19110173× sample-and-hold upsampling
100% of 540 rows flagged; adjacent repetition 67.23% ± 0.35%; nominal row length 1,536 samples; effective rate ≈ 170.67 Hz.
A historical single-channel dataset of unpaced mental arithmetic, re-engineered end-to-end in ten phases: raw audit, boundary-safe filtering, artifact screening, 33-feature extraction, statistics and leak-free machine learning — ending in a function that can score a live headset.
This dataset contains zero music conditions.
Rest (eyes closed) vs Medium vs Hard arithmetic (eyes open) — no audio, no music files, no music condition. It cannot test whether music improves cognition; it is a methodological baseline for single-channel DSP, leak-free validation and the eyelid confound.
Phase 2 · Data integrity
The raw CSV (8,249 rows, SHA-256 verified) was audited row by row. Four anomalies shape every later decision.
Raw recording rows
8,249
across 51 unique participant IDs, 151 subject-task files
Complete 3-task subjects
50
1911019 completed Task 1 only (61 rows) and is excluded
Repeated-measures cohort
N = 48
2104041 and 2311026 lost 100% of Task 3 to detachment
Task order
Fixed
100% of subjects ran Task 1 → Task 2 → Task 3; never counterbalanced
1911017100% of 540 rows flagged; adjacent repetition 67.23% ± 0.35%; nominal row length 1,536 samples; effective rate ≈ 170.67 Hz.
1911018100% of 428 rows flagged; adjacent repetition 50.72% ± 0.44%; nominal row length 1,024 samples; effective rate 256 Hz. Decimated with polyphase anti-aliasing in preprocessing.
Task 3All 157 poor_signal == 200 rows (1.90%) occurred in Task 3. 2104041 (58/58 rows) and 2311026 (57/57 rows) lost the entire task.
is_correctis_correct is constant within 100% of 151 recordings — a broadcast session outcome, not trial-level performance. Missing in 18.57% of Task 2 and 33.20% of Task 3 rows.
| poor_signal == 0 | 7,858 rows (95.26%) |
|---|---|
| poor_signal 25–107 | 234 rows (2.84%) |
| poor_signal == 200 | 157 rows (1.90%) |
Behaviour can't be predicted from this EEG
Across 32 subjects with paired data, Medium Math accuracy was 87.5% vs Hard81.25% — McNemar χ² = 0.2353, p = 0.6276 (exact binomial p =0.6291). EEG–is_correct correlations: 0 significant (all Holm-adjusted p = 1.000).
Source: reports/full_data_audit.md · §1–4
Phases 4–6 · Signal processing
Pipeline v2
Stream reconstruction & segmentation
· 1911017 / 1911018 decimated with anti-aliasing to their true bandwidth.
Bandpass
4th-order zero-phase Butterworth bandpass, 1.0–40.0 Hz, SOS + sosfiltfilt
Artifact screening
Ceiling ptp ≤ 2000 ADC counts; flatline rejection std ≥ 1.
Feature extraction v2 — 33 base features
Deliberately not applicable
ERP processing: No time-locked trial event markers or stimulus triggers exist in this dataset.
Frontal alpha asymmetry: Single unilateral electrode (FP1); no bilateral homologous channel.
| Ceiling | 1-s chunks | Acc | BalAcc | Data loss |
|---|---|---|---|---|
| 1,000 counts | 7,606 | 85.58% | 68.47% | 17.5% rejected |
| 1,500 counts | 8,767 | 80.57% | 71.32% | 4.9% rejected |
| 2,000 counts (selected) | 9,074 | 78.92% | 71.78% | 1.6% rejected |
| 3,000 counts | 9,219 | 78.45% | 71.40% | 0.0% rejected |
2,000 counts keeps > 98.4% of usable chunks across all 50 subjects and gives the highest balanced accuracy. Lower ceilings discard genuine high-amplitude resting delta and distort the class balance — note how raw accuracy rises at 1,000 counts while balanced accuracy falls.
Source: reports/full_data_audit.md · §5
Phase 7 · Neurophysiology (N = 48 repeated measures)
Subject-by-task means, Wilcoxon signed-rank tests with Holm correction and paired Cohen's dz. Two subjects lost all Hard-Math data to detachment, leaving N = 48 for three-condition tests.
Frontal midline theta
dz = 1.388
rel_theta 0.182 → 0.305 from Rest to Medium Math (Δ = +0.123, W = 39.0, Holm p = 4.85 × 10⁻¹⁰). theta_beta 1.385 → 7.256 (dz = 0.953).
Eyes-closed alpha confound
Rest → Medium
−0.1886
dz = -1.378
Medium → Hard
−0.0029
p = 0.2553 (n.s.)
Relative alpha falls 51.0% at eye opening, then changes only 1.6% between difficulty levels (Holm p = 0.4702). This negative control isolates the alpha drop to eyelid state.
Hjorth complexity scales
1.544 → 1.945 → 2.204
Rest → Medium dz = 0.691; Medium → Hard dz = 0.627 (Holm p = 2.54 × 10⁻⁵) — one of the few features that keeps rising when eyes are open in both conditions.
| Feature | Rest | Medium | Hard | χ²(2) | Holm p | Kendall W |
|---|---|---|---|---|---|---|
| theta_alpha | 0.965 | 2.453 | 2.807 | 63.875 | 1.62 × 10⁻¹³ | 0.665 |
| theta_beta | 1.385 | 7.256 | 8.204 | 63.167 | 2.11 × 10⁻¹³ | 0.658 |
| hjorth_complexity | 1.544 | 1.945 | 2.204 | 55.542 | 8.70 × 10⁻¹² | 0.579 |
| rel_theta | 0.182 | 0.305 | 0.330 | 54.875 | 9.71 × 10⁻¹² | 0.572 |
| rel_alpha | 0.370 | 0.181 | 0.178 | 48.792 | 1.78 × 10⁻¹⁰ | 0.508 |
| alpha_beta | 2.690 | 2.209 | 2.424 | 0.542 | 1.0000 | 0.006 |
Source: reports/statistical_analysis.md · §2–4
Phase 8 · Machine learning (57 configurations)
Every generalised model is validated with participant-grouped 5-fold GroupKFold or leave-one-subject-out (LOSO). 95% CIs: 1,000 bootstrap resamples of per-subject balanced accuracy.
| Model | CV | Acc | BalAcc | 95% CI |
|---|---|---|---|---|
| Random Forest (300 trees) | 5-Fold GroupKFold | 78.92% | 71.78% | 70.52–78.07 |
| Logistic Regression | 5-Fold GroupKFold | 78.64% | 71.69% | 69.90–77.50 |
| Linear SVM | 5-Fold GroupKFold | 78.70% | 71.51% | 69.55–77.22 |
| XGBoost (100 trees) | 5-Fold GroupKFold | 78.14% | 71.27% | 69.37–77.32 |
| Random Forest (300 trees) | LOSO (50 folds) | 79.53% | 72.34% | 71.27–78.70 |
| Logistic Regression | LOSO (50 folds) | 79.46% | 72.33% | 70.68–78.08 |
| Linear SVM | LOSO (50 folds) | 79.59% | 72.19% | 70.40–77.85 |
| XGBoost (100 trees) | LOSO (50 folds) | 80.06% | 73.03% | 71.66–79.24 |
| Model | CV | Acc | BalAcc | 95% CI |
|---|---|---|---|---|
| Random Forest | 5-Fold GroupKFold | 55.68% | 52.98% | 50.06–58.01 |
| Logistic Regression | 5-Fold GroupKFold | 55.26% | 52.67% | 51.99–59.26 |
| Linear SVM | 5-Fold GroupKFold | 55.47% | 52.73% | 51.95–59.17 |
| XGBoost | 5-Fold GroupKFold | 55.08% | 52.28% | 50.76–58.99 |
| Random Forest | LOSO (50 folds) | 56.01% | 53.29% | 50.46–57.92 |
| Logistic Regression | LOSO (50 folds) | 54.80% | 52.01% | 51.43–58.51 |
| Linear SVM | LOSO (50 folds) | 55.06% | 52.09% | 51.57–58.83 |
| XGBoost | LOSO (50 folds) | 55.21% | 52.14% | 50.79–58.70 |
Every Medium-vs-Hard model's raw accuracy is below the 56.99% majority baseline — the classes are inseparable from 1-s single-channel EEG.
Source: reports/modeling_results.md · Exp. 8.1
Exp. 8.4 · Alpha ablation
| Subset | n | BalAcc | Δ |
|---|---|---|---|
| All features | 33 | 71.78% | Reference |
| Without alpha features | 26 | 72.16% | +0.38 pp |
| Band power only | 16 | 70.83% | −0.95 pp |
| Ratios only | 4 | 68.93% | −2.85 pp |
| Hjorth & complexity only | 5 | 61.32% | −10.46 pp |
| Time-domain only | 6 | 55.28% | −16.50 pp |
Removed: rel_alpha, theta_alpha, alpha_beta, theta_alpha_beta, abs_alpha, log_alpha, peak_alpha_freq.
Exp. 8.8 · Leakage demonstration
| Condition | Value (%) | Spread |
|---|---|---|
| RF · random KFold | 83.12 | — |
| RF · GroupKFold | 71.78 | — |
| LogReg · random KFold | 76.37 | — |
| LogReg · GroupKFold | 71.69 | — |
Random chunk-level splitting puts temporally autocorrelated windows from the same person in train and test, so the model memorises subject morphology. RF inflates by +11.33 pp; LogReg by +4.68 pp.
Exp. 8.6 · Permutation test
Within-subject label shuffles; empirical p = (0 + 1) / (200 + 1).
Exp. 8.5 · Personal calibration
| k (s) | Subjects | Generic | Calibrated | Δ pp |
|---|---|---|---|---|
| 0 | 50 | 71.24% | 71.24% | +0.00 |
| 5 | 50 | 71.20% | 57.59% | −13.61 |
| 10 | 47 | 71.89% | 63.15% | −8.73 |
| 20 | 41 | 71.18% | 68.81% | −2.37 |
| 40 | 37 | 63.64% | 66.02% | +2.39 |
First k contiguous chunks calibrate; evaluation strictly on chunk index ≥ k, so the evaluated set shrinks as k grows.
Source: reports/ml_results.csv
Curated figures
Click any figure for the full-resolution viewer (← / → to step, Esc to close). All 11 MindWave figures →
Butterworth 1–40 Hz filter response
Magnitude (dB) and phase across 0–60 Hz for the 4th-order zero-phase SOS bandpass. −3 dB cut-offs at 1 Hz and 40 Hz; DC drift suppressed by > 40 dB at 0.2 Hz and mains harmonics by > 30 dB at 50/60 Hz. Identical relative bandwidth at 512 Hz and the decimated 256 Hz rate.
Mean power spectrum per task
Grand-average PSDs (thick) over individual spectra (thin) from 1–45 Hz. Rest shows an 8–12 Hz alpha peak from eyes-closed idling; both math tasks show alpha suppression and elevated theta/beta power.
The eyes-closed alpha confound
Panel A: relative alpha collapses from eyes-closed Rest to eyes-open Medium Math (Δ = −0.1886, d_z = −1.378) and then flatlines between Medium and Hard (Δ = −0.0029, p = 0.2553). Panel B: non-alpha markers (rel_theta, hjorth_complexity) rise progressively with demand.
Data-leakage demonstration
Random chunk-level KFold (leaked) vs subject-grouped GroupKFold (honest). Leakage inflates RF balanced accuracy from 71.78% to 83.12% (+11.33 pp) and LogReg from 71.69% to 76.37% (+4.68 pp).
Within-subject permutation null
200 within-subject label permutations: null mean 0.5084 (SD 0.0039, 95th percentile 0.5145) vs observed 0.7131. Empirical p = 1/201 = 0.0050.
Phases 9–10 · Headset pilot bridge
The trained pipeline is serialised and exposed as one function, with five gatekeeper checks before any prediction is trusted.
src/predict.pyPythondef predict_from_raw(raw_512_samples, fs=512) -> dict:
# raw_512_samples: 512 raw integer ADC counts
# (1 s from the MindWave; do NOT convert to µV)
...
return {
'high_load_probability': float | None,
'prediction': 0 | 1 | None, # 0 Rest, 1 High load
'quality_ok': bool,
'quality_reason': str,
'features': dict | None,
}Inside predict_from_raw(raw_512_samples, fs=512)
Model artifact: models/best_pipeline.joblib (git-ignored)
Buffer length
exactly 512 samples
Contact byte
poor_signal == 0
Repeated samples
dup_fraction ≤ 0.20
Flatline
std ≥ 1.0 and ptp ≥ 2.0 counts
Rail saturation
ptp ≤ 2000 counts
Units warning
Pass raw integer ADC counts. The model was trained on unscaled counts; converting to µV shrinks band powers by ≈ 0.0484× and causes severe misclassification.
What the pilot can and can't tell you
It estimates broad rest-vs-task state. It cannot grade task difficulty (Medium vs Hard ≈ chance) and — on its own — says nothing about music.
Source: docs/pilot_integration.md