Skip to content
Music×EEG

Cross-dataset comparison

Two datasets, side by side — never pooled

MindWave and PhysioNet were preprocessed, trained and validated completely separately. This page compares their designs and results so you can see where they agree, where they diverge, and why their accuracies must not be compared head-to-head.

A · Acquisition

Hardware, montage & units

Hardware and design comparison
DimensionMindWave · N = 50PhysioNet Exp. 1 · N = 10
CohortN = 50 subjectsN = 10 participants (A1–A10)
Montage1 dry frontal electrode (FP1), earlobe reference/ground4 dry electrodes (TP9, AF7, AF8, TP10), mastoid references
HardwareNeuroSky TGAM1 ASICInteraXon Muse headband
Sampling rate512 Hz nominal (two subjects host-upsampled)256.0 Hz nominal
UnitsUnscaled 16-bit ADC counts; power in (counts)²Calibrated microvolts (µV); PSD in µV²/Hz
Windowing1-s continuous chunks, no stimulus alignment2.0-s stimulus-locked epochs, mean alignment error 0.98 ms
Cognitive paradigmUnpaced mental arithmetic: Rest (eyes closed), Medium, Hard (eyes open)Visual N-back: 1-back (low) vs. 3-back (high), both eyes open
Music conditionsZERO — no audio, no music condition4 conditions in a FIXED order: No Music → Relaxing → Stressing → AI Music
Behavioural metricsSession-level is_correct only (broadcast, invariant)Chronos button box: accuracy, RT, d′, criterion c
Data volume9,074 clean 1-s chunks (v2 pipeline)14,080 trials; 13,772 eligible after screening
Repository roleHistorical audit, leak-free baseline, alpha ablationPrimary benchmark: synchronisation, personalisation, confound audit

Source: reports/CONSOLIDATED_RESEARCH_FINDINGS.md · §A.3, §D

B · Signal processing

Preprocessing & artifact screening

Same principle (zero-phase SOS Butterworth, amplitude screening), different implementations — because the units differ by orders of magnitude.

Preprocessing comparison
StepMindWavePhysioNet
Filtering tracks1 track — 1–40 Hz Butterworth (order 4), SOS, sosfiltfilt2 tracks — A: 1–40 Hz (bandpower) · B: 0.5–30 Hz (ERP/time-domain), SOS, sosfiltfilt
Where filtering runsWithin continuous segments; breaks at packet drops / poor-signal rowsOn the whole ~87-min continuous recording before epoching
Amplitude screenPTP ≤ 2000 ADC counts (gross artifact / rail screen)PTP ≤ 200 µV on AF7/AF8 (heuristic model-eligibility screen)
Flatline screenstd ≥ 1.0 countσ ≥ 0.5 µV
Channel fallbackNot applicable (single channel)Frontal fallback: A4 → AF8, A9 → AF7; FAA masked to NaN
Yield9,074 clean chunks; 1.6% rejected vs. the 3,000-count reference13,772 / 14,080 eligible (97.81%); 308 trials excluded (2.19%)
Hardware anomalies1911017 (3×) and 1911018 (2×) sample repetition — decimatedA3, A5, A10 periodic duplicate sample every 1,285 samples — left unmodified, N = 7 sensitivity cohort
Feature space33 base features (bandpower, ratios, entropy, Hjorth, moments)229-column feature table; 88-feature full model; 9 ablation groups

PTP ≤ 2000 counts ≠ PTP ≤ 200 µV

The MindWave ceiling is in unscaled ADC counts and targets gross disconnection and rail saturation. The PhysioNet ceiling is in calibrated µV. Neither removes all ocular or EMG contamination.

1-track vs 2-track filtering

MindWave has no stimulus markers, so ERP processing is not applicable and one 1–40 Hz track suffices. PhysioNet adds a 0.5–30 Hz track for stimulus-locked ERP and time-domain features.

C · Classification benchmarks

Benchmarks, filtered by question

Pick a comparison. Each view shows both datasets on the same 40–85% axis (leakage view: 40–90%) with the error bars the reports give. Bars are drawn from the verified data layer, not from the PNGs.

No calibration, unseen subjects

MindWaveBalanced accuracy · bars show 95% bootstrap CI

MindWave · subject-independent (no calibration)

MindWave · subject-independent (no calibration)
ConditionValue (%)Spread
Rest vs Math · RF · GroupKFold71.7895% CI 70.52–78.07
Rest vs Math · XGBoost · LOSO73.0395% CI 71.66–79.24
Medium vs Hard · RF · GroupKFold52.9895% CI 50.06–58.01
Medium vs Hard · RF · LOSO53.2995% CI 50.46–57.92
50% balanced-accuracy chanceerror bar = reported SD or 95% CI

Takeaway. Rest-vs-Math separates well across unseen subjects, but that contrast also changes eyelid state. Medium vs Hard (eyes open in both) collapses to ~53% — below the 56.99% majority-class accuracy baseline.

Caveat. Majority-class accuracy baselines: Rest vs Math 67.49%, Medium vs Hard 56.99%. Balanced accuracy chance is 50%.

Source: reports/modeling_results.md

PhysioNetBalanced accuracy · bars show ± SD across folds

PhysioNet · generic cross-participant models

PhysioNet · generic cross-participant models
ConditionValue (%)Spread
Random Forest · all sessions51.19± 2.55 SD
Logistic Regression · all sessions50.98± 3.33 SD
Linear SVM · all sessions50.97± 3.52 SD
Random Forest · No Music only (condition-matched)50.00± 3.82 SD
50% chance (perfectly balanced classes)error bar = reported SD or 95% CI

Takeaway. 1-back vs 3-back from held-out participants is at chance for every model family — inter-subject baseline variance swamps the within-subject workload shift.

Source: reports/physionet/experiment_1/phase4_modeling_results.md

Do not read across the two columns as a league table

MindWave Rest-vs-Math (~72–73% balanced accuracy) contrasts eyes-closed rest with active arithmetic — the eyelid state and arousal change help. PhysioNet 1-back-vs-3-back (~55.6–56.1% personalized balanced accuracy) contrasts two eyes-open active workload levels. When MindWave is given an equally subtle contrast (Medium vs Hard, eyes open in both), it falls to ~53%.

D · Evidence strength

The evidence matrix

All twelve observations from the consolidated report, graded by how strongly the current analyses support them. Filter by grade.

  1. 01
    MindWavePhysioNetStrongly supported

    Generic cross-participant models perform near chance on fine-grained workload classification.

    PN RF 51.19%, LogReg 50.98%, SVM 50.97%; MW Med vs Hard RF 51.60% (historical).

  2. 02
    MindWavePhysioNetStrongly supported

    Personalized baseline calibration significantly outperforms generic models.

    PN RF +4.45% (p = 0.0025, d_z = 1.31, Holm 0.0075); LogReg +4.98%; SVM +5.10%. MW resting calibration +3.75%.

  3. 03
    MindWavePhysioNetStrongly supported

    Bandpower ratios (θ/β, θ/α, α/β) are the most diagnostic representation.

    PN band_ratios ranked #1 (56.25% ± 5.30%); MW theta_alpha_z #1 in full model (+0.0829 drop).

  4. 04
    PhysioNetStrongly supported

    Single-trial ERP features fail on consumer dry headbands.

    PN erp subsystem 50.26% ± 2.91%.

  5. 05
    PhysioNetStrongly supported

    Behavioural gains across PhysioNet sessions reflect practice, not music.

    Accuracy 36.5% → 51.8% in fixed order; music RM-ANOVA p = 0.207 (synthesis); no Holm-surviving pairwise contrast.

  6. 06
    MindWaveSupported but limited

    Single-channel Rest vs Math is heavily confounded by eyelid closure (Berger effect).

    rel_alpha dominant; Med vs Hard with constant eyelid state fails.

  7. 07
    MindWaveSupported but limited

    Non-alpha features (frontal theta, Hjorth complexity) still separate rest from calculation.

    MW alpha ablation keeps 72.16% (Phase 8) / 73.23% (historical) balanced accuracy.

  8. 08
    PhysioNetSupported but limited

    Baseline frontal theta reflects trait capacity rather than momentary state.

    PN Spearman ρ = −0.8788 (p = 0.0008) with d′; within-subject |r_rm| < 0.20.

  9. 09
    PhysioNetSupported but limited

    More calibration data shows a sample-efficiency progression on a fixed test set.

    PN 52.19% (10%) → 57.06% (50%) on Sessions 3–4.

  10. 10
    PhysioNetExploratory

    Temporoparietal sites show stronger workload modulation than frontal sites.

    PN TP10_rel_theta d_z = +1.26 vs AF7 d_z = +0.68; TP subsystem 55.33% vs frontal 53.67%.

  11. 11
    MindWavePhysioNetNot established

    Background music causally improves working memory or alleviates fatigue.

    MW has zero music; PN conditions confounded by fixed order; EEG condition differences q ≥ 0.90.

  12. 12
    MindWavePhysioNetNot established

    Wearable dry-electrode EEG can reach > 70% single-trial workload decoding in real time.

    MW Med vs Hard 51.6%; PN personalized plateaus ≈ 55.6–56.1%.

Source: reports/CONSOLIDATED_RESEARCH_FINDINGS.md · Section E

E · Synthesis

Where they agree — and where they don't

✓ Consistent across both

  • Inter-subject variance barrier

    In both datasets, between-person differences in amplitude and spectral power (relative theta 8%–24% across PhysioNet subjects) swamp within-person workload shifts (Δ ≈ 1%–2%). Uncalibrated cross-subject models fail on fine-grained load.

  • Personalization imperative

    Referencing each person against their own baseline — resting windows in MindWave, initial task blocks in PhysioNet — gives verifiable gains (+3.75 pp; +4.45 to +5.10 pp).

  • Spectral ratios as primary markers

    theta/alpha and theta/beta style ratios self-normalise across electrode impedance and outperform raw power or single-trial ERPs.

  • Single-trial ceiling

    Neither dataset supports > 70% fine-grained workload decoding from single 1–2 s trials.

≠ Divergences & non-comparables

  • Different task contrasts — accuracies are NOT comparable

    MindWave ~72–73% separates eyes-closed rest from eyes-open arithmetic, so ocular and arousal changes help the classifier. PhysioNet ~55.6–56.1% separates two eyes-open active workload levels (1-back vs 3-back), a far subtler physiological contrast. A higher MindWave number does not mean better hardware or a better model.

  • Topographical coverage

    MindWave has one forehead channel (FP1) with substantial ocular contamination. PhysioNet adds bilateral frontal (AF7/AF8) and temporoparietal (TP9/TP10) sites, where the largest paired load effects appeared (TP10 d_z = 1.26 vs AF7 d_z = 0.68).

  • Units and calibration metrology

    MindWave features are in raw ADC counts; PhysioNet in calibrated µV. Absolute power values cannot be compared across the two datasets — only relative bandpowers and ratios travel.

  • Primary design threat

    MindWave: eyes-closed rest vs eyes-open math (Berger effect) and fixed task order. PhysioNet: fixed No Music → Relaxing → Stressing → AI Music order that makes music collinear with practice and fatigue.

Source: reports/CONSOLIDATED_RESEARCH_FINDINGS.md · Section D