tankalapavankalyan/exp01-eeg-to-text-sentences

Exp01 — Sentence-level EEG-to-text training data (unified)

This is a private working corpus for experiment 1 (fine-tuning EEG / time-series foundation models on EEG-to-English-text). It bundles several public EEG-while-reading datasets into a single, raw-lossless parquet schema where one row = one sentence read by one participant.

⚠️ License: Per-source licenses are preserved verbatim in each row's license column and source_url. Do not re-distribute publicly without re-checking the upstream terms; the other SPDX tag here is only because this repo aggregates several distinct licenses (CC0-1.0, CC-BY-4.0, MIT, Apache-2.0).

Source datasets and provenance

Skipped (redundant): pyinglie/EEG-To-Text_Preprocessed (only adds derived BART pickles), sajjad5221/eeg2text-emmt-dataset non-unique variant (the unique-sentences variant covers the same EEG with richer gaze metadata).

Unified schema

Every row is a single sentence read by a single participant. EEG is stored at the native sampling rate, with the native channel count for that source — no resampling, no channel selection, no rereferencing.

EEG storage convention

All word_eeg_segments arrays are stored channels-first ([channels][time]), matching the MNE convention. EMMT's source [time][channels] layout is transposed on conversion. The original layout is recorded in extra_json.original_eeg_layout.

Why some columns are NULL for some sources

Each source dataset only provides a subset of the available metadata. We never invent values. For example:

• —ZuCo carries the richest set of per-word band features and eye-tracking metrics.
• —EMMT provides per-word EEG + a small set of gaze counts but no band features.
• —Quoron provides per-word EEG + topic-relevance labels but no eye-tracking.
• —DERCo provides per-word EEG and per-article cloze probabilities (will be filled from the behavioural CSVs in a follow-up).

Use dataset to filter rows when training a model that requires a specific feature.

Dedup decisions

The user's brief was: "if there is repetition, leave those, otherwise do everything." The following candidate sources were skipped because they are strict subsets / derived products of other sources we already include:

• —pyinglie/EEG-To-Text_Datasets — used only as a fast HF mirror of the ZuCo MAT files; the data is the original ZuCo content.
• —pyinglie/EEG-To-Text_Preprocessed — only contains BART-style pickled features derived from the same MAT files; raw EEG is in zuco_v*_* rows.
• —sajjad5221/eeg2text-emmt-dataset (non-unique) — superseded by the -unique-sentences variant which has 23 fewer duplicate samples but adds the per-word gaze feature columns.

Re-using and reproducing

Every row carries source_url, license, and dataset_extra_blob so you can always trace back to the upstream and reload the source-specific arrays:

import io, numpy as np
from datasets import load_dataset

ds = load_dataset("tankalapavankalyan/exp01-eeg-to-text-sentences",
                  data_files="data/zuco_v1_sr__sub-ZAB.parquet", split="train")
row = ds[0]
print(row["sentence_text"])
print("EEG shape (chan, time):",
      len(row["sentence_eeg"]), len(row["sentence_eeg"][0]))
extras = np.load(io.BytesIO(row["dataset_extra_blob"]))
print("extras keys:", extras.files[:10])

Conversion pipeline

The full pipeline is a streaming download → convert → push → delete loop with peak local disk usage bounded to ~2 GB at any time. The orchestrator is experiments/exp01_finetune_foundation_models/scripts/preprocess/stream_pipeline.py, backed by per-source converters (convert_emmt.py, convert_quoron.py, convert_zuco.py, convert_derco.py) and the unified schema definition in experiments/exp01_finetune_foundation_models/src/data/unified_schema.py. Source files are downloaded one at a time from HuggingFace mirrors (or OSF for DERCo), converted in /tmp, uploaded to this repo, and deleted before the next file is fetched. Nothing persists in the project tree.