ciaochris/neuro2-neuroscience-datasets

Neuro2 Neuroscience Dataset Atlas

<div align="center">

   

</div>

Neuro2 is an interactive 3D knowledge graph, discovery engine, and metadata atlas for exploring open neuroscience datasets across the global research ecosystem.

This repository publishes an authoritative, cryptographically verified snapshot of the complete public Neuro2 catalog together with 5 query-optimized Parquet tables designed for instant analysis in Python, Hugging Face datasets, DuckDB, Polars, Pandas, NetworkX, and Graph Neural Network (GNN) pipelines.

Key Highlights & Statistics

• —12,011 Datasets Indexed: Aggregating public metadata from 18 major research repositories (OpenNeuro, Zenodo, DANDI, OSF, Figshare, Dataverse, NeuroVault, PhysioNet, CONP, GIN, Dryad, DataLad, NeuralBench, FCP/INDI, BNCI, NITRC, NeuroAtlas, and Hugging Face).
• —Comprehensive Modality Coverage: Spanning EEG (3,856+), MRI/fMRI/sMRI/dMRI (3,275+), Electrophysiology / Neural Signals (2,405+), MEG (1,330+), Eye-tracking (285+), iEEG/ECoG (239+), NIRS/fNIRS (159+), PET (108+), ECG, Optical Physiology (OPhys), and behavioral experiments.
• —Enriched Knowledge Graph (40,934 Nodes & 63,390 Edges): Capturing interconnectivity between datasets, experimental tasks, research papers, authors, institutions, funding agencies, GitHub code repositories, and recording hardware manufacturers.
• —Large-Scale Cohort Metrics: Covering 128,450+ recorded subjects and over 741 million recording seconds (~205,840+ hours / >23.5 years of continuous neural recording data).
• —Semantic & Sparse Embeddings: Includes 11,183 TF-IDF sparse embedding vectors and full-text search indices for instant semantic discovery, keyword filtering, and topic modeling.
• —Byte-for-Byte Raw Mirror: The raw/ directory contains exact byte copies of the 5 top-level graph JSON documents and 32 sharded detail files (detail/00.json through detail/31.json) backed by manifest.json.

Dataset Configurations

The repository exposes five distinct Hugging Face configurations:

Quickstart & Usage Examples

1. Load with Hugging Face datasets

from datasets import load_dataset

# Load the primary dataset catalog
datasets = load_dataset(
    "ciaochris/neuro2-neuroscience-datasets",
    "datasets",
    split="train",
)

print(f"Total datasets: {len(datasets)}")
print("Sample record:", datasets[0])

# Load knowledge graph nodes and edges
nodes = load_dataset("ciaochris/neuro2-neuroscience-datasets", "nodes", split="train")
edges = load_dataset("ciaochris/neuro2-neuroscience-datasets", "edges", split="train")

2. Fast Direct Parquet Loading (Pandas / Polars)

Because the files are standard Parquet, you can read them directly from Hugging Face without cloning the full repository:

import pandas as pd

# Load directly from the Hub URL
url = "https://huggingface.co/datasets/ciaochris/neuro2-neuroscience-datasets/resolve/main/data/datasets.parquet"
df = pd.read_parquet(url)

# Filter for human EEG datasets with at least 30 recorded subjects
eeg_large_cohorts = df[
    (df["species"] == "human") &
    (df["modalities"].apply(lambda mods: "eeg" in mods if mods is not None else False)) &
    (df["subject_count"] >= 30)
]

print(f"Found {len(eeg_large_cohorts)} large-cohort EEG datasets:")
print(eeg_large_cohorts[["id", "name", "source", "subject_count", "license"]].head(10))

3. Serverless SQL Analytics (DuckDB)

Query remote Parquet tables using standard SQL:

import duckdb

con = duckdb.connect()

# Query top neuroscience repositories by dataset volume
query = """
SELECT 
    source, 
    COUNT(*) AS total_datasets,
    SUM(subject_count) AS total_subjects,
    ROUND(SUM(recording_seconds) / 3600, 1) AS total_hours
FROM 'https://huggingface.co/datasets/ciaochris/neuro2-neuroscience-datasets/resolve/main/data/datasets.parquet'
GROUP BY source
ORDER BY total_datasets DESC
LIMIT 10;
"""

print(con.execute(query).df())

4. Knowledge Graph Analysis (NetworkX)

Construct a heterogeneous multi-relational graph from nodes and edges:

import networkx as nx
import pyarrow.parquet as pq

# Load nodes and edges tables
nodes_table = pq.read_table("data/nodes.parquet")
edges_table = pq.read_table("data/edges.parquet")

G = nx.MultiDiGraph()

# Add nodes with attributes
for node_id, kind, label, layer in zip(
    nodes_table["id"].to_pylist(),
    nodes_table["kind"].to_pylist(),
    nodes_table["label"].to_pylist(),
    nodes_table["layer"].to_pylist(),
):
    G.add_node(node_id, kind=kind, label=label, layer=layer)

# Add edges with relationships
for src, dst, rel in zip(
    edges_table["source_id"].to_pylist(),
    edges_table["target_id"].to_pylist(),
    edges_table["relationship"].to_pylist(),
):
    G.add_edge(src, dst, relationship=rel)

print(f"Constructed Knowledge Graph: {G.number_of_nodes():,} nodes, {G.number_of_edges():,} edges.")

# Find the most connected research institutions
inst_nodes = [n for n, d in G.nodes(data=True) if d.get("kind") == "institution"]
top_institutions = sorted(
    [(G.degree(n), G.nodes[n].get("label")) for n in inst_nodes],
    reverse=True
)

print("\nTop 5 Connected Institutions:")
for degree, label in top_institutions[:5]:
    print(f"  {label} ({degree} linked datasets/entities)")

5. Semantic Search via Sparse TF-IDF Embeddings

Discover datasets matching complex conceptual queries via cosine similarity:

import json
import math
import pyarrow.parquet as pq

emb_table = pq.read_table("data/embeddings.parquet")

query_terms = {"sleep": 1.0, "spindle": 0.8, "eeg": 0.5}
query_norm = math.sqrt(sum(v**2 for v in query_terms.values()))

results = []
for doc_id, top_terms, weights_json in zip(
    emb_table["id"].to_pylist(),
    emb_table["top_terms"].to_pylist(),
    emb_table["weights_json"].to_pylist(),
):
    if not weights_json:
        continue
    weights = json.loads(weights_json)
    dot_product = sum(query_terms[t] * weights[t] for t in query_terms if t in weights)
    if dot_product > 0:
        doc_norm = math.sqrt(sum(w**2 for w in weights.values()))
        sim = dot_product / (query_norm * doc_norm)
        results.append((sim, doc_id, top_terms[:5]))

results.sort(reverse=True)

print("Top 5 Semantically Similar Datasets:")
for sim, doc_id, terms in results[:5]:
    print(f"  [{sim:.3f}] {doc_id} -> Keywords: {terms}")

Dataset Breakdown & Distributions

Source Repositories Indexed

Modalities

• —EEG (Electroencephalography): 3,856 datasets
• —MRI (fMRI, sMRI, dMRI, DWI, BOLD): 3,275+ datasets
• —Signals & Electrophysiology (Patch clamp, Neuropixels, Spikes): 2,405+ datasets
• —MEG (Magnetoencephalography): 1,330 datasets
• —Eye-Tracking & Pupillometry: 285 datasets
• —iEEG / ECoG (Intracranial EEG / Electrocorticography): 239 datasets
• —NIRS / fNIRS (Near-Infrared Spectroscopy): 159 datasets
• —PET (Positron Emission Tomography): 108 datasets
• —ECG & Autonomic Physiology: 75 datasets
• —OPhys (Optical Physiology / Two-Photon Calcium Imaging): 56 datasets

Species Distribution

• —Human (`human`): 11,196 datasets (93.2%)
• —Animal (`animal` - Non-human primates, Rodents, etc.): 471 datasets (3.9%)
• —Unknown / Cross-species (`unknown`): 344 datasets (2.9%)

Schema Reference

1. datasets Configuration

2. nodes Configuration

3. edges Configuration

4. search_text Configuration

5. embeddings Configuration

Provenance, Pipeline & Reproducibility

This repository is maintained as an automated, reproducible mirror of datasets.neuro2.ai.

Snapshot Provenance

• —Snapshot Timestamp: 2026-08-14T11:54:30Z
• —Source Byte Total: 35,330,883 bytes across 37 validated JSON documents.
• —Manifest: manifest.json provides cryptographic SHA-256 hashes, HTTP headers (ETag, Last-Modified), schemas, and row counts for every mirrored file.

Refresh & Verification Commands

To reproduce the synchronization, normalize the Parquet tables, and verify cryptographic integrity locally:

# Set Python path to include synchronization modules
$env:PYTHONPATH='scripts'
$env:PYTHONDONTWRITEBYTECODE='1'

# 1. Sync public catalog from upstream into repository layout
python 'scripts/sync_neuro2.py' sync --repo-root .

# 2. Verify all Parquet tables, schemas, and SHA-256 digests against manifest.json
python 'scripts/sync_neuro2.py' verify --repo-root .

# 3. Execute automated test suite
python -m pytest -q

Licensing, Ethics & Data Access

Licensing

The repository-level license is `other` because this catalog aggregates public metadata from 18 disparate research platforms, each governed by its own terms:

• —Every row in the datasets table explicitly preserves the license and canonical source_url provided by the original host.
• —Common upstream licenses include Creative Commons Zero (CC0), Creative Commons Attribution (CC-BY 4.0), Open Data Commons PDDL, MIT, and specific institutional open-access terms.
• —Downstream researchers must consult and comply with the individual license terms of the underlying datasets they access.

Data Access & Scientific Payloads

This repository distributes metadata, relational graph topology, and semantic indices. Linked raw electrophysiology, neuroimaging, and behavioral payloads (amounting to ~151.7 TB across global servers) remain hosted on their respective scientific platforms (OpenNeuro, Zenodo, DANDI, Figshare, PhysioNet, etc.).

Human Subjects & Ethical Standards

All metadata in this atlas originated from publicly published, de-identified research archives. No private health information (PHI) or unshared participant data is collected or exposed.

Citation & Attribution

The Neuro2 live 3D atlas and catalog were created and maintained by [Nataliya Kosmyna](https://www.linkedin.com/in/nataliekosmina/) and [Eugene Hauptmann](https://github.com/eugenehp).

If you use Neuro2 in your research, software, or meta-analyses, please cite the project as follows:

@misc{neuro2_2026,
  author       = {Kosmyna, Nataliya and Hauptmann, Eugene},
  title        = {Neuro2: The Interactive 3D Open Neuroscience Dataset Atlas},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://datasets.neuro2.ai/}},
  note         = {Hugging Face Dataset: ciaochris/neuro2-neuroscience-datasets}
}

When utilizing underlying datasets identified through this atlas, please also cite the primary dataset authors, DOIs, and originating host repositories.