Datasetpublic

Tejaskumar/Emergent-NCA-Sequences-5M

✨ Why this dataset? Emergent NCA Sequences 5M generates complex global behaviors entirely from frozen random Neural Cellular Automata. What makes this approach powerful? Controlled Diversity: Each rollout uses a fresh set of random weights, creating massive diversity in dynamical systems without hand-crafting rules. Stable Semantics: Continuous hidden states are compressed into a global 32-token vocabulary (centroids.pt), guaranteeing… See the full description on the dataset page: https://huggingface.co/datasets/Tejaskumar/Emergent-NCA-Sequences-5M.

sourceHugging Facemitupdated 3mo agoView on Hugging Face

8likes130downloads

Dataset Card

<br>

![License: MIT](https://opensource.org/licenses/MIT) ![Dataset Size](https://huggingface.co/datasets) ![Format](https://numpy.org/)

</div>

✨ Why this dataset?

Emergent NCA Sequences 5M generates complex global behaviors entirely from frozen random Neural Cellular Automata. What makes this approach powerful?

—Controlled Diversity: Each rollout uses a fresh set of random weights, creating massive diversity in dynamical systems without hand-crafting rules.
—Stable Semantics: Continuous hidden states are compressed into a global 32-token vocabulary (centroids.pt), guaranteeing structurally comparable but dynamically unique sequences.
—No Memorization: Because dynamics are deterministic-given-weights but highly diverse across rollouts, sequence models must genuinely internalize transition rules.

📊 Dataset Dynamics Analysis

The Emergent NCA Sequences dataset represents a rich, spatiotemporal dynamical system. Rather than static text or images, it encodes complex local rules that evolve over 500 steps. Below is a comprehensive analysis of the emergent behaviors, dimensional distributions, and temporal dynamics.

1. Global Dynamics & Behavioral Taxonomy

We categorize and evaluate the emergent behaviors of the cellular automata across the dataset. The dynamics are overwhelmingly active and structured:

<div align="center"> <table border="0" cellspacing="0" cellpadding="5"> <tr> <td align="center" valign="top" width="50%"> <img src="assets/behaviordistribution.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 12px rgba(0,255,157,0.25);" /> <br><em>Emergent Behavior Distribution (Donut Chart)</em> </td> <td align="center" valign="top" width="50%"> <img src="assets/phasespace.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 12px rgba(0,184,255,0.25);" /> <br><em>NCA Phase Space: Volatility vs. Token Entropy</em> </td> </tr> </table> </div>

—Static vs. Dynamic Rollouts: The dataset is designed to be highly active. Almost all rollouts remain continuously active or chaotic, with less than 0.1% collapsing into completely static attractors (frozen states).
—Chaos vs. Periodic (Oscillators): A significant portion of the sequences fall into robust periodic loops (breathers), allowing models to learn recurring temporal rhythms. Other sequences exhibit chaotic, turbulent, or wave-like diffusion across the grid.

2. Geometry & Spatial Dimensionality

Grid sizes vary dynamically across rollouts, presenting an ideal testbed for variable-length sequence modeling and multi-scale generalization.

<div align="center"> <img src="assets/dimension_distributions.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 15px rgba(236, 72, 153, 0.2);" /> <br><em>Distribution of Heights, Widths, Shape Ratios, and Tokens per Frame</em> </div>

—Grid Dimensions: Heights range from 11 to 45 cells; widths range from 9 to 46 cells.
—Token Length Profile:
—Average Tokens per Frame: 724.4 cells (Median: 495.0).
—Average Sequence Tokens per Rollout: 362,178.2 total tokens (reaching up to a maximum of 880,000 tokens over 500 frames).
—Variable-shaped grid dimensions ensure sequence models must dynamically allocate memory and scale attention mechanisms across varying lengths.

3. Symbolic Vocabulary & Token Distribution

The continuous hidden states are mapped to a discrete 32-token vocabulary using MiniBatch KMeans. The state distribution reveals the density and structures of the active shapes:

<div align="center"> <table border="0" cellspacing="0" cellpadding="5"> <tr> <td align="center" valign="top" width="50%"> <img src="assets/tokenfrequencies.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 12px rgba(139, 92, 246, 0.25);" /> <br><em>Discrete Token Vocabulary Frequency (%)</em> </td> <td align="center" valign="top" width="50%"> <img src="assets/complexitydist.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 12px rgba(245, 158, 11, 0.25);" /> <br><em>Token Complexity (Unique Tokens used per Rollout)</em> </td> </tr> </table> </div>

—Top Vocab Occupancy: The discrete cell states are non-uniformly distributed. The top 5 most frequent states dominate:
—Token 19: 24.43%
—Token 31: 16.62%
—Token 7: 10.87%
—Token 17: 9.83%
—Token 24: 7.77%
—Empty Space Mapping: Token 0 and dominant low-activity tokens represent background space, enabling sparse active boundaries and localized structures.

4. Rollout Evolution & Spatiotemporal Physics

Let's look at a selected rollout (Rollout 0, size 11x38) as it evolves over the 500-frame horizon:

<div align="center"> <img src="assets/rollout_trajectory.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 15px rgba(0, 0, 0, 0.4);" /> <br><em>Physical grid state evolution across time steps t ∈ [0, 499]</em> </div>

—Initial Chaotic Phase: The automaton starts in an active state and rapidly diffuses local updates through 3x3 residual convolutions.
—Attractor/Periodic State: Over time, the local transitions converge into highly structured, repeating spatial shapes and periodic breathers.

5. Transition Curves & Temporal Decay

By calculating frame-to-frame change dynamics, we can quantify the entropy and rate of change of the NCA:

<div align="center"> <table border="0" cellspacing="0" cellpadding="5"> <tr> <td align="center" valign="top" width="50%"> <img src="assets/entropynoveltycurves.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 12px rgba(59, 130, 246, 0.25);" /> <br><em>Entropy, Novelty (Frame-to-Frame change), and Cumulative Difference</em> </td> <td align="center" valign="top" width="50%"> <img src="assets/temporal_decay.png" width="100%" style="border-radius:12px; box-shadow: 0 4px 12px rgba(16, 185, 129, 0.25);" /> <br><em>Temporal Similarity Decay: Chaotic phase (t = 0) vs. Stabilized phase (t = 100)</em> </td> </tr> </table> </div>

—Entropy Curves: Quantifies cellular state diversity. The system maintains high structural complexity, with clear indicators of the transition to stable states.
—Transition Activity (Novelty): Shows the frame-to-frame changes. The novelty peaks early during the emergent phase and then stabilizes as the system converges to attractors.
—Lag Similarity Decay: In the early chaotic phase (t = 0), matching states decay rapidly over a short time lag, showing high volatility. In contrast, once the attractor phase (t = 100) is reached, matching remains high even over hundreds of steps, verifying robust periodic or stable attractors.

🧬 Neural Cellular Automata Architecture

The dataset employs a lightweight Residual NCA architecture, uniquely initialized for every single rollout:

<div align="center" style="background-color: #1e1e1e; padding: 20px; border-radius: 12px; border: 1px solid #333;"> <img src="https://readme-typing-svg.demolab.com?font=Fira+Code&weight=600&size=20&pause=1000&color=ff5e5e&center=true&vCenter=true&width=400&lines=Random+Frozen+Weights" alt="Weights" /><br> ⬇️ <i>Inject</i><br> <b>🧱 Local 3x3 Interaction Convolutions</b><br> ⬇️ <i>Flow</i><br> <b>🔄 Residual Hidden-State Updates</b><br> ⬇️ <i>Add Noise</i><br> <b>🌫️ Stochastic Perturbation Noise</b><br> ⬇️ <i>Produce</i><br> <img src="https://readme-typing-svg.demolab.com?font=Fira+Code&weight=600&size=20&pause=1000&color=00FF9D&center=true&vCenter=true&width=400&lines=16+Hidden+Channels" alt="Channels" /> </div>

<br>

Why randomize? Each sequence uses a fresh set of random weights, creating unparalleled diversity in the dynamical systems while strictly sharing a common symbolic vocabulary.

🔤 Symbolic Vocabulary

Continuous hidden states are intelligently compressed into discrete symbolic tokens using MiniBatch KMeans clustering and cosine-similarity assignments.

<div align="center" style="background-color: #1e1e1e; padding: 15px; border-radius: 8px; border: 1px solid #333;"> <code>🎲 Random NCA</code> ➡️ <code>🎞️ 500 Frame Rollout</code> ➡️ <code>🧩 Hidden-State Extraction</code> ➡️ <code>🎯 KMeans Quantization</code> ➡️ <b style="color:#00FF9D;">✨ Symbolic Sequences</b> </div>

<br>

What is a token? A token represents a specific, quantized combination of the 16 hidden channels. By assigning each cell a discrete ID from 0 to 31, we map high-dimensional continuous dynamics into a text-like representation.

—Vocabulary Size: 32 distinct symbols.
—Shared Reference: The centroids.pt file defines this global vocabulary across all 5M+ rollouts. This means Token 7 in sequence A means exactly the same structural latent state as Token 7 in sequence B.

📊 Dataset Statistics

Property	Value	Property	Value
Total Samples	5M+	Grid Sizes	8×8 → 48×48
Rollout Length	500 Frames	Quantization	MiniBatch KMeans
Hidden Channels	16	Storage Format	`.npz` Shards
Vocabulary	32 Tokens	Dynamics	Frozen Random NCA

Shard Information: The full dataset is split into manageable .npz shards. Ensure your pipeline streams or handles shard loading efficiently to avoid memory bottlenecks.

📁 Repository Structure & Scripts

Data & Labels Mapping

The dataset is generated in massive chunks. Each data folder has a corresponding CSV file containing the computed behavioral metrics (e.g., activity, complexity, stable states) for every rollout:

—dataset_labels_set.csv ➡️ Describes rollouts in nca_dataset/
—dataset_labels_set2.csv ➡️ Describes rollouts in nca_dataset_set2/
—dataset_labels_set3.csv ➡️ Describes rollouts in nca_dataset_set3/

Utility Scripts

The repository includes several Python scripts to help you generate, load, and visualize the data:

—generate_local.py: The core dataset generator. It initializes a random TinyNCA model, runs the dynamics, quantizes the continuous hidden states using centroids.pt, and writes the 32-token symbolic sequences into compressed .npz shards.
—sample_usage.py: A lightweight snippet demonstrating how to iterate through the data. It streams the .npz shards and yields individual frame transitions (frame[t] to frame[t+1]), which is the standard format for training sequence or world models.
—visualize_dataset.py: A helper script that picks a random rollout from the shards, maps the symbolic tokens to grayscale values, and renders an animated .gif to let you visually inspect the emergent patterns.

🧠 The Sparse Long-Horizon Prediction Objective

To bypass simple identity-mapping shortcuts (where sequence models learn to simply copy-paste consecutive frames), the dataset is optimized for a sparse long-horizon prediction task:

—Context Inputs: t₀, t₁₆, t₃₂, t₄₈ (spaced 16 frames apart)
—Prediction Target: t₁₁₂ (a large 64-frame gap after the context)

<div align="center"> <img src="assets/predictiontaskvisualization.png" width="100%" style="border-radius: 12px; box-shadow: 0 4px 15px rgba(255, 94, 94, 0.3);" /> <br> <em>Sparse Long-Horizon Prediction: Mapping t₀, t₁₆, t₃₂, t₄₈ → t₁₁₂</em> </div>

🎯 Task Difficulty & Mechanics

—Bypassing the Continuity Shortcut: Between t₄₈ and t₁₁₂, approximately 83.01% of the cells undergo state changes. Standard copy-paste operations or identity mappings yield terrible cross-entropy loss, forcing models to genuinely model and internalize the underlying ResNCA transition dynamics.
—Temporal Abstraction: Models must learn high-level temporal transitions over the 64-step interval, testing their capacity for long-term reasoning, scale-generalization, and dynamic system emulation.

🎯 Use Cases

—Sequence Reasoning & Pretraining: Train/fine-tune small transformers on structured reasoning. The dataset acts as a synthetic "physics" engine for sequence models.
—World Model Learning: Multi-scale grids (8×8 → 48×48) make this a perfect testbed for scale-generalization in predictive world models.
—Evaluating Abstraction: Test if your SSM (Mamba, etc.) or Transformer generalizes rules instead of memorizing patterns.
—Artificial Life Research: Study how lifelike behaviors (oscillators, diffusion) emerge from simple localized rules.
—Anomaly Detection: Train a model on "normal" NCA dynamics and probe its detection of out-of-distribution transitions.

⚠️ Limitations

—Uncontrolled Diversity: Because the NCA weights are completely random and frozen, the emergent phenomena are heavily diverse but not systematically curated or balanced.
—Coarse Vocabulary: The 32-token limit compresses high-dimensional behavior heavily. Certain fine-grained structural changes might be smoothed out.

📄 Contributor

@LegendaryCheese

📄 Citation

If you use this dataset in your research, please cite it:

bibtex

@misc{nca_sequences_5m,
  authors = {Tejaskumar Reddy J, Tharun N Nanju},
  title = {Emergent NCA Sequences 5M: Massive-Scale Synthetic Symbolic Dynamics},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/Tejaskumar/Emergent-NCA-Sequences-5M}},
}

<br>

<div align="center"> <img src="https://capsule-render.vercel.app/api?type=waving&color=timeGradient&height=100&section=footer" width="100%" alt="Footer Animation"/> <h3>“Complexity emerging from locality.”</h3> <p>🌀 <i>Local rules → emergent worlds.</i></p> </div>