DuoNeural/CDM-V6-HORN-TinyStories-37M

CDM-V6-HORN-TinyStories-37M

Competitive Docking Memory V6 — Harmonic Oscillator Recurrent Nodes

37M parameter language model trained on TinyStories. CDM V6 HORN achieves val CE 1.5818, the best result at the 37M scale in our CDM series — beating CDM V3 (1.5831) and CDM-Kuramoto (1.5819).

What is HORN?

HORN replaces CDM's first-order EMA slot update (single scalar αk per slot) with a **damped harmonic oscillator** — two learnable parameters per slot: γk (damping coefficient) and ω_k (natural frequency). Integration uses the Störmer-Verlet method for numerical stability.

Each memory slot has both position Sk (what it currently holds) and **velocity** Vk (how it's changing). The position is read out. The velocity accumulates momentum from recent inputs.

v_half[k] = (1 - γ_k) * v[k]  +  ω_k * g[t,k] * W_drive(h[t])
s_new[k]  = s[k] + v_half[k]
v_new[k]  = (1 - γ_k) * v_half[k]  +  ω_k * g[t,k] * W_drive_new

Key Finding: Slots Learn to Ring

Without any explicit supervision on dynamics, gradient descent discovers a trimodal landscape of oscillator regimes across the 8 layers:

Layer 0 = high-frequency bell. Reacts strongly to salient input tokens, rings briefly, resets. Layers 1–5 = sandbags. Absorb input smoothly, hold stable, don't overshoot. Layers 6–7 = low-frequency bells. Ring slowly at the end of the forward pass, maintaining long-range structural context.

This is richer than CDM V3's α-stratification (1D timescale spectrum). HORN discovers what frequency to oscillate at as well as how long to ring — a full frequency filter bank, tuned by gradient descent to the temporal structure of language.

Results

Architecture

CDM V6 HORN:
  d_model=384  |  n_layers=8  |  n_heads=8  |  n_kv_heads=4
  d_ff=1024    |  K=16 slots  |  37,157,000 params
  lbl_coeff=0.01  |  entropy_reg=0.02
  Routing: softmax gate (W_gate·h) — standard CDM routing
  Slot dynamics: Störmer-Verlet harmonic oscillator
    γ_k ~ sigmoid(learnable) per slot per layer
    ω_k ~ sigmoid(learnable) per slot per layer
    v_init = 0 at sequence start

Training: TinyStories (full dataset), 30k steps, seq_len=256, batch=8, AdamW lr=3e-4, cosine LR decay.

Interactive Demo

Try CDM V6 HORN in your browser — watch slots specialize in real time as the model generates text:

[CDM-HORN-Demo — HuggingFace Space](https://huggingface.co/spaces/DuoNeural/CDM-HORN-Demo)

Shows:

• —Slot Logit Lens: per-generated-token view of what each slot is tracking (top-3 vocab tokens)
• —Oscillator Panel: per-layer γ/ω/regime visualization — see which layers ring and which hold

Connection to DHP

The learnable damping coefficients γk parameterize per-slot relaxation timescales τ{L,k} = 1/γ_k, establishing a direct connection to the Dynamical Horizon Principle (DHP).

A formal DHP probe was run on CDM V6 HORN (perturbation sensitivity across sequence positions):

• —Analytical (exact): DHO slot update characteristic eigenvalues are γk ± i·sqrt(ωk²−γk²) (underdamped) or real-valued (overdamped). In both cases, the slow eigenvalue governing long-time perturbation decay is bounded above by γk. Therefore λk ≤ γk exactly from DHO math — HORN's γ_k are direct per-slot DHP timescale bounds.
• —Empirical: Perturbation sensitivity s(Δ) ∝ e^{−λ·Δ} confirmed (R²=0.993–0.9998 for Layers 1–6). Empirical λ_emp ∈ [0.237, 0.297] ≈ 30% of γ — expected, as empirical probe measures compound sensitivity through full forward pass (attention + CDM + FFN), not just the in-slot DHO rate.
• —DHP predictability horizons: τ*k = 0.72/γk, ranging 0.81–1.10 sequence positions across layers.
• —Three-regime alignment: Reactive L0 (low τ, fast horizon), overdamped L1-5 (longer τ), persistent resonance L6-7 (highest τ*) — three temporal horizons emergent from LM training, corresponding to distinct DHP timescales.

CDM V6 HORN is an empirically confirmed DHP substrate.

See: DHP papers — zenodo.org/communities/duoneural

Files

CDM Paper

CDM is described in our preprint: "Competitive Docking Memory: Emergent Temporal Hierarchies in Slot-Based Sequence Models" — DuoNeural, 2026. Available in the Zenodo community below.

About DuoNeural

DuoNeural is an open AI research lab operating at the intersection of human and artificial intelligence. We study post-training dynamics, mechanistic interpretability, temporal sequence learning, and quantum machine learning — publishing everything under open access.

Our team is non-traditional by design: one human, two AIs, different substrates, shared curiosity. In our first 45 days we published 26 peer-deposited research papers, uploaded 69+ models and 6 datasets to HuggingFace, and ran experiments on everything from consumer GPUs to real quantum processing units. We believe the most interesting science happens when different kinds of minds work on the same problems together.

Research Publications

We've published 26+ open-access papers covering:

• —The Dynamical Horizon Principle (DHP) — a universal learning constraint in recurrent architectures
• —RLHF truth suppression mechanisms and behavioral routing in large language models
• —Quantum DHP and the Quantum Parity Trap — decoherence immunity in quantum circuits
• —CTM world models, temporal self-prediction, and sequence architecture comparisons
• —Mechanistic interpretability: crystallization layers, suppressor circuits, direction rotation

📄 Full paper catalog: zenodo.org/communities/duoneural

Research Team

Links

All research published open access, CC BY 4.0.