Jasonjiao2023/CEDL

CEDL: A Hippocampal-Inspired Architecture for Advancing LLMs

Author: Dian Jiao. Manuscript submitted to Neurocomputing, 2026.

CEDL combines contextual language prediction with direct access to token evidence in the visible input. Four stages divide contextual encoding, expansion, retrieval and context–memory comparison. Two shared processing sweeps are followed by a small recurrent probability readout, allowing the model to use both learned language probabilities and previously observed token-successor associations.

The design draws functional inspiration from the entorhinal cortex, dentate gyrus, CA3 and CA1. Its contribution is a tested language-and-retrieval trade-off: lower mean language loss and high isolated lexical recall at modest scale, with greater execution cost. The biological analogy guides the organization; the experiments do not establish biological causation.

Release contents and checkpoint availability

The submitted model and the older downloadable checkpoint are different releases.

• —`standalone/` contains the approximately 45.6M-parameter model implementation associated with the submitted study, loading instructions and identities of the six evaluated checkpoint pairs. Those final trained weights are not yet included in this repository.
• —The root-level pytorch_model.bin, CEDL.py, config.json, cedl_config.json, MANIFEST.json, examples/ and probes/ belong to the older 136.3M-parameter research checkpoint. They are preserved for existing users. See legacy release information.
• —The results below describe the submitted 45.6M study. They must not be attributed to the older pytorch_model.bin. The old checkpoint is not compatible with the new standalone loader.

Six corpus/seed-specific states were evaluated: three trained on WikiText-103 and three trained on FineWeb-Edu. They are separate models, not a single checkpoint trained jointly on both corpora. `standalone/CHECKPOINTS.json` records the matching backbone/readout pairs and SHA256 checksums. Its paths identify the retained experimental files, not files currently hosted on the Hub.

Architecture

• —C — Contextual encoding. Six shared layers construct contextual states using positive-feature normalized retention with content-selective decay.
• —E — Expansion. A dense fourfold expansion and contraction transform the contextual states. A training-time separation objective acts on the expanded codes; the implementation does not sparsify them.
• —D — Directed retrieval. A learned causal relational path is combined with suffix-weighted token-successor retrieval and an adaptive routing rule. Only evidence already available in the prefix can be retrieved.
• —L — Linkage. Context–memory comparison combines a preserved contextual reference with retrieved information and supplies feedback between the two sweeps.

The probability readout combines the native vocabulary distribution with suffix-weighted successor evidence and a latest-best-match successor distribution. A 20-unit recurrent controller adjusts the mixture using 16 features derived from cache statistics and the model's own probabilities. It has 802 trainable parameters and 32 fitted frozen coefficients. Its state resets for every request.

The backbone has width 448, seven retention heads and tied input/output token embeddings. Including the readout, CEDL has 45,643,901 learned coefficients. The deployed model uses GPT-2's 50,257-token vocabulary and a default maximum context of 2,048 tokens.

Training and comparison protocol

The submitted study compares CEDL with public Transformer and Mamba-2 implementations and a specified looped-Transformer control. The baselines retain their architectures; CEDL's probability readout is not added to them.

Each corpus/model/seed run receives 536,870,912 main language-training targets: 65,536 updates, a 2,048-token context and 8,192 targets per update. Three confirmation seeds are used per corpus. Language training uses BF16, AdamW and a registered learning-rate calibration procedure with an outward trial for a grid-edge winner.

The language corpora are WikiText-103, using its raw configuration, and FineWeb-Edu, using a fixed sample from sample-10BT. WikiText training amounts to approximately 4.55 packed passes. FineWeb-Edu training and evaluation documents are partitioned by a fixed document-hash rule. Both use GPT-2 tokenization; their packing and document-boundary conventions are specified in the manuscript.

Task adaptation then gives every model 4,096 updates with identically indexed language/recall presentations. CEDL freezes its language backbone and fits the small probability readout; baselines fit their existing parameters. Development selection maximizes recall subject to a language-NLL increase of at most 0.01 relative to the better of the native parent and initial deployed state. The threshold and candidate steps 0, 1,024, 2,048 and 4,096 were fixed before confirmation.

All baseline runs retain step 0 under this rule. Later adaptation loses too much development language performance. Some rejected states improve development recall, so the selected-state contrast is not a limit on what a Transformer or Mamba-2 can learn. Adaptation subspaces also differ, and CEDL's retained development-fitted router/readout initializers add historical exposure outside the common main-training budget. Lifetime tuning effort and computation were not equalized.

Language likelihood and isolated recall

These are results at context 2,048 for the same deployed states on both tasks. NLL and recall are mean ± sample standard deviation over three seeds. PPL is the exponential of mean NLL. Lower NLL/PPL and higher recall are better.

Language evaluation scores all 283,286 WikiText-103 test targets and 1,048,576 held-out FineWeb-Edu targets, using windows that advance by 256 targets with up to 2,048 tokens of input history. Each target is scored once. Reference scoring uses FP32 for every model.

Relative to Mamba-2, CEDL's mean perplexity is 7.27% lower on WikiText-103 and 6.82% lower on FineWeb-Edu. The WikiText Transformer has substantial seed variability; its paired NLL-contrast interval includes zero. The descriptive mean ranking should not be read as a stable ranking under wider Transformer tuning.

What recall measures

The recall panel contains 56 controlled lexical-binding conditions, varying binding load, distractor gap, key/value length and whether a key has been updated. Each checkpoint receives 1,792 isolated queries. Earlier question–answer pairs are excluded from the query context, and accuracy requires the complete value to be generated correctly.

On the 28 updated-key conditions, CEDL recovers the latest lexical binding with 99.33% ± 0.73% accuracy for WikiText-trained states and 99.96% ± 0.06% for FineWeb-Edu-trained states. This is evidence for recovering an updated symbolic association. It is not a natural-language contradiction-resolution benchmark, and the implementation does not delete an old fact when a newer one appears.

Downstream evaluation

Full official splits are used for four zero-shot tasks. Values below are descriptive mean accuracies (%) across three seeds. ARC-Easy and HellaSwag use length-normalized accuracy; WinoGrande and LAMBADA use ordinary accuracy. LAMBADA requires the complete target word.

Transfer is mixed. CEDL has higher mean LAMBADA and ARC-Easy scores than Mamba-2, while FineWeb-Edu HellaSwag is essentially tied and WinoGrande shows no consistent advantage. Near-perfect synthetic recall does not imply near-perfect reading comprehension or general reasoning.

Mechanism evidence

Same-checkpoint inference interventions on seed 7703 separate the roles of the hidden-state pathway and the probability readout. These use a smaller assessment subset: 8,192 language targets and 256 isolated queries per corpus, without retraining.

• —Bypassing D increases language NLL by 1.3615 on WikiText and 2.0600 on FineWeb-Edu. Disabling feedback increases it by 0.6403 and 0.5796. Recall remains at 100% with the probability route active.
• —Disabling the probability readout reduces recall from 99.61% to 13.67% and 12.89%, respectively. Disabling only its recurrent correction leaves 87.89% and 96.48% recall.

The trained language pathway relies on D and feedback; exact lexical recall relies heavily on the probability route. Fully retrained component ablations and a baseline fitted with the same readout are still needed for stronger attribution. The final comparison did not include that baseline-plus-readout control.

Execution cost and limitations

On the reference RTX PRO 6000 Blackwell Server Edition, CEDL at context 2,048 takes approximately 2.75–3.02× Mamba-2's language-training time and 4.64–4.86× its full-window scoring time across the two corpus probes. Scoring includes CEDL's CPU readout and transfers; these are not cached token-by-token decoding measurements. Mamba-2 uses its official fused execution path. The quality comparison is parameter- and main-target-matched, not compute-matched.

C's retention operator has bounded state at a fixed width. The complete model also constructs token-addressable evidence and dense relational/feedback tensors, so whole-model memory and work grow with context length.

This is a single-scale, finite-budget research study. Its limitations include repeated WikiText exposure, unequal historical development effort, different adaptation subspaces, three-seed uncertainty, and incomplete retrained attribution. It establishes neither general superiority over large pretrained language models nor a biological explanation for the observed gains.

Intended use

CEDL is intended for research into language prediction, isolated associative retrieval and the interaction between contextual representations and probability-space evidence. The implementation can support controlled inference studies and further model development.

The deployed interface handles one unpadded request at a time and resets its memory/readout state between requests. It has no persistent cross-request store, semantic invalidation operation or calibrated abstention policy. It is not instruction-tuned or a clinical decision system. Generated text can be incorrect or reflect biases in the training data; research recall accuracy does not make its free-form answers reliable.

Standalone implementation

The new implementation is `standalone/cedl.py`. It uses ordinary PyTorch operators and requires neither the older root-level code nor Mamba/FLA dependencies.

Use Python 3.10 or later with PyTorch 2.11.0. After downloading the standalone folder, install the appropriate PyTorch build for the target machine:

cd standalone
python -m pip install -r requirements.txt

The following example requires a separately supplied, matching final checkpoint exported as cedl.pt, as described in the usage guide. It does not download trained weights and must not use the older root-level pytorch_model.bin.

import torch
from cedl import configure_execution, load_checkpoint

configure_execution()
model = load_checkpoint("cedl.pt", device="cpu")
ids = torch.tensor([[11, 23, 11]], dtype=torch.long, device=model.device)
log_probs = model(ids)  # [time, vocabulary]
continuation = model.generate(ids, max_new_tokens=16)

The IDs illustrate the interface, not a text prompt. Encode and decode real text with the GPT-2 tokenizer. A score_mask selects output positions; it is not a padding mask. Generation recomputes the visible prefix and does not silently truncate input. The 2,048-token default limit includes the requested continuation. Constructing CEDL() without a checkpoint creates an untrained model.

The standalone implementation was checked against the reference on CPU PyTorch 2.11.0, including observed exact output agreement through 2,048 tokens with a compatible trained full-width backbone and small-geometry gradient/update checks. Independent tests cover causality, request reset and checkpoint handling. CUDA execution and all six final trained states have not been requalified with this standalone extraction. The paper's reference GPU timings must not be treated as a fresh benchmark of this file.

Citation

This implementation accompanies the submitted manuscript:

Jiao, D. (2026). CEDL: A Hippocampal-Inspired Architecture for Advancing LLMs. Manuscript submitted to Neurocomputing.

@unpublished{jiao2026cedl,
  author = {Jiao, Dian},
  title = {{CEDL}: A Hippocampal-Inspired Architecture for Advancing {LLMs}},
  year = {2026},
  note = {Manuscript submitted to Neurocomputing}
}

This is a submitted-manuscript citation, not a claim of acceptance or publication. When referring to the older downloadable checkpoint, also identify the repository revision and checkpoint used.

About the author

CEDL was developed by Dian Jiao, a multidisciplinary researcher and technology leader with more than 15 years of experience across digital transformation, neuroscience and technology innovation. His work connects brain-inspired theory with practical AI systems, with a focus on NeuroAI, digital therapeutics and human-centered model design.

His research interests include AI-enhanced digital therapeutics using EEG, adaptive biofeedback and machine learning for cognitive enhancement and neurological rehabilitation, alongside biologically inspired models of memory and neural dynamics.

Selected publications and contributions:

• —Jiao, D. (2025). Elliptic cortical networks: A mathematically constrained architecture for biologically-inspired intelligence. Neurocomputing, 658, 131802. DOI.
• —Jiao, D. (2025). Leveraging neurotechnology for neurodivergent education: a narrative review. Learning: Research and Practice, 1–25. DOI.
• —Jiao, D. (2025). AI-enhanced digital therapeutics for cognitive impairment: Integrating mobile applications, virtual reality, and wearable devices. Discover Artificial Intelligence, 5, Article 69. DOI.
• —Jiao, D. (2025). Advancing personalized digital therapeutics: integrating music therapy, brainwave entrainment methods, and AI-driven biofeedback. Frontiers in Digital Health, 7, 1552396. DOI.
• —Jiao, D. (2025). From hypoxic pockets to daily routines: linking brain oxygenation and cognitive resilience. Frontiers in Aging Neuroscience, 17, 1534198. DOI.
• —Cognition and Brain Teaching Technology, a book on brain-based teaching and cognitive technology.
• —Chinese translations of How Breakthroughs Happen (2020), Mastering the Dynamics of Innovation (2022), and Neuroscience of You (2025).

LinkedIn