ethicalabs/Echo-DSRN-114M-v0.1.2-Base
Model Card for ethicalabs/Echo-DSRN-114M-v0.1.2-Base
     
[!WARNING] This repository contains experimental models designed strictly for academic evaluation and research purposes. Critical Constraints: No Production Deployment: Experimental models must not be deployed in commercial, enterprise, or mission-critical environments under any circumstances. No Liability: Experimental models are provided "as-is" without warranties of any kind. The developers assume zero liability for downstream consequences, system integration failures, or regulatory non-compliance resulting from unauthorized deployment.
The Echo-DSRN(N) (Dual State Recurrent Neural Network, short name: Echo-DSRN, also know as echo) is a novel architecture specifically designed to be a viable alternative for low-resource tasks that are currently being inefficiently handled by the excessive scale of Large Language Models (LLMs) 🌱
⚠️ Important Notice
This is a research prototype and demo model.
- Not production-ready
- Will hallucinate and give incorrect answers
- Do not use for any real-world decisions
- Intended for architecture experimentation only
What Works
- Text generation is fluent
- Memory usage is constant O(1)
- Runs on CPUs, NPUs, GPUs (Tested on AMD's ROCm and Apple's MPS)
What Doesn't Work
- Factual accuracy
- Instruction following
- Common sense reasoning
🏗️ Architecture Details
📊 Parameter Breakdown
🧩 Internal Block Structure (Per Layer)
Pre-Training
Truncated Backpropagation Through Time (TBPTT) on Fineweb-EDU + Smoltalk2 (no think) (6BT)
1 epoch on a single AMD Instinct MI300X 192 GB
Evaluation
Gradio App - Next Word Prediction
Echo-DSRN-114M-Base - Next Word Prediction
Citation
If you use this model in your research, please cite it as follows:
@misc{Massimo Roberto Scamarcia, title={Echo-DSRN-114M: Surprise-Gated Dual-State Recurrent Architecture for Efficient Language Modeling and Classification}, DOI={10.5281/zenodo.19848279}, publisher={Zenodo}, author={Massimo Roberto Scamarcia} }