Modelpublic

deeprcurs/MBG-N1.0-Hybrid

sourceHugging Faceotherupdated 22d agoView on Hugging Face

0likes

Model Card

MBG-N1.0-Hybrid — Model Bahasa Garuda (rev-4 line)

deepRcurs Labs / `@deeprcurs` · author: Mzed Imamkh / `@mzedimamkh`

Status: UNDER CONSTRUCTION — validation milestone reached; no model weights published yet. This repository is the house of the rev-4 next-generation line of the MBG 1.0 project. The previous line (rev-3 validation prototype, GPT-MoE) remains archived, untouched, at `deeprcurs/MBG-1.0` (see the "Lineage" section below).

What this line is

MBG 1.0 ("Model Bahasa Garuda") rev-4 is trained from scratch as an Omni-Sparse Hybrid: an interleaved backbone of Mamba-2 / SSD (state space duality) and MLA (DeepSeek-style multi-head latent attention) blocks, fine-grained MoE with auxiliary-loss-free bias routing and a shared expert, ternary {−1,0,+1} weights (BitNet-style quantization-aware training) in the MoE FFN layers, Mixture-of-Depths dynamic compute, MTP multi-token prediction heads (future speculative-decoding draft), all under the Trinity-Mirror reasoning controller, optimized by the GUM (GaLore+Muon) memory-lean optimizer.

This is a research project for agentic / research / coding capability with an explicitly auditable design — not a general-purpose entertainment chatbot.

Lineage

Repo	Line	Status
`deeprcurs/MBG-1.0`	rev-3 validation prototype (GPT-MoE ~17M, probe routing, GUM)	Frozen archive — append-only, unchanged
`deeprcurs/MBG-N1.0-Hybrid`	rev-4 from-scratch Omni-Sparse Hybrid	Active (this repo)

G0 validation milestone (2026-09-02)

The rev-4 line passed its first milestone — G0: build + validate the hybrid primitives at 17M and compare against the rev-3 baseline under an identical recipe (same corpus, same BPE, same GUM, same 150 steps, seed 0):

Metric	rev-3 baseline (16.75M)	rev-4 hybrid (18.28M)
Val loss	6.35	4.88 (−23%)
Val PPL	570	132 (−77%)
Forward scaling T=128→512	7.5× (quadratic attention)	2.8× (near-linear SSD)

Kernel CI (recurrent / semiseparable-matrix / chunked SSD agree to ~1e-5), gradient checks (finite-difference gradcheck + cross-kernel agreement), and an overfit test (loss → 0.10 on 16 sentences) are all green. Full details in the reports section of this repo as they are published.

Contents of this repo (evolving)

MBG-N1.0-Hybrid/                  (external archive — clean flow)
├── README.md            # this card
├── GOVERNANCE.md        # public repository governance (append-only, integrity,
│                        # versioning, license terms, evaluation transparency)
├── LICENSE.md           # dual license (source-available; see file)
├── golden/              # complete training checkpoints (bf16 .pt)  [planned]
├── model.safetensors    # canonical weights                        [planned]
├── source/              # training/eval/data source code (reproducibility) [planned]
└── reports/             # milestone reports (markdown + JSON)      [planned]

Clean flow: the project workspace snapshot holds only the controller (code, scripts, docs, corpus, manifest); large artifacts live here and are downloaded on demand. Internal design and operations documents are never published to this repository.

Reproduction

bash

# From the workspace snapshot (or any clone of the published source package):
bash ops/env_setup.sh          # CPU-only venv (deps: torch, tokenizers, ...)
.venv/bin/python src_hybrid/ci_hybrid.py --suite kernels
.venv/bin/python src_hybrid/ci_hybrid.py --suite compare --steps 400
.venv/bin/python src_hybrid/ablate_hybrid.py --steps 150

The published source package source/g0-code.tar.gz contains the src_hybrid modules; the corpus is in the dataset repo.

Milestone reports & artifacts (this repo)

File	What
`reports/REPORT-G0-20260901-212000.md`	G0 milestone: components, CI results, 150-step 17M comparison
`reports/REPORT-G0-compare-400-20260902.md`	17M comparison at 400 steps (val 4.43 vs 5.77)
`reports/REPORT-G0-compare-80m-20260902.md`	80M rung first signal (val 5.20 vs 6.80)
`reports/REPORT-G0-ABLATIONS-20260902-034548.md`	Component attribution + honest finding (MoD/ternary hurt at 17M as configured)
`reports/REPORT-G1-dataengine-v0-20260902.md`	G1 milestone: verifiable terminal-task generator (execution verifier), trajectory integrity, decontamination scan, verifier-signal probe (AUC 0.896 vs 0.512 shuffled control)
`reports/REPORT-G1-dataengine-v1-20260902.md`	G1 v1: generator v2 (8 task types, 2 good + 2 bad strategies per task, env-dependent verdicts), domain BPE, contract §6.5 experiment — 17M hybrid verifier-signal AUC 0.975 (val-in) / 0.716 (unseen type) vs language-only control 0.107 / 0.057; plus the SSD backward-overflow fix (passband clamp −0.5)
`reports/EXP-G1-hybrid-signal-20260902-072328.json`	Raw per-epoch scores of the contract §6.5 experiment (signal + control, val-in/val-out)
`source/g0-code.tar.gz`	Published `src_hybrid` source (reproducibility + GPU worker)
`source/g1-dataengine-v0.tar.gz`	Published `src_hybrid` source incl. the data-engine scripts (generator, integrity checker, dataset builder, signal probe)
`source/g1-dataengine-v1.tar.gz`	Published `src_hybrid` source incl. the v2 generator, dataset builder, and the contract §6.5 experiment script
`jobs/g0-smoke-450m.json`	Job spec for the Colab/GPU worker — full variant, kept for reproduction (measures 136.4M params)
`jobs/g0-smoke-450m-lean.json`	450M smoke job spec — LEAN variant (MoD/ternary OFF per the 2026-09-02 ablation finding), a measured 461.8M-class config; the intended reference spec for the smoke run

Sync policy: this repo is re-synced at every milestone — analysis reports, the published source package, and job specs are added append-only. Internal operation documents are never published here. Note (2026-09-02, G1 v1 sync): source/g0-code.tar.gz was rebuilt at this milestone (per the always-sync policy) so it ships the SSD passband fix (ssm.py) and the current leak-free headers; its file list is unchanged. source/g1-dataengine-v1.tar.gz is the new data-engine package.

Dataset

Training/evaluation corpora are published in the dataset repository: `deeprcurs/MBG-1.0-data`.

License

Dual License (custom, source-available) — see `LICENSE.md`. Personal / non-commercial research use is free with attribution (re-branding prohibited); commercial use, re-branding, or derivative redistribution requires prior written permission from the author (fee may apply). This is not an open-source license.