deeprcurs/MBG-1.0

MBG 1.0 — Model Bahasa Garuda

deepRcurs Labs / `@deeprcurs` · author: Mzed Imamkh / `@mzedimamkh`

MBG 1.0 ("Model Bahasa Garuda") is a research project building an Omni-Sparse Hybrid language model — a laddered-scaling architecture that combines an SSM/MLA core (Mamba-2 + DeepSeek-style multi-head latent attention), fine-grained MoE, and ternary {−1,0,1} weights (BitNet-style), with Trinity-Mirror as the core reasoning / answer-production controller.

This Hugging Face model repo is the external archive for the project: it stores the golden checkpoints and canonical weights. The dataset/corpus lives in a separate dataset repo (linked below), and the workspace snapshot holds only the controlling code and lightweight pointers.

Status: L0 (0.17M) and L1 (17M) rungs validated. This repo currently holds the L0/L1 validation checkpoints produced by the mbg_mini_gpt.py prototype, which exercises the same MoE + probe-routing + GUM-optimizer machinery used to validate the scaling, memory, and routing claims. The full Omni-Sparse Hybrid (SSM+MLA) is the target architecture and is specified in the internal design blueprint (MBG-1.0-rev-3_Blueprint.md), which is kept in the workspace snapshot only — not published to this repo.

Architecture summary (per blueprint rev-3)

• —Omni-Sparse Hybrid core: interleaved Mamba-2/SSM + MLA (KV-compressed) layers with fine-grained MoE FFN, aux-loss-free bias routing, and ternary FFN-MoE weights (embeddings/attention/router stay BF16).
• —Trinity-Mirror controller: Thinker → Critic → Refiner chained processor (bounded loop, confidence calibration, auditable mirror ledger). Inquiry tokens route probes to specialist MoE experts (probe→expert coupling).
• —GUM optimizer (Innovation I1): GaLore-Unbiased + Muon — Muon FLOP efficiency with GaLore low-rank memory savings (~72% optimizer-state reduction, state/param ≈ 2.2 B).

Scaling ladder ([1 7, 0 8, 4 5] — mandatory mantissa pattern)

The scaling pattern is the mantissa digits `[1 7, 0 8, 4 5]` = 0.17M, 0.8M, 4.5M, each column multiplied by ×100 per level: 0.17M → 17M → 1.7B≈1.8B · 0.8M → 80M → 8B · 4.5M → 450M → 45B. This pattern is canonical and is reproduced here unchanged from the blueprint (§5).

Scaling beyond L1 is deferred until implementation planning is finalized and snapshot safety is guaranteed. L2 and above require external funding ("L3 ke atas = jika ada pendanaan"); capital buys compute, not debugging of an architecture that should already be validated at L0/L1.

Checkpoints in this repo

.pt files are complete training checkpoints (weights + config + history) in bf16; model.safetensors is the canonical 17M weight set.

File layout

MBG-1.0/                          (external archive — clean flow)
├── README.md            # this model card
├── config.json          # current model config (L1 quality model, v2x)
├── model.safetensors    # canonical weights (follows latest model update)
├── manifest.json        # provenance: config, metrics, params, SHA-256 of every golden
├── tokenizer/
│   └── mbg_bpe.json     # BPE tokenizer artifact
├── golden/              # complete training checkpoints (bf16 .pt)
├── source/              # training/eval/data source code (reproducibility)
└── reports/             # experiment reports (markdown + JSON)

Clean flow: the workspace snapshot holds only the controller (code, scripts, docs, corpus, manifest) — no `.pt`/`.safetensors`. All model weights and checkpoints live in this repo and are downloaded on demand (e.g. internal/ops/sync.sh latest / pull). Public vs internal: everything published here (source code, reports, weights) is intentionally public. Internal design/governance artifacts — the design blueprint, the Trinity-Mirror component spec, the internal protocols/ops documents, and the ops scripts — live under internal/ in the snapshot and are never pushed to this repo.

Dataset

The English corpora used for training/evaluation are published in the dataset repo:

`deeprcurs/MBG-1.0-data` — see the dataset card there.

Quick start (load 17M weights)

pip install torch safetensors

from safetensors.torch import load_file
weights = load_file("model.safetensors")   # 117 tensors (bf16; embeddings/norms fp32)
# Rebuild with src/mbg_mini_gpt.MbGPT(config.json) and load_state_dict(weights).

Integrity

manifest.json records the SHA-256 of every golden checkpoint so downloads can be verified before use.

License

Dual License (custom, source-available) — see `LICENSE.md`.

• —Personal / Non-Commercial Use: free, for private experimentation and research. Re-branding and claiming the work as your own are prohibited.
• —Commercial Use / Re-branding / derivative redistribution: requires the author's prior written permission (commercial license fee applies).

The license applies to the model weights/checkpoints, tokenizer, config, training/evaluation code, and the datasets & corpus. This is not an open-source license; it intentionally restricts redistribution and use.