CMSManhattan/JiRackTernaryGemma4_26b

Gemma 4 26B-A4B migrated to Ternary Architedure

Gemma 4 26B-A4B (MoE) migrated to the JiRack ternary architecture.

• —Text-only: official vision tower dropped (~550M params), decoder kept
• —~26B total / ~4B active per token (128 experts, top-8)
• —Ternary (BitNet b1.58) QAT path for CPU / GGUF / Ollama
• —Vision removed for RAM/VRAM optimization . So it safes much memory for large context
• —Robotics, routing, coding and tool-call tags via CMSManhattan/GemmaRoboticsTokenizer

JiRack service options

• —Current quantizations were done from the FP16 model.
• —If you need custom compression or fine-tuning, please write to me and I'll perform QAT from your dataset, tailored specifically to your task.
• —Plus double QAT via ONNX QAT.
• —Adapt train process to avoid catastrophic forgetting with NDA
• —Adapt train process to avoid fast plateau in training with NDA
• —Adapts to agentic or instruct models for tool calling, using the JiRack tokenizer to enable high-quality tool calling on small models — built as a domain-specific tool expert.
• —Deployment and scale

Partnership

• —NVIDIA Inception
• —FISERV

Architecture

From google/gemma-4-26B-A4B-it text config:

What is ternarized: q/k/v/o, dense MLP, all 128 experts. What stays full precision: embeddings / tied lm_head, router, norms, layer_scalar.

Class: JiRackTernaryGemma4_26b (JiRackTernaryGemma4_26b.py).

Files

• —config.json — Hub card + auto_map to the JiRack class
• —model.safetensors.index.json + model-00001-of-00013.safetensors … 00013 (~49 GB bf16)
• —GGUF: JiRackTernaryGemma4-26b-f16.gguf, Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M
• —Tokenizer: [CMSManhattan/GemmaRoboticsTokenizer](https://huggingface.co/CMSManhattan/GemmaRoboticsTokenizer)

Tokenizer

Use [CMSManhattan/GemmaRoboticsTokenizer](https://huggingface.co/CMSManhattan/GemmaRoboticsTokenizer) with this checkpoint.

• —Gemma 4 chat template (<|turn>user / <|turn>model / <turn|>)
• —Extra tags: robotics, routing (__ROBOTICS__, __CODING__, …), FIM, media, mood
• —Vocab: 262,251 (pad=0, eos=1)

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("CMSManhattan/GemmaRoboticsTokenizer")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Hello"}],
    add_generation_prompt=True,
    enable_thinking=False,
    tokenize=False,
)

Official google/gemma-4-26B-A4B-it tokenizer also works for plain chat (vocab 262,144). Robotics / routing tags need the extended tokenizer.

The extra 107 tokens sit past this checkpoint’s token_emb (262,144). They are safe in the template and in datasets; do not emit those ids until you resize_token_embeddings(len(tokenizer)) and train the new rows.

Training Notice

• —During Quantization-Aware Training, the routing mechanism should remain frozen. Router training should only commence once the lambda parameter reaches 1.0 and the base model's quality has been verified. (Note: "lamba" was corrected to "lambda" in these revisions, assuming it refers to the standard scaling parameter often used in QAT and straight-through estimators).

Run Notice

• —Please note that the model has not been tested on large contexts yet. Let me know if you notice any difference in quality compared to the original. ---

Chat template

Gemma 4 native (<|turn>user / <|turn>model / <turn|>). Thinking is a template flag (enable_thinking), not a </think> suffix. With thinking off the model still emits an empty <|channel>thought\n<channel|> block — strip it from history.

Stop tokens: <turn|> (106) and <eos> (1). Sampling (Gemma 4 rec.): temperature=1.0, top_p=0.95, top_k=64.

Quick start (PyTorch)

from transformers import AutoTokenizer
from JiRackTernaryGemma4_26b import JiRackTernaryGemma4_26b, JiRackConfig

tok = AutoTokenizer.from_pretrained("CMSManhattan/GemmaRoboticsTokenizer")
# load shards via model.safetensors.index.json into JiRackTernaryGemma4_26b
# model.set_lambda(0.0)

Or the included chat script:

export JIRACK_TOKENIZER=CMSManhattan/GemmaRoboticsTokenizer
python chat_jirack_gemma4_26b.py

GGUF / llama.cpp / Ollama

Current quants were made from the F16 export.

Recommended everyday: Q4_K_M. CPU boxes without AVX2: export MKL_ENABLE_INSTRUCTIONS=AVX.

Ollama tag (when published): cmsmanhattan/JiRackTernaryGemma4-26b-q4

• —https://ollama.com/cmsmanhattan/JiRackTernaryGemma4-26b-q4

ollama run cmsmanhattan/JiRackTernaryGemma4-26b-q3 "Hello"
ollama run cmsmanhattan/JiRackTernaryGemma4-26b-q4 "Hello"
ollama run cmsmanhattan/JiRackTernaryGemma4-26b-q5 "Hello"
ollama run cmsmanhattan/JiRackTernaryGemma4-26b-q6 "Hello"
ollama run cmsmanhattan/JiRackTernaryGemma4-26b-q8 "Hello"

Hardware

bf16 safetensors need ~50 GB RAM/VRAM resident even though only ~4B params fire per token.

Lineage

Text decoder follows google/gemma-4-26B-A4B-it. Tokenizer: CMSManhattan/GemmaRoboticsTokenizer. This card is the JiRack ternary port, not a Google release. Do not treat upstream Gemma 4 leaderboard numbers as this checkpoint’s scores.

JiRack Codding Agent IDE

• —It is Agent Coding IDE for JiRack Models to run via Ollama on home PC
• —It good choose for Agent Coding IDE such as Cursor , Windsurf IDE or Devin IDE etc but more safe that ask you to apply changes and review.
• —Web site https://www.jirack.com
• —Plugin https://marketplace.eclipse.org/content/jirack-coding-agent
• —Final release version https://huggingface.co/CMSManhattan/JiRackDeltaNet27b/resolve/main/jirackide_final.zip

Contact

• —Email: grabko@cmsmanhattan.com
• —Phone: +1 (516) 777-0945
• —New York, USA

License

MIT for the Gemma 4 base lineage. JiRack UI / Docker / paid Ollama builds are separate commercial products.