CoolFace
Modelpublic

JANGQ-AI/Nemotron-Cascade-2-30B-A3B-JANG_4M

sourceHugging Faceotherupdated 19d agoView on Hugging Face
5likes209downloads
Model Card

<p align="center"> <a href="https://mlx.studio"><img src="https://raw.githubusercontent.com/jjang-ai/jangq/main/assets/mlx-studio-light.png" alt="MLX Studio" width="500"></a> </p>

<p align="center"> <a href="https://mlx.studio"><img src="https://mlx.studio/assets/screenshots/mlx-studio-featured.png?v=1" alt="MLX Studio App" width="600"></a> </p>

<h4 align="center"><a href="https://mlx.studio">MLX Studio</a> — the only app that natively supports JANG models with reasoning</h4>

IMO Gold Medal reasoning in 17 GB. Nemotron-Cascade-2 achieves 88% MMLU with reasoning at just 17 GB — fits on 16 GB MacBooks. Hybrid Mamba-2 SSM + MoE + Attention. Only 6 KV cache attention layers = minimal memory at long context.
LM Studio, Ollama, oMLX do NOT support JANG format. Use [MLX Studio](https://mlx.studio) or pip install "jang[mlx]>=2.1.5".

<p align="center"> <img src="https://raw.githubusercontent.com/jjang-ai/jangq/main/assets/jangq-logo-dark.png" alt="JANG" width="300"> </p>

<p align="center"> <a href="https://vmlx.net"><img src="vmlx-app.png" alt="vMLX — run JANG models on Apple Silicon" width="820"></a> </p>

<h3 align="center">⚡ All JANG models are meant to be run in <a href="https://vmlx.net">vMLX</a></h3>

<h3 align="center">Nemotron-Cascade-2-30B-A3B — JANG_4M (4.1-bit, 8-bit attention) — Reasoning</h3> <p align="center"><b>JANG</b> — Jang Adaptive N-bit Grading | The GGUF Equivalent for MLX</p>

<p align="center"> <a href="https://github.com/jjang-ai/jangq"><img src="https://img.shields.io/badge/GitHub-Source_Code-blue?logo=github" alt="GitHub"></a>&nbsp; <a href="https://pypi.org/project/jang/"><img src="https://img.shields.io/pypi/v/jang?label=PyPI&color=green" alt="PyPI"></a>&nbsp; <a href="https://jangq.ai"><img src="https://img.shields.io/badge/Web-jangq.ai-orange" alt="Website"></a>&nbsp; <a href="https://x.com/dealignai"><img src="https://img.shields.io/badge/X-@dealignai-black?logo=x" alt="X/Twitter"></a> </p>

JANG is fully open-source. Quantization engine, research, and full commit history: github.com/jjang-ai/jangq. Created by Jinho Jang.

Key Features

  • —93.0% MMLU (200 questions, reasoning mode) — IMO Gold Medal model in 17 GB
  • —55 tok/s generation, 154 tok/s prefill
  • —10.3 GB on disk, 10.3 GB GPU RAM (peak 11.1 GB)
  • —Reasoning mode: <think>...</think> step-by-step problem solving
  • —Tiny KV cache: only 6 attention layers, 0.2 GB at 32K context
  • —Hybrid architecture: Mamba-2 SSM + MoE (128 experts, top-6) + Attention

Results: JANG vs MLX (200-question MMLU)

Per-subject comparison. All models tested with and without reasoning.

SubjectJANG_2L No-ThinkJANG_2L ReasoningJANG_4M No-ThinkJANG_4M ReasoningMLX 4-bit No-ThinkMLX 4-bit ReasoningMLX 6-bit No-ThinkMLX 6-bit Reasoning
Abstract Algebra4/2015/209/2019/208/2018/207/2019/20
Anatomy13/2017/2015/2019/2014/2018/2017/2019/20
Astronomy17/2019/2018/2020/2017/2019/2019/2020/20
College CS7/2017/2010/2018/2011/2017/2011/2017/20
College Physics13/2020/2014/2019/2015/2020/2014/2020/20
HS Biology16/2019/2018/2020/2018/2020/2018/2020/20
HS Chemistry12/2019/2014/2019/2013/2019/2017/2019/20
HS Mathematics8/2015/208/2018/2010/2019/208/2020/20
Logical Fallacies12/2018/2014/2016/2014/2017/2013/2017/20
World Religions16/2017/2018/2018/2018/2018/2018/2018/20
Total118/200 (59.0%)176/200 (88.0%)138/200 (69.0%)186/200 (93.0%)138/200 (69.0%)185/200 (92.5%)142/200 (71.0%)189/200 (94.5%)

Summary

JANG_2LJANG_4MMLX 4-bitMLX 6-bit
MMLU (no-think)59.0%69.0%69.0%71.0%
MMLU (reasoning)88.0%93.0%92.5%94.5%
Size10.3 GB17 GB16.6 GB23.9 GB
GPU RAM10.3 GB17 GB~17 GB~24 GB
Speed55 tok/s———
Fits 24 GB?YESNONONO

JANG2L is the **only quantization that fits 24 GB Macs** while delivering 88% MMLU with reasoning. JANG4M beats MLX 4-bit (93.0% vs 92.5%) at the same 17 GB size.

Also see: JANG_2L (10 GB) — fits 16 GB Macs, 88% reasoning MMLU.

Specs

MetricValue
SourceNemotron-Cascade-2-30B-A3B
ArchitectureHybrid Mamba-2 SSM + MoE + Dense Attention
Layers52 (Mamba-2 + MoE + 6 Attention)
Experts128 per MoE layer, top-6 active (3B active params)
KV cache6 attention layers, 2 KV heads, 128 dim — 0.2 GB at 32K context
ProfileJANG_2L (CRITICAL=8, IMPORTANT=6, COMPRESS=2)
Average bits4.12 bpw
Disk size10.3 GB
GPU RAM10.3 GB (peak 11.1 GB)
Speed55 tok/s generation, 154 tok/s prefill

Requirements

  • —Apple Silicon Mac with 24+ GB unified memory
  • —[MLX Studio](https://mlx.studio) or pip install "jang[mlx]>=2.1.5"

Quick Start

bash
pip install "jang[mlx]>=2.1.5"
python
from jang_tools.loader import load_jang_model
from mlx_lm import generate

model, tokenizer = load_jang_model("JANGQ-AI/Nemotron-Cascade-2-30B-A3B-JANG_4M")

# With reasoning (recommended)
messages = [{"role": "user", "content": "Solve: what is the integral of x^2 * e^x?"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=True)
result = generate(model, tokenizer, prompt=prompt, max_tokens=2048)

# Without reasoning (faster)
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=False)
result = generate(model, tokenizer, prompt=prompt, max_tokens=100)

Technical Notes

  • —Mamba-2 SSM: Most layers use state-space models, enabling efficient long-context with minimal KV cache.
  • —Only 6 attention layers: KV cache is tiny (0.2 GB at 32K). Most models use 25-100% attention layers.
  • —nemotron_h architecture: Requires JANG loader for proper weight mapping. Standard mlx-lm has incomplete support.
  • —IMO Gold Medal: This model achieves competition-level mathematical reasoning at 30B scale.

<p align="center"> <b>JANG</b> — Created by <a href="https://jangq.ai">Jinho Jang</a> (eric@jangq.ai) · <a href="https://x.com/dealignai">@dealignai</a><br> <a href="https://github.com/jjang-ai/jangq">GitHub</a> · <a href="https://pypi.org/project/jang/">PyPI</a> · <a href="https://huggingface.co/JANGQ-AI">HuggingFace</a> </p>