Modelpublic

apus-ailab/APUS-OpenJev-v1-35B-A3B-GGUF

sourceHugging Faceapache-2.0updated 4d agoView on Hugging Face

1likes3.4kdownloads

Model Card

APUS-OpenJev-v1-35B-A3B-GGUF

English | 中文 · Source model · Collection · GGUF collection · MLX collection · MLX (Apple Silicon)

GGUF conversions of APUS-OpenJev-v1-35B-A3B (revision 494738529137) for Ollama, llama.cpp and LM Studio on Linux, Windows and macOS (Metal).

OpenJev is a decision model: each request supplies a state, an instruction and 2–16 candidates; the model answers with one candidate label (A–P) and the application reads the distribution over those labels. It is not a chat model.

Files and parity

Every file was scored on the Frozen80 panel with identical prompt tokens and compared with the HF BF16 release run by its own runtime (full depth, 71/80 · 88.75%).

File	Size	Frozen80	Decisions = HF BF16	Max Δp vs HF BF16
Q8_0	34.4 GiB	71/80 · 88.75%	80/80	0.2384
Q4_K_M	19.7 GiB	72/80 · 90.00%	79/80	0.2944
BF16	64.6 GiB	72/80 · 90.00%	79/80	0.1790

Rows are llama-server b11118 on an NVIDIA RTX PRO 6000 (CUDA) unless marked Apple M5 (Metal). Q80 is the recommended default; Q4KM uses an importance matrix. Quantized scores are measured separately and do not inherit the BF16 result. Frozen80 is a reused development panel, not a blind benchmark. Per-file details: [evaluation/](evaluation/). Per-question rows (candidate probabilities, choice, correctness; join with Frozen80 by `panelindex): [evaluation/per-question/](evaluation/per-question/). The BF16 file exceeds the Hub's 50 GB limit and is split with llama-gguf-split; point llama.cpp at APUS-OpenJev-v1-35B-A3B-BF16-00001-of-00002.gguf` and it loads the remaining parts (Ollama does not import split GGUFs).

Ollama

bash

ollama run hf.co/apus-ailab/APUS-OpenJev-v1-35B-A3B-GGUF:Q8_0 --think=false

Always disable thinking. Ollama renders this architecture with its built-in Qwen3.5 renderer (the Modelfile TEMPLATE is not applied), and that renderer opens a thinking block unless told otherwise, which breaks the trained no-thinking contract. Use --think=false on the CLI, "think": false in /api/chat / /api/generate, or send the fully rendered prompt with "raw": true. params sets temperature 0 and num_predict 1; a local Modelfile is included.

To get candidate probabilities, use the bundled client (raw mode):

bash

python examples/openjev_local.py --backend ollama --model hf.co/apus-ailab/APUS-OpenJev-v1-35B-A3B-GGUF:Q8_0

Ollama returns at most 20 top_logprobs and cannot report named tokens, so the distribution is exact only when every candidate label is in the top 20 (Q8_0: 71/80 Frozen80 prompts); otherwise use the selected label or llama-server.

Measured with Ollama 0.34.3 and the Q8_0 file on Linux (NVIDIA RTX PRO 6000, CUDA):

Request	Label = llama-server	Frozen80	Prompt tokens = training
`/api/generate`, `raw: true`, full prompt (examples/openjev_local.py)	80/80	71/80	80/80
`/api/chat` or `ollama run`, `think: false`	80/80	71/80	80/80
`/api/generate`, thinking left on (wrong)	0/80	0/80	0/80

llama.cpp (exact distribution)

bash

llama-server -m APUS-OpenJev-v1-35B-A3B-Q8_0.gguf -c 9216 -ngl 999
python examples/openjev_local.py --backend llama-server --url http://127.0.0.1:8080

examples/openjev_local.py renders prompts with openjev_contracts.py, the same contract used in training.

Conversion

—llama.cpp b11118 (e6ab7c1a4), convert_hf_to_gguf.py --no-mtp (the merged release has no MTP weights).
—Q4KM importance matrix: 448 training-course decisions, 64 per source, disjoint from Frozen80 (details, imatrix.gguf).
—All 1-D tensors, including GDN A_log / dt_bias and norms, stay F32 in every file (check).
—Scope: full depth only (the 16/20-layer low exit is not available), text only (no vision tower), probabilities are not calibrated.

License

Apache-2.0, inherited from the source model; see LICENSE. Base model: Qwen/Qwen3.5-35B-A3B.

Authors: gumpcheng (xDAN2099), zhangxu, APUS AI-LAB.