StandardThinking/StandardOne-8B-LoRA

Standard One 8B (LoRA adapter)

Standard One scores a bounded set of answers for a supplied scenario and returns probabilities through POST /v1/systemone. It does not generate free-form response text. This repository contains the 8B LoRA adapter and a merge recipe; serving requires a merged checkpoint and the server code.

For ready-to-serve merged BF16 weights and server code, use StandardOne-8B.

In the reported served evaluations, 8B scores higher than 3B on the public standard and hard tiers; 3B has a lower median latency on the measured short-request profile. See Benchmarks for the measurement conditions and limitations.

[image]

The figure combines results from different measurement paths. See [Benchmarks](#benchmarks) for served versus offline conditions; measured 24 September 2026.

At a glance

• —Send a state and a bounded rubric to receive probabilities for the supplied labels: choice selects among labeled options, noul is yes/no, and score uses an ordinal scale. The endpoint scores the labels in one forward pass without decoding answer text after this adapter is merged and served.
• —In the same-run offline comparison with the untuned base, 8B improves on four of six suites, ties on public easy, and declines on public hard. These are not served-endpoint results.
• —Probabilities are temperature-scaled and calibration-checked (hard-tier ECE, distribution total-variation) — see Benchmarks below.
• —Multilingual: English plus Japanese, Chinese, Spanish, French, German, Portuguese and Russian, with a smaller Korean share. The merged checkpoint retains the Pixtral vision tower and accepts image data URLs; this card does not report a separate image-input benchmark.
• —Apache-2.0 throughout: base model, adapter, merged weights and server code.

Merge the adapter

import torch
from transformers import Mistral3ForConditionalGeneration
from peft import PeftModel

base = Mistral3ForConditionalGeneration.from_pretrained(
    "mistralai/Ministral-3-8B-Instruct-2512-BF16",
    revision="f6fae9795746f63c9be8344932f01275f3c63734",
    torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(base, ".").merge_and_unload()
model.save_pretrained("./StandardOne-8B-merged", safe_serialization=True)
# then copy the base snapshot's tokenizer / chat template / preprocessor / generation config
# files into ./StandardOne-8B-merged alongside the merged weights.

Then serve ./StandardOne-8B-merged with the sglang.launch_server and jev-adapter setup in StandardOne-8B's QUICKSTART.md, substituting the merged directory for --model-path (served model name standard-one-8b, no system prompt, --default-temperature 1.35). That guide also has the virtual-environment installation, request format and curl example.

Benchmarks

Served endpoint results (the release configuration). Merged BF16 weights through SGLang 0.5.20 and jev-adapter, native wording, no system prompt, one option order, T=1.35. Jev 1.13 was measured on the same items through its hosted endpoint; its probabilities are raw, with no temperature applied. These are our measurements, not official sealed-set JevBench scores.

Offline comparison with the untuned base. This separate transformers runner used native wording, no system prompt, one option order, T=1. The base and tuned checkpoint were scored by the same offline path; these numbers are indicative of the base-model change, not the served scores above.

Against the untuned base, four suites improve, public easy ties, and public hard falls by 5.41 percentage points on this offline run. Served and offline probabilities differ even on identical prompts, so use the served table for expected endpoint behavior. Hard-tier ECE at served T is 0.178 (T=1.35) against Jev 1.13's 0.099 raw, and mean TV to the stated distributions is 0.127 at served T against Jev's 0.192 raw; full calibration table: `docs/BENCHMARKS.md`.

Speed — raw serial latency on one H200 with SGLang 0.5.20, using a 242-decision profile averaging about 280 input tokens per decision. The 25.8 ms figure is p50 for this profile, not a latency guarantee for other request lengths, concurrency or hardware. Qwen checkpoints are untuned and shown for speed only; no accuracy comparison is implied.

Throughput on one H200 (hard+standard mix, 1,322 tokens/request): 29.3k tok/s at concurrency 1, rising to 39.5k tok/s at concurrency 64 (≈40 decisions/s at concurrency 8 on the 280-token profile above).

On the public classification and decision suites (400 cases/suite, seed 13, served endpoints): AG News 84.8 %, typed decisions 69.8 %, MASSIVE intent mean 86.4 %, email spam 87.2 %, phishing 84.8 %. Full table, per-language and per-workflow breakdown: `docs/BENCHMARKS.md` and `docs/public-classification-suites.md`.

A JevBench v1.4.1 run has been requested; the sealed-set result is not yet available.

Full report: `docs/BENCHMARKS.md`.

Model details

• —Base model: mistralai/Ministral-3-8B-Instruct-2512-BF16, revision f6fae9795746f63c9be8344932f01275f3c63734 (Apache-2.0).
• —Adapter: LoRA r=16, α=32, dropout 0, on q_proj k_proj v_proj o_proj gate_proj up_proj down_proj of the language-model projections only (vision tower and multimodal projector excluded), 44,564,480 trainable parameters, PEFT 0.21.0. Adapter file adapter_model.safetensors, 214,559,872 bytes, sha256 a2d9f02d5dd5d3ade3fab88145fc7321315609aa52d3847319dc4ec827f0c684.
• —Merged BF16 checkpoint (as published in StandardThinking/StandardOne-8B): merging this adapter into the base changes 238 tensors (293 unchanged), none outside the language-model projections, maximum absolute weight change 0.00201.
• —Serving details: native chat-template wording, no system prompt, fixed temperature (T = 1.35, fitted on held-out calibration data); served model name standard-one-8b behind stock SGLang 0.5.20 via jev-adapter (POST /v1/systemone); single caller-supplied option order, no rotation ensemble; 8,192-token context.

The merged BF16 checkpoint, the server code and the full quick-start guide are published in StandardThinking/StandardOne-8B.

Training data

Training data spans document and field normalisation, judge/routing and answer-adequacy, stated-distribution probability, verification/scoring, adequacy-rubric style, and game- and Tetris board-state cohorts — every row is synthetic and solver-generated (deterministic generators with visible checkers; a subset was teacher-reviewed), licensed Apache-2.0 (ours); the JevBench public tiers used only for evaluation carry MIT. Full per-cohort breakdown (row counts, what each covers, licence): `docs/BENCHMARKS.md`.

An exact-text overlap audit against the public JevBench tiers found 0 exact scenario matches and 181 exact instruction matches — rows in two adequacy-rubric cohorts whose entire instruction field, a generic 58-character adequacy question, is byte-identical to one public hard-tier instruction (0.05 % of the 359,497-row training mixture). These rows are kept and disclosed here rather than regenerated, since the overlap is limited to one rubric question's wording and never touches a scenario or an answer.

Limitations

• —Public hard tier: the served 8B score is 54.05 %, versus 72.07 % for Jev 1.13. In the separate offline base comparison, Standard One 8B scores 54.95 %, below the untuned base's 60.36 %.
• —Served probabilities are temperature-scaled by a single global value; if you apply this model to a materially different question distribution, re-fitting that temperature is advisable rather than assuming 1.35 transfers.
• —At most 26 options per question (one uppercase letter per option, A–Z).
• —The sealed JevBench set has not been measured for this model.
• —Served and offline probabilities can differ on identical prompts (mean total-variation ≈0.07 on the hard tier); served numbers are treated as authoritative.
• —Korean is a small share of multilingual training relative to the other seven languages.
• —The card reports text benchmarks; it does not establish decision accuracy on image inputs.
• —Ten-way support triage (34–40 %) and RAG passage relevance (58–61 %) are weak zero-shot; fine-tune for those.

Licence

Adapter weights, merge recipe and this card: Apache-2.0. Base model mistralai/Ministral-3-8B-Instruct-2512 (and -BF16): Apache-2.0 per its Hugging Face model card, which adds that the model must not be used in a way that infringes, misappropriates, or otherwise violates any third party's rights. jev-adapter and SGLang: Apache-2.0. The JevBench harness and public tiers used for evaluation: MIT; other benchmark items keep their own upstream terms.

Citation

StandardThinking/StandardOne-8B-LoRA (this repository, adapter + merge recipe) · StandardThinking/StandardOne-8B (merged weights + server code).