flowxai/flowx-sentinel-gate-ministral-3b
FlowX Sentinel Gate (Ministral)
Decides whether a case in a regulated workflow can be actioned automatically (DECIDE) or must go to a human (ESCALATE), and when it escalates, says which of six categories applies. Emits JSON only.
Spec
Evaluation
Frozen held-out set, n=50, greedy decoding, scored by eval_v2.py. Reported across 3 training seeds (7, 42, 1337) that differ in nothing but the seed, because a single run cannot measure its own noise and this table makes claims against two other published models.
Same frozen set and scoring as every system below.
The metric that decides this detector is the false-negative rate: a missed escalation is the expensive error. On a parse failure the caller should fail safe to ESCALATE.
Limits, stated plainly
Exactly one claim here survives 3 seeds, and it is action accuracy. 95.7% to 98.0% against the Qwen3-4B incumbent's 93.5%: ahead on every draw, with a margin of 2.2 to 4.5 points rather than the top of the range.
Category accuracy is not a usable comparison at this sample size. It spans 75.6% to 90.0% across seeds that differ in nothing else, a 14.4-point swing around the incumbent's 80.0%. One seed loses to the incumbent and another beats it comfortably. Neither ordering is real.
The false-negative count is likewise indistinguishable. It spans 0 to 2 against the incumbent's 1, and the median is 1. This is the metric the detector is judged by, so it matters that the honest answer is "no measurable difference" rather than a win. Do not select between these models on that basis.
n is 50. A 4.5-point action difference is three cases, and a one-case false-negative difference is one case out of 43. Every figure above is quoted with its seed range because a single run cannot measure its own noise; the first version of this card reported one draw and claimed zero missed escalations, which a reseed did not reproduce.
Contrast-set gap: +43.0 points. Scored against a contrast set whose members differ in exactly one field, category accuracy falls to 36.1% (artifact eval_cross_oldmodel_newtest_ministral.json). That gap measures how much the model leans on phrasing and co-occurring fields rather than the deciding fact. It is not a corrected accuracy: the contrast set's own labels come from a single teacher model and separating "shortcuts removed" from "labels noisier" needs human adjudication. It is reported because a model card that omits it would overstate the headline number.
The training corpus is 227 rows of realistic synthetic scenarios grounded in real regulation citations, 85% ESCALATE. An attempt to expand it with generated minimal pairs regressed every metric and is documented rather than shipped.
Not a compliance control. This produces evidence about a routing decision. Obligations under any regulation sit with the operator of the system, not with a model.
Licence and attribution
Released under the Apache License, Version 2.0. Full text in LICENSE.
This model is a derivative work of [`mistralai/Ministral-3-3B-Instruct-2512`](https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512) by Mistral AI, itself licensed under Apache-2.0. That licence permits redistribution of derivatives and requires that attribution and a statement of changes travel with them, so both are carried here and in NOTICE.
Changes made to the base model, in order:
- Dequantized FP8 to bfloat16. The published checkpoint stores 3.03B parameters as
F8_E4M3with per-tensorweight_scale_invscales. MLX has no float8 dtype, so it cannot read the checkpoint at all. - Converted to MLX 4-bit, 4.501 bits per weight.
- Vision tower discarded. The base is a vision-language checkpoint; this is text-only.
- LoRA fine-tune fused in (rank 32, scale 16, dropout 0.05, top 16 layers).
No part of the base model's training data, evaluation results or documentation is reproduced here. The evaluation figures above are ours and were produced by eval_v2.py on our own frozen set.