vinci00/ministral-3-benchmark-prompts

Ministral 3 MLX benchmark prompts

This tiny dataset contains the four fixed prompts used by the reproducible smoke benchmark for the Ministral 3 MLX 4-bit model. It is a benchmark fixture, not a training or fine-tuning dataset.

Schema

Each JSONL row contains:

• —id: stable case identifier;
• —language: prompt language;
• —prompt: exact input sent to the model;
• —expected_keywords: lowercase substrings used by the smoke check.

The benchmark uses greedy decoding and checks whether every expected keyword appears in the generated text. The four checks cover arithmetic, a short science explanation, Italian output, and constrained JSON. They are intended to catch loading or regression failures, not to estimate general model quality.

Reproduction

The runner and exact settings are in `benchmark_mlx.py`. The benchmark records outputs and latency while using max_tokens=96.