Datasetpublic

PS-098/single-turn-eval-int_qwen3-4b_distill_teacher_reverse_kl_lr1e-7-n32

Single-turn eval — violetxi/int_qwen3-4b_distill_teacher_reverse_kl_lr1e-7 Generated by teaching/inference/single_turn_eval_vllm.py. One row per problem; samples is the list of model responses, scores is per-sample correctness, and mean/best/worst are the aggregates used by mean@N / best@N / worst@N. Eval results (n_samples_per_example = 32) Overall metric value n_examples 566 mean@32 0.3146 best@32 0.5883 worst@32 0.0919… See the full description on the dataset page: https://huggingface.co/datasets/PS-098/single-turn-eval-int_qwen3-4b_distill_teacher_reverse_kl_lr1e-7-n32.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes20downloads

Dataset Card

Single-turn eval — `violetxi/int_qwen3-4b_distill_teacher_reverse_kl_lr1e-7`

Generated by teaching/inference/single_turn_eval_vllm.py. One row per problem; samples is the list of model responses, scores is per-sample correctness, and mean/best/worst are the aggregates used by mean@N / best@N / worst@N.

Eval results (nsamplesper_example = 32)

Overall

metric	value
n_examples	566
mean@32	0.3146
best@32	0.5883
worst@32	0.0919
pass_rate	0.5883

Per data source

data_source	n	mean@32	best@32	worst@32	pass_rate
Hwilner/imo-answerbench	400	0.3165	0.5700	0.0975	0.5700
MathArena/aime_2026	30	0.5719	0.8333	0.1667	0.8333
MathArena/hmmtnov2025	30	0.4240	0.7000	0.0333	0.7000
d1shs0ap/unified-hard-set-with-student-solutions-guided-rl	76	0.0970	0.4737	0.0132	0.4737
opencompass/AIME2025	30	0.4740	0.7667	0.2000	0.7667

Run config

json

{
  "data_files": [
    "data/instruct/aime-2025/test.parquet",
    "data/instruct/aime-2026/train.parquet",
    "data/instruct/hmmt-nov-2025/train.parquet",
    "data/instruct/imo-answerbench/train.parquet",
    "data/instruct/int_dataset_split/val.parquet"
  ],
  "limit": null,
  "output": "teaching/inference/results/int_qwen3-4b_distill_teacher_reverse_kl_lr1e-7/summary.json",
  "record_log": "teaching/inference/results/int_qwen3-4b_distill_teacher_reverse_kl_lr1e-7/records.jsonl",
  "solver_model_path": "violetxi/int_qwen3-4b_distill_teacher_reverse_kl_lr1e-7",
  "solver_max_new_tokens": 16384,
  "temperature": 0.8,
  "top_p": 1.0,
  "n_samples": 32,
  "seed": 0,
  "tensor_parallel_size": 1,
  "data_parallel_size": 8,
  "gpu_memory_utilization": 0.85,
  "max_model_len": 40000,
  "max_num_batched_tokens": 40000,
  "enforce_eager": true,
  "dtype": "bfloat16",
  "enable_thinking": false,
  "hf_dataset_id": null,
  "hf_push": true,
  "hf_private": false,
  "hf_token": null,
  "git_sha": "fbe54923edf778b314b0e05bbb4ee9a806a43033",
  "wall_time_seconds": 13836.383792638779
}