Datasetpublic

violetxi/single-turn-eval-meta_feedback_qwen3-4b_step2_gpt-5.4_gepa-n32

Single-turn eval — violetxi/meta_feedback_qwen3-4b_step2_gpt-5.4_gepa Generated by teaching/inference/single_turn_eval_vllm.py. One row per problem; samples is the list of model responses, scores is per-sample correctness, and mean/best/worst are the aggregates used by mean@N / best@N / worst@N. Eval results (n_samples_per_example = 32) Overall metric value n_examples 1006 mean@32 0.1796 best@32 0.3588 worst@32 0.0477 pass_rate… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/single-turn-eval-meta_feedback_qwen3-4b_step2_gpt-5.4_gepa-n32.

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes26downloads

Dataset Card

Single-turn eval — `violetxi/meta_feedback_qwen3-4b_step2_gpt-5.4_gepa`

Generated by teaching/inference/single_turn_eval_vllm.py. One row per problem; samples is the list of model responses, scores is per-sample correctness, and mean/best/worst are the aggregates used by mean@N / best@N / worst@N.

Eval results (nsamplesper_example = 32)

Overall

metric	value
n_examples	1006
mean@32	0.1796
best@32	0.3588
worst@32	0.0477
pass_rate	0.3588

Per data source

data_source	n	mean@32	best@32	worst@32	pass_rate
CohenQu/POPE-hard-dataset-Qwen3-4B-Instruct-32k-128-filtered-iter3-gemini-success	440	0.0097	0.0705	0.0045	0.0705
Hwilner/imo-answerbench	400	0.3155	0.5700	0.0850	0.5700
MathArena/aime_2026	30	0.5177	0.8333	0.1000	0.8333
MathArena/hmmtnov2025	30	0.4240	0.7667	0.0667	0.7667
d1shs0ap/unified-hard-set-with-student-solutions-guided-rl	76	0.1106	0.4079	0.0263	0.4079
opencompass/AIME2025	30	0.4510	0.7667	0.1667	0.7667

Run config

json

{
  "data_files": [
    "data/instruct/aime-2025/test.parquet",
    "data/instruct/aime-2026/train.parquet",
    "data/instruct/hmmt-nov-2025/train.parquet",
    "data/instruct/imo-answerbench/train.parquet",
    "data/instruct/int_dataset_split/val.parquet",
    "data/instruct/pope_full/test.parquet"
  ],
  "limit": null,
  "output": "teaching/inference/results/meta_feedback_qwen3-4b_step2_gpt-5.4_gepa_20260504_171552/summary.json",
  "record_log": "teaching/inference/results/meta_feedback_qwen3-4b_step2_gpt-5.4_gepa_20260504_171552/records.jsonl",
  "solver_model_path": "violetxi/meta_feedback_qwen3-4b_step2_gpt-5.4_gepa",
  "solver_max_new_tokens": 16384,
  "temperature": 0.8,
  "top_p": 1.0,
  "n_samples": 32,
  "seed": 0,
  "tensor_parallel_size": 1,
  "data_parallel_size": 5,
  "gpu_memory_utilization": 0.85,
  "max_model_len": 40000,
  "max_num_batched_tokens": 40000,
  "enforce_eager": true,
  "dtype": "bfloat16",
  "enable_thinking": false,
  "hf_dataset_id": null,
  "hf_push": true,
  "hf_private": false,
  "hf_token": null,
  "git_sha": "d6c6facd2310705959045a27e6c7e087bf9361be",
  "wall_time_seconds": 18859.761454820633
}