laion/delphi-1e23-25b-stageE-rl-eval-artifacts
delphi-1e23 (25B) Stage-E RL — raw evalchemy eval artifacts Raw evalchemy (lm-eval v0.4.12) outputs for the delphi-1e23 25B Stage-E RL sweep (marin issue #6279). For each of 5 models (SFT wc50m baseline + 4 RL cells D1–D4) and 2 tasks: <TASK>_<MODEL>_results.json — aggregate metrics + full run config (accuracy for MATH500; exact_match flexible/strict for gsm8k). <TASK>_<MODEL>_samples.jsonl — per-example: problem, gold, model_output, extracted answer, correctness. MATH500 =… See the full description on the dataset page: https://huggingface.co/datasets/laion/delphi-1e23-25b-stageE-rl-eval-artifacts.
delphi-1e23 (25B) Stage-E RL — raw evalchemy eval artifacts
Raw evalchemy (lm-eval v0.4.12) outputs for the delphi-1e23 25B Stage-E RL sweep (marin issue #6279). For each of 5 models (SFT wc50m baseline + 4 RL cells D1–D4) and 2 tasks:
<TASK>_<MODEL>_results.json— aggregate metrics + full run config (accuracy for MATH500; exact_match flexible/strict for gsm8k).<TASK>_<MODEL>_samples.jsonl— per-example: problem, gold, model_output, extracted answer, correctness.
MATH500 = 0-shot, EVAL_MAX_GEN_TOKS=3072 (the #6279 held-out selector). gsm8k = 5-shot, EVAL_MAX_GEN_TOKS=2048 (2048 to avoid the 4k-context 5-shot overflow that made 3072 model-blind). All served via vLLM TP=6 (delphi has 42 attention heads → TP must divide 42), delphiv0 chat template, `skipspecialtokens=false`, `maxmodel_len=4096`.
Models: laion/delphi-1e23-wc50m-warmup-levanter-sft (baseline) + laion/delphi-1e23-wc50m-rl-{d1-rlvrmath,d2-ifeval,d3-dapomath,d4-math500}-32gpu.