atrost/nanochat-d24-sft-chat-eval-v1
nanochat d24 SFT Chat Eval Capture Per-example outputs for the nanochat chat eval tasks across dense, nested, and MatFormer SFT models. Each dataset config corresponds to one model. Each split corresponds to one eval task. Important columns include input_prompt, rendered_prompt, model_response, correct, task_logical_index, example_key, and forward-inference FLOP estimates split into flops_prefill, flops_decode, and flops_total. Dataset repo:… See the full description on the dataset page: https://huggingface.co/datasets/atrost/nanochat-d24-sft-chat-eval-v1.
nanochat d24 SFT Chat Eval Capture
Per-example outputs for the nanochat chat eval tasks across dense, nested, and MatFormer SFT models.
Each dataset config corresponds to one model. Each split corresponds to one eval task.
Important columns include input_prompt, rendered_prompt, model_response, correct, task_logical_index, example_key, and forward-inference FLOP estimates split into flops_prefill, flops_decode, and flops_total.
Dataset repo: atrost/nanochat-d24-sft-chat-eval-v1
The categorical tasks keep nanochat's default evaluation semantics: the model response is the highest-logit answer letter among the allowed choices, not a sampled free-form completion.