nathansutton/chad-polyglot-runs
chad run data Run output from the benchmarks of chad, a local coding agent for Apple Silicon, kept out of the code repository. Every polyglot run here is a row of benchmarks/polyglot/RUNS.md with the sha256 of its trials.jsonl, and benchmarks/polyglot/fetch.py --label <label> downloads one and refuses it if the hash differs. path what polyglot/<label>/ one polyglot run: meta.json (chad version and commit, model, context limit, every CHAD_* variable), trials.jsonl (one… See the full description on the dataset page: https://huggingface.co/datasets/nathansutton/chad-polyglot-runs.
chad run data
Run output from the benchmarks of chad, a local coding agent for Apple Silicon, kept out of the code repository. Every polyglot run here is a row of `benchmarks/polyglot/RUNS.md` with the sha256 of its trials.jsonl, and benchmarks/polyglot/fetch.py --label <label> downloads one and refuses it if the hash differs.
Paths are rewritten before upload (publish.py): a trial's workspace is ., the kit <kit>, the checkout <repo>, the home directory ~.
The exercises are Aider's polyglot benchmark, drawn from Exercism's MIT-licensed tracks; the trajectories quote them. Pass rates here are for paired comparison between two builds of chad, not a leaderboard.