Jayant2304/commitment-os

1# Improvement Evaluation Artifacts2 3This folder contains deterministic baseline-vs-trained-style evaluation outputs for all 15 CommitmentOS tasks.4 5This is **not** the same as the real LLM checkpoint comparison; see root **README** section **B) True LLM Learning Eval** and `artifacts/evals_llm/`.6 7## Files8 9- `eval_protocol.json`: fixed protocol (task set, seed, max steps, decode config)10- `baseline_eval.json`: per-task baseline rollouts11- `trained_eval.json`: per-task improved/trained-style rollouts (same protocol)12- `improved_eval.json`: alias of trained outputs for backward compatibility13- `comparison.csv`: task-by-task delta table14- `summary.json`: aggregate metrics (mean/median deltas, difficulty splits, steps, success)15- `case_study_hard_011.md`: concise before/after narrative for one hard scenario16- `reward_by_task.svg`: visual comparison of final reward by task17- `violations_before_after.svg`: visual comparison of commitment violations18 19## Reproduce20 21```bash22cd commitment_os23python3 evaluation/evaluate_improvement.py24python3 evaluation/plot_improvement.py25```26