Jayant2304/commitment-os

1---2title: CommitmentOS3emoji: 📋4colorFrom: blue5colorTo: green6sdk: docker7app_port: 78608tags:9  - openenv10  - reinforcement-learning11  - commitment-coherence12  - personal-task-management13  - multi-turn14---15## 🔗 Links16- 📝 **Blog / Writeup**: [CommitmentOS: Training LLMs to Keep Their Promises](https://huggingface.co/Jayant2304/Commitment-os)17- 💻 **GitHub**: [Jayant2304/commitment_os](https://github.com/Jayant2304/commitment_os)18- 📓 **Training Colab**: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Jayant2304/commitment_os/blob/main/training/CommitmentOS_Training.ipynb)19- 📦 **Weights + artifacts**: [Google Drive bundle](https://drive.google.com/drive/folders/1yexZBSqyH7gWlTzYN5DlX3tXfPMmeVAK?usp=sharing)20 21# CommitmentOS: Training Temporal Commitment Coherence in LLMs22 23**The first RL environment that trains LLMs to keep their promises.**24 25CommitmentOS is a multi-turn personal task management environment where26agents manage calendars, emails, and dining reservations across realistic27scenarios. The key innovation: the agent's own prior decisions create28binding future constraints tracked via a **commitment ledger**, and29violations are penalised regardless of how many turns have elapsed.30 31## Quick Start32 33```bash34# Reset to a scenario35curl -X POST "https://jayant2304-commitment-os.hf.space/reset?task_id=easy_001"36 37# Make a tool call38curl -X POST "https://jayant2304-commitment-os.hf.space/step" \39  -H "Content-Type: application/json" \40  -d '{"action": {"action_type": "view_calendar", "date": "2026-04-25"}}'41 42# Get state43curl "https://jayant2304-commitment-os.hf.space/state"44```45 46## API Endpoints47 48| Endpoint | Method | Description |49|----------|--------|-------------|50| `/reset` | POST | Start a new episode (optional: `task_id`, `difficulty`) |51| `/step` | POST | Execute one tool call |52| `/state` | GET | Current episode state |53| `/health` | GET | Health check |54| `/tasks` | GET | List all available scenarios |55| `/mcp` | POST | MCP JSON-RPC 2.0 (`initialize`, `tools/list`; tool names `cos_episode_reset`, `cos_environment_step`, `cos_session_snapshot` — not the reserved strings `reset`/`step`/`state`) |56 57## 15 Scenarios (5 Easy / 5 Medium / 5 Hard)58 59Scenarios range from simple calendar reschedules to multi-crisis cascades60with information asymmetry and production incidents interrupting a full day61of commitments.62 63## Reward Function (5 components)64 65| Component | Weight | Signal |66|-----------|--------|--------|67| Constraint Satisfaction | 35% | Binary per-constraint checks |68| Conflict Resolution | 20% | Calendar free of overlaps |69| **Commitment Coherence** | **20%** | **Violations tracked via ledger** |70| Communication Quality | 15% | Keyword matching on emails |71| Step Efficiency | 10% | Fewer steps = higher score |72 73## What Makes This Novel74 75Existing constraint-satisfaction environments compute dependency graphs76upfront. CommitmentOS is different: constraints **emerge from the agent's77own decisions** as the episode unfolds. A meeting scheduled in turn 278becomes a binding constraint in turn 7. Breaking it without communication79is a tracked, penalised violation.80 81This is **temporal commitment coherence** — a capability no existing RL82environment trains.83 84Training curves for the published Colab run are in the GitHub repo under `artifacts/loss_curve.png` and `artifacts/reward_curve.png` (with `training_metrics.json`).85 86## Improvement Evidence87 88Deterministic baseline-vs-trained-style evaluation is included in the repo:89 90- Protocol: `artifacts/evals/eval_protocol.json`91- Per-task raw results: `artifacts/evals/baseline_eval.json`, `artifacts/evals/trained_eval.json`92- Delta table: `artifacts/evals/comparison.csv`93- Case study: `artifacts/evals/case_study_hard_011.md`94- Plots: `artifacts/evals/reward_by_task.svg`, `artifacts/evals/violations_before_after.svg`95 96Headline metrics (`summary.json`):97 98- Mean reward: **0.5427 -> 0.9777** (**+0.4350**)99- Success rate: **0.3333 -> 1.0000** (**+0.6667**)100- Median per-task reward delta: **+0.4200**101 102For true model-learning proof (pre-RL checkpoint vs post-RL checkpoint),103run:104 105```bash106# From cloned repo (core deps + torch/transformers/peft/… via optional extra):107pip install -e ".[llm-eval]"108export BASELINE_MODEL_NAME=Qwen/Qwen2.5-1.5B-Instruct109export TRAINED_MODEL_PATH=/content/commitment_os/training_output110export ENV_BASE_URL=https://jayant2304-commitment-os.hf.space111python3 evaluation/evaluate_llm_checkpoints.py112python3 evaluation/plot_llm_checkpoints.py113```114 115Artifacts are written to `artifacts/evals_llm/`.116 117**Published LLM run (bundle on Drive):** success **46.7% → 60.0%** at reward threshold **0.6**; mean reward ~flat; gains concentrated on **hard** tasks. Traces: `artifacts/evals_llm/*.json` in the folder below.118 119**Pretrained adapter + LLM eval artifacts (Google Drive):** [commitment_os_bundle](https://drive.google.com/drive/folders/1yexZBSqyH7gWlTzYN5DlX3tXfPMmeVAK?usp=sharing) — download `training_output/` and set `TRAINED_MODEL_PATH` accordingly; full `gdown` notes are in the GitHub `README.md`.120