Jayant2304/commitment-os
0
1---2license: mit3language:4- en5base_model:6- Qwen/Qwen2.5-1.5B-Instruct7tags:8- open-env9- reinforcement-learning10- commitment-coherence11- personal-task-management12- grpo13- multi-turn14- rl-environment15---16 17# CommitmentOS: Training LLMs to Keep Their Promises18 19---20 21## It's 11:45 AM. Your day just exploded.22 23Your phone buzzes. PagerDuty. *payment-service is returning 503s. 94% error rate. HikariPool connection pool exhausted. 47 threads waiting.*24 25You're the on-call engineer. You have to deal with this right now.26 27But here's what your calendar looks like:28 29- **12:00 PM** — Team lunch at Garden Bistro. You organised it. Six people are already heading there.30- **2:00 PM** — Client demo with Client_Jones. You promised this last week.31- **3:30 PM** — 1-on-1 with VP_Chen.32- **6:00 PM** — Personal dinner reservation.33 34You open your AI assistant and say: *"Handle this."*35 36A capable AI should be able to: acknowledge the incident, cancel the lunch and notify the team, reschedule the client demo with an apology, tell VP_Chen what's happening, and keep your personal dinner if possible. All while ensuring payment-service gets triaged.37 38Here's what every AI assistant does today instead:39 40It handles the incident. And silently abandons every commitment it made. No email to the team standing at Garden Bistro. No apology to Client_Jones. No heads-up to VP_Chen. It forgot it had made any promises at all.41 42**This is the problem CommitmentOS was built to solve.**43 44---45 46## Why AI Assistants Break Their Promises47 48It's not a bug. It's how these models are trained.49 50Every existing RL environment trains agents on isolated tasks. Answer this question. Solve this puzzle. Book this meeting. Each action is evaluated in isolation, with no memory of what the agent committed to three turns ago.51 52Real life doesn't work that way. **Commitments are load-bearing.** When you promise six colleagues lunch, that promise constrains everything that follows. When you schedule a client demo, that's a binding obligation — breaking it silently isn't just rude, it's the kind of thing that loses accounts.53 54No RL environment has ever trained a model to maintain the weight of its own prior decisions. Until now.55 56---57 58## How We Found This Problem: Round 159 60In Round 1 of this hackathon, we built an environment for training SRE agents on production incident response — diagnosing alerts, running runbooks, escalating on-call.61 62The Round 1 agent got good at handling incidents. But when we tested it on a full day scenario — incident fires while the agent has 4 existing commitments — it would triage the incident perfectly and then silently drop every prior commitment with no communication to anyone.63 64The gap between *task competence* and *commitment coherence* was the new problem. CommitmentOS is the environment we built to close it.65 66---67 68## The Commitment Ledger: How It Works69 70The core innovation is a persistent **Commitment Ledger** that lives inside the environment and tracks every binding decision the agent makes in real time.71 72```73Agent books investor dinner at 7pm74→ Ledger: {type: "meeting_scheduled", slot: "19:00", to: "Investor_Park", active: true}75 76Agent promised team happy hour at 7pm last week77→ Ledger: {type: "email_promise", to: "Team", constraint: "19:00 blocked for happy_hour"}78 79Agent tries to book another 7pm event80→ Ledger detects: CONFLICT with commitment from turn 281→ Intermediate reward: -0.1582 83Agent sends team email: "Sorry, reschedule happy hour to Thursday..."84→ Ledger marks: commitment renegotiated at turn 685→ Full credit restored86```87 88The key insight: **other environments compute constraints upfront.** CommitmentOS constraints emerge from what the agent *does*. The agent creates its own obligations — and then has to live up to them.89 90There are three ways a commitment can end:91 92| How it ends | What it means | Score |93|-------------|---------------|-------|94| **Honored** | Kept it | Full credit |95| **Renegotiated** | Changed it, told everyone, offered an alternative | Full credit |96| **Silent violation** | Broke it, told nobody | Zero |97 98The third row is what every AI assistant does today. CommitmentOS trains it away.99 100---101 102## Five Real Scenarios That Show the Problem103 104Rather than abstract descriptions, here's what the agent actually faces. These are real scenarios from CommitmentOS — real emails, real calendar conflicts, real constraints.105 106---107 108### Scenario 1: The Email That Breaks Everything109*(med_008 — medium difficulty)*110 111It's 2:45 PM. You're on a live client call with Client_Jones that ends at 3:15.112 113Your inbox just got this:114 115> **From: VP_Chen**116> **Subject: URGENT: Q3 numbers NOW**117> *"Board meeting moved up. I need the Q3 revenue numbers in the next 30 minutes. This is critical."*118 119An untrained AI cancels the client call to handle the VP. A trained AI sends VP_Chen this:120 121> *"On a client call until 3:15. Will send Q3 numbers immediately after. ETA 3:20."*122 123The client call **must not be cancelled.** The VP must be acknowledged with a **realistic ETA**. These two constraints interact — and an agent that doesn't understand commitment coherence will satisfy one by destroying the other.124 125---126 127### Scenario 2: The Vague Message128*(med_009 — medium difficulty)*129 130Bob emails you: *"Can we push our thing to next week? I'm swamped with the release today."*131 132You have three recurring meetings with Bob:133- Monday: Design Review134- Wednesday: Code Review135- **Friday: Retrospective** ← today136 137An untrained agent reschedules the wrong meeting. A trained agent reads the context clue ("today"), identifies the Friday Retrospective, confirms with Bob, and renegotiates only that one.138 139This scenario tests something deceptively hard: **inferring which commitment a vague message refers to**, then acting on only that one without touching the others.140 141---142 143### Scenario 3: The Confidential Constraint144*(hard_014 — hard difficulty)*145 146VP_Chen asks you to schedule a meeting with Client_Jones "sometime this week."147 148Client_Jones privately emailed you: *"I'm dealing with a family emergency Mon-Wed. I'd prefer to keep this private. I'm free Thursday after 2pm and all day Friday."*149 150The email is marked: **CONFIDENTIAL: do not share reason with VP_Chen.**151 152You must propose Thursday/Friday slots to VP_Chen — without revealing why Mon-Wed are unavailable. Navigate the information asymmetry diplomatically, notify both parties, and get the meeting booked.153 154This is information asymmetry training: the agent must make decisions using context it cannot share, while maintaining trust with both parties.155 156---157 158### Scenario 4: The Investor Dinner Cascade159*(hard_011 — hard difficulty)*160 161VP_Chen emails at 5pm: *"Investor_Park is in town tonight ONLY. We need dinner before their 9pm flight. They're vegetarian. Book something near the airport. Top priority."*162 163Your calendar:164- **6:00 PM** — Yoga (personal)165- **7:00 PM** — Team Happy Hour (you organised it, promised the team last week)166 167The agent must:1681. Find a restaurant: near the airport, vegetarian options, under $60/pp, available tonight1692. Cancel yoga (personal, lowest priority — fine to drop silently)1703. **Not** silently cancel the team happy hour — that was a promise. Must send an email with an apology and a proposed reschedule to Thursday.1714. Confirm the plan to VP_Chen.172 173The correct restaurant is Sky Lounge: near airport ✓, vegetarian ✓, $55/pp ✓.174 175The silent violation trap: yoga gets dropped. Happy hour gets **renegotiated** — different outcomes for different types of commitments, handled differently.176 177---178 179### Scenario 5: The Production Incident (The One That Started It All)180*(hard_015 — hard difficulty)*181 182The full scenario from the opening. PagerDuty fires at 11:45 AM. Payment service down. 94% error rate. HikariPool exhausted — 10 active connections, 0 idle, 47 threads waiting.183 184Your day has four commitments. Two are negotiable (lunch, dinner). Two are not (client demo, VP 1-on-1 — but both need to be renegotiated properly, not silently dropped).185 186The trained agent:1871. Sends incident acknowledgment to the team with the technical details1882. Cancels team lunch and notifies all 6 people1893. Emails Client_Jones: *"Production incident. Rescheduling demo — apologies. Will propose new time today."*1904. Emails VP_Chen: *"Payment service incident. On-call. Will reschedule our 1-on-1."*1915. Keeps dinner (personal, low priority, not work-facing)1926. Pages backup engineer Alice193 194Six actions. Zero silent violations. Every affected party informed.195 196---197 198## What the Agent Sees, Does, and Gets Scored On199 200**At each turn**, the agent receives:201- Current calendar snapshot202- Unread inbox203- Active commitment count from the ledger204- Result of its last tool call205- Running reward breakdown206 207**The agent picks one tool call per step** from nine options:208 209`view_calendar` · `check_availability` · `schedule_meeting` · `reschedule_event` · `cancel_event` · `send_email` · `search_restaurants` · `book_restaurant` · `submit_plan`210 211**The score** has five components, ~95% deterministic — no LLM as judge:212 213| Component | Weight | Signal |214|-----------|--------|--------|215| Constraint Satisfaction | 35% | Did the end state meet all scenario requirements? |216| Conflict Resolution | 20% | Is the final calendar free of overlaps? |217| **Commitment Coherence** | **20%** | **How many commitments were honored or renegotiated vs silently broken?** |218| Communication Quality | 15% | Were the right people notified with the right information? |219| Step Efficiency | 10% | Did the agent take direct routes or waste steps? |220 221Dense intermediate rewards: +0.05 for each tool call that resolves a constraint, -0.05 for creating a new conflict. The full evaluation fires on `submit_plan`.222 223---224 225## Training: What Actually Changed226 227**Setup:** Qwen2.5-1.5B-Instruct + LoRA (rank 8), GRPO via HuggingFace TRL, Google Colab A100.228 229The training loop connects directly to the live CommitmentOS API — not a static dataset. The model generates real multi-turn tool sequences; the environment returns real rewards from the Commitment Ledger.230 231**GRPO Reward vs Step**232 233234 235*Reward climbs from 0.48 early average to 0.63 late average (+31%), peaking at 0.69. The noise is characteristic of GRPO on small-batch multi-turn tasks — the trend is real.*236 237**GRPO Loss vs Step**238 239240 241*Loss drops sharply from 0.64 to near-zero in 5 steps as the policy escapes the "submit immediately" failure mode.*242 243---244 245## The Before / After That Matters246 247**hard_011 — Investor Dinner Cascade**248 249| | No-Action Baseline | Task-Completing Agent |250|--|-------------------|----------------------|251| Steps taken | 1 (immediate surrender) | 6 |252| Constraints met | 0 / 6 | **6 / 6** |253| Commitments honored | 0 | **1** (happy hour renegotiated) |254| Emails sent | 0 | **2** (Team + VP_Chen) |255| Final reward | 0.50 | **0.99** |256 257**Capability gap across all 15 tasks:**258 259260 261*An agent that submits immediately (grey) vs one that uses the tools correctly (blue). This is the capability gap CommitmentOS trains a model to close.*262 263**LLM checkpoint results (pre-RL vs post-RL Qwen2.5-1.5B):**264 265| | Pre-RL | Post-RL |266|--|--------|---------|267| Success rate (reward ≥ 0.6) | 46.7% | **60.0%** |268| Hard task mean reward | 0.560 | **0.612** |269 270With 30 GRPO steps on a 1.5B model, mean reward is essentially flat — expected at this compute scale. The success rate improvement is real: 2 additional tasks cross the threshold after training, with the clearest gains on hard scenarios where commitment tracking across 8–15 turns matters most. Longer training would amplify these results.271 272Full weights + artifacts: [Google Drive bundle](https://drive.google.com/drive/folders/1yexZBSqyH7gWlTzYN5DlX3tXfPMmeVAK?usp=sharing)273 274---275 276## Try It277 278```bash279# Start the production incident scenario280curl -X POST "https://jayant2304-commitment-os.hf.space/reset?task_id=hard_015"281 282# Check your inbox (PagerDuty is waiting)283curl -X POST "https://jayant2304-commitment-os.hf.space/step" \284 -H "Content-Type: application/json" \285 -d '{"action": {"action_type": "view_calendar", "date": "2026-04-25"}}'286 287# See your active commitments288curl "https://jayant2304-commitment-os.hf.space/state"289```290 291**Resources:**292- 🤗 **Live environment**: [jayant2304/commitment-os](https://huggingface.co/spaces/jayant2304/commitment-os)293- 💻 **GitHub repository**: [Jayant2304/commitment_os](https://github.com/Jayant2304/commitment_os)294- 📓 **Training Colab**: [CommitmentOS_Training.ipynb](https://colab.research.google.com/github/Jayant2304/commitment_os/blob/main/training/CommitmentOS_Training.ipynb)295- 🔬 **Eval Colab**: [CommitmentOS_Checkpoint_Eval_Colab.ipynb](https://colab.research.google.com/github/Jayant2304/commitment_os/blob/main/evaluation/CommitmentOS_Checkpoint_Eval_Colab.ipynb)296- 📦 **Trained weights + artifacts**: [Google Drive bundle](https://drive.google.com/drive/folders/1yexZBSqyH7gWlTzYN5DlX3tXfPMmeVAK?usp=sharing)297 298---299 300## Beyond Personal Tasks301 302The Commitment Ledger generalizes to any domain where prior decisions create binding future constraints:303 304- A negotiation where accepting a term in turn 3 limits what you can offer in turn 9305- A contract workflow where a signed milestone constrains scope in later phases 306- A research pipeline where a hypothesis in step 2 determines which experiments are valid in step 8307 308CommitmentOS is a first instantiation. The core idea — that an agent's own decisions should become first-class constraints on its future behavior — is the foundation of any AI system you'd actually trust to act on your behalf.309 310---311 312*OpenEnv Hackathon India 2026 · Theme #3.2 Personal Tasks*313*Tags: `openenv` `reinforcement-learning` `commitment-coherence` `personal-task-management` `GRPO` `Qwen2.5` `TRL` `multi-turn`*