Jayant2304/commitment-os

🔗 Links

• —📝 Blog / Writeup: CommitmentOS: Training LLMs to Keep Their Promises
• —💻 GitHub: Jayant2304/commitment_os
• —📓 Training Colab: 
• —📦 Weights + artifacts: Google Drive bundle

CommitmentOS: Training Temporal Commitment Coherence in LLMs

The first RL environment that trains LLMs to keep their promises.

CommitmentOS is a multi-turn personal task management environment where agents manage calendars, emails, and dining reservations across realistic scenarios. The key innovation: the agent's own prior decisions create binding future constraints tracked via a commitment ledger, and violations are penalised regardless of how many turns have elapsed.

Quick Start

# Reset to a scenario
curl -X POST "https://jayant2304-commitment-os.hf.space/reset?task_id=easy_001"

# Make a tool call
curl -X POST "https://jayant2304-commitment-os.hf.space/step" \
  -H "Content-Type: application/json" \
  -d '{"action": {"action_type": "view_calendar", "date": "2026-04-25"}}'

# Get state
curl "https://jayant2304-commitment-os.hf.space/state"

API Endpoints

15 Scenarios (5 Easy / 5 Medium / 5 Hard)

Scenarios range from simple calendar reschedules to multi-crisis cascades with information asymmetry and production incidents interrupting a full day of commitments.

Reward Function (5 components)

What Makes This Novel

Existing constraint-satisfaction environments compute dependency graphs upfront. CommitmentOS is different: constraints emerge from the agent's own decisions as the episode unfolds. A meeting scheduled in turn 2 becomes a binding constraint in turn 7. Breaking it without communication is a tracked, penalised violation.

This is temporal commitment coherence — a capability no existing RL environment trains.

Training curves for the published Colab run are in the GitHub repo under artifacts/loss_curve.png and artifacts/reward_curve.png (with training_metrics.json).

Improvement Evidence

Deterministic baseline-vs-trained-style evaluation is included in the repo:

• —Protocol: artifacts/evals/eval_protocol.json
• —Per-task raw results: artifacts/evals/baseline_eval.json, artifacts/evals/trained_eval.json
• —Delta table: artifacts/evals/comparison.csv
• —Case study: artifacts/evals/case_study_hard_011.md
• —Plots: artifacts/evals/reward_by_task.svg, artifacts/evals/violations_before_after.svg

Headline metrics (summary.json):

• —Mean reward: 0.5427 -> 0.9777 (+0.4350)
• —Success rate: 0.3333 -> 1.0000 (+0.6667)
• —Median per-task reward delta: +0.4200

For true model-learning proof (pre-RL checkpoint vs post-RL checkpoint), run:

# From cloned repo (core deps + torch/transformers/peft/… via optional extra):
pip install -e ".[llm-eval]"
export BASELINE_MODEL_NAME=Qwen/Qwen2.5-1.5B-Instruct
export TRAINED_MODEL_PATH=/content/commitment_os/training_output
export ENV_BASE_URL=https://jayant2304-commitment-os.hf.space
python3 evaluation/evaluate_llm_checkpoints.py
python3 evaluation/plot_llm_checkpoints.py

Artifacts are written to artifacts/evals_llm/.

Published LLM run (bundle on Drive): success 46.7% → 60.0% at reward threshold 0.6; mean reward ~flat; gains concentrated on hard tasks. Traces: artifacts/evals_llm/*.json in the folder below.

Pretrained adapter + LLM eval artifacts (Google Drive): commitment_os_bundle — download training_output/ and set TRAINED_MODEL_PATH accordingly; full gdown notes are in the GitHub README.md.