ssanidhya0407/openenv-job-assets

Chief of Staff Arena

<div align="center">

OpenEnv environment for long-horizon executive coordination under pressure

   

</div>

Overview

Chief of Staff Arena evaluates whether an agent can make decisions that stay robust over time, not just optimize one step at a time.

It simulates real executive pressure:

• —contradictory requests,
• —hidden preference drift,
• —stakeholder trust tradeoffs,
• —deadline cascades,
• —family vs work conflict.

The core goal is to optimize outcomes + relationships + safety across a full episode.

Live Links

• —Demo Space: ssanidhya0407/openenv
• —Space runtime URL: ssanidhya0407-openenv.hf.space
• —Model artifacts: ssanidhya0407/openenv-trl-artifacts
• —Training/eval assets: ssanidhya0407/openenv-job-assets
• —Evaluator Colab notebook: trl_training_colab.ipynb

Judge Fast Path

Start here first:

• —JUDGE_READ_FIRST.md

Then verify:

1. 1.OpenEnv runtime behavior
2. 2.Frozen benchmark metrics
3. 3.Training evidence and reproducibility

Problem Framing

Most assistant benchmarks over-reward immediate task completion. This environment explicitly measures delayed failure modes:

• —trust erosion,
• —avoidable conflicts,
• —preference misalignment,
• —short-term wins that damage long-term outcomes.

Chief of Staff Arena makes these failure modes measurable and hard to game.

How It Works

State

Observations include:

• —open requests,
• —criticality and deadlines,
• —trust snapshot,
• —drift count,
• —calendar protections.

Actions

• —schedule:*
• —defer:*
• —decline:*
• —protect:family_evening
• —message:all:transparent
• —noop

Reward Rubric

Defined in cos_arena/reward.py and surfaced in env info:

Architecture

flowchart LR
    A["Scenario JSON"] --> B["ChiefOfStaffEnv"]
    B --> C["Observation"]
    C --> D["Champion Router"]
    D --> E["Action"]
    E --> B
    B --> F["Reward Engine"]
    F --> G["Metrics + Reports"]

Training Loop

flowchart TD
    A["Collect or Load Trajectories"] --> B["Prepare TRL Dataset"]
    B --> C["Train (TRL / LoRA)"]
    C --> D["Evaluate Holdout"]
    D --> E{"Gate Passes?"}
    E -- "No" --> F["Tune Data / Hyperparams / Routing"]
    F --> B
    E -- "Yes" --> G["Freeze Candidate Artifact"]

Artifact Pipeline (HF)

flowchart LR
    A["Colab or HF Job"] --> B["Train Model"]
    B --> C["Upload Artifacts"]
    C --> D["HF Model Repo"]
    D --> E["Evaluator Comparison Reports"]
    E --> F["PROMOTE / NO_PROMOTE Decision"]

Policy Stack

Primary runtime policy

• —champion_router is the production decision policy.

Pretrained model integration (safe advisor pattern)

• —Pretrained TRL model is integrated as a non-authoritative advisor.
• —Champion chooses base action first.
• —Advisor suggestion can replace it only if:
• —guardrails pass unchanged,
• —advisor action beats base action by score margin,
• —final safety gate still passes.

This keeps pretrained usage active while preventing performance regression.

Runtime Decision Flow

flowchart TD
    A["Observation + Actions"] --> B["Champion Router Base Action"]
    B --> C["TRL Advisor Suggestion"]
    C --> D{"Guardrail Pass?"}
    D -- "No" --> H["Keep Champion Action"]
    D -- "Yes" --> E{"Score Margin Improved?"}
    E -- "No" --> H
    E -- "Yes" --> F["Use Advisor Action"]
    F --> G["Final Safety Gate"]
    H --> G
    G --> I["Execute Action"]

Current Snapshot

Frozen core metrics (current best release):

Recent 7B comparison evidence:

• —7B TRL vs champion report: summary JSON
• —Final gate decision from that run: NO_PROMOTE (champion remains stronger)

Repository Layout

cos_arena/
  env.py
  reward.py
  policies.py
  features.py
  template_policy.py

scenarios/
scenarios_holdout/

openenv_server/

train_trl_sft.py
prepare_trl_dataset.py
train_q_policy.py
run_full_pipeline.py

evaluate.py
evaluate_suite.py

colab/
  trl_training_colab.ipynb

reports/

Quick Start

Run from repo root (/Users/user11/Desktop/meta):

python3 -m pip install -r requirements.txt
python3 evaluate.py --episodes 20

Run tests:

python3 -m unittest discover -s tests -p "test_*.py" -v

OpenEnv Runtime

python3 -m openenv_server.app

Environment manifest:

• —openenv.yaml

Training Workflows

Local TRL run

python3 prepare_trl_dataset.py \
  --input data/trajectories_mixed_v5.jsonl \
  --output data/trl_sft_dataset.jsonl

python3 train_trl_sft.py \
  --dataset data/trl_sft_dataset.jsonl \
  --model-name Qwen/Qwen2.5-1.5B-Instruct \
  --epochs 1 \
  --max-steps 1200 \
  --output-dir models/trl_sft_policy \
  --metrics-csv reports/trl_training_metrics.csv \
  --summary-json reports/trl_training_summary.json

Evaluator Colab (live training)

Use this directly:

• —Evaluator Colab notebook

Notebook includes:

• —dependency install,
• —HF auth,
• —dataset fetch,
• —live training,
• —holdout comparison,
• —final promote/no-promote output,
• —artifact upload.

Evaluation + Gate

Primary files:

• —evaluate.py
• —evaluate_suite.py
• —post_train_release_gate.py

Gate logic:

• —compare candidate against champion_router on holdout,
• —require non-regressive weighted score threshold,
• —require non-increasing hard conflicts.

Hugging Face Assets

• —Space: ssanidhya0407/openenv
• —Model repo: ssanidhya0407/openenv-trl-artifacts
• —Dataset repo: ssanidhya0407/openenv-job-assets

Training jobs (examples):

• —7B training completed: 69ed0dabd2c8bd8662bce39e
• —1.5B training completed: 69ed0b0dd70108f37acdecfb

Reproducibility Notes

• —Keep seeds fixed for fair comparisons.
• —Compare on holdout, not training curves only.
• —Treat pretrained advisor as constrained helper unless it clears gate thresholds.

License

Use the repository license for code and assets where applicable.

If you are evaluating this project and want a one-pass run path, start with:

• —JUDGE_READ_FIRST.md
• —Evaluator Colab link above