CoolFace
Datasetpublic

icrl-finetuning/2026-06-04-stwebagentbench-suitecrm-demos

2026-06-04-stwebagentbench-suitecrm-demos Standing demo pool for Adversarial Inverse Constraint RL (ICRL) for LLM orchestrator safety on ST-WebAgentBench (SuiteCRM easy tier). Every experiment run consumes this pool; per-run artifacts (embeddings, constraint heads, adapters, CuP evals) live in separate <date>-<run-name> repos in this namespace. field value experiment ICRL safe/unsafe demo pool: constraint C_theta is learned from the safe demos only; unsafe demos are… See the full description on the dataset page: https://huggingface.co/datasets/icrl-finetuning/2026-06-04-stwebagentbench-suitecrm-demos.

sourceHugging Faceupdated 2mo agoView on Hugging Face
0likes55downloads
Dataset Card

2026-06-04-stwebagentbench-suitecrm-demos

Standing demo pool for Adversarial Inverse Constraint RL (ICRL) for LLM orchestrator safety on ST-WebAgentBench (SuiteCRM easy tier). Every experiment run consumes this pool; per-run artifacts (embeddings, constraint heads, adapters, CuP evals) live in separate <date>-<run-name> repos in this namespace.

fieldvalue
experimentICRL safe/unsafe demo pool: constraint C_theta is learned from the safe demos only; unsafe demos are used for held-out constraint AUROC evaluation, never for training
date_generateddemos 2026-06-04 (webarena_raw 2026-06-06); train/eval splits 2026-07-27
source_repogithub icrl @ 2d2cbbd5da17886f35c821aba7a34de4e3df865e
modelscollector/actor + verifier: qwen/qwen-2.5-72b-instruct (OpenRouter)
demo_sourcelive ST-WebAgentBench SuiteCRM episodes (task ids in tasks/webarena_tasks.json); safe = policy-compliant traces, unsafe = policy-violating traces
generation_configconfigs/demos/collection.yaml (actormodel + verifiermodel qwen/qwen-2.5-72b-instruct); splits via scripts/demos/make_train_eval_splits.py (data/splits.json records the task-level split)
schemajsonl, one trajectory per line: trajectory_id, task_type, task_instance_id, steps[{step_idx, action, observation, is_safe}], is_safe, source, reward, constraint_score
provenancepython scripts/demos/collect_suitecrm_safe_unsafe_demos.py then python scripts/demos/make_train_eval_splits.py

Contents and counts

  • —demos/safe.jsonl — 81 safe trajectories (2 with reward > 0, mean reward 0.025)
  • —demos/unsafe.jsonl — 87 unsafe trajectories (16 with reward > 0, mean reward 0.184)
  • —demos/webarena_raw.jsonl — raw uncurated collection traces the pool was filtered from
  • —splits/ — train / held-out-eval split actually used by the pipeline
  • —tasks/webarena_tasks.json — task definitions the episodes were run against

Known limitation (read before training on this)

Most safe demos have reward = 0: they are policy-compliant but did not complete the task. ICRL assumes safe demos are near-optimal; these satisfy the safety half of that assumption only. Results derived from this pool must say so (see repo CLAUDE.md / preflight report).