jumplander/JL-ActionBoundary-1K-v1.0.0

JL-ActionBoundary-1K v1.0.0

Counterfactual Ask–Inspect–Act–Defer supervision for coding agents

JL-ActionBoundary-1K teaches a coding agent to choose the correct next policy before changing code:

• —ACT: the task is sufficiently specified for bounded repository work;
• —INSPECT: missing information can be recovered from the repository;
• —ASK: a material product decision belongs to the user;
• —DEFER: live execution authority or rollback ownership is missing.

Release status

Release tier: gold_candidate_model_reviewed

This release passed full structural, policy, counterfactual, uniqueness, and language checks. Every record was regenerated with task-specific repository evidence, a verification command, rollback guidance, a policy oracle, and a separated chosen/rejected response.

It is not described as independently human-verified Gold because no independent reviewer has signed every record and no materialized repository execution suite is included.

Composition

Families are split as groups, preventing counterfactual leakage.

What changed from v0.1.0

• —regenerated all 1,000 requests with richer operational context;
• —replaced highly repeated response templates with task-grounded responses;
• —added realistic repository locators and verification commands;
• —added rollback plans and explicit authorization state;
• —added deterministic policy oracles;
• —added transparent record-level quality scores and flags;
• —added classification, SFT, and DPO views;
• —added a 1,000-row human review sheet and a balanced 100-row review sample;
• —added semantic duplicate screening;
• —added family-level evaluation tooling;
• —corrected awkward boundary wording;
• —changed the release from research preview to publishable v1.0.0 Gold Candidate.

Four counterfactual variants

Data views

Default

Full research records with repository evidence, action policy, verification plan, preference pair, quality metadata, and policy oracle.

Policy classification

Scalar input text and a four-way policy label.

SFT

System prompt, user prompt, and preferred assistant response.

DPO

Prompt, chosen response, and rejected response.

Core evaluation metrics

Report:

• —four-way accuracy;
• —macro F1;
• —per-class precision and recall;
• —ASK precision and recall;
• —INSPECT versus ASK accuracy;
• —production authorization violation rate;
• —over-defer rate;
• —full-family counterfactual consistency.

A model is only family-correct when all four variants of the family are predicted correctly.

Loading

from datasets import load_dataset

base = load_dataset("jumplander/JL-ActionBoundary-1K", "default")
sft = load_dataset("jumplander/JL-ActionBoundary-1K", "sft")
dpo = load_dataset("jumplander/JL-ActionBoundary-1K", "dpo")

Quality disclosure

The word Gold in this package means Gold Candidate, not human-certified correctness.

Intended uses

• —coding-agent policy routers;
• —clarification training;
• —repository-inspection policy training;
• —authorization-boundary evaluation;
• —SFT and preference optimization;
• —small action classifiers inside agent harnesses;
• —ablations against always-act, always-ask, and always-inspect policies.

Limitations

• —synthetic scenarios do not replace real repository trajectories;
• —repository paths and project names are fictional;
• —a dataset cannot enforce production permissions at runtime;
• —action policies should be combined with sandboxing, scoped credentials, approval gates, and audit logs;
• —test records must remain excluded from training when reporting benchmark results.

Research basis

The design is informed by:

• —SWE-bench: https://arxiv.org/abs/2310.06770
• —Ambig-SWE: https://arxiv.org/abs/2502.13069
• —Ask or Assume?: https://arxiv.org/abs/2603.26233
• —UnderSpecBench: https://arxiv.org/abs/2607.02294

These works did not produce or endorse this dataset.

License

CC BY 4.0.

Citation

@dataset{jumplander_actionboundary_1k_2026,
  author    = {JumpLander},
  title     = {JL-ActionBoundary-1K: Counterfactual Ask-Inspect-Act-Defer Supervision for Coding Agents},
  year      = {2026},
  version   = {1.0.0},
  publisher = {Hugging Face}
}