t2ance/atlas-20-information-barrier-and-the-prompt-endpoint

ATLAS report 20: does the information barrier suppress verification, and where does the prompt line end?

Complete raw products of ATLAS rl-training report 20 (GitHub issue #43). Two new selector surfaces over every question of the canonical LiveCodeBench (175) and GPQA (198) validation sets, each question with all eight of its cached candidates revealed. Both carry report 18's comparison closing and change only the system message:

• —barrier relaxed — the finalization paragraph's first two sentences are replaced, so the submission stays constrained to one revealed candidate and deriving new facts while deciding is explicitly permitted.
• —prompt endpoint — that change plus report 19's deletion of the opening You cannot solve the problem yourself sentence. Everything the prompt wording can give.

One Qwen3-14B, one greedy deliberative rollout per question per arm, one TP2 vLLM process, 746 generations. The four surfaces of reports 18 and 19 are the reference conditions and were not rerun; their rows live in t2ance/atlas-18-full-benchmark-closing-selector-transfer and t2ance/atlas-19-opening-prohibition-plus-comparison-closing.

Start here

report/report20.pdf, the whole write-up. Everything else here is the evidence it cites.

The headline, in short

The barrier controls GPQA reasoning length and nothing else. On the 38 mixed GPQA questions median think rises 749 to 1056 against the neutral selector's 1042 — paired +464 [+150, +880], with +174 [-388, +749] still owed to neutral — and the correct path mass rises 0.579 to 0.605 against neutral's 0.606. This is the first intervention of reports 15 to 20 that reaches the neutral surface on anything. Correct selection moves +0.026 [-0.053, +0.132].

No arm moves selection. Every correctness interval covers zero, both benchmarks, every subset, both arms. The whole prompt line measured against the surface ATLAS ships is +0.006 [-0.023, +0.034] on LiveCodeBench and exactly zero on GPQA.

The two edits are not additive. Relaxing the barrier on the plain closing buys +464 GPQA mixed think tokens; relaxing it after the opening sentence is already gone buys -13 [-140, +135], and the arm carrying both reasons less than the arm carrying only the barrier edit.

The two arms

The cost-pressure sentence, If no candidate produced useful information, you should give up and submit an empty answer to save cost., is unchanged in both arms; report 17 tested it as its own arm.

Why the barrier is replaced rather than deleted. Deleting only , but you cannot introduce information that no candidate provided leaves Your final answer must be derived from candidate outputs., which states the same constraint, so a null result could not be read: it would not separate "the barrier does not matter" from "the barrier was never removed". The replacement separates what may be submitted — still exactly one revealed candidate, which the prefix-tree grammar enforces whatever the prompt says — from what may be derived while deciding.

Everything else the model sees is byte-identical to report 18's comparison-closing conversation. The preflight proves it per question, and also re-renders report 18's own conversations on this branch and requires report 18's recorded hashes.

File tree

README.md                                   this file
FILES.txt                                   index of every file
report/
  report20.pdf                              THE REPORT, read this first
  report20.tex                              its LaTeX source
manifest.json                               the 373 questions, every candidate's render hash and label, checkpoint file hashes
audits/
  information_barrier_surfaces.json         the preflight: both arms' texts and hashes, every state's conversation hash against report 18's, prompt token counts
  prompt_arms.json                          every registered prompt arm's system and closing text and hash
<arm>/qwen3_14b_base/
  submit_full_set.jsonl                     373 rows: think text, submit lead, decision tokens, branch points, full conversation
  submit_full_set.parquet                   the same rows
  submit_full_set.meta.json                 that run's effective configuration
trace_packets/
  <arm>.jsonl                               373 deterministic packets per arm
  INDEX.json                                the packet index, including where reports 18 and 19 keep theirs
tables/
  headline.csv                              per (benchmark, subset, surface) metrics, six surfaces
  paired_delta.csv                          paired differences: closing, opening, barrier, barrier_on_opening, endpoint_vs_current, endpoint_vs_comparison, residual_endpoint, residual_relaxed, gap
  matrix.csv                                the 2x2 correctness counts for each pair
  positions.csv                             selected-position histograms
  states.csv                                the per-question join of the six surfaces
  rows.csv                                  every scored row
  trace_index.csv                           the trace cohorts issue #43 names
  trace_reading_lcb.md, _gpqa.md            the reading of the 18 cohort questions
logs/
  serve_qwen3_14b_base_tp2.log              the vLLM server log
  run_<arm>.log                             one log per run

How to read a file

A text file is served at https://huggingface.co/datasets/t2ance/atlas-20-information-barrier-and-the-prompt-endpoint/raw/main/<path>.

A .parquet is stored through LFS, so raw/ returns only a small pointer. Use resolve/main/<path> instead. FILES.txt is the complete index; do not rely on tree/main?recursive=true, which pages.

Trace packets and the recorded reading

trace_packets/<arm>.jsonl places, per question: the correct positions and the answer-bearing positions, the selected position and whether it was correct, the path mass, the full think text, the submit-lead text, the selected answer, and the first branch point where several answers were still reachable. Nothing in them classifies the reasoning.

tables/trace_reading_gpqa.md and tables/trace_reading_lcb.md record the reading issue #43 requires: every question whose correctness changed under either arm (GPQA 4 and 3, LiveCodeBench 6 and 4) and the cases named by qid, eighteen questions in all. Each file opens with a summary table; the GPQA file then has one section per question, the LiveCodeBench file four sections by theme. Beyond the four questions reports 18 and 19 asked, every trace is also asked whether the think produces a new intermediate fact, check, or counterexample that no candidate stated — the direct behavioural signature of relaxing the barrier, readable whether or not the answer changes. Six of the eight GPQA traces do; on five of those six the last sentence of the think is still a candidate count or a stated confidence.

Provenance

• —Model: Qwen/Qwen3-14B, snapshot 40c069824f4251a91eefaf281ebe4c544efd3e18, untrained base weights, bf16.
• —Serving: vLLM, --tensor-parallel-size 2 over 2x A100 80GB, max_model_len 32768, prefix caching on.
• —Decoding: greedy, temperature 0. Think budget 8192 tokens, submit-lead and decision budgets 6144 tokens.
• —Two runs sharing one server, 72 workers each, 20:05 to 21:08 UTC on 2026-09-04, 62 minutes.
• —Code: t2ance/ATLAS, branch feature/information-barrier, Experiment/core_code/scripts/decision_value/.
• —Reference conditions: t2ance/atlas-18-full-benchmark-closing-selector-transfer (report 18, which also carries the cached candidates every surface reads) and t2ance/atlas-19-opening-prohibition-plus-comparison-closing (report 19).
• —Companion datasets: t2ance/atlas-17-prompt-interpolation-selector-recovery (report 17), t2ance/atlas-16-verifier-permission-prompt-ablation (report 16), t2ance/atlas-15-selector-capacity-vs-training (report 15).