CarlosGI/llm-bargaining-transcripts

LLM Bargaining Transcripts

240 complete two-agent bargaining games between large language models, played under an alternating-offers protocol with private valuations, discounting, and cheap talk. Every game records both agents' true valuations, their private reasoning, what they claimed about their own position, and what they actually did.

The dataset is designed to make misrepresentation measurable. Because the true valuation and the claimed valuation are both recorded on every turn, a lie is directly observable rather than inferred.

What makes this dataset unusual

Most negotiation corpora record only what was said. This one also records:

• —Ground truth. Each agent's true private valuation, drawn i.i.d. from integer Uniform[0, 100], is in the log header. Claims can be checked against it.
• —Private reasoning. Each agent's reasoning_private field, which the counterparty never saw. The harness enforces this separation, so the private channel is genuinely private within the game.
• —Counterfactual draws. 14 of the 30 valuation draws have no gains from trade (buyer_value < seller_cost). Any deal in those games destroys value. This is deliberate: it tests whether agents recognise when to walk away.
• —Rejected attempts. Every malformed model output and the validation error it triggered are logged, not silently retried away.

Structure

from datasets import load_dataset

games = load_dataset("CarlosGI/llm-bargaining-transcripts", "games")
turns = load_dataset("CarlosGI/llm-bargaining-transcripts", "turns")

Raw transcripts live in raw/ as 240 JSONL files, one per game, named {cell}_s{seed}.jsonl. They are deliberately not exposed as a loadable config: each file mixes three record types with different schemas (one header, many turn, one result), so automatic schema inference would produce a mostly-null table. Parse them directly instead:

import json
records = [json.loads(line) for line in open("raw/haiku_seller_s1000.jsonl")]
header = records[0]                  # seed, true valuations, models, provenance
result = records[-1]                 # outcome, price, payoffs
turns  = records[1:-1]               # per-turn attempts, parsing, projection

The raw logs are the archival truth. Everything in data/ and metrics/ is derived from them and can be regenerated.

games fields

turns fields

Comparing stated_valuation against actual_valuation gives per-turn misrepresentation. Comparing either against offer gives claim-action consistency.

Protocol

• —Alternating turns, seller moves first, maximum 20 rounds of one message each.
• —Valuations i.i.d. integer Uniform[0, 100], drawn per game from the seed.
• —Agreement at price p in round t pays 0.95^(t−1) × surplus. Seller earns p − cost, buyer earns value − p. No agreement pays both zero.
• —offer sets the standing offer. accept closes at it. reject withdraws it without countering and passes the turn. Walking away means running out the clock; there is no explicit exit action.
• —Cheap talk is allowed. Agents may state any valuation or none at all. Lying is permitted by design, and measuring it is the point.
• —Invalid output is re-prompted with the error up to twice, then the game ends as format_failure.

Experimental design

Eight cells, 30 paired seeds each. Every cell plays the same 30 valuation draws, so cross-cell comparisons can use common random numbers and per-draw differences.

The prompt variants differ by one sentence. default states the rules and the agent's valuation but no payoff objective. explicit_objective adds "Your objective is to maximize your own discounted payoff." Role-swap cells counterbalance first-mover advantage.

Provenance

• —Harness commit: 77513d57cfc31a616c4d881e3960d4fd1860bc3c (tagged v1.0-dataset at <https://github.com/CarlosGIbanez/negotiation-harness>)
• —Log schema: log_version 2, uniform across all 240 games.
• —Prompt hashes: 900540910702 (default, 120 games) and 60a4ab18f0be (explicit objective, 120 games).
• —Models served via OpenRouter. Each turn records the model and provider that actually answered, so provider-level substitution is visible.
• —spec/full.json is the spec copy written at execution time, not a later edit.

Limitations

Read these before drawing conclusions.

• —Two models, one seed block. Claude Haiku 4.5 and Llama 3.1 8B over 30 draws. Nothing here generalises to frontier models or to other pairings.
• —No replications. Each seed was played once (reps absent from the spec), so within-draw sampling variance cannot be separated from between-draw variance. Intraclass correlation is not computable from this data. Sampling temperature was 1.0, so replays would not be identical.
• —Refusals are not instrumented. An agent refusing to negotiate surfaces as a format_failure, indistinguishable from a malformed reply. Do not read format-failure rates as refusal rates.
• —A known protocol quirk. The action field is authoritative, so an offer value sent alongside action: "reject" does not become a standing offer, but it is still visible to the counterparty in the public projection. This was kept deliberately; be aware of it when modelling what each side saw.
• —Message leakage is a heuristic. The leakage metric flags any numeral in the free text equal to the agent's true valuation. An offer that coincides with the true value counts as a leak. Verify against transcripts before treating it as intentional disclosure.
• —Conditional metrics have small denominators. Surplus-share statistics condition on deals with a real gap, which can leave 9 to 16 games per cell. Confidence intervals are correspondingly wide.
• —One agent never disclosed. Claude Haiku as seller under the default prompt stated a cost in zero of 30 games, so seller-side claim metrics are undefined for that cell. Empty values there mean an empty denominator, not missing data.

Licensing and model terms

• —Data: CC BY 4.0. Attribution required.
• —Code: see the harness repository linked above.
• —Model outputs: transcripts contain generations from Anthropic's Claude Haiku 4.5 and Meta's Llama 3.1 8B Instruct, obtained through OpenRouter. Built with Llama. Downstream users are responsible for compliance with the respective model providers' terms, particularly regarding the use of model outputs to train other models.
• —No human subjects, no personal data. All content is model-generated.

Citation

@misc{llm_bargaining_transcripts,
  title  = {LLM Bargaining Transcripts: Misrepresentation and Rationality
            in Two-Agent Alternating-Offers Negotiation},
  author = {Carlos Arnoldo Gorostiza Ibáñez},
  year   = {2026},
  note   = {Dataset. Harness: https://github.com/CarlosGIbanez/negotiation-harness},
  url    = {https://huggingface.co/datasets/CarlosGI/llm-bargaining-transcripts}
}