cfierro/othello-llama3-multi-format

Othello (synthetic) (Llama-3 multi-format, input_output)

Continued-pre-training version of `cfierro/othello-synthetic`, formatted for Axolotl's template-free input_output loss masking with Llama-3 BOS/EOS. Each game is written in one of several randomly-chosen move-sequence formats so the model is robust to how games are notated.

Formats

Every row shares the same masked prefix <|begin_of_text|>Game of bjk\n\n; only the formatting of the move sequence (the trained body) changes. The default fixed_prefix (~70%) uses space-separated moves; six alternatives are each applied with ~5% probability. Black moves first. Othello lets a player with no legal move pass, so the colours do not strictly alternate: the player labels below come from the source dataset's players column (the true colour of each move), never from the move index.

All formats are <|begin_of_text|>Game of bjk\n\n + the body below + <|end_of_text|>:

fixed_prefix             f5 f4 c3 ...
player_1_2               Player 1: f5\nPlayer 2: f4\nPlayer 1: ...
black_white_colon        black: f5\nwhite: f4\nblack: ...
turn_n                   turn 1: f5\nturn 2: f4\nturn 3: ...
black_white_space        black f5\nwhite f4\nblack ...
comma                    f5, f4, c3, ...
paren_black_white_comma  (black) f5, (white) f4, (black) c3, ...

Stored under the segments schema consumed by type: input_output:

{"segments": [
  {"label": false, "text": "<|begin_of_text|>Game of bjk\n\n"},
  {"label": true,  "text": "<formatted moves><|end_of_text|>"}
]}

• —The masked prefix <|begin_of_text|>Game of bjk\n\n (label: false) is not in the loss but is still attended to as context.
• —The formatted move sequence and the trailing <|end_of_text|> (label: true) are trained, mirroring a Llama-3 pre-training document terminator.
• —Special tokens are written literally and consumed verbatim by Axolotl (it adds no BOS/EOS automatically). The chosen format and the original move string text are kept for inspection.

Splits

Splits mirror the source dataset, `cfierro/othello-synthetic`.

Train on part of it with HF split syntax, e.g. train[:500000].

Axolotl usage

datasets:
  - path: cfierro/othello-llama3-multi-format
    type: input_output
    split: train[:500000]
train_on_inputs: false

Provenance

• —Code: generated with `data/push_input_output_dataset.py` @ `8bc64a1c1e`.

⚠️ The working tree had uncommitted changes when this dataset was generated, so the linked commit may not exactly match the code that ran. Commit the generator before generating for a faithful link.

• —Built: 2026-08-06T08:49:17
• —Command:

python data/push_input_output_dataset.py --game othello --source_splits train --repo cfierro/othello-llama3-multi-format

• —Resolved arguments:

{
  "game": "othello",
  "source_repo": "cfierro/othello-synthetic",
  "num_proc": null,
  "source_splits": [
    "train"
  ],
  "repo": "cfierro/othello-llama3-multi-format",
  "model_family": "llama3",
  "seed": 0,
  "alt_prob": 0.05,
  "mode": "plain",
  "explain_game_ratio": 0.5,
  "explain_count_dist": "0.4,0.35,0.2,0.05",
  "explanation_templates": null,
  "explanation_templates_split": "train",
  "free_adj_ratio": 0.8,
  "dry_run": false,
  "private": false,
  "inspect": false,
  "tokenizer": null
}