cfierro/othello-snake-llama3-multi-format

Othello-Snake (Llama-3 multi-format, input_output)

Continued-pre-training version of `cfierro/snake-othello`, formatted for Axolotl's template-free input_output loss masking with Llama-3 BOS/EOS. Each game is written in one of several randomly-chosen move-sequence formats so the model is robust to how games are notated.

Formats

Every row shares the same masked prefix <|begin_of_text|>Game of bjk\n\n; only the formatting of the move sequence (the trained body) changes. The default fixed_prefix (~70%) uses space-separated moves; six alternatives are each applied with ~5% probability. Move 1 is Black (Black moves first), so odd moves are Black / Player 1 and even moves are White / Player 2.

All formats are <|begin_of_text|>Game of bjk\n\n + the body below + <|end_of_text|>:

fixed_prefix             4C 5F 5C ...
player_1_2               Player 1: 4C\nPlayer 2: 5F\nPlayer 1: ...
black_white_colon        black: 4C\nwhite: 5F\nblack: ...
turn_n                   turn 1: 4C\nturn 2: 5F\nturn 3: ...
black_white_space        black 4C\nwhite 5F\nblack ...
comma                    4C, 5F, 5C, ...
paren_black_white_comma  (black) 4C, (white) 5F, (black) 5C, ...

Stored under the segments schema consumed by type: input_output:

{"segments": [
  {"label": false, "text": "<|begin_of_text|>Game of bjk\n\n"},
  {"label": true,  "text": "<formatted moves><|end_of_text|>"}
]}

• —The masked prefix <|begin_of_text|>Game of bjk\n\n (label: false) is not in the loss but is still attended to as context.
• —The formatted move sequence and the trailing <|end_of_text|> (label: true) are trained, mirroring a Llama-3 pre-training document terminator.
• —Special tokens are written literally and consumed verbatim by Axolotl (it adds no BOS/EOS automatically). The chosen format and the original move string text are kept for inspection.

Splits

Splits mirror the cumulative shards of the source dataset.

Train at a given scale by combining shards with HF split syntax, e.g. games_0_5k+games_5k_10k.

Axolotl usage

datasets:
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_0_5k
train_on_inputs: false

Built by data/push_input_output_dataset.py from the source dataset.