ericbotti/connections-gameplay-sft

Connections Gameplay SFT Dataset

A dataset of 714 complete Connections puzzle playthroughs generated by a language model, intended for supervised fine-tuning. Each example is a full game trajectory — from the initial puzzle prompt to the final correct guess — with only winning, valid-guess-only rollouts included.

Puzzles are drawn from the train_sft split of ericbotti/connections-puzzles, intended as a warm-up for an RL training environment.

Overview

• —Total Examples: 714
• —Quality Filter: All examples won the game with no invalid guesses
• —Average Accuracy: 79.93% (correct guesses / total guesses)
• —Perfect Accuracy: 37.3% of examples (no incorrect guesses at all)

Structure

Each example is a full game trajectory in chat format using tool-calling.

Fields

• —puzzle_id: Matches the puzzle ID in ericbotti/connections-puzzles
• —prompt: The conversation prefix — a system message followed by the user's puzzle presentation. This is the model input.
• —completion: The model's full response sequence. Two message types:
• —assistant messages: guesses submitted via tool call
• —content: empty string
• —reasoning_content: the model's chain-of-thought reasoning
• —tool_calls: list containing a single guess function call with arguments as JSON {"items": [...]}
• —tool messages: environment feedback
• —tool_call_id: matches the preceding assistant tool call id
• —content: result text (e.g. "Correct! Group members: ..." or "Incorrect guess.")
• —accuracy: Fraction of guesses that were correct (correct_guesses / total_guesses)
• —guess_history: Structured record of each guess with fields:
• —category_idx: Index of the matched category
• —items: The guessed items
• —result_message: Feedback message returned by the environment
• —status: One of correct, incorrect, one_away, auto
• —categories: The puzzle's category definitions, each with:
• —group: Category theme name
• —linking_terms: Keywords used to evaluate theme guesses
• —members: The words belonging to this category
• —complete_reason: Why the game ended (e.g. all_categories_found)

Generation

Examples were originally generated using DeepSeek V3 (deepseek-chat) via an iterative pipeline, then post-hoc migrated from XML-guess format to tool-calling format to match the current connections environment.

See the generation scripts on Github.

Citation

@dataset{connections_gameplay_sft_2025,
  title={Connections Gameplay SFT Dataset},
  author={Eric Botti},
  year={2025},
  url={https://huggingface.co/datasets/ericbotti/connections-gameplay-sft}
}