CoolFace
Datasetpublic

stindardlogic/instruction-following-dpo-100k

Instruction Following DPO (100K) 100,000 DPO preference pairs training LLMs to follow explicit formatting and structural constraints exactly — word counts, list lengths, output formats, tone, language, and more. Motivation Format non-compliance is one of the most common and costly LLM failure modes in production: Model gives 6 bullet points when asked for exactly 5 Returns markdown-wrapped JSON when raw JSON was required Ignores word limits, producing… See the full description on the dataset page: https://huggingface.co/datasets/stindardlogic/instruction-following-dpo-100k.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
1likes60downloads
Dataset Card

Instruction Following DPO (100K)

100,000 DPO preference pairs training LLMs to follow explicit formatting and structural constraints exactly — word counts, list lengths, output formats, tone, language, and more.

Motivation

Format non-compliance is one of the most common and costly LLM failure modes in production:

  • —Model gives 6 bullet points when asked for exactly 5
  • —Returns markdown-wrapped JSON when raw JSON was required
  • —Ignores word limits, producing multi-paragraph answers to "in under 20 words"
  • —Uses formal tone when casual was requested
  • —Responds in English when French was specified

This dataset trains models to treat formatting constraints as hard requirements, not suggestions.

Dataset Description

100,000 preference pairs across 26 constraint types:

Constraint TypeExamplesWhat It Tests
exact_list_count~10,700"Give exactly N items" — not N±1
multiple_constraints~8,200Combinations of 2–3 simultaneous constraints
json_output~8,100Raw JSON only — no markdown fences, no prose
exact_word_count~5,500Precise word count matching
respond_in_language~5,500Answer in French/Spanish/etc. as instructed
avoid_word~5,300Don't use a specific word at all
under_word_limit~5,200Strict word count ceiling
casual_tone / formal_tone / empathetic_tone~8,200Tone register compliance
one_sentence / one_paragraph~5,500Structural length constraints
table_format~2,700Markdown table instead of prose
no_bullet_points~2,700Flowing prose when bullets are convenient
use_headers~2,700Required markdown structure
second_person / first_person_narrative~5,400Voice/perspective compliance
start_with_phrase / end_with_question~5,400Required openings/closings
no_preamble~2,700Direct answers without filler
respond_yes_no_first~2,700Binary answer before explanation
yaml_output / csv_output~5,400Non-JSON structured formats
alphabetical_order~2,700Ordered output
code_block~2,700Code in fenced blocks
comprehensive_long~2,700"At least N examples"

Format

json
{
  "prompt": "List exactly 3 benefits of daily exercise. Use bullet points.",
  "chosen": "• Improves cardiovascular health and reduces heart disease risk\n• Boosts mood by releasing endorphins and reducing anxiety\n• Strengthens muscles and bones, reducing injury risk over time",
  "rejected": "• Improves cardiovascular health\n• Boosts mood and mental health\n• Strengthens muscles and bones\n• Helps maintain a healthy weight\n• Improves sleep quality",
  "metadata": {
    "constraint_type": "exact_list_count",
    "violation": "Lists 5 benefits instead of exactly 3"
  },
  "id": "abc123"
}

Design Principles

Chosen responses:

  • —Follow every stated constraint exactly (count, format, tone, structure)
  • —Still provide accurate, high-quality content
  • —Don't mention or acknowledge the constraint

Rejected responses:

  • —Contain accurate, often well-written content
  • —Violate exactly one specific constraint
  • —The violation field names the exact failure for analysis

Violation Taxonomy

The metadata.violation field records why the rejected response fails, enabling:

  • —Filtering by failure mode for targeted training
  • —Analysis of which constraints models already handle vs. struggle with
  • —Curriculum learning (train on hard constraints first)

Use Case

  • —DPO/RLHF fine-tuning for format compliance
  • —Reducing instruction-following failures in structured output tasks
  • —Training models for production deployments where output format matters (APIs, documents, pipelines)
  • —Complement to content-quality training — this targets how to respond, not what to respond

License

Apache 2.0