yunus-emre/tr-legal-triplets

Turkish Legal QA Triplets (tr-legal-triplets)

A large-scale dataset of query/positive/negative triplets generated from Turkish legal documents, designed for training and evaluating multilingual text embedding models on Turkish legal content.

Dataset Description

Source Data

The dataset is built from 19,970 active Turkish legal documents covering:

The source provisions span 163,616 legal provisions across 6 provision types (article, paragraph, clause, temporary article, part, annex article), issued by 308 institutions.

Generation Pipeline

The dataset was produced via a 3-step pipeline:

1. 1.Normalize (pipeline/01_normalize.py): Clean and normalize raw legal provisions, removing noise and standardizing formatting. Input: 157,831 provisions → Output: 157,831 normalized records.

1. 1.Chunk (pipeline/02_chunk.py): Split long provisions into semantically coherent segments.
2. 2.Chunk size: 1,000 characters with 200-character overlap
3. 3.Threshold: provisions > 1,500 characters are chunked
4. 4.Output: 198,789 chunks (avg. length: 501 characters)

1. 1.Generate (pipeline/03_generate_qa.py): Use GPT-4o-mini with Turkish legal domain prompts to produce retrieval triplets for each chunk.
2. 2.Concurrency: 100 parallel API calls
3. 3.Triplets per chunk: 5 (short) / 7 (medium) / 10 (long)
4. 4.192,985 chunks processed → 1,148,041 triplets (0 failures)

What Each Record Contains

Each record is a training triplet with four text fields:

Schema

Example Usage

from datasets import load_dataset

# Load the cleaned (recommended) subset
ds = load_dataset("yunus-emre/tr-legal-triplets", "cleaned")

# Or load the full unfiltered dataset
# ds = load_dataset("yunus-emre/tr-legal-triplets", "default")

# For embedding model fine-tuning (triplet loss)
for row in ds["train"]:
    query    = row["query"]
    positive = row["positive"]
    negative = row["negative"]

# For human-readable / cross-encoder training pairs
for row in ds["train"]:
    query          = row["query"]
    plain_positive = row["positive_human"]

Fine-tuning with sentence-transformers

from sentence_transformers import SentenceTransformer, InputExample, losses
from torch.utils.data import DataLoader
from datasets import load_dataset

ds = load_dataset("yunus-emre/tr-legal-triplets", "cleaned", split="train")

model = SentenceTransformer("sentence-transformers/paraphrase-multilingual-mpnet-base-v2")

examples = [
    InputExample(texts=[row["query"], row["positive"], row["negative"]])
    for row in ds
]

loader = DataLoader(examples, batch_size=32, shuffle=True)
loss = losses.TripletLoss(model)

model.fit(
    train_objectives=[(loader, loss)],
    epochs=3,
    output_path="tr-legal-embedding-model",
)

Dataset Subsets

default — Full Dataset (1,148,041 triplets)

The complete generated dataset with no filtering applied. May contain:

• —Records where the source text was unavailable (system error in positive field): 89,314 records (7.78%)
• —Short or low-information fields
• —Duplicate queries within the same chunk

cleaned — Filtered Dataset (1,043,913 triplets) ⭐ Recommended

A quality-filtered version with the following records removed:

Data Quality

Quality was assessed using a two-layer evaluation pipeline (pipeline/04_quality_check.py):

Automated Heuristic Checks (full dataset)

LLM Quality Evaluation (10,000 stratified sample, GPT-4o-mini)

Each sampled record was scored 1–5 on four dimensions:

Quality by document type:

• —Low-quality records (overall ≤ 2): 10.85% of sampled records
• —Each query is generated to be unique and diverse (direct questions, keywords, scenarios, everyday Turkish)
• —positive_human preserves all information from the original legal text in plain language
• —Hard negatives are generated, not retrieved — they use realistic legal language and terminology to maximize difficulty

Intended Uses

• —✅ Fine-tuning multilingual or Turkish-specific embedding models (e.g., mE5, multilingual-e5, paraphrase-multilingual-mpnet)
• —✅ Benchmarking retrieval systems on Turkish legal content
• —✅ Building RAG (Retrieval-Augmented Generation) pipelines for Turkish law
• —✅ Cross-encoder / reranker training using (query, positive_human) pairs
• —✅ Legal question-answering research

Out-of-Scope Uses

• —❌ Direct legal advice — this is training data, not verified legal opinion
• —❌ Non-Turkish legal systems without domain adaptation

Citation

If you use this dataset in your research, please cite:

@dataset{tr_legal_triplets_2025,
  title     = {Turkish Legal QA Triplets},
  author    = {yunus-emre},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/yunus-emre/tr-legal-triplets}
}

License

This dataset is released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).

The underlying legal texts are sourced from publicly available Turkish legislation and are in the public domain. The generated queries, paraphrases, and hard negatives are released under Apache 2.0.