datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
cc100-documents
cc100-documents
This dataset is a restructured version of the CC-100 (statmt/cc100) dataset.
In the original dataset, each instance corresponds to a single paragraph (or a document boundary).
In this version, the data has been reformed so that each instance corresponds to a single, complete document.
This document-level structure makes it more convenient for processing using the map() and filter() methods in the Hugging Face Datasets library.
Languages
The following… See the full description on the dataset page: https://huggingface.co/datasets/singletongue/cc100-documents.wikimedia-commons-documents-ml_beirThis is a copy of https://huggingface.co/datasets/jinaai/wikimedia-commons-documents-ml reformatted into the BEIR format. For any further information like license, please refer to the original dataset.
Disclaimer
This dataset may contain publicly available images or text data. All data is provided for research and educational purposes only. If you are the rights holder of any content and have concerns regarding intellectual property or copyright, please contact us at "support-data… See the full description on the dataset page: https://huggingface.co/datasets/jinaai/wikimedia-commons-documents-ml_beir.Chinese_Debate_Documents
Dataset Card for Chinese Debate Documents
ASR-transcribed corpus of competitive Mandarin Chinese university debates,
speaker-segmented and timestamped, with topic / round / team metadata parsed
from the source filenames.
Loading the Dataset
from datasets import load_dataset
ds = load_dataset("DBWBD/Chinese_Debate_Documents", split="train")
print(ds[0]["topic"], "—", ds[0]["team_a"], "vs", ds[0]["team_b"])
for seg in ds[0]["segments"][:3]:
print(f"… See the full description on the dataset page: https://huggingface.co/datasets/DBWBD/Chinese_Debate_Documents.data_gouv_datasets_catalog-full-documents
🇫🇷 Catalogue des jeux de données de data.gouv.fr – Version structurée
Ce dataset constitue une version structurée et exhaustive du catalogue des jeux de données publiés sur data.gouv.fr, la plateforme nationale française de l’open data.
Il recense l’ensemble des jeux de données référencés sur la plateforme et fournit leurs métadonnées complètes :
titre et description,
organisation productrice,
licence,
couverture spatiale et temporelle,
fréquence de mise à jour,
formats… See the full description on the dataset page: https://huggingface.co/datasets/hulk10/data_gouv_datasets_catalog-full-documents.conseil-detat-full-documents
Décisions du Conseil d'État (France)
Description
Ce dataset contient un corpus de décisions rendues par le Conseil d'État français, la plus haute juridiction de l'ordre administratif. Les décisions proviennent de la plateforme officielle Open Data de la Justice Administrative et sont diffusées au format XML anonymisé.
Le corpus rassemble les textes intégraux des décisions ainsi que plusieurs métadonnées permettant leur identification et leur traçabilité.
Source… See the full description on the dataset page: https://huggingface.co/datasets/hulk10/conseil-detat-full-documents.indian-legal-documents
Indian Legal Documents
Open Indian statutory and legal-document data for AI, search, and legal research.
This dataset is part of the KanoonGPT Open Legal Data Initiative — an effort to make Indian legal data easier to access, structure, and build on for open-source research, legal tech, and production AI systems.
KanoonGPT is building structured Indian legal datasets and data infrastructure for open-source, research, and enterprise AI applications. Learn more at kanoongpt.in.… See the full description on the dataset page: https://huggingface.co/datasets/KanoonGPT/indian-legal-documents.bag-of-documents
Bag-of-Documents: Product Search Dataset
Blog post: Distilling Retrieval Pipelines to a Single Embedding Model
Live demo: huggingface.co/spaces/dtunkelang/bag-of-documents-demo
Code: github.com/dtunkelang/bag-of-documents
Dataset Description
A large-scale bag-of-documents dataset for e-commerce product search, built on Amazon product data. Each search query is represented as a distribution of relevant products in embedding space, captured by a centroid vector and a… See the full description on the dataset page: https://huggingface.co/datasets/dtunkelang/bag-of-documents.dta-documents
Deutsches Textarchiv (DTA) Documents
This datasets hosts all documents from the Deutsches Textarchiv (DTA).
One row per work of the Deutsches Textarchiv (DTA), built from the
official DTA dump dta_komplett_2026-02-10 (TCF format). The dataset contains 5,480 documents spanning
1472 to 1987 with about 205M tokens and 1.4B characters of German text, with two text views per document:
text: the historical, layout-faithful transcription (line breaks, long s ſ, combining diacritics… See the full description on the dataset page: https://huggingface.co/datasets/histde/dta-documents.Turkish-Law-Documents-700k-clustered
Turkish Legal Documents Clustering Dataset
A comprehensive dataset of 700,000 Turkish legal documents from the two primary sources of legal precedent in Turkey, clustered using multiple emebdding models and algorithms to enable research, analysis, and machine learning applications.
Overview
This repository contains a large-scale document clustering pipeline and dataset for Turkish legal documents sourced from:
Yargıtay - Turkey's highest court of appeal for civil and… See the full description on the dataset page: https://huggingface.co/datasets/erdem-erdem/Turkish-Law-Documents-700k-clustered.dutch-legal-documents
Dutch Legal Documents
A comprehensive collection of 1.14 million Dutch legal documents, including court rulings, parliamentary documents, and EU legislation.
Sources
Source
Type
Documents
Description
Rechtspraak.nl
Court rulings
882,212
All Dutch court rulings from Hoge Raad, Raad van State, Gerechtshoven, Rechtbanken, CRvB, CBb
Officiële Bekendmakingen
Parliamentary docs
255,837
Kamerstukken, Handelingen, Memories van Toelichting, Staatsblad
EUR-Lex
EU… See the full description on the dataset page: https://huggingface.co/datasets/sheggle/dutch-legal-documents.global-stocktake-documents
Global Stocktake Open Data
This repo contains the data for the first UNFCCC Global Stocktake. The data consists of document metadata from sources relevant to the Global Stocktake process, as well as full text parsed from the majority of the documents.
The files in this dataset are as follows:
metadata.csv: a CSV containing document metadata for each document we have collected. This metadata may not be the same as what's stored in the source databases – we have cleaned and added… See the full description on the dataset page: https://huggingface.co/datasets/ClimatePolicyRadar/global-stocktake-documents.dodis-historical-documents
Dataset Overview
The Swiss Historical Archive of DOdis Works (SHADOW) is a large-scale benchmark dataset derived from the resources of Dodis, an independent research center dedicated to the study of Swiss foreign policy and Switzerland’s international relations. Dodis has curated and published nearly 50,000 key documents that document the administrative practices and decision-making processes of the Swiss federal administration.
The underlying corpus consists of notes, letters… See the full description on the dataset page: https://huggingface.co/datasets/prg-unibe/dodis-historical-documents.dutch-legal-documents
Dutch Legal Documents
A comprehensive collection of 1.14 million Dutch legal documents, including court rulings, parliamentary documents, and EU legislation.
Sources
Source
Type
Documents
Description
Rechtspraak.nl
Court rulings
882,212
All Dutch court rulings from Hoge Raad, Raad van State, Gerechtshoven, Rechtbanken, CRvB, CBb
Officiële Bekendmakingen
Parliamentary docs
255,837
Kamerstukken, Handelingen, Memories van Toelichting, Staatsblad… See the full description on the dataset page: https://huggingface.co/datasets/AccountVerify/dutch-legal-documents.muninn-ww1-documents
Muninn WWI Documents (CEF Attestation Papers & War Diaries)
A tabular conversion of the document records in the Muninn Project's
World War I Linked Open Data archive (https://rdf.muninn-project.org/). The Muninn Project is a
research project that extracts structured data from digitized WWI-era archival documents.
The dataset covers 28,742 documents — 28,718 Canadian Expeditionary Force (CEF) attestation
papers
(enlistment forms of individual soldiers) and 24 Canadian unit war… See the full description on the dataset page: https://huggingface.co/datasets/biglam/muninn-ww1-documents.bioreason-pro-sft-reasoning-documents
BioReason-Pro SFT Reasoning Documents
Complete, self-contained training documents reconstructed from the BioReason-Pro SFT data, ready for LLM pre-training.
The upstream dataset wanglab/bioreason-pro-sft-reasoning-data
ships the assistant side of each training example (reasoning, final_answer) alongside the raw
biological context columns, but not the assembled prompt. The prompt cannot be recovered from the
data card alone, because two of its three parts were non-textual… See the full description on the dataset page: https://huggingface.co/datasets/timodonnell/bioreason-pro-sft-reasoning-documents.rag_documentsindian-legal-documents
Indian Legal Documents
Open Indian statutory and legal-document data for AI, search, and legal research.
This dataset is part of the KanoonGPT Open Legal Data Initiative — an effort to make Indian legal data easier to access, structure, and build on for open-source research, legal tech, and production AI systems.
KanoonGPT is building structured Indian legal datasets and data infrastructure for open-source, research, and enterprise AI applications. Learn more at kanoongpt.in.… See the full description on the dataset page: https://huggingface.co/datasets/abby2231/indian-legal-documents.FineWeb-long-context-documentsnyan_documents
Nyan documents
Documents scraped for НЯН Telegram channel from March 2022 to December 2023. The dataset includes documents from 100+ different Telegram news channels.
Usage
pip3 install datasets
from datasets import load_dataset
for row in load_dataset("NyanNyanovich/nyan_documents", split="train", streaming=True):
print(row)
break
Other datasets
Documents (this dataset): https://huggingface.co/datasets/NyanNyanovich/nyan_documents
Clusters:… See the full description on the dataset page: https://huggingface.co/datasets/NyanNyanovich/nyan_documents.NDC_documents_masterhighpassfilter-medical-documentshighpassfilter-medical-documents-o4-miniafrica-morocco-situation-des-documents-durbanisme-2021-ee975c01
Situation Des Documents Durbanisme 2021 | Africa (Morocco Open Data)
42 rows - 1 Africa country/area - time not specified - source table - Engineered by Electric Sheep Africa
TL;DR
This dataset contains 42 rows from Morocco Open Data, covering Situation Des Documents Durbanisme 2021. It is published as ML-ready Parquet with consistent Hugging Face metadata, source provenance, and analysis-friendly loading examples.
What This Dataset Measures… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepafrica/africa-morocco-situation-des-documents-durbanisme-2021-ee975c01.legal-documents-questionsMNLP_rag_documentsMNLP_M3_rag_documents_300tokMNLP_M3_rag_documentsdocuments-preveniaDocuments used by the RAG component of the prevenIA chatbot to answer questions about suicide.
synthdoc-raw_documents-multilingualAI-related-documents
