Datasetpublic

superdoc-dev/docx-corpus

docx-corpus The largest classified corpus of Word documents. 736K+ .docx files from the public web, classified into 10 document types and 9 topics across 76 languages. Dataset Description This dataset contains metadata for publicly available .docx files collected from the web. Each document has been classified by document type and topic using a two-stage pipeline: LLM labeling (Claude) of a stratified sample, followed by fine-tuned XLM-RoBERTa classifiers applied… See the full description on the dataset page: https://huggingface.co/datasets/superdoc-dev/docx-corpus.

sourceHugging Faceodc-byupdated 7mo agoView on Hugging Face

5likes382downloads

Dataset Card

docx-corpus

The largest classified corpus of Word documents. 736K+ .docx files from the public web, classified into 10 document types and 9 topics across 76 languages.

Dataset Description

This dataset contains metadata for publicly available .docx files collected from the web. Each document has been classified by document type and topic using a two-stage pipeline: LLM labeling (Claude) of a stratified sample, followed by fine-tuned XLM-RoBERTa classifiers applied at scale.

Schema

Column	Type	Description
`id`	string	SHA-256 hash of the file (unique identifier)
`filename`	string	Original filename from the source URL
`type`	string	Document type (10 classes)
`topic`	string	Document topic (9 classes)
`language`	string	Detected language (ISO 639-1 code)
`word_count`	int	Number of words in the document
`confidence`	float	Classification confidence (min of type and topic)
`url`	string	Direct download URL for the `.docx` file

Document Types

legal, forms, reports, policies, educational, correspondence, technical, administrative, creative, reference

Topics

government, education, healthcare, finance, legal_judicial, technology, environment, nonprofit, general

Download Files

Each row includes a url column pointing to the .docx file on our CDN. You can download files directly:

python

from datasets import load_dataset
import requests

ds = load_dataset("superdoc-dev/docx-corpus", split="train")

# Filter and download
legal_en = ds.filter(lambda x: x["type"] == "legal" and x["language"] == "en")
for row in legal_en:
    resp = requests.get(row["url"])
    with open(f"corpus/{row['id']}.docx", "wb") as f:
        f.write(resp.content)

Or use the manifest API for bulk downloads:

bash