tadad/french-fiction-16-18th-century

French Fiction of the 16th–18th Centuries

A Hugging Face conversion of Pierre-Carl Langlais's French Fiction of the 16–18th century deposit for the BigLAM community. It contains historical French OCR, bibliographic metadata, a genre-labeled and lemmatized subset, and the source R model.

The Zenodo deposit is the source of record. This conversion preserves its OCR, metadata, work assignments, and labels without scholarly correction.

Structure

The split names follow the deposited filenames and are not strict date filters. Metadata dates range from 1473 to 1867 in century_16 and from 1510 to 1880 in century_17.

from datasets import load_dataset

pages = load_dataset("tadad/french-fiction-16-18th-century", "pages")
metadata = load_dataset("tadad/french-fiction-16-18th-century", "metadata")
volumes = load_dataset("tadad/french-fiction-16-18th-century", "volumes")
fresnoy = load_dataset("tadad/french-fiction-16-18th-century", "fresnoy")

Fields and joins

• —pages: page_id, source partition and row, original and normalized file IDs, provider, item URL, absolute file-page number, verbatim OCR text, and links in metadata_ids and work_uids.
• —metadata: titles, authors, dates, deposited work and edition IDs, digitized-file IDs, provider URLs, and BnF edition IDs. work_uid qualifies the deposited work_id with its source partition.
• —volumes: one derived row per digitized file. page_ids and page_numbers preserve page membership and order; text joins the page OCR with "\n\f\n". Character counts and the union of metadata and work links are included.
• —fresnoy: labeled pages with deposited genre_label and lemma_text. parent_page_id links every row to its source row in pages/century_17; file ID, page number, and OCR text are validated.

The Fresnoy subset contains four labels:

lemma_text was produced with the historical-French LEM17 model. The original and lemmatized text are both retained.

Provenance and conversion

The source creator identified novels through the Bibliothèque nationale de France's historical Y2 catalog classification, then located digitized copies in Gallica, Google Books, Internet Archive, and other libraries. The deposit reports coverage of 78% of identified novels from 1450–1600 and 68% from 1600–1700, and describes its work and multi-volume assignments as tentative.

The conversion verifies the size and MD5 checksum of each Zenodo file, parses quoted TSV records, assigns stable row IDs, normalizes filename-like IDs for joins, derives provider URLs, converts missing-value markers to nulls, writes typed Parquet, and produces the volume view. reports/build-report.json records output checksums and corpus statistics.

The deposited fresnoy_model.rda is preserved byte-for-byte as an archival artifact. The source reports 93% accuracy and possible overfitting, but provides no locked R environment, training script, or evaluation split. This repository does not claim to reproduce that score.

Uses and limitations

The page and volume views support corpus analysis, retrieval, historical-language modeling, and OCR research. Fresnoy supports genre-classification experiments.

• —OCR is uncorrected and includes recognition errors, page furniture, provider notices, and segmentation artifacts.
• —Bibliographic coverage is incomplete. The metadata splits contain 77 and 336 rows without a digitized file ID; 566 and 543 page rows have no metadata match.
• —A file can match several metadata rows. All links are retained; the maximum is three matches.
• —Work, edition, and multi-volume relationships are research data, not authoritative catalog records.
• —Provider and catalog selection introduce collection and digitization bias.
• —Fresnoy has no predefined evaluation split. Split by work_uids, or at least file_id, to prevent pages from the same text appearing in training and testing.

Rebuild

uv sync
uv run python -m conversion.build_dataset
uv run python -m conversion.validate_dataset
uv run pytest

Downloads are cached under .cache/. The validator checks schemas, row counts, unique IDs, joins, volume reconstruction, genre counts, and source checksums.

License and citation

The Zenodo deposit is CC BY 4.0. Attribute the source creator and cite the deposit. Digitized items originate from Gallica, Google Books, and Internet Archive; consult their terms before retrieving or redistributing additional assets. No page images are included here.

@dataset{langlais_french_fiction_2021,
  author    = {Langlais, Pierre-Carl},
  title     = {French Fiction of the 16-18th century},
  year      = {2021},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.5770866},
  url       = {https://doi.org/10.5281/zenodo.5770866}
}

This conversion responds to BigLAM issue #86.