CoolFace
Datasetpublic

synthetix-institute/latex-data-pub

Hyperion: Scientific LaTeX Corpus (Public) This dataset constitutes the Public Scientific Corpus for the Hyperion Project at the Synthetix Institute. It contains high-fidelity LaTeX source text extracted from diverse scientific repositories, optimized for topological knowledge discovery and relational mapping. Dataset Details Total Documents: ~1,318,468 Average Document Length: Variable (approx. 32KB - 256KB) Primary Domain: Mathematics, Physics, and Chemistry.… See the full description on the dataset page: https://huggingface.co/datasets/synthetix-institute/latex-data-pub.

sourceHugging Facecc-by-sa-4.0updated 6mo agoView on Hugging Face
1likes1.5kdownloads
Dataset Card

Hyperion: Scientific LaTeX Corpus (Public)

This dataset constitutes the Public Scientific Corpus for the Hyperion Project at the Synthetix Institute. It contains high-fidelity LaTeX source text extracted from diverse scientific repositories, optimized for topological knowledge discovery and relational mapping.

Dataset Details

  • —Total Documents: ~1,318,468
  • —Average Document Length: Variable (approx. 32KB - 256KB)
  • —Primary Domain: Mathematics, Physics, and Chemistry.
  • —Goal: Provide the raw evidence layer for the Stage I.A Hyperion Pipeline (Anchor Detection and Morphism Extraction).

Dataset Structure

The dataset is provided in a flat format with the following schema:

ColumnTypeDescription
latexstringThe full LaTeX source of the document.
paper_idstringUnique identifier (arXiv ID or DOI equivalent).
idstringInternal UUID for the document.

Usage

You can load this dataset directly using the Hugging Face datasets library:

python
from datasets import load_dataset

# Stream the dataset to avoid massive memory usage
ds = load_dataset("synthetix-institute/latex-data-pub", split="train", streaming=True)

# Fetch the first document
for row in ds:
    print(row['paper_id'])
    break

Maintenance

This dataset is maintained by the Synthetix Institute. For questions regarding the Hyperion Discovery Engine or the Epistemic Manifold, please refer to the project documentation.