PlethoraSolutions/open-scholarly-document-catalog-10k

<p align="center"> <img src="assets/plethora-catalog-banner.png" alt="Plethora Solutions scholarly catalog" width="100%"> </p>

<p align="center"> <a href="https://huggingface.co/spaces/PlethoraSolutions/scholarly-catalog-explorer">Search interactively</a> &nbsp;&middot;&nbsp; <a href="https://plethorasolutions.llc/scholarly-catalog/">Product and methodology</a> &nbsp;&middot;&nbsp; <a href="https://huggingface.co/datasets/PlethoraSolutions/open-scholarly-document-catalog-full">Full catalog</a> </p>

Rights-Audited Scholarly PDF Catalog: 10K Validated Sample

Evaluation sample / inspect before you buy. Metadata and source links only. No PDFs or extracted full text are redistributed.

Build scientific-literature search, RAG, discovery, and corpus-acquisition workflows from scholarly metadata with current PDF-link evidence and per-record Creative Commons or NASA rights signals.

This is a deterministic 10,000-row evaluation sample from Plethora Solutions' rights-audited catalog of 965,792 scholarly-document records. It lets data and ML teams evaluate the schema, source coverage, rights evidence, and link quality before licensing the maintained full catalog.

The sample contains metadata and links. It does not contain PDF files or extracted document text. Rights in linked documents remain with their respective authors, publishers, agencies, or licensors.

Start with evidence

Explore or request the full catalog

• —Interactive explorer: https://huggingface.co/spaces/PlethoraSolutions/scholarly-catalog-explorer
• —Full 965,792-row gated catalog: https://huggingface.co/datasets/PlethoraSolutions/open-scholarly-document-catalog-full
• —Product details and pricing: https://plethorasolutions.llc/scholarly-catalog/
• —Founding-customer checkout ($499 USD): https://www.paypal.com/ncp/payment/4F6QSYDG5DBQJ

The founding-customer price for the full dated commercial snapshot is $499 USD for the first 10 customers; the standard snapshot price is $1,250 USD. The commercial product is the curated metadata compilation, rights evidence, normalization, QA, and delivery—not ownership of public facts or linked papers.

The full-catalog network pass is now complete: all 776,075 distinct non-empty PDF URLs were checked between 2026-08-31 and 2026-09-02, and 611,609 (78.81%) began serving content with a PDF signature. The check read at most 16 KiB per URL and did not download or store complete PDFs. Full results and their evidence hashes are included with the gated catalog.

Designed for

• —Scientific-literature and academic search systems
• —Retrieval-augmented generation and research assistants
• —Rights-aware corpus acquisition and filtering
• —Link-health analysis and repository monitoring
• —Custom scholarly-data exports and due-diligence workflows

Sample composition

The full 2026-08-30 catalog contains 316,314 strong CC candidates and 266,400 strong NASA public-use candidates under the same operational criteria.

Fresh PDF-link validation

Every one of the 8,500 ready-tier rows was checked on 2026-08-31 with a streamed HTTP GET requesting only bytes 0-16,383. The response was closed after at most 16 KiB, and %PDF- had to appear within the first 1,024 bytes. Redirect target, HTTP status, PDF version, size headers, ETag, modification date, and check time are preserved in the link_probe_* fields.

The validation did not download or store complete PDFs. It establishes that the URL began serving PDF content at the recorded time; it does not prove that the entire file is complete or uncorrupted.

The first validation pass found 320 failed CC-ready rows and 74 failed NASA-ready rows. They were removed and replaced by the next deterministic sample-hash candidates only after those replacements passed the same PDF check. All lower-rights tiers remain unchanged so users can evaluate exclusion logic.

Important rights distinction

NASA public-use permission is represented by rights tier N; it is never relabeled as CC-BY. NASA-hosted material can contain contractor, publisher, or third-party content, so the sample preserves NASA's exact determination and risk fields. Tier C and tier D records are not presented as commercially redistributable documents.

See RIGHTS.md before using links or downloading documents.

Selection method

Selection is deterministic and rights-stratified. Eligible rows are ranked by:

SHA256("plethora-public-sample-v1" + NUL + source_id)

The lowest hashes within each sample bucket are retained, skipping ready-tier rows that failed fresh PDF validation. This avoids manual cherry-picking and makes the sample reproducible from the dated release plus the hashed probe evidence recorded in sample_stats.json.

Core fields

Nulls are meaningful. For example, a null NASA field normally means the row is not from NASA, while a null content SHA-256 means the linked PDF bytes were not stored and hashed by this catalog.

Load with Python

from datasets import load_dataset

sample = load_dataset(
    "PlethoraSolutions/open-scholarly-document-catalog-10k",
    split="train",
)
print(sample.num_rows)  # 10000

Or read the Parquet file directly:

import pyarrow.parquet as pq

table = pq.read_table("data/public-sample-10000.parquet")
print(table.schema)

Files and integrity

• —data/public-sample-10000.parquet: the sample data.
• —sample_stats.json: source release, selection, coverage, and population statistics.
• —CHECKSUMS.csv: file size, row count, and SHA-256.
• —sample_validation.json: independent package validation results.
• —RIGHTS.md: operational rights interpretation and limitations.

Limitations

• —This is an operational evidence classification, not a legal opinion.
• —A URL confirmed on 2026-08-31 can later move, expire, or return non-PDF content.
• —Prefix validation does not establish full-file integrity or expected document identity.
• —Tier A reflects machine-readable license evidence, not legal review of each work.
• —NASA records require NASA-specific acknowledgment and non-endorsement handling.
• —The sample is metadata-only and does not itself grant rights to linked documents.
• —Coverage is source-driven and is predominantly English.

Full catalog and update feed

The maintained 965,792-row catalog, dated snapshots, rights-evidence updates, and custom validated exports are available separately through the gated catalog linked above. For questions or custom delivery, email michael@plethorasolutions.llc.