wangekxy/classical-tcm-canon

Classical Chinese Medicine Canon — 中医经典文本数据集 (v1)

A curated Traditional Chinese Medicine (TCM) text dataset: clean, full-text digitizations of the foundational Chinese medicine canon — the 内经 (Inner Canon), 难经, 伤寒论 (Treatise on Cold Damage), 金匮要略, and 温病 (warm-disease) classics — assembled from public-domain source works. Useful for LLM training, RAG, and search over classical Chinese medicine / 中医药 literature.

Summary

• —115 distinct works, 9,401,166 Chinese characters (~9.40M)
• —Source: pre-modern works whose authors died before 1912 (China copyright = life + 50y)
• —Zero OCR — every record is extracted directly from a text-format source, not a scan
• —Held back (not in this release): 10 single-source works pending second-source cross-validation (see canon_v1_hold.jsonl)

Provenance & rights

Each record is a faithful transcription of a public-domain classical work. The underlying texts are out of copyright; the digital transcription carries no new copyright (faithful reproduction of a PD text). Rights basis per record is recorded in rights.basis, backed by an author death-year ledger (../rights_ledger.txt). Editions containing a modern annotator's copyrighted commentary were excluded (see Quality control).

Table of contents

Schema (one JSON object per line)

Quality control (pipeline)

1. 1.Source priority — text-format twin preferred; the curated 中华医书集成 numbered series is treated as authoritative.
2. 2.Garbage gate — mojibake extractions (common-Hanzi ratio < 4%) rejected.
3. 3.De-junking — web-compilation ads/nav/pinyin/contact lines stripped; CJK de-spaced.
4. 4.繁简-normalized cross-validation — copies of a work compared via OpenCC-normalized 12-gram overlap; ≥0.85 = validated.
5. 5.Contamination exclusion — editions weaving a modern annotator's copyrighted commentary into the PD base text are removed (e.g. 唐步祺 闡釋 of 伤寒恒论).
6. 6.External spot-check — flagship works verified against Wikisource PD reference.

Validation mix (release): {'singlesource': 61, 'validatedmultisource': 53, 'validatedminorvariance': 1}

Known limitations

• —Punctuation/collation follows each source edition (modern editors' 标点).
• —异体字 variants (e.g. 脏/藏, 腑/府) appear as in the source.
• —10 single-source .chm works are held for v1.1 pending an independent second source (their scanned-PDF twins will be OCR'd in Phase 2 to cross-validate).