lkevincc0/Akkadian-OCR-Corpus

Akkadian OCR Corpus

OCR-extracted corpus from Old Assyrian cuneiform tablet publications and Akkadian linguistic resources, designed for LLM pretraining on ancient Mesopotamian languages.

Dataset Description

This dataset contains OCR-processed text from academic publications on Old Assyrian studies, including cuneiform tablet transliterations, translations, and the Chicago Assyrian Dictionary (CAD).

Splits

Data Fields

Usage

Load with HuggingFace Datasets

from datasets import load_dataset

# Load all splits
dataset = load_dataset("path/to/data")

# Load specific split
tablets = load_dataset("path/to/data", split="tablets")
grammars = load_dataset("path/to/data", split="grammars")

# Access data
for example in tablets:
    print(example["text"])

Load with Pandas

import pandas as pd

tablets = pd.read_parquet("data/tablets.parquet")
grammars = pd.read_parquet("data/grammars.parquet")

# Combine if needed
combined = pd.concat([tablets, grammars], ignore_index=True)

Data Sources

Tablets (AKT Series)

• —AKT 1-12: Ankara Kültepe Tabletleri (Ankaraner Kültepe-Tafeln)
• —Old Assyrian trade documents, letters, and legal texts
• —Languages: Akkadian (Old Assyrian dialect), with Turkish and German annotations

Grammars (CAD)

• —Chicago Assyrian Dictionary: Comprehensive Akkadian dictionary
• —Volumes A-Z with detailed word entries, citations, and translations
• —Language: English with Akkadian citations

Processing Pipeline

1. 1.OCR: GLM-OCR with vLLM inference
2. 2.Page Splitting: Split by ## Page X markers
3. 3.Quality Filtering:
4. 4.Removed pages < 200 characters
5. 5.Removed blurry/unreadable pages
6. 6.Removed mostly numeric pages
7. 7.Removed table of contents pages

Intended Use

• —LLM Pretraining: Continual pretraining on Akkadian/Old Assyrian language
• —Translation Models: Training Akkadian-English translation systems
• —Academic Research: Cuneiform studies and Assyriology

Limitations

• —OCR may have errors, especially for cuneiform transliteration characters
• —Some pages with complex tables or figures may have formatting issues
• —Content is from copyrighted academic publications

Citation

If you use this dataset, please cite the original publications:

@book{akt_series,
  title = {Ankara Kültepe Tabletleri},
  publisher = {Türk Tarih Kurumu},
  year = {1990-2015}
}

@book{cad,
  title = {The Assyrian Dictionary of the Oriental Institute of the University of Chicago},
  author = {{Oriental Institute}},
  publisher = {University of Chicago},
  year = {1956-2010}
}

License

This dataset is derived from academic publications. Please respect the original copyright holders and use for research purposes only.