Datasetpublic

ghananlpcommunity/ghanaian-corpus-generate-clean

Ghanaian Corpus Generate — Cleaned Cleaned version of ghananlpcommunity/ghanaian-corpus-generate. What changed The text_clean column was produced by stripping out non-sentence content from text: Removed section/chapter numbering (e.g. 24 5.3, III, 6.1) Removed document headers (e.g. Chapter 6: Conclusion and Recommendations, Abstract, Acknowledgements) Removed figure/table/code references (e.g. Figure 4.10 QR recognition class) Removed very short fragments (< 15… See the full description on the dataset page: https://huggingface.co/datasets/ghananlpcommunity/ghanaian-corpus-generate-clean.

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes77downloads

Dataset Card

Ghanaian Corpus Generate — Cleaned

Cleaned version of ghananlpcommunity/ghanaian-corpus-generate.

What changed

The text_clean column was produced by stripping out non-sentence content from text:

—Removed section/chapter numbering (e.g. 24 5.3, III, 6.1)
—Removed document headers (e.g. Chapter 6: Conclusion and Recommendations, Abstract, Acknowledgements)
—Removed figure/table/code references (e.g. Figure 4.10 QR recognition class)
—Removed very short fragments (< 15 chars) and non-alphabetic noise
—Kept only well-formed English sentences

Stats

	Original (`text`)	Cleaned (`text_clean`)
Mean length	2,377 chars	2,322 chars
Median length	2,275 chars	2,219 chars
Empty rows	0	117 (0.02%)

Columns

Column	Description
`source_type`	Type of source document (e.g. `research`)
`source`	Institution or origin
`page_range`	Page range from the source
`text`	Original text (unmodified)
`generated_question`	Generated question paired with the text
`text_clean`	Cleaned text — only well-formed sentences

Intended use

Translation to Twi (Akan) via the Thai pivot method. Use text_clean as the source column for translation.