Datasetpublic

wannaphong/KhanomTanLLM-pretrained-dataset-thai-subset

KhanomTanLLM pretrained dataset (Thai subset) This daataset collect all raw text for pretraining LLM. (Thai subset) Codename: numfa v2 Repository: https://github.com/pythainlp/KhanomTanLLM Thai pythainlp/thai_food_v1.0 pythainlp/thailaw-v1.0 pythainlp/thai-tnhc2-books pythainlp/thai-constitution-corpus pythainlp/thai-it-books pythainlp/prd_news_3011202 pythainlp/thailand-policy-statements pythainlp/thai-cc-license pythainlp/blognone_news pythainlp/goethe-website… See the full description on the dataset page: https://huggingface.co/datasets/wannaphong/KhanomTanLLM-pretrained-dataset-thai-subset.

sourceHugging Faceodc-byupdated 2y agoView on Hugging Face

0likes83downloads

Dataset Card

KhanomTanLLM pretrained dataset (Thai subset)

This daataset collect all raw text for pretraining LLM. (Thai subset)

Codename: numfa v2

Repository: https://github.com/pythainlp/KhanomTanLLM

Thai

—pythainlp/thaifoodv1.0
—pythainlp/thailaw-v1.0
—pythainlp/thai-tnhc2-books
—pythainlp/thai-constitution-corpus
—pythainlp/thai-it-books
—pythainlp/prdnews3011202
—pythainlp/thailand-policy-statements
—pythainlp/thai-cc-license
—pythainlp/blognone_news
—pythainlp/goethe-website
—pythainlp/thai-wiki-dataset-v3
—pythainlp/thai-open-data-go-th
—pythainlp/thai_usembassy
—pythainlp/thai-open-data-text-v1
—pythainlp/thaigov-corpus
—pythainlp/thaigov-v2-corpus-31032024
—pythainlp/thaisum and add more news that update 01042024)
—pythainlp/thai-financial-dataset
—pythainlp/thai-culturax-clean-dataset
—Thai cleaned subset from https://hplt-project.org/datasets/v1.2