Datasetpublic

zh-tw-llm-dv/zh-tw-pythia-ta8000-v1-e1-tr_sg-302-c1024

zh-tw-pythia-ta8000-v1-e1-tr_sg-302-c1024 This dataset is a part of the zh-tw-llm project. Tokenizer: zh-tw-pythia-tokenizer-a8000-v1 Built with: translations, sharegpt Rows: train 305958, test 195 Max length: 1024 Full config:{"build_with": ["translations", "sharegpt"], "preview_length": 128, "translations_settings": {"source_dataset": "zetavg/coct-en-zh-tw-translations-twp-300k", "lang_1_key": "en", "lang_2_key": "ch", "templates": ["English: {lang_1}\nChinese: {lang_2}"… See the full description on the dataset page: https://huggingface.co/datasets/zh-tw-llm-dv/zh-tw-pythia-ta8000-v1-e1-tr_sg-302-c1024.

sourceHugging Faceupdated 3y agoView on Hugging Face

0likes21downloads

Dataset Card

zh-tw-pythia-ta8000-v1-e1-tr_sg-302-c1024

This dataset is a part of the zh-tw-llm project.

—Tokenizer: zh-tw-pythia-tokenizer-a8000-v1
—Built with: translations, sharegpt
—Rows: train 305958, test 195
—Max length: 1024
—Full config:

json

  {"build_with": ["translations", "sharegpt"], "preview_length": 128, "translations_settings": {"source_dataset": "zetavg/coct-en-zh-tw-translations-twp-300k", "lang_1_key": "en", "lang_2_key": "ch", "templates": ["English: {lang_1}\nChinese: {lang_2}", "Chinese: {lang_2}\nEnglish: {lang_1}"], "use_template": "random", "rows_limit": 300000, "test_size": 100, "test_split_seed": 42}, "sharegpt_settings": {"source_dataset": "zetavg/ShareGPT-Processed", "train_on_inputs": false, "languages": [{"en": 0.4}, "zh_Hant"], "rows_limit": 8000, "test_size": 0.02, "test_split_seed": 42, "test_rows_limit": 100}}