datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
twitch-chat
Twitch Chat Messages
Each Twitch channel is stored as its own dataset config, with its chat messages under data/<channel>/. Data arrives as small append-only chunk files (data/<channel>/<chunk-id>.jsonl) added on every publish cycle -- existing chunks are immutable and never re-uploaded, so cost per publish only scales with new messages, not the dataset's total size. Sharding chunks into a per-channel directory also keeps any single directory well under the Hub's 10… See the full description on the dataset page: https://huggingface.co/datasets/Daankular/twitch-chat.Twin-2K-500
Twin-2K-500 Dataset
This dataset Twin-2K-500 contains comprehensive persona information from a representative sample of 2,058 US participants, providing rich demographic and psychological data. The dataset is specifically designed for building digital twins for LLM simulations.
More information on how to use this dataset can be found in our Documentation and GitHub repository.
Details on how the dataset was generated are available in our Paper.
Dataset Creation… See the full description on the dataset page: https://huggingface.co/datasets/LLM-Digital-Twin/Twin-2K-500.tw-leetcode
Dataset Card for tw-leetcode
A curated Traditional Chinese LeetCode solution dataset with high-efficiency answers (Beats 100%), structured explanation in "Top Concept → Step Implement → Complexity Analysis" style, updated daily.
Dataset Details
Dataset Description
tw-leetcode 是一個針對 LeetCode 題目的繁體中文資料集，內容包含高效能程式解法、完整的解題思路，以及時間與空間複雜度分析。每份題解都經由人工清洗與優化，並依循「Top Concept → Step Implement → Complexity Explanation」的結構撰寫，方便機器學習模型或人類讀者理解程式邏輯的推理過程。
本資料集適合作為：… See the full description on the dataset page: https://huggingface.co/datasets/twinkle-ai/tw-leetcode.tw-reasoning-instruct-50k
Dataset Card for tw-reasoning-instruct-50k
tw-reasoning-instruct-50k 是一個精選的 繁體中文（台灣） 推理資料集，旨在提升語言模型於逐步邏輯思考、解釋生成與語言理解等任務中的表現。資料內容涵蓋日常思辨、教育對話、法律推理等多元主題，並結合「思考步驟」與「最終答案」的結構設計，引導模型以更清晰、條理分明的方式進行推論與回應，特別強調符合台灣本地語言與文化背景的應用需求。
Dataset Details
Dataset Description
本資料集專為發展具備強大推理能力的繁體中文大型語言模型（Large Reasoning Models, LRM）所設計，內容深度結合台灣的語言與文化脈絡。每筆資料通常包含使用者的提問、模型的回應，以及清楚的推理過程。資料集設計目標為培養模型具備類人邏輯的逐步思考與解釋能力。
此資料集適用於訓練與評估以下任務：
台灣社會的日常推理
教育性對話
以解釋為導向的生成任務… See the full description on the dataset page: https://huggingface.co/datasets/twinkle-ai/tw-reasoning-instruct-50k.Twin-2K-500-Mega-Study
Twin-2K-500-Mega-Study Dataset
GitHub Repository: https://github.com/TianyiPeng/Twin-2K-500-Mega-Study
To see more details for how to process these data, please refer to this GitHub repository.
This dataset contains survey data from the Twin-2K-500 Mega Study, which tests the validity of using large language models to predict people's future answers based on their answers to past surveys (creating "digital twins" of participants).
Dataset Structure
The dataset is… See the full description on the dataset page: https://huggingface.co/datasets/LLM-Digital-Twin/Twin-2K-500-Mega-Study.tw-drug-labels-vision
Dataset Card for tw-drug-labels-vision
💊 tw-drug-labels-vision 是一份涵蓋臺灣食品藥物管理署（TFDA）核發之 44,663 筆藥品仿單／外盒 的繁體中文多模態資料集。每一筆紀錄同時包含 PDF 全部頁面的渲染圖（WebP 多頁）以及一份依統一 17 欄 JSON Schema 抽取自原始藥品標示文件的結構化資料，可直接用於語言模型微調、視覺語言模型訓練、文件問答、藥品知識檢索、繁體中文醫藥 NLP 任務之素材。
Dataset Details
Dataset Description
本資料集源自臺灣 TFDA 公開的藥品許可證查詢系統。每筆紀錄對應一份藥品文件（仿單或外盒），原始為 PDF 圖檔形式。處理流程分為三階段：
下載：依據 20251222政府開放資料集_仿單與藥品外盒_66032.xlsx 中的 PDF URL，下載原始檔。
頁面渲染：將 PDF 各頁渲染為 WebP 圖檔，封裝在 images 欄位中。
OCR +… See the full description on the dataset page: https://huggingface.co/datasets/twinkle-ai/tw-drug-labels-vision.twitch_chat
Twitch Chat Dataset
This dataset is a large-scale collection of Twitch chat logs aggregated from multiple streamers across various categories. It is designed to support the research and development of models for real-time, informal, and community-driven conversation, such as:
Chatbots tailored for livestream platforms
Simulating the behavior of Twitch chat
Modeling how chat reacts during hype moments, events, or memes
The code for it can be found here
📂 Dataset… See the full description on the dataset page: https://huggingface.co/datasets/lparkourer10/twitch_chat.twi-english-reasoning-sft-mix
Twi-English Reasoning SFT Mix
A ready-to-train supervised fine-tuning dataset for English<->Twi (Akan)
translation, combining a chain-of-thought reasoning pool with a much larger
direct-translation pool into ONE shuffled file, so a standard trainer/
dataloader sees the right mix without any extra sampling code. The messages
format with <think>...</think> in the assistant turn works with any model
whose chat template splits on those tags (or that's simply trained to treat
them as… See the full description on the dataset page: https://huggingface.co/datasets/ghanaopenai/twi-english-reasoning-sft-mix.twi_text_c3Twi Text C3 is the largest Twi texts collected and used to train FastText embeddings in the
YorubaTwi Embedding paper: https://www.aclweb.org/anthology/2020.lrec-1.335/tw-function-call-reasoning-10k
Dataset Card for tw-function-call-reasoning-10k
本資料集為繁體中文版本的函式呼叫（Function Calling）資料集，翻譯自 AymanTarig/function-calling-v0.2-with-r1-cot，而該資料集本身是 Salesforce/xlam-function-calling-60k 的修正版。我們利用語言模型翻譯後，經人工修改，旨在打造高品質的繁體中文工具使用語料。
Dataset Details
Dataset Description
tw-function-call-reasoning-10k 是一個專為語言模型「工具使用能力（Function Calling）」訓練所設計的繁體中文資料集。其內容源自 AymanTarig/function-calling-v0.2-with-r1-cot，該資料集又為 Salesforce/xlam-function-calling-60k… See the full description on the dataset page: https://huggingface.co/datasets/twinkle-ai/tw-function-call-reasoning-10k.Twin-2K-500
Twin-2K-500 Dataset
This dataset Twin-2K-500 contains comprehensive persona information from a representative sample of 2,058 US participants, providing rich demographic and psychological data. The dataset is specifically designed for building digital twins for LLM simulations.
More information on how to use this dataset can be found in our Documentation and GitHub repository.
Details on how the dataset was generated are available in our Paper.
Dataset Creation… See the full description on the dataset page: https://huggingface.co/datasets/krajavi3/Twin-2K-500.Sudoku_DLM_Reasoning
Sudoku DLM Reasoning
stwistzz/Sudoku_DLM_Reasoning is a deterministic 9x9 Sudoku benchmark for studying masked
diffusion language models, depth, and iterative decoding. Version 1.1.0
trains only on original, r0, and r1_4; r5_19 is held out for adjacent
difficulty extrapolation.
Version 1.1.0 changes only the deterministic selection seed to 0
relative to v1.0.2. The raw CSV files, sources, bucket definitions, split quotas,
schema, and validation rules are unchanged; the four… See the full description on the dataset page: https://huggingface.co/datasets/twistshan/Sudoku_DLM_Reasoning.twi-llm-reasoning-dataset-1k
This dataset is shared under CC BY-NC 4.0, which means you are free to use, share, and adapt it for non-commercial research and educational purposes with attribution. You can read the full license at https://creativecommons.org/licenses/by-nc/4.0/.
This dataset is made available because of Ghana NLP's volunteer driven research work. Please consider contributing to any of our projects on Github
Twi Reasoning Dataset
A Twi (Akan) translation of the Multilingual-Thinking… See the full description on the dataset page: https://huggingface.co/datasets/ghanaopenai/twi-llm-reasoning-dataset-1k.Code-170k-twi
This dataset is shared under CC BY-NC 4.0, which means you are free to use, share, and adapt it for non-commercial research and educational purposes with attribution. You can read the full license at https://creativecommons.org/licenses/by-nc/4.0/.
This dataset is made available because of Ghana NLP's volunteer driven research work. Please consider contributing to any of our projects on Github
Dataset Description
Code-170k-twi is a groundbreaking dataset containing 176,999… See the full description on the dataset page: https://huggingface.co/datasets/ghanaopenai/Code-170k-twi.task512_twitter_emotion_classification
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task512_twitter_emotion_classification
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task512_twitter_emotion_classification.Twin-2K-500
Twin-2K-500 Dataset
This dataset Twin-2K-500 contains comprehensive persona information from a representative sample of 2,058 US participants, providing rich demographic and psychological data. The dataset is specifically designed for building digital twins for LLM simulations.
More information on how to use this dataset can be found in our Documentation and GitHub repository.
Details on how the dataset was generated are available in our Paper.
Dataset Creation… See the full description on the dataset page: https://huggingface.co/datasets/chadreadey/Twin-2K-500.TwinRouterBench
TwinRouterBench Static
This dataset contains the static track for TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing.
Paper: arXiv:2605.18859
Contents
data/train.parquet: Hugging Face viewer-friendly table with 970 rows. Nested fields such as messages and optional tool/function schemas are stored as JSON strings so all benchmark sources share a stable schema.
question_bank.jsonl: the original static question bank exported by… See the full description on the dataset page: https://huggingface.co/datasets/Amorph/TwinRouterBench.Twin-2K-500_edit
Twin-2K-500 Dataset
This dataset Twin-2K-500 contains comprehensive persona information from a representative sample of 2,058 US participants, providing rich demographic and psychological data. The dataset is specifically designed for building digital twins for LLM simulations.
More information on how to use this dataset can be found in our Documentation and GitHub repository.
Details on how the dataset was generated are available in our Paper.
Dataset Creation… See the full description on the dataset page: https://huggingface.co/datasets/Shar999/Twin-2K-500_edit.twi-english-reasoning-sft-mix
Twi-English Reasoning SFT Mix
A ready-to-train supervised fine-tuning dataset for English<->Twi (Akan)
translation, combining a chain-of-thought reasoning pool with a much larger
direct-translation pool into ONE shuffled file, so a standard trainer/
dataloader sees the right mix without any extra sampling code. The messages
format with <think>...</think> in the assistant turn works with any model
whose chat template splits on those tags (or that's simply trained to treat
them as… See the full description on the dataset page: https://huggingface.co/datasets/ghananlpcommunity/twi-english-reasoning-sft-mix.tw-instruct-500k-Q-R1
tw-instruct-500k-Q-R1
台灣常見任務對話集（Common Task-Oriented Dialogues in Taiwan） 為台灣社會裡常見的任務對話，從 lianghsun/tw-instruct 截取出 50 萬筆的子集合版本，進行理解力(Reasoning)資料補充生成。
Dataset Details
Dataset Description
這個資料集為合成資料集（synthetic datasets），內容由 a. reference-based 和 b. reference-free 的子資料集組合而成。生成 reference-based 資料集時，會先以我們收集用來訓練 lianghsun/Llama-3.2-Taiwan-3B 時的繁體中文文本作為參考文本，透過 LLM 去生成指令對話集，如果參考文本有特別領域的問法，我們將會特別設計該領域或者是適合該文本的問題；生成 reference-free 時，則是以常見的種子提示（seed prompts）作為參考，讓 LLM… See the full description on the dataset page: https://huggingface.co/datasets/NLTF-mock/tw-instruct-500k-Q-R1.twi_sentences_320k
This dataset is shared under CC BY-NC 4.0, which means you are free to use, share, and adapt it for non-commercial research and educational purposes with attribution. You can read the full license at https://creativecommons.org/licenses/by-nc/4.0/.
This dataset is made available because of Ghana NLP's volunteer driven research work. Please consider contributing to any of our projects on Github
Twi Sentences Dataset
This dataset contains sentences in the Twi language… See the full description on the dataset page: https://huggingface.co/datasets/ghanaopenai/twi_sentences_320k.tw-instruct-500k
Dataset Card for tw-instruct-500k
[👋歡迎加入 Discord 討論，我們正在找人一塊擴充這個對話集🎉]
台灣常見任務對話集（Common Task-Oriented Dialogues in Taiwan） 為臺灣社會裡常見的任務對話，從 lianghsun/tw-instruct 截取出 50 萬筆的子集合版本。最新格式請改用 lianghsun/tw-instruct-500k-2511。
Dataset Details
Dataset Description
本資料集為合成資料集（synthetic dataset），由 a. reference-based 與 b. reference-free 兩種子流程組成：
reference-based：以收集自臺灣的繁中文本（用於訓練 lianghsun/Llama-3.2-Taiwan-3B 之語料）為參考，請 LLM 根據文本特性產生對應領域的指令對話。… See the full description on the dataset page: https://huggingface.co/datasets/lianghsun/tw-instruct-500k.twi-english-reasoning-translation
Twi-English Reasoning Translation Dataset
Chain-of-thought English<->Twi (Akan) translation data, generated for fine-tuning
MiniCPM5-1B. Each row is a two-turn
chat conversation: the user asks for a translation, the assistant answers with its
reasoning inside <think>...</think> (matching MiniCPM5-1B's chat template, which
splits assistant content on those tags) followed by the translation itself.
Sources
source
direction
rows
translation
pristine (from… See the full description on the dataset page: https://huggingface.co/datasets/ghanaopenai/twi-english-reasoning-translation.twi-english-generation-prompts
Twi-English Generation Prompts
English instruction prompts with Twi (Akan) translations, sourced from the Ghanaian Corpus Generation dataset.
Description
534k instruction prompts originally generated from a Ghanaian English corpus. Each prompt was translated from English to Twi via a two-hop pipeline: English -> Thai -> Twi using Google Translate.
Translation Stats
Status
Count
Description
Translated
~532k
Successfully translated to Twi… See the full description on the dataset page: https://huggingface.co/datasets/ghananlpcommunity/twi-english-generation-prompts.pristine-twi
Pristine Twi Dataset
A large-scale Twi language dataset containing clean and naturally sounding Twi text across
four distinct styles — monologue, narrative, dialogue, and storytelling — generated
from real Ghanaian news topics as inspiration to keep it grounded on Ghanaian named entities and vocabulary.
This dataset was built to support the development of Twi language models, tokenizers,
and other NLP tools for the Akan language family.
Dataset Details
Field… See the full description on the dataset page: https://huggingface.co/datasets/fiifinketia/pristine-twi.tw-instruct-500k-cleaned
Dataset Description
此資料集為 lianghsun/tw-instruct 的修正版，資料筆數為 499148 筆。
主要修正以下兩點:
(1) 簡轉繁套件 OpenCC 轉換的一些缺漏及錯誤。
(2) 刪除 模型回答無窮回覆 的資料
(1) 錯誤包含但不限於:
自「制」果醬 → 自「製」果醬
「酸奶」 → 「優酪乳」
小「貼士」 → 小「提醒」
「俯臥撐」 → 「伏地挺身」
QR「碼」 → QR 「code」
「幹」擾 → 「干」擾
濃「鬱」 → 濃「郁」
適「閤」 → 適「合」
「瞭」解 → 「了」解
「引」數 → 「參」數
以上僅為部分舉例。而在修改過程中，並非只作字詞轉換，會考慮到許多包含 關鍵字 前後語的情況
舉例說明:
例如上述範例1：
法「制」作業 並不會 轉換為 法「製」作業。
此資料集 已知但未處理 的錯誤如以下:
生抽、老抽 未作轉換
程序、程式 的誤用
(2) 模型回答無窮回覆
刪除852筆 無窮回覆資料，刪除資料舉例如下:
{
'conversations':
[
{… See the full description on the dataset page: https://huggingface.co/datasets/minyichen/tw-instruct-500k-cleaned.twinkle-dialogue-gemma3-2025-08
Twinkle Dialogue (Gemma-3-12B-it, 2025-08)
本資料集由 Gemma-3-12B-it（Twinkle AI 社群服務） 生成之對話資料，採用 OpenAI Chat Messages 格式（.jsonl），並整合：
Reference-free（由 seed 派生單輪問答）
Reference-based（依據參考文本生成單輪問答）
檔案路徑：data/train.jsonl（選配：data/train.parquet）
結構說明
每列為一筆樣本：{"id": "...", "type": "...", "messages": [{"role":"system","content":"..."}, ...]}
訓練時可擷取第一個 user 與對應 assistant 形成 (instruction, response) pair，或直接使用 chat 格式的 trainer。
來源與限制… See the full description on the dataset page: https://huggingface.co/datasets/tw-llama/twinkle-dialogue-gemma3-2025-08.pristine-twi
This dataset is shared under CC BY-NC 4.0, which means you are free to use, share, and adapt it for non-commercial research and educational purposes with attribution. You can read the full license at https://creativecommons.org/licenses/by-nc/4.0/.
Pristine Twi Dataset
A large-scale Twi language dataset containing clean and naturally sounding Twi text across
four distinct styles — monologue, narrative, dialogue, and storytelling — generated
from real Ghanaian news topics as… See the full description on the dataset page: https://huggingface.co/datasets/ghananlpcommunity/pristine-twi.tw-math-reasoning-2k
Dataset Card for tw-math-reasoning-2k
tw-math-reasoning-2k 是一個繁體中文數學語言資料集，從 HuggingFaceH4/MATH 英文數學題庫中精選 2,000 題，並透過 perplexity-ai/r1-1776 模型以繁體中文重新生成具邏輯性且詳盡的解題過程與最終答案。此資料集可作為訓練或評估繁體中文數學推理模型的高品質參考語料。
Dataset Details
Dataset Description
tw-math-reasoning-2k 是一個繁體中文數學語言資料集，旨在提供高品質的解題語料以支援中文數學推理模型的訓練與評估。此資料集從 HuggingFaceH4/MATH 英文數學題庫中精選 2,000 題，涵蓋代數、幾何、機率統計等各類題型，並確保題目類型分佈均衡。
所有題目皆經由 perplexity-ai/r1-1776… See the full description on the dataset page: https://huggingface.co/datasets/twinkle-ai/tw-math-reasoning-2k.twitch_chat
Twitch Chat Dataset
This dataset is a large-scale collection of Twitch chat logs aggregated from multiple streamers across various categories. It is designed to support the research and development of models for real-time, informal, and community-driven conversation, such as:
Chatbots tailored for livestream platforms
Simulating the behavior of Twitch chat
Modeling how chat reacts during hype moments, events, or memes
The code for it can be found here
📂 Dataset… See the full description on the dataset page: https://huggingface.co/datasets/S1lver404/twitch_chat.