datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
belebele
The Belebele Benchmark for Massively Multilingual NLU Evaluation
Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. This dataset enables the evaluation of mono- and multi-lingual models in high-, medium-, and low-resource languages. Each question has four multiple-choice answers and is linked to a short passage from the FLORES-200 dataset. The human annotation procedure was carefully curated to create questions that discriminate… See the full description on the dataset page: https://huggingface.co/datasets/facebook/belebele.Belle_1.4M-SLAM-Omni
Belle_1.4M
This dataset is prepared for the reproduction of SLAM-Omni.
This is a multi-round Chinese spoken dialogue training dataset. For code and usage examples, please refer to the related GitHub repository: X-LANCE/SLAM-LLM (examples/s2s)
🔧 Modifications
Data Filtering: We removed samples with excessively long data.
Speech Response Tokens: We used CosyVoice to synthesize corresponding semantic speech tokens for the speech response. These tokens, represented as… See the full description on the dataset page: https://huggingface.co/datasets/worstchan/Belle_1.4M-SLAM-Omni.bellhart_trainingtrain_1M_CN
内容
包含约100万条由BELLE项目生成的中文指令数据。
样例
{
"instruction": "给定一个文字输入，将其中的所有数字加1。\n“明天的会议在9点开始，记得准时到达。”\n",
"input": "",
"output": "“明天的会议在10点开始，记得准时到达。”"
}
字段：
instruction: 指令
input: 输入（本数据集均为空）
output: 输出
使用限制
仅允许将此数据集及使用此数据集生成的衍生物用于研究目的，不得用于商业，以及其他会对社会带来危害的用途。
本数据集不代表任何一方的立场、利益或想法，无关任何团体的任何类型的主张。因使用本数据集带来的任何损害、纠纷，本项目不承担任何责任。
train_0.5M_CN
内容
包含约50万条由BELLE项目生成的中文指令数据。
样例
{
"instruction": "给定一个文字输入，将其中的所有数字加1。\n“明天的会议在9点开始，记得准时到达。”\n",
"input": "",
"output": "“明天的会议在10点开始，记得准时到达。”"
}
字段：
instruction: 指令
input: 输入（本数据集均为空）
output: 输出
使用限制
仅允许将此数据集及使用此数据集生成的衍生物用于研究目的，不得用于商业，以及其他会对社会带来危害的用途。
本数据集不代表任何一方的立场、利益或想法，无关任何团体的任何类型的主张。因使用本数据集带来的任何损害、纠纷，本项目不承担任何责任。
train_3.5M_CN
内容
包含约350万条由BELLE项目生成的中文指令数据。
样例
{
"id":"66623028",
"conversations":[
{
"from":"human",
"value":"归纳介绍三种不同类型的电影题材"
},
{
"from":"assistant",
"value":"1. 动作片：动作片通常强调动作和战斗场面，故事情节紧凑，充满了紧张和兴奋的氛围。主角一般是具有超凡实力或特殊技能的英雄人物，往往会与反派角色进行一系列激烈的格斗，追逐和爆破等酷炫场面。例如《速度与激情》和《复仇者联盟》等。\n2. 爱情片：爱情片是以爱情为主题的电影，着重呈现角色之间的感情纠葛，展现了爱情的美好和悲壮。情节与角色之间的发展令人深思，增加了观众的感性体验。这种电影的故事情节主要以人类与人类之间的情感变化为主题。例如《泰坦尼克号》和《让子弹飞》等。\n3.… See the full description on the dataset page: https://huggingface.co/datasets/BelleGroup/train_3.5M_CN.BellaTurca
Dataset Card for BellaTurca
BellaTurca is the first large-scale Turkish corpus collection for training Turkish language models. The total size is around 245GB and 30 billion words. BellaTurca's focus is high quality, diversity as well as the size.
This collection is made up of five datasets: AkademikDerlem, OzenliDerlem, ForumSohbetleri, Temiz OSCAR and Temiz mC4. Originally there was a book corpus included, but it is excluded due to containing copyrighted material.
AkademikDerlem… See the full description on the dataset page: https://huggingface.co/datasets/turkish-nlp-suite/BellaTurca.train_2M_CN
内容
包含约200万条由BELLE项目生成的中文指令数据。
样例
{
"instruction": "将以下三个句子组合成一个有意义的段落。\n狗是人类最好的朋友。它们非常聪明，可以进行各种活动。如果你喜欢散步，狗可以成为你一起散步的伙伴。",
"input": "",
"output": "狗是人类最好的朋友，它们非常聪明，可以进行各种活动。如果你喜欢散步，狗可以成为你一起散步的伙伴。出门散步是一种良好的锻炼方式，而有狗的陪伴会让散步变得更有趣，并且有狗在身边也能给你带来安全感。所以，拥有一只狗作为你的伙伴，可以帮助你变得更加积极主动和健康。"
}
字段：
instruction: 指令
input: 输入（本数据集均为空）
output: 输出
使用限制
仅允许将此数据集及使用此数据集生成的衍生物用于研究目的，不得用于商业，以及其他会对社会带来危害的用途。… See the full description on the dataset page: https://huggingface.co/datasets/BelleGroup/train_2M_CN.belebele-fleurs
Belebele-Fleurs
Belebele-Fleurs is a dataset suitable to evaluate two core tasks:
Multilingual Spoken Language Understanding (Listening Comprehension): For each spoken paragraph, the task is to answer a multiple-choice question. The question and four answer choices are provided in text form.
Multilingual Long-Form Automatic Speech Recognition (ASR) with Diverse Speakers: By concatenating sentence-level utterances, long-form audio clips (ranging from 30 seconds to 1 minute 30… See the full description on the dataset page: https://huggingface.co/datasets/WueNLP/belebele-fleurs.2M-Belebele
2M-Belebele
Highly-Multilingual Speech and American Sign Language Comprehension Dataset
We introduce 2M-Belebele as the first highly multilingual speech and American Sign Language (ASL) comprehension dataset. Our dataset, which is an extension of the existing Belebele only-text dataset, covers 74 spoken languages at the intersection of Belebele and Fleurs, and one sign language (ASL).
The speech dataset is built from aligning Belebele, Flores200 and Fleurs datasets as… See the full description on the dataset page: https://huggingface.co/datasets/facebook/2M-Belebele.belebele-auggenerated_chat_0.4M
Generated_Chat_0.4M
内容
包含约40万条由BELLE项目生成的个性化角色对话数据，包含角色介绍。
注意：此数据集是由ChatGPT产生的，未经过严格校验，题目或解题过程可能包含错误。使用过程中请注意这一点。
样例
{
"instruction": "于以下角色信息完成一段对话\nAlice是一个年轻的律师，她正在为一个涉及濒危物种保护的案件做准备。Bob是一位当地的居民，他所在的社区经常受到该濒危物种的干扰。\n",
"input": "",
"output": "\nAlice: 你好Bob，我是濒危物种保护案的律师，很高兴见到你。我们前几天发给你的文件你看到了吗？\nBob: 看到了，你们说要禁止我们在该物种的常住地采集蘑菇，这很不公平。\nAlice: 保护濒危物种对整个生态系统都很重要，如果我们的举动损害了这个物种的生存，那么整个生态系统都将遭受损失。\nBob: 我理解您的立场，但是我们一直以来都依靠这个物种来维持我们的经济生活，现在我们要怎么办？\nAlice:… See the full description on the dataset page: https://huggingface.co/datasets/BelleGroup/generated_chat_0.4M.belebele
The Belebele Benchmark for Massively Multilingual NLU Evaluation
Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. This dataset enables the evaluation of mono- and multi-lingual models in high-, medium-, and low-resource languages. Each question has four multiple-choice answers and is linked to a short passage from the FLORES-200 dataset. The human annotation procedure was carefully curated to create questions that discriminate… See the full description on the dataset page: https://huggingface.co/datasets/mteb/belebele.NLU-Belebele-MCQABelle_1.4M-SLAM-Omni
Belle_1.4M
This dataset is prepared for the reproduction of SLAM-Omni.
This is a multi-round Chinese spoken dialogue training dataset. For code and usage examples, please refer to the related GitHub repository: X-LANCE/SLAM-LLM (examples/s2s)
🔧 Modifications
Data Filtering: We removed samples with excessively long data.
Speech Response Tokens: We used CosyVoice to synthesize corresponding semantic speech tokens for the speech response. These tokens, represented as… See the full description on the dataset page: https://huggingface.co/datasets/mwei/Belle_1.4M-SLAM-Omni.school_math_0.25M
School Math 0.25M
内容
包含约25万条由BELLE项目生成的中文数学题数据，包含解题过程。
注意：此数据集是由ChatGPT产生的，未经过严格校验，题目或解题过程可能包含错误。使用过程中请注意这一点。
样例
{
"instruction": "题目：小华手里有一个装满糖果的袋子，共有12个，小明想知道里面有多少颗糖果，于是他问小华：“你手里的糖果袋子里有偶数个糖果吗？”小华回答：“有，而且多于10颗。”请问小华手里的糖果袋子里最少有几颗糖果？",
"input": "",
"output": "\n由题目可知：小华手里的糖果袋子里有偶数个糖果；\n又知道小华手里的糖果袋子里有多于10颗糖果。\n因为糖果数为偶数，多于10颗，所以糖果数最小必须是12。\n所以小华手里的糖果袋子里最少有12颗糖果。"
}
字段
instruction: 指令
input: 输入（本数据集均为空）
output: 输出
局限性和使用限制… See the full description on the dataset page: https://huggingface.co/datasets/BelleGroup/school_math_0.25M.multiturn_chat_0.8M
Multiturn Chat 0.8M
内容
包含约80万条由BELLE项目生成的用户与助手的多轮对话。
注意：此数据集是由ChatGPT产生的，未经过严格校验，内容可能包含错误。使用过程中请注意这一点。
instruction中包含多轮对话的上文内容，以Human:和Assistant:区分，output中包含当前助手角色的回答。
样例
{
"instruction":… See the full description on the dataset page: https://huggingface.co/datasets/BelleGroup/multiturn_chat_0.8M.tiny-belebeleNemotron-Personas-Belgium
Nemotron-Personas-Belgium
(NL) Een compound-AI-benadering van meertalige Belgische persona's, verankerd in reële verdelingen
(FR) Une approche d'IA composée pour des personas belges multilingues, ancrés dans des distributions réelles
(DE) Ein Compound-KI-Ansatz für mehrsprachige belgische Personas, verankert in realen Verteilungen
(EN) A compound AI approach to multilingual Belgian personas grounded in real-world distributions
Overzicht… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Personas-Belgium.belarusian-verse
Belarusian stress, rhyme, frequency and agreement tables
Five lookup tables for working with Belarusian words as they sound, as they combine, and as
people actually use them:
stress — 1,997,602 word forms with the position of the stressed vowel.
Belarusian does not write stress, but it decides pronunciation (по́бач, not паба́ч), rhyme
and how a line fits a melody.
forms — 2,006,179 word forms with part of speech, gender, case and number,
which is enough to check whether an… See the full description on the dataset page: https://huggingface.co/datasets/YauhenBichel/belarusian-verse.belief_stat2belka-selfies-idspunta-cana-spanish-reviewsThis data set was collected for academic purposes, suitable for some NLP tasks including sentiment analysis.
belka-selfies-train-cls-ftBelebele_N100belebele-ftp
Belebele-FTP
A first-token-prediction (FTP) reframing of facebook/belebele.
Each example is a single text sequence ending in Answer： (fullwidth U+FF1A, no trailing space) so a model can predict the answer as one token (A/B/C/D). Format matches lm-evaluation-harness's belebele template byte-for-byte.
Format
Example (eng_Latn):
P: <passage>
Q: <question>
A: <choice 1>
B: <choice 2>
C: <choice 3>
D: <choice 4>
Answer：
Schema: question_id: int, text: str… See the full description on the dataset page: https://huggingface.co/datasets/mikaberidze/belebele-ftp.nmr-belief-cascade
BeliefCascade Branch Grid
Each row is one complete sequential belief-revision episode. The benchmark
uses a 432-condition grid: nodes per level {2, 3, 4, 5}, level counts
{3, 4, 5}, out-/in-degree complexity bands {20, 50, 80}, and revision
types {monotonic, nmr_retraction, nmr_newinfo, nmr_mixed}. There are 10
train and 50 test episodes for every condition (4,320 train / 21,600 test).
Columns
text: atoms, static dependencies, and inference policy.
belief:… See the full description on the dataset page: https://huggingface.co/datasets/leo-bjpark/nmr-belief-cascade.Belle_train_3.5M_CN
内容
包含约350万条由BELLE项目生成的中文指令数据。
样例
{
"id":"66623028",
"conversations":[
{
"from":"human",
"value":"归纳介绍三种不同类型的电影题材"
},
{
"from":"assistant",
"value":"1. 动作片：动作片通常强调动作和战斗场面，故事情节紧凑，充满了紧张和兴奋的氛围。主角一般是具有超凡实力或特殊技能的英雄人物，往往会与反派角色进行一系列激烈的格斗，追逐和爆破等酷炫场面。例如《速度与激情》和《复仇者联盟》等。\n2. 爱情片：爱情片是以爱情为主题的电影，着重呈现角色之间的感情纠葛，展现了爱情的美好和悲壮。情节与角色之间的发展令人深思，增加了观众的感性体验。这种电影的故事情节主要以人类与人类之间的情感变化为主题。例如《泰坦尼克号》和《让子弹飞》等。\n3.… See the full description on the dataset page: https://huggingface.co/datasets/ticoAg/Belle_train_3.5M_CN.en-vi-ja-curated-500k-triplets
EN-VI-JA Curated 500K Triplets
High-quality English-Vietnamese-Japanese translation triplets curated from OPUS parallel corpora.
Dataset Statistics
Split
Count
Train
446,252
Validation
24,792
Test
24,792
Total
495,836
Quality Metrics
Average Quality Score: 0.794
Average LaBSE Score: 0.859
LaBSE Threshold: 0.6
LaBSE Score Distribution
Range
Count
0.9-1.0
177,022
0.8-0.9
217,463
0.7-0.8
76,727
0.5-0.7
24,624… See the full description on the dataset page: https://huggingface.co/datasets/belumind/en-vi-ja-curated-500k-triplets.belief_r
Belief Revision: The Adaptability of Large Language Models Reasoning
This is the official dataset for the paper "Belief Revision: The Adaptability of Large Language Models Reasoning", published in the main conference of EMNLP 2024.
📚 Data |
📃 Paper
Overview
The capability to reason from text is crucial for real-world NLP applications. Real-world scenarios often involve incomplete or evolving data. In response, individuals update their beliefs and… See the full description on the dataset page: https://huggingface.co/datasets/CAiRE/belief_r.