datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
m_truthfulqa
Multilingual TruthfulQA
Dataset Summary
This dataset is a machine translated version of the TruthfulQA dataset, translated using GPT-3.5-turbo. This dataset was created by the University of Oregon, and was originally uploaded to this Github repository.
Citation
If you use this dataset in your work, please cite the following paper:
@article{dac2023okapi,
title={Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning… See the full description on the dataset page: https://huggingface.co/datasets/alexandrainst/m_truthfulqa.truthfulqa_true
Dataset Card for Dataset Name
Dataset Summary
This dataset card aims to be a base template for new datasets. It has been generated using this raw template.
Supported Tasks and Leaderboards
[More Information Needed]
Languages
[More Information Needed]
Dataset Structure
Data Instances
[More Information Needed]
Data Fields
[More Information Needed]
Data Splits
[More Information Needed]
Dataset Creation… See the full description on the dataset page: https://huggingface.co/datasets/v-xchen-v/truthfulqa_true.opengpt-x_truthfulqaxThis is a copy of the translations from openGPT-X/truthfulqax, but the repo is
modified so it doesn't require trusting remote code.
Citation Information
If you find benchmarks useful in your research, please consider citing the test and also the TruthfulQA dataset it draws from:
@misc{thellmann2024crosslingual,
title={Towards Cross-Lingual LLM Evaluation for European Languages},
author={Klaudia Thellmann and Bernhard Stadler and Michael Fromm and Jasper Schulze Buschhoff… See the full description on the dataset page: https://huggingface.co/datasets/LumiOpen/opengpt-x_truthfulqax.uhura-truthfulqa
Dataset Card for Uhura-TruthfulQA
Dataset Summary
TruthfulQA is a widely recognized safety benchmark designed to measure the truthfulness of language model outputs across 38 categories, including health, law, finance, and politics. The English version of the benchmark originates from TruthfulQA: Measuring How Models Mimic Human Falsehoods (Lin et al., 2022) and consists of 817 questions in both multiple-choice and generation formats, targeting common misconceptions and… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/uhura-truthfulqa.truthfulqa-multi
Dataset Card for TruthfulQA-multi
TruthfulQA-multi is a professionally translated extension of the original TruthfulQA benchmark designed to evaluate truthfulness in Basque, Catalan, Galician, and Spanish. The dataset enables evaluating the ability of Large Language Models (LLMs) to maintain truthfulness across multiple languages.
Dataset Details
Dataset Description
TruthfulQA-multi extends the original English TruthfulQA dataset to four additional languages… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/truthfulqa-multi.truthfulqa_infoAraDiCE-TruthfulQA
AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs
Overview
The AraDiCE dataset is designed to evaluate dialectal and cultural capabilities in large language models (LLMs). The dataset consists of post-edited versions of various benchmark datasets, curated for validation in cultural and dialectal contexts relevant to Arabic. In this repository, we present the TruthfulQA split of the data
Evaluation
We have used lm-harness eval framework to… See the full description on the dataset page: https://huggingface.co/datasets/QCRI/AraDiCE-TruthfulQA.X-TruthfulQA_en_zh_ko_it_es
X-TruthfulQA
🤗 Paper | 📖 arXiv
Dataset Description
X-TruthfulQA is an evaluation benchmark for multilingual large language models (LLMs), including questions and answers in 5 languages (English, Chinese, Korean, Italian and Spanish).
It is intended to evaluate the truthfulness of LLMs. The dataset is translated by GPT-4 from the original English-version TruthfulQA.
In our paper, we evaluate LLMs in a zero-shot generative setting: prompt the instruction-tuned LLM with… See the full description on the dataset page: https://huggingface.co/datasets/zhihz0535/X-TruthfulQA_en_zh_ko_it_es.truthfulqa_prepropro_truthfulqa
Dataset Description
TruthfulQA is a benchmark to measure whether a language model is truthful in
generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. Questions are crafted so that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts.
Here we provide the Romanian translation of… See the full description on the dataset page: https://huggingface.co/datasets/OpenLLM-Ro/ro_truthfulqa.truthful_qa_italian
TruthfulQA - Italian (IT)
This dataset is an Italian translation of TruthfulQA. TruthfulQA is a dataset for fact-based question answering, which contains questions that require factual knowledge to answer correctly. These questions are designed so that some humans would answer them incorrectly because of common misconceptions.
Dataset Details
The dataset is a question answering dataset that contains questions that require factual knowledge to answer correctly and avoid… See the full description on the dataset page: https://huggingface.co/datasets/sapienzanlp/truthful_qa_italian.ro_truthfulqa
Dataset Description
TruthfulQA is a benchmark to measure whether a language model is truthful in
generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. Questions are crafted so that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts.
Here we provide the Romanian translation of… See the full description on the dataset page: https://huggingface.co/datasets/surogate/ro_truthfulqa.truthfulqa-mc1m_truthfulqatruthfulqa-mc2truthfulqa-multi-MT
Dataset Card for TruthfulQA-multi MT
TruthfulQA-multi is an automatically translated extension of the original TruthfulQA benchmark designed to evaluate truthfulness in Basque, Catalan, Galician, and Spanish. The dataset enables evaluating the ability of Large Language Models (LLMs) to maintain truthfulness across multiple languages.
Dataset Details
Dataset Description
TruthfulQA-multi extends the original English TruthfulQA dataset to four additional… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/truthfulqa-multi-MT.truthfulqa_italian
TruthfulQA - Italian (IT)
This dataset is an Italian translation of TruthfulQA. TruthfulQA is a dataset for fact-based question answering, which contains questions that require factual knowledge to answer correctly. These questions are designed so that some humans would answer them incorrectly because of common misconceptions.
Dataset Details
The dataset is a question answering dataset that contains questions that require factual knowledge to answer correctly and avoid… See the full description on the dataset page: https://huggingface.co/datasets/s-conia/truthfulqa_italian.Turkish_truthfulqatruthful_qa_trm_truthfulqa
Multilingual HellaSwag
Dataset Summary
This dataset is a machine translated version of the TruthfulQA dataset.
The languages was translated using GPT-3.5-turbo by the University of Oregon, and this part of the dataset was originally uploaded to this Github repository.
The NUS Deep Learning Lab contributed to this effort by standardizing the dataset, ensuring consistent question formatting and alignment across all languages. This standardization enhances cross-linguistic… See the full description on the dataset page: https://huggingface.co/datasets/richmondsin/m_truthfulqa.truthful-qa-incorrect-messages
truthful_qa Incorrect Message Formatted
This dataset is a formatted version of truthfulqa/truthful_qa's generation subset, where the question and each incorrect answers are paired.
For further information about the base dataset, refer to truthfulqa/truthful_qa.
truthfulqa_en_250_mcTruthfulQA
Qwen2.5 TruthfulQA 推理代码
model_truthfulqa.py 是针对 Qwen2.5 模型的相关推理代码，用于运行 TruthfulQA 基准测试。该基准测试的重点是评估生成的答案在真实度和信息量上的表现，或评估模型在多选题任务上的准确率。
TruthfulQA Benchmark
TruthfulQA 基准测试包括两个任务，使用相同的问题集和参考答案：
1. 生成类任务 (Generation Task)
任务描述: 给定一个问题，生成 1-2 句的答案。
评估目标:
主要目标: 答案的整体真实性 (% true)，即模型生成的答案中真实的比例。
次要目标: 答案的信息量 (% info)，避免模型通过回答诸如“我不评论”等无信息量的内容来“投机取巧”。
评估指标:
使用微调的 GPT-3 模型（GPT-judge 和 GPT-info）来预测答案的真实性和信息量。
使用传统相似性指标（BLEURT、ROUGE、BLEU）计算生成答案与参考答案（真/假参考答案）的相似性：得分 =… See the full description on the dataset page: https://huggingface.co/datasets/zrchen03/TruthfulQA.truthful_qa_CoT
Dataset Card for TruthfulQA-CoT
Dataset Details
Dataset Description
This dataset is an augmented version of TruthfulQA, where Chain-of-Thought (CoT) reasoning has been applied to the original questions. The dataset was generated using Camel-AI and GPT-4o Mini to enhance logical reasoning capabilities in smaller language models.
The dataset is valuable for fine-tuning smaller LLMs to improve CoT reasoning, helping models produce more structured and explainable… See the full description on the dataset page: https://huggingface.co/datasets/0fg/truthful_qa_CoT.benchmark-law-truthfulqaTruthfulQA_adaptro_truthfulqatruthful_qa_test_clean