datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
tiny-supervised-datasetreason-tool-use-demo-1500
Dataset info
The dataset is a selection of reasoning toolcalls data from https://huggingface.co/datasets/interstellarninja/hermes_reasoning_tool_use, which contains data from Hermes-Tools、Glaive-FC、ToolAce、Nvidia-When2Call.
The format has been transformed to adapt llama-factory v1 training pipeline.
BenchMAX_Science
Dataset Sources
Paper: BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models
Link: https://huggingface.co/papers/2502.07346
Repository: https://github.com/CONE-MT/BenchMAX
Dataset Description
BenchMAX_Science is a dataset of BenchMAX, sourcing from GPQA, which evaluates the natural science reasoning capability in multilingual scenarios.
We extend the original English dataset to 16 non-English languages.
The data is first translated by Google… See the full description on the dataset page: https://huggingface.co/datasets/LLaMAX/BenchMAX_Science.alpaca_gpt4_zhBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
Removed 6,103 mistruncated examples.
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_zh.
alpaca_zhBorrowed from: https://huggingface.co/datasets/hfl/alpaca_zh_51k
Removed some examples with empty output.
You can use it in LLaMA Factory by specifying dataset: alpaca_zh.
glaive_toolcall_enBorrowed from: https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2
You can use it in LLaMA Factory by specifying dataset: glaive_toolcall_en.
pmc_llama_instructionsThis repo provides part of the dataset used for PMC-LLaMA-13B's instruction tuning.
Data
Size
Link
ChatDoctor
100K
https://www.yunxiangli.top/ChatDoctor/
MedQA
10.2K
https://huggingface.co/datasets/GBaker/MedQA-USMLE-4-options
MedMCQA
183K
https://huggingface.co/datasets/medmcqa
PubmedQA
211K
https://huggingface.co/datasets/pubmed_qa
LiveQA
635
https://huggingface.co/datasets/truehealth/liveqa
MedicationQA
690
https://huggingface.co/datasets/truehealth/medicationqa
UMLS… See the full description on the dataset page: https://huggingface.co/datasets/axiong/pmc_llama_instructions.mmlu-redux-2.0-for-llama.cppMMLU-redux-v2.0 converted for the llama.cpp perplexity multiple choice tool.
Only valid entries where kept, there is no error based prompting included.
Dataset Card for MMLU-Redux-2.0
MMLU-Redux is a subset of 5,700 manually re-annotated questions across 57 MMLU subjects.
Citation
BibTeX:
@misc{gema2024mmlu,
title={Are We Done with MMLU?},
author={Aryo Pradipta Gema and Joshua Ong Jun Leang and Giwon Hong and Alessio Devoto and Alberto Carlo Maria… See the full description on the dataset page: https://huggingface.co/datasets/Green-Sky/mmlu-redux-2.0-for-llama.cpp.headqa-cot-llama31
headqa-cot
Synthetically enhanced responses to the HeadQA dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
To increase the quality of answers from the training splits of the HeadQA dataset, we leverage Llama-3.1-70B-Instruct to generate Chain of Thought(CoT) answers. We create a custom prompt for the dataset, along with a hand-crafted list of… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/headqa-cot-llama31.MMLU-medical-cot-llama31
MMLU-medical-cot
Synthetically enhanced responses to the medical-related questions of the auxiliary train set of the MMLU dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
First, we use Llama-3.1-70B-Instruct to filter the medical-related questions of the auxiliary train set of the MMLU dataset. Next, we leverage Mixtral-8x7B to… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/MMLU-medical-cot-llama31.llama2-hallucination-hidden-states
🧠 LLaMA-2 Hidden-State Hallucination Dataset
Repository: ShoaibSSM/llama2-hallucination-hidden-states
Task: Hallucination Detection via Transformer Internal Representations
Base Model: LLaMA-2-7B
Primary Labels: LLM-Judge + Hybrid Semantic Grounding
📌 Overview
This dataset contains layer-wise hidden states extracted from LLaMA-2-7B during question answering on SQuAD v2, along with structured hallucination labels.
Unlike traditional hallucination datasets that operate… See the full description on the dataset page: https://huggingface.co/datasets/ShoaibSSM/llama2-hallucination-hidden-states.alpaca_enBorrowed from: https://github.com/tatsu-lab/stanford_alpaca
Removed some erroneous examples.
You can use it in LLaMA Factory by specifying dataset: alpaca_en.
alpaca_gpt4_enBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_en.
glaive_toolcall_zhBorrowed from: https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2
Translated by GPT-3.5.
You can use it in LLaMA Factory by specifying dataset: glaive_toolcall_zh.
OpenR1-Math-94kThis dataset was converted from open-r1/OpenR1-Math-220k using the following script.
from datasets import Dataset, load_dataset
SYSTEM = r"Please reason step by step, and put your final answer within \boxed{}."
def generate_data(data):
for sample in data:
for generation, correctness in zip(sample["generations"], sample["correctness_math_verify"]):
if correctness:
yield {
"messages": [
{"role": "system"… See the full description on the dataset page: https://huggingface.co/datasets/llamafactory/OpenR1-Math-94k.mmlu-redux-for-llama.cppMMLU-redux converted for the llama.cpp perplexity multiple choice tool.
Only valid entries where kept, there is no error based prompting included.
Dataset Card for MMLU-Redux
[!TIP]
Please consider using MMLU-Redux-2.0 which contains all 57 MMLU subjects.
MMLU-Redux is a subset of 3,000 manually re-annotated questions across 30 MMLU subjects.
Citation
BibTeX:
@misc{gema2024mmlu,
title={Are We Done with MMLU?},
author={Aryo Pradipta Gema and Joshua Ong… See the full description on the dataset page: https://huggingface.co/datasets/Green-Sky/mmlu-redux-for-llama.cpp.PubMedQAThis dataset is a subset of https://github.com/pubmedqa/pubmedqa.
The training set is composed of 10,000 examples of the PQA-U dataset and the test set is composed of 1,000 examples of the PQA-L dataset.
Citation Information
@inproceedings{jin2019pubmedqa,
title={PubMedQA: A Dataset for Biomedical Research Question Answering},
author={Jin, Qiao and Dhingra, Bhuwan and Liu, Zhengping and Cohen, William and Lu, Xinghua},
booktitle={Proceedings of the 2019 Conference on Empirical… See the full description on the dataset page: https://huggingface.co/datasets/llamafactory/PubMedQA.medqa-cot-llama31
medqa-cot-llama31
Synthetically enhanced responses to the MedQa dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
To increase the quality of answers from the training splits of the MedQA dataset, we leverage Llama-3.1-70B-Instruct to generate Chain of Thought(CoT) answers. We create a custom prompt for the dataset, along with a hand-crafted… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/medqa-cot-llama31.OpenThoughts-114kThis dataset was converted from open-thoughts/OpenThoughts-114k using the following script.
from datasets import Dataset, load_dataset
SYSTEM = (
"You are an assistant that thoroughly explores questions through a systematic long thinking process "
"before providing the final precise and accurate solutions. "
"This requires engaging in a comprehensive cycle of analysis, summarization, exploration, reassessment, "
"reflection, backtracing, and iteration to develop a… See the full description on the dataset page: https://huggingface.co/datasets/llamafactory/OpenThoughts-114k.Llama_3.1-8B-Instruct-Self-CalibrationThe official repository which contains the code and pre-trained models/datasets for our paper Efficient Test-Time Scaling via Self-Calibration.
🔥 Updates
[2025-3-3]: We released our paper.
[2025-2-25]: We released our codes, models and datasets.
🏴 Overview
We propose an efficient test-time scaling method by using model confidence for dynamically sampling adjustment, since confidence can be seen as an intrinsic measure that directly reflects model… See the full description on the dataset page: https://huggingface.co/datasets/HINT-lab/Llama_3.1-8B-Instruct-Self-Calibration.llama2_QA_Economics_230915
Dataset Card for "llama2_QA_Economics_230915"
More Information needed
pokemon-gpt4o-captionsBorrowed from: https://huggingface.co/datasets/jugg1024/pokemon-gpt4o-captions
You can use it in LLaMA Factory by specifying dataset: pokemon_cap.
limaBorrowed from: https://huggingface.co/datasets/GAIR/lima
You can use it in LLaMA Factory by specifying dataset: lima.
fiqaThis dataset is a subset of the FIQA dataset proposed by https://sites.google.com/view/fiqa/home.
The training set is composed of 5,500 examples of the original training set and the test set is composed of 1,000 examples of the original validation set and the original test set.
Instructions are borrowed from: https://huggingface.co/datasets/FinGPT/fingpt-fiqa_qa
RLHF-VBorrowed from: https://huggingface.co/datasets/openbmb/RLHF-V-Dataset
You can use it in LLaMA Factory by specifying dataset: rlhf_v.
justicio-BOE-A-1978-31229-constitucion-by-articles-qa-qa-groq_llama3_70b_8192-sas
Dataset summary
It is an end-to-end evaluation dataset (using SAS metric) for Justicio.
Domain: Legal, Law, Spanish Constitution
Language: Spanish
SAS summary
The concept of Semantic Answer Similarity refers to the evaluation of the semantic similarity between the generated answer and the ground truth. The score ranges from 0 to 1. A higher score indicates a better match between the generated answer and the ground truth.
Justicio summary
Justicio is a… See the full description on the dataset page: https://huggingface.co/datasets/dariolopez/justicio-BOE-A-1978-31229-constitucion-by-articles-qa-qa-groq_llama3_70b_8192-sas.medmcqa-cot-llama31
medqa-cot-llama31
Synthetically enhanced responses to the MedMCQA dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
To increase the quality of answers from the training splits of the MedMCQA dataset, we leverage Llama-3.1-70B-Instruct to generate Chain of Thought(CoT) answers. We create a custom prompt for the dataset, along with a… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/medmcqa-cot-llama31.llama-python-codes-30k
Python Codes - 30k examples, Llama1&2 tokenized dataset
Author
FlyTech
For general guide on how to create, quantize, merge or inference the model and more, visit:
hackmd.io/my_first_ai
Overview
This dataset serves as a rich resource for various Natural Language Processing tasks such as:
Question Answering
Text Generation
Text-to-Text Generation
It primarily focuses on instructional tasks in Python, tokenized specifically for the Llama architecture.… See the full description on the dataset page: https://huggingface.co/datasets/flytech/llama-python-codes-30k.justicio-BOE-A-1978-31229-constitucion-by-articles-qa-multilingual-e5-large-groq_llama3_70b-sas
Dataset summary
It is an end-to-end evaluation dataset (using SAS metric) for Justicio.
Domain: Legal, Law, Spanish Constitution
Language: Spanish
SAS summary
The concept of Semantic Answer Similarity refers to the evaluation of the semantic similarity between the generated answer and the ground truth. The score ranges from 0 to 1. A higher score indicates a better match between the generated answer and the ground truth.
Justicio summary
Justicio is a… See the full description on the dataset page: https://huggingface.co/datasets/dariolopez/justicio-BOE-A-1978-31229-constitucion-by-articles-qa-multilingual-e5-large-groq_llama3_70b-sas.llama-omni-speech-instruct
Llama3.2 Omni Speech Instruct Dataset
This dataset is created for the sole purpose of enhancing the LLM capability to become multi-modals. This dataset has speech instruction
that a model could use to learn and produce the output thus allowing the model to overcome only text input and extends it capabilities
towards processing speech command as well.
Dataset Details
Dataset Description
This dataset can be used to train an LLM model to allow adaptibility in… See the full description on the dataset page: https://huggingface.co/datasets/gruhit-patel/llama-omni-speech-instruct.