datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
tiny-supervised-datasetreason-tool-use-demo-1500
Dataset info
The dataset is a selection of reasoning toolcalls data from https://huggingface.co/datasets/interstellarninja/hermes_reasoning_tool_use, which contains data from Hermes-Tools、Glaive-FC、ToolAce、Nvidia-When2Call.
The format has been transformed to adapt llama-factory v1 training pipeline.
BenchMAX_Science
Dataset Sources
Paper: BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models
Link: https://huggingface.co/papers/2502.07346
Repository: https://github.com/CONE-MT/BenchMAX
Dataset Description
BenchMAX_Science is a dataset of BenchMAX, sourcing from GPQA, which evaluates the natural science reasoning capability in multilingual scenarios.
We extend the original English dataset to 16 non-English languages.
The data is first translated by Google… See the full description on the dataset page: https://huggingface.co/datasets/LLaMAX/BenchMAX_Science.alpaca_gpt4_zhBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
Removed 6,103 mistruncated examples.
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_zh.
alpaca_zhBorrowed from: https://huggingface.co/datasets/hfl/alpaca_zh_51k
Removed some examples with empty output.
You can use it in LLaMA Factory by specifying dataset: alpaca_zh.
glaive_toolcall_enBorrowed from: https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2
You can use it in LLaMA Factory by specifying dataset: glaive_toolcall_en.
pmc_llama_instructionsThis repo provides part of the dataset used for PMC-LLaMA-13B's instruction tuning.
Data
Size
Link
ChatDoctor
100K
https://www.yunxiangli.top/ChatDoctor/
MedQA
10.2K
https://huggingface.co/datasets/GBaker/MedQA-USMLE-4-options
MedMCQA
183K
https://huggingface.co/datasets/medmcqa
PubmedQA
211K
https://huggingface.co/datasets/pubmed_qa
LiveQA
635
https://huggingface.co/datasets/truehealth/liveqa
MedicationQA
690
https://huggingface.co/datasets/truehealth/medicationqa
UMLS… See the full description on the dataset page: https://huggingface.co/datasets/axiong/pmc_llama_instructions.MMLU-medical-cot-llama31
MMLU-medical-cot
Synthetically enhanced responses to the medical-related questions of the auxiliary train set of the MMLU dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
First, we use Llama-3.1-70B-Instruct to filter the medical-related questions of the auxiliary train set of the MMLU dataset. Next, we leverage Mixtral-8x7B to… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/MMLU-medical-cot-llama31.headqa-cot-llama31
headqa-cot
Synthetically enhanced responses to the HeadQA dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
To increase the quality of answers from the training splits of the HeadQA dataset, we leverage Llama-3.1-70B-Instruct to generate Chain of Thought(CoT) answers. We create a custom prompt for the dataset, along with a hand-crafted list of… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/headqa-cot-llama31.alpaca_enBorrowed from: https://github.com/tatsu-lab/stanford_alpaca
Removed some erroneous examples.
You can use it in LLaMA Factory by specifying dataset: alpaca_en.
alpaca_gpt4_enBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_en.
glaive_toolcall_zhBorrowed from: https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2
Translated by GPT-3.5.
You can use it in LLaMA Factory by specifying dataset: glaive_toolcall_zh.
PubMedQAThis dataset is a subset of https://github.com/pubmedqa/pubmedqa.
The training set is composed of 10,000 examples of the PQA-U dataset and the test set is composed of 1,000 examples of the PQA-L dataset.
Citation Information
@inproceedings{jin2019pubmedqa,
title={PubMedQA: A Dataset for Biomedical Research Question Answering},
author={Jin, Qiao and Dhingra, Bhuwan and Liu, Zhengping and Cohen, William and Lu, Xinghua},
booktitle={Proceedings of the 2019 Conference on Empirical… See the full description on the dataset page: https://huggingface.co/datasets/llamafactory/PubMedQA.medqa-cot-llama31
medqa-cot-llama31
Synthetically enhanced responses to the MedQa dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
To increase the quality of answers from the training splits of the MedQA dataset, we leverage Llama-3.1-70B-Instruct to generate Chain of Thought(CoT) answers. We create a custom prompt for the dataset, along with a hand-crafted… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/medqa-cot-llama31.fiqaThis dataset is a subset of the FIQA dataset proposed by https://sites.google.com/view/fiqa/home.
The training set is composed of 5,500 examples of the original training set and the test set is composed of 1,000 examples of the original validation set and the original test set.
Instructions are borrowed from: https://huggingface.co/datasets/FinGPT/fingpt-fiqa_qa
limaBorrowed from: https://huggingface.co/datasets/GAIR/lima
You can use it in LLaMA Factory by specifying dataset: lima.
medmcqa-cot-llama31
medqa-cot-llama31
Synthetically enhanced responses to the MedMCQA dataset. Used to train Aloe-Beta model.
Dataset Details
Dataset Description
To increase the quality of answers from the training splits of the MedMCQA dataset, we leverage Llama-3.1-70B-Instruct to generate Chain of Thought(CoT) answers. We create a custom prompt for the dataset, along with a… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/medmcqa-cot-llama31.llama-python-codes-30k
Python Codes - 30k examples, Llama1&2 tokenized dataset
Author
FlyTech
For general guide on how to create, quantize, merge or inference the model and more, visit:
hackmd.io/my_first_ai
Overview
This dataset serves as a rich resource for various Natural Language Processing tasks such as:
Question Answering
Text Generation
Text-to-Text Generation
It primarily focuses on instructional tasks in Python, tokenized specifically for the Llama architecture.… See the full description on the dataset page: https://huggingface.co/datasets/flytech/llama-python-codes-30k.gov-report-qs-llama2-format
Government Report Question Answering Dataset in LLAMA2 Format
Dataset Description
This dataset is a LLAMA2 formatted dataset of the GovReport Dataset which is a report dataset, consisting of reports written by government research agencies including Congressional Research Service and US Government Accountability Office.
The purpose of creating this dataset is to provide those trying to finetune LLAMA2 and other LLM models for Government domain a formatted and easier to use… See the full description on the dataset page: https://huggingface.co/datasets/Kira-Floris/gov-report-qs-llama2-format.Llama-3-70b-battlesChatbot Arena user conversations between Llama-3-70b VS GPT-4-1025 or Llama-3-70b VS Claude-3-Opus with user preference votes. Single turn. Excludes ties.
Used in Llama Data Analysis blog post and "VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models" (Paper, Code).
Citation
@article{dunlap_vibecheck,
title={VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models},
author={Lisa Dunlap and Krishna Mandal and Trevor… See the full description on the dataset page: https://huggingface.co/datasets/lmarena-ai/Llama-3-70b-battles.OpenO1-SFTBorrowed from: https://huggingface.co/datasets/O1-OPEN/OpenO1-SFT
You can use it in LLaMA Factory by specifying dataset: openo1_sft.
LongBench-v2-for-llama.cppLongBench v2 converted for the llama.cpp perplexity multiple choice tool.
[!WARNING]
!! Currently does not work, will fix it in the near future. Probably.
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
🌐 Project Page: https://longbench2.github.io
💻 Github Repo: https://github.com/THUDM/LongBench
📚 Arxiv Paper: https://arxiv.org/abs/2412.15204
LongBench v2 is designed to assess the ability of LLMs to handle long-context problems… See the full description on the dataset page: https://huggingface.co/datasets/Green-Sky/LongBench-v2-for-llama.cpp.llama2-TR-recipeLlama2-7B-data-course
Dataset Description
This dataset is designed to support a teaching assistance model for an introductory computer science course. It includes structured content such as course syllabi, lesson plans, lecture materials, and exercises related to topics such as computer fundamentals, algorithms, hardware, software, and IT technologies. The dataset integrates practical assignments, theoretical knowledge, and ethical education, aiming to enhance teaching efficiency and improve student… See the full description on the dataset page: https://huggingface.co/datasets/2imi9/Llama2-7B-data-course.llama2-jinyong-style
LLama2微调医学数据集
开源数据集存在与预训练模型数据集相似度较高的问题，导致微调结果不明显，效果不突出，因此就需要构建某种风格的专属数据集。
因此，构建了带金庸创作风格的医学数据集，可直接用于LLama2的微调使用。
微调实验结果证明，当输入医学问题时，微调模型回答的风格为金庸创作风格回答。
Instruction
Instruction: 我睡眠不足，很头痛
Output
少侠保重身体。江湖人称我为医仙，今日得闻君有头痛之症，实非善兆。睡眠不足，乃是身体之不适，需慎之又慎。头痛之症，或许源自于血液循环不畅，或许源于神经压力过大。当务之急，当调整生活习惯，保持良好的睡眠规律，避免过度劳累。此外，可尝试调整饮食，避免辛辣刺激之物，以免加重头痛之苦。如君仍遭此病痛，可寻求名医良药，以求解忧。</s>
可以发现，模型的回答已经学习到金庸创作风格的能力。
Devanagari-Ecommerce-fomatted-for-llama2-chat-Dataset
Dataset Card for Dataset Name
यो देवनागरी नेपाली भाषाको डेटासेट विशेषगरी च्याटबोट प्रणालीहरू बनाउनको लागि डिजाइन गरिएको हो। यसमा विभिन्न श्रेणीहरूको डेटासेटहरू समावेश गरिएको छ, जसलाई JSON मा ढाँचा बनाईएको छ, जसले नेपाली वार्तालाप एआई अनुप्रयोगहरूको लागि भाषा मोडेलहरूलाई तालिम र फाइन-ट्यून गर्नको लागि व्यापक स्रोत प्रदान गर्दछ।
This dataset card aims to be a base template for new datasets. It has been generated using this raw template.
Dataset Details… See the full description on the dataset page: https://huggingface.co/datasets/kshitizgajurel/Devanagari-Ecommerce-fomatted-for-llama2-chat-Dataset.marathi-alpaca-llama-finetune
Marathi Alpaca Dataset for llama-finetune
This dataset contains 48,897 high-quality Marathi instruction-following examples, converted to the llama-finetune format.
Format
Each line in the JSONL file contains:
{
"messages": [
{
"role": "user",
"content": "निरोगी राहण्यासाठी तीन टिपा द्या."
},
{
"role": "assistant",
"content": "1. संतुलित आणि पौष्टिक आहार घ्या..."
}
]
}
Usage
Download
from datasets import… See the full description on the dataset page: https://huggingface.co/datasets/aghatage/marathi-alpaca-llama-finetune.InnerILLM-Llama2-training-dataset
Inner I LLM Llama 2 Training Dataset
Overview
This dataset is designed for fine-tuning the Llama 2 model to explore, express, and expand upon concepts related to the True Self, the Inner 'I', the Impersonal 'I', 'I Am', and the singularity of human intelligence. The dataset aims to foster a deeper understanding and reflection on these themes, contributing to the development of an LLM that can engage in meaningful dialogues about self-awareness and consciousness.… See the full description on the dataset page: https://huggingface.co/datasets/InnerI/InnerILLM-Llama2-training-dataset.dolly-llama-qa
Dataset Card for dolly-llama-qa
This dataset has been created with dataformer.
Dataset Details
Dataset Description
The dolly-llama-qa dataset is a synthetic QA pair dataset created using the context from databricks-dolly-15k. We used Meta-Llama-3-8B-Instruct and Meta-Llama-3.1-8B-Instruct models for the generation and evolution part. Openai's gpt-4o was used for evaluating the refined questions and refined answers.
Dataset Columns
context:… See the full description on the dataset page: https://huggingface.co/datasets/dataformer/dolly-llama-qa.rouzhiba-llama3-tt