datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
alpaca-cleaned
Dataset Card for Alpaca-Cleaned
Repository: https://github.com/gururise/AlpacaDataCleaned
Dataset Description
This is a cleaned version of the original Alpaca Dataset released by Stanford. The following issues have been identified in the original release and fixed in this dataset:
Hallucinations: Many instructions in the original dataset had instructions referencing data on the internet, which just caused GPT3 to hallucinate an answer.
"instruction":"Summarize… See the full description on the dataset page: https://huggingface.co/datasets/yahma/alpaca-cleaned.alpaca-zh
Dataset Card for "alpaca-zh"
本数据集是参考Alpaca方法基于GPT4得到的self-instruct数据，约5万条。
Dataset from https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
It is the chinese dataset from https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM/blob/main/data/alpaca_gpt4_data_zh.json
Usage and License Notices
The data is intended and licensed for research use only. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not… See the full description on the dataset page: https://huggingface.co/datasets/shibing624/alpaca-zh.alpaca-data-gpt4-chineseTested-143k-Python-AlpacaContributors: Nicolas Mejia Petit
Vezora's CodeTester Dataset
Introduction
Today, on March 6, 2024, we are excited to release our internal Python dataset with 143,327 examples of code. These examples have been meticulously tested and verified as working. Our dataset was created using a script we developed.
Dataset Creation
Our script operates by extracting Python code from the output section of Alpaca-formatted datasets. It tests each extracted piece of code… See the full description on the dataset page: https://huggingface.co/datasets/Vezora/Tested-143k-Python-Alpaca.code_instructions_122k_alpaca_stylealpaca-cleaned
Dataset Card for Alpaca-Cleaned
Forked from https://huggingface.co/datasets/yahma/alpaca-cleaned
Repository: https://github.com/gururise/AlpacaDataCleaned
Dataset Description
This is a cleaned version of the original Alpaca Dataset released by Stanford. The following issues have been identified in the original release and fixed in this dataset:
Hallucinations: Many instructions in the original dataset had instructions referencing data on the internet, which just caused… See the full description on the dataset page: https://huggingface.co/datasets/unsloth/alpaca-cleaned.finance-alpacaThis dataset is a combination of Stanford's Alpaca (https://github.com/tatsu-lab/stanford_alpaca) and FiQA (https://sites.google.com/view/fiqa/) with another 1.3k pairs custom generated using GPT3.5
Script for tuning through Kaggle's (https://www.kaggle.com) free resources using PEFT/LoRa: https://www.kaggle.com/code/gbhacker23/wealth-alpaca-lora
GitHub repo with performance analyses, training and data generation scripts, and inference notebooks: https://github.com/gaurangbharti1/wealth-alpaca… See the full description on the dataset page: https://huggingface.co/datasets/gbharti/finance-alpaca.alpaca-gpt4-chineseThe dataset is used in the research related to MultilingualSIFT.
alpaca-gpt4-data-zh
Dataset Card for "alpaca-gpt4-data-zh"
All of the work is done by this team.
Usage and License Notices
The data is intended and licensed for research use only. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes.
English Dataset
Found here
Citation
@article{peng2023gpt4llm,
title={Instruction Tuning with GPT-4},
author={Baolin Peng, Chunyuan Li… See the full description on the dataset page: https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data-zh.Tested-22k-Python-AlpacaContributors: Nicolas Mejia Petit
Vezora's CodeTester Dataset
Introduction
Today, on November 2, 2023, we are excited to release our internal Python dataset with 22,600 examples of code. These examples have been meticulously tested and verified as working. Our dataset was created using a script we developed.
Dataset Creation
Our script operates by extracting Python code from the output section of Alpaca-formatted datasets. It tests each extracted piece of… See the full description on the dataset page: https://huggingface.co/datasets/Vezora/Tested-22k-Python-Alpaca.alpaca_gpt4_zhBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
Removed 6,103 mistruncated examples.
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_zh.
open-instruct-uncensored-alpacaOriginal dataset page from ehartford.
810,102 entries. Sourced from open-instruct-uncensored.jsonl.
Converted the jsonl to a json which can be loaded into something like LLaMa-LoRA-Tuner.
I've also included smaller datasets that includes less entries depending on how much memory you have to work with.
Each one is randomized before being converted, so each dataset is unique in order.
Count of each Dataset:
code_alpaca: 19991
unnatural_instructions: 68231
baize: 166096
self_instruct: 81512… See the full description on the dataset page: https://huggingface.co/datasets/xzuyn/open-instruct-uncensored-alpaca.mitre-stix-cve-exploitdb-dataset-alpaca-chatml-harmony
MITRE+NVD+ExploitDB Dataset (Alpaca/ChatML/Harmony)
A dataset for training AI assistants/agents on vulnerability analysis and pentesting Q&A. It is built by the pentestds pipeline, which fetches and merges data from MITRE CVE, NVD (CVSS enrichment), ExploitDB, and a small set of HuggingFace datasets. Provenance is recorded for every entry, and the pipeline emits Alpaca, ChatML, and Harmony JSONL files.
Dataset Summary
This dataset is designed for training AI agents to… See the full description on the dataset page: https://huggingface.co/datasets/jason-oneal/mitre-stix-cve-exploitdb-dataset-alpaca-chatml-harmony.alpaca_data_chinese_51kalpaca_zhBorrowed from: https://huggingface.co/datasets/hfl/alpaca_zh_51k
Removed some examples with empty output.
You can use it in LLaMA Factory by specifying dataset: alpaca_zh.
Turkish-AlpacaStanford alpaca turkish: Stanford Alpaca
alpaca_zh_51k
alpaca_data_zh_51k.json
中文Alpaca数据，包含51k个从ChatGPT (gpt-3.5-turbo)爬取的指令数据。
Chinese Alpaca dataset, containing 51k instruction data crawled from ChatGPT (gpt-3.5-turbo).
项目地址 / Project：https://github.com/ymcui/Chinese-LLaMA-Alpaca
alpaca_enBorrowed from: https://github.com/tatsu-lab/stanford_alpaca
Removed some erroneous examples.
You can use it in LLaMA Factory by specifying dataset: alpaca_en.
alpacaepoccode-alpaca-20k
Dataset Card for CodeAlpaca 20K
This dataset originates from the Code Alpaca repository.
The CodeAlpaca 20K dataset is specifically used for training code generation models.
Dataset Details
Dataset Description
Each sample is comprised of three columns: instruction, input and output.
Language(s): English
License: Apache-2.0 License
Dataset Sources
The code from the original repository was adopted to post it here.
Repository:… See the full description on the dataset page: https://huggingface.co/datasets/flwrlabs/code-alpaca-20k.alpaca_chinese_datasetThanks for hikariming.
This dataset was created by merging "翻译后的中文数据" and "其他中文问题补充" from hikariming/alpaca_chinese_dataset in a certain format.
alpaca_gpt4_enBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_en.
alpaca-gpt4-data
Dataset Card for "alpaca-gpt4-data"
All of the work is done by this team.
Usage and License Notices
The data is intended and licensed for research use only. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes.
Chinese Dataset
Found here
Citation
@article{peng2023gpt4llm,
title={Instruction Tuning with GPT-4},
author={Baolin Peng, Chunyuan Li… See the full description on the dataset page: https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data.alpaca-chat-smallalpaca-gpt4-indonesianThe dataset is used in the research related to MultilingualSIFT.
DeepSeek-R1-Distilled-Translate-en-zh_CN-39k-Alpaca-GPT4
DeepSeek R1 满血蒸馏英中翻译数据集 Alpaca GPT-4（带 CoT 版本）
本数据集是 @FradSer/DeepSeek-R1-Distilled-Translate-en-zh_CN-39k 的 Alpaca GPT-4 版本，专门用于微调语言模型的英中翻译任务。采用标准的指令微调格式，更适合直接用于 SFT（Supervised Fine-tuning）训练。
本项目主要基于以下工具完成数据处理和生成：
llm-tools: 用于大语言模型数据处理的工具集合
数据集概览
关键统计
总样本数：38,981
数据集结构
字段说明
features:
- name: instruction # 待翻译的英文文本
dtype: string
- name: input # 空字符串，保持与标准指令格式一致
dtype: string
- name: output #… See the full description on the dataset page: https://huggingface.co/datasets/FradSer/DeepSeek-R1-Distilled-Translate-en-zh_CN-39k-Alpaca-GPT4.Vietnamese-Multi-turn-Chat-Alpacaalpaca-gpt4-arabicThe dataset is used in the research related to MultilingualSIFT.
tigerbot-alpaca-zh-0.5mTigerbot 自有基于alpaca生成中文问答对
Usage
import datasets
ds_sft = datasets.load_dataset('TigerResearch/tigerbot-alpaca-zh-0.5m')
WizardLM_alpaca_evol_instruct_70k_unfilteredThis dataset is the WizardLM dataset victor123/evol_instruct_70k, removing instances of blatant alignment.
54974 instructions remain.
inspired by https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered
All credit to anon8231489123 for the cleanup script that I adapted to wizardlm_clean.py
license: apache-2.0
language:
- en
pretty_name: wizardlm-unfiltered