datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
alpaca-data-gpt4-chinesesharegpt_gpt4
Dataset Card
Dataset Summary
ShareGPT中挑选出的GPT4多轮问答数据，多语言问答。
Languages
数据集是多语言，包括中文、英文、日文等常用语言。
Dataset Structure
Data Fields
The data fields are the same among all splits.
conversations: a List of string .
head -n 1 sharegpt_gpt4.jsonl
{"conversations":[
{'from': 'human',
'value': '採用優雅現代中文，用中文繁體字型，回答以下問題。為所有標題或專用字詞提供對應的英語翻譯：Using scholarly style, summarize in detail James Barr\'s book "Semantics of Biblical Language". Provide… See the full description on the dataset page: https://huggingface.co/datasets/shibing624/sharegpt_gpt4.alpaca-gpt4-chineseThe dataset is used in the research related to MultilingualSIFT.
alpaca-gpt4-data-zh
Dataset Card for "alpaca-gpt4-data-zh"
All of the work is done by this team.
Usage and License Notices
The data is intended and licensed for research use only. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes.
English Dataset
Found here
Citation
@article{peng2023gpt4llm,
title={Instruction Tuning with GPT-4},
author={Baolin Peng, Chunyuan Li… See the full description on the dataset page: https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data-zh.alpaca_gpt4_zhBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
Removed 6,103 mistruncated examples.
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_zh.
gpt-4o-toksuite-detokenizedTraining data of the model detokenized in the exact order seen by the model.
The training data is partitioned into 8 chunks (chunk-0 through chunk-7), based on the GPU rank that generated the data. Each chunk contains detokenized text files in JSON Lines format (.jsonl).
ruozhiba_gpt4
ruozhiba_gpt4
本仓库包含使用GPT-4（4T/4o）构建的ruozhiba指令数据[^1]，共计2449条。其中包含以下两个版本，题目相同，仅回答内容不同。
ruozhiba_qa2449_gpt4t.json：利用gpt-4-turbo-20240409 对问题进行了回答。
ruozhiba_qa2449_gpt4o.json：利用gpt-4o-20240514 对问题进行了回答。
注意：指令数据中可能包含冒犯用语。
所属项目
Chinese-LLaMA-Alpaca-3：https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
This repository contains the ruozhiba instruction data[^1] constructed using GPT-4 (4T/4o), totaling 2449 entries. It includes the following two versions with the same questions… See the full description on the dataset page: https://huggingface.co/datasets/hfl/ruozhiba_gpt4.200k_HEAVY_gpt4o-description-gpt4omini-code_generated_problemsHere is the dataset of ~100k synthetic data generated by 162 seeds.
We generate the dataset with the following steps and two approaches:
Generate ~110k descriptions by GPT4o.
Approach 1: Generate ~110k codes follow each description by GPT4o-mini.
Approach 2: Generate ~110k codes follow each description by GPT4o-mini and suggest it to use specific library functions.
Run the ~220k codes and do auto-filtering.
Get the final ~200k legitimate ARC-like tasks with examples.
GPT4-LLM-CleanedThis is the GPT4-LLM dataset from : https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
It has been filtered of all OpenAI disclaimers and refusals. (Disclaimer: It may have removed some additional things besides just OAI disclaimers, as I used the followings script which is a bit more broad: https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered/blob/main/wizardlm_clean.py)
There is a modified script of that in the repo that was used specifically for… See the full description on the dataset page: https://huggingface.co/datasets/teknium/GPT4-LLM-Cleaned.roleplay-zh-sharegpt-gpt4-data
roleplay 数据集
数据
我们有4个数据集文件:
"sharegpt_formatted_data-evol-gpt4.jsonl" 来自 bai-roleplay/evol-character-entire 将其转换为sharegpt格式。
"sharegpt_formatted_data-evol-gpt35.jsonl" 来自 bai-roleplay/evol-character-entire 将其转换为sharegpt格式。
"sharegpt_formatted_data-evol-male-gpt35.jsonl" 来自 bai-roleplay/evol-character-entire 将其转换为sharegpt格式。
"sharegpt_formatted_data-roleplay-chat-1k.jsonl" 来自 Minami-su/roleplay_multiturn_chat_1k_zh_v0.1 将其转换为sharegpt格式。… See the full description on the dataset page: https://huggingface.co/datasets/shibing624/roleplay-zh-sharegpt-gpt4-data.lightblue-tagengo-gpt4
lightblue/tagengo-gpt4
An unofficial, reformatted version of lightblue/tagengo-gpt4.
Tagengo is described by its author as the world's largest high-quality multilingual chat dataset - containing over 75,000 single-turn conversations between humans and GPT‑4 (gpt-4-0125-preview) across 74 languages. It fills a major gap in multilingual chat data, which has so far been limited compared to English.
Additional Processing
Split by language
Kept only entries with both… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/lightblue-tagengo-gpt4.alpaca_gpt4_enBorrowed from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
You can use it in LLaMA Factory by specifying dataset: alpaca_gpt4_en.
MMInstruct-GPT4V
MMInstruct
The official implementation of the paper "MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity".
The data engine is available on GitHub at yuecao0119/MMInstruct.
Todo List
Data Engine.
Open Source Datasets.
Release the checkpoint.
Introduction
Vision-language supervised fine-tuning effectively enhances VLLM performance, but existing visual instruction tuning datasets have limitations:
Instruction Annotation… See the full description on the dataset page: https://huggingface.co/datasets/yuecao0119/MMInstruct-GPT4V.alpaca-gpt4-data
Dataset Card for "alpaca-gpt4-data"
All of the work is done by this team.
Usage and License Notices
The data is intended and licensed for research use only. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes.
Chinese Dataset
Found here
Citation
@article{peng2023gpt4llm,
title={Instruction Tuning with GPT-4},
author={Baolin Peng, Chunyuan Li… See the full description on the dataset page: https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data.alpaca-gpt4-indonesianThe dataset is used in the research related to MultilingualSIFT.
gpt4_datasetalpaca-gpt4-arabicThe dataset is used in the research related to MultilingualSIFT.
DeepSeek-R1-Distilled-Translate-en-zh_CN-39k-Alpaca-GPT4
DeepSeek R1 满血蒸馏英中翻译数据集 Alpaca GPT-4（带 CoT 版本）
本数据集是 @FradSer/DeepSeek-R1-Distilled-Translate-en-zh_CN-39k 的 Alpaca GPT-4 版本，专门用于微调语言模型的英中翻译任务。采用标准的指令微调格式，更适合直接用于 SFT（Supervised Fine-tuning）训练。
本项目主要基于以下工具完成数据处理和生成：
llm-tools: 用于大语言模型数据处理的工具集合
数据集概览
关键统计
总样本数：38,981
数据集结构
字段说明
features:
- name: instruction # 待翻译的英文文本
dtype: string
- name: input # 空字符串，保持与标准指令格式一致
dtype: string
- name: output #… See the full description on the dataset page: https://huggingface.co/datasets/FradSer/DeepSeek-R1-Distilled-Translate-en-zh_CN-39k-Alpaca-GPT4.Evol-Instruct-Chinese-GPT4The dataset is created by (1) translating English questions of Evol-instruct-70k into Chinese and (2) requesting GPT4 to generate Chinese responses.
For more details, please refer to:
Repository:
https://github.com/FreedomIntelligence/AceGPT
https://github.com/FreedomIntelligence/LLMZoo
Paper:
AceGPT, Localizing Large Language Models in Arabic
Phoenix: Democratizing ChatGPT across Languages
BibTeX entry and citation info
@article{huang2023acegpt,
title={AceGPT, Localizing… See the full description on the dataset page: https://huggingface.co/datasets/FreedomIntelligence/Evol-Instruct-Chinese-GPT4.100k-gpt4omini-description-gpt4omini-code_generated_problemsHere is the dataset of 100k synthetic data generated by 100 seeds.
We generate the dataset with the following steps:
Generate 120k descriptions by GPT4o-mini.
Generate 120k codes follow each description by GPT4o-mini.
Run the 120k codes and do auto-filtering.
Get the final 100k legitimate ARC-like tasks with examples.
Vietnamese-alpaca-gpt4-gg-translatedalpaca-gpt4-spanishThe dataset is used in the research related to MultilingualSIFT.
alpaca-gpt4-japaneseThe dataset is used in the research related to MultilingualSIFT.
gpt4v-raw-chunksalpaca-gpt4-deutschThe dataset is used in the research related to MultilingualSIFT.
alpaca-gpt4-hindiThe dataset is used in the research related to MultilingualSIFT.
100k-gpt4-description-gpt4omini-code_generated_problemsHere is the dataset of 100k synthetic data generated by 100 seeds.
We generate the dataset with the following steps:
Generate 120k descriptions by GPT4.
Generate 120k codes follow each description by GPT4o-mini.
Run the 120k codes and do auto-filtering.
Get the final 100k legitimate ARC-like tasks with examples.
gpt-4o-distilDPO formatted dataset for distilling GPT-4o-like behavior into other models. If you're new to HuggingFace and looking for a 4o replacement, this is probably not what you want. This is just a dataset, not a trained model. For models you can actually chat with, check out this collection of models trained on this dataset.
alpaca-gpt4-frenchThe dataset is used in the research related to MultilingualSIFT.
StrategyQA_CoT_GPT4o