datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ShareGPT4Video
ShareGPT4Video 4.8M Dataset Card
Dataset details
Dataset type:
ShareGPT4Video Captions 4.8M is a set of GPT4-Vision-powered multi-modal captions data of videos.
It is constructed to enhance modality alignment and fine-grained visual concept perception in Large Video-Language Models (LVLMs) and Text-to-Video Models (T2VMs). This advancement aims to bring LVLMs and T2VMs towards the capabilities of GPT4V and Sora.
sharegpt4video_40k.jsonl is generated by GPT4-Vision… See the full description on the dataset page: https://huggingface.co/datasets/ShareGPT4Video/ShareGPT4Video.ShareGPT-4otrain_video_and_instruction
ShareGPTVideo Training Data
All dataset and models can be found at ShareGPTVideo.
Contents:
Train 300k video frames: contains video frames used for SFT and DPO model, which is a subset of total 900k.
ActivityNet 50k + vidal 150k + webvid 100k.
Train 600k video frames: contains the rest 600k frames, the total 900k frames are used for pre-training stage. If you just do finetuning using our video QA, you can just download the 300k above.
900k composition is 400k WebVid +… See the full description on the dataset page: https://huggingface.co/datasets/ShareGPTVideo/train_video_and_instruction.ShareGPT-Chinese-English-90k
ShareGPT-Chinese-English-90k Bilingual Human-Machine QA Dataset
A high-quality Chinese-English parallel bilingual human-machine QA dataset, covering user questions in real and complex scenarios. It is used for training high-quality dialogue models (more robust in instruction distribution than those datasets generated by repeatedly calling API interfaces to simulate machine-generated Q&A, like Moss)
Features:
Provides fully semantically equivalent Chinese-English parallel corpus… See the full description on the dataset page: https://huggingface.co/datasets/shareAI/ShareGPT-Chinese-English-90k.ShareGPT4V
News
[2024/5/8] We released ShareGPT4Video, a large-scale video-caption dataset, with 40K captions annotated by GPT4V and 4.8M captions annotated by our ShareCaptioner-Video. The total videos last with 300 hours and 3000 hours separately!
ShareGPT4V 1.2M Dataset Card
Dataset details
Dataset type:
ShareGPT4V Captions 1.2M is a set of GPT4-Vision-powered multi-modal captions data.
It is constructed to enhance modality alignment and fine-grained visual concept… See the full description on the dataset page: https://huggingface.co/datasets/Lin-Chen/ShareGPT4V.MT-SFT-ShareGPT
MT-SFT-ShareGPT
💻 Github Repo • 🤗 HuggingFace • 🤖 ModelScope
Introduction
Data has always been an important part of advancing large language models forward. Based on this, we have collected dozens of high-quality open source datasets from the open source community, with a total data volume of 20 M.
After some cleaning actions, we have open sourced a set of high-quality datasets for fine-tuning the instructions of the… See the full description on the dataset page: https://huggingface.co/datasets/thomas-yanxin/MT-SFT-ShareGPT.OpenMathReasoning_ShareGPTOriginal README:
OpenMathReasoning
OpenMathReasoning is a large-scale math reasoning dataset for training large language models (LLMs).
This dataset contains
540K unique mathematical problems sourced from AoPS forums,
3.2M long chain-of-thought (CoT) solutions
1.7M long tool-integrated reasoning (TIR) solutions
566K samples that select the most promising solution out of many candidates (GenSelect)
We used Qwen2.5-32B-Instruct to preprocess problems, and
DeepSeek-R1 and QwQ-32B… See the full description on the dataset page: https://huggingface.co/datasets/MaziyarPanahi/OpenMathReasoning_ShareGPT.train_raw_video
ShareGPTVideo Raw ActivityNet Videos for Train data
All dataset and models can be found at ShareGPTVideo.
Contents:
Due to our scene split, we provide our processed activityNet videos corresponding to test frames in
train video frames
the processing script is process_activitynet.py
German-RAG-SFT-ShareGPT-HESSIAN-AI
German-RAG-SFT (Supervised Fine-Tuning) Share-GPT Format
German-RAG - German Retrieval Augmented Generation
Dataset Summary
The SFT Tasks Dataset represents a specialized collection for fine-tuning language models with a focus on RAG-specific capabilities. Most tasks were developed using synthetically enhanced data derived from the German Wikipedia, accessed through Cohere's dataset (wikipedia-22-12-de-embeddings). The data is structured in a training knowledge… See the full description on the dataset page: https://huggingface.co/datasets/avemio/German-RAG-SFT-ShareGPT-HESSIAN-AI.lihuowang-sharegpt开源项目地址：https://github.com/wangerzi/lihuowang
数据集仓库：
huggingface 地址
ModelScope 地址
Paddle 飞桨地址
数据集名称
描述
lihuowang-alpaca-dpo.json
【推荐】使用 Alpaca 格式的 DPO 数据集，包含李火旺的疯言疯语回答和正常回答的对比
daoguiyixian-sharegpt-summary-v2.json
【推荐】ShareGPT 格式，章节摘要的 QA 第二版，优化了问题质量和回答准确性
daoguiyixian-sharegpt-qa-v2.json
【推荐】2W 条数据，ShareGPT 格式，针对章节内容的细致问答第二版，增加了更多细节问题和更准确的回答
daoguiyixian-summary-v2.json
各章节的摘要总结第二版，优化了摘要的连贯性和关键事件的覆盖度
daoguiyixian-sharegpt-summary.jsonShareGPT 格式，章节摘要的 QA… See the full description on the dataset page: https://huggingface.co/datasets/wj2015/lihuowang-sharegpt.German-RAG-ORPO-ShareGPT-HESSIAN-AI
German-RAG-ORPO (Odds Ratio Preference Optimization) ShareGPT-Format
German-RAG - German Retrieval Augmented Generation
Dataset Summary
The ORPO Tasks Dataset represents a specialized collection for fine-tuning language models with a focus on RAG-specific capabilities.
The subsets can be for this training step are derived from 3 different sources:
SauerkrautLM Preference Datasets:
SauerkrautLM-Fermented-GER-DPO: is a specialized dataset designed for training… See the full description on the dataset page: https://huggingface.co/datasets/avemio/German-RAG-ORPO-ShareGPT-HESSIAN-AI.test_video_and_instruction
ShareGPTVideo Testing Data
All dataset and models can be found at ShareGPTVideo.
Contents:
Test video frames: contains video frames used for testing
In-domain: WebVid, Vidal (Youtube shorts), ActivityNet
Out-of-domain: MSRVTT, MSVD, TGIF, SSV2
Testing pipeline refer to LLaVA-Hound-DPO test
Inference and results: result, replying to issue
DPO ranking results: dpo ranking
Set up:
git clone git@github.com:RifleZhang/LLaVA-Hound-DPO.git
source setup/setup_env.sh
source… See the full description on the dataset page: https://huggingface.co/datasets/ShareGPTVideo/test_video_and_instruction.German-RAG-DPO-ShareGPT-HESSIAN-AI
German-RAG-DPO Share-GPT Format
German-RAG - German Retrieval Augmented Generation
Dataset Summary
The DPO Tasks Dataset represents a specialized collection for fine-tuning language models with a focus on RAG-specific capabilities. Most tasks were developed using synthetically enhanced data derived from the German Wikipedia, accessed through Cohere's dataset (wikipedia-22-12-de-embeddings). The data is structured in a training knowledge graph where… See the full description on the dataset page: https://huggingface.co/datasets/avemio/German-RAG-DPO-ShareGPT-HESSIAN-AI.Vietnamese-ShareGPT4Video-ShareGPT4Video-gg-translatedVietnamese-ShareGPT4Vision-gg-translatedtest_raw_video_data
ShareGPTVideo Raw Videos for Testing data
All dataset and models can be found at ShareGPTVideo.
Contents:
In case of need, this contains raw videos corresponding to test frames in
Test video frames
python18k_instruct_sharegpt
Note:
This dataset builds upon the iamtarun/python_code_instructions_18k_alpaca dataset and adheres to the ShareGPT format with a unique “conversations” column containing messages in JSONL. Unlike simpler formats like Alpaca, ShareGPT is ideal for storing multi-turn conversations, which is closer to how users interact with LLMs.
Example:
from datasets import load_dataset
dataset = load_dataset("AnelMusic/python18k_instruct_sharegpt", split = "train")
def… See the full description on the dataset page: https://huggingface.co/datasets/AnelMusic/python18k_instruct_sharegpt.German-RAG-ORPO-Long-Context-ShareGPT-HESSIAN-AI
German-RAG-ORPO (Odds Ratio Preference Optimization) Long Context ShareGPT-Format
German-RAG - German Retrieval Augmented Generation
Dataset Summary
The ORPO Long Context Tasks Dataset represents a specialized collection for fine-tuning language models with a focus on RAG-specific capabilities.
The subsets are derived from Synthetic generation inspired by Tencent's (“Scaling Synthetic Data Creation with 1,000,000,000 Personas”).
Dataset Structure… See the full description on the dataset page: https://huggingface.co/datasets/avemio/German-RAG-ORPO-Long-Context-ShareGPT-HESSIAN-AI.Vietnamese-OpenGVLab-ShareGPT-4o-gg-translatedsharegpt_cot_dataset
A data set inspired by the "Reflection" method, three-dimensional thinking and cot
This is the ShareGPT format.
The data set was generated using multiple llm synthesis.
tw-sharegpt
Dataset Card for tw-sharegpt
本資料集彙整來自使用者在 🪣 data-contribute 主動上傳、並經過授權同意開放分享的 GPT 對話資料。資料內容採用 JSONL 格式儲存，每筆包含對話內容（messages）及基本上傳 metadata。本資料集旨在提供高品質繁體中文對話語料，以支援語言模型訓練、研究與評估。
歡迎貢獻你個人的對話紀錄 🤗
Dataset Details
Dataset Description
本資料集為一個持續更新（continual collection）的繁體中文人機對話語料庫，由真實使用者在本平台上傳其與 GPT… See the full description on the dataset page: https://huggingface.co/datasets/lianghsun/tw-sharegpt.ShareGPT4Video
ShareGPT4Video 4.8M Dataset Card
Dataset details
Dataset type:
ShareGPT4Video Captions 4.8M is a set of GPT4-Vision-powered multi-modal captions data of videos.
It is constructed to enhance modality alignment and fine-grained visual concept perception in Large Video-Language Models (LVLMs) and Text-to-Video Models (T2VMs). This advancement aims to bring LVLMs and T2VMs towards capabilities of GPT4V and Sora.
sharegpt4video_40k.jsonl is generated by GPT4-Vision… See the full description on the dataset page: https://huggingface.co/datasets/lodestone-horizon/ShareGPT4Video.Reddit-SFW-Writing_Prompts_ShareGPT_Curated
Normalized SFW Reddit Writing Prompts
Dataset Description
This dataset is a normalized, flattened version of curated Reddit writing prompts, specifically derived from ChaoticNeutrals/Reddit-SFW-Writing_Prompts_ShareGPT. It maps nested conversational arrays into a strict instruction-response schema, making it highly optimized for instruction-tuning Large Language Models.
Dataset Schema
Column Name
Type
Description
prompt
string
The input prompt, user… See the full description on the dataset page: https://huggingface.co/datasets/rafy2342/Reddit-SFW-Writing_Prompts_ShareGPT_Curated.ShareGPT4V-PT-ru
ShareGPT4V-PT-ru
This is translated version of ShareGPT4V-PT(1.25M) subset of Lin-Chen/ShareGPT4V dataset into Russian.
capybara-sharegpt
capybara-sharegpt
LDJnr/Capybara converted to ShareGPT format for use in common training repositories.
Please refer to the original repository's dataset card for more information. All credit goes to the original creator.
ShareGPT4V-ru
ShareGPT4V-ru
This is translated version ShareGPT4V(102k) subset of Lin-Chen/ShareGPT4V dataset into Russian.
medical_meadow_wikidoc_sharegptGoogle Collab Notebook:- https://colab.research.google.com/drive/1XL5jZaLCZFFfI9LaxQ3zP66YbX41ZMnn#scrollTo=3f4lR3Rndf3D
guanaco-sharegpt-style-serbian
Guanaco Sharegpt-style Serbian
Dataset Description
This dataset is a Serbian-translated version of the philschmid/guanaco-sharegpt-style
Dataset Structure
Usage
To load the dataset in Serbian, run:
from datasets import load_dataset
ds = load_dataset("datatab/guanaco-sharegpt-style-serbian")
Data Splits
The dataset has one splits, suitable for:
Supervised fine-tuning (sft).
The dataset is stored in parquet format with each entry using… See the full description on the dataset page: https://huggingface.co/datasets/datatab/guanaco-sharegpt-style-serbian.ShareGPT4V-hin
Dataset details
It is constructed to enhance modality alignment and fine-grained visual concept perception in Large Multi-Modal Models (LMMs) during both the pre-training and supervised fine-tuning stages. This advancement aims to bring LMMs towards GPT4-Vision capabilities.
sharegpt4v_instruct_gpt4-vision_cap100k.json is generated by GPT4-Vision (ShareGPT4V).
This dataset is Hindi-translated version of the ShareGPT4V
This dataset is intended only for Fine-tuning
The images can be… See the full description on the dataset page: https://huggingface.co/datasets/damerajee/ShareGPT4V-hin.Alpaca_ShareGPT_10G
Dataset Description
This dataset consists of 10GB of open-source bilingual data (Chinese and English), sourced from platforms such as Hugging Face. The data covers a wide range of topics, with an emphasis on multi-round conversational logic and reasoning. It includes both general and technical question-answer pairs, making it ideal for training AI models that need to handle extended conversations and maintain context across multiple exchanges.
The dataset is designed to improve the… See the full description on the dataset page: https://huggingface.co/datasets/2imi9/Alpaca_ShareGPT_10G.