datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MSC-Self-Instruct
MemGPT
This is the self-instruct dataset of MSC conversations used for MemGPT paper. For more information please refer to memgpt.ai
The MSC dataset is a multi-round human conversations. In this dataset, our goal is to come up with a conversation opener, that is personalized to the user by referencing topics from the previous conversations.
These were generated while evaluating MemGPT.
self-instruct-safety-alignment[EMNLP 2024] Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models
🌐 Homepage | 📖 Paper | 🤗 Dataset (Data Advisor) | 🤗 Dataset (Self-Instruct)
Disclaimer
The dataset contains content that may be offensive or harmful. This dataset is intended for research purposes, specifically to support efforts aimed at creating safer and less harmful AI systems. Please engage with it responsibly and at your own risk.
Citation… See the full description on the dataset page: https://huggingface.co/datasets/fwnlp/self-instruct-safety-alignment.GPT-4-Self-Instruct-GermanHere we share a German dataset synthesized using the OpenAI GPT-4 model with Self-Instruct, utilizing some excess Azure credits. Please feel free to use it. All questions and answers are newly generated by GPT-4, without specialized verification, only simple filtering and strict semantic similarity control have been applied.
We hope that this will be helpful for fine-tuning open-source models for non-English languages, particularly German. This dataset will be updated continuously.
GPT-4-Self-Instruct-TurkishAs per the community's request, here we share a Turkish dataset synthesized using the OpenAI GPT-4 model with Self-Instruct, utilizing some excess Azure credits. Please feel free to use it. All questions and answers are newly generated by GPT-4, without specialized verification, only simple filtering and strict semantic similarity control have been applied.
We hope that this will be helpful for fine-tuning open-source models for non-English languages, particularly Turkish. This dataset will be… See the full description on the dataset page: https://huggingface.co/datasets/CausalLM/GPT-4-Self-Instruct-Turkish.self-instruct-base
Dataset Card for Dataset Name
Dataset Summary
This dataset is a copy of yizhongw's data from the github above, note this was created on 24th Jan 2023.
Dataset Structure
GPT3-finetuning format (prompt + completion)
Data Fields
Prompt
"Task: [Instruction] Output:"
Completion
"[Answer]<|endoftext|>"
Data Splits
No splits
Dataset Creation
Curation Rationale
Effeciently create a large dataset by using GPT3 to generate the… See the full description on the dataset page: https://huggingface.co/datasets/srinivasbilla/self-instruct-base.GPT-4-Self-Instruct-JapaneseHere we share a Japanese dataset synthesized using the OpenAI GPT-4 model with Self-Instruct, utilizing some excess Azure credits. Please feel free to use it. All questions and answers are newly generated by GPT-4, without specialized verification, only simple filtering and strict semantic similarity control have been applied.
We hope that this will be helpful for fine-tuning open-source models for non-English languages, particularly Japanese. This dataset will be updated continuously.
self-instruct-seedManually created seed dataset used in bootstrapping in the Self-instruct paper https://arxiv.org/abs/2212.10560. This is part of the instruction fine-tuning datasets.
self-instruct-evalro_sft_selfinstruct_gpt4
Dataset Description
selfinstruct-gpt4 represents a colecction of modular instruction finetune datasets generated by GPT-4.
Here we provide the Romanian translation of the selfinstruct-gpt4 dataset, translated with Systran.
This dataset is part of the instruction finetune protocol for Romanian LLMs proposed in "Vorbeşti Româneşte?" A Recipe to Train Powerful Romanian LLMs with English Instructions (Masala et al., 2024).
Citation… See the full description on the dataset page: https://huggingface.co/datasets/surogate/ro_sft_selfinstruct_gpt4.GPT-4-Self-Instruct-GreekAs per the community's request, here we share a Greek dataset synthesized using the OpenAI GPT-4 model with Self-Instruct, utilizing some excess Azure credits. Please feel free to use it. All questions and answers are newly generated by GPT-4, without specialized verification, only simple filtering and strict semantic similarity control have been applied.
We hope that this will be helpful for fine-tuning open-source models for non-English languages, particularly Greek. This dataset will be… See the full description on the dataset page: https://huggingface.co/datasets/CausalLM/GPT-4-Self-Instruct-Greek.self-instruct-data-concept-need-responsebigcode_self-oss-instruct-sc2-exec-filter-50k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
bigcode_self-oss-instruct-sc2-exec-filter-50k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
bigcode/self-oss-instruct-sc2-exec-filter-50k with responses regenerated with gemini-2.0-flash-thinking-exp-1219.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was skipped.
If ["candidates"][0]["safety_ratings"] == "SAFETY" the sample was skipped.
If ["candidates"][0]["finish_reason"] != 1 the sample was… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/bigcode_self-oss-instruct-sc2-exec-filter-50k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.self-instruct-seed-ca
Catalan self-instruct seed
Manual translation of the seed instructions from self-instruct.
Note that some examples could not be literally translated (e.g. jokes, puns, code) and had to be adapted to the target language.
Self-Discover-MM-InstructThis dataset was synthetically generated using the Mistral Medium model for a project I am currently developing. It draws inspiration from the Self-Discover framework outlined in a paper by Google Deepmind 1. While this implementation is a basic interpretation and does not fully capture the essence of the original framework, it resulted in a robust Instruct dataset that meets the project's objectives. Further details will be shared upon the project's release. Below is the Python code utilized… See the full description on the dataset page: https://huggingface.co/datasets/Crystalcareai/Self-Discover-MM-Instruct.Self-Discover-MM-Instruct-Alpacasft_data_manufacturing_self-instruct
Dataset Card for SFT_data_manufacturing_self-instruct
本データセットは、製造業における現場の課題解決（異常検知、保守履歴分析、予防保全計画の最適化など）を目的として、Self-Instructを用いて生成されたSFT (Supervised Fine-Tuning)データセットです。
製造現場特有の専門的なデータ（センサーログ、保守履歴、停止データ）をAIがどのように解釈し、論理的な推論に基づいて対策立案まで行えるかを検証・学習するために設計されました。
Dataset Details
Dataset Description
本データセットは、製造業における現場の疑問や専門的な指示に対する回答を学習させるために作成された、100件の質問応答データです。
各データはJSONL形式で以下の構造を持っています。
フィールド名
データ型
内容の説明
instruction
String
現場における具体的なタスク指示。… See the full description on the dataset page: https://huggingface.co/datasets/Podtech/sft_data_manufacturing_self-instruct.ro_sft_selfinstruct_gpt4
Dataset Description
selfinstruct-gpt4 represents a colecction of modular instruction finetune datasets generated by GPT-4.
Here we provide the Romanian translation of the selfinstruct-gpt4 dataset, translated with Systran.
This dataset is part of the instruction finetune protocol for Romanian LLMs proposed in "Vorbeşti Româneşte?" A Recipe to Train Powerful Romanian LLMs with English Instructions (Masala et al., 2024).
Citation… See the full description on the dataset page: https://huggingface.co/datasets/OpenLLM-Ro/ro_sft_selfinstruct_gpt4.MSC-Self-Instruct
MemGPT
This is the self-instruct dataset of MSC conversations used for MemGPT paper. For more information please refer to memgpt.ai
The MSC dataset is a multi-round human conversations. In this dataset, our goal is to come up with a conversation opener, that is personalized to the user by referencing topics from the previous conversations.
These were generated while evaluating MemGPT.
self-oss-instruct-50k-sharegpthttps://huggingface.co/datasets/bigcode/self-oss-instruct-sc2-exec-filter-50k converted to the sharegpt format with all the extra info stripped out
license is odc-by same as the original dataset
all interactions are single turn, so you should probably augment this with a dataset such as code-feedback for multi-turn abilities
self-instruct-seedManually created seed dataset used in bootstrapping in the Self-instruct paper https://arxiv.org/abs/2212.10560. This is part of the instruction fine-tuning datasets.
self-generated-instructions-plThis repository contains combined and a little bit cleaned set of 104527 instructions from https://huggingface.co/datasets/chrisociepa/raw-self-generated-instructions-pl and https://huggingface.co/datasets/mmosiolek/pl_alpaca_data_cleaned
Please check corresponding repositories to find more details.
tulu_self_instructindic-self-instruct-gemma
Indic Self-Instruct Dataset
This dataset is generated using Sarvam-30B as a teacher model to distill knowledge into smaller models like Gemma.
Tasks
Translation
Summarization
Code-mixed language
Politeness rewriting
Languages
Telugu
Tamil
Hindi
Hinglish
self-instruct-unfiltered-dedupedThis dataset is https://github.com/yizhongw/self-instruct unfiltered and deduped, removing 1600 instances of blatant alignment and 26 duplicates.
80813 instructions remain.
clean.py was first ran on https://github.com/yizhongw/self-instruct/blob/0b26ccaa415992100fa32df62d41b994cf928e23/data/gpt3_generations/batch_221203/all_instances_82K.jsonl and then dedupe.py was ran on it, renamed to .json not .jsonl
inspired by… See the full description on the dataset page: https://huggingface.co/datasets/ewof/self-instruct-unfiltered-deduped.raw-self-generated-instructions-plThis repository contains generated instructions with GPT-3.5-Turbo in Polish.
The dataset has been used to finetune alpaca-lora-7b-pl.
uz.self-instructReST-MCTS_Mistral-MetaMATH-7b-Instruct_Self-Rewarding-DPO_2ndqwen2-self-align-instructReST-MCTS_Llama3-8b-Instruct_Self-Rewarding-DPO_2ndReST-MCTS_Mistral-MetaMATH-7b-Instruct_Self-Rewarding-DPO_1st