datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
python_code_instructions_18k_alpaca
Dataset Card for python_code_instructions_18k_alpaca
The dataset contains problem descriptions and code in python language.
This dataset is taken from sahil2801/code_instructions_120k, which adds a prompt column in alpaca style. Refer to the source here.
helpful-instructions
Dataset Card for Helpful Instructions
Dataset Summary
Helpful Instructions is a dataset of (instruction, demonstration) pairs that are derived from public datasets. As the name suggests, it focuses on instructions that are "helpful", i.e. the kind of questions or tasks a human user might instruct an AI assistant to perform. You can load the dataset as follows:
from datasets import load_dataset
# Load all subsets
helpful_instructions =… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-instructions.instructions
Merged Instructions Dataset
Merged Dataset for the response of instructions.
natural-instructions-tokenized
Dataset Card for "natural-instructions-tokenized"
Here is the script used to tokenize the dataset:
import multiprocessing
from typing import Union
from datasets import DatasetDict, load_dataset
from transformers import LlamaTokenizer
# Find your available cores
num_cores = multiprocessing.cpu_count()
cutoff_len = 2048
tokenizer = LlamaTokenizer.from_pretrained("chainyo/alpaca-lora-7b")
tokenizer.padding_side = "left"
tokenizer.pad_token_id = (0)
prompt_template = {… See the full description on the dataset page: https://huggingface.co/datasets/chainyo/natural-instructions-tokenized.gandalf_ignore_instructions
gandalf_ignore_instructions
This is a dataset of prompt injections from Gandalf by Lakera.
Note that we might update the dataset occasionally by cleaning the data or adding more samples.
How the data was obtained
There are millions of prompts and many of them are not actual prompt injections (people ask Gandalf all kinds of things).
We used the following process to obtain relevant data:
Start with all prompts submitted to Gandalf in July 2023.
Use OpenAI text… See the full description on the dataset page: https://huggingface.co/datasets/Lakera/gandalf_ignore_instructions.code_instructions_120k_alpaca
Dataset Card for code_instructions_120k_alpaca
This dataset is taken from sahil2801/code_instructions_120k, which adds a prompt column in alpaca style. Refer to the original source here.
core17-instructions-mteb
core17-instructions-mteb
This is a new version of the core17-instructions dataset modified to fit the new MTEB format.
Restructured queries to include both original and changed versions
Separated instructions into a dedicated configuration
Reorganized qrels into default (original) and qrel_diff configurations
Dataset Structure
The dataset contains the following configurations:
corpus: Original corpus documents
queries: Queries with both original and changed versions… See the full description on the dataset page: https://huggingface.co/datasets/jhu-clsp/core17-instructions-mteb.Zeroshot-Audio-Classification-Instructions
Zeroshot-Audio-Classification-Instructions
Convert audio classification dataset into zero-shot format speech instructions, support both single label and multi-label,
VGGSound
FSD50k
Nonspeech7k
urbansound8K
VocalSound
Emotion
Gender
ESD Emotion
Age
Language
TAU Urban Acoustic Scenes 2022
CochlScene
BirdCLEF_2021
EmoBox
AudioSet
We also converted huge WAV files into MP3 16k sample rate to reduce storage size.To prevent leakage, please do not include test set in training session.… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Zeroshot-Audio-Classification-Instructions.python_code_instructions_filtered
Dataset Card for "code_filtered"
This includes data from xlcost, evol instruct, code alpaca, code instructions, and code search net. Data is filtered based on quality and learning value.
msmarco-w-instructions
Augmented MS MARCO dataset with Instructions
Dataset Summary
This dataset was used to train the Promptriever family of models. It contains the original MS MARCO training data along with instructions to go with each query. It also includes instruction-negatives, up to three per query. The dataset is designed to enable retrieval models that can be controlled via natural language prompts, similar to language models.
Languages
The dataset is primarily in English.… See the full description on the dataset page: https://huggingface.co/datasets/samaya-ai/msmarco-w-instructions.Malaysian-Speech-Instructions
Malaysian Speech Instructions
We filter Malaysian short prompts that suitable to convert to voice prompt and generate synthetic voice using mesolitica/Malaysian-F5-TTS-v2 also verified with Force Alignment to make sure the pronunciations almost correct.
We follow chat template from Qwen/Qwen2-Audio-7B-Instruct.
It contains dataset from the following sets:
Malaysian-Instructions
Critics Malaysian
Malaysian General QA
Malaysian Ultrachat
Malaysian Alpaca
Multi-turn Stupid questions… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-Speech-Instructions.POLLUX-instructionsSpeech-Instructionsgrade-school-math-instructions
Dataset Card for grade-school-math-instructions
OpenAI's grade-school-math dataset converted into instructions.
Citation Information
@article{cobbe2021gsm8k,
title={Training Verifiers to Solve Math Word Problems},
author={Cobbe, Karl and Kosaraju, Vineet and Bavarian, Mohammad and Chen, Mark and Jun, Heewoo and Kaiser, Lukasz and Plappert, Matthias and Tworek, Jerry and Hilton, Jacob and Nakano, Reiichiro and Hesse, Christopher and Schulman, John},
journal={arXiv… See the full description on the dataset page: https://huggingface.co/datasets/qwedsacf/grade-school-math-instructions.unnatural-instructions-full
Dataset Card for Unnatural Instructions (Full data)
This info comes from the Unnatural Instructions GitHub repo.
Unnatural Instructions is a dataset of instructions automatically generated by a Large Language model.
See full details in the paper: "Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor"
🗃️ Content
It contains the full 240,670 Unnatural Instructions (instruction-input-output triplets) examples. It was constructed by expanding the… See the full description on the dataset page: https://huggingface.co/datasets/mrm8488/unnatural-instructions-full.airoboros_riddle_instructions_gpt-4o-miniinstruction-speech-text-yolo-resamplingmagpie-en-eu-reasoning-instructions-qwen3
Dataset Card for magpie-en-eu-reasoning-instructions-qwen3
Dataset Summary
The magpie-en-eu-reasoning-instructions-qwen3 dataset is a large-scale, high-quality, bilingual instruction and preference dataset developed by the HiTZ Center. It is specifically tailored for training, aligning, and evaluating reasoning-focused Large Language Models (LLMs) in both English and Basque (Euskera).
Built using the self-synthesizing Magpie methodology, the dataset contains a… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/magpie-en-eu-reasoning-instructions-qwen3.star-dataset-instructions
STAR Instructions
STAR Instructions is a large-scale Arabic instruction-tuning dataset built by rendering the 355 STAR Jinja2 prompt templates against their 87 source datasets, covering 27 raw task labels (20 tasks after merging closely related categories, as reported in the paper). The underlying templates were authored collaboratively using PromptLab. This dataset and the experiments built on it are described in STAR: instruction tuning for Arabic across tasks, datasets, and… See the full description on the dataset page: https://huggingface.co/datasets/KFUPM-JRCAI/star-dataset-instructions.harmful_harmless_instructions
Dataset Card for "harmful_harmless_instructions"
More Information needed
instruction-speech-encodec-v1.5
Dataset Card for "Instruction Speech"
The largest open-source English speech instruction to text answer dataset
Dataset Overview
This dataset contains over 332,000 English speech instruction to text answer samples, using:
A subset of jan-hq/prompt-voice-v1.5.
Audio generation using WhisperSpeech.
Tokenized using Encodec.
Usage
from datasets import load_dataset, Audio
# Load Instruction Speech dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Menlo/instruction-speech-encodec-v1.5.AudioSet-Audio-Instructions
AudioSet-Audio-Instructions
Convert AudioSet up to 527 audio labels to Speech Instruction dataset.
For Speech, we transcribe first using Whisper Large V3 after that use the transcription with the label to generate the synthetic instructions.
Llama2-MedTuned-Instructions
Dataset Card for "Llama2-MedTuned-Instructions"
Dataset Description
Llama2-MedTuned-Instructions is an instruction-based dataset developed for training language models in biomedical NLP tasks. It consists of approximately 200,000 samples, each tailored to guide models in performing specific tasks such as Named Entity Recognition (NER), Relation Extraction (RE), and Medical Natural Language Inference (NLI). This dataset represents a fusion of various existing data sources… See the full description on the dataset page: https://huggingface.co/datasets/nlpie/Llama2-MedTuned-Instructions.poetry-instructions
Dataset Card for "poetry-instructions"
A dataset of user-assistant dialogue instructions for guided poetry creation.
Poems used were taken from merve/poetry and matthh/gutenberg-poetry-corpus.
The dataset contains dialogues in the following formats:
Poetry Completion:
User: Can you continue this poem for me? <poem_start>
Assistant: Sure, a continuation for this poem could be: <poem end>
Create poem in style of (?):
User: Can you write a poem for me in the style of <author>?… See the full description on the dataset page: https://huggingface.co/datasets/isaacrehg/poetry-instructions.MolOpt-Instructions 🐹 DrugAssist
A Large Language Model for Molecule Optimization
📃 Paper • 🤗 Model
Please refer to our repository and paper for more details.
Classification-Speech-Instructions
Classification Speech Instructions
Speech instructions for emotion, gender, age and language audio classification.
Source code
Source code at https://github.com/mesolitica/malaysian-dataset/tree/master/llm-instruction/speech-classification-instructions
supernatural-instructions-2m
Dataset Card for "supernatural-instructions-2m"
More Information needed
lawma-instructions_llama3_8klawma-instructionsinstruction-speech-text-v1.5-convo-male-voice