datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SlimPajama-Meta-rater
Annotated SlimPajama Dataset
Dataset Description
This dataset contains the first fully annotated SlimPajama dataset with comprehensive quality metrics for data-centric large language model research. The dataset includes approximately 580 billion tokens from the training set of the original SlimPajama dataset, annotated across 25 different quality dimensions.
Note: This dataset contains only the training set portion of the original SlimPajama dataset, which is why the… See the full description on the dataset page: https://huggingface.co/datasets/opendatalab/SlimPajama-Meta-rater.SlimOrca
Overview
This is a new curated subset of our OpenOrca data. This release provides an efficient means of reaching performance on-par with using larger slices of our data, while only including ~500k GPT-4 completions.
The key change in this dataset is that we've done an additional pass, using GPT-4 to remove answers which appear wrong based on the human annotations from the FLAN dataset.
This reduces the dataset size to only ~500k entries, allowing training to a similar quality level… See the full description on the dataset page: https://huggingface.co/datasets/Open-Orca/SlimOrca.SlimPajama-chunked
SlimPajama-Chunked
Dataset Description
This is a chunked re-upload of Cerebras' SlimPajama-627B. The original upload has split
the dataset into 10 chunks, with each containing upwards of 5,000 files. This makes it cumbersome to download and process. We've downloaded the entire
dataset for our own purposes, and decided to upload the chunked version for easier usage.
Each file is ~45GB due to HuggingFace's limitation of 50GB per LFS file.
slimpajama_labeledThis is the dataset used in the paper Multi-Agent Collaborative Data Selection for Efficient LLM Pretraining.
It is a labeled version of the SlimPajama-627B train dataset.
An example in this dataset:
{
"id": "BkiUdvk25V5jCITp144_",
"content": "At the time of Federation most Australian colonies had introduced income taxes, each with its own rules and administered in its own way. This was further complicated with some jurisdictions recording tax according to a taxpayer's residence, and… See the full description on the dataset page: https://huggingface.co/datasets/beccabai/slimpajama_labeled.SlimPajama-Meta-rater-Readability-30B
Top 30B token SlimPajama Subset selected by the Readability rater
This repository contains the dataset described in the paper Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models.
Code: https://github.com/opendatalab/Meta-rater
Dataset Description
This dataset contains the top 30B tokens from the SlimPajama-627B corpus, selected using the Readability dimension of the PRRC (Professionalism, Readability, Reasoning, Cleanliness) framework.… See the full description on the dataset page: https://huggingface.co/datasets/opendatalab/SlimPajama-Meta-rater-Readability-30B.SlimPajama-Meta-rater-Professionalism-30B
Top 30B token SlimPajama Subset selected by the Professionalism rater
This repository contains the dataset described in the paper Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models.
Code: https://github.com/opendatalab/Meta-rater
Dataset Description
This dataset contains the top 30B tokens from the SlimPajama-627B corpus, selected using the Professionalism dimension of the PRRC (Professionalism, Readability, Reasoning, Cleanliness)… See the full description on the dataset page: https://huggingface.co/datasets/opendatalab/SlimPajama-Meta-rater-Professionalism-30B.SlimPajama-Meta-rater-Reasoning-30B
Top 30B token SlimPajama Subset selected by the Reasoning rater
This repository contains the dataset described in the paper Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models.
Code: https://github.com/opendatalab/Meta-rater
Dataset Description
This dataset contains the top 30B tokens from the SlimPajama-627B corpus, selected using the Reasoning dimension of the PRRC (Professionalism, Readability, Reasoning, Cleanliness) framework. Each… See the full description on the dataset page: https://huggingface.co/datasets/opendatalab/SlimPajama-Meta-rater-Reasoning-30B.SlimPajama-Meta-rater-Cleanliness-30B
Top 30B token SlimPajama Subset selected by the Cleanliness rater
This repository contains the dataset described in the paper Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models.
Code: https://github.com/opendatalab/Meta-rater
Dataset Description
This dataset contains the top 30B tokens from the SlimPajama-627B corpus, selected using the Cleanliness dimension of the PRRC (Professionalism, Readability, Reasoning, Cleanliness) framework.… See the full description on the dataset page: https://huggingface.co/datasets/opendatalab/SlimPajama-Meta-rater-Cleanliness-30B.Slim-Wildchat-zhA big shout out to AllenAI, you guys rock!
从WildChat中抽出中文对话,但是因为发现了很多重复对话，有的人会反复的用一个prompt进行提问，有的人会换3.5或4去问同样的问题，所以进行了简单的去重。
去重方法大致为，使用bert-base-chinese将第一个问题转换为embedding，使用类knn的方法抽取了1万条。并转换成了sharegpt格式。
注意！在对话中发现了NSFW的内容，并没有进行过滤，使用请注意甄别。
你会找到三个jsonl文件：
wildchat-seed-multi-200.json 是使用每一个单独的Dialogue的首个HumanQuestion为基础，采样的200个种子任务，用于EvolInsturction。
Subsample_10K.jsonl 原始版本，是使用每一个单独的Dialogue的首个HumanQuestion为基础，采样的1万个对话。
1213_Wildchat_zh_Sharegpt_ConcatSubsample_20k.jsonl… See the full description on the dataset page: https://huggingface.co/datasets/lorinma/Slim-Wildchat-zh.prompt-slimmer-slm
Prompt Slimmer SLM — Demo Dataset
Synthetic examples for experimenting with prompt rewriting and sentence selection. Exported without changing the examples or their original splits from the shared GitHub codebase.
Model · Project page
Configuration
Train
Validation
Test
Purpose
rewrites-expanded (default)
41
2
2
Expanded rewriting dataset: 45 examples
rewrites
9
2
2
Original dataset used by the first adapter
selector
256
64
64
KEEP/DROP labels for source spans… See the full description on the dataset page: https://huggingface.co/datasets/ai-mitra/prompt-slimmer-slm.SlimOrca-enPurified-openai-messages
Dataset Card for enPurified/SlimOrca-enPurified-openai-messages
This dataset was updated on January 14th to remove even more math, code, and low quality prose from the dataset. That's why the number below shows it was trimming from 270k.
Dataset Summary
The enPurified collection is an initiative to curate high-fidelity English prose datasets for language modeling. The primary objective is to isolate high-quality natural language text by strictly excising code… See the full description on the dataset page: https://huggingface.co/datasets/enPurified/SlimOrca-enPurified-openai-messages.SlimOrca-Llama-3-Preference-DPO-Pairs
SlimOrca-Llama-3-Preference-DPO-Pairs
This dataset is based on instructions of SlimOrca-Dedup-Alpaca, with Llama-3 generated response to form a preference dataset.
SlimPajama
SlimPajama-Chunked
Dataset Description
This is a chunked re-upload of Cerebras' SlimPajama-627B. The original upload has split
the dataset into 10 chunks, with each containing upwards of 5,000 files. This makes it cumbersome to download and process. We've downloaded the entire
dataset for our own purposes, and decided to upload the chunked version for easier usage.
Each file is ~45GB due to HuggingFace's limitation of 50GB per LFS file.
Ko.SlimOrca원본 데이터셋: Open-Orca/SlimOrca
Slim-LCCC-zh在LLM横行的今天，大家都在讲究SFT数据质量。相比于各种一板一眼的AI回复，又是step by step又是detailed reasoning，这种非常casual的对话显得那么的独特，更适合用作情感陪伴闲聊机器人的目的。
本项目提供了一个大规模中文对话数据集，原始数据来自于清华大学的LCCC(Large-scale Cleaned Chinese Conversation)数据集
基于LCCC-large，但因为有1200万。故使用bert-base-chinese转换为embedding，且使用类knn的方法抽取了1万条。并转换成了sharegpt格式。
从实用的角度来说，因为对话都只有两句，需要通过GPT进行续写。但是实测发现openai系列的太严肃了，失去了casual的味道。浅测了一下文心一言可以续写这种闲聊对话。只是测试了一下，并没有放在这个数据集中。
当然了，最好的还是收集真实世界的对话。
Slim-Moss003sft-zh因为原生的Moss003数量太大，所以进行了简单的去重。
去重方法大致为，只选择中文的对话，使用bert-base-chinese将第一个问题转换为embedding，使用类knn的方法抽取了1万条。并转换成了sharegpt格式。
glados
Dataset Card for Dataset Name
This dataset card aims to be a base template for new datasets. It has been generated using this raw template.
Dataset Details
Dataset Description
Curated by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Dataset Sources [optional]
Repository: [More… See the full description on the dataset page: https://huggingface.co/datasets/slimShaders/glados.SlimOrcaVi