cosmopedia
cosmopedia
Cosmopedia v0.1
Image generated by DALL-E, the prompt was generated by Mixtral-8x7B-Instruct-v0.1
Note: Cosmopedia v0.2 is available at smollm-corpus
User: What do you think "Cosmopedia" could mean? Hint: in our case it's not related to cosmology.
Mixtral-8x7B-Instruct-v0.1: A possible meaning for "Cosmopedia" could be an encyclopedia or collection of information about
different cultures, societies, and topics from around the world, emphasizing diversity and global… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceTB/cosmopedia.cosmopedia-100k
Dataset description
This is a 100k subset of Cosmopedia dataset. A synthetic dataset of textbooks, blogposts, stories, posts and WikiHow articles generated by Mixtral-8x7B-Instruct-v0.1.
Here's how you can load the dataset
from datasets import load_dataset
ds = load_dataset("HuggingFaceTB/cosmopedia-100k", split="train")
cosmopedia_quality_score_v2Adding quality score v2 to HuggingFaceTB/cosmopedia
chinese-cosmopedia
Chinese Cosmopedia Dataset [中文] [English]
[OpenCSG Community] [👾github] [wechat] [Twitter]
📖Technical Report
The Chinese Cosmopedia dataset contains a total of 15 million entries, approximately 60B tokens. Two key elements in constructing the synthetic dataset are seed data and prompts. Seed data determines the theme of the generated content, while prompts define the style of the data (such as textbooks, stories, tutorials, or children's books). The data… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/chinese-cosmopedia.cosmopedia-logprobssmollm-cosmopedia-v2