datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Fineweb-Edu-Chinese-V2.1
Chinese Fineweb Edu Dataset V2.1 [中文] [English]
[OpenCSG Community] [👾github] [wechat] [Twitter]
📖Technical Report
The Chinese Fineweb Edu Dataset V2.1 is an enhanced version of the V2 dataset, designed specifically for natural language processing (NLP) tasks in the education sector. This version introduces two new data sources, map-cc and opencsg-cc, and retains data with scores ranging from 2 to 3. The dataset entries are organized into different… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.1.chinese-fineweb-edu
This version is deprecated. We recommend you to use the newest version Fineweb-edu-chinese-v2.1 !
Chinese Fineweb Edu Dataset [中文] [English]
[OpenCSG Community] [👾github] [wechat] [Twitter]
📖Technical Report
Chinese Fineweb Edu dataset is a meticulously constructed high-quality Chinese pre-training corpus, specifically designed for natural language processing tasks in the education domain. This dataset undergoes a rigorous selection and… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/chinese-fineweb-edu.Fineweb-Edu-Chinese-V2.2
Chinese Fineweb Edu Dataset V2.2 (Instruct & Pre-train)
[[中文]] | [[English]]
OpenCSG Community | 👾 GitHub | 📖 Technical Report
Dataset Introduction: Filling the Data Puzzle for Chinese Education LLMs
Chinese Fineweb Edu Dataset V2.2is a rare high-quality dataset in the open-source community that covers the full process from Pre-training to Supervised Fine-Tuning (SFT) for the Chinese education domain.
This project aims to solve the core pain point of… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.2.chinese-fineweb-edu-v2
This version is deprecated. We recommend you to use the newest version Fineweb-edu-chinese-v2.1 !
Chinese Fineweb Edu Dataset V2 [中文] [English]
[OpenCSG Community] [👾github] [wechat] [Twitter]
📖Technical Report
Chinese Fineweb Edu Dataset V2 is a comprehensive upgrade of the original Chinese Fineweb Edu, designed and optimized for natural language processing (NLP) tasks in the education sector. This high-quality Chinese pretraining dataset has… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/chinese-fineweb-edu-v2.smoltalk-chinese
Chinese SmolTalk Dataset [中文] [English]
[OpenCSG Community] [👾github] [wechat] [Twitter]
📖Technical Report
smoltalk-chinese is a Chinese fine-tuning dataset constructed with reference to the SmolTalk dataset. It aims to provide high-quality synthetic data support for training large language models (LLMs). The dataset consists entirely of synthetic data, comprising over 700,000 entries. It is specifically designed to enhance the performance of Chinese… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/smoltalk-chinese.chinese-cosmopedia
Chinese Cosmopedia Dataset [中文] [English]
[OpenCSG Community] [👾github] [wechat] [Twitter]
📖Technical Report
The Chinese Cosmopedia dataset contains a total of 15 million entries, approximately 60B tokens. Two key elements in constructing the synthetic dataset are seed data and prompts. Seed data determines the theme of the generated content, while prompts define the style of the data (such as textbooks, stories, tutorials, or children's books). The data… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/chinese-cosmopedia.LLaVA-Instruct-600K-Chinese
仿照 LLaVA-Instruct-150K ，使用 Qwen2.5-VL-32B-Instruct 合成的用于微调中文VLM的数据；也可以与英文数据集混合使用，训练多语言VLM
任务类型为基于单张图片的问答和对话，每个样本都对应一张不同的图片，其中大部分图片包含中文字符，更适合中文场景下视觉语言模型的训练。
图片从各类中文网站上爬取
包含3类任务：日常对话、复杂推理、描述图片。日常对话通常是5轮对话，其余任务是1轮对话。
每种任务的数量如下：
任务类型
数量
日常对话
247,431
复杂推理
194,646
描述图片
199,791
用于生成对话数据的prompt如下
日常对话
设计一个你和一个询问这张照片的人之间的对话。答案应该是视觉AI助手看到图像并回答问题的语气。
你需要提出不同的问题并给出相应的答案。问题可以包括询问图像视觉内容的问题，包括对象类型、对象计数、对象动作、对象位置、对象之间的相对位置等。必须是有明确答案的问题，即
（1） 人们可以在图像中明确看到问题所问的内容，并且可以自信地回答；
（2）… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/LLaVA-Instruct-600K-Chinese.UltraFeedback-chinese
Chinese SmolTalk Dataset [中文] [English]
[OpenCSG Community] [👾github] [wechat] [Twitter]
UltraFeedback Chinese Dataset
UltraFeedback-Chinese is a Chinese version developed based on the construction method of the UltraFeedback dataset, designed specifically for training robust reward and critic models. This dataset supports two training methods: PPO (Proximal Policy Optimization) and DPO (Direct Preference Optimization). UltraFeedback-Chinese… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/UltraFeedback-chinese.Fineweb-Edu-Chinese-V2.3
Chinese Fineweb Edu Dataset V2.3
中文 | English
OpenCSG 社区 | GitHub | 数据集许可协议
数据集简介
Chinese Fineweb Edu Dataset V2.3 是 OpenCSG 面向中文教育、知识问答、指令微调和文本生成场景构建的高质量中文教育 SFT 数据集。
该版本包含 23.04 万条高质量中文教育 QA pairs，并将同一批问答对发布为 Alpaca、Messages、Messages-no-system 三种训练格式。三种格式面向不同训练模板，建议训练时按模型和框架选择其中一种格式使用，而不是将不同格式简单相加作为独立知识规模。
V2.3 是在 V2.2 基础上的质量升级版本。针对 V2.2 社区反馈和内部质量审计中出现的重复模式、异常中英文混入、噪声片段、弱证据支撑回答和低质量合成输出等问题，V2.3 提高了源文本进入生成环节的门槛，并优化了问答生成与过滤逻辑。
在数据构建上，V2.3 从约 2.3T… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.3.Fineweb-Edu-Chinese-V3
Fineweb-Edu-Chinese-V3
中文 | English
OpenCSG 社区 | 数据集许可协议
数据集简介
Fineweb-Edu-Chinese-V3 是 OpenCSG 面向学科知识问答、教材理解和推理型指令微调场景构建的高质量中英双语教育 SFT 数据集，也是 Fineweb-Edu-Chinese 系列的最新版本。
该版本包含 18.81 万条 SFT 样本，来自 100,442 篇高质量图书、教材、学科文献与技术长文，覆盖计算机、自然科学、社科人文、法学、经济五大学科方向，并同步提供 Messages、Messages-no-system、Alpaca 三种训练格式。三种格式是同一批问答对的不同导出视图，训练时应按模型模板选择其中一种，而不是简单相加作为独立数据规模。
V3 是 Fineweb-Edu-Chinese 系列的一次数据源与构造范式的整体切换。V1.0 至 V2.3 均以大规模中文网页语料为基础：通过打分器筛选出具备教育属性的网页文本，再由大模型生成问答。V3… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V3.csg-robomaster
csg-robomaster [中文] [English]
[OpenCSG Community] [github] [wechat] [Twitter]
OpenCSG stands for Converged resources, Software refinement, and Generative LM. The 'C' represents Converged resources, indicating the integration and full utilization of hybrid resources. The 'S' stands for Software refinement, signifying software that is refined by large models. The 'G' represents Generative LM, which denotes widespread, inclusive, and democratized generative… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/csg-robomaster.CIMD
CIMD
[[中文]] | [[English]]
CSGHub Dataset Page | Hugging Face | OpenCSG Community
中文说明
数据集概述
CIMD 是一个面向文档智能任务的跨来源、多语言 JSONL 语料库。当前公开快照包含 111,308 条解析记录，覆盖制度参考、学术与长文档资料、机构分析、企业运营、公共讨论和市场相关材料等来源家族。每条记录都把正文与来源类型、语言、时间、关键词、授权标签和来源字段放在同一个结构里，用户拿到数据后可以直接做检索、抽样、审计和数据治理。
公开数据已转换为统一字段，并按来源家族拆分为可单独加载的子集；它不是原始文件夹的简单打包。用户可以只读取制度参考、学术长文档或公共讨论记录，也可以合并多个子集构建检索库、抽取训练候选样本、构造评测样本池，并按来源、语言和时间字段继续筛选。
CIMD 和通用网页语料的差别在于记录级元数据。它不只提供可索引文本，还提供… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/CIMD.autohub-benchmark
autohub-benchmark
This project designs common use scenarios for web-based code, model, and dataset hosting platforms, and provides corresponding prompts and ground truth. These resources can be used to evaluate the localization performance of visual language models (VLMs) in specialized scenarios.
Model Hosting Platform GUI Inference
Model
Platform
Accuracy (%)
Error (%)
Invalid (%)
Completion Rate (%)
AriaUI
Huggingface
70.8
12.5
6.7
100.0
ModelScope
57.6… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/autohub-benchmark.details_opencsg__csg-wukong-1B-chat-v0.1PR_review_deepseek
Pull request review task
Given the original code snippet (may be truncated) and a pull request (in diff format), the model reviews the PR and decides whether it should be merged.
The answers are generated by Deepseek-V2
chinese-fineweb-v2-scorer-train-dataChinese-fineweb-v2 和 v2.1 中使用了 基于 bert 的文本打分模型，此数据集为模型的训练数据。
text: 被打分的文本
edu_eval: 由 qwen2.5-14b-instruct 产生的打分结果
score: 打分结果中提取出来的得分
details_opencsg__csg-wukong-1B
Dataset Card for Evaluation run of opencsg/csg-wukong-1B
Dataset automatically created during the evaluation run of model opencsg/csg-wukong-1B on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_opencsg__csg-wukong-1B.details_opencsg__csg-wukong-1B-orpo-bf16details_opencsg__csg-wukong-1B-sft-bf16details_opencsg__csg-wukong-1B-sft-dpo-bf16