datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
OmegaUse-OfficeVal
OmegaUse-OfficeVal
Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding
OmegaUse-OfficeVal is a benchmark for evaluating LLM agents on long-horizon,
real-world office-suite tasks that span word-processing documents, spreadsheets,
presentations, and cross-file productivity workflows. Tasks are derived from
authentic office requests proposed by practitioners and drawn from freelance
platforms, grounding the benchmark in real economic demand. Each task… See the full description on the dataset page: https://huggingface.co/datasets/baidu-frontier-research/OmegaUse-OfficeVal.baidu-baike-dataset
Baidu Baike Dataset
This dataset contains 5,634,898 entries extracted from Baidu Baike (百度百科), which is one of the largest Chinese online encyclopedias. This is a mirror of the original dataset from https://github.com/BIT-ENGD/baidu_baike, with data originally crawled around 2019.
Format
Each entry in the JSON format includes the following fields:
title: The title of the Baidu Baike entry
summary: A brief summary of the entry content
sections: A list of sections, where… See the full description on the dataset page: https://huggingface.co/datasets/lars1234/baidu-baike-dataset.BaiduBaike-5.63Mbaidu-ultr-pretrainbaiduBaidu_Tieba_KangYaBeiGuo说明
随机爬取的百度贴吧抗压背锅吧的内容，10万条左右，不包含视频和图片，比较适合用于风格微调（大概）（心虚）。 数据遵循ChatGLM4使用的格式（有需要别的格式请自己调整QWQ）。 清洗的不是很干净，所以把没有清洗的数据也发上来了（QWQ）。
original.json是爬取后未经清洗的数据
Description
This dataset consists of roughly 100,000 samples randomly scraped from the "Kang Ya Bei Guo" bar on Baidu Tieba. It does not contain videos or images and is generally suitable for style fine-tuning (probably... kind of... maybe 👀). The data follows the format used by ChatGLM4 (please adjust to other formats if needed, QWQ). Note: The… See the full description on the dataset page: https://huggingface.co/datasets/Orphanage/Baidu_Tieba_KangYaBeiGuo.Baidu_Tieba_SunXiaochuan
说明
随机爬取的百度贴吧孙笑川吧的内容，10万条左右，不包含视频和图片，比较适合用于风格微调（大概）（心虚）。
数据遵循ChatGLM4使用的格式（有需要别的格式请自己调整QWQ）。
清洗的不是很干净，所以把没有清洗的数据也发上来了（QWQ）。
train.jsonl是训练集
dev.jsonl是验证集
No_train_validation_split.jsonl是清洗后并未划分训练和验证集的数据
original.json是爬取后未经清洗的数据
Description
This dataset consists of roughly 100,000 samples randomly scraped from the "Sun Xiaochuan" bar on Baidu Tieba. It does not contain videos or images and is generally suitable for style fine-tuning (probably... kind of... maybe 👀).
The… See the full description on the dataset page: https://huggingface.co/datasets/Orphanage/Baidu_Tieba_SunXiaochuan.baidu_baike
Baidu Baike Dataset
A large-scale dataset of Baidu Baike articles first introduced in the paper An Analysis of Chinese Censorship Bias in LLMs.
Articles were automatically scraped from the Internet Archive's snapshots of the encyclopedia.
Citation
If you publish work using our datasets or CensorshipDetector, please cite our work using the following citation:
@inproceedings{ahmed2025censorshipbias
title = {An Analysis of Chinese Censorship Bias in LLMs},
author… See the full description on the dataset page: https://huggingface.co/datasets/mohamedah/baidu_baike.chinese-street-text-baiduTaken from here: https://aistudio.baidu.com/datasetdetail/8429 ; This solely acts as a nice way to integrate it to my scripts
baidubaike
Dataset Card for "baidubaike"
More Information needed
baidubaike2
Dataset Card for "baidubaike2"
More Information needed
baiduwiki
数据集介绍
这是百度百科数据集，对原始数据集进行了简单清洗，最终变成了类似:{"text":....}对结构，jsonl文件。
清洗代码
import json
input_file = "百度百科.jsonl"
output_file = "filtered_data.jsonl"
with open(input_file, 'r', encoding='utf-8') as infile, \
open(output_file, 'w', encoding='utf-8') as outfile:
for line in infile:
if not line.strip():
continue
try:
data = json.loads(line)
outfile.write(json.dumps(data, ensure_ascii=False) + '\n')
except… See the full description on the dataset page: https://huggingface.co/datasets/Zhaoming213/baiduwiki.baidu-ruozhiba-llama3-tt