datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
RoadmapBench
RoadmapBench
A benchmark for evaluating AI coding agents on multi-target, long-horizon software development tasks derived from open-source project version upgrades.
Overview
RoadmapBench contains 115 tasks spanning 17 open-source repositories across 5 programming languages (Python, TypeScript, Go, Rust, C++). Each task requires an agent to implement multiple interdependent features that correspond to a real version upgrade of the target project.
Quick Start… See the full description on the dataset page: https://huggingface.co/datasets/UnipatAI/RoadmapBench.alpaca-data-gpt4-chineseRoadmapBench
RoadmapBench
A benchmark for evaluating AI coding agents on multi-target, long-horizon software development tasks derived from open-source project version upgrades.
Overview
RoadmapBench contains 115 tasks spanning 17 open-source repositories across 5 programming languages (Python, TypeScript, Go, Rust, C++). Each task requires an agent to implement multiple interdependent features that correspond to a real version upgrade of the target project.
Task Structure… See the full description on the dataset page: https://huggingface.co/datasets/benchmark-anon-2026/RoadmapBench.RoadSocial
About Dataset
This dataset accompanies the paper RoadSocial: A Diverse Dataset and Benchmark for Road Event Understanding from Social Video Narratives.
RoadSocial is a large-scale, diverse VideoQA dataset designed for generic road event understanding from social media narratives. It can help enhance the road event comprehension capabilities of general-purpose Video LLMs and improve their performance in traffic scene understanding, planning, and other autonomous vehicle (AV) related… See the full description on the dataset page: https://huggingface.co/datasets/chiragp26/RoadSocial.ChatHaruhi-RolePlaying
ChatHaruhi
Reviving Anime Character in Reality via Large Language Model
Chat-Haruhi-Suzumiyais a language model that imitates the tone, personality and storylines of characters like Haruhi Suzumiya,
https://github.com/LC1332/Chat-Haruhi-Suzumiya
Using this to load character and chat with him/her
from ChatHaruhi import ChatHaruhi
chatbot = ChatHaruhi( role_from_hf = "silk-road/ChatHaruhi-RolePlaying/haruhi",\
llm = 'openai' ,\… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/ChatHaruhi-RolePlaying.Wizard-LM-Chinese-instruct-evolWizard-LM-Chinese是在MSRA的Wizard-LM数据集上，对指令进行翻译，然后再调用GPT获得答案的数据集
Wizard-LM包含了很多难度超过Alpaca的指令。
中文的问题翻译会有少量指令注入导致翻译失败的情况
中文回答是根据中文问题再进行问询得到的。
我们会陆续将更多数据集发布到hf，包括
Coco Caption的中文翻译
CoQA的中文翻译
CNewSum的Embedding数据
增广的开放QA数据
WizardLM的中文翻译
如果你也在做这些数据集的筹备，欢迎来联系我们，避免重复花钱。
骆驼(Luotuo): 开源中文大语言模型
https://github.com/LC1332/Luotuo-Chinese-LLM
骆驼(Luotuo)项目是由冷子昂 @ 商汤科技, 陈启源 @ 华中师范大学 以及 李鲁鲁 @ 商汤科技 发起的中文大语言模型开源项目，包含了一系列语言模型。
( 注意: 陈启源 正在寻找2024推免导师，欢迎联系 )
骆驼项目不是商汤科技的官方产品。
Citation… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/Wizard-LM-Chinese-instruct-evol.ro-aya_collectionThis dataset is a translation of CohereLabs/aya_collection, an instruction dataset, using LLMic, a bilingual Romanian-English LLM.
The Aya Collection is a massive multilingual collection consisting of 513 million instances of prompts and completions covering a wide
range of tasks. This collection incorporates instruction-style templates from fluent speakers and applies them to a curated
list of datasets, as well as translations of instruction-style datasets into 101 languages.
Only the… See the full description on the dataset page: https://huggingface.co/datasets/faur-ai/ro-aya_collection.roam
About RoAM
We introduce Robot Autonomous Motion (RoAM), a unique video-action dataset that includes 50 long video sequences collected over 7 days at 14 different indoor spaces, capturing various indoor human activities from the ego-motion perspective of the Turtlebot3 robot. Along with the stereo image sequences, RoAM also contains time-stamped robot action sequences that are synchronised with the video data. The dataset primarily includes a range of human movements, such as walking… See the full description on the dataset page: https://huggingface.co/datasets/meenakshi-roam/roam.ChatHaruhi-from-RoleLLMAdapt English Role in RoleBench into ChatHaruhi format
only using profiles part in ZenMoore/RoleBench
Great thanks to on authors of RoleLLM!
usage:
# if you pip installed chatharuhi it should be
# from chatharuhi import ChatHaruhi
from ChatHaruhi import ChatHaruhi
chatbot = ChatHaruhi( role_from_hf = 'silk-road/ChatHaruhi-from-RoleLLM/Sherlock Holmes', \
llm = 'openai',
embedding = 'bge_en')
response = chatbot.chat(role='Police Chief', text = 'Oh… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/ChatHaruhi-from-RoleLLM.ChatHaruhi-54K-Role-Playing-Dialogue
ChatHaruhi
Reviving Anime Character in Reality via Large Language Model
github repo: https://github.com/LC1332/Chat-Haruhi-Suzumiya
Chat-Haruhi-Suzumiyais a language model that imitates the tone, personality and storylines of characters like Haruhi Suzumiya,
The project was developed by Cheng Li, Ziang Leng, Chenxi Yan, Xiaoyang Feng, HaoSheng Wang, Junyi Shen, Hao Wang, Weishi Mi, Aria Fei, Song Yan, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun,etc.
This… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/ChatHaruhi-54K-Role-Playing-Dialogue.ro_arc_challenge
Dataset Description
ARC is a dataset of 7,787 genuine grade-school level, multiple-choice science questions assembled to encourage research in
advanced question-answering.
Here we provide the Romanian translation of the ARC Challenge Set from the paper "Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback" (Lai et al., 2023).
This dataset is used as a benchmark and is part of the evaluation protocol for Romanian… See the full description on the dataset page: https://huggingface.co/datasets/OpenLLM-Ro/ro_arc_challenge.chinese-dolly-15kChinese-Dolly-15k是骆驼团队翻译的Dolly instruction数据集
最后49条数据因为翻译长度超过限制，没有翻译成功，建议删除或者手动翻译一下
原来的数据集'databricks/databricks-dolly-15k'是由数千名Databricks员工根据InstructGPT论文中概述的几种行为类别生成的遵循指示记录的开源数据集。这几个行为类别包括头脑风暴、分类、封闭型问答、生成、信息提取、开放型问答和摘要。
在知识共享署名-相同方式共享3.0（CC BY-SA 3.0）许可下，此数据集可用于任何学术或商业用途。
我们会陆续将更多数据集发布到hf，包括
Coco Caption的中文翻译
CoQA的中文翻译
CNewSum的Embedding数据
增广的开放QA数据
WizardLM的中文翻译
MMC4的中文翻译
如果你也在做这些数据集的筹备，欢迎来联系我们，避免重复花钱。
骆驼(Luotuo): 开源中文大语言模型
https://github.com/LC1332/Luotuo-Chinese-LLM… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/chinese-dolly-15k.Haruhi-Zero
用于ChatHaruhi-Zero Extend的训练数据
目前还不知道数据规模 知道的话回头会更名为Haruhi-Zero-XXX K
目前只放出每个source的sample，完整的数据将在1.0 模型放出之后发布
主项目链接 https://github.com/LC1332/Chat-Haruhi-Suzumiya
如果有兴趣加入我们的训练请联系chengli.thu@gmail.com
计划加入的数据源
数据源
中文小说数据
erotics小说数据
ChatHaruhi 52K, （转了message格式）
Chinese 13.2k, 转了message格式）
Waifu-extended 0.2K, 看看方不方便转成message格式，不行就简单的user-AI
Claude-Baize数据 7.2K
PIPPA数据 1.68K JanitorAI数据
PIPPA翻译数据
RoleLLM 1.6K, 看看方不方便转成message格式，不行就简单的user-AI
0.2
进一步去掉AI助理的相关数据… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/Haruhi-Zero.ChatHaruhi_NovelWriting50-Chinese-Novel-CharactersChatHaruhi-Expand-118K
ChatHaruhi Expanded Dataset 118K
62663 instance from original ChatHaruhi-54K
42255 English Data from RoleLLM
13166 Chinese Data from
github repo:
https://github.com/LC1332/Chat-Haruhi-Suzumiya
Please star our github repo if you found the dataset is useful
Regenerate Data
If you want to regenerate data with different context length, different embedding model or using your own chracter
now we refactored the final data generating pipeline
RoleLLM Data was generated by… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/ChatHaruhi-Expand-118K.ChatHaruhi-English-62K-RolePlaying
ChatHaruhi English_62K
20000 instance from original ChatHaruhi-54K
(translate original some chinese prompt into English)
42255 English Data from RoleLLM
token_len count via tokenizer from Phi-1.5
github repo:
https://github.com/LC1332/Chat-Haruhi-Suzumiya
Please star our github repo if you found the dataset is useful
Regenerate Data
If you want to regenerate data with different context length, different embedding model or using your own chracter
now we refactored the… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/ChatHaruhi-English-62K-RolePlaying.ro_arc_challenge
Dataset Description
ARC is a dataset of 7,787 genuine grade-school level, multiple-choice science questions assembled to encourage research in
advanced question-answering.
Here we provide the Romanian translation of the ARC Challenge Set from the paper "Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback" (Lai et al., 2023).
This dataset is used as a benchmark and is part of the evaluation protocol for Romanian… See the full description on the dataset page: https://huggingface.co/datasets/surogate/ro_arc_challenge.Road_construction_recommendationHaruhi-Baize-Role-Playing-Conversation
Haruhi-Zero的Conversation训练数据
我们计划拓展ChatHaruhi，从Few-shot到Zero-shot，这个数据集记录使用各个（中文）角色扮演api进行Baize式相互聊天后得到的数据结果
ids代表聊天的时候两张bot的角色卡片， 角色卡片的信息可以在https://huggingface.co/datasets/silk-road/Haruhi-Zero-RolePlaying-movie-PIPPA 中找到
并且对于第一次出现的id0，也会在prompt字段中进行记录。
聊天的时候id和ids的卡片进行对应
openai 代表两个聊天的bot都使用openai
GLM 代表两个聊天的bot都使用CharacterGLM
Claude 代表两个聊天的bot都使用Claude
Claude_openai 代表id0的使用Claude， id1的使用openai
Baichuan 代表两个聊天的bot都使用Character-Baichuan-Turbo… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/Haruhi-Baize-Role-Playing-Conversation.k8sbench
K8sBench: Kubernetes Configuration Generation Benchmark
K8sBench is a structured benchmark of 30 prompts covering 17 Kubernetes resource types, designed to evaluate LLMs on schema-validated Kubernetes manifest generation.
Metrics
Metric
Description
YAML%
YAML syntax validity (yaml.safe_load)
K8s%
Schema compliance (kubeconform --strict)
Sem%
Semantic field completeness (required fields present)
Resource Types Covered
Deployment, Service… See the full description on the dataset page: https://huggingface.co/datasets/roanbrasil/k8sbench.Kabyle_Road_Traffic_Code
Kabyle-English Road Traffic Code Dataset
A bilingual parallel corpus of 102 road traffic signs and regulations in English and Kabyle (Taqbaylit), an Amazigh language spoken in Algeria.
Categories
Dangers (Imihiten): Warning signs (39 entries)
Prohibitions (Tigedlin): Prohibitory signs (35 entries)
Obligations (Timariwin): Mandatory signs (16 entries)
End of Restrictions: End of regulation signs (12 entries)
Splits
Split
Size
Train
62… See the full description on the dataset page: https://huggingface.co/datasets/boffire/Kabyle_Road_Traffic_Code.taiwan-urban-roads
taiwan-urban-roads
Taiwan urban road networks, 1 km tiles. Vectors only (GeoJSON).
orig: 142
aug: 994 (rot/flip of orig)
files: data/tiles.jsonl, geojson/orig, geojson/aug
Derived from OpenStreetMap via Geofabrik Taiwan extract. License: ODbL-1.0. © OpenStreetMap contributors.
Haruhi-Zero-RolePlaying-movie-PIPPA
2000 Chinese RoleCards from IMDB_250 Movies and PIPPA
用于拓展zero-shot角色扮演的角色卡片。
其中870个角色来自电影字幕总结(id为movie_xx)，其中406张翻译成了简体中文，剩下的没翻（所以有些繁体或者英文混杂）
1270个角色来自于对PIPPA数据集的翻译
凌云志@伯恩茅斯大学 使用射手api爬取了电影的字幕
李鲁鲁 完成了从字幕到角色卡片的总结，以及对数据的翻译(openai)
后续
我们后续打算用这些卡片 从openai, CharacterGLM, KoboldAI的api中，利用Baize的方式去获得数据。
项目主页 https://github.com/LC1332/Chat-Haruhi-Suzumiya
如果你要讨论加入我们的项目
可以把你的联系方式私信发给 https://www.zhihu.com/people/cheng-li-47
MMC4-130k-chinese-imageMMC4-130k-chinese是对MMC4中，抽样了130k左右 simliarty较高的图文pair得到的数据集
Chinese版本是对这里所有的caption进行了翻译。
我们会陆续将更多数据集发布到hf，包括
Coco Caption的中文翻译
CoQA的中文翻译
CNewSum的Embedding数据
增广的开放QA数据
WizardLM的中文翻译
如果你也在做这些数据集的筹备，欢迎来联系我们，避免重复花钱。
骆驼(Luotuo): 开源中文大语言模型
https://github.com/LC1332/Luotuo-Chinese-LLM
骆驼(Luotuo)项目是由冷子昂 @ 商汤科技, 陈启源 @ 华中师范大学 以及 李鲁鲁 @ 商汤科技 发起的中文大语言模型开源项目，包含了一系列语言模型。
( 注意: 陈启源 正在寻找2024推免导师，欢迎联系 )
骆驼项目不是商汤科技的官方产品。
Citation
Please cite the repo if you use the data or code… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/MMC4-130k-chinese-image.ChatHaruhi-Waifu本数据集是为了部分不适合直接显示的角色进行hugging face存储。text部分做了简单的编码加密
使用方法
载入函数
from transformers import AutoTokenizer, AutoModel, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("silk-road/Chat-Haruhi_qwen_1_8", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("silk-road/Chat-Haruhi_qwen_1_8", trust_remote_code=True).half().cuda()
model = model.eval()
具体看https://github.com/LC1332/Chat-Haruhi-Suzumiya/blob/main/notebook/ChatHaruhi_x_Qwen1_8B.ipynb 这个notebook
from ChatHaruhi… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/ChatHaruhi-Waifu.Haruhi-Dialogue-Speaker-Extract
Chat凉宫春日的对话抽取模型
我们希望有一个模型能够从小说的chunk中批量去提取摘要和对话
这个模型就是实现了这一点。模型使用了大约30k的中文小说数据和20k的英文小说数据进行训练，在qwen-1.8上进行了3个epoch的finetune。 原则上模型同时支持中文和英文小说的训练
主项目链接 https://github.com/LC1332/Chat-Haruhi-Suzumiya
李鲁鲁完成了数据的收集，以及进一步将inference程序扩展到连续的chunks
刘崇寒完成了模型的训练
米唯实测试并上传模型到hugging face
Chat Haruhi Suzumiya's Dialogue Extraction Model
We hope to have a model that can extract summaries and dialogues in batches from chunks of novels.
This model achieves just that. It was trained… See the full description on the dataset page: https://huggingface.co/datasets/silk-road/Haruhi-Dialogue-Speaker-Extract.Carla-Road-X1
Carla-Road-X1: OpenDRIVE Road Network Generation Dataset
Text-to-xodr dataset for training language models to generate OpenDRIVE (.xodr) road network files from natural language descriptions.
Dataset Summary
Train samples: 385,929
Val samples: 42,882
Template samples: 40 (train: 36, val: 4)
Format: JSONL with Gemma chat template (system/user/assistant messages)
Data Sources
Source
Count
Description
OSM-converted xodr sub-networks
~428K… See the full description on the dataset page: https://huggingface.co/datasets/NCUT-AI/Carla-Road-X1.CharacterEval2sharegpt将CharacterEval转为适合Haruhi Zero测试的格式
之后还会上一个rag版本，把中间一些有rag信息的人物进行补全。
RoAd