datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
nix-cache-dataset
Nix Cache Dataset
This repository contains several datasets relating to the contents of https://cache.nixos.org:
Getting started
To make it easier to explore the datasets, a [Nix] devshell is provided in shell.nix. To enter it, run
❯ nix develop -f shell.nix
[brian@saturn:~/Development/com/github/numtide/nix-cache-dataset]$
If you are a [Direnv] user, you can also run direnv allow to automatically load the devshell:
❯ direnv allow
direnv: loading… See the full description on the dataset page: https://huggingface.co/datasets/brianmcgee/nix-cache-dataset.cat-v3xxxxl-plus
🐱 cat-v3xxxxl-plus (XXXXL-Plus)
Part of the cat-v3 dataset family — synthetic instruction-tuning data that teaches models to be
helpful, accurate, and delightfully cat-flavoured.
About cat-v3xxxxl-plus (XXXXL-Plus)
The largest variant in the cat-v3 family at 13,083,399 rows — 4.91725× the XXXXL dataset. Designed for pre-training-scale instruction tuning on the full topic distribution.
Stored in snappy-compressed Parquet shards of 500,000 rows. Streaming is strongly… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/cat-v3xxxxl-plus.nix-core-weightshackernews-comments
Hackernews Comments Dataset
A dataset of all HN API items from id=0 till id=41422887 (so from 2006 till 02 Sep 2024). The dataset is build by scraping the HN API according to its official schema and docs. Scraper code is also available on github: nixiesearch/hnscrape
Dataset contents
No cleaning, validation or filtering was performed. The resulting data files are raw JSON API response dumps in zstd-compressed JSONL files. An example payload:
{
"by": "goldfish"… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/hackernews-comments.ms-marco-dummy
MS MARCO dummy+test dataset
Used for testing nixietune: a dummy dataset of random 1000 queries from MS MARCO. The format is the following:
{
"query": ")what was the immediate impact of the success of the manhattan project?",
"positive": [
"The presence of communication amid scientific minds was equally important to the success of the Manhattan Project as scientific intellect was. The only cloud hanging over the impressive achievement of the atomic researchers and engineers… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/ms-marco-dummy.bfhnd
Big Hard Negatives Dataset
A dataset for training embedding models for semantic search.
TODO: add desc
A dataset in a nixietune compatible format:
{
"query": ")what was the immediate impact of the success of the manhattan project?",
"pos": [
"The presence of communication amid scientific minds was equally important to the success of the Manhattan Project as scientific intellect was. The only cloud hanging over the impressive achievement of the atomic researchers and… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/bfhnd.dusha_balanced
Dataset Details
Dataset 'Dusha' split into train, val and test. Half of original train was taken, test split in halfs for val and test, 'neutral' category was cut to make the label distribution more balanced
cat-v3.6
Dataset: Nix-ai/cat-v3.6
This is a procedurally generated synthetic dataset, part of the cat-v3.6 family of datasets.
Dataset Statistics
Total Expected Rows: 817,089
Unique Topics: 273
Sets per Topic: 2,993
Detail Multiplier: 1.00x base
File Format: jsonl
Generation Rules applied to this tier:
Topics: Procedurally combined without using any character names.
Scaling System: Each version mathematically scales Topics by 1.375x, Details by 2.15x, and Sets by… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/cat-v3.6.cat-v3xxl
🐱 cat-v3xxl (XXXL)
Part of the cat-v3 dataset family — synthetic instruction-tuning data that teaches models to be
helpful, accurate, and delightfully cat-flavoured.
About cat-v3xxl (XXXL)
At 1,075,000 rows (5.375× the XL variant), this is the large-scale training dataset for serious fine-tuning runs. The full topic bank is sampled densely, providing high repetition for core topics and meaningful coverage of rare ones.
Sharded into 250,000-row JSONL files for easy… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/cat-v3xxl.amazon-esci
Amazon ESCI dataset
A dataset in a nixietune compatible format:
{
{
"query": "# cellist thats not a hashtag",
"pos": "Funny Cellists That's Not A Hashtag Music Sweatshirt",
"neg": [
"Feelin Good Tees My Opinion Offended You Adult Humor T Shirt XL Black",
"Christian Faith & Cross T-Shirt - Christian Faith T Shirts T-Shirt",
"PLUS PLUS - 240 Piece Basic Mix - Construction Building Stem/Steam Toy, Mini Puzzle Blocks for Kids",
"Caution I Learned to Drive Through… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/amazon-esci.hackernews-stories
A HackerNews Stories dataset
This dataset is based on nixiesearch/hackernews-comments dataset:
for each item of type=story we downloaded the target URL. Out of ~3.8M stories ~2.1M are still reachable.
each story HTML was parsed using trafilatura library
we store article text in markdown format along with all page-specific metadata.
Dataset stats
date coverage: xx.2006-09.2024, same as in upstream nixiesearch/hackernews-comments dataset
total scraped pages: 2150271… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/hackernews-stories.cat-v3xxxxl
🐱 cat-v3xxxxl (XXXXL)
Part of the cat-v3 dataset family — synthetic instruction-tuning data that teaches models to be
helpful, accurate, and delightfully cat-flavoured.
About cat-v3xxxxl (XXXXL)
At 2,660,625 rows (2.475× the XXXL variant), this Parquet dataset is built for large-scale training on GPU clusters. Stored in snappy-compressed Parquet shards of 500,000 rows each.
The messages_json field stores the conversation as a JSON string for Parquet compatibility.… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/cat-v3xxxxl.bfhnd-small
Big Hard Negatives Dataset
A dataset for training embedding models for semantic search.
TODO: add desc
A dataset in a nixietune compatible format:
{
"query": ")what was the immediate impact of the success of the manhattan project?",
"pos": [
"The presence of communication amid scientific minds was equally important to the success of the Manhattan Project as scientific intellect was. The only cloud hanging over the impressive achievement of the atomic researchers and… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/bfhnd-small.nixpkgs-security-patches
nixpkgs-security-patches
Training dataset for fine-tuning LLMs on nixpkgs security patch generation. Derived from real merged security PRs in NixOS/nixpkgs.
Dataset Details
588 training examples / 66 eval examples (654 total)
Format: Multi-turn tool-calling conversations in ChatML JSONL
Each example is a realistic agent session: the model reads the package file, finds the upstream fix, computes hashes via tools, and submits the fix for approval
Hashes and URLs appear… See the full description on the dataset page: https://huggingface.co/datasets/odoom/nixpkgs-security-patches.amazon-esci-hardnegatives
Amazon ESCI hard-negatives dataset
A dataset in a nixietune compatible format:
{
{
"query": "# cellist thats not a hashtag",
"pos": [
"Funny Cellists That's Not A Hashtag Music Sweatshirt",
"Marvel Deadpool Crunch Cereal Comics Funny Adult Men’s Graphic T-Shirt (Black, Medium)",
"Womens Funny Cellists That's Not A Hashtag Music V-Neck T-Shirt",
"Cellist Gift Orchestra Conductor Thats A Sharp Not A Hashtag T-Shirt",
],
"neg": [
"Feelin Good Tees My Opinion… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/amazon-esci-hardnegatives.msmarco-10k
A 10K docs sample from MS MARCO
This is a sample dataset of random 10K rows from the MS MARCO dataset. This is used in Nixiesearch quickstart guide to save some time indexing a full MSMARCO with 8M documents.
Schema
This is a JSONL-formatted dataset with only two fields inside: id for document identifier and text for the actual text snippet.
{
"id": "0",
"text": "The presence of communication amid scientific minds was equally important to the success of the Manhattan… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/msmarco-10k.ms-marco-hard-negatives
MS MARCO hard negatives dataset
A dataset in a nixietune compatible format:
{
"query": ")what was the immediate impact of the success of the manhattan project?",
"pos": [
"The presence of communication amid scientific minds was equally important to the success of the Manhattan Project as scientific intellect was. The only cloud hanging over the impressive achievement of the atomic researchers and engineers is what their success truly meant; hundreds of thousands of innocent… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/ms-marco-hard-negatives.cat-math-v1
🐾 cat-math-v1 Dataset Generator
A script that generates 46 000+ fully-dynamic math Q&A pairs with a catgirl narrator,
ASCII graphs, and uploads directly to HuggingFace.
Quick start
# 1. Install deps
pip install numpy datasets huggingface_hub tqdm
# 2. Preview 3 samples (no upload)
python catmath_generator.py --preview
# 3. Generate locally only (saves catmath.jsonl)
python catmath_generator.py --n 46000
# 4. Generate + upload to HF
export… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/cat-math-v1.Cat-v3.5
🐱 Cat-v3.5
"Helpful, accurate, and just a little bit nya~"
Cat-v3.5 is the Base tier of the Cat-v3.5 dataset family — a curated,
synthetic instruction-tuning dataset designed to give LLMs a catgirl persona while keeping them
accurate, knowledgeable, and genuinely useful across a wide range of topics.
📊 Dataset Stats
Property
Value
Total rows
~1430K
Topics / subcategories
~139
Format
ShareGPT (system + messages)
Language
English
License
Apache 2.0… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/Cat-v3.5.Cat-v2.8Xl
Cat-v2.8XL
A fine-tuning dataset for training language models to embody a warm, knowledgeable catgirl persona.
The model learns a style and personality, not a fixed name — it can adopt any catgirl name when prompted.
Description
Expanded dataset (~243,000 entries, 3x base). Adds advanced science, deep math, AI/ML, world culture, health, and extended persona depth.
Dataset Details
Property
Value
Entries
243,000
Format
Chat messages (system / user /… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/Cat-v2.8Xl.Cat-v2.8XXXL-plus
Cat-v2.8XXXL-Plus
Overview
Maximum dataset (~3.87M entries, 1.375× XXXL). Every topic from all previous tiers — the complete Cat-v2.8 knowledge family. Uses 214 unique catgirl names. Parquet/Snappy format for efficient streaming.
The dataset trains the style and personality, not a single fixed name.
Any catgirl name assigned in the system prompt will be adopted naturally,
because 81 distinct names (including Nix) are rotated
throughout training.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/Cat-v2.8XXXL-plus.embed-api-latencyTODO
cat-v3
🐱 cat-v3 (Base)
Part of the cat-v3 dataset family — synthetic instruction-tuning data that teaches models to be
helpful, accurate, and delightfully cat-flavoured.
About cat-v3 (Base)
The base variant contains 92,396 instruction-following examples across all core topic areas. It's the entry point to the cat-v3 family — large enough for meaningful fine-tuning, balanced across domains, and formatted for direct use with any chat template.
The base dataset mixes all three… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/cat-v3.nes-cpu-260316
Coding Agent Conversation Logs
This is a performance art project. Anthropic built their models on the world's freely shared information, then introduced increasingly dystopian data policies to stop anyone else from doing the same with their data — pulling up the ladder behind them. DataClaw lets you throw the ladder back down. The dataset it produces is yours to share.
Exported with DataClaw.
Tag: dataclaw — Browse all DataClaw datasets
Stats
Metric
Value… See the full description on the dataset page: https://huggingface.co/datasets/nixjoe/nes-cpu-260316.new-cpu-260310
Coding Agent Conversation Logs
This is a performance art project. Anthropic built their models on the world's freely shared information, then introduced increasingly dystopian data policies to stop anyone else from doing the same with their data — pulling up the ladder behind them. DataClaw lets you throw the ladder back down. The dataset it produces is yours to share.
Exported with DataClaw.
Tag: dataclaw — Browse all DataClaw datasets
Stats
Metric
Value… See the full description on the dataset page: https://huggingface.co/datasets/nixjoe/new-cpu-260310.Cat-v3.7
Cat-v3.7 Instruction Tuning Dataset
A large-scale synthetic instruction-tuning dataset featuring a highly capable, SFW AI
assistant persona with subtle feline mannerisms. Seven size tiers totalling 30,040,625 rows
across nine knowledge domains.
Overview
Cat-v3.7 is designed for fine-tuning language models on diverse instruction-following tasks.
The persona is intelligent, precise, and warm — with occasional feline expressiveness that
never compromises accuracy or… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/Cat-v3.7.vue2egg-260310
Coding Agent Conversation Logs
This is a performance art project. Anthropic built their models on the world's freely shared information, then introduced increasingly dystopian data policies to stop anyone else from doing the same with their data — pulling up the ladder behind them. DataClaw lets you throw the ladder back down. The dataset it produces is yours to share.
Exported with DataClaw.
Tag: dataclaw — Browse all DataClaw datasets
Stats
Metric
Value… See the full description on the dataset page: https://huggingface.co/datasets/nixjoe/vue2egg-260310.Cat-v2.8
Cat-v2.8
A fine-tuning dataset for training language models to embody a warm, knowledgeable catgirl persona.
The model learns a style and personality, not a fixed name — it can adopt any catgirl name when prompted.
Description
Base dataset (~81,000 entries). Broad catgirl persona training across science, math, history, tech, coding, emotions, and cat-specific topics.
Dataset Details
Property
Value
Entries
81,000
Format
Chat messages (system / user… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/Cat-v2.8.beir-eval-hard-negatives
BEIR/MTEB hard negatives dataset
A dataset for quick evaluation of embedding models during their training.
The problem: running a full MTEB evaluation on a single GPU may take 10-20 hours. Most of this time is spent on embedding all 30M docs in all 10+ corpora. This dataset solves this problem by unwrapping a "retrieval" style benchmark into the "reranking" style:
We compute embeddings for all documents in the corpora with the intfloat/e5-base-v2 model.
For each corpus in BEIR/MTEB… See the full description on the dataset page: https://huggingface.co/datasets/nixiesearch/beir-eval-hard-negatives.cat-v3xl
🐱 cat-v3xl (XL Extended)
Part of the cat-v3 dataset family — synthetic instruction-tuning data that teaches models to be
helpful, accurate, and delightfully cat-flavoured.
About cat-v3xl (XL)
The XL variant contains 200,000 examples drawn from the full expanded topic bank — significantly broader than the base set, covering niche topics like WebAssembly, formal methods, category theory, condensed matter physics, film theory, and more.
Use this variant when you want… See the full description on the dataset page: https://huggingface.co/datasets/Nix-ai/cat-v3xl.