datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
tests-raw-jsonlraw_jsonlner-jsonlformat-jsonldoc-formats-jsonl-1
[doc] formats - jsonl - 1
This dataset contains one jsonl file at the root.
hermes-function-calling-v1-jsonl
Hermes Function-Calling V1
This dataset is the compilation of structured output and function calling data used in the Hermes 2 Pro series of models.
This repository contains a structured output dataset with function-calling conversations, json-mode, agentic json-mode and structured extraction samples, designed to train LLM models in performing function calls and returning structured output based on natural language instructions. The dataset features various conversational scenarios… See the full description on the dataset page: https://huggingface.co/datasets/minpeter/hermes-function-calling-v1-jsonl.stackexchange_title_body_jsonljsonl.gz format from https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml
Each line contains a dict in the format: {"text": ["title", "body"], "tags": ["tag1", "tag2"]}
The following parameters have been used for filtering: min_title_len = 20 min_body_len = 20 max_body_len = 4096 min_score = 0
If a stackexchange contained less than 10k questions (after filtering), it is written to the small_stackexchanges.jsonl.gz file.
This is a dump of the files from… See the full description on the dataset page: https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_title_body_jsonl.step35-en2pl-conv-pass4-jsonlconversations: 1,251,034
chat-template tokens (role+content, incl. special tokens): 2,664,206,408
reasoning_content tokens (not covered by chat template, counted separately): 6,662,763,429
avg tokens/conversation: 2129.6
used tokenizer: APT4
step35-en2pl-conv-pass5-jsonlstep35-en2pl-conv-pass2-jsonlstep35-en2pl-conv-pass7-jsonlswe_v0.1_jsonl_wo_mlang_large100_wo_v0.0_deltagjsonl-mls-hubert_large_ll60k-layer_22south-african-monolingual-corpora-jsonl
South African Languages Pretraining Dataset
This dataset contains pretraining text data for 9 South African languages, compiled from multiple sources including CC100, Glot500, mC4, ParaCrawl, and various corpora collections.
The datasets were gathered as part of the University of Cape Town's SALLM project. Where data was gathered from multiple sources, extensive filtering and deduplication was conducted to ensure dataset integrity
Languages Included
Language… See the full description on the dataset page: https://huggingface.co/datasets/SimbaMaw1547/south-african-monolingual-corpora-jsonl.claude-code-fable-5-traces.jsonlsuper cleaned
from: Glint-Research/Fable-5-traces and armand0e/claude-fable-5-claude-code
bigbench_jsonlBIG-Bench but it doesn't require the hellish dependencies (tensorflow, pypi-bigbench, protobuf) of the official version.
dataset = load_dataset("tasksource/bigbench",'movie_recommendation')
Code to reproduce:
https://colab.research.google.com/drive/1MKdLdF7oqrSQCeavAcsEnPdI85kD0LzU?usp=sharing
Datasets are capped to 50k examples to keep things light.
I also removed the default split when train was available also to save space, as default=train+val.
@article{srivastava2022beyond… See the full description on the dataset page: https://huggingface.co/datasets/NJUDeepEngine/bigbench_jsonl.tests-public-raw-jsonlMagpie-Pro-DPO-200K-JSONLkbd-ru-jsonl-tmpinstruct_chat_50k.jsonlinstruct_chat_50k.jsonl which is composed of 30k Chinese sharegpt dataset and 20k alpaca-instruction-Chinese-dataset
claude-sonnet5-jsonl
Claude Sonnet 5 Dataset
This dataset was collected during a chat with Claude Sonnet 5.
Format:
{"instruction": "...", "output": "..."}
Note
Created/Collected by: mondk
YO, thanks to @mradermacher for taking an interest in my repo.
Don't forget to leave a like if you find this helpful!
--- Thank you!
poziomka-fun-v9-jsonlall_news_finance_jsonlTextual Time Series Dataset for finetuning / pretraining.
Json version of original dataset.
Original Dataset : https://huggingface.co/datasets/PaulAdversarial/all_news_finance_sm_1h2023
claude-v2-super.jsonl
Claude Dataset
This dataset was collected during a chat with Claude Sonnet 5/Fable 5.
Format:
{"instruction": "...", "output": "..."}
Note
Created/Collected by: mondk
Please give me credit when using or copying this dataset.
Don't forget to leave a like if you find this helpful!
Model I created: mondk/claude-sonnet5-base.qwen2.5code1.5b-it (you can refer to this)
--- Thank you!
pg19-jsonl
PG19 JSONL
This dataset is a JSONL conversion of the Hugging Face dataset
emozilla/pg19.
The source dataset is a parquet version of
pg19.
Dataset Structure
The dataset has three splits:
train: 28,602 rows in 23 JSONL shards
validation: 50 rows in 1 JSONL shard
test: 100 rows in 1 JSONL shard
Each line is a JSON object with the same fields as the source parquet dataset:
short_book_title: short title for the book
publication_date: publication year
url: Project Gutenberg… See the full description on the dataset page: https://huggingface.co/datasets/virtualkevin/pg19-jsonl.NSFW-Stories-JsonLConverted to JsonL from: bluuwhale/nsfwstory2
chatgpt-gpt-chat-jsonlData is collected during genuine chat with chatgpt.
and openai_gsm8k_0_7473.jsonl by openai, real: openai/gsm8k. It has been shortened.
ty
Qwen2.5-3B-jsonlcvedataset.jsonl
CVE Dataset (1999-2024) for LLM Fine-Tuning
Overview
This dataset comprises Common Vulnerabilities and Exposures (CVE) records spanning from 1999 to 2024. Each entry provides essential information on software vulnerabilities, their descriptions, affected products and versions, CVSS scores, and relevant references. The data is formatted in a JSON Lines (.jsonl) structure, making it suitable for fine-tuning Large Language Models (LLMs) for tasks such as cybersecurity… See the full description on the dataset page: https://huggingface.co/datasets/iamthierno/cvedataset.jsonl.lsv2-mathlib-v4.28.0-rc1-jsonl
LeanSearch v2 — Mathlib v4.28.0-rc1 corpus (JSONL)
One record per Mathlib v4.28.0-rc1 declaration with the LLM-generated informal description used as the embedding input — the raw source from which the companion cuVS index is built.
Code: https://github.com/frenzymath/LeanSearch-v2
Paper: https://arxiv.org/abs/2605.13137