datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Magpie-Reasoning-V2-250K-CoT-Llama3
Project Web: https://magpie-align.github.io/
Arxiv Technical Report: https://arxiv.org/abs/2406.08464
Codes: https://github.com/magpie-align/magpie
Abstract
Click Here
High-quality instruction data is critical for aligning large language models (LLMs). Although some models, such as Llama-3-Instruct, have open weights, their alignment data remain private, which hinders the democratization of AI. High human labor costs and a limited, predefined scope for prompting prevent… See the full description on the dataset page: https://huggingface.co/datasets/Magpie-Align/Magpie-Reasoning-V2-250K-CoT-Llama3.llama-3.1-medprm-reward-training-set
Med-PRM-Reward (Version 1.0)
🚀 Med-PRM-Reward is among the first Process Reward Models (PRMs) specifically designed for the medical domain. Unlike conventional PRMs, it enhances its verification capabilities by integrating clinical knowledge through retrieval-augmented generation (RAG). Med-PRM-Reward demonstrates exceptional performance in scaling-test-time computation, particularly outperforming majority‐voting ensembles on complex medical reasoning tasks. Moreover, its… See the full description on the dataset page: https://huggingface.co/datasets/dmis-lab/llama-3.1-medprm-reward-training-set.luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags-pausefilled
luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags-pausefilled
Every non-first chunk of every document carries a thought: the gpt-5.6-luna reasoning thought
where one was generated, and a content-free pause thought everywhere else.
luna chunk <|reserved_special_token_1|> luna reasoning <|reserved_special_token_2|>
filler chunk <|reserved_special_token_1|> 256x <|reserved_special_token_0|> <|reserved_special_token_2|>
The filler is 258 tokens. Chunk 0 is excluded… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags-pausefilled.luna-reason-only.k-8.statml-arxiv-llama32
luna-reason-only.k-8.statml-arxiv-llama32
Prefix-only "thoughts" for next-token prediction on stat.ML arXiv LaTeX. Each thought is
visible reasoning about the next 8 Llama-3.2 tokens after a cut, written without ever
seeing that continuation. Intended to be spliced into the document before the chunk so a
small model (Llama 3.2 3B base) can read the reasoning and predict the chunk.
Complete: every designated chunk has a thought.
split
thoughts
coverage
documents… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.llama-3.1-8b-funding-extraction-sft-ablations
LLaMA 3.1 8B Funding Extraction SFT Ablations
Ablation study results for LoRA SFT of Meta LLaMA 3.1 8B Instruct on structured funding metadata extraction from scholarly text.
The model extracts four fields: funder_name, award_ids, funding_scheme, and award_title.
Key findings
Factor
Best config
Avg F1
Overall best
synthetic, twostage (2+1 epochs), LoRA r=64, lr=3e-5
0.588
Data type
Synthetic >> non-synthetic (+0.126 avg F1)
—
LoRA rank
r=64 > r=32 > r=16
—… See the full description on the dataset page: https://huggingface.co/datasets/cometadata/llama-3.1-8b-funding-extraction-sft-ablations.luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags-echo
luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags-echo
Tokenized, tag-wrapped form of JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.
Each thought is wrapped as <|reserved_special_token_1|> … thought … <|reserved_special_token_2|> … last prefix token and stored both as text (thought_text) and as Llama 3.2
token ids (input_ids).
The trailing token is the document token immediately before the cut (input_ids[chunk_start_index - 1]), copied from the document rather… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags-echo.luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.kv-tags
luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.kv-tags
Tokenized, tag-wrapped form of JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.
Each thought is wrapped as
<|reserved_special_token_1|>
KEY: <last 8 prefix tokens>
VALUE: <thought>
<|reserved_special_token_2|>
and stored both as text (thought_text) and as Llama 3.2
token ids (input_ids).
Longest thought: 514 tokens — a training run's max_thought_length must be at least
this.
Intended to be PREPENDED to the… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.kv-tags.luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.kv-tags-explained
luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.kv-tags-explained
Tokenized, tag-wrapped form of JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.
Each thought is wrapped as
<|reserved_special_token_1|>
This is a hint about a span that appears later in this document. KEY is the text immediately before that span; VALUE is a note about what might come next.
KEY: <last 8 prefix tokens>
VALUE: <thought>
<|reserved_special_token_2|>
and stored both as text (thought_text)… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.kv-tags-explained.Alpaca-Llama3.1-KD
Dataset Card for Alpaca-Llama3.1-KD
This dataset was introduced in the paper SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices.
The official code repository can be found here: ernlavr/SigmaScale.
Dataset Summary
This dataset is a distilled version of the classic tatsu-lab/alpaca dataset. It utilizes Meta-Llama-3.1-8B-Instruct as an answer generation model to generate high-quality, instruction-following responses for… See the full description on the dataset page: https://huggingface.co/datasets/ernlavr/Alpaca-Llama3.1-KD.luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags
luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags
Tokenized, tag-wrapped form of JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.
Each thought is wrapped as <|reserved_special_token_1|> … thought … <|reserved_special_token_2|> and stored both as text (thought_text) and as Llama 3.2
token ids (input_ids).
Provenance
Source thoughts: JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32 — prefix-only reasoning about the next 8 Llama-3.2 tokens of
stat.ML… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.k-8.statml-arxiv-llama32.llama32-ids.tags.llama-3.1-8b-instruct_aime-all
meta-llama/Llama-3.1-8B-Instruct — aime-all
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: aime-all (933 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 32768
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt application)… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_aime-all.statML-arxiv-40M-20M-llama32
statML-arxiv-40M-20M-llama32
A Llama 3.2-tokenized re-windowing of JackHsieh/statML-arxiv-40M-20M
(originally tokenized with Qwen3), the Llama analogue of
JackHsieh/statML-arxiv-40M-20M-olmo3.
Each row is a contiguous span of exactly 4096 tokens under the Llama 3.2 tokenizer
(meta-llama/Llama-3.2-3B, byte-identical to the 1B tokenizer, vocab 128 256), anchored at the same
character offset as the corresponding Qwen3 window of the same paper.
train: 9,728 sequences (39.8M tokens)… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/statML-arxiv-40M-20M-llama32.llama-3.1-medprm-reward-training-set
Med-PRM-Reward (Version 1.0)
🚀 Med-PRM-Reward is among the first Process Reward Models (PRMs) specifically designed for the medical domain. Unlike conventional PRMs, it enhances its verification capabilities by integrating clinical knowledge through retrieval-augmented generation (RAG). Med-PRM-Reward demonstrates exceptional performance in scaling-test-time computation, particularly outperforming majority‐voting ensembles on complex medical reasoning tasks. Moreover, its… See the full description on the dataset page: https://huggingface.co/datasets/jysyoh/llama-3.1-medprm-reward-training-set.llama-3.1-8b-instruct_arena-hard-creative-writing
meta-llama/Llama-3.1-8B-Instruct — arena-hard-creative-writing
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: arena-hard-creative-writing (250 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_arena-hard-creative-writing.llama-3.1-8b-instruct_ifeval
meta-llama/Llama-3.1-8B-Instruct — ifeval
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: ifeval (541 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt application)… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_ifeval.llama-3.1-8b-instruct_bookmia-label0-5pct-raw
meta-llama/Llama-3.1-8B-Instruct — bookmia-label0-5pct-raw
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: bookmia-label0-5pct-raw (247 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_bookmia-label0-5pct-raw.llama-3.1-8b-instruct_tinystories-val1pct-raw
meta-llama/Llama-3.1-8B-Instruct — tinystories-val1pct-raw
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: tinystories-val1pct-raw (220 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_tinystories-val1pct-raw.llama-3.1-8b-instruct_writingbench-en100
meta-llama/Llama-3.1-8B-Instruct — writingbench-en100
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: writingbench-en100 (100 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 8192
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_writingbench-en100.llama-3.1-8b-instruct_creativemath-with-answers
meta-llama/Llama-3.1-8B-Instruct — creativemath-with-answers
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: creativemath-with-answers (188 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 32768
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_creativemath-with-answers.llama-3.1-8b-instruct_storygen-prompts-200
meta-llama/Llama-3.1-8B-Instruct — storygen-prompts-200
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: storygen-prompts-200 (200 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_storygen-prompts-200.llama-3.1-8b-instruct_alpaca-text-generation-384
meta-llama/Llama-3.1-8B-Instruct — alpaca-text-generation-384
Model outputs from the micro-creativity inference suite.
Model: meta-llama/Llama-3.1-8B-Instruct
Dataset: alpaca-text-generation-384 (384 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/llama-3.1-8b-instruct_alpaca-text-generation-384.