datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
quantized-llama-3.1-leaderboard-v2-evals
Open LLM Leaderboard v2 Benchmark Results
This artifact contains all the data from evaluations of Neural Magic's quantized Llama-3.1 models.
These evaluations were produced with lm-evaluation-harness by running the following command:
lm_eval \
--model vllm \
--model_args pretrained="<model_path>",dtype=auto,add_bos_token=False,max_model_len=4096,tensor_parallel_size="<num_gpus>",gpu_memory_utilization=0.8,enable_chunked_prefill=True \
--apply_chat_template \… See the full description on the dataset page: https://huggingface.co/datasets/neuralmagic/quantized-llama-3.1-leaderboard-v2-evals.quantized-retrieval-dataimagenet-sdxl-quantized
ImageNet SDXL Quantized
This repository provides the ImageNet-1K dataset pre-encoded with the Stable Diffusion XL VAE encoder and quantized to uint8, allowing for faster training of latent diffusion models by eliminating the need for on-the-fly encoding.
Key Features
Reduces quantization error by 2dB PSNR compared to a linear encoding scheme
Provided in both 256 and 512 resolutions
Compatible with NumPy, JAX, and PyTorch
Usage
Loading the dataset… See the full description on the dataset page: https://huggingface.co/datasets/jon-kyl/imagenet-sdxl-quantized.ToneWebinars_quantized-bigcodecdaily-paper-2026-08-20-quantized-embedding-skill-router
Quantizing the Gatekeeper: Accuracy-Cost Tradeoffs of Compressing the Embedding Model in Hybrid Skill Routers
TL;DR — Quantizing the dense embedding half of a hybrid (BM25 + embedding) skill router to INT8 leaves fused hit@1/hit@3 unchanged on a multilingual stand-in encoder because the lexical term dampens the error, but an off-the-shelf dynamic quantization backend made single-query encoding slower rather than faster on CPU, failing the latency side of the compression's own… See the full description on the dataset page: https://huggingface.co/datasets/thaki-AI/daily-paper-2026-08-20-quantized-embedding-skill-router.details_dball__zephyr-tiny-sft-qlora-quantized-2
Dataset Card for Evaluation run of dball/zephyr-tiny-sft-qlora-quantized-2
Dataset automatically created during the evaluation run of model dball/zephyr-tiny-sft-qlora-quantized-2 on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_dball__zephyr-tiny-sft-qlora-quantized-2.quantized_burgers_vqmair-floating-point-vs-quantized-benchmarks
Dataset Coverage
The benchmarks evaluate performance across 14+ specialized datasets covering:
Legal & Regulatory: ACORDAR, AILA2019-Case, AILA2019-Statutes, LeCaRDv2, LegalQuAD, REGIR-EU2UK, REGIR-UK2EU.
Financial: ConvFinQA, FinanceBench, FinQA, FiQA, HC3Finance.
Medical & Clinical: NFCorpus.
General/API: Apple Documentation.
Metrics
NDCG@10: Normalized Discounted Cumulative Gain at rank 10, measuring retrieval quality.
Latency (ms): Mean search latency measured… See the full description on the dataset page: https://huggingface.co/datasets/moorcheh/mair-floating-point-vs-quantized-benchmarks.ggml-vicuna-v0-quantizedThese are quantized ggml binary files for vicuna 7B and 13B models. The version of vicuna for these models are v0.
These files can be used in conjunction with minigpt4 ggml models 7B and 13B in minigpt4.cpp
Recommended are the Q5_K and Q6_K implementations. If there are any issues, use Q4_1 or Q4_0.
Vicuna Model Card
Model details
Model type:
Vicuna is an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT.
It is… See the full description on the dataset page: https://huggingface.co/datasets/maknee/ggml-vicuna-v0-quantized.MNLP_M3_quantized_datasetdetails_fzzhang__Marcoroni-neural-chat-7B-v2_gsm8k_quantized_mergedfloat_s
Dataset Card for Evaluation run of fzzhang/Marcoroni-neural-chat-7B-v2_gsm8k_quantized_mergedfloat_s
Dataset automatically created during the evaluation run of model fzzhang/Marcoroni-neural-chat-7B-v2_gsm8k_quantized_mergedfloat_s on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_fzzhang__Marcoroni-neural-chat-7B-v2_gsm8k_quantized_mergedfloat_s.MNLP_M3_quantized_datasetquantized-llama-3.1-humaneval-evals
Coding Benchmark Results
The coding benchmark results were obtained with the EvalPlus library.
HumanEvalpass@1
HumanEval+pass@1
meta-llama_Meta-Llama-3.1-405B-Instruct
67.3
67.5
neuralmagic_Meta-Llama-3.1-405B-Instruct-W8A8-FP8
66.7
66.6
neuralmagic_Meta-Llama-3.1-405B-Instruct-W4A16
66.5
66.4
neuralmagic_Meta-Llama-3.1-405B-Instruct-W8A8-INT8
64.3
64.8
neuralmagic_Meta-Llama-3.1-70B-Instruct-W8A8-FP8
58.1
57.7
neuralmagic_Meta-Llama-3.1-70B-Instruct-W4A16
57.1… See the full description on the dataset page: https://huggingface.co/datasets/neuralmagic/quantized-llama-3.1-humaneval-evals.MNLP_M2_quantized_dataset1quantized-wikipedia-indicesMNLP_M2_quantized_dataset
Dataset Card for MNLP_M2_sft_dataset
Dataset Description
A unified STEM instruction-following dataset comprising 240,500 examples drawn from six existing benchmarks: SciQ, Deepmind Code Contests, TIGER-Lab MathInstruct, TULU Algebra, TULU Code, and Facebook Natural Reasoning. Each example is formatted as a chat-style message pair for supervised fine-tuning of instruction-following models.
Curated by: Sarra Chabane
Shared by: GingerBled (https://huggingface.co/GingerBled)… See the full description on the dataset page: https://huggingface.co/datasets/arthurrpp/MNLP_M2_quantized_dataset.ToneBooksPlus_quantized-bigcodecemilia_en_quantized-wav-unifydetails_FabbriSimo01__Facebook_opt_1.3b_Quantized
Dataset Card for Evaluation run of FabbriSimo01/Facebook_opt_1.3b_Quantized
Dataset Summary
Dataset automatically created during the evaluation run of model FabbriSimo01/Facebook_opt_1.3b_Quantized on the Open LLM Leaderboard.
The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_FabbriSimo01__Facebook_opt_1.3b_Quantized.quantized-llama-3.1-arena-hard-evals
Arena-Hard Benchmark Results
This artifact contains all the data neccessary to reproduce the results of the Arena-Hard benchmark for Neural Magic's quantized Llama-3.1 models.
The model_answers directory includes the generated answers from all models, and the model_judgements directory contains the evaluations by gpt-4-1106-preview.
The Arena-Hard version used for benchmarking is v0.1.0, corresponding to commit efc012e192b88024a5203f5a28ec8fc0342946df.
All model answers were… See the full description on the dataset page: https://huggingface.co/datasets/neuralmagic/quantized-llama-3.1-arena-hard-evals.details_FabbriSimo01__Cerebras_1.3b_Quantized
Dataset Card for Evaluation run of FabbriSimo01/Cerebras_1.3b_Quantized
Dataset Summary
Dataset automatically created during the evaluation run of model FabbriSimo01/Cerebras_1.3b_Quantized on the Open LLM Leaderboard.
The dataset is composed of 3 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_FabbriSimo01__Cerebras_1.3b_Quantized.quantized_wiki_faissMNLP_M2_quantized_dataset
MNLP M2 Quantized MCQA Dataset
Train split of prompt/completion examples with an extra dataset column indicating source.
Column
Type
Description
prompt
string
The input question prompt
completion
string
The ground-truth answer
dataset
string
Source label (e.g. scienceqa, M1_chatgpt, qasc, mathqa, commonsenseqa, openbookqa)
ToneSlavic_quantized-bigcodecquantized-common-voice-enMNLP_M2_quantized_datasetMNLP_M3_quantized_datasetquantized_modelsMNLP_M2_quantized_datasetMNLP_M2_quantized_dataset