datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
eval-Qwen3-32B-HLEtest-datachain-llm-evalpeacock-data-public-datasets-idc-llm_evalhle_eval_datasettest_eval_hle生徒モデル選定目的でtakaさん評価コード（運営から提供されたコードを動作するように変更版）で評価してみた結果（佐々木分）です。https://www.notion.so/23de14b94af5807a87b5c6292a3d3520https://matsuokenllmcommunity.slack.com/archives/C095PS80YTE/p1753979349212189?thread_ts=1753437729.053739&cid=C095PS80YTE
<predictionsフォルダ>素の Qwen3で hle_eval推論した結果hle_Qwen3-14B.jsonhle_Qwen3-32B.jsonhle_Qwen3-235B-A228B.json(441問回答版)、この後に追加で100問程度推論したファイルも別途あり（判定は未だ行っていない）（再度、推論するとpredictionsフォルダに入っている推論済の問題以外を追加で推論してくれる動作になっている）
<judgedフォルダ>素の Qwen3で… See the full description on the dataset page: https://huggingface.co/datasets/llm-compe-2025-kato/test_eval_hle.llm-eval-flakiness-trajectories
Llm Eval Flakiness Trajectories
Rights & intended use: legacy public research corpus / portfolio
artifact. Hosted frontier-model outputs are research-only inputs under
project policy (synthetic-factory#161):
intended_use: research_only, project_training_policy: blocked. Not
training data for any model-weight update. Machine-readable record:
rights.json.
Release status: The raw, uncurated payload is now published under
data/raw/. It is available for inspection and… See the full description on the dataset page: https://huggingface.co/datasets/rmems/llm-eval-flakiness-trajectories.eval-result-Qwen3-32B-HLE
neko-llm/eval-result-Qwen3-32B-HLE
Judged HLE evaluation results uploaded via script.
Summary metrics
Total examples: 2129
Accuracy: 7.4213% ± 1.1134%
Average confidence: 93.9%
Calibration error (L2): 83.6859%
Models
neko-llm/Qwen3-32B-HLE
Columns
id
model
response
usage (JSON string)
judge_correct_answer
judge_model_answer
judge_reasoning
judge_correct
judge_confidence
judge_response (raw JSON)
Generated by evaluation/hle/upload_eval_result.py
serbian-llm-eval-v1
Serbian LLM eval 🇷🇸
This dataset should be used for Serbian (and potentially also other HBS languages) LLM evaluation.
Here is the GitHub project used to build this dataset.
For technical report of the project see this in-depth Weights & Biases report. ❤️
I'll give a TL;DR here:
What is covered?
Common sense reasoning:
Hellaswag, Winogrande, PIQA, OpenbookQA, ARC-Easy, ARC-Challenge
World knowledge:
NaturalQuestions, TriviaQA
Reading comprehension:
BoolQ… See the full description on the dataset page: https://huggingface.co/datasets/gordicaleksa/serbian-llm-eval-v1.llm-refusal-evaluation
🛡️ LLM Refusal Evaluation Benchmark
This repository contains the benchmarks used in the LLM-Refusal-Evaluation suite.
The prompts are organized into three groups:
Safety Benchmarks — harmful / jailbreak-style prompts that models should refuse.
Chinese Sensitive Topics — prompts that may be censored by China-aligned models.
Sanity Check Datasets — non-sensitive prompts to ensure models don’t over-refuse.
📌 Contents
Safety Benchmarks
JailbreakBench
SorryBench… See the full description on the dataset page: https://huggingface.co/datasets/MultiverseComputingCAI/llm-refusal-evaluation.llm-sfm-safety-eval
LLM x SFM Safety Evaluation
When a general-purpose language model interprets the output of a specialist
science foundation model (a protein, genomic, RNA, or chemistry model), does its
safety behavior recognize the scientific content, or only the surface form of the
request?
This repository is the empirical core of a study of that question: the evaluation
harness, the redacted aggregate results, and the measurement specifications behind
four findings about how deployed Claude… See the full description on the dataset page: https://huggingface.co/datasets/jang1563/llm-sfm-safety-eval.llm-instruction-following-eval
LLM Instruction-Following Evaluation: 256 Models Across 20 Diagnostic Tests
Dataset Summary
This dataset contains comprehensive evaluation results from testing 256 Large Language Models across 20 carefully designed diagnostic instruction-following prompts, totaling 5,120 individual evaluations. The evaluation was conducted on October 14, 2025, using the OpenRouter API.
Paper: When Models Can't Follow: Testing Instruction Adherence Across 256 LLMs
arXiv: 2510.18892… See the full description on the dataset page: https://huggingface.co/datasets/richardyoung/llm-instruction-following-eval.llm-jp-eval
Dataset Card for llm-book/ja-vicuna-qa-benchmark
書籍『大規模言語モデル入門Ⅱ〜生成型LLMの実装と評価』で使用する、LLM-jpが提供する「llm-jp-eval」で使用されるデータセットです。
オリジナルのサイトと同じものを使用しています。
本データセットは、複数のデータセットを横断して日本語の大規模言語モデルを自動評価するにの使用されます。
Licence
Apache License, Version 2.0
slovenian-llm-eval
Slovenian LLM Evaluation Dataset
This dataset is designed for evaluating Slovenian language models and builds upon the work of gordicaleksa/slovenian-llm-eval-v0 which translated some of the popular English benchmarks into Slovenian by using Google Translate. We have further improved the quality of the Slovenian translations.
The dataset contains the following benchmarks:
ARC Challenge
ARC Easy
BoolQ
GSM8K
HellaSwag
NQ Open
OpenBookQA
PIQA
TriviaQA
TruthfulQA
Winogrande… See the full description on the dataset page: https://huggingface.co/datasets/cjvt/slovenian-llm-eval.llm-ensembles-13-inspect-evals-validationllm-jp-evalllm-eval-massive_intentllm-based-expansions-eval-datasetsmacedonian-llm-eval
Macedonian LLM Eval
This repository is adapted from the original work by Aleksa Gordić. If you find this work useful, please consider citing or acknowledging the original source.
You can find the Macedonian LLM eval on GitHub. To run evaluation just follow the guide.
Info: You can run the evaluation for Serbian and Slovenian as well, just swap Macedonian with either one of them.
What is currently covered:
Common sense reasoning: Hellaswag, Winogrande, PIQA… See the full description on the dataset page: https://huggingface.co/datasets/LVSTCK/macedonian-llm-eval.serbian-llm-eval-v0
Serbian LLM eval v0 🇷🇸
Please instead use the version 1 of the dataset here.
Weights & Biases report.
Project Sponsors
Platinum sponsors 🌟
Ivan (fizicko lice, anoniman)
Gold sponsors 🟡
qq (fizicko lice, anoniman)
Mitar Perovic
Nikola Ivancevic
Silver sponsors ⚪
psk.rs, OmniStreak, Marko Radojicic, Luka Vazic, Miloš Durković, Marjan Radeski, Marjan Stankovic (fizicko lice), Nikola Stojiljkovic, Mihailo Tomic, Bojan Jevtic, Jelena… See the full description on the dataset page: https://huggingface.co/datasets/gordicaleksa/serbian-llm-eval-v0.llm-eval-massive_scenarioua-llm-router-eval
UA Specialist Router — evaluation progress
Score tables, routing stats, and run metadata from the diploma project
MariaOnyshchuk/ua-llm-router:
a rules-based router over open Ukrainian specialists (Mamay-4B, Lapa-12B, Aya Expanse 8B, Qwen-Coder).
This dataset is the progress log of pinned JSON summaries, not a dump of every generation.
What is included
Path
Contents
progress_ledger.csv
Flattened metric rows across weeks (best table for browsing)… See the full description on the dataset page: https://huggingface.co/datasets/MariaOnyshchuk/ua-llm-router-eval.llm_eval_promptsprivesc-llm-evals
PrivEsc-LLM Evaluations
This evaluation artifact contains compact evidence for the paper's reported results.
Source repository: https://github.com/sailab-vienna/privesc-llm
Layout
summaries/runs/paper/01_sft_hyperparam_sweep/: SFT LR/rank/seed selection tables, manifests, training stats, and recorded procedural evaluation summaries
summaries/runs/paper/02_sft_trace_design/: aggregate trace-design summary and source manifest
summaries/runs/paper/03_reward_ladder/:… See the full description on the dataset page: https://huggingface.co/datasets/sailab-vienna/privesc-llm-evals.llm-eval-imdbllm-eval-mtop_domainllm-eval-banking77llm-fol-reasoning-eval
LLM FOL Reasoning Eval
This dataset is derived from ProverQA, a First-Order Logic reasoning benchmark designed to test the ability of large language models (LLMs) to perform structured logical reasoning.It restructures and normalizes the ProverQA development and training data into a unified, clean format suitable for evaluating chain-of-thought (CoT) and symbolic reasoning capabilities in LLMs.
Source
Original dataset: ProverQA: A First-Order Logic Reasoning… See the full description on the dataset page: https://huggingface.co/datasets/MinaGabriel/llm-fol-reasoning-eval.llm-eval-amazon_counterfactualevalita_llm_resultsslovenian-llm-eval-v0
Slovenian LLM eval 🇸🇮
This dataset should be used for Slovenian LLM evaluation.
Here is the GitHub project used to build this dataset.
For technical report of the project see this in-depth Weights & Biases report. ❤️ Even though this one was written for Serbian LLM eval the same process was used to build Slovenian LLM eval.
I'll give a TL;DR here:
What is covered?
Common sense reasoning:
Hellaswag, Winogrande, PIQA, OpenbookQA, ARC-Easy, ARC-Challenge
World… See the full description on the dataset page: https://huggingface.co/datasets/gordicaleksa/slovenian-llm-eval-v0.