datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ModalityFaultLines-SCEval
SCEval — Modality Fault Lines
Data for Modality Fault Lines: Structural Corruptions Reveal Fragile Omni-Modal Reasoning (Findings of EMNLP 2026).
SCEval is a human-verified benchmark for omni-modal robustness. Text, vision, and audio all remain
present, but controlled corruptions make the evidence inside a channel unreliable. Each corrupted
item is paired with its clean counterpart at the example level, so clean-to-corrupted comparisons are
made on the same underlying question… See the full description on the dataset page: https://huggingface.co/datasets/KZL96/ModalityFaultLines-SCEval.frontier-csautoinference-agentic-mix-v1
Autoinference Agentic Mix v1
This is a prompt set for the online_agentic serving benchmark. That profile stands
in for long-horizon agent traffic: a large context that grows turn over turn, with
short structured outputs at each step. The usual way to run it uses
generated-shared-prefix, which builds a synthetic shared prefix out of random tokens.
This dataset uses real agent trajectories instead, so the prefix reuse, the context
growth, and the token mix all match what an agent… See the full description on the dataset page: https://huggingface.co/datasets/modal-labs/autoinference-agentic-mix-v1.swebenchproautoinference-agentic-mix-v2
Autoinference Agentic Mix v2
300 real SWE-agent trajectories from
TIGER-Lab/SWE-Next-SFT-Trajectories,
expanded into one request per assistant turn. Each row carries the conversation
up to that turn and the model generates the turn. Replaying a trajectory in
turn_index order re-sends a growing prefix, which is how an agent loop
actually hits a prefix cache.
What changed from v1
v1 kept only requests with at least 34k prefix tokens. That cut trajectories
down to… See the full description on the dataset page: https://huggingface.co/datasets/modal-labs/autoinference-agentic-mix-v2.GatewayBench-v1
Dataset Card for GatewayBench v1
Dataset Summary
GatewayBench v1 is a synthetic benchmark dataset for evaluating LLM gateway systems and routing decisions. It provides 2,000 test cases with ground truth labels across four distinct task types, each designed to test different aspects of gateway performance: tool selection from large sets (tool-heavy), information retrieval (retrieval), pure conversation (chat), and high-complexity scenarios (stress).
Key Features:
Ground… See the full description on the dataset page: https://huggingface.co/datasets/ModaLabs/GatewayBench-v1.usacoshared-emergence-icl-modalities-128
Shared-emergence ICL replication at T=128
This dataset contains the complete raw result archive for the paper
“Many Next-Token Predictors are In-Context Learners.”
The campaign evaluates a fixed suite of 100 program-synthesis tasks using 128
sampled prompts per task, for every clean and deranged shot cell described by
the paper:
21 run keys;
281 experiment cells;
12,800 predictions per cell;
3,596,800 predictions in total.
The archive expands to a top-level results_128/… See the full description on the dataset page: https://huggingface.co/datasets/N8Programs/shared-emergence-icl-modalities-128.swebench-multilingualmodality-routing-dataset
Modality Routing Dataset
This dataset materializes the dynamic modality routing data builder used by the local
mmBERT-32K modality router training pipeline. The export is intended for review,
versioning, and uploading to a Hugging Face dataset repository.
Labels
Label
ID
Description
AR
0
Text-only requests that should route to an autoregressive LLM.
DIFFUSION
1
Image-generation requests that should route to a diffusion model.
BOTH
2
Requests that benefit… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/modality-routing-dataset.modal-semantics-reasoning
Modal Semantics Reasoning
Can a language model change its answer when the rules of modal logic change?
Each example contains the same premises and conclusion under two semantic
specifications. Only one rule about possible worlds or objects changes, and
the correct answer changes with it. Automated theorem provers verify every
label.
This dataset accompanies Same Formulas, Different Semantics: Do Language
Models Follow Modal Logic Specifications?
Dataset subsets… See the full description on the dataset page: https://huggingface.co/datasets/sileod/modal-semantics-reasoning.MixBench
MixBench: A Benchmark for Mixed Modality Retrieval
MixBench is a benchmark for evaluating retrieval across text, images, and multimodal documents. It is designed to test how well retrieval models handle queries and documents that span different modalities, such as pure text, pure images, and combined image+text inputs.
MixBench includes four subsets, each curated from a different data source:
MSCOCO
Google_WIT
VisualNews
OVEN
Each subset contains:
queries.jsonl: each entry… See the full description on the dataset page: https://huggingface.co/datasets/mixed-modality-search/MixBench.FW_EDU_SUBSET_500k_docs
FineWeb-Edu Subset
This dataset contains 483,606 documents sampled from the FineWeb-Edu dataset.
The dataset is used throughout various tutorials on modalities.
For licensing, see their conditions.
MoD-Alpacamodalitieshttps://connect.helmholtz-imaging.de/
sensory-modality-ratingsmodal-vllm-cache-h200-minimax-v43autoinference-realtime-mix-v1
Autoinference Real-Time Generation Mix v1
This is a prompt set for the real_time_generation serving benchmark. That profile
stands in for medium-context, single-shot interactive traffic: roughly 3000 input
tokens, 100 output tokens, one request at a time with no shared context between
requests. The usual way to run it feeds the server random token IDs of a fixed
length. This dataset keeps the same input and output shape but uses real prompts.
The reason real text matters: random… See the full description on the dataset page: https://huggingface.co/datasets/modal-labs/autoinference-realtime-mix-v1.