datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
signature-to-mechanisms
Signature-to-Mechanisms (S2M)
Signature-to-Mechanisms (S2M) provides standardized tasks designed to enable the assessment of mechanistic reasoning in AI agents. Each task supplies the elements necessary for evaluation, including experimental context, molecular signatures, and task prompts, so that agents can be tested on their ability to reconstruct mechanistic explanations reported in peer-reviewed biological studies.
S2M formalizes a core challenge in computational biology:… See the full description on the dataset page: https://huggingface.co/datasets/vida-nyu/signature-to-mechanisms.shadow-llm-mia-signals
Shadow LLM MIA Signals (OLMo-2-1B)
Membership Inference Attack (MIA) signal tensors extracted from 128 shadow models
fine-tuned from allenai/OLMo-2-0425-1B.
Overview
This dataset enables research on membership inference attacks against large language models.
Each of 128 shadow models was trained on a different random subset of 64 out of 128 candidate
documents from the OLMo-mix-1124 pretraining dataset.
For each (model, document) pair, we extracted softmax prediction… See the full description on the dataset page: https://huggingface.co/datasets/matthewwicker/shadow-llm-mia-signals.e15-context-budget
SignalDepth E15 Context Budget
This is a small prompt-sensitivity benchmark slice for separating two explanations that often get conflated:
the prompt is too short
the task contract is underspecified
The narrow result: on this deterministic Python code-task suite, making sparse prompts longer did not help. Making the task contract explicit did.
Key Result
Condition
Average pass rate
Read
short_sparse
0.25
short and underspecified
long_sparse
0.25
longer… See the full description on the dataset page: https://huggingface.co/datasets/signaldepth/e15-context-budget.