knows
Datasets
All datasets matching “knows”knows-benchmark
KNOWS Benchmark
KNOWS evaluates web agents on the work people actually do in Google Workspace: writing documents,
building spreadsheets, and composing slide decks that require web research, multi-step tool use, and
faithful grounding in retrieved sources.
This dataset contains the task definitions — the prompt an agent receives, plus the structured
evaluation rubric used to grade the artifact it produces.
Tasks
110 (22 templates × 5 instances)
Domains
20
Mean… See the full description on the dataset page: https://huggingface.co/datasets/utahnlp/knows-benchmark.know_sqlplease use the val ign file for training, its much cleaner. thanks :)
factprobe-replication-exposure-knows-trajectory-v1
factprobe-replication-exposure-knows-trajectory-v1
Does EXACT per-checkpoint cumulative corpus exposure predict whether OLMo-2 KNOWS a fact? Two measures per (checkpoint, relation): PAIRED (P(Yes|true) > P(Yes|hard-negative), the 'beats' defs) and MARGINAL (P(Yes|true), r2_p_true + per-feature Spearman). Checkpoints: after-phase1, base(s1+s2), SFT, DPO, RLVR + 7 pre-merge s2-anneal ingredients; both models; both relations (P26 spouse, P3373 sibling); surface/name level; all 7… See the full description on the dataset page: https://huggingface.co/datasets/latkes/factprobe-replication-exposure-knows-trajectory-v1.factprobe-replication-hardneg-knows-labels-surface-v1
factprobe-replication-hardneg-knows-labels-surface-v1
NAME-level (surface-form) paired 'knows the fact' judgements of OLMo-2 against hard negatives, across the training ladder. One row per subject NAME form (not the entity aggregate). subject_name held fixed; object side is the any-of max. Forward direction; spouse (P26) and sibling (P3373).
Dataset Info
Rows: 1299300
Columns: 11
Columns
Column
Type
Description
tag
Value('string')
7b or… See the full description on the dataset page: https://huggingface.co/datasets/latkes/factprobe-replication-hardneg-knows-labels-surface-v1.KnowSelf-Train-DataThis repository contains the data presented in Agentic Knowledgeable Self-awareness.
Code: https://github.com/zjunlp/KnowSelf
factprobe-replication-hardneg-knows-labels-v1
factprobe-replication-hardneg-knows-labels-v1
Paired 'knows the fact' judgements of OLMo-2 (7B, 13B) against GOOD hard negatives, across the training ladder. One row per (model, stage, relation, phrasing, true pair): P(Yes) on the true pair, best P(Yes) on the hardest hard negative, and beats_all (does the model prefer the true partner). Forward direction; spouse (P26) and sibling (P3373).
Dataset Info
Rows: 284260
Columns: 10
Columns
Column… See the full description on the dataset page: https://huggingface.co/datasets/latkes/factprobe-replication-hardneg-knows-labels-v1.