CoolFace
Datasetpublic

vineet-datasets/probe-benchmark-hard50

PROBE hard50 — human-reviewed hard development set This is a separate standard-format review bank of 62 Astra-conditioned hard development episodes. It is NOT an independently evaluated holdout. Do not merge into or modify benchmark_600. The original 50 human-reviewed questions were augmented with 12 human-accepted inverse comparison prompts, yielding 24 compare questions total. Contents and ordering Type Count Directories beneath 10… See the full description on the dataset page: https://huggingface.co/datasets/vineet-datasets/probe-benchmark-hard50.

sourceHugging Faceupdated 5d agoView on Hugging Face
0likes60downloads
validation.json21 linesDownload Raw Back to root
1{2  "episodes": 62,3  "by_type": {4    "beneath": 10,5    "compare": 24,6    "count": 13,7    "find": 158  },9  "review_image_paths_checked": 872,10  "copied_files_hash_checked": 10502,11  "snapshots_byte_identical": true,12  "all_review_statuses": "accepted",13  "official_manifest_unchanged_sha256": "72344affeab5dd48dd3737edacf16042db35f40b37d4a8b6c8d7ff9eefee3c0f",14  "extra_asset_folders": [15    "allsides_objects/allsides_stacking_ring_toy",16    "allsides_objects/allsides_turtle_figurine"17  ],18  "website_not_modified": true,19  "inverse_compare_episodes_added": 1220}21