vineet-datasets/probe-benchmark-hard50
PROBE hard50 — human-reviewed hard development set This is a separate standard-format review bank of 62 Astra-conditioned hard development episodes. It is NOT an independently evaluated holdout. Do not merge into or modify benchmark_600. The original 50 human-reviewed questions were augmented with 12 human-accepted inverse comparison prompts, yielding 24 compare questions total. Contents and ordering Type Count Directories beneath 10… See the full description on the dataset page: https://huggingface.co/datasets/vineet-datasets/probe-benchmark-hard50.
060
1{2 "episodes": 62,3 "by_type": {4 "beneath": 10,5 "compare": 24,6 "count": 13,7 "find": 158 },9 "review_image_paths_checked": 872,10 "copied_files_hash_checked": 10502,11 "snapshots_byte_identical": true,12 "all_review_statuses": "accepted",13 "official_manifest_unchanged_sha256": "72344affeab5dd48dd3737edacf16042db35f40b37d4a8b6c8d7ff9eefee3c0f",14 "extra_asset_folders": [15 "allsides_objects/allsides_stacking_ring_toy",16 "allsides_objects/allsides_turtle_figurine"17 ],18 "website_not_modified": true,19 "inverse_compare_episodes_added": 1220}21 