Apppublic

visual-riddles/Visual-Riddles-Leaderboard

sourceHugging Facecc-by-4.0updated 2y agoView on Hugging Face

0likes

Visual-Riddles-Leaderboard.csv17 linesDownload Raw Back to root

1Model,Open Ended VQA: % Human Rating,Multiple Choice VQA: % Accuracy,Hints-Multiple Choice VQA: % Accuracy ,Attributions-Multiple Choice VQA: % Accuracy ,Refernce Based-Automatic Evaluation: Accuracy of Judge Prediction Compared to Human Ratings,Refernce Free-Automatic Evaluation: Accuracy of Judge Prediction Compared to Human Ratings,Automatic Evaluation: % Auto-Rater Ratings,Hints-Automatic Evaluation: % Auto-Rater Ratings,Attributions-Automatic Evaluation: % Auto-Rater Ratings
2Humans,82,*,*,*,*,*,78,*,*
3Gemini Pro 1.5,40,38,66,72,87,52,53,62,29
4Gemini Pro Vision,30,41,62,*,75,38,34,47,
5GPT4,34,45,69,82,86,51,38,61,25
6LlaVA-1.6-34B,15,24,30,*,76,43,21,16,*
7LlaVA-1.5-7B,13,17,29,*,70,35,19,30,*
8InstructBlip,13,*,*,*,*,*,20,28,*
9Gemini Pro 1.5 Caption _ Gemini Pro 1.5,23,*,*,*,*,*,*,*,*
10Human (Oracle) Caption _ Gemini Pro 1.5,50,*,*,*,*,*,*,*,*
11Claude 3.5 Sonnet,*,46,45,*,*,*,39,*,*
12GPT4o,*,55,83,*,*,*,50,*,*
13Qwen-VL-Max,*,35,53,*,*,*,26,*,*
14Molmo-7B,*,34,42,*,*,*,36,*,*
15OpenAI o1,*,58,82,*,*,*,58,*,*
16Gemini 2.0 thinking,*,60,84,*,*,*,51,*,*
17Gemini 2.0,*,46,72,*,*,*,55,*,*