sshchoholiev/research-paper-read-bench

Research Paper Read - On-device LLM Benchmark

Leaderboard for the Research Paper Read iOS app's on-device inference benchmark. Users run a fixed prefill + 256-token greedy decode on their iPhone; the app POSTs the results to this Space which appends them to the companion dataset.

Submission format

POST /submit with Content-Type: application/json. See schema.json in the dataset repo for the full payload shape.

The Space returns { "status": "ok" | "duplicate", "submission_id": "...", "leaderboard_url": "..." }.

Rate limit: 1 submission per IP per 5 minutes.

Filters applied to the public leaderboard

• —bench_version == 1
• —thermal_state == "nominal" (fair/serious/critical filtered out)
• —is_simulator == false

Best gen_tps per device_id is shown.