openrouter
OpenRouter-ZebraLogicBench
OpenRouter-ZebraLogicBench
This repository contains a single Python file evaluation script for the allenai/ZebraLogicBench dataset. The script is adapted from ZeroEval and can be used to evaluate language models on logical reasoning tasks.
Key Features
Single file implementation for easy use
Compatible with OpenAI-like APIs (base URL can be modified in eval_zebra.py)
Example results provided for Claude 3 Haiku
Usage Requirements
Access to the private dataset:… See the full description on the dataset page: https://huggingface.co/datasets/leafspark/OpenRouter-ZebraLogicBench.attributiontag-vl-22k-openrouter-imagesconversations_dataset_openrouter4conversations_dataset_openrouterconversations_dataset_openrouter3conversations_dataset_openrouter2