latam-gpt/eval-runner

Webhook receiver mode: Space receives POSTs, spawns GPU Jobs

Reduce max_length to 2560 (observed max is 2049, saves KV cache)

Set max_length=4096 for olmes (5-shot MMLU exceeds 2048)

Fix patches.sh: use find_spec instead of import (CUDA unavailable at build)

Force-reinstall lm-eval 0.4.3, add patch verification

Remove separate lm-eval install: olmes pins 0.4.3, patches.sh handles vLLM compat

Add lm-eval[math] extras (fixes minerva_math import)

Pin lm-eval to 0.4.11 (TokensPrompt fix for vLLM 0.17)

Remove deep_ep: fixes libnvshmem ImportError on A100

Add python symlink (olmes needs python not python3)

Bundle eval.yaml, fallback when GitHub repo is private

Reset ENTRYPOINT: vllm base crashes on CPU Space

Use CMD instead of ENTRYPOINT: keep Space alive for Jobs

Fix entrypoint: use python3

Fix olmes install: clone then pip install

Fix Dockerfile: install olmes from GitHub, use base image torch/vLLM

Initial eval-runner pipeline

initial commit