iol-ai-challenge/iol-ai-2026

IOL-AI Challenge 2026

Can your model solve International Linguistics Olympiad problems? This is a script competition. Participants submit a Hugging Face model repository containing a script.py that reasons over each problem's data and produces answers, which are scored against the hidden IOL 2026 individual-contest set.

Task

Each problem gives a small amount of data from a language you have almost certainly never seen (Koryak, Hadza, Komnzo, Dâw, Yanyuwa) along with the hints needed to solve it. From those few examples your model has to work out enough of the grammar and vocabulary to answer 90 hidden sub-questions.

How to participate

1. 1.Create a public Hugging Face model repo containing a script.py (and any model files it needs). Private repos cannot be submitted.
2. 2.Your script reads the hidden test set from /tmp/data/test.csv and writes submission.csv (columns id,pred, 90 rows) to its working directory.
3. 3.Paste your model repo id into the submission form in the Space.

Evaluation

Scored by the geometric mean of points-weighted exact-match and chrF:

score = 100 × sqrt(weighted_exact_match × weighted_chrF)

Higher is better. A public subset is scored on each submission; the private split is revealed at the deadline. See the Rules, Dataset, and Submission tabs inside the Space for full details.

Problems are © their original IOL 2026 authors and used here for research and benchmarking. Learn more about IOL at ioling.org.