evalitahf/cruciverb_it

Cruciverb-IT

Dataset adaptation of `cruciverb-it/evalita2026` for EVALITA-LLM. This repository contains only the data; prompts, parsers and evaluation metrics are defined in the evaluation harness.

Task 1

Each record contains an Italian crossword clue, the expected answer length and the gold answer:

{"id": "task1_test_000001", "clue": "...", "answer_length": 7, "answer": "..."}

The gold test data were cleaned and deterministically sampled into three nested splits: test_1k ⊂ test_2k ⊂ test_5k. The cleaned dev split contains 20,729 records.

Task 2

Each record combines the empty grid, its clues and the gold grid, which are stored in separate files in the original repository:

{"id": "task2_test_000001", "empty_grid": [[" "]], "clues": [{"target": "A", "clue": "...", "row": 0, "col": 0, "direction": "A", "length": 1}], "gold_grid": [["A"]]}

The splits contain 500 training, 50 development and 50 test crosswords. The target field contains the gold answer for a clue and must not be included in the model prompt.

License

The data are distributed under the conditions specified in the original repository.

Citation

Cristiano Ciaccio, Gabriele Sarti, Alessio Miaschi, Felice Dell'Orletta and Malvina Nissim. 2026. *Cruciverb-IT at EVALITA 2026: Overview of the Crossword Solving in Italian Task*. In Proceedings of EVALITA 2026, pages 392–409.

Credits

EVALITA-LLM dataset adaptation: Pietro Botarelli.