isaacmg/qwen3-vl-8b-hebrew-v20a-ckpt

Qwen3-VL-8B Hebrew — v2.0a checkpoints (KTIV corpus + grounding; merger frozen)

Status: current best on both transcription benchmarks (Sep 2026) and the first checkpoint in the series with text grounding — recommended for evaluation. The stable flagship pointer of the series remains v19a-ckpt until v2.0 is promoted; see the table at the bottom.

• —Canonical revision: step 1800, `af9df6a0ad4743bc8493a7cf0c14bc3b4b796dd8`. The 2000-step cosine schedule ended at 1800 when the (resumed) training session reached its runtime limit; the learning rate was already ~1e-6 and the eval loss flat (0.6733 at both 1700 and 1800), so this is the run's effective final.
• —Warm-started from v1.9a step 1300; vision tower and language LoRA (r16) continue training; the vision→language merger is frozen (the v1.9b/c experiments showed merger training changes spatial behaviour without improving transcription — see those cards).
• —Same inference contract as v1.9: 6.5–7 MP image budget (min_pixels=6_500_000, max_pixels=7_000_000), 12k sequence length.

What is new in v2.0

A new manuscript corpus and new task families, mixed with the v1.9 data:

• —KTIV manuscript pages (796 manuscripts, 2,005 pages, ~1.9M letters of editorial transcription with word geometry) as full-page, region, section and line transcription rows — roughly 3× the manuscript text of the v1.9 training set.
• —Grounding tasks (~8% of the mixture), derived from the same word geometry: locate (unique phrase → bbox_2d), read_box (box → text), templated layout QA, and grounded page transcription (JSON lines with boxes). Coordinates are 0–1000 normalized to the input image.
• —The v1.9 mixture is retained for continuity: Cairo Genizah pages (genizah_clean_v2), synthetic Hebrew renders (synthetic_hebrew_v3) and printed-Talmud replay.

Benchmark manuscripts (the religious-140 set and the frozen PGP-131 set) are excluded from training by manuscript identifier.

Results

Transcription (median aligned F1, single decode at temperature 0.1, same harness as every card in the series):

The religious gain is well outside the series' median noise band (±~0.008); it is also the benchmark closest in kind to the new KTIV training pages (decontaminated by manuscript, but the same corpus family). The PGP gain on documentary fragments is small.

Grounding (frozen 133-query eval on religious-benchmark pages; none of the v1.9 arms saw grounding data):

Box-conditioned tasks are where the new capability lives (phrase→box IoU 0.29→0.47, box→text CER 0.58→0.13). Templated layout QA on held-out pages is roughly tied with the parent (first/last-line CER 0.286 vs 0.329; find-the-line CER 0.242 vs 0.274); grounded page transcription pays a per-line text cost relative to plain transcription in this checkpoint.

Caveats. Single seed, single decode; per-page comparisons are not interpretable at this granularity (a same-model rerun control flips several pages across 0.5 F1). Aggregates above are the basis of every claim.

Prompts

• —Transcription: the same diplomatic-transcription prompts as v1.9.
• —Locate: "Locate the phrase … on this page. Answer with JSON `{"bbox_2d": [x1, y1, x2, y2]}` in 0-1000 normalized coordinates."
• —Read box: "Transcribe only the text inside bbox_2d = [x1, y1, x2, y2] (0-1000 normalized)…"
• —Grounded page: JSON list of {"bbox_2d": […], "text": "…"} per line.

Data and credits

Manuscript images and transcriptions used for the KTIV portion of the training set were accessed through the Ktiv project of the National Library of Israel, with holding institutions including Cambridge University Library, the Jewish Theological Seminary and the British Library; that training set is not redistributed. Genizah documentary transcriptions derive from the Princeton Geniza Project. Printed-Talmud replay data is described on the v1.6 card. The synthetic renders are released as synthetic_hebrew_v3 and synthetic_rashi.

The series at a glance

One line per generation — which checkpoint to use and which are historical:

\* CER over substantive attempts only, alignment-based scorer. Benchmarks differ across generations (Talmud print vs Genizah manuscripts) — compare within a row's named benchmark, not across rows.