anisiraj/code-image-to-text

Code Snippet Image → Text

A multimodal dataset for fine-tuning vision-language models (VLMs) on the task of transcribing an image of a code snippet back into its source text — syntax-aware OCR.

Each example pairs a syntax-highlighted PNG of code with the exact code text that produced it. It spans 8 programming languages and deliberately mixes two capture types:

• —`block` — a complete function / unit (6–45 lines).
• —`fragment` — a contiguous partial view (3–14 lines) that may start or end mid-statement, simulating someone screenshotting only part of a snippet (possibly with a line or two of surrounding context). This makes the model robust to partial inputs.

• —Total examples: 96,000 · Languages: 8 · Splits: train (85,634) / validation (10,366)
• —Target column: text (the exact code; never contains line numbers).

Samples

Complete blocks

C — complete block

C++ — complete block

Go — complete block

Java — complete block

JavaScript — complete block

PHP — complete block

Python — complete block

Ruby — complete block

Fragments (partial captures)

JavaScript — partial fragment

Python — partial fragment

Dataset structure

Fields

Composition (images per language)

Splits

train / validation, split by repository — a function and any fragments derived from it always land in the same split, so there is no train/validation leakage.

Load

from datasets import load_dataset

ds = load_dataset("anisiraj/code-image-to-text")        # DatasetDict with 'train' and 'validation'
print(ds)
ex = ds["train"][0]
print(ex["language"], ex["kind"])  # e.g. 'Python' 'block'
ex["image"]                        # PIL.Image.Image
ex["text"]                         # the code string to predict

Stream (no full download)

ds = load_dataset("anisiraj/code-image-to-text", split="train", streaming=True)
for ex in ds.take(5):
    print(ex["language"], ex["kind"], ex["image"].size)

One split only

val = load_dataset("anisiraj/code-image-to-text", split="validation")

View the images

ds = load_dataset("anisiraj/code-image-to-text", split="train")

# Jupyter / notebook — renders inline:
ds[0]["image"]

# Save or open in an OS image viewer:
img = ds[0]["image"]
img.save("example.png")
img.show()

Show a grid of samples with matplotlib:

import matplotlib.pyplot as plt
fig, axes = plt.subplots(2, 3, figsize=(20, 8))
for ax, ex in zip(axes.ravel(), ds.select(range(6))):
    ax.imshow(ex["image"]); ax.axis("off")
    ax.set_title(f"{ex['language']} / {ex['kind']}")
plt.tight_layout(); plt.show()

Filter

# Python complete blocks only
py_blocks = ds.filter(lambda r: r["language"] == "Python" and r["kind"] == "block")

# fragments only
fragments = ds.filter(lambda r: r["kind"] == "fragment")

# one language
go = ds.filter(lambda r: r["language"] == "Go")

How images are stored

Images use the HF Image feature: in the Parquet shards each is a struct<bytes: binary, path: string> holding the raw PNG bytes (PNG signature 89 50 4E 47), not base64. datasets decodes them to PIL.Image on access. To get the undecoded bytes:

from datasets import Image
raw = load_dataset("anisiraj/code-image-to-text", split="train").cast_column("image", Image(decode=False))
raw[0]["image"]["bytes"][:8]   # b'\x89PNG\r\n\x1a\n'

Fine-tune a VLM (image → text)

from datasets import load_dataset
ds = load_dataset("anisiraj/code-image-to-text")

PROMPT = "Transcribe the code shown in this image exactly, preserving indentation."

def to_chat(ex):
    return {"image": ex["image"], "prompt": PROMPT, "target": ex["text"]}

train = ds["train"].map(to_chat)
# Feed `image` + `prompt` through your VLM's processor/chat template
# (e.g. Qwen2-VL, Idefics3, Llava, MiniCPM-V) and train to produce `target`.

Rendering & augmentation

Rendered with Pygments ImageFormatter: 18 themes (light & dark) × 3 monospace fonts × 5 font sizes (14–20) × line-numbers on/off × 3 paddings. Balanced to an equal number of images per language. The line-number gutter, when present, is visual only and is never part of the text target.

Sources & license

• —CodeSearchNet — code-search-net/code_search_net: Go, Java, JavaScript, PHP, Python, Ruby (complete functions from open-source GitHub repos).
• —GitHub Code Snippets by Bugout.dev / Simiotic — kaggle (CC BY 4.0): C, C++ blocks.

Released under CC BY 4.0. The underlying code originates from public GitHub repositories under their respective open-source licenses; please retain attribution to the sources above.

Citation

@misc{code_image_to_text,
  title  = {Code Snippet Image to Text},
  author = {anisiraj},
  year   = {2026},
  url    = {https://huggingface.co/datasets/anisiraj/code-image-to-text}
}