HanClinto/solring-eval

Sol Ring Dataset

(c) 2026, HanClinto Games, LLC

A collection of 307 reference frames for benchmarking Magic: The Gathering card identification — specifically edition (set) discrimination under real-world camera conditions.

Purpose

To provide a meaningful, reproducible metric for measuring and comparing the accuracy of card recognition algorithms, with particular focus on set / edition identification rather than just card-name recognition.

Theory

In Magic: The Gathering, Commander is the most popular way to play the game.

In Commander, the single most-popular card (ranked #1 on EDHREC) is Sol Ring.

The Mike Bierik artwork for Sol Ring is the most-reprinted artwork in the entire game, appearing across dozens of Commander precon sets with nearly identical artwork and card layout.

This makes Sol Ring uniquely valuable as a benchmark: it is simultaneously the most-played card in the most-played format, and the card whose printings are most easily confused with one another. A system that can reliably distinguish a C17 Sol Ring from a C18 Sol Ring from a CMR Sol Ring — all sharing the same artwork — has demonstrated meaningful edition discrimination, not just card-name lookup.

This dataset therefore represents a practical, high-stakes standard for edition identification accuracy across a wide swath of modern sets.

Dataset construction

21 distinct printings of Sol Ring were acquired through TCGPlayer — each from a different edition, each bearing the iconic Mike Bierik artwork.

Short videos were recorded of each card using a mobile phone against a plain white background, capturing dozens of frames per card across varied lightings, angles, and minor motion blur.

Each video filename is labeled with the Scryfall UUID of the correct card.

Keyframes were extracted with FFmpeg, and blur detection was used to filter out unwanted frames. The remaining sharp ("good") frames are what appear in this dataset under data/frames/.

Corner coordinates for each frame were then detected via a SIFT homography pipeline matching against the known Scryfall reference image for that card. These are stored in corners.csv and can be used to dewarp each frame to a clean, perspective-corrected card crop before running an identification model.

Temporal structure

Frames within each edition are temporally ordered by frame_number (the source video frame index, spaced roughly every 60 source frames ≈ 1–2 seconds at 30 fps). This ordering is critical for simulating a live-camera rolling-buffer evaluation:

from collections import deque, defaultdict
import csv, cv2
from pathlib import Path

rows = list(csv.DictReader(open("corners.csv")))
by_card = defaultdict(list)
for r in rows:
    by_card[r["card_id"]].append(r)
for frames in by_card.values():
    frames.sort(key=lambda r: int(r["frame_number"]))

# Simulate a rolling buffer of up to 5 embeddings
for card_id, frames in by_card.items():
    buffer = deque(maxlen=5)
    for row in frames:
        img  = cv2.imread(row["img_path"])
        emb  = embed(dewarp(img, row))          # your model here
        kept = [e for e in buffer
                if cosine_sim(emb, e) >= 0.7]  # filter bad grabs
        search_emb = normalize(mean([emb] + kept)) if kept else emb
        top1 = gallery_search(search_emb)
        buffer.append(emb)
        record(top1 == card_id)

File layout

corners.csv          307-row metadata file (schema below)
data/frames/*.jpg    source JPEG frames (original camera perspective, not cropped)

corners.csv schema

Edition list

All 21 printings share the Mike Bierik Sol Ring artwork.

License

This dataset is released under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0).

You are free to share and adapt this material for any purpose, including commercially, as long as you provide appropriate credit and distribute any derivative datasets under the same license. You are explicitly free to use this dataset for commercial purposes under those terms.

The goal is a universal, openly-accessible standard for measuring card identification accuracy — usable for comparing closed-source and open-source solutions alike. If the above terms don't work for your situation, reach out and we can discuss alternative licensing.

Contributions are welcome. Additions or corrections to the dataset are appreciated but not required.