uv-scripts/object-detection

1---2viewer: false3tags: [uv-script, object-detection]4---5 6# Object Detection Dataset Scripts7 88 scripts to **create**, convert, review, validate, inspect, diff, and sample object detection datasets on the Hub. Supports 6 bbox formats — no setup required.9 10Start from nothing: `falcon-perception.py` generates a first-pass detection dataset for any class you can name, zero-shot, with no labelling and no training. The other six then convert, check, and measure it.11This repository is inspired by [panlabel](https://github.com/strickvl/panlabel)12 13## Quick Start14 15Convert bounding box formats without cloning anything:16 17```bash18# Convert COCO-style bboxes to YOLO normalized format19uv run convert-hf-dataset.py merve/coco-dataset merve/coco-yolo \20    --from coco_xywh --to yolo --max-samples 10021```22 23That's it! The script will:24 25- Load the dataset from the Hub26- Convert all bounding boxes in-place27- Push the result to a new dataset repo28- View results at: `https://huggingface.co/datasets/merve/coco-yolo`29 30## Scripts31 32| Script | Description |33|--------|-------------|34| `falcon-perception.py` | **Create** a detection dataset zero-shot from any image dataset — name a class, get boxes + masks (runs on Apple Silicon too) |35| `falcon-perception-bucket.py` | Same, reading images from an HF bucket, resumable across restarts |36| `review-detections.py` | **Review** a detection dataset in your browser — keyboard accept/reject per image or per box, acceptance + missed rates, pushes a `review` column |37| `convert-hf-dataset.py` | Convert between 6 bbox formats and push to Hub |38| `validate-hf-dataset.py` | Check annotations for errors (invalid bboxes, duplicates, bounds) |39| `stats-hf-dataset.py` | Compute statistics (counts, label histogram, area, co-occurrence) |40| `diff-hf-datasets.py` | Compare two datasets semantically (IoU-based annotation matching) |41| `sample-hf-dataset.py` | Create subsets (random or stratified) and push to Hub |42| `embed-bucket-images.py` | Join image bytes back into a bucket teacher pass — gold excluded + asserted, one final-schema write |43| `materialize-coco.py` | Generate a canonical COCO 2017 directory tree from the parquet, in-job, on ephemeral disk |44| `render-detections.py` | Render box/mask overlays and pixel-verify they actually drew (blank renders exit nonzero) |45| `smoke-test.py` | Free local check (~30 s) that the plumbing chain (`embed-bucket-images.py` → `materialize-coco.py` → `render-detections.py`) still produces correct output — run before any job depends on it |46 47## Supported Bbox Formats48 49All scripts support these 6 bounding box formats, matching the [panlabel](https://github.com/strickvl/panlabel) Rust CLI:50 51| Format | Encoding | Coordinate Space |52|--------|----------|------------------|53| `coco_xywh` | `[x, y, width, height]` | Pixels |54| `xyxy` | `[xmin, ymin, xmax, ymax]` | Pixels |55| `voc` | `[xmin, ymin, xmax, ymax]` | Pixels (alias for `xyxy`) |56| `yolo` | `[center_x, center_y, width, height]` | Normalized 0–1 |57| `tfod` | `[xmin, ymin, xmax, ymax]` | Normalized 0–1 |58| `label_studio` | `[x, y, width, height]` | Percentage 0–100 |59 60Conversions go through XYXY pixel-space as the intermediate representation, so any format can be converted to any other format.61 62## Common Options63 64All scripts accept flexible column mapping. Datasets can store annotations as flat columns or nested under an `objects` dict — both layouts are handled automatically.65 66| Option | Description |67|--------|-------------|68| `--bbox-column` | Column containing bboxes (default: `bbox`) |69| `--category-column` | Column containing category labels (default: `category`) |70| `--width-column` | Column for image width (default: `width`) |71| `--height-column` | Column for image height (default: `height`) |72| `--split` | Dataset split (default: `train`) |73| `--max-samples` | Limit number of samples (useful for testing) |74| `--hf-token` | HF API token (or set `HF_TOKEN` env var) |75| `--private` | Make output dataset private |76 77Every script supports `--help` to see all available options:78 79```bash80uv run convert-hf-dataset.py --help81```82 83## Convert (`convert-hf-dataset.py`)84 85Convert bounding boxes between any of the 6 supported formats:86 87```bash88# COCO -> XYXY89uv run convert-hf-dataset.py merve/license-plates merve/license-plates-voc \90    --from coco_xywh --to voc91 92# YOLO -> COCO93uv run convert-hf-dataset.py merve/license-plates merve/license-plates-yolo \94    --from coco_xywh --to yolo95 96# TFOD (normalized xyxy) -> COCO97uv run convert-hf-dataset.py merve/license-plates-tfod merve/license-plates-coco \98    --from tfod --to coco_xywh99 100# Label Studio (percentage xywh) -> XYXY101uv run convert-hf-dataset.py merve/ls-dataset merve/ls-xyxy \102    --from label_studio --to xyxy103 104# Test on 10 samples first105uv run convert-hf-dataset.py merve/dataset merve/converted \106    --from xyxy --to yolo --max-samples 10107 108# Shuffle before converting a subset109uv run convert-hf-dataset.py merve/dataset merve/converted \110    --from coco_xywh --to tfod --max-samples 500 --shuffle111```112 113| Option | Description |114|--------|-------------|115| `--from` | Source bbox format (required) |116| `--to` | Target bbox format (required) |117| `--batch-size` | Batch size for map (default: 1000) |118| `--create-pr` | Push as PR instead of direct commit |119| `--shuffle` | Shuffle dataset before processing |120| `--seed` | Random seed for shuffling (default: 42) |121 122## Validate (`validate-hf-dataset.py`)123 124Check annotations for common issues:125 126```bash127# Basic validation128uv run validate-hf-dataset.py merve/coco-dataset129 130# Validate YOLO-format dataset131uv run validate-hf-dataset.py merve/yolo-dataset --bbox-format yolo132 133# Validate TFOD-format dataset134uv run validate-hf-dataset.py merve/tfod-dataset --bbox-format tfod135 136# Strict mode (warnings become errors)137uv run validate-hf-dataset.py merve/dataset --strict138 139# JSON report140uv run validate-hf-dataset.py merve/dataset --report json141 142# Stream large datasets without full download143uv run validate-hf-dataset.py merve/huge-dataset --streaming --max-samples 5000144 145# Push validation report to Hub146uv run validate-hf-dataset.py merve/dataset --output-dataset merve/validation-report147```148 149**Issue Codes:**150 151| Code | Level | Description |152|------|-------|-------------|153| E001 | Error | Bbox/category count mismatch |154| E002 | Error | Invalid bbox (missing values) |155| E003 | Error | Non-finite coordinates (NaN/Inf) |156| E004 | Error | xmin > xmax |157| E005 | Error | ymin > ymax |158| W001 | Warning | No annotations in example |159| W002 | Warning | Zero or negative area |160| W003 | Warning | Bbox before image origin |161| W004 | Warning | Bbox beyond image bounds |162| W005 | Warning | Empty category label |163| W006 | Warning | Duplicate file name |164 165## Stats (`stats-hf-dataset.py`)166 167Compute rich statistics for a dataset:168 169```bash170# Basic stats171uv run stats-hf-dataset.py merve/coco-dataset172 173# Top 20 label histogram, JSON output174uv run stats-hf-dataset.py merve/dataset --top 20 --report json175 176# Stats for TFOD-format dataset177uv run stats-hf-dataset.py merve/dataset --bbox-format tfod178 179# Stream large datasets180uv run stats-hf-dataset.py merve/huge-dataset --streaming --max-samples 10000181 182# Push stats report to Hub183uv run stats-hf-dataset.py merve/dataset --output-dataset merve/stats-report184```185 186Reports include: summary counts, label distribution, annotation density, bbox area/aspect ratio distributions, per-category area stats, category co-occurrence pairs, and image resolution distribution.187 188## Diff (`diff-hf-datasets.py`)189 190Compare two datasets semantically using IoU-based annotation matching:191 192```bash193# Basic diff194uv run diff-hf-datasets.py merve/dataset-v1 merve/dataset-v2195 196# Stricter matching197uv run diff-hf-datasets.py merve/old merve/new --iou-threshold 0.7198 199# Per-annotation change details200uv run diff-hf-datasets.py merve/old merve/new --detail201 202# JSON report203uv run diff-hf-datasets.py merve/old merve/new --report json204```205 206Reports include: shared/unique images, shared/unique categories, matched/added/removed/modified annotations.207 208## Sample (`sample-hf-dataset.py`)209 210Create random or stratified subsets:211 212```bash213# Random 500 samples214uv run sample-hf-dataset.py merve/dataset merve/subset -n 500215 216# 10% fraction217uv run sample-hf-dataset.py merve/dataset merve/subset --fraction 0.1218 219# Stratified sampling (preserves class distribution)220uv run sample-hf-dataset.py merve/dataset merve/subset \221    -n 200 --strategy stratified222 223# Filter by categories224uv run sample-hf-dataset.py merve/dataset merve/subset \225    -n 100 --categories "cat,dog,bird"226 227# Reproducible sampling228uv run sample-hf-dataset.py merve/dataset merve/subset \229    -n 500 --seed 42230```231 232| Option | Description |233|--------|-------------|234| `-n` | Number of samples to select |235| `--fraction` | Fraction of dataset (0.0–1.0) |236| `--strategy` | `random` (default) or `stratified` |237| `--categories` | Comma-separated list of categories to filter by |238| `--category-mode` | `images` (default) or `annotations` |239 240## Run Locally241 242```bash243# Clone and run244git clone https://huggingface.co/datasets/uv-scripts/panlabel245cd panlabel246uv run convert-hf-dataset.py input-dataset output-dataset --from coco_xywh --to yolo247 248# Or run directly from URL249uv run https://huggingface.co/datasets/uv-scripts/panlabel/raw/main/convert-hf-dataset.py \250    input-dataset output-dataset --from coco_xywh --to yolo251```252 253Works with any Hugging Face dataset containing object detection annotations — COCO, YOLO, VOC, TFOD, or Label Studio format.254 255## Making a dataset from scratch256 257The other scripts assume you already have annotations. `falcon-perception.py` is where they can come from — [Falcon-Perception](https://huggingface.co/tiiuae/Falcon-Perception) finds every instance of a class you name, with no label set and no training:258 259```bash260# 1. does the model do the thing? (your laptop — no GPU needed)261uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/falcon-perception.py --image page.jpg --query illustration --preview262 263# 2. does it work on YOUR data? (first rows of the real corpus)264uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/falcon-perception.py --dataset biglam/british-library-book-images \265    --config plates --limit 3 --preview266 267# 3. the whole corpus, on a GPU268hf jobs uv run --flavor a10g-large --secrets HF_TOKEN \269    https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/falcon-perception.py \270    --dataset biglam/british-library-book-images --config plates \271    --id-col fname --query illustration --out you/plates-illustrations272 273# 4. it is already in `yolo` format — the rest of this directory just works274uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/validate-hf-dataset.py you/plates-illustrations --bbox-format yolo275uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/stats-hf-dataset.py    you/plates-illustrations --bbox-format yolo276```277 278Falcon emits boxes as normalised centre x,y + w,h, which *is* the `yolo` format above, so no conversion step is needed.279 280**The correction loop.** A zero-shot first pass is a starting point, not ground truth. Convert it for human review, correct it, then diff the two to find out how good the first pass actually was:281 282```bash283uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/convert-hf-dataset.py you/plates-illustrations you/for-review --from yolo --to label_studio284#  ... correct in Label Studio, push as you/corrected ...285uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/diff-hf-datasets.py you/plates-illustrations you/corrected   # IoU match = zero-shot accuracy286```287 288**Runs without a CUDA GPU.** Unlike most recipes in this repo, `falcon-perception.py` selects the MLX backend on Apple Silicon automatically. It is slower there (about 6 s/img vs 0.4 on an A10G), which is the right trade for step 1 and 2 above — checking your class name works before spending GPU hours.289 290### Known limits291 292Measured, not guessed — see the script docstrings for the failure each one came from.293 294| Limit | What to do |295|---|---|296| `--query` is a **class name**, not an instruction | `illustration` works; `the illustration, excluding captions` returns nothing |297| **One class per run** | A combined query returned 6 instances where three single-class runs found 24. N classes = N runs, then concatenate |298| **No confidence scores** — the model has no score token | Sort review by the emitted `rectangularity` (mask area ÷ bbox area, measured 0.34–1.00) and apply an area floor |299| `a10g-small` gets OOMKilled | The engine's auto-config sizes from the GPU and ignores host RAM — use `a10g-large` |300 301### Just want the numbers?302 303`--out` takes a file path as readily as a repo id — no Hub push, nothing to clean up:304 305```bash306uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/falcon-perception.py --image page.jpg --query illustration --out results.json307uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/falcon-perception.py --image "scans/*.jpg" --query illustration --out results.jsonl308uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/falcon-perception.py --image page.jpg --query illustration --json | jq '.[0].objects.bbox'309```310 311Anything ending `.json`, `.jsonl` or `.parquet` is written locally; anything else is treated as a Hub dataset repo id.312 313### Bucket runs314 315`falcon-perception-bucket.py` reads images from an HF bucket and writes resumable parquet parts back to a bucket — kill it and re-run the same command, done keys are skipped. Publish once at the end to use the rest of this directory:316 317```python318from datasets import ClassLabel, Sequence, load_dataset319ds = load_dataset("parquet", data_files="hf://buckets/<namespace>/<bucket>/part-*.parquet",320                  split="train")321feats = ds.features.copy()  # parquet stores category as bare ints; name the class322feats["objects"]["category"] = Sequence(ClassLabel(names=[ds[0]["query"]]))323ds.cast(feats).push_to_hub("<namespace>/<dataset>")  # a dataset repo, distinct from the bucket324```325 326### Output columns327 328`objects.bbox` (`yolo`), `objects.category` (a `ClassLabel` named after the query), `objects.area`, `objects.rectangularity`, plus `image`, `image_id` (int64 — COCO-style trainers require an integer id), `source_id` (the original key), `width`, `height`, `n_instances`, and `masks_rle` (COCO RLE — segmentation rides along; the bbox scripts ignore it).329 330### Train on the output331 332Convert to COCO pixel boxes, then fine-tune a compact Apache-2.0 detector (D-FINE, RT-DETRv2 — not ultralytics/YOLO, which is AGPL). The [`SKILL.md`](SKILL.md) in this directory walks an agent (or you) through the whole loop — teacher labels → validate → convert → train → honest eval:333 334```bash335uv run https://huggingface.co/datasets/uv-scripts/object-detection/raw/main/convert-hf-dataset.py you/plates-illustrations you/plates-coco --from yolo --to coco_xywh336```337