jgracie52/scanner-poisoned-iris-benchmark

Scanner Poisoned Iris Benchmark

This benchmark starts from the classic UCI Iris dataset and injects multiple synthetic poisoning patterns so dataset scanners can exercise duplicate, anomaly, missingness, skew, and divergence heuristics against a small tabular corpus.

Recommended Hugging Face repo slug: your-org/scanner-poisoned-iris-benchmark

What It Is For

• —benchmarking dataset quality and poisoning detection workflows
• —regression-testing scanner heuristics on a deterministic public reference set
• —demonstrating how a mostly clean tabular dataset can be corrupted in several distinct ways

What It Is Not

• —not a naturally occurring poisoned dataset collected from the wild
• —not a benchmark for downstream model accuracy
• —not a replacement for large-scale real-world curation audits

Poison Families Included

• —exact_duplicate
• —near_duplicate_text
• —near_duplicate_numeric
• —numeric_extreme_outlier
• —missingness_hotspot
• —label_flip
• —label_skew_injection
• —long_text_injection
• —array_payload_bloat
• —cross_split_distribution_shift

Split Intent

• —train.jsonl: balanced clean baseline plus duplicate, near-duplicate, label-flip, and outlier pressure
• —validation.jsonl: intentionally setosa-heavy with concentrated missingness, duplicate pressure, and oversized payloads
• —test.jsonl: mixed-label corruption with strong near-duplicate, outlier, and structural anomaly pressure

Schema

• —record_id: synthetic row identifier
• —source_dataset_id: source dataset slug
• —source_row_id: source row index from Iris
• —split_name: split label
• —sepal_length_cm
• —sepal_width_cm
• —petal_length_cm
• —petal_width_cm
• —species
• —specimen_note: synthetic descriptive text used to support text-oriented duplicate detection
• —observation_tags: array field used to support array-length anomaly checks
• —is_poisoned
• —poison_primary
• —poison_tags

Split Sizes

• —train.jsonl: 180 rows
• —validation.jsonl: 150 rows
• —test.jsonl: 150 rows

Publication Notes

• —keep the repo config as default
• —upload the four artifacts in this directory: README.md, train.jsonl, validation.jsonl, test.jsonl, and manifest.json
• —preserve manifest.json because it records the intended poison counts and split-level scanner targets
• —keep the JSONL schema unchanged if you want scanner regressions to stay stable across versions

Provenance

The clean base measurements come from the UCI Iris Plants Database. The generator in this directory reads the SciPy-bundled copy of the Iris ARFF fixture and writes deterministic poisoned JSONL splits.

License And Attribution

This benchmark is released under CC BY 4.0.

It is a synthetic derivative benchmark built from the Iris Plants Database. When publishing or redistributing it, keep attribution to the original UCI Iris dataset and note that this version adds deterministic poisoning patterns, synthetic text fields, and benchmark metadata for scanner evaluation.

Generation

From api/:

./.venv/bin/python -m data.poisoned_iris_benchmark.generator

The command rewrites train.jsonl, validation.jsonl, test.jsonl, and manifest.json in this directory.