Joshyxwa/data_draft

1---2pretty_name: TechJam 2026 Data Draft3license: other4size_categories:5- 1K<n<10K6task_categories:7- image-classification8task_ids:9- multi-class-image-classification10tags:11- ai-generated-image-detection12- image-forensics13- research-only14- private-dataset15---16 17# TechJam 2026 Data Draft18 19## Dataset Summary20 21This **private research draft** validates a binary image-classifier pipeline:22the local data loader, feature extractor, classifier head, calibration step,23and robustness evaluation. It is not a public benchmark, a claim of real-world24detection quality, or part of the production 80k-master corpus.25 26The package has **10,000 canonical RGB PNG images**. Every image has a27portable manifest record with a label, split, source provenance, hashes, image28properties, and processing history.29 30## Composition31 32| Source | Real | Fully AI | Total | Notes |33| --- | ---: | ---: | ---: | --- |34| SID-Set | 2,500 | 2,500 | 5,000 | Canonical local snapshot; source-provided real/synthetic label |35| WildFake | 2,500 | 2,500 | 5,000 | Deterministic subset from complete local source archives |36| **Total** | **5,000** | **5,000** | **10,000** | Exact class and source balance |37 38WildFake real examples are evenly selected from `afhq`, `celebahq`, `church`,39`ffhq`, and `imagenet`. Fully-AI examples are evenly selected from `adm`,40`ddim`, `ddpm`, `imagen`, and `gan_based`. Each WildFake family contributes41500 images. COCO and DALL-E families are excluded.42 43## Splits44 45| Split | Real | Fully AI | Total | Intended use |46| --- | ---: | ---: | ---: | --- |47| `train` | 3,500 | 3,500 | 7,000 | Fit an architecture or classifier head |48| `dev` | 750 | 750 | 1,500 | Select settings and inspect errors |49| `calibration` | 750 | 750 | 1,500 | Set thresholds or fit probability calibration only |50 51Do not move records between splits or train on `dev`/`calibration` images or52their derivatives. The builder rejects cross-split collisions by asset ID,53base ID, lineage ID, exact SHA-256, exact perceptual hash, and near perceptual54hash distance at most five during WildFake selection.55 56## Labels57 58| Value | Meaning | Source evidence |59| --- | --- | --- |60| `real` | A source-labelled non-synthetic photograph/image | SID source path role or WildFake real archive role |61| `ai_full` | A source-labelled, fully generated AI image | SID source path role or WildFake generator archive role |62 63Labels are source-provided, not manually re-annotated. `label_evidence` and64`label_confidence` are present for every row; the confidence is `0.8` for this65draft. There are no `partial_ai`, `recaptured_ai`, or edited-image labels.66 67## Files And Metadata68 69| File | Purpose |70| --- | --- |71| `images/<split>/<label>/<asset_id>.png` | Canonical RGB image files |72| `manifest.parquet` | Full provenance, integrity, quality, and image-property manifest |73| `metadata.jsonl` | Compact image/label/split/source index for simple loaders |74| `verification.json` | Completed local decode, hash, and split-leakage audit |75 76`manifest.parquet` includes these field groups:77 78| Group | Key fields |79| --- | --- |80| Identity and split | `asset_id`, `base_id`, `lineage_id`, `split`, `label` |81| Label traceability | `label_evidence`, `label_confidence`, `real_subtype`, `ai_subtype` |82| Source provenance | `source_dataset`, `source_family`, `source_uri`, `source_archive`, `archive_member` |83| Image integrity | `sha256`, `phash`, `width`, `height`, `aspect_ratio`, `file_format`, `file_size_bytes` |84| Processing and quality | `processing_track`, `processing_history_status`, `provenance_status`, `quality_status` |85| Rights policy | `licence`, `licence_name`, `allowed_for_training`, `allowed_for_public_demo` |86 87Paths in the manifest are relative to this dataset repository. After download:88 89```python90from pathlib import Path91 92import pandas as pd93from PIL import Image94 95root = Path("path/to/data_draft")96manifest = pd.read_parquet(root / "manifest.parquet")97row = manifest.iloc[0]98image = Image.open(root / row.path).convert("RGB")99label = row.label  # "real" or "ai_full"100```101 102## Quality Checks103 104The packaged manifest passed the following local checks:105 106- 10,000 of 10,000 images decode as canonical RGB files.107- Recorded SHA-256, perceptual hash, width, and height match every image file.108- No detected cross-split collision by base ID, lineage ID, parent ID, exact109  hash, or exact perceptual hash.110- Final class counts are exactly balanced in every split.111- The package contains no raw source archives, API keys, model checkpoints,112  production generations, COCO records, or DALL-E records.113 114## Provenance And Restrictions115 116SID-Set rows retain upstream source URI and CC-BY-4.0 metadata. WildFake rows117retain the exact ZIP archive and member path but are marked118`licence_audit_required`. Therefore:119 120- Keep this repository private.121- Do not redistribute WildFake-derived files or make public demos from them122  until the upstream licence review is complete.123- Do not add these records to the production 80k corpus or use them for124  benchmark/generalization claims.125 126The `other` licence tag describes the mixed, restricted state of the package;127it grants no rights beyond the underlying source datasets.128 129## Limitations130 131This small mixture is useful for pipeline and architecture smoke tests, but it132is source-family constrained. Its results may be optimistic because training,133development, and calibration all originate from the same two upstream datasets.134It does not model social-media reposting, video-frame extraction, partial edits,135recapture, contemporary held-out generators, or real-world class prevalence.136Report per-source and per-transform errors, not only aggregate AUROC/AUPRC.137 138## Reproducibility139 140The deterministic selection seed is `20260830`. `manifest.parquet` preserves141the selection and processing lineage, stable asset/base/lineage identifiers,142and content hashes required to audit a local rebuild before training.143