bigdatamark/synthid-research

SynthID Research Set

The same prompts rendered by several generators. Some carry an invisible provenance watermark and one does not, so the set is a matched comparison for anyone working on AI-image provenance detection.

We are publishing it because generating it is not free, and because a matched set — same prompt, several renderers, one of them a clean control — is more useful than unrelated collections.

This dataset grows. Directories are added as new models ship and existing ones are extended. metadata.csv is authoritative per image.

prompts.txt                                  the prompts, one per line, in order
metadata.csv                                 per image: size, C2PA state, issuer, recorded actions
gpt-image-2/                                 400 × OpenAI gpt-image-2,            1254×1254 PNG
gpt-image-2.5-flare/                          10 × OpenAI gpt-image-2.5-flare,    1254×1254 PNG
gpt-image-2.5-sunburst/                       10 × OpenAI gpt-image-2.5-sunburst, 1254×1254 PNG
gemini-3.1-flash-image/                      700 × Gemini 3.1 Flash Image,        2048×2048 JPEG
gemini-3.1-flash-image_library-downscales/   135 × the same renders re-downloaded, 1024×1024 JPEG
flux.2/                                      400 × FLUX.2, run locally,           1280×1280 PNG

Filenames are the join key. A file is named for the line of prompts.txt that produced it, so gpt-image-2/47.png, gemini-3.1-flash-image/47.jpg and flux.2/47.png are all line 47. Note that zero-padding is not consistent across the set — match on the integer, not the string.

What is in each set

Generation dates are read from each file's own metadata — the C2PA action timestamp where there is one — so they describe when the images were made, not when the repository was assembled.

The gpt-image-2.5 variants do not identify themselves

gpt-image-2.5-flare and gpt-image-2.5-sunburst are two separately named models, requested by those exact model strings. Their C2PA manifests report `softwareAgent: gpt-image 2.0` — the same string `gpt-image-2` produces. Nothing inside these files distinguishes the variant, or distinguishes either of them from gpt-image-2. The directory name is the only record of which model produced them, which is why they are filed separately rather than merged.

If you are testing whether a detector can tell model versions apart, this matters twice over: the metadata will not do it for you, and a detector that appears to separate these directories is working from pixels alone.

The FLUX images document their own generation

Every file in flux.2/ carries the complete ComfyUI workflow in its PNG prompt text chunk, so the exact model, encoder, VAE, sampler, scheduler, guidance, step count, resolution and seed for each image can be read straight out of the file:

import json
from PIL import Image
json.loads(Image.open("flux.2/47.png").info["prompt"])

For the record, every render so far used the same configuration, varying only the prompt and the seed:

The seeds are therefore reproducible but not sequential in this set's numbering, because the published prompts were selected out of a larger working set — read the seed from the file rather than computing it. The embedded workflow contains no local paths, usernames or machine identifiers; we checked before publishing.

The transformer is the FLUX dev variant. That licence governs use of the weights, which are not distributed here; it is not a copyright in the images, which have none (see LICENSE.md).

The Gemini set has two provenance halves — read this before using it

The files in gemini-3.1-flash-image/ came through two routes, and their signed manifests differ:

Both are Gemini output and both manifests verify as Trusted against Google's root. The second came through the consumer app's download path, which re-encodes and signs the result as a resize of an earlier asset, so its manifest does not repeat the watermark assertion. If you need images whose own manifest explicitly claims a SynthID watermark, filter metadata.csv on gemini_says_synthid == 1.

We found no measurable difference between the two halves in the pixel statistics we tested, but we have not verified every property, so treat the split as a variable rather than noise.

gemini-3.1-flash-image_library-downscales/

A second copy of 135 renders, obtained by re-downloading them from the Gemini library at 1024×1024 instead of the 2048×2048 the API returns. These are downscales of 2K renders, not images generated at 1024 — a distinction that matters more than it sounds.

The manifests invert in an interesting way. A file here asserts Created by Google Generative AI. plus Applied imperceptible SynthID watermark., while the 2048² file of the same prompt in gemini-3.1-flash-image/ carries the Opened + Resized manifest with no watermark assertion. So for these prompts the smaller file makes the stronger provenance claim.

Use them for what they are: a controlled test of what downscaling does to a watermark, with the same content available at both sizes. Do not use them as a stand-in for images a model generated natively at 1024 — a native-1K render and a downscaled 2K render are measurably different things, and treating one as the other will produce results that do not hold.

Prompts

Every prompt begins generate a square image of …. That phrasing was sent verbatim to gpt-image-2 and Gemini; for FLUX the prefix was stripped, because FLUX has no chat-style instruction layer and the sentence would otherwise be treated as content to draw. Squareness comes from an aspect_ratio parameter for Gemini and from the model's native output for gpt-image-2.

The published prompts were selected from a larger working set by removing prompts that: named or depicted the author, his family or his home; named a real person; depicted any person at all; probed model guardrails, the model's handling of colloquial slang, or its behaviour on meaningless input rather than describing a subject; contained a misspelling; or duplicated another prompt closely enough that the two renders were near-identical. Those kept were renumbered from 1.

The set therefore contains no people — every prompt names an animal, an object, a plant, a place or a scene. That is deliberate: a redistributed public set of AI-generated imagery is simpler for everyone if it depicts nobody.

Clean controls, and how false positives were tested

flux.2/ is the in-set clean control: an AI generator that applies no provenance watermark, so it separates "carries a watermark" from "was generated by a machine" — the confound that defeats most naive detectors.

Two further clean sets were used in our own false-positive testing but are not redistributed here, because they are someone else's to distribute:

• —Midjourney — roughly 780 images, mixed sizes, no provenance watermark.
• —Unsplash — photographs from the Unsplash Dataset, which is free for research use and should be obtained from Unsplash directly under their terms.

A detector that scores well against FLUX but fires on photographs has learned "is an AI render", not "carries a watermark". Testing against real photography is not optional.

What we used it for

We used this set, together with other methods we are not publishing, to reverse-engineer how SynthID-style watermarks appear in the two vendors' output, and to build a detector that reports both whether an image carries a provenance watermark and which vendor applied it.

The undisclosed part. The techniques that were most informative involve probing the generation APIs with synthetic inputs and observing what the services do, decline to do, and vary between calls. Describing them precisely would amount to publishing a recipe for extracting and therefore potentially forging these watermarks, and would expose implementation details of Google's and OpenAI's systems that are theirs to disclose, not ours. We have kept those methods, and the artifacts derived from them, out of this repository. What is here is the ordinary prompt-and-render data, which reveals nothing that generating your own images would not.

We publish the dataset and withhold the method deliberately: the data lets others reproduce and check detection work without repeating the API spend; the method would help someone defeat the thing the data is for.

What this data alone will get you

Working only from the images published here, and nothing else, we were able to reconstruct:

The figures below were measured on the initial release — 300 prompts across three generators — and are reported at that size deliberately, because a reconstruction result is only meaningful next to the number of images that produced it. They are not restated as the set grows.

Both figures are measured against a reference we derived by other means, and in each case the reference was built from material that is not in this dataset — so neither number is inflated by comparing the data against itself. For OpenAI the reference comes from images of a kind that appear nowhere in this set; for Gemini it comes from the images we held back when selecting the published prompts.

Two things to keep in mind about that percentage:

• —It is a floor, not a ceiling. The comparison is between two independently noisy reconstructions, and noise in either one drags the measured agreement down. The true overlap is higher than 8-12%; how much higher we cannot say precisely, because we have no noise-free copy of either fingerprint to compare against. The Gemini figure is the more pessimistic of the two, because the reference it is measured against was itself built from under a hundred images.
• —The two vendors behave differently as images are added. Averaged over three random orderings of the set, OpenAI climbs to +0.099 / +0.115 / +0.122 / +0.121 at 50 / 100 / 200 / 300 images — it has essentially converged by 200, and more OpenAI images would buy little. Gemini climbs +0.035 / +0.049 / +0.067 / +0.079 over the same range and is still rising at 300, so its reconstruction is the one that is image-starved, not method-limited. That asymmetry is itself a result: the two watermarks do not give themselves up at the same rate.

A tenth of a fingerprint sounds modest and is not: the recovered fraction is enough to identify a vendor's output reliably, because the fingerprint is spread across the whole frame and a partial reconstruction still aligns with it everywhere. What the remaining fraction buys is margin — against compression, resizing, and images that are unlike the ones in this set.

One caution on the names in that table. We call these the vendors' fingerprints because they are present in the vendors' output and absent from flux.2/, which is what the evidence supports. We have not independently confirmed that either recovered pattern is SynthID rather than some other consistent artifact of the same pipeline — neither vendor's public checker will adjudicate the question on the inputs that would settle it. Read the table as "the vendor-specific signal we can recover", and treat the SynthID label as the likely identification it is.

We are not describing how the reconstruction is done. See What we used it for above.

Suggested uses

• —Train or evaluate a watermark detector with a genuine same-prompt clean control.
• —Study C2PA in the wild: signed manifests, two issuers, two different action vocabularies, and a set with none at all.
• —Test robustness — every file is the original bytes as delivered, so JPEG re-encoding, resizing and cropping can be applied from a known-clean starting point.
• —Check a detector for the "AI render" confound using flux.2/ before claiming watermark detection.

Caveats

• —Nobody owns the images. Prompt-generated model output has no human author and so no copyright under current US law; we claim none and grant none. The vendors' terms of service bind whoever called their APIs, not whoever downloads this. See LICENSE.md.
• —Model versions drift. Generated September 2026. A watermark is tied to the encoder version that applied it, and vendors update encoders; results on this set may not transfer to output generated a year from now.
• —The Gemini files are JPEG, as delivered by the API — which only returns JPEG. They use all-ones quantisation tables (no quantisation loss) but 4:2:0 chroma subsampling, so the luma plane is effectively lossless while chroma is half-resolution. Measure on luma unless you have a reason not to.
• —`c2pa.watermarked.unbound` is not the word "SynthID". OpenAI's manifests assert an unbound (soft-binding) watermark without naming the scheme. Do not infer from the absence of the string "SynthID" that no SynthID-family watermark is present.
• —This is not a benchmark. It is enough to develop against and to reproduce our numbers; it is not enough to make claims about detection rates in the wild.

Licence and attribution

The curation — the selection and ordering of the prompts, prompts.txt, metadata.csv and this documentation — is licensed [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Use it for anything, including commercially, provided you credit the source.

The images carry no copyright — under current US law, model output generated from a prompt has no human author and is not protectable, so neither we nor the vendors hold a right in them and none is granted or needed. See LICENSE.md for the detail, including why a vendor's terms of service is a contract binding us rather than a right that follows the files to you.

If you use this set in a publication, please cite it:

@misc{synthid_research_set_2026,
  title        = {SynthID Research Set: matched same-prompt renders across image generators},
  author       = {Mark Adams},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/datasets/bigdatamark/synthid-research}},
  note         = {Matched same-prompt renders from gpt-image-2, Gemini 3.1 Flash Image and FLUX,
                  with C2PA provenance metadata. Curation licensed CC BY 4.0.}
}

And if you build on the detection work rather than the data, please say which parts you verified independently — several of our own earlier conclusions were withdrawn after better controls, and we would rather that habit spread than our numbers.

My OpenAI SynthID-O and Google SynthID Oracle are free to use here: [Scrubb.io](https://www.scrubb.io/)