pamela-dataset/pamela

<div align="center"> <h1 style="font-family: serif; font-size: 3rem; margin-bottom: 0;">PAM∃LA</h1> <p style="font-family: serif; font-size: 1.5rem; color: gray;">Personalizing Text-to-Image Generation to Individual Taste</p> <p><em>Anonymous submission — author and affiliation details withheld during review.</em></p> </div>

PAM∃LA is a dataset of AI-generated images rated by human participants for aesthetic quality, built specifically for personalization research. It pairs each rating with rich participant demographics and image metadata, enabling research on personalized aesthetic prediction, demographic variation in visual preference, and reward modelling for generative content.

Dataset Statistics

Splits ending in `_seen` evaluate model performance on known users rating new images. Splits ending in `_unseen` evaluate generalization to completely new users — the harder, more realistic setting for personalization research.

Evaluation settings

Each sample contains a text prompt, a generated image, a user ID, and a preference rating. PAM∃LA defines two complementary tasks for predicting that rating:

• —Seen users. Predict ratings for new prompt–image pairs rated by users already present in the training set.
• —Unseen users. Predict ratings for users not present in the training set.

Image content

• —5,077 unique images
• —21 visual groups: abstract, animal, architecture, automotive, cinematic, cityscape, fashion, flower painting, food, landscape (art and photorealism), lifestyle, portrait (art and photorealism), product, scene, special, sports, still life, street, travel
• —26 art-historical styles applied to the Art subset (e.g., Impressionism, Synthetic_Cubism, Art_Nouveau_Modern, Romanticism, Baroque, Cubism, ...). Photorealism images carry no style label (null).
• —2 image types: Art (26,237 ratings) and Photorealism (43,667 ratings)

The 5,077 images split into 1,977 artistic images, generated from prompts sourced from the LAPIS benchmark, and 3,100 photorealistic images, generated from a curated set of thematic prompts. All images were synthesized with Flux 2 and Nano Banana (Gemini 2.5 Flash Image). Prompts vary along two orthogonal axes — visual style (how an image is rendered) and semantic content (what the image depicts) — to disentangle annotator preferences for style and semantic content. All images were manually reviewed to remove harmful or not-safe-for-work content prior to collection.

Annotation protocol

Ratings were collected on the Mabyduck online study platform. Each image was presented in isolation and rated on a 5-point slider (continuous score 1.0–5.0). The slider uses a click-drag-confirm format (cannot click through). Sessions were ≤20 minutes and compensated at UK minimum wage. Post-hoc checks (range use, no streaks) confirmed no participant exclusions were needed. The platform also collects participant demographic metadata (age, gender, nationality), used here to inform personalized predictors.

Participants

• —199 unique participants
• —Age range: 19–60
• —Gender: male (40,525), female (29,086), prefer not to say (293)
• —23 nationalities represented

Field reference

Dataset Structure

Each entry in the annotation JSON files has the following schema:

{
  "image_path": "./images/2f8e75f4-da55-4074-9328-052199f7cfb0.png",
  "image_id": 1049071,
  "participant_id": "p_5d2aeb40",
  "image_metadata": {
    "group": "landscape",
    "style": "Art_Nouveau_Modern",
    "category": "AI image",
    "prompt": "A grand palace, rendered in muted greens and greys ...",
    "type": "Art"
  },
  "user_demographics": {
    "age": 28,
    "age_binned": "twenties",
    "gender": "male",
    "nationality": "United States"
  },
  "quality_label": "good",
  "original_score": 3.497
}

Annotator Demographic Distribution

Per participant

Age: mean 36.19, std 9.62, median 35, min 19, max 60.

Per rating

Agreement Metrics

All 5,077 images have at least two ratings and are included in the inter-rater reliability analysis. Images receive 13.77 ratings on average (median 14, max 17), for 69,904 ratings used in the analysis.

Krippendorff's α

Intraclass correlation

ICC(2,k̄) and ICC(3,k̄) use mean k̄ as an approximation because the number of raters per image varies.

Notes

• —The participant pool (199 users, 23 nationalities) is not demographically representative of the global population. Aggregated scores should not be treated as universal aesthetic judgments.
• —Subgroups defined by combinations of demographics (e.g., women aged 50+ from a specific country) may be very small. Avoid claims about narrow subpopulations without checking sample sizes.

Intended use

PAM∃LA is built for personalization research — modelling individual aesthetic preference for AI-generated imagery. It is not intended for establishing population-level aesthetic norms, for image-quality assessment in the technical sense (artefacts, distortion, fidelity), or for aesthetic judgements outside the 21 visual groups and 26 art styles included.

Hosting & Licensing

Primary host. HuggingFace Hub at `pamela-dataset/pamela`. PNG images are stored via Git LFS (per the standard HF .gitattributes filter); annotation JSONs are committed as plain text. Releases are tagged via Git tags on the Hub; the main revision tracks the latest curated version. No mirror host is currently maintained.

Dataset license. The annotations, prompts, dataset structure, README, and the accompanying croissant.json metadata file are released under CC-BY-4.0. You may share and adapt the material with attribution.

Image license. The 5,077 PNG images are AI-generated outputs of two text-to-image models — Nano Banana (Gemini 2.5 Flash Image) and FLUX.2 (Black Forest Labs). We release these images alongside the annotations under CC-BY-4.0, but downstream users should additionally observe the generators' terms of service governing model outputs (Google Generative AI Terms; BFL FLUX.2 license). Where those terms restrict commercial use of outputs, that restriction propagates to the images.

Personal data. The dataset contains no directly-identifying information. We do not release or use Mabyduck-provided user IDs: although Mabyduck changes rater IDs between experiments, releasing those IDs could create a re-identification risk if the ID-generation procedure were inferred. Instead, participant IDs are dataset-local pseudonyms (p_* hashes) that are not linked to names, emails, IP addresses, devices, or Mabyduck account identifiers. Released demographics are limited to age, gender, and nationality; other collected demographics, such as ethnicity and student status, were removed because we do not assume they are indicative of aesthetic preference and excluding them further reduces re-identification risk. Demographics were collected with informed consent obtained prior to participation. Collection took place in February 2026 on the Mabyduck online study platform.

Responsible AI summary

A full machine-readable RAI section is shipped in croissant.json (rai:dataLimitations, rai:dataBiases, rai:personalSensitiveInformation, rai:dataUseCases, rai:dataSocialImpact, rai:hasSyntheticData, plus prov:wasDerivedFrom / prov:wasGeneratedBy). Headline points:

• —Synthetic data: all images are AI-generated by Nano Banana and FLUX.2; aesthetics may reflect the failure modes and stylistic biases of these specific models.
• —Demographic coverage: convenience sample of 199 participants across 23 nationalities, with heavier concentration in a few countries.
• —Sensitive attributes captured: age, gender, nationality. Raw Mabyduck user IDs and additional demographics such as ethnicity and student status are not released. No health, political, religious, biometric, contact, or geolocation data.
• —Validated for: personalized aesthetic prediction, demographic-bias auditing of T2I aesthetics, RLHF reward modelling for personalization, style/group-conditional preference analysis.
• —Not validated for: general (non-AI-generated) image aesthetics, universal aesthetic norms, technical image-quality assessment, identity inference.

Citation

Anonymous Authors. PAM∃LA: Personalizing Text-to-Image Generation to Individual Taste. Under review, 2026.