naomiKenKorem/HDR-user-study

Text-to-HDR — Pairwise User Study

Pairwise human comparison of three text-to-HDR methods (HDR-LTX, X2HDR, LEDiff) on 30 cinematic prompts. Raters see 90 stacked 3-EV bracket pairs and pick which row looks more natural / more like a real photograph.

Files in this Space

• —app.py — Gradio app
• —pairs.json — 90 pair definitions (with hidden top/bottom assignment)
• —prompts.json — 30 source prompts
• —pairs/pair_NNN.png — 90 stacked-bracket comparison images
• —requirements.txt — gradio, huggingface_hub

Vote storage

Each completed rater session writes one JSONL file (votes/votes_<rater_id>.jsonl) to a private HF dataset repo (HF_DATASET_REPO env var). One line per pair, with the chosen label and the recorded top/bottom method assignment.

Local dev

pip install -r requirements.txt
python app.py

When HF_TOKEN / HF_DATASET_REPO env vars are not set, the app runs locally and stores votes in votes_<rater_id>.jsonl next to app.py (no upload).

Deploy to a free HF Space

# 1. create the space + dataset on huggingface.co (or via the CLI)
# 2. set Space secrets:
#    HF_TOKEN          — write-scoped token for the dataset
#    HF_DATASET_REPO   — e.g. "naomi/t2hdr-user-study-votes"
# 3. push the contents of this directory to the Space repo:
huggingface-cli login
git lfs install
git clone https://huggingface.co/spaces/<your-username>/t2hdr-user-study
cp -r * /path/to/cloned/space/
cd /path/to/cloned/space
git lfs track "pairs/*.png"
git add . && git commit -m "Initial study upload" && git push

Scoring

After the study closes, run score_study.py (TODO) to download all JSONL files from the dataset repo, compute Thurstone Case V scores per method, and write results.json.