LawrenceYin/CRISP_AtomAudit_Survey_v2

CRISP-AtomAudit Survey

Standalone Streamlit survey app for the CRISP human audit. Suggested HuggingFace Space name: crisp-atom-audit-survey. It follows the same interaction pattern as the previous accelaes-survey Space, but it is a separate survey: annotators enter a handle, receive a deterministic subset, can resume later, and responses are upserted into this survey's own responses.csv.

Files

• —app.py: Streamlit app.
• —audit_manifest.jsonl: one row per image with prompt and atom questions.
• —images/: copied audit images.
• —responses.csv: created after the first submission and synced to a HF Dataset when credentials are configured.
• —assignments.csv: created automatically to keep each annotator's assigned image list stable while prioritizing globally least-audited images.
• —selected_images.csv, annotation_atoms.csv: source tables for analysis/reference.

Run Locally

cd /home/runkai/xuanhua/CRISP/outputs/human_audit_survey_space
streamlit run app.py --server.port 7860

HuggingFace Space

Create a new Streamlit Space and upload this folder. Do not reuse the old accelaes-survey Space or response dataset.

Recommended environment variables/secrets:

• —N_REQUIRED_IMAGES: number of images per annotator, default 60.
• —RESPONSE_DATASET: HF Dataset repo for syncing audit state. Default: LawrenceYin/crisp-atom-audit-responses.
• —HF_TOKEN: token with write access to RESPONSE_DATASET.
• —Backward-compatible fallback: if a secret named response exists, the app treats it as the HF token.

The app also exposes sidebar download buttons for responses.csv and assignments.csv whenever local copies exist.

Response Format

responses.csv stores one row per (annotator, annotation_id) with:

• —image metadata: image_id, prompt_id, backbone, split, atom_count, prompt, image_path
• —question metadata: annotation_id, question_id, question, expected_answer, skill
• —annotation: human_label (Yes/No or blank), human_uncertain, notes, timestamp