Cerberus149/remembrance-soundscapes

Image-to-Sound Pipeline

Turns an uploaded image into a 30-second soundscape using AI vision + audio generation. Part of the Hope & Despair project — outputs are pushed to GitHub and displayed as an interactive visualization at fraloisio.github.io/remembrance-soundscapes.

What it does

1. 1.Upload an image in the Generator tab.
2. 2.Vision analysis with Qwen/Qwen3-VL-8B-Instruct produces:
3. 3.TITLE — short name for the soundscape
4. 4.ANALYSIS — visual description of the image
5. 5.LONG PROMPT — sound-design brief, fed directly to the audio model
6. 6.Audio generation with Stable Audio Open 1.0:
7. 7.~30 seconds, 150 diffusion steps, CFG 12
8. 8.Files saved locally and pushed to GitHub:
9. 9.images/ — the uploaded image (.png)
10. 10.audio/ — the generated soundscape (.wav)
11. 11.texts/ — metadata including title, analysis, prompt (.txt)
12. 12.records.csv — updated index of all runs

⚡ Switching hardware

Edit the two lines at the top of this file (the frontmatter between ---) on GitHub. requirements.txt and app.py work for both modes — nothing else to touch. Save → Space rebuilds → pick hardware in Space Settings → Hardware.

ZeroGPU (free):

sdk: gradio
sdk_version: 5.25.0

Paid GPU (A100 etc.):

sdk: docker

GitHub mirroring (required for visualization)

Each completed run is pushed to a GitHub repo via the Contents API. Set these secrets in Space Settings:

• —GITHUB_TOKEN — personal access token (repo write scope)
• —GITHUB_REPO — e.g. fraloisio/remembrance-soundscapes
• —GITHUB_BRANCH — main
• —HF_TOKEN — Hugging Face token (write scope)

Without these, files are saved locally but the visualization won't update (local files are wiped on Space restart).

Limitations

• —The generated long prompt is used directly for audio — no manual editing step.
• —Duration, steps, and CFG are fixed in code.
• —No content moderation — review outputs yourself.