build-small-hackathon/ink-witch

1---2title: InkWitch3emoji: 🪄4colorFrom: indigo5colorTo: purple6sdk: gradio7sdk_version: 6.16.08python_version: '3.12'9app_file: app.py10pinned: false11tags:12  - track:wood13  - sponsor:openbmb14  - achievement:offgrid15  - achievement:offbrand16---17 18# 🪄 InkWitch19 20> **Build Small Hackathon submission**. *An Adventure in Thousand Token Wood* Track21 22Your research grant is running dry, and the loan shark has sent you an ultimatum for the debt. 23He repossessed your lab and turned it into a witchcraft store. 24 25To keep your roof and clear the debt, you serve the customers who wander in — each wants an object with a few specific26qualities, *a key with wings*, *a mug with a cat face*, etc.27 28Sketch glyphs and transmute them to fulfill the customers' orders, so you can receive payment and build your reputation. But watch out, every stroke drains your mana. Reinvest wisely, sketch sparingly, and DON'T GO BANKRUPT!29 30## Model Pipeline311. **Glyph appraisal** — `MiniCPM-V-4.6` looks at your raw glyph and scores it32   for *recognition* (is it the right object at all?) plus a confidence on each33   requested *quality*.342. **Transmutation** — if the object is recognized, your glyph is fed as a35   scribble control image into an `SDXL-Flash + Scribble-ControlNet` pipeline,36   which renders a finished illustration that follows your lines.373. **Re-check** — the VLM then re-judges the *transmuted image*, but only on the38   qualities your glyph **failed**.39   Your gold and reputation reflect this final artwork — so a rough glyph that40   the transmutation fleshes out faithfully can win back reputation it missed.41 42In other words, you're judged not just on what you drew, but on what your glyph43*becomes*. The AI is load-bearing twice over: once as the judge, once as the artist.44 45## Models (all running on the Space — Off the Grid)46 47| Role | Model | Params |48|---|---|---|49| Vision judge | [`openbmb/MiniCPM-V-4.6`](https://huggingface.co/openbmb/MiniCPM-V-4.6) | ~1.3B |50| Image generator | [`sd-community/sdxl-flash`](https://huggingface.co/sd-community/sdxl-flash) | ~2.6B |51| Scribble control | [`xinsir/controlnet-scribble-sdxl-1.0`](https://huggingface.co/xinsir/controlnet-scribble-sdxl-1.0) | ~1.25B |52| VAE | [`madebyollin/sdxl-vae-fp16-fix`](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix) | ~84M |53 54The image pipeline is adapted from55[`linoyts/scribble-sdxl-flash`](https://huggingface.co/spaces/linoyts/scribble-sdxl-flash).56No external inference APIs are called — both heavy models are eagerly loaded at57Space startup and every appraisal + transmutation happens inside one `@spaces.GPU`58acquisition.59 60## Tech Notes61 62- **Custom frontend on `gr.Server`** — `/` serves a hand-written `index.html` +63  `app.js` + `style.css` (the drawing canvas, shop, and night phase), driven64  through the official `@gradio/client` so ZeroGPU per-user quota is attributed65  correctly. `app.launch(ssr_mode=False)` keeps Gradio's SSR from shadowing it.66- **Per-user state** is keyed on `gr.Request.session_hash`, so simultaneous67  players get independent games.68- **One GPU call per Transmute** — glyph appraisal, transmutation, and re-check69  are fused into a single `@spaces.GPU` function.70- **Hidden test mode** — click **Reset** 5 times in a row to flip on a test session: 71  unlimited gold/mana, the full request pool, and the image-gen tuning panel. 72  Refresh the page to return to a normal run.73 74## Team Member75- [gizzleon](https://huggingface.co/gizzleon)76 77## 🎥 Demo video78 79https://youtu.be/3R139SneUzA?si=zGK5RLvHhFSDOs4480 81## 📣 Social post82 83https://www.reddit.com/r/aigamedev/comments/1u55h2u/webbased_ai_doodle_game/84 85## Run it locally86 87Only the **image generation** stage (SDXL-Flash + ControlNet) requires a CUDA88GPU — it does not run on CPU/MPS. The MiniCPM judge runs fine without one, so on89a machine with no CUDA GPU, disable image-gen with `WITCH_WOOD_GEN_ENABLED=0`.90 91```bash92# Set up the project with uv (https://github.com/astral-sh/uv)93uv sync94 95# Full pipeline — real MiniCPM judge + image generation (image-gen needs a CUDA GPU):96python app.py97 98# No CUDA GPU? Disable image-gen only — the real MiniCPM judge still runs99# (on CPU/MPS) and the SDXL pipe is never loaded:100WITCH_WOOD_GEN_ENABLED=0 python app.py101 102# Fully model-free — skips ALL model calls (mock judge, no image generation):103WITCH_WOOD_MOCK=1 python app.py104```105 106Runs on Hugging Face Spaces **ZeroGPU** in production.107