build-small-hackathon/hackathon-advisor

1---2title: Hackathon Advisor3emoji: "📜"4colorFrom: yellow5colorTo: green6sdk: gradio7sdk_version: 6.16.08python_version: "3.11"9app_file: app.py10pinned: true11license: mit12short_description: Originality advisor for small-model project ideas.13tags:14  - gradio15  - build-small-hackathon16  - small-models17  - agent18  - originality19  - track:wood20  - sponsor:openbmb21  - sponsor:openai22  - sponsor:nvidia23  - sponsor:modal24  - achievement:offgrid25  - achievement:welltuned26  - achievement:offbrand27  - achievement:llama28  - achievement:sharing29  - achievement:fieldnotes30  - tiny-titan31  - best-demo32  - best-agent33  - bonus-quest-champion34models:35  - openbmb/MiniCPM5-1B36  - build-small-hackathon/hackathon-advisor-minicpm5-lora37  - build-small-hackathon/hackathon-advisor-quest-minicpm5-lora38  - ggml-org/embeddinggemma-300m-qat-q8_0-GGUF39  - nvidia/nemotron-speech-streaming-en-0.6b40datasets:41  - build-small-hackathon/hackathon-advisor-quest-dataset42  - build-small-hackathon/hackathon-advisor-codex-traces43---44 45# Hackathon Advisor46 47**Hackathon Advisor** is a live map of the Build Small Hackathon and a small-model originality coach for builders. It48opens on an atlas of public `build-small-hackathon` Spaces, then lets a builder search the field, inspect project49clusters, see quest evidence, and open **The Unwritten Almanac** to evaluate an idea against the work already on the50trail.51 52The [Build Small Hackathon](https://huggingface.co/build-small-hackathon) asks participants to build under a 32B53parameter cap, solve a concrete problem for someone nearby or make a delightful AI-native experience, and submit a Space,54demo video, and social post. Hackathon Advisor treats that setting as the data surface: every public Space becomes part55of a continuously refreshed project atlas, and every advisor response is grounded in that shared map.56 57## Demo58 59- Live app: <https://build-small-hackathon-hackathon-advisor.hf.space>60- Hugging Face Space: <https://huggingface.co/spaces/build-small-hackathon/hackathon-advisor>61- Source code (GitHub): <https://github.com/JacobLinCool/hackathon-advisor>62- Demo video: <https://youtu.be/Gq-FUiL-ZPw>63- Social post: <https://x.com/JacobLinCool/status/2066156056724848965>64- Team HF username: [`JacobLinCool`](https://huggingface.co/JacobLinCool)65- Submission notes: [`docs/submission-notes.md`](docs/submission-notes.md)66- Start at the Idea Map, search for a theme, click nearby projects, hover quest badges for evidence, and open the67  advisor when you are ready to test an idea.68 69## What This Establishes70 71Builders enter a fast-moving hackathon with limited context. A promising idea can already be crowded, a quiet niche can72be hard to see, and prize alignment can be scattered across READMEs, tags, and app files. Hackathon Advisor turns the73field itself into the starting point. The app shows where projects cluster, which submissions sit near each other, which74quests they appear to satisfy, and where a new idea may still have room to breathe.75 76The project also exists because the dataset is already there. A public hackathon organization is a living corpus of77Spaces, READMEs, model declarations, app files, tags, and demos. Once that corpus is indexed, builders can explore what78others are making, which methods they are using, and which results are emerging across the field.79 80That visibility gives an online hackathon some of the creative force of OpenAI's Parameter Golf challenge: the event81becomes a shared surface for ideas to interact while people are still building. Participants can find adjacent work,82recognize overlap, borrow useful patterns, extend ideas into new domains, and connect with others working on similar83problems. The result is a faster open-source feedback loop, where good ideas become easier to find, improve, and build84on together.85 86The atlas is the default experience because the map is the evidence. The advisor is available behind `Open advisor`,87where it uses the same project snapshot to cite overlap, propose whitespace, score the idea, draft a build plan, and88export the session evidence.89 90## Hackathon Submission91 92Hackathon Advisor is submitted primarily for the **Thousand Token Wood** track. The product is useful to builders, but93its core form is an AI-native field guide: a living map, an almanac voice, quest evidence, and shareable artifacts that94make the hackathon field itself explorable.95 96The demo video is part of that submission evidence. It is built from real app footage of the atlas and advisor flows. Codex helped draft the storyboard, drive the app, capture the screen, generate voice-over, compose the cut, and verify frames and ASR transcripts against the intended narration.97 98The Space is also targeting the official sponsor and achievement tags shown in the README front matter:99 100- `sponsor:openbmb`: MiniCPM5-1B is the central planner and quest-classifier base model.101- `sponsor:openai`: Codex served as the engineering partner across the build. It helped translate the hackathon102  requirements into implementation slices, inspect and revise the repository, implement the atlas103  refresh/storage/search paths, add the quest-evidence UI, run tests and browser checks, review deployed Space behavior,104  prepare commits and deployment updates, produce the demo storyboard/app-footage/voice-over checks, and refine the105  README narrative. The redacted project-facing Codex session traces are published as a Hugging Face dataset.106- `sponsor:nvidia`: voice input runs `nvidia/nemotron-speech-streaming-en-0.6b` through NVIDIA NeMo ASR.107- `sponsor:modal`: Modal is used for development compute, including the quest LoRA training path and remote index-build108  path, and is documented in this README.109- `achievement:offgrid`: runtime inference is local to the Space process; no proprietary cloud inference API is called.110- `achievement:welltuned`: the advisor and quest classifier use published MiniCPM LoRA adapters.111- `achievement:offbrand`: the app uses a custom `gradio.Server` frontend instead of the stock Gradio interface.112- `achievement:llama`: retrieval embeddings run through llama.cpp via `llama-cpp-python`.113- `achievement:sharing`: redacted Codex build traces are published as a Hugging Face dataset.114- `achievement:fieldnotes`: the repo includes build reports and field-note exports.115 116Additional prize evidence for Tiny Titan, Best Agent, Best Demo, and Bonus Quest Champion is summarized in117[`docs/submission-notes.md`](docs/submission-notes.md).118 119## What You Can Do120 121- Explore a full-screen t-SNE atlas of public hackathon Spaces, with KMeans clusters and nearest-neighbor links.122- Search projects with BM25 over titles, slugs, summaries, tags, declared models, cluster labels, quest evidence, README123  text, and declared app-file source.124- Filter by cluster or quest, then inspect the selected project's summary, Space link, tags, quest matches, and evidence125  hints.126- Chat with the atlas: the "Ask the atlas" drawer answers questions like "who completed the most quests" or "what127  clusters exist" through the BASE MiniCPM5-1B model's native tool calling, with thinking enabled — the reasoning128  trace streams live into a collapsible block. Verified tool results render as cards and can filter or highlight the129  map directly; the model's prose is grounded on a compact digest of the same result.130- Refresh the atlas from the Space backend; validated artifacts are written to the mounted cache directory and swapped131  into the live app atomically.132- Open the advisor workspace for idea comparison, gap exploration, score seals, profile-aware plans, voice input, and133  shareable exports.134- Export from the workspace UI: build notes, the Almanac chapter, and the page PNG. Further reviewer artifacts — trace135  JSONL, demo bundle, submission packet, LoRA dataset, and LoRA training kit — are served through the API endpoints136  listed below.137 138## How It Works139 140The refresh path snapshots public Spaces in the `build-small-hackathon` organization, reads each README and declared141main app file, rebuilds the EmbeddingGemma project index, resolves official `track:*`, `sponsor:*`, and142`achievement:*` metadata tags first, then asks MiniCPM only for quest evidence that metadata did not declare. The active143dashboard contains project points, nearest links, clusters, quest coverage, provenance, and refresh state.144 145`ADVISOR_CACHE_DIR` is the artifact store. On Hugging Face Spaces it points to the mounted Storage Bucket; locally it can146be a normal directory such as `.cache/advisor-dashboard`. Each refresh writes147`runs/{run_id}/projects.json`, `project_index.json`, `dashboard.json`, `quest_analysis.json`, and `manifest.json`, then148updates `latest.json` through an atomic swap. Quest analysis is cached per project using the metadata-first inference149prompt hash, taxonomy hash, MiniCPM model id, adapter id/revision, local adapter digest, and generation config.150 151The app can start an hourly scheduler when `ADVISOR_CACHE_DIR` is configured. The submitted Space keeps scheduled152refresh disabled and updates the mounted bucket from local builds, so index construction and quest analysis do not run153inside the public demo runtime. Manual and scheduled refreshes still acquire `$ADVISOR_CACHE_DIR/refresh.lock`,154heartbeat while active, and leave the current validated dashboard in place if a new run fails validation.155 156## Models And Data157 158| Role | Model | Runtime | Evidence |159| --- | --- | --- | --- |160| Advisor | [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) + [`build-small-hackathon/hackathon-advisor-minicpm5-lora`](https://huggingface.co/build-small-hackathon/hackathon-advisor-minicpm5-lora) | ZeroGPU, Transformers, PEFT | A 1.08B OpenBMB model plans which tool to call each turn; advisor prose is rendered from deterministic templates grounded in the retrieved tool results. |161| Quest analysis | Official Space metadata + [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) + [`build-small-hackathon/hackathon-advisor-quest-minicpm5-lora`](https://huggingface.co/build-small-hackathon/hackathon-advisor-quest-minicpm5-lora) | ZeroGPU, Transformers, PEFT | The analyzer trusts official README tags for declared tracks, sponsor prizes, and badges, then uses a task-specific MiniCPM LoRA to classify remaining README/app-file evidence into strict quest JSON. |162| Project retrieval | [`ggml-org/embeddinggemma-300m-qat-q8_0-GGUF`](https://huggingface.co/ggml-org/embeddinggemma-300m-qat-q8_0-GGUF) | Local llama.cpp index build plus llama.cpp query embeddings | The atlas and retrieval index use a GGUF embedding model through llama.cpp. |163| Voice input | [`nvidia/nemotron-speech-streaming-en-0.6b`](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b) | ZeroGPU; NVIDIA NeMo ASR | Voice notes are transcribed with NVIDIA NeMo using the same Nemotron model in local and deployed runs. |164 165MiniCPM is loaded following the official demo shape (`trust_remote_code=True`, `bfloat16`, and166`apply_chat_template(..., enable_thinking=False)`) for stable tool calls and strict quest JSON.167 168| Data / released material | Link | How it is used |169| --- | --- | --- |170| Hackathon project corpus | [`build-small-hackathon`](https://huggingface.co/build-small-hackathon) | Public Spaces are crawled as the live field for the atlas, search, advisor citations, and quest coverage. |171| Project snapshot | [`data/projects.json`](https://huggingface.co/spaces/build-small-hackathon/hackathon-advisor/blob/main/data/projects.json) | Stores Space metadata, README text, declared models/datasets, tags, and declared app-file evidence. |172| Project embedding index | [`data/project_index.json`](https://huggingface.co/spaces/build-small-hackathon/hackathon-advisor/blob/main/data/project_index.json) | Stores normalized EmbeddingGemma vectors and retrieval metadata for map construction and advisor search. |173| Quest SFT dataset | [`build-small-hackathon/hackathon-advisor-quest-dataset`](https://huggingface.co/datasets/build-small-hackathon/hackathon-advisor-quest-dataset) | Trains the MiniCPM quest classifier from README/app-file prompts with source-attributed quest labels. |174| Codex session traces | [`build-small-hackathon/hackathon-advisor-codex-traces`](https://huggingface.co/datasets/build-small-hackathon/hackathon-advisor-codex-traces) | Publishes real Codex session logs for this project after selection, minimization, and OpenAI Privacy Filter redaction. |175| Advisor LoRA examples | `lora_dataset` and [`/api/lora-training-kit.zip`](https://build-small-hackathon-hackathon-advisor.hf.space/api/lora-training-kit.zip) | Regenerates chat JSONL examples, recipe metadata, and the adapter card from exact advisor sessions. |176 177## How Codex Was Used178 179[Codex](https://developers.openai.com/codex) served as the engineering partner for the project. It helped translate the180hackathon requirements into implementation slices, inspect the existing codebase, build the atlas refresh/storage/cache181path, add the dashboard search and quest-evidence UI, run local tests and browser checks, review deployed Space behavior,182prepare commits and deployment updates, and revise the README into a submission narrative. This was an evidence loop:183Codex could read the repository, operate the local or deployed app, inspect the result, and then revise the184implementation or presentation from what it observed. The live app runtime uses the models and data listed above; Codex185appears in the development record as the assistant that helped design, implement, validate, and document the system.186 187The demo video used the same agentic workflow. Codex helped draft the narrated storyboard, drive the live app, capture188real screen footage, generate voice-over, compose the final video, and check the artifact by reading frames and ASR189transcripts. That media workflow ran outside Hackathon Advisor's submitted runtime and is not counted in the model190budget, but it documents how Codex was used to produce and verify submission materials rather than only code changes.191 192The redacted session-level Codex traces are published as a Hugging Face dataset at193[`build-small-hackathon/hackathon-advisor-codex-traces`](https://huggingface.co/datasets/build-small-hackathon/hackathon-advisor-codex-traces).194 195The full development history is public at <https://github.com/JacobLinCool/hackathon-advisor>.196 197## Prize Evidence198 199This submission targets the **Thousand Token Wood** main track, plus the OpenBMB, OpenAI/Codex, NVIDIA, and Modal200sponsor awards and the six bonus-quest badges.201 202| Prize path | Implemented evidence |203| --- | --- |204| Thousand Token Wood | The Almanac and Idea Map make the AI output visible as a playful, evidence-grounded exploration surface; the embedding index and the MiniCPM tool loop are load-bearing for the whitespace and originality experience. |205| Off the Grid | Every model runs from open weights on the Space's own GPU/CPU (or a local box); no third-party inference API is called at runtime, and retrieval vectors are local and embedded through llama.cpp. |206| Well-Tuned | Two MiniCPM5-1B PEFT LoRA adapters (advisor + quest classifier) are published publicly on the Hub; the local quest adapter is byte-identical to its published repo, and the training kit reproduces them. |207| Off-Brand | The custom `gradio.Server` frontend ships a bespoke atlas and Almanac experience, with no default Gradio UI in the runtime path. |208| Llama Champion | EmbeddingGemma GGUF vectors and every runtime query embedding run through llama.cpp; the index validator rejects any non-llama.cpp runtime. |209| Sharing is Caring | Real Codex session logs for this project are published on the Hub at [`build-small-hackathon/hackathon-advisor-codex-traces`](https://huggingface.co/datasets/build-small-hackathon/hackathon-advisor-codex-traces); the publisher selects project-relevant sessions, minimizes internal metadata, applies [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter), and records source hashes for audit. |210| Field Notes | A build report on the quest-classifier fine-tune is published at [`docs/quest-classification-lora.md`](docs/quest-classification-lora.md), and the app exports session Field Notes as markdown. |211| Tiny Titan | The largest single model is MiniCPM5-1B at ~1.08B — well under the 4B Tiny Titan ceiling; the full runtime stack totals ≈1.98B, far under the 32B cap. |212| Best Demo | The YouTube demo and public X post are linked from this README, and a backup MP4 is committed at [`static/assets/hackathon-advisor-demo.mp4`](static/assets/hackathon-advisor-demo.mp4). The video uses real app footage and a Codex-assisted production loop for storyboard, app driving, screen capture, voice-over generation, final composition, and frame/ASR verification. |213| OpenBMB | MiniCPM5-1B is the central language model for both tool planning and quest classification. |214| OpenAI Codex | Codex acted as an engineering partner across implementation, debugging, documentation, deployment review, demo production, and submission preparation. The public GitHub and Space histories use `Co-authored-by: Codex <codex@openai.com>` trailers, and selected project-facing session traces are published after minimization and privacy-filter redaction. |215| NVIDIA Nemotron | Voice input runs `nvidia/nemotron-speech-streaming-en-0.6b` through NVIDIA NeMo. |216| Modal | Modal trains the quest-classifier LoRA (`scripts/modal_train_quest_lora.py`), and a Modal remote index-build path is provided; the index shipped in this repo was built locally. |217| Best Agent | Each turn MiniCPM5 selects one tool; the engine then orchestrates the search → whitespace → score → plan chain over the live project field. |218 219## Run Locally220 221```bash222python3.11 -m venv .venv223. .venv/bin/activate224pip install -r requirements.txt225mkdir -p .cache/advisor-dashboard226ADVISOR_CACHE_DIR=.cache/advisor-dashboard \227ADVISOR_MODEL_BACKEND=minicpm-transformers \228ADVISOR_MODEL_ID=openbmb/MiniCPM5-1B \229ADVISOR_ADAPTER_ID=build-small-hackathon/hackathon-advisor-minicpm5-lora \230ADVISOR_ADAPTER_REVISION=25de69bcde397e1bcdd852923b56a42f10222650 \231ADVISOR_QUEST_ANALYZER_BACKEND=minicpm-transformers \232python app.py233```234 235Then open <http://127.0.0.1:7860>. The atlas refresh button runs locally with the same artifact swap path used in236deployment. It writes refreshed runs under `.cache/advisor-dashboard/runs/` and atomically updates237`.cache/advisor-dashboard/latest.json`.238 239## Refresh The Project Snapshot240 241```bash242python scripts/crawl_hf_spaces.py --org build-small-hackathon --out data/projects.json243python scripts/build_project_index.py --location modal --projects data/projects.json --out data/project_index.json244```245 246The checked-in development snapshot lives in `data/projects.json` and `data/project_index.json`. A configured247`ADVISOR_CACHE_DIR` supplies the latest validated dashboard artifacts.248 249## Publish Codex Trace Dataset250 251Local privacy-filter run:252 253```bash254uv run --with 'transformers>=5.6,<6' --with 'torch>=2.8,<3' \255  python scripts/publish_codex_trace_dataset.py \256  --project-root . \257  --repo-id build-small-hackathon/hackathon-advisor-codex-traces \258  --verbose259```260 261Faster Modal GPU run:262 263```bash264python scripts/publish_codex_trace_dataset.py --location modal \265  --project-root . \266  --repo-id build-small-hackathon/hackathon-advisor-codex-traces267```268 269The publisher scans `~/.codex/sessions` and `~/.codex/archived_sessions`, selects sessions that mention this project,270keeps project-facing Codex events, removes system/developer prompts and compaction internals, normalizes local paths,271caps long tool-output text with truncation counts in the manifest, applies OpenAI Privacy Filter to the published log272text, writes `codex_sessions.jsonl` and `dataset_manifest.json`, then uploads the filtered data to the configured273Hugging Face dataset. The Modal wrapper uploads the selected raw JSONL files to a private Modal Volume, runs the same274publisher core on a GPU, returns the filtered dataset to local disk, and performs the Hugging Face upload from local275credentials.276 277## API And Artifacts278 279| Surface | Purpose |280| --- | --- |281| `GET /api/dashboard` | Atlas points, links, clusters, quest report, provenance, and refresh status. |282| `GET /api/dashboard/search?q=...` | BM25 search over project, cluster, quest, README, and app-file text. |283| `POST /api/dashboard/chat` | Atlas chat turn (NDJSON stream): base-model tool call, verified result + map action, grounded answer. |284| `POST /api/dashboard/refresh` | Starts one background refresh job. |285| `GET /api/dashboard/refresh` | Reports refresh stage, result, and status. |286| `POST /api/transcribe` | Transcribes uploaded voice notes with NVIDIA NeMo and Nemotron ASR. |287| `GET /api/prize-ledger` | Model stack, parameter budget, runtime status, and prize evidence. |288| `GET /api/demo-bundle.zip` | Demo session JSON, prize ledger, trace, notes, chapter, LoRA files, submission packet, and PNG. |289| `GET /api/lora-training-kit.zip` | SFT data, recipe, adapter card, and training command. |290 291The Gradio API also exposes `trace_artifact`, `field_notes`, `chapter`, `lora_dataset`, and `submission_packet` for292submission evidence and reviewer inspection.293 294## Advisor Workspace295 296The advisor workspace preserves the working loop from the original app. `Ink` compares the current idea against the297project index, `Gap` rotates through unused whitespace candidates, `Plan` drafts a practical build path, and `Compare`298rescans the saved idea board to select the strongest page. The `Profile` panel adds skills, time, preferences, and299constraints to the plan so the output can reflect "one evening", "frontend prototyping", or "CPU-only Space" as real300scoping facts.301 302Each scored page includes a deterministic `wood_map`: background dots for indexed Spaces, red dots for closest cited303echoes, and a green/red point for the current idea. The live UI and PNG export use the same Pillow renderer.304 305## Runtime Backend306 307The deployed Space is configured for ZeroGPU inference with:308 309```bash310ADVISOR_ZERO_GPU=1311ADVISOR_ZERO_GPU_DURATION=120312ADVISOR_MODEL_BACKEND=minicpm-transformers313ADVISOR_MODEL_ID=openbmb/MiniCPM5-1B314ADVISOR_ADAPTER_ID=build-small-hackathon/hackathon-advisor-minicpm5-lora315ADVISOR_ADAPTER_REVISION=25de69bcde397e1bcdd852923b56a42f10222650316ADVISOR_QUEST_ANALYZER_BACKEND=minicpm-transformers317ADVISOR_QUEST_ADAPTER_ID=build-small-hackathon/hackathon-advisor-quest-minicpm5-lora318ADVISOR_QUEST_ANALYSIS_BATCH_SIZE=8319ADVISOR_CACHE_DIR=/data/advisor-cache320ADVISOR_DISABLE_SCHEDULED_REFRESH=1321ADVISOR_REFRESH_COMPUTE=cpu322ADVISOR_SCHEDULED_REFRESH_COMPUTE=cpu323ADVISOR_REFRESH_INTERVAL_SECONDS=3600324ADVISOR_REFRESH_INITIAL_DELAY_SECONDS=300325ADVISOR_REFRESH_LOCK_TTL_SECONDS=7200326ADVISOR_REFRESH_EMBEDDING_TIMEOUT_SECONDS=1800327ADVISOR_EMBEDDING_MODEL_REPO=ggml-org/embeddinggemma-300m-qat-q8_0-GGUF328ADVISOR_EMBEDDING_MODEL_FILE=embeddinggemma-300m-qat-Q8_0.gguf329ADVISOR_EMBEDDING_N_CTX=2048330ADVISOR_ASR_MODEL_ID=nvidia/nemotron-speech-streaming-en-0.6b331```332 333The retrieval query embedder downloads the GGUF model through `huggingface_hub` unless334`ADVISOR_EMBEDDING_MODEL_PATH` points to a local file. `/api/transcribe` uses the same ZeroGPU wrapper for Nemotron ASR.335On macOS local runs, the app automatically runs llama.cpp query embedding in a worker process so the MiniCPM PyTorch336runtime and llama.cpp stay isolated from each other's OpenMP runtime. Dashboard refresh also builds the GGUF embedding337index in a subprocess before returning to the app process for MiniCPM quest analysis. When338`ADVISOR_CACHE_DIR` is set and `HF_HOME` is not, the refresh subprocess stores Hugging Face downloads under339`$ADVISOR_CACHE_DIR/huggingface` so the mounted bucket keeps the embedding model cache across refreshes and restarts.340 341## Test342 343```bash344pytest345```346