dawood/spotsound-temporal-grounding

SpotSound — fine-grained audio temporal grounding

Demo of [SpotSound](https://huggingface.co/Loie/SpotSound) (Sun et al., 2026), a LoRA on top of NVIDIA Audio Flamingo 3 that gives large audio–language models fine-grained temporal grounding: given a long, untrimmed recording and a natural-language query, it answers with the start/end timestamps of the described sound event.

• —Paper: <https://huggingface.co/papers/2604.13023>
• —Project page: <https://loiesun.github.io/spotsound/>
• —Code: <https://github.com/LoieSun/SpotSound>
• —Benchmark: <https://huggingface.co/datasets/Loie/SpotSound-Bench>

How it works

spotsound.py ports the two pieces of the official implementation onto current transformers:

1. 1.Timestamp-interleaved audio tokens — the <sound> placeholder is expanded to "timestamp: t seconds; feature: " + <sound> × 25 for every second of audio (AF3 produces exactly 25 post-pool audio tokens per second), so the language model can read absolute time off the audio stream.
2. 2.Tolerant audio scatter — the audio features are scattered into the audio-token slots without the base class's strict #tokens == #features check, since the interleaving keeps floor(N / 25) × 25 of the N features.

The published LoRA adapter is merged into the base model at startup and served on ZeroGPU.

Examples

The four example clips come from Loie/SpotSound-Bench (MIT licensed), the paper's own "needle-in-a-haystack" benchmark.

Licensing

The SpotSound adapter and benchmark are MIT licensed. The base model, nvidia/audio-flamingo-3-hf, is released under the NVIDIA OneWay Noncommercial License — this demo is for research/non-commercial use.