NextNeural/NINA

NINA — Neural Information & Narrative Abstraction

Upload a video and a prompt; NINA answers using only the video — fusing what is said (Whisper ASR) and what is shown (CLIP), with semantic retrieval, a cross-encoder rerank, confidence-gated precise re-transcription, and a streaming LLM synthesis. See the header of app.py for the full v0.80 pipeline + perf notes.

Running this Space

• —Required secret: set HF_TOKEN in Settings → Variables and secrets — the LLM synthesis step calls the Hugging Face Inference API. Without it, NINA still retrieves regions but falls back to raw excerpts instead of a written summary.
• —Hardware: runs on CPU (auto-falls back from MPS/CUDA). large-v3-turbo is CPU-bound, so precise re-transcription is gated to only fire on low-confidence audio. A larger CPU/GPU tier improves latency on long videos.

API (used by the custom front-end at app.nnlabs.pl/nina)

The Gradio app doubles as the API. From the browser:

import { Client } from "https://cdn.jsdelivr.net/npm/@gradio/client/+esm";
const app = await Client.connect("NextNeural/NINA");
const job = app.submit("/run", { video_path: fileBlob, prompt: "summarize briefly" });
for await (const ev of job) if (ev.type === "data") console.log(ev.data[0]); // streams

Config reference: https://huggingface.co/docs/hub/spaces-config-reference