CoolFace
Datasetpublic

jkorstad/anyrag-prototype

AnyModal RAG — validated end-to-end prototype Reference run: job 6ab1136251992417dfccf413 (a10g-small, 4m35s wall-clock, inference only). Stack (all open weights) Unified embedder: Qwen/Qwen3-VL-Embedding-2B — one 2048-d space for text, images, video keyframes, 3D-proxy views Reranker: Qwen/Qwen3-Reranker-0.6B (cross-encoder) Audio: laion/larger_clap_general (Apache-2.0 audio/text space), faster-whisper tiny for STT Generator: Qwen/Qwen3-1.7B (decoder-only… See the full description on the dataset page: https://huggingface.co/datasets/jkorstad/anyrag-prototype.

sourceHugging Faceupdated 1d agoView on Hugging Face
0likes61downloads
Dataset Card

AnyModal RAG — validated end-to-end prototype

Reference run: job 6ab1136251992417dfccf413 (a10g-small, 4m35s wall-clock, inference only).

Stack (all open weights)

  • —Unified embedder: Qwen/Qwen3-VL-Embedding-2B — one 2048-d space for text, images, video keyframes, 3D-proxy views
  • —Reranker: Qwen/Qwen3-Reranker-0.6B (cross-encoder)
  • —Audio: laion/larger_clap_general (Apache-2.0 audio/text space), faster-whisper tiny for STT
  • —Generator: Qwen/Qwen3-1.7B (decoder-only, citations pinned to chunk ids)
  • —Index: sqlite-vec, binary (1-bit/dim) top-200 → float MRL-512 top-20 → cross-encoder top-5

Measured results

MetricValue
Text retrieval: binary ANN top-200 recall@51.0
Float MRL-512 top-20 recall@51.0
Cross-encoder recall@51.0
Text→image hits@54/4
Text→audio hits@14/4
Storage compression vs full float3.88× (68 chunks)
Query pipeline latency (embed→ANN→float→cross-encoder)146 ms total (binary ANN 1.1 ms, float rerank 9.7 ms, cross-encoder 135 ms)
Query embedding6.4 ms
STT (faster-whisper tiny)correct LibriSpeech transcript
RAG answers3/3 correct, grounded, with citations

Files

  • —anyrag_demo.py — full pipeline (ingest → embed → index → retrieve → rerank → generate), all modalities in one script
  • —metrics.json — raw metrics from the passing v6 run

Full design doc: see conversation blueprint artifact (stage × modality matrix, architecture rationale, cost analysis).