CoolFace
Modelpublic

ruv/ruvltra-small

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
2likes128downloads
Model Card

<div align="center">

RuvLTRA Small

![License](https://opensource.org/licenses/Apache-2.0) ![HuggingFace](https://huggingface.co/ruv/ruvltra-small) ![GGUF](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md)

📱 Compact Model Optimized for Edge Devices

Quick Start • Use Cases • Integration

</div>

Overview

RuvLTRA Small is a compact 0.5B parameter model designed for edge deployment. Perfect for mobile apps, IoT devices, and resource-constrained environments.

Model Card

PropertyValue
Parameters0.5 Billion
QuantizationQ4KM
Context4,096 tokens
Size~398 MB
Min RAM1 GB

🚀 Quick Start

bash
# Download
wget https://huggingface.co/ruv/ruvltra-small/resolve/main/ruvltra-0.5b-q4_k_m.gguf

# Run with llama.cpp
./llama-cli -m ruvltra-0.5b-q4_k_m.gguf -p "Hello, I am" -n 64

💡 Use Cases

  • —Mobile Apps: On-device AI assistant
  • —IoT: Smart home device intelligence
  • —Edge Computing: Local inference without cloud
  • —Prototyping: Quick model experimentation

🔧 Integration

Rust (RuvLLM)

rust
use ruvllm::hub::ModelDownloader;

let path = ModelDownloader::new()
    .download("ruv/ruvltra-small", None)
    .await?;

Python

python
from huggingface_hub import hf_hub_download

model = hf_hub_download("ruv/ruvltra-small", "ruvltra-0.5b-q4_k_m.gguf")

Hardware Support

  • —✅ Apple Silicon (M1/M2/M3)
  • —✅ NVIDIA CUDA
  • —✅ CPU (x86/ARM)
  • —✅ Raspberry Pi 4/5

License: Apache 2.0 | GitHub: ruvnet/ruvector

⚡ TurboQuant KV-Cache Compression

RuvLTRA models are fully compatible with TurboQuant — 2-4 bit KV-cache quantization that reduces inference memory by 6-8x with <0.5% quality loss.

QuantizationCompressionQuality LossBest For
3-bit10.7x<1%Recommended — best balance
4-bit8x<0.5%High quality, long context
2-bit32x~2%Edge devices, max savings

Usage with RuvLLM

bash
cargo add ruvllm    # Rust
npm install @ruvector/ruvllm   # Node.js
rust
use ruvllm::quantize::turbo_quant::{TurboQuantCompressor, TurboQuantConfig, TurboQuantBits};

let config = TurboQuantConfig {
    bits: TurboQuantBits::Bit3_5, // 10.7x compression
    use_qjl: true,
    ..Default::default()
};
let compressor = TurboQuantCompressor::new(config)?;
let compressed = compressor.compress_batch(&kv_vectors)?;
let scores = compressor.inner_product_batch_optimized(&query, &compressed)?;

v2.1.0 Ecosystem

  • —Hybrid Search — Sparse + dense vectors with RRF fusion (20-49% better retrieval)
  • —Graph RAG — Knowledge graph + community detection for multi-hop queries
  • —DiskANN — Billion-scale SSD-backed ANN with <10ms latency
  • —FlashAttention-3 — IO-aware tiled attention, O(N) memory
  • —MLA — Multi-Head Latent Attention (~93% KV-cache compression)
  • —Mamba SSM — Linear-time selective state space models
  • —Speculative Decoding — 2-3x generation speedup

RuVector GitHub | ruvllm crate | @ruvector/ruvllm npm

Benchmarks (L4 GPU, 24GB VRAM)

MetricResult
Inference Speed75.4 tok/s
Model Load Time1.44s
Parameters0.5B
TurboQuant KV (3-bit)10.7x compression, <1% PPL loss
TurboQuant KV (4-bit)8x compression, <0.5% PPL loss

Benchmarked on Google Cloud L4 GPU via `ruvltra-calibration` Cloud Run Job (2026-03-28)