Leon4gr45/fable5-inference
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Document split KV quant + cache-reuse env vars
Split K/V cache quant (K=q8_0,V=q4_0), 32K ctx, cache-reuse, tool-call smoke test
safety: cap threads at CPU_THREADS_MAX (default 2) to prevent oversubscription if detection over-reports
feat: cgroup-aware vCPU auto-detection (respects container CPU limit; no CPU_THREADS env needed)
fix: startup readiness loop must sleep on HTTP 503 (model loading), not only on connection errors — false 'startup timeout' left instance marked unloaded
harden: normalize legacy KV_CACHE_QUANT values (4bit->q4_0) so stale env vars can't crash llama-server
build: use prebuilt llama.cpp CPU binary (b9895) instead of source compile — fixes mtmd OOM hang, ~20x faster build
build: pull model at container runtime instead of baking into image (faster/reliable build)
v2.0: single-instance CPU proxy, native OpenAI tools+streaming, flash-attn+q4_0 KV, all-cores
Fix llama-server arguments and increase timeouts
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Upload folder using huggingface_hub
Fix: Use hf_hub_download for specific model file from deepreinforce-ai/Ornith-1.0-9B-GGUF
Fix: Skip Python bindings if directory doesn't exist
Fix llama.cpp build with CMake
Fix pip install with --break-system-packages
Update README with Ornith 1.0 details
Update Dockerfile for 2-instance build
Update app.py with 2-instance setup
Upload gemma-inference space
Upload gemma-inference space
Upload gemma-inference space
Upload gemma-inference space
Upload gemma-inference space
initial commit