CoolFace
Modelpublic

RunSLM-AI/SmolLM2-1.7B-Instruct-GGUF

sourceHugging Faceapache-2.0updated 15h agoView on Hugging Face
0likes97downloads
Model Card

SmolLM2-1.7B-Instruct (GGUF for RunSLM Mobile Workstation)

Official, production-calibrated Q4_K_M GGUF quantization of `HuggingFaceTB/SmolLM2-1.7B-Instruct`, purpose-built for physical, zero-cloud execution in the RunSLM AI mobile workstation on Apple Silicon (Metal UMA) and Android (ARM64-v8.2a+ / KleidiAI SIMD).

⚡ Hardware Footprint & Operational Profile

MetricTarget Specification
Model ArchitectureLLaMA-based (SmolLM2)
Parameter Count1.71 Billion
Quantization MethodQ4_K_M (4-bit Medium K-Quants)
Binary ArtifactSmolLM2-1.7B-Instruct-Q4_K_M.gguf
File Size1.06 GB (1,142,482,944 bytes)
Runtime KV Cache Footprint~144 MB (Q8_0 Key / Q4_0 Value at 4,096 tokens)
Recommended Device RAM4 GB+ (iOS & Android)
Observed Throughput40–55+ t/s (Apple Silicon Metal) · 18–26+ t/s (ARM KleidiAI SIMD)
Time-To-First-Token (TTFT)< 350 ms on modern mobile SoCs

🛠️ Direct Model Download

You can fetch the weights directly using curl or Hugging Face CLI:

bash
# Direct HTTP download
curl -L -o SmolLM2-1.7B-Instruct-Q4_K_M.gguf \
  https://huggingface.co/RunSLM-AI/SmolLM2-1.7B-Instruct-GGUF/resolve/main/SmolLM2-1.7B-Instruct-Q4_K_M.gguf

USB Sideloading (RunSLM iOS / Android)

  • —iPhone / iPad: Connect via USB to Windows/Mac → Open Apple Devices / iTunes → File Sharing → RunSLM → Drag and drop SmolLM2-1.7B-Instruct-Q4_K_M.gguf.
  • —Android / Tablets: Connect via USB → Transfer directly into Android/data/com.tking0000.runslm/files/models/.

💬 Prompt Template: ChatML

SmolLM2-1.7B-Instruct natively parses the standard ChatML format. Ensure your tokenizer or inference wrapper uses the following structure:

text
<|im_start|>system
You are RunSLM, a concise, sovereign on-device assistant.<|im_end|>
<|im_start|>user
What is the difference between CPU cache and main memory?<|im_end|>
<|im_start|>assistant

Stop Sequences

Configure your engine to break generation on:

  • —<|im_end|>
  • —<|endoftext|>

🎙️ Intended Mobile Workloads

  1. 1.Air-Gapped Sovereign Chat: Real-time general reasoning, code generation, and instruction following with zero server communication.
  2. 2.Dual-Persona AI Podcast Studio: Low-latency turn generation between dialectical Host and Expert personas with zero acoustic overlap.
  3. 3.Local Workspace Synthesis: Drafting calendar events, parsing offline documents, and generating Markdown notes directly into mobile storage.

📜 Licensing & Legal Attribution

  • —Base Model: `SmolLM2-1.7B-Instruct` released by Hugging Face, Inc. under the Apache 2.0 License.
  • —Quantization & Packaging: Released by RunSLM AI under the Apache 2.0 License.
  • —Commercial Use: Permitted under Apache 2.0 terms. Please preserve original copyright and license notices when redistributing.