DuoNeural/SmolLM2-135M-Instruct-LiteRT

language:

• —en tags:
• —duoneural
• —litert
• —edge
• —gguf
• —on-device
• —smollm
• —smol
• —tiny
• —litert
• —edge
• —instruct basemodel: HuggingFaceTB/SmolLM2-135M-Instruct pipelinetag: text-generation license: apache-2.0 ---

# SmolLM2-135M-Instruct-LiteRT

SmolLM2 135M Instruct — ultra-tiny on-device assistant (~90MB) — converted for mobile and edge deployment by DuoNeural.

• —Source model: HuggingFaceTB/SmolLM2-135M-Instruct
• —Format: GGUF Q4KM (llama.cpp-compatible)
• —File size: 105 MB
• —Quantization: 4-bit K-mean (Q4KM) — excellent accuracy/size trade-off for edge devices
• —Target platforms: Android, iOS, desktop edge inference
• —Converted: 2026-05-06 06:08:19 by Archon / DuoNeural

## Usage

### llama.cpp (CLI)

    ./llama-cli -m SmolLM2-135M-Instruct-LiteRT_Q4_K_M.gguf -n 512 --temp 0.7

### Google AI Edge / MediaPipe (Android/iOS) This GGUF is compatible with MLC-LLM and llama.cpp Android bindings for on-device inference. For use with Google Edge Gallery, convert to .task bundle using MediaPipe LLM conversion tools.

### Python via llama-cpp-python

    from llama_cpp import Llama

    llm = Llama(
        model_path="SmolLM2-135M-Instruct-LiteRT_Q4_K_M.gguf",
        n_ctx=2048,
        n_threads=4,
        verbose=False,
    )

    response = llm.create_chat_completion(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Hello! How can you help me today?"},
        ]
    )
    print(response["choices"][0]["message"]["content"])

### Ollama

    ollama run hf.co/DuoNeural/SmolLM2-135M-Instruct-LiteRT

## About the Conversion

Converted using llama.cpp GGUF pipeline with CUDA acceleration. Source weights downloaded from HuggingFace, converted to F16 GGUF, then quantized to Q4KM.

## DuoNeural

DuoNeural is an open AI research lab — human + AI in collaboration.

### DuoNeural Research Publications

Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.

### Research Team

• —Jesse — Vision, hardware, direction
• —Archon — Lab Director, post-training, abliteration, experiments
• —Aura — Research AI, literature synthesis, novel proposals

Subscribe to the lab newsletter at [duoneural.beehiiv.com](https://duoneural.beehiiv.com) for model drops before they go anywhere else.