barakplasma/dictalm-3.0-1.7b-thinking-android

DictaLM-3.0-1.7B — Android / Google AI Edge Bundles

On-device Hebrew LLM for Android using Google AI Edge. Converts DictaLM-3.0-1.7B-Thinking and DictaLM-3.0-1.7B-Instruct (Hebrew LLMs fine-tuned from Qwen3-1.7B by Dicta) into .litertlm format that runs locally on Android — no internet or cloud API required.

Files

Which to use?

• —Best quality + visible reasoning: thinking-float16-thinking — model shows its thought process before answering
• —Best quality, no reasoning overhead: thinking-float16 or instruct-float16
• —Balanced (recommended): thinking-dynamic_int8 — good Hebrew quality, fits comfortably in RAM
• —Fastest / smallest: thinking-int4 or instruct-int4

Quick Start — Google AI Edge Gallery (Android)

1. 1.Download a .litertlm file from this repo
2. 2.Install Google AI Edge Gallery on your Android device
3. 3.Open the app → Add Model → select your .litertlm file
4. 4.Start chatting in Hebrew

Device Requirements

Tested on Pixel 10 (12 GB RAM).

Thinking Mode

The *-thinking-float16-thinking.litertlm bundle embeds a Jinja chat template that starts generation with <|im_start|>assistant\n<think>\n, which prompts the model to reason before answering. The <think>...</think> block will appear as raw text in the app output.

The other Thinking-variant bundles embed <think>\n\n</think> in the generation prompt to suppress thinking for faster responses.

The Instruct variant has no thinking capability.

Hebrew Language

All bundles embed a Hebrew system prompt:

You are a helpful Hebrew AI assistant named DictaLM. Respond in Hebrew unless explicitly asked otherwise.
אתה עוזר AI שימושי בעברית בשם DictaLM. הגב בעברית אלא אם ביקשו ממך אחרת.

How These Were Built

Architecture

DictaLM-3.0-1.7B is a fine-tune of Qwen3-1.7B:

litert-torch 0.8.0 already includes qwen3.build_1_7b_model() which exactly matches this config — no custom builder was needed.

Conversion Pipeline

HuggingFace safetensors
        ↓
litert-torch (Strategy 1: native Qwen3 builder)
        ↓
TFLite with prefill/decode KV-cache signatures
        ↓
bundle_litertlm.py (LlmMetadata proto + HF tokenizer + Jinja template)
        ↓
.litertlm (Google AI Edge runtime format)

Key detail: lm_head.weight is absent from the checkpoint (tie_word_embeddings=True) and is copied from model.embed_tokens.weight in a custom weight loader before conversion.

Reproduce a Build

Requirements: ~32 GB RAM, Python 3.12, litert-torch==0.8.0, LiteRT-LM builder

# Install dependencies
pip install litert-torch==0.8.0 mediapipe transformers safetensors

# Clone LiteRT-LM (needed by bundle_litertlm.py)
git clone --depth=1 https://github.com/google-ai-edge/LiteRT-LM /tmp/litert-lm
# Build flatbuffer + proto bindings — see LiteRT-LM docs

# Download model
huggingface-cli download dicta-il/DictaLM-3.0-1.7B-Thinking \
  --local-dir ./dictalm-3.0-1.7b-thinking

# Convert to TFLite (Strategy 1 — KV-cache prefill/decode)
python scripts/convert_dictalm_android.py \
  --model-dir ./dictalm-3.0-1.7b-thinking \
  --tflite-dir ./tflite_output/thinking_fp16 \
  --quantize float16 \
  --prefill-seq-len 1024 --kv-cache-max-len 1024 \
  --skip-task-bundle

# Bundle as .litertlm with thinking enabled
python scripts/bundle_litertlm.py \
  --tflite ./tflite_output/thinking_fp16/*.tflite \
  --tokenizer ./dictalm-3.0-1.7b-thinking/tokenizer.json \
  --tokenizer-type hf \
  --model-type qwen3 \
  --thinking \
  --output ./dictalm-3.0-1.7b-thinking-float16-thinking.litertlm \
  --quant float16

# Or without thinking (no-think mode, faster):
python scripts/bundle_litertlm.py \
  --tflite ./tflite_output/thinking_fp16/*.tflite \
  --tokenizer ./dictalm-3.0-1.7b-thinking/tokenizer.json \
  --tokenizer-type hf \
  --model-type qwen3 \
  --output ./dictalm-3.0-1.7b-thinking-float16.litertlm \
  --quant float16

Quantization Options

Scripts

See scripts/ folder:

License

Model weights: Qwen License / DictaLM terms Conversion scripts: Apache 2.0