saliltambe/gemma-4-E2B-it-nepali-lora

Nepali LoRA for Gemma 4 E2B

A LoRA adapter for google/gemma-4-E2B-it that answers in Nepali even when prompted in English, with no system prompt required at inference time.

How it was made

The base model already produces fluent Nepali when given the system prompt "You are a helpful AI assistant that answers in Nepali.", and English without it. This adapter distills that system prompt into the weights, so the behaviour is unconditional. The point is on-device efficiency: a system prompt costs prefill tokens on every request, and training it in once removes that cost permanently.

• —Prompts: OpenAssistant/oasst1, English conversation openers (Apache-2.0, human-written)
• —Targets: generated by google/gemma-4-E2B-it itself with the system prompt above
• —Training pairs: 468 (quality-gated at Devanagari >= 0.7)

Measured result

On 60 held-out English prompts, with no system prompt, mean fraction of Devanagari characters in the response:

Training configuration

Limitations

• —Nepali quality is inherited from the base model; this adapter changes which language is used, not how well it is spoken.
• —Targets are model-generated, so any factual errors the base model makes were trained in.
• —A strong unconditional language bias can compete with an explicit contrary instruction.
• —Trained on single-turn conversation openers; long multi-turn behaviour is untested.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("google/gemma-4-E2B-it", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "saliltambe/gemma-4-E2B-it-nepali-lora")
tok = AutoTokenizer.from_pretrained("google/gemma-4-E2B-it")

For on-device deployment with LiteRT-LM, see the companion conversion notebook.