buddhist-nlp/gemma-2-mitra-chat

gemma-2-mitra-chat

A multi-turn chat model for Buddhist studies from the Dharmamitra project: closed-book Buddhism Q&A plus a translation/refinement assistant for classical languages (Sanskrit, Tibetan, Buddhist Chinese, Pāli), speaking the standard gemma-2 chat protocol — it works out of the box with Ollama / llama.cpp multi-turn templates.

Built by full-parameter SFT of `buddhist-nlp/gemma2-mitra-base` (9.5B, Buddhist-domain continued pretraining) on ~10k examples: ~5k closed-book Buddhism Q&A (mined open-book, references removed so the knowledge is distilled into the weights) and ~5k translation/refine tasks (zh/sa/bo/pi, half with retrieved reference passages). Completion-only loss over full chat histories; EOS is <end_of_turn>, so turns close correctly in stock chat runtimes.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("buddhist-nlp/gemma-2-mitra-chat")
model = AutoModelForCausalLM.from_pretrained(
    "buddhist-nlp/gemma-2-mitra-chat", dtype=torch.bfloat16, device_map="cuda"
)

messages = [{"role": "user", "content":
    "Translate into English: 'di skad bdag gis thos pa dus gcig na"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                 return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Tibetan may be given in Wylie transliteration (as in the mitra convention); Sanskrit and Pāli in IAST.

Training details

• —Base: buddhist-nlp/gemma2-mitra-base
• —Full-parameter SFT (TRL, completion-only loss, gemma-2 chat template), LR 1e-5, effective batch 64, cosine schedule, bf16; best checkpoint at step 220
• —Data: combined closed-book Q&A + translation/refinement corpus (~10k multi-turn examples)

Related models

Part of the buddhist-nlp gemma-2 Mitra family; see also the newer Qwen3.5-based generation: `buddhist-nlp/mitra-qwen35-base`, `buddhist-nlp/mitra-qwen35-embedder`.

Citation

If you use this model, please cite the Dharmamitra project (https://dharmamitra.org).