CoolFace
Modelpublic

lubzo/gemma-2-2b-marathi-lora

sourceHugging Facegemmaupdated 5d agoView on Hugging Face
0likes26downloads
Model Card

gemma-2-2b-marathi-lora

This repository contains LoRA PEFT adapter weights for google/gemma-2-2b fine-tuned on a Cleaned Alpaca Marathi-translated dataset.

This model attempts to reproduce the experimental setup described in: Khade et al., CHiPSAL 2025.

Model description

  • —Base model: google/gemma-2-2b
  • —Tuning Method: LoRA (Low-Rank Adaptation)
  • —Target Modules: q_proj, k_proj, v_proj, o_proj
  • —LoRA Hyperparameters: r = 16, alpha = 32, dropout = 0.05
  • —Language: Marathi (mr)
  • —Training data: Marathi-translated Alpaca instruction dataset (source)

Training Details

  • —Dataset: 51,760 instruction-response pairs translated into Marathi from the Cleaned Stanford Alpaca dataset(unsloth/alpaca-cleaned).
  • —Precision: Pure FP16 / BF16 (unquantized base weights).
  • —Epochs: 3
  • —Optimizer: AdamW

Intended use

Research and educational use, not a production-ready assistant.

Automated Evaluation Results

Evaluated using the AI4Bharat Airavata benchmark harness: | Model | Metric | IndicSentiment | ARC-Easy | ARC-Challenge | IndicCOPA | IndicXNLI | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | | gemma-2-2b (Base) | Accuracy | 0.9180 | 0.8085 | 0.6570 | 0.5698 | 0.3782 | | | Binary F1 (Cls 1) | 0.9093 | N/A | N/A | 0.6013 | N/A | | | Macro F1 | 0.9172 | 0.6463 | 0.5247 | 0.5671 | 0.3075 | | | Paper Reported F1 | 0.9206 | 0.6384 | 0.6463 | 0.6577 | 0.2191 | | gemma-2-2b (Mr) | Accuracy | 0.9520 | 0.7424 | 0.4300 | 0.5158 | 0.4439 | | | Binary F1 (Cls 1) | 0.9502 | N/A | N/A | 0.6687 | N/A | | | Macro F1 | 0.9519 | 0.7924 | 0.3043 | 0.3846 | 0.3551 | | | Paper Reported F1 | | | | | - |

Usage

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "google/gemma-2-2b"
adapter_id = "lubzo/gemma-2-2b-marathi-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)

prompt = "खालील प्रश्नाचे उत्तर द्या:\nभारताची राजधानी कोणती आहे?\nउत्तर:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

## Limitations

- Trained on machine-translated data, not natively-authored Marathi — inherits translation artifacts and biases.
  

## Citation

@inproceedings{khade2025challenges, title={Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning}, author={Khade, Omkar and Jagdale, Shruti and Phaltankar, Abhishek and Takalikar, Gauri and Joshi, Raviraj}, booktitle={Proceedings of the First Workshop on Challenges in Processing South Asian Languages (CHiPSAL 2025)}, year={2025} }