5ivatej/qwen2.5-1.5B-india-finetuned

qwen1.5B-india-finetuned

Overview

This repository contains Qwen2.5-1.5B fine-tuned with LoRA on small Indic instruction-following datasets. The LoRA adapters were merged into the base weights, producing a standalone checkpoint that can be used directly with mlx_lm.

License

• —The base model Qwen/Qwen2.5-1.5B is released under the Qwen License.
• —This fine-tuned checkpoint is subject to the same license. Please review the terms before use, especially for commercial scenarios.
• —Marked here as license: other to follow Hugging Face conventions.

Training Configuration

• —Method: LoRA-SFT (attention + MLP)
• —LoRA hyperparams: r=16, alpha=32, dropout=0.05
• —Max sequence length: 1024
• —Steps: 1500
• —Batch size: 1
• —Optimizer: AdamW (default in mlx_lm)
• —Hardware: Apple Silicon (MacBook Pro M4)
• —Framework: `mlx_lm`

The training configuration YAML used can be found at: `configs/qwen2.5-3b_lora.yaml`

Data

• —Subsets from ai4bharat/indic-align were used: Dolly_T and Anudesh.
• —Converted into completion-style prompts (prompt/completion pairs).
• —The focus is on Indic languages (Kannada, Hindi, Tamil, Telugu, Marathi, Gujarati) with some English instructions.
• —Preprocessed into train.jsonl / valid.jsonl (not included here).

Usage

Run with mlx_lm:

python -m mlx_lm generate \
  --model 5ivatej/qwen2.5-1.5B-india-finetuned \
  --max-tokens 200 \
  --prompt "Reply ONLY in Kannada written in English letters. Question: kannada dalli mathadoo?\n\nAnswer:"