CoolFace
Modelpublic

mohamedrayyan/dhivehi-news-lora-qwen3-30b

sourceHugging Facemitupdated 9mo agoView on Hugging Face
0likes15downloads
Model Card

Dhivehi News LoRA - Qwen3-30B-A3B

A LoRA fine-tuned adapter that teaches Qwen3-30B-A3B to generate Dhivehi (Maldivian) news articles.

Model Details

PropertyValue
Base ModelQwen/Qwen3-30B-A3B
Training MethodLoRA (Low-Rank Adaptation)
LoRA Rank32
Task TypeCausal Language Modeling

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-30B-A3B",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-30B-A3B")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "mohamedrayyan/dhivehi-news-lora-qwen3-30b")

# Generate
prompt = "Write a Dhivehi news article with the title: މާލޭގައި ބޮޑު އަލިފާނުގެ ހާދިސާއެއް ހިނގައިފި"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

  • —Dataset: alakxender/dhivehi-news-corpus (64.9k news articles)
  • —Training Platform: Tinker API
  • —Epochs: 3
  • —Learning Rate: 2e-4 (with linear decay)
  • —Max Sequence Length: 2048 tokens
  • —Batch Size: 128

Training Format

The model was trained on instruction-following format:

User: Write a Dhivehi news article with the title: {title}
Assistant: {article content}

Results

The fine-tuned model shows significant improvement over the base model for Dhivehi text generation:

Fine-Tuned Model

  • —Generates coherent Dhivehi news article content
  • —Follows the news writing style from the training corpus
  • —Produces grammatically correct Dhivehi text

Base Model (without fine-tuning)

  • —Struggles with the Dhivehi language task
  • —Often produces repetitive or garbled text
  • —Falls back to English "thinking" mode
  • —Misinterprets Dhivehi words

Example

Prompt:

Write a Dhivehi news article with the title: މާލޭގައި ބޮޑު އަލިފާނުގެ ހާދިސާއެއް ހިނގައިފި

Fine-Tuned Output:

މާލޭގައި ބޮޑު އަލިފާނުގެ ހާދިސާއެއް ހިނގައިފި އެވެ.

އިމާމް ނައިބު ރޭ މައުލޫމާތު ދެއްވަމުން މާލެ ކައުންސިލުން ވިދާޅުވީ މި ހާދިސާގައި މީހަކަށް އަނިޔާ ނުވާކަމަށާއި އެ މީހާ ހައްޔަރުކޮށްފައިވާ ކަމަށެވެ...

Requirements

  • —transformers>=4.40.0
  • —peft>=0.10.0
  • —torch>=2.0.0

Citation

If you use this model, please cite:

bibtex
@misc{dhivehi-news-lora-2026,
  author = {mohamedrayyan},
  title = {Dhivehi News LoRA for Qwen3-30B-A3B},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/mohamedrayyan/dhivehi-news-lora-qwen3-30b}}
}

License

This model adapter follows the license of the base model (Qwen3-30B-A3B).

Acknowledgments