CoolFace
Modelpublic

Afeefzeed/Llama-3.2-1B-Singlish-FT

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face
0likes15downloads
Model Card

Uploaded finetuned model

  • —Developed by: Afeefzeed
  • —License: apache-2.0
  • —Finetuned from model : unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
  • —--- language:
  • —en
  • —si tags:
  • —transliteration
  • —singlish
  • —sinhala
  • —llama-3.2
  • —unsloth license: llama3.2 datasets:
  • —indonlp ---

🇱🇰 Llama-3.2-1B-Singlish-FT

This model is a fine-tuned version of Llama-3.2-1B-Instruct, optimized for transliterating Singlish (phonetic English) into Sinhala. It was trained on a large-scale corpus of 500,000 sentence pairs using the Unsloth framework (QLoRA) to handle diverse spelling variations and casual social media text.

📊 Evaluation Results

The model was rigorously evaluated on a held-out test set of 10,003 rows derived from the IndoNLP Sinhala Dataset. This large-scale testing ensures the metrics reflect real-world performance on unseen data.

MetricScoreInterpretation
CER (Character Error Rate)5.46%Excellent. The model predicts 95% of characters correctly, demonstrating mastery of Singlish phonetics.
BERT Score (F1)0.9898Near-Perfect. This confirms that even if the specific words differ slightly, the semantic meaning is 99% identical to the human reference.
BLEU Score27.61Strong. A solid fluency score for a 1B parameter model on a low-resource language task.
WER (Word Error Rate)20.29%Expected. The 20% variation largely comes from valid synonyms (e.g., mata vs maa-ta) rather than incorrect translations, as evidenced by the high BERT score.

🧪 Testing Methodology

  • —Test Dataset: IndoNLP (Sinhala Subset) - Held-out test split.
  • —Sample Size: 10,003 distinct sentences.
  • —Model Configuration: 4-bit Quantization (via Unsloth).

🚀 Usage

You can run this model using the transformers library:

python
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the model
model_name = "afeefzeed/Llama-3.2-1B-Singlish-FT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Input text (Singlish)
input_text = "Transliterate to Sinhala: oya koheda yanne"

# Tokenize & Generate
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# Output: ඔයා කොහෙද යන්නෙ