abdulhafis/dagbani-english-translator

dagbani-english-translator

This repository contains a fine-tuned Large Language Model (LLM) for Dagbani-English translation, developed as part of a Google Colab project. The model is based on unsloth/Meta-Llama-3.1-8B-Instruct and has been fine-tuned using a parallel dataset of Dagbani and English sentences.

Model Description

This model is a specialized translation assistant designed to translate between English and Dagbani, aiming for natural and culturally relevant outputs. It is built upon the powerful Llama 3.1 8B Instruct model, enhanced with LoRA adapters via Unsloth for efficient fine-tuning.

Key Features:

• —Translation: Bidirectional translation between English and Dagbani.
• —Cultural Context: Aims to provide natural and respectful translations, considering cultural nuances where possible.
• —Instruction-tuned: Designed to follow instructions for translation tasks.

Dataset

The model was fine-tuned on the dagbani_english_parallel_dataset.tsv dataset, consisting of {insertnumberof_examples} parallel English and Dagbani examples. The dataset was preprocessed to handle potential None values and formatted into an instruction-based prompt structure for optimal performance with the Llama 3.1 Instruct model.

Training Details

• —Base Model: unsloth/Meta-Llama-3.1-8B-Instruct
• —Framework: Unsloth for efficient 4-bit LoRA fine-tuning.
• —LoRA Configuration:
• —r: 16
• —target_modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
• —lora_alpha: 16
• —lora_dropout: 0
• —bias: "none"
• —Training Arguments:
• —per_device_train_batch_size: 2
• —gradient_accumulation_steps: 4
• —warmup_steps: 10
• —max_steps: {insertmaxstepsfromnotebook} (e.g., 400 or more if extended)
• —learning_rate: 2e-4
• —optim: "adamw_8bit"
• —lr_scheduler_type: "linear"

How to Use

You can use this model for inference using the transformers library, similar to how it was tested in the Colab notebook.

First, install the necessary libraries:

pip install transformers torch accelerate peft unsloth

Then, load the model and tokenizer and perform inference:

import torch
from unsloth import FastLanguageModel

# Load the fine-tuned model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "abdulhafis/dagbani-english-translator", # Your Hugging Face repository
    max_seq_length = 2048,
    dtype = None, # Auto detects based on your GPU
    load_in_4bit = True,
)

# Ensure model is in inference mode
FastLanguageModel.for_inference(model)

def translate_to_dagbani(text):
    prompt = f"""You are a helpful assistant that translates between English and Dagbani naturally.\n\nEnglish: {text}\n\nDagbani:"""

    inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    
    # Extract only the Dagbani part
    dagbani_part = response.split("Dagbani:")[-1].strip()
    return dagbani_part

# Test the translator
english_sentence = "What is the price of yam today?"
dagbani_translation = translate_to_dagbani(english_sentence)
print(f"English: {english_sentence}")
print(f"Dagbani: {dagbani_translation}")

Limitations

• —The quality of translation depends heavily on the size and diversity of the fine-tuning dataset.
• —May struggle with highly idiomatic expressions or very niche terminology not present in the training data.
• —Generated translations should always be reviewed by a native speaker for accuracy and appropriateness.

Acknowledgements

• —Unsloth: For providing efficient fine-tuning tools.
• —Hugging Face: For hosting models and datasets.
• —Llama 3.1: The powerful base model.

Note: Replace `{insert_number_of_examples}` with the actual number of examples from your dataset and `{insert_max_steps_from_notebook}` with the `max_steps` value used in your training configuration if you want to be more specific. These values can be found in the notebook cell that performs training.