Blaqadonis/Qwen2.5-0.5B-Nigerian-News-Headlines

Qwen2.5-0.5B-Nigerian-News-Headlines

Fine-tuned Qwen 2.5 0.5B Instruct model for generating compelling headlines from Nigerian news articles using QLoRA (Quantized Low-Rank Adaptation).

Model Description

This model adapts the lightweight Qwen 2.5 0.5B Instruct base model to generate concise, informative headlines specifically for Nigerian news content. It was fine-tuned using parameter-efficient QLoRA on over 4,000 Nigerian news articles from AriseTv.

• —Base Model: Qwen/Qwen2.5-0.5B-Instruct
• —Model Type: Causal Language Model (Headline Generation)
• —Fine-tuning Method: QLoRA (4-bit quantization + LoRA adapters)
• —Language: English (Nigerian context)
• —License: Same as base model

Intended Use

Primary Use Cases

• —News Headline Generation: Generate engaging headlines from Nigerian news excerpts
• —Content Summarization: Create concise summaries of news articles
• —Media Applications: Assist journalists and content creators in headline writing

Out-of-Scope Use

• —General-purpose text generation outside Nigerian news context
• —Non-English language headline generation
• —Real-time news processing without human review

Training Data

Dataset

• —Source: okite97/news-data
• —Description: Collection of Nigerian news articles from AriseTv
• —Size: 4,686 articles
• —Split:
• —Training: 4,286 samples
• —Validation: 200 samples
• —Test: 200 samples

Data Format

Each sample consists of:

• —Excerpt: News article excerpt (context)
• —Title: Target headline (gold standard)

Training Procedure

Fine-tuning Configuration

Hardware:

• —GPU: 1x NVIDIA T4 (16GB VRAM)
• —Platform: Google Colab

Hyperparameters:

# Model Configuration
base_model: Qwen/Qwen2.5-0.5B-Instruct
sequence_length: 512

# QLoRA Configuration
quantization: 4-bit NF4
lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
target_modules: [q_proj, v_proj]

# Training Parameters
num_epochs: 2
max_steps: 300
batch_size: 16
gradient_accumulation_steps: 2
learning_rate: 2e-4
lr_scheduler: cosine
warmup_steps: 50
optimizer: paged_adamw_8bit
bf16: true

Trainable Parameters:

• —Total parameters: 495.1M
• —Trainable parameters: 1.08M (0.22%)
• —Training method: QLoRA (4-bit + LoRA adapters)

Training Metrics

Training Curve:

• —Loss decreased steadily from 2.92 (step 25) to 2.42 (step 300)
• —Validation loss improved from 2.87 to 2.55
• —No signs of overfitting observed

Evaluation Results

ROUGE Scores

Evaluated on 200 held-out Nigerian news samples:

Key Findings

✅ Significant improvements across all metrics

• —ROUGE-1 improved by 17%, indicating better content overlap
• —ROUGE-2 improved by 41%, showing improved bigram matching
• —ROUGE-L improved by 28%, demonstrating better sequence structure

✅ Better headline quality

• —More concise and focused headlines
• —Better keyword selection
• —Improved grammatical structure

Example Predictions

Example 1: Sports News

Excerpt:

Lewis Hamilton was gracious in defeat after Red Bull rival Max Verstappen ended 
the Briton's quest for an unprecedented eighth...

Reference: F1: Hamilton Gracious in Title Defeat as Mercedes Lodge Protests

Baseline: Lewis Hamilton's Gracious Victory After Red Bull's Max Verstappen Seeks Record-Setting Eighth Win

Fine-tuned: Hamilton Gracious After Red Bull Victory ✅

Example 2: Business News

Excerpt:

Following improved corporate earnings by companies, low yield in fixed income 
market, among other factors, the stock market segment of...

Reference: Nigeria's Stock Market Sustains Bullish Trend, Gains N5.64trn in First Half 2022

Baseline: "Boosting Corporate Profits: The Impact on Stock Market Performance Amidst Yield Challenges"

Fine-tuned: Nigeria's Stock Market Suffers as Corporate Earnings Slow ✅

Example 3: Politics

Excerpt:

Amidst the worsening insecurity in the country, governors elected on the platform 
of the Peoples Democratic Party (PDP) on Wednesday...

Reference: Nigeria: PDP Governors Restate Case for Decentralised Police

Baseline: "Governors Rally to Defend Statehood Amidst Growing Security Concerns"

Fine-tuned: Nigeria: PDP Governors Elected Amidst Worsening Security Crisis ✅

Usage

Installation

pip install transformers peft torch bitsandbytes accelerate

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

# Configure 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# Load LoRA adapters
model = PeftModel.from_pretrained(
    base_model,
    "Blaqadonis/Qwen2.5-0.5B-Nigerian-News-Headlines"
)
model.eval()

Generating Headlines

def generate_headline(news_excerpt):
    """Generate headline from news excerpt."""
    prompt = f"""Generate a concise and engaging headline for the following Nigerian news excerpt.

## News Excerpt:
{news_excerpt}
## Headline:"""
    
    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    
    outputs = model.generate(
        **inputs,
        max_new_tokens=50,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )
    
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    # Extract only the generated headline
    headline = response.split("## Headline:")[-1].strip()
    return headline

# Example usage
excerpt = """
Nigeria's inflation rate has risen to 33.40% in July 2024, 
according to the National Bureau of Statistics...
"""

headline = generate_headline(excerpt)
print(headline)
# Output: "Nigeria's Inflation Rate Rises to 33.40% in July 2024"

Using with Pipeline

from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Generate headline
prompt = """Generate a concise and engaging headline for the following Nigerian news excerpt.

## News Excerpt:
The Central Bank of Nigeria has announced new forex policies...
## Headline:"""

output = pipe(prompt, max_new_tokens=50, do_sample=False)
print(output[0]['generated_text'])

Limitations

Known Issues

1. 1.Domain Specificity: Optimized for Nigerian news; may not perform well on other news sources
2. 2.Context Length: Limited to 512 tokens; very long articles may need truncation
3. 3.Factual Accuracy: May generate plausible but inaccurate headlines; always verify content
4. 4.Quantization Effects: 4-bit quantization may introduce minor variations in output
5. 5.Language: English only; no support for Nigerian languages (Yoruba, Igbo, Hausa)

Bias Considerations

• —Trained on Nigerian news corpus, may reflect biases present in the training data
• —May favor certain news topics or political perspectives present in AriseTv content
• —Should not be used as sole source for headline generation without human review

Ethical Considerations

⚠️ Important Notes:

• —This model should not replace human journalists in editorial decisions
• —Generated headlines should be reviewed for accuracy and appropriateness
• —Be mindful of potential amplification of biases present in training data
• —Ensure compliance with journalistic ethics and standards

Citation

If you use this model in your work, please cite:

@misc{qwen25-nigerian-headlines,
  author = {Blaqadonis},
  title = {Qwen2.5-0.5B-Nigerian-News-Headlines: Fine-tuned Model for Nigerian News Headline Generation},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/Blaqadonis/Qwen2.5-0.5B-Nigerian-News-Headlines}}
}

Model Card Contact

• —Developer: Blaqadonis
• —HuggingFace: @Blaqadonis
• —Repository: GitHub

Acknowledgments

• —Base model: Qwen Team
• —Dataset: okite97
• —Training framework: Hugging Face Transformers, PEFT
• —Training support: LLMED Program by Ready Tensor

Model Version: 1.0 Last Updated: December 2025 Model Card Version: 1.0