Umair1710/Bert-Lora-Finedtuned-Hala_Haram_Detection

metrics:

• —accuracy
• —f1
• —precision
• —recall ---

BERT-LoRA Halal/Haram Ingredient Detection System

This model is a Parameter-Efficient Fine-Tuned (PEFT) implementation based on bert-base-uncased engineered specifically for fine-grained binary classification of food ingredients, composition text, and menu item descriptions into Halal (Class 0) or Haram (Class 1) categories.

🛠️ Project Methodology & Pipeline Architecture

1. Exploratory Data Analysis & Class Balance Chasm

The initial raw master training data presented a substantial class distribution skew, with over 75% of text entries belonging to the majority class (Halal). Unchecked, standard language models converge on a trivial majority-guess heuristic, rendering them entirely blind to rare or complex Haram ingredients.

2. Feature Engineering & Counterfactual Data Augmentation (CDA)

To force the network to construct rigid semantic boundaries around dietary restrictions, we applied a systematic Counterfactual Data Augmentation (CDA) loop across the minority pool. Major Haram ingredient triggers (pork, gelatin, lard, bacon, alcohol, wine, shortening, cochineal) were isolated and dynamically injected with contextual negation prefixes including non-X, X free, zero X, no X, and without X—synthesizing 206,500 unique, hard-negative Halal references.

3. Cryptographic Anti-Leakage Sanitation Guardrail

A strict, automated data-integrity auditing layer was built using cryptographic Python sets. Before model ingestion, this layer systematically cross-referenced and purged cross-contamination points between the generated augmentation pool and the isolated validation/test splits. This guaranteed a 0% data leakage rate, validating that all downstream evaluations reflect real-world generalization.

🚀 Training Dynamics & Overfitting Prevention Verification

Instead of destructive full parameter manipulation, we implemented Low-Rank Adaptation (LoRA) targeting the query and value attention projection modules. By tuning a rank matrix of $r=8$ and a scaling factor of $\alpha=16$, the total active footprint was restricted to just 296,450 parameters (0.27% of the total network) while freezing the underlying 110M parameter foundational architecture.

Generalized Convergence Profile

Overfitting was actively mitigated through heavily optimized structural controls: a robust weight decay configuration ($0.01$), an intense attention/hidden unit dropout factor ($0.2$), and an early stopping patience window targeting Macro F1 gains. The health of the convergence loop is fully demonstrated by the training tracking matrix:

Diagnostic Assessment: The margin between your training loss and validation loss remained exceptionally narrow and strictly parallel all the way to Epoch 5. This provides mathematical proof that the model is acquiring generalized semantic representations rather than memorizing positional keywords.

📊 Cross-Model Performance Evaluation Report

Overall Test Benchmarks

Evaluated across an uncorrupted, isolated test footprint:

Per-Class Detailed Dissection (5,000 Sample Resolution)

1. Un-Tuned Base BERT Model

                 precision    recall  f1-score   support
Halal (Class 0)       0.00      0.00      0.00      3821
Haram (Class 1)       0.24      1.00      0.38      1179

Defect Analysis: The base model completely collapsed due to class distribution bias, taking a uniform path of least resistance by assigning all predictions exclusively to Class 1.

2. Fine-Tuned LoRA Adapter Model

                 precision    recall  f1-score   support
Halal (Class 0)       0.99      0.95      0.97      3821
Haram (Class 1)       0.85      0.98      0.91      1179

Success Metric Highlight: Critical Haram Recall hit 0.98. The model possesses a highly precise ability to extract and flag forbidden targets, securely parsing nested structures (e.g. classifying raw white wine as Haram while tracking white wine vinegar accurately as Halal).

💻 Local Inference Implementation Quickstart

import torch
from transformers import BertTokenizer, BertForSequenceClassification
from peft import PeftModel

base_model_name = 'bert-base-uncased'
peft_model_id = 'Umair1710/Bert-Lora-Finedtuned-Hala_Haram_Detection'

tokenizer = BertTokenizer.from_pretrained(base_model_name)
base_model = BertForSequenceClassification.from_pretrained(base_model_name, num_labels=2)
model = PeftModel.from_pretrained(base_model, peft_model_id)

inputs = tokenizer('gourmet sauce with white wine vinegar', return_tensors='pt')
with torch.no_grad():
    logits = model(**inputs).logits
    pred_class = torch.argmax(logits, dim=-1).item()
print('Prediction:', 'HALAL' if pred_class == 0 else 'HARAM')

✍️ Author & Citation

Main Author: Umair (Umair1710)

If you utilize this model, the underlying data augmentation techniques, or the classification system in your research, applications, or downstream projects, please attribute the work by citing it as follows:

@misc{umair2026berthelalharam,
  author       = {Umair},
  title        = {BERT-LoRA Fine-Tuned Halal/Haram Detection System},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{[https://huggingface.co/Umair1710/Bert-Lora-Finedtuned-Hala_Haram_Detection](https://huggingface.co/Umair1710/Bert-Lora-Finedtuned-Hala_Haram_Detection)}},
  note         = {Final Year Project (FYP) Core Implementation Engine}
}