81melody/algerianDeBERTa-realestate-ner
algerianDeBERTa-realestate-ner
A Name Entity Recognition model based on AlgerianDeBERTa, finetuned to extract 13 real-estate entities from Algerian Facebook posts
Handles the exact language mix found on Facebook Marketplace and Algerian classified groups: Darja (Algerian dialect), Arabizi (Arabic written in Latin script), French, and heavy code-switching, finetuned on +7k examples of pure algerian real-estate posts from Facebook
Model Highlights
Quick Start
Option 1 : pipeline (standard, recommended for short texts)
Uses aggregation_strategy="max": for each surface word the subword token with the highest entity-class score wins, then consecutive spans that have the same type are merged automatically
from transformers import pipeline
ner = pipeline(
"token-classification",
model="81melody/algerianDeBERTa-realestate-ner",
aggregation_strategy="max",
)
print(ner("سلام ، خصني اف2 فالعاصمة في ميسوني ولا اودان ولا ديدوش ، في هاد الجويه لي عندو يتوصل معيا في الخاص"))
print(ner("Appartement F4 à vendre Oran centre 120m² 4ème étage acte notarié"))
For texts that may exceed 192 tokens, pass sliding-window arguments directly to the pipeline call:
result = ner(
long_text,
truncation=True,
max_length=192,
stride=64,
)Each result dict contains entity_group, word, score, start, end.
Option 2 : Manual sliding window inference (production / long texts)
For real estate posts that frequently exceed one chunk + to adapt with The small vocab size of the first version of the base model (30k), the approach below is more robust for production use: it averages the probability vectors of overlapping tokens across all chunks, then merges subword pieces that form the same surface word before BIO decoding
This fixes a common artefact where words like "cherche" (tokenised as ["cher", "che"]) or prices like "1.700" (tokenised as ["1", ".", "700"]) get truncated mid-word if a trailing subword happens to predict O
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
import numpy as np
from typing import List
MODEL_NAME = "81melody/algerianDeBERTa-realestate-ner"
MAX_SEQ_LEN = 192
STRIDE = 64
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForTokenClassification.from_pretrained(MODEL_NAME)
model.eval()
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
def extract_entities(text: str) -> List[dict]:
enc = tokenizer(
text,
return_tensors="pt",
max_length=MAX_SEQ_LEN,
stride=STRIDE,
truncation=True,
return_overflowing_tokens=True,
return_offsets_mapping=True,
padding="max_length",
)
enc.pop("overflow_to_sample_mapping", None)
offsets = enc.pop("offset_mapping")
with torch.no_grad():
logits = model(**{k: v.to(device) for k, v in enc.items()}).logits
probs = torch.softmax(logits, dim=-1).cpu().numpy()
attention = enc["attention_mask"].numpy()
off_np = offsets.numpy()
char_probs = {}
for c in range(probs.shape[0]):
for t in range(off_np.shape[1]):
if attention[c, t] == 0:
continue
cs, ce = int(off_np[c, t, 0]), int(off_np[c, t, 1])
if cs == 0 and ce == 0:
continue
if cs not in char_probs:
char_probs[cs] = {"end": ce, "vecs": [probs[c, t]]}
else:
char_probs[cs]["vecs"].append(probs[c, t])
if not char_probs:
return []
items = sorted(char_probs.items())
word_tokens = []
w_start, w_info = items[0]
w_end = w_info["end"]
w_avg_p = np.mean(w_info["vecs"], axis=0)
for cs, info in items[1:]:
if cs == w_end:
w_end = info["end"]
else:
word_tokens.append({"start": w_start, "end": w_end, "avg_p": w_avg_p})
w_start = cs
w_end = info["end"]
w_avg_p = np.mean(info["vecs"], axis=0)
word_tokens.append({"start": w_start, "end": w_end, "avg_p": w_avg_p})
label_map = model.config.id2label
entities, current = [], None
for w in word_tokens:
idx = int(np.argmax(w["avg_p"]))
label = label_map[idx]
score = float(w["avg_p"][idx])
if label == "O":
if current:
entities.append(current)
current = None
elif label.startswith("B-"):
if current:
entities.append(current)
current = {
"entity": label[2:],
"word": text[w["start"]:w["end"]],
"score": score,
"_sc": [score], "_s": w["start"], "_e": w["end"],
}
elif label.startswith("I-"):
etype = label[2:]
if current and current["entity"] == etype:
current["word"] = text[current["_s"]:w["end"]]
current["_e"] = w["end"]
current["_sc"].append(score)
current["score"] = float(np.mean(current["_sc"]))
else:
if current:
entities.append(current)
current = {
"entity": etype,
"word": text[w["start"]:w["end"]],
"score": score,
"_sc": [score], "_s": w["start"], "_e": w["end"],
}
if current:
entities.append(current)
return [
{"entity": e["entity"], "word": e["word"], "score": round(e["score"], 6)}
for e in entities
]
Entity Schema
The model uses a 27-label BIO scheme covering 13 entity types drawn directly from Algerian real estate Facebook market
Performance
Evaluated on a held-out test set of 95 posts / 596 tokenized chunks — never seen during training or validation.
precision recall f1-score support
AMENITY 0.9735 0.9971 0.9852 700
APT_CLASS 0.9728 1.0000 0.9862 357
CONDITION 0.9793 0.9861 0.9827 144
DOCUMENT 0.9755 0.9848 0.9801 526
FLOOR 0.9416 0.9928 0.9665 276
NEIGHBORHOOD 0.8434 0.8642 0.8537 81
PHONE 0.8101 0.9846 0.8889 65
PRICE 0.8739 0.9336 0.9028 557
PROPERTY_TYPE 0.9869 0.9912 0.9890 682
SURFACE 0.9259 0.9420 0.9338 517
TRANSACTION 0.9950 0.9934 0.9942 603
WILAYA 0.9810 0.9841 0.9825 629
micro avg 0.9566 0.9780 0.9672 5137
macro avg 0.9382 0.9712 0.9538 5137
weighted avg 0.9576 0.9780 0.9675 5137Metric: seqeval entity-level strict match (not token-level an entity prediction counts as correct only if both the span and the label match the annotation exactly)
Training Data
Sourced from the companion dataset `81melody/algerian-realestate-ner-dataset`.
Language distribution: Arabic (Darja) 53% · French 34% · Mixed / Arabizi 13% Listing intent: Seller 95.4% · Buyer 4.6%
All posts were collected from public Algerian Facebook real estate groups(using Facebook API in the Apify platforl) Phone numbers are replaced with [PHONE] before publishing (BIO tags preserved so the model learns positional context without memorizing digits)
Training Details
base_model: algerianDeBERTa (DeBERTa-v2)
architecture: DebertaV2ForTokenClassification
num_labels: 27 (BIO, 13 entity types)
max_seq_len: 192
stride: 64
optimizer: AdamW
peak_lr: 2e-5
llrd_factor: 0.9
weight_decay: 0.01
adam_eps: 1e-6
adam_beta1: 0.9
adam_beta2: 0.999
max_grad_norm: 1.0
grad_accum_steps: 2
epochs: 20 (early stop at epoch 14, patience=5)
warmup_ratio: 0.1
schedule: cosine with warmup
label_smoothing: 0.05
class_weighting: inverse-frequency, capped at 10× (rare tags: CITY, WILAYA, CONDITION)
dropout: 0.1 (attention + hidden)
best_val_f1: 0.9858
test_f1: 0.9672
test_precision: 0.9566
test_recall: 0.9780Training highlights
Layerwise Learning Rate Decay (LLRD): The classifier head trains at peak_lr=2e-5, each successive DeBERTa layer is scaled by 0.9×, reaching ≈4.3e-6 at the embedding layer, This preserves general language representations while aggressively adapting the top layers to the NER task
Weighted label-smoothed cross-entropy: Rare entity tags (CITY, WILAYA, CONDITION) carry up to 10× the loss weight of frequent tags , Label smoothing (ε=0.05) prevents the model from becoming over-confident on the abundant O tag
Sliding-window tokenization: Posts exceeding 192 tokens are split into overlapping chunks (stride=64). Predictions from overlapping windows are reconciled at entity boundaries, ensuring long posts are fully covered without truncation
Limitations
- NEIGHBORHOOD F1=0.85: Neighbourhood names in Algeria are highly variable in spelling across Arabic, French, and Arabizi. This entity is underrepresented in the training data (188 annotations). Performance will improve with more annotated data from underrepresented neighbourhoods
- PRICE edge cases: Non-standard price expressions that rely heavily on slang are occasionally missed, The model handles the most common formats reliably
- Platform distribution: Trained on Facebook posts — casual, informal register. May underperform on formal Arabic (MSA) or structured portal listings
- Purely extractive: This is a span classifier, not a generative model. It labels tokens; it does not summarise or rewrite listings
Intended Use
Citation
If you use this model or the dataset in your research, please cite:
@misc{himeur2026algeriandeberta_ner,
title = {algerianDeBERTa-realestate-ner: Named Entity Recognition
for Algerian Real Estate Text in Darja, Arabizi, and French},
author = {Himeur, Ayoub},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/81melody/algerianDeBERTa-realestate-ner},
note = {Fine-tuned DeBERTa-v2 on annotated
Algerian Facebook real estate posts, 13 entity types}
}License
Contact
mohamed.himeur@student.unamur.be