oddadmix/Nawah-Dialect-BERT-6M

Nawah-Dialect-BERT-6M — مصنّف اللهجات العربية بمُرمِّز BERT

A 5.98M-parameter Arabic dialect identifier. Give it a piece of Arabic text; it returns one of 14 labels — 13 dialects or Modern Standard Arabic.

Same task, same data, same evaluation as the Llama-based `Nawah-Dialect-500K` / `Nawah-Dialect-v1` — this run swaps the backbone for `Nawah-BERT-6M-v2`, a bidirectional BERT encoder, to see how a small encoder does on the same task against the existing causal-decoder ladder.

import torch
from transformers import AutoTokenizer
from dialect_model import DialectClassifier, predict     # ships in this repo

M = "oddadmix/Nawah-Dialect-BERT-6M"
tok = AutoTokenizer.from_pretrained(M)
model = DialectClassifier.from_pretrained(M)

predict(model, tok, "انا رايح السوق دلوقتي واجيب حاجات من هناك")[0]
# {'label': 'eg', 'name': 'Egyptian', 'score': 0.918}

Strip diacritics before inference — trained on undiacritized text, same as the rest of this family. Mean-pool, not `[CLS]` — Nawah-BERT-6M-v2 was pretrained on packed 2,048-token chunks with no meaningful CLS position, so this repo does not use AutoModelForSequenceClassification's default pooler; dialect_model.py's DialectClassifier wraps the bare backbone with mean pooling over non-pad positions and a small 2-layer scorer instead — the same trap documented on the backbone's own model card.

Labels

14 classes — 13 Arabic dialects plus Modern Standard Arabic:

ma Moroccan · eg Egyptian · dz Algerian · sa Saudi · msa MSA · sd Sudanese · bh Bahraini · tn Tunisian · lb Lebanese · ye Yemeni · sy Syrian · ps Palestinian · iq Iraqi · ly Libyan

Results

Test set: 3,079 held-out rows, identical split to the Llama-based models. Majority-class baseline 0.140.

The 6M BERT lands within 0.13 points of accuracy and 0.53 of macro-F1 of the 51.8M Llama model — essentially matching it — at 1/8.6 the size, and clearly beats the 500K model on both metrics.

Also roughly half the size of the two dialect-router releases

Outside this Llama-based ladder, two earlier dialect-ID models were released as part of the Lahgtna TTS routing pipeline: `dialect-router-v0.1` and `dialect-router-v0.2`, both fine-tunes of asafaya/bert-mini-arabic at 11.55M parameters — essentially double this model's size:

Not a strictly apples-to-apples comparison — different training corpus, different test set, and dialect-router-v0.2 also carries an English label this model doesn't — but on their own reported numbers, this model matches or slightly beats dialect-router-v0.2 at about half the parameters. (dialect-router-v0.1 reports no accuracy/F1 in its card, so it isn't in the table.)

Per-class accuracy (worst first)

The same short-text-starved classes that were weak on the 51.8M model (ps, sy, ly, ye) are weak here too — this is a data-scarcity property of the task, not something specific to this backbone.

Confusions are linguistic, not artefactual

Top confusions: ye→bh (8), ps→sy (7), sd→ye (7), sy→sd/ps/lb (6/6/5), bh→ye (6), iq→bh (5), sd→bh/ps (6/5) — all within one dialect family (Levantine: ps/sy/lb; Gulf and African: bh/ye/sd/iq). Nothing confuses across family boundaries.

The register confound, checked

The MSA class mixes transcript rows with chunks of written articles (see prepare_dialect.py). If the model were reading written-vs-spoken register instead of MSA, the transcript slice would collapse relative to the article slice:

0.968 on transcripts — matching the 51.8M model's reported 0.968 on the same check almost exactly. The model is reading MSA, not register.

Limitations

Same as the rest of this family: inputs shorter than ~25 words are unreliable across all classes; ps/sy/ly/ye are under-resourced in the source corpus; source text is YouTube speech transcripts, so performance on formal writing, social media orthography, or Arabizi is untested.

Training

Base `Nawah-BERT-6M-v2` — BERT, hidden 128, 8 layers, 2 heads. Data prep is prepare_dialect.py, unchanged from the Llama-based models (it is backbone-agnostic — text normalisation, dedup, and the train/test split don't depend on what reads the text afterward): 573,829 train / 3,079 test rows from `oddadmix/dialectal-arabic-text-no-audio-v2` (YouTube transcripts) plus MSA chunks from `oddadmix/msa_texts_summaries_new`.

train_dialect_cls_bilingual.py's MeanPoolClassifier: bare AutoModel backbone → mean pool over non-pad positions → Linear(h,h) → GELU → Linear(h, 14), cross-entropy loss. LR 3e-4 cosine, batch 64, 3 epochs, bf16, max_length 192, checkpoint selected on macro-F1. 26,901 steps, 6m16s on a single consumer GPU.

prepare_dialect.py and train_dialect_cls_bilingual.py are in this repo.

© KAND CA 2026 — PROJECT NAWAH