algerian-nlp/DZAIR-ONNX

DZAIR-ONNX

ONNX fp32 export of algerian-nlp/DZAIR, the 105.3M-parameter encoder for Algerian Darija, for CPU inference through onnxruntime with no torch dependency at serving time.

Fidelity and latency

Measured on an Apple Silicon host (macOS arm64) against the torch fp32 release. The full report ships as onnx_report.json.

Latency is 20 warmup plus 100 timed forwards at batch 8 × length 128 on the CPU provider. It is measured and reported, never gated, and it is a property of that host — re-measure on yours before sizing anything.

For a quarter of the disk at a small fidelity cost, see DZAIR-ONNX-INT8.

Usage

onnxruntime alone — no torch, no transformers at serving time. The graph takes input_ids and attention_mask and returns last_hidden_state, with both batch and sequence axes dynamic.

import numpy as np
import onnxruntime
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

REPO = "algerian-nlp/DZAIR-ONNX"
tokenizer = AutoTokenizer.from_pretrained("algerian-nlp/DZAIR", trust_remote_code=True)
session = onnxruntime.InferenceSession(
    hf_hub_download(REPO, "model.onnx"), providers=["CPUExecutionProvider"]
)

texts = ["واش راك خويا لاباس عليك؟", "rani rayeh lel dar bech nchouf la famille ya kho"]
encoded = tokenizer([t.lower() for t in texts], padding=True, return_tensors="np")
hidden = session.run(
    None,
    {"input_ids": encoded["input_ids"], "attention_mask": encoded["attention_mask"]},
)[0]

mask = encoded["attention_mask"][..., None]
embeddings = (hidden * mask).sum(axis=1) / mask.sum(axis=1)
print(hidden.shape, embeddings.shape)  # (2, N, 768) (2, 768)

The tokenizer comes from the base repo and is also shipped here, so the graph can be served without it once ids are produced upstream.

`optimum` does not work here. ORTModelForFeatureExtraction fails to import against transformers 5.x — optimum 2.1.0 still reads is_offline_mode from transformers.utils, which 5.x removed (verified 2026-09-19). Use the onnxruntime session above, or pin transformers<5 if you need the optimum wrapper.

Input rules. Lowercase Latin spans before encoding; the tokenizer wraps each row as [CLS] ... [SEP] and pads on the right. Do not transliterate Arabizi phoneme digits (3, 7, 9) — they are atomic pieces in this vocabulary.

Files

Licence

Apache-2.0, inherited from the base model. Read the licence composition on the base card before redistributing derivatives.