Modelpublic

aufklarer/Nemotron-3-Diarization-100M-MLX-INT8

sourceHugging Faceopenmdw-1.1updated 3d agoView on Hugging Face

0likes70downloads

Model Card

Nemotron 3 Diarization · MLX INT8

Apple Silicon MLX export of NVIDIA Nemotron 3 Diarization. It emits frame-level activity probabilities for up to eight speakers. It does not transcribe speech or identify people.

Model

Property	Value
Parameters	99.2 million in the source model
Weights	INT8 affine, group size 64; other tensors FP16
Format	MLX safetensors
File size	About 102 MiB of model weights
Audio	16 kHz mono, 128 mel bands
Chunk	30.4 seconds of mel features; 10 ms output frames
Speakers	Up to eight

Files

File	Size	Purpose
`model.safetensors`	About 102 MiB	Quantized weights
`config.json`	About 2 KiB	Shapes, cache settings, source revision, and export checks
`mlx_reference.py`	About 6 KiB	Low-level MLX inference implementation
`run_chunk.py`	About 2 KiB	First-chunk command line example
`LICENSE`	About 3 KiB	OpenMDW 1.1 license
`NOTICE`	Under 1 KiB	Source and copyright notice

Performance

Measured on an Apple M5 Pro running macOS 26.6.2 with one 30.4-second chunk from each of three locally held speech recordings. Times are the median of three warm runs of the model stages. Mel extraction, file loading, speaker-cache updates, and segment postprocessing are excluded. The comparison is against the final unquantized checkpoint; this is a numerical parity check, not a diarization error rate benchmark.

Measure	MLX INT8	Meaning
Median model-stage time	32.7 ms	About 930× faster than the chunk's audio duration; lower is better
Mean absolute probability difference	0.00036–0.00073	Close to the source checkpoint; lower is better
Speaker-activity decision disagreement at 0.25 threshold	0.01–0.07% of frame/channel pairs	Quantization changes few decisions; lower is better

The speech-swift host runtime scored 10.98% pooled DER on 132 internal benchmark recordings (12.82 hours) across eight languages, using a 0.25-second collar, overlap included, and automatic speaker count. All files completed. This measures the full Swift audio-to-segment pipeline, not the first-chunk example below. Other hosts must reproduce the source model's cache and FIFO update algorithm to preserve speaker labels across chunks.

Usage

Install mlx, numpy, and safetensors. Supply a NumPy file containing preprocessed 128-band log-mel features with shape [1, 3040, 128], matching the source NeMo preprocessor. The command below processes the first chunk with empty speaker memory:

bash

python run_chunk.py --format mlx --bundle . --mel chunk.npy --output probabilities.npy

python

import mlx.core as mx
from mlx_reference import Nemotron3MLXReference, load_weights

model = Nemotron3MLXReference(load_weights("model.safetensors"))
# mel: MLX array [1, 3040, 128]; first chunk only
embeddings = model.preencode(mel)
packed = mx.concatenate((embeddings, mx.zeros((1, 304, 512))), axis=1)
probabilities_10ms, probabilities_80ms = model.infer_preencoded(packed, 380)

The first 3040 output frames correspond to the 30.4-second input. Binarize speaker activity with 0.5 onset and offset thresholds, as in the source model's default postprocessor. The pred_score_threshold: 0.25 value in config.json is used to select speaker-cache frames; it is not the segment activity threshold. For later chunks, preserve the arrival-order speaker cache and FIFO state as described by the source model.

Source