aufklarer/Ultra-Sortformer-Diarization-CoreML

Ultra-Sortformer 8-Speaker Streaming Diarization — CoreML

CoreML conversion of `devsy0117/ultra_diar_streaming_sortformer_8spk_v1`, the Ultra-Sortformer fine-tune that widens NVIDIA's streaming Sortformer speaker head from four to eight speakers with SVD-initialized rows and split-learning-rate training.

Same conversion pipeline, streaming state protocol, and Arrival-Order Speaker Cache host algorithm as `aufklarer/Sortformer-Diarization-CoreML` (the 4-speaker base). Only the head width and the emitted probability tensor change: speaker_preds [1, 242, 8] (188 speaker-cache + 40 FIFO + 14 chunk frames), with the base model's cache geometry preserved.

Contents

Validation

The exported step was driven by NeMo's own streaming_feat_loader and streaming_update_async and matched the checkpoint's native forward_streaming loop (streaming-parity pytest in the conversion pipeline, passing).

Lineage and licensing

• —Base: `nvidia/diar_streaming_sortformer_4spk-v2.1` (NVIDIA Open Model License).
• —Fine-tune: `devsy0117/ultra_diar_streaming_sortformer_8spk_v1` (Ultra-Sortformer, Apache-2.0), trained on synthetic multi-speaker sessions built from the AI Hub multi-speaker speech synthesis corpus (Korean).
• —This repository redistributes a format conversion of that fine-tune; no additional training was performed.

Note: the upstream project publishes real-corpus rankings only for the 4-speaker base model. Evaluate this 8-speaker variant on your own data before preferring it — the wider head targets crowded scenes, and behavior on 2–4 speaker audio should be regression-checked per application.