FluidInference/speaker-diarization-coreml

1---2license: other3license_name: scoped-cc-by-4.04license_link: https://huggingface.co/FluidInference/speaker-diarization-coreml/blob/main/NOTICE.md5tags:6- speech7- audio8- voice9- speaker-diarization10- speaker-change-detection11- coreml12- speaker-segmentation13base_model:14- pyannote/speaker-diarization-community-115pipeline_tag: voice-activity-detection16---17 18# Speaker Diarization Core ML19 20Core ML conversions used by21[FluidAudio](https://github.com/FluidInference/FluidAudio) for on-device speaker22diarization on Apple platforms.23 24The supported Community-1 artifact set is distributed under CC-BY-4.0. See25[NOTICE.md](NOTICE.md) for attribution and the precise license scope, and26[PROVENANCE.md](PROVENANCE.md) plus [provenance.json](provenance.json) for27source, conversion, environment, and file-integrity records.28 29## Supported Community-1 artifacts30 31| Artifact | Source |32| --- | --- |33| `Segmentation.mlmodelc` | `pyannote/speaker-diarization-community-1/segmentation/pytorch_model.bin` |34| `FBank.mlmodelc` | Deterministic feature-extraction graph configured from the embedding checkpoint |35| `Embedding.mlmodelc` | `pyannote/speaker-diarization-community-1/embedding/pytorch_model.bin` |36| `PLDA.mlmodelc`, `PldaRho.mlmodelc` | `plda/plda.npz` and `plda/xvec_transform.npz` |37| `plda-parameters.json`, `xvector-transform.json` | Serialized tensors from the same two PLDA files |38 39The snapshot includes uncompiled packages for `Segmentation`, `FBank`,40`Embedding`, and `PldaRho` under `mlpackages/`; no uncompiled `PLDA` package was41published. These are Core ML conversions, not fine-tuned models.42 43## Provenance status44 45The artifacts are pinned by SHA-256 in `provenance.json`. Their historical46source lineage has been reconstructed from the public Community-1 repository,47artifact metadata, and the published conversion source. The exact local48upstream checkout and Mobius commit used for the original 2025 conversion were49not recorded, so the existing binaries are not described as a fully attested50reproducible build.51 52The immutable upstream reference used for reconstruction is53[`pyannote/speaker-diarization-community-1@3533c8cf`](https://huggingface.co/pyannote/speaker-diarization-community-1/tree/3533c8cf8e369892e6b79ff1bf80f7b0286a54ee).54The public historical conversion reference is55[`FluidInference/mobius@33fd6eab`](https://github.com/FluidInference/mobius/tree/33fd6eab634966ae7db4d73da3376a90379642fb/models/speaker-diarization/pyannote-community-1/coreml).56 57The current conversion pipeline requires a full upstream commit SHA, records58input and output hashes, embeds source metadata in each model, and captures the59build environment. It is available at60[`FluidInference/mobius@ffbc3c8`](https://github.com/FluidInference/mobius/tree/ffbc3c8cae2d0ac83912a005a25a2874ced98a3a/models/speaker-diarization/pyannote-community-1/coreml).61 62## Legacy compatibility artifacts63 64The repository also retains `pyannote_segmentation.mlmodelc`,65`wespeaker.mlmodelc`, `wespeaker_v2.mlmodelc`, and `wespeaker_int8.mlmodelc`66for FluidAudio's legacy online diarizer. They predate the Community-1 export,67record a different toolchain, and are not included in the Community-168provenance or license-scope confirmation in `NOTICE.md`.69 70## Technical specifications71 72- Input: 16 kHz mono audio73- Output: speaker segments with timestamps and speaker identifiers74- Framework: Core ML converted from PyTorch75- Deployment target: iOS 17 / macOS 14 or later76- Community-1 conversion metadata: PyTorch 2.8.0, coremltools 9.0b1,77  TorchScript source dialect78- Compiled MIL metadata: coremlc 3500.32.1, MIL 3500.14.179 80## Usage81 82See the [FluidAudio diarization documentation](https://github.com/FluidInference/FluidAudio/tree/main/Documentation/Diarization).83 84## Citations85 86### Speaker segmentation87 88```bibtex89@inproceedings{Plaquet23,90  author={Alexis Plaquet and Hervé Bredin},91  title={{Powerset multi-class cross entropy loss for neural speaker diarization}},92  year=2023,93  booktitle={Proc. INTERSPEECH 2023},94}95```96 97### Speaker embedding98 99```bibtex100@inproceedings{Wang2023,101  title={Wespeaker: A research and production oriented speaker embedding learning toolkit},102  author={Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin},103  booktitle={ICASSP 2023, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},104  pages={1--5},105  year={2023},106  organization={IEEE}107}108```109 110### Speaker clustering111 112```bibtex113@article{Landini2022,114  author={Landini, Federico and Profant, J{\'a}n and Diez, Mireia and Burget, Luk{\'a}{\v{s}}},115  title={{Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks}},116  year={2022},117  journal={Computer Speech & Language},118}119```120