Modelpublic

soniqo/MOSS-Transcribe-Diarize-0.9B-ONNX-INT8

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

1likes7downloads

Model Card

MOSS-Transcribe-Diarize-0.9B-ONNX-INT8

ONNX conversion of OpenMOSS-Team/MOSS-Transcribe-Diarize, pinned to revision e6d68cdfcddbdad1a7e8454f0cb859cad76e2502. The model produces timestamped, speaker-attributed text in the form [start][Sxx]text[end].

Model

Property	Value
Parameters	908,513,280 total: 312,463,360 audio/VQ + 596,049,920 text decoder
Architecture	24-layer Whisper-medium encoder, 4x VQ adaptor, 28-layer Qwen3 decoder
Quantization	Core ML FP16 audio + WebGPU symmetric INT8 block-32 ONNX decoder
Format	Compiled Core ML FP16 audio plus ONNX opset 23 WebGPU MatMulNBits decoder
Bundle size	2.06 GiB
Audio	16 kHz mono; non-overlapping 30-second encoder chunks
Output	Timestamps, anonymous speaker labels, transcription text
Context	Fixed 1,024-token in-place WebGPU K/V cache

Files

File	Size	Description
`added_tokens.json`	707 B	Additional token IDs
`audio_encoder.mlmodelc/`	596.3 MiB	Compiled Whisper encoder and VQ adaptor
`audio_encoder.onnx`	596.2 MiB	FP16 Whisper encoder and VQ adaptor graph
`chat_template.jinja`	4.7 KiB	Prompt and audio-placeholder template
`config.json`	2.6 KiB	Root loader metadata and download-counting query file
`decoder.onnx`	387.6 KiB	Qwen3 decoder graph with documented K/V-cache inputs
`decoder.onnx.data`	900.8 MiB	External decoder tensor data
`export_config.json`	18.5 KiB	Source revision, artifact hashes, and graph contract
`generation_config.json`	107 B	Greedy generation token defaults
`merges.txt`	1.6 MiB	Byte-pair merge rules
`preprocessor_config.json`	315 B	16 kHz Whisper feature-extractor settings
`processing_moss_transcribe_diarize.py`	10.7 KiB	Upstream processor implementation
`processor_config.json`	292 B	Audio-token and timestamp-marker settings
`source_config.json`	2.3 KiB	Pinned upstream model geometry
`special_tokens_map.json`	613 B	Special token definitions
`tokenizer.json`	10.9 MiB	Qwen tokenizer vocabulary and rules
`tokenizer_config.json`	503 B	Tokenizer configuration
`validation.json`	8.4 KiB	Measured quality, speed, memory, and parity results
`vocab.json`	2.6 MiB	Byte-pair vocabulary

Performance

Measured with greedy decoding on an Apple M5 Pro with 48 GB unified memory. Word error rate (WER), character error rate (CER), and real-time factor (RTF) are lower when better. Throughput is 1 / RTF, is higher when better, and reports how many seconds of audio are processed per wall-clock second. RTF excludes model loading. RSS is process memory sampled at clip boundaries; OS high-water RSS also includes transient peaks when the operating system reports it.

Slice	Samples	WER	CER	RTF	Throughput	Sampled / OS high-water RSS	Plain parity vs FP32
english	80	8.43	5.36	0.0458	21.8x real-time	2,186 / 2,623 MB	76/80

Aggregate inference phases across 80 clips: Processor 0.18 s, Audio encoder 5.57 s, Decoder prefill 3.90 s, Token decode 25.12 s, Other host work 0.01 s; 5.602 ms/generated token.

Multilingual precision check

Slice	Samples	WER	CER	RTF	Throughput	Plain parity vs FP32
german	10	13.37	5.99	0.0389	25.7x real-time	10/10
french	10	5.36	2.07	0.0610	16.4x real-time	9/10
msa	10	27.00	8.64	0.0470	21.3x real-time	8/10

Round trips

—30-second overlapping AMI window: RTF 0.077; text=exact, speaker_sequence=exact, raw=drift. The source predicted 3 speakers where RTTM contains 4.
—33.12-second two-chunk German clip: RTF 0.036; text=exact, speaker_sequence=exact, raw=exact.

The multilingual check covers only ten German, ten French, and ten Modern Standard Arabic clips. It is a conversion check, not proof of the upstream model's full 50-language quality. The meeting round trip checks transcript and speaker-sequence parity but is not a diarization error-rate benchmark.

Usage

Python

python

from pathlib import Path

import coremltools as ct
import onnxruntime as ort
import onnxruntime_ep_webgpu as webgpu_ep
from huggingface_hub import snapshot_download

bundle = Path(snapshot_download("soniqo/MOSS-Transcribe-Diarize-0.9B-ONNX-INT8"))
ort.register_execution_provider_library("webgpu", webgpu_ep.get_library_path())
devices = [
    device for device in ort.get_ep_devices()
    if device.ep_name == webgpu_ep.get_ep_name()
]
options = ort.SessionOptions()
options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_BASIC
options.add_provider_for_devices(devices, {})
audio = ct.models.CompiledMLModel(str(bundle / "audio_encoder.mlmodelc"))
decoder = ort.InferenceSession(str(bundle / "decoder.onnx"), sess_options=options)

Allocate each past_key_N / past_value_N as a 1,024-row WebGPU OrtValue. Use ONNX Runtime I/O binding to bind the same OrtValue to the corresponding present_key_N / present_value_N output. Pass the real seqlens_k and total_sequence_length; capacity is not the logical token position.

Command line

bash

hf download soniqo/MOSS-Transcribe-Diarize-0.9B-ONNX-INT8 --local-dir ./moss-transcribe-diarize

This downloads a complete low-level model bundle. SDK integration is tracked in speech-swift issue #388; until that integration lands, applications must implement the documented host contract around the exported graphs or weights.

Runtime contract

Decoder linear weights use homogeneous symmetric INT8 block-32 MatMulNBits. Q/K/V and gate/up projections are fused into wide calls. Its per-layer FP16 K/V buffers remain on the WebGPU device and are aliased as both past inputs and present outputs; the host passes the logical cache lengths directly. Prompt construction, audio chunking, generation, cache management, and transcript parsing remain host responsibilities. Machine-readable details and measured results are in config.json, export_config.json, and validation.json.

Source

No training was performed for this conversion. The source weights contain 908,513,280 parameters and are licensed under Apache 2.0.

Limitations

Speaker IDs are anonymous within each inference. The source model can emit malformed or overlapping timestamps and can miss speakers; conversion parity does not correct those behaviors. The WebGPU execution provider is a preview plugin and this profile requires macOS plus the bundled Core ML audio encoder. The fixed decoder cache limits total prompt plus generated tokens to 1,024. The validation here does not establish the upstream 90-minute claim for this deployment format.

License

Apache License 2.0, inherited from the upstream model.