FluidInference/paraformer-large-zh-coreml

Paraformer-large (zh) — CoreML (Apple Neural Engine)

CoreML conversion of FunASR's Paraformer-large (Mandarin Chinese) for on-device inference on Apple Silicon, for FluidInference/FluidAudio.

Paraformer is a non-autoregressive ASR model: a SANM encoder, a CIF predictor that emits one acoustic-embedding token per output character, and a parallel (single-pass) decoder. Upstream: iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch.

Files (3 CoreML stages + host CIF)

The CIF predictor runs on the host between encoder and decoder (it emits a dynamic token count, which a fixed-shape CoreML graph can't express). It's a small conv1d + linear + sigmoid → integrate-and-fire; a numpy reference (cif_numpy.py) is in the conversion repo as the Swift blueprint.

Pipeline

waveform → [Preprocessor fp32/CPU] → features [1,T,560]
        → [Encoder fp16/ANE] → enc_out [1,T,512]
        → [host CIF] → acoustic_embeds [1,L,512], token_count L
        → [Decoder fp16/ANE] → logits [1,L,8404]
        → argmax per token → drop sos(1)/eos(2)/blank(0) → CharTokenizer

Both fp16 encoder/decoder are correct on the Neural Engine. The front-end runs FP32/CPU (power-spectrum + log exceed the FP16 range). Run the encoder/decoder with MLModelConfiguration.computeUnits = .cpuAndNeuralEngine.

Conversion notes

Two SANM-specific fixes were required for fp16/ANE under bucket padding (see the conversion repo): a fp16-safe attention mask fill (-inf → -1e4), and building the encoder/decoder pad-masks from the input tensor's seq dim (so EnumeratedShapes generalize) rather than lengths.max().

Benchmark — AISHELL-1 test (CoreML on ANE)

Full test set (7,176 utts), full-CoreML pipeline on M5 Pro ANE:

Official Paraformer-large AISHELL-1 ≈ 1.95% CER (the ~0.17 pp gap is fp16 + the fixed-shape decoder padding). int8 weight quantization is accuracy-neutral (CER unchanged), ~half the size/memory.

Reproduces the published Paraformer-large AISHELL-1 number — confirming the conversion (front-end + encoder + CIF + decoder) is faithful.

License & attribution

Weights derive from FunASR's Paraformer-large; the upstream license applies. This repo is a format conversion only (no retraining). See FunASR.