Modelpublic

smdesai/canary-1b-v2-pal6-coreml

sourceHugging Facecc-by-4.0updated 18d agoView on Hugging Face

0likes

Model Card

Canary 1B v2 — CoreML (6-bit palettized encoder, KV-cache)

Memory-optimized CoreML build of NVIDIA canary-1b-v2 for Apple silicon: the encoder's weights are k-means palettized to 6 bits (per grouped channel, 16 channels per lookup table) and decompressed in flight on the Apple Neural Engine. Smaller than the INT8-encoder build (canary-1b-v2-int8-coreml) with accuracy matching the FP16 reference build (canary-1b-v2-coreml). Decoder and cross-KV stay FP16.

Base model: nvidia/canary-1b-v2 (NVIDIA NeMo EncDecMultiTaskModel, 1B parameters, 25 European languages, ASR + speech translation). License: the base model is released under CC-BY-4.0; this conversion carries the same license. Please attribute NVIDIA for the model.

This build


Encoder	6-bit k-means palette, 16-channel groups; Neural Engine-resident, 614 MB on disk, ~0.63 GB resident
Decoder / cross-KV	FP16 (unchanged), 271 MB + 34 MB
Download	0.92 GB
iOS RAM while transcribing	1.0 GB (measured) vs 1.9 GB for the FP16 build
Encoder latency	~43 ms per 15 s window (M3 Max, warm; ~7 ms slower than INT8 because of the lookup-table decode); first-load Neural Engine compile ~24 s vs ~14 s

Build family

All 1B v2 builds share the same preprocessor, tokenizer, package layout, and decode contract; only the weight format of the encoder and/or decoder differs. Download sizes are as hosted on the Hub; iOS RAM was measured in an app while transcribing a 6-minute file.

Repo	Build	Encoder weights	Decoder weights	Download	iOS RAM
`canary-1b-v2-coreml`	FP16 (reference)	FP16	FP16	1.89 GB	1.9 GB (measured)
`canary-1b-v2-int8-coreml`	INT8 encoder	INT8 per-channel	FP16	1.10 GB	1.2 GB (measured)
`canary-1b-v2-pal6-coreml` (this repo)	6-bit encoder	6-bit palette, g=16	FP16	0.92 GB	1.0 GB (measured)
`canary-1b-v2-int8full-coreml`	INT8 full	INT8 per-channel	INT8 per-channel	0.95 GB	1.0 GB (measured)
`canary-1b-v2-pal6-int8-coreml`	6-bit + INT8	6-bit palette, g=16	INT8 per-channel	0.77 GB	910 MB (measured)
`canary-180m-flash-coreml`	180M Flash	FP16 (17L, d=512)	FP16 (4L)	0.37 GB	470 MB (measured)

Files

File	Contents
`canary_preprocessor.mlmodelc`	FP32 mel front end (128 mel bins, 16 kHz mono, 15 s window)
`canary_encoder.mlmodelc`	FastConformer encoder, 32 layers, d=1024, subsampling 8
`canary_cross_kv.mlmodelc`	Cross-attention K/V projection, 8 layers (run once per window)
`canary_decoder_kv.mlmodelc`	Stateful single-step Transformer decoder, 8 layers, 16384-way LM head
`canary_spe.model`	SentencePiece tokenizer (16384 pieces)
`metadata.json`	Shapes, decode geometry, seed tokens, and the weight recipe of this build

Credits

Model: NVIDIA NeMo team, canary-1b-v2, CC-BY-4.0.