passpoo/talyno-whisperkit-coreml

talyno-whisper-small-finance (WhisperKit CoreML)

WhisperKit CoreML build of talyno-whisper-small-finance, a LoRA fine-tune of openai/whisper-small specialized for earnings-call and investor-meeting speech. This is the on-device model used by Talyno — an iOS app that transcribes and translates multilingual earnings calls entirely on-device.

• —Source (PyTorch) weights: `passpoo/talyno-whisper-small-finance`
• —Base model: openai/whisper-small
• —Format: WhisperKit CoreML (.mlmodelc), converted with whisperkittools
• —Languages: English (en), Chinese — Simplified + Traditional (zh)
• —Size on disk: ~466 MB
• —License: Apache 2.0

What makes this different from stock Whisper

This is not a generic ASR model. It is fine-tuned specifically on earnings-call and investor- relations language:

1. 1.Financial vocabulary & IR patterns — trained on real earnings-call audio (SPGISpeech) and financial-terminology TTS corpora, so it recognizes terms like EPS, EBITDA, guidance, revenue/margin figures, and analyst Q&A phrasing more reliably than a general-purpose model.
2. 2.Focused language scope, intentionally — rather than a shallow "50+ language" claim, this model covers exactly the languages it was actually trained on: English and Chinese (Simplified/Traditional). See the training guide for why the scope was narrowed from an earlier 6-language generation.
3. 3.On-device, offline-first — designed to run fully on iPhone Neural Engine via WhisperKit; no audio or transcript ever needs to leave the device.

Repo layout

talyno-whisper-small-finance/
├── AudioEncoder.mlmodelc/
├── TextDecoder.mlmodelc/
├── TextDecoderCoreML.mlmodelc/
└── config.json

Usage (WhisperKit / iOS)

import WhisperKit

let whisper = try await WhisperKit(
    model: "talyno-whisper-small-finance",
    modelRepo: "passpoo/talyno-whisperkit-coreml",
    downloadBase: FileManager.default
        .urls(for: .documentDirectory, in: .userDomainMask)[0]
        .appendingPathComponent("WhisperModels")
)

WhisperKit downloads the .mlmodelc bundle from this repo on first use and caches it locally. Talyno bundles the much smaller openai_whisper-tiny model for immediate offline use and offers this model as an optional "Accurate" upgrade in Settings → AI Models.

Training summary

Two-phase LoRA fine-tune on a frozen Whisper encoder:

Limitations

• —English and Chinese (Simplified/Traditional) only — other languages will not produce reliable transcripts with this model. Use the bundled openai_whisper-tiny or a stock Whisper model for broader language coverage.
• —Optimized for earnings-call / IR register; performance on casual conversational speech is not specifically validated.
• —Traditional Chinese coverage relies partly on synthesized (TTS) training audio for finance-specific vocabulary, since public real-recording Chinese speech corpora (FLEURS) do not include a Taiwan Mandarin locale.

License

Apache 2.0, consistent with the base openai/whisper-small model and the Talyno app.