Modelpublic

sbintuitions/kana-whisper

sourceHugging Facemitupdated 3mo agoView on Hugging Face

16likes45kdownloads

Model Card

Kana Whisper

![Arxiv](https://arxiv.org/abs/2606.25369)

Kana Whisper is a fine-tuned Whisper large-v3-turbo model that transcribes Japanese speech directly into katakana sequences.

This model is the ASR component of the Joyo Kanji Yomi Benchmark, a systematic evaluation framework for kanji-level reading accuracy of Japanese TTS systems. It was developed as part of the Sarashina2.2-TTS project, where it serves as the backbone of the Kana CER (Kana Character Error Rate) metric proposed in our paper.

Usage

With Transformers Pipeline

python

from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
import torch

model_id = "sbintuitions/kana-whisper"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True,
    use_safetensors=True,
).to("cuda")

processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch.float16,
    device="cuda",
    generate_kwargs={"language": "ja", "task": "transcribe"},
)

result = pipe("path/to/audio.wav")
print(result["text"])
# Example output: "キョーワイイテンキデスネ"

Citation

bibtex

@misc{liu2026sarashina22ttstacklingkanjipolyphony,
      title={Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesis}, 
      author={Lianbo Liu and Shiao Zhu and Kai Washizaki and Reo Yoneyama and Haesung Jeon and Mengjie Zhao and Yusuke Fujita and Hao Shi and Nao Yoshida and Yuan Gao and Roman Koshkin and Yukiya Hono and Yui Sudo},
      year={2026},
      eprint={2606.25369},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2606.25369}, 
}