CoolFace
Modelpublic

GilgameshWind/X-ASR-zh-en

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
16likes70downloads
Model Card

<h1 align="center">🎙️ X-ASR-zh-en</h1>

<p align="center"> <b>Chinese-English offline-streaming unified ASR model artifacts for low-latency deployment.</b> </p>

<table align="center" border="0" cellspacing="0" cellpadding="0"> <tr> <td align="center" width="25%" style="border: none; padding: 0 14px;"> <a href="https://www.sjtu.edu.cn/"><img src="figure/institutions/sjtu.png" height="64" alt="Shanghai Jiao Tong University"></a> </td> <td align="center" width="25%" style="border: none; padding: 0 14px;"> <a href="https://www.sii.edu.cn/"><img src="figure/institutions/sii.png" height="64" alt="Shanghai Innovation Institute"></a> </td> <td align="center" width="25%" style="border: none; padding: 0 14px;"> <a href="https://www.fudan.edu.cn/en/"><img src="figure/institutions/fudan.png" height="64" alt="Fudan University"></a> </td> <td align="center" width="25%" style="border: none; padding: 0 14px;"> <a href="https://www.hust.edu.cn/"><img src="figure/institutions/hust.png" height="64" alt="Huazhong University of Science and Technology"></a> </td> </tr> </table>

<p align="center"> <sub><b>Participating Institutions</b></sub> </p>

<p align="center"> <a href="https://github.com/Gilgamesh-J/X-ASR">🌐 GitHub Project</a> | <a href="https://huggingface.co/GilgameshWind/X-ASR-zh-en">🤗 Hugging Face Hub</a> | <a href="https://www.modelscope.ai/Gilgamesh-J/X-ASR-zh-en">🧩 ModelScope</a> | <a href="https://huggingface.co/spaces/chenxie95/X-ASR">🪐 Hugging Face Space</a> | <a href="https://stream-asr.sjtuxlance.com/">🎧 Online Demo</a> | <a href="deployment/README.md">🚀 Deployment Guide</a> </p>

<p align="center"> <b>📄 X-ASR-zh-en Technical Report: Coming Soon</b> </p>

<p align="center"> <img src="https://img.shields.io/badge/Model%20Released-X--ASR--zh--en-blue" alt="Model released"> <img src="https://img.shields.io/badge/Languages-zh%20%7C%20en-green" alt="Languages"> <img src="https://img.shields.io/badge/Streaming-low%20latency%20%7C%20multi--mode-orange" alt="Streaming"> <img src="https://img.shields.io/badge/Deployment-sherpa--onnx-red" alt="Deployment"> <img src="https://img.shields.io/badge/License-Apache--2.0-lightgrey" alt="License"> </p>

<p align="center"> <a href="#model-card-scope">🔍 Model Card Scope</a> | <a href="#repository-contents">📦 Repository Contents</a> | <a href="#evaluation">📊 Evaluation</a> | <a href="#download">⬇️ Download</a> | <a href="#deployment">🚀 Deployment</a> </p>

<a id="model-card-scope"></a>

🔍 Model Card Scope

🧩 X-ASR Series

X-ASR is a series of automatic speech recognition models built with the icefall framework. The series focuses on streaming ASR and low-latency deployment, while also supporting offline recognition. The broader project roadmap, source organization, issue tracking, and bilingual documentation are maintained on the GitHub project page.

🤖 X-ASR-zh-en

X-ASR-zh-en is trained on approximately 1 million hours of open-source and collected speech data. It is designed as an offline-streaming unified transducer ASR model with the Zipformer architecture, supporting both offline decoding and true streaming decoding. The model provides multiple streaming chunk sizes: 160 ms, 480 ms, 960 ms, and 1920 ms, supports punctuation and casing, and can be deployed with sherpa-onnx.

<p align="center"> <img src="figure/zipformer.png" width="700" alt="Zipformer architecture"> </p>

✨ Artifact Page Notes

This repository is the model artifact page for X-ASR-zh-en.

What this artifact page providesWhat the GitHub project provides
Downloadable model artifactsProject-level overview
ONNX encoder / decoder / joiner filesBilingual README and release notes
sherpa-onnx deployment entry pointSource layout and issue tracking
Model-card metadata, tags, license, and metricsDevelopment history and contribution workflow

<a id="repository-contents"></a>

📦 Repository Contents

PathPurpose
deployment/Deployment-ready sherpa-onnx runtime files and examples
deployment/models/Exported streaming ONNX model variants
deployment/infer_and_client/WebSocket server, inference wrapper, and test client
figure/Architecture figure and demo preview media
demo/Demo video asset
applications/vibe-xasr/Vibe XASR desktop application package, manifest, and download notes
streaming_exp/Averaged/pretrained checkpoint artifact for research reference

Directory Layout

text
.
|-- README.md
|-- config.json
|-- demo/
|   `-- demo.mov
|-- applications/
|   `-- vibe-xasr/
|       |-- README.md
|       |-- download_manifest.json
|       `-- VibeXASR-1.1.2-macos-universal.dmg
|-- deployment/
|   |-- README.md
|   |-- infer_and_client/
|   |   |-- README.md
|   |   |-- sherpa_streaming_client.py
|   |   |-- sherpa_streaming_infer.py
|   |   `-- sherpa_streaming_server.py
|   `-- models/
|       |-- chunk-160ms-model/
|       |   |-- encoder-160ms.onnx
|       |   |-- decoder-160ms.onnx
|       |   |-- joiner-160ms.onnx
|       |   `-- tokens.txt
|       |-- chunk-480ms-model/
|       |-- chunk-960ms-model/
|       `-- chunk-1920ms-model/
|-- figure/
|   |-- zipformer.png
|   |-- demo-preview.png
|   `-- institutions/
`-- streaming_exp/
    `-- pretrained.pt

🧩 Model Variants

Each streaming variant contains a matched encoder, decoder, joiner, and tokens.txt. Do not mix files across model folders.

DirectoryEncoderDecoderJoinerIntended chunk
deployment/models/chunk-160ms-modelencoder-160ms.onnxdecoder-160ms.onnxjoiner-160ms.onnx160 ms
deployment/models/chunk-480ms-modelencoder-480ms.onnxdecoder-480ms.onnxjoiner-480ms.onnx480 ms
deployment/models/chunk-960ms-modelencoder-960ms.onnxdecoder-960ms.onnxjoiner-960ms.onnx960 ms
deployment/models/chunk-1920ms-modelencoder-1920ms.onnxdecoder-1920ms.onnxjoiner-1920ms.onnx1920 ms

⭐ Highlights

CategoryDescription
Frameworkicefall / k2
ArchitectureZipformer transducer
Runtimesherpa-onnx
LanguagesChinese and English
Training scaleApproximately 1 million hours of open-source and collected speech data
Recognition modesOffline decoding and true streaming decoding
Streaming chunks160 ms, 480 ms, 960 ms, 1920 ms
Text outputSupports punctuation and casing

<a id="evaluation"></a>

📊 Evaluation

The following results are for the current X-ASR-zh-en release. Values are WER/CER percentages; lower is better. All results are reported with greedy search.

<table> <thead> <tr> <th align="center" rowspan="2">Mode</th> <th align="center" rowspan="2">Chunk size</th> <th align="center" colspan="2">LibriSpeech</th> <th align="center" rowspan="2">GigaSpeech</th> <th align="center" colspan="2">WenetSpeech</th> </tr> <tr> <th align="center">clean</th> <th align="center">other</th> <th align="center">net</th> <th align="center">meeting</th> </tr> </thead> <tbody> <tr> <td align="center">Streaming</td> <td align="center">160 ms</td> <td align="center">3.91</td> <td align="center">10.17</td> <td align="center">10.97</td> <td align="center">9.45</td> <td align="center">12.04</td> </tr> <tr> <td align="center">Streaming</td> <td align="center">480 ms</td> <td align="center">3.14</td> <td align="center">7.57</td> <td align="center">9.77</td> <td align="center">7.38</td> <td align="center">9.31</td> </tr> <tr> <td align="center">Streaming</td> <td align="center">960 ms</td> <td align="center">3.12</td> <td align="center">7.22</td> <td align="center">9.62</td> <td align="center">6.96</td> <td align="center">8.84</td> </tr> <tr> <td align="center">Streaming</td> <td align="center">1920 ms</td> <td align="center">2.84</td> <td align="center">6.47</td> <td align="center">9.46</td> <td align="center">6.42</td> <td align="center">8.03</td> </tr> <tr> <td align="center">Offline</td> <td align="center">-</td> <td align="center"><b>2.69</b></td> <td align="center"><b>5.76</b></td> <td align="center"><b>9.23</b></td> <td align="center"><b>5.96</b></td> <td align="center"><b>7.20</b></td> </tr> </tbody> </table>

Note: Bold numbers indicate the best result among the listed modes for each benchmark column.

Public Benchmark Model Comparison

The following table compares representative ASR models on the same public benchmark columns. Ranks are computed by AVG across the five listed columns; lower is better. Parameter sizes are shown when provided by the source sheet.

<table> <thead> <tr> <th align="center" rowspan="2">Rank</th> <th align="center" rowspan="2">Model</th> <th align="center" rowspan="2">Params</th> <th align="center" colspan="2">LibriSpeech</th> <th align="center" rowspan="2">GigaSpeech</th> <th align="center" colspan="2">WenetSpeech</th> <th align="center" rowspan="2">AVG</th> </tr> <tr> <th align="center">clean</th> <th align="center">other</th> <th align="center">net</th> <th align="center">meeting</th> </tr> </thead> <tbody> <tr><td align="center">1</td><td align="center">Qwen3-ASR</td><td align="center">1.7B</td><td align="center">1.65</td><td align="center">3.45</td><td align="center">8.56</td><td align="center">5.29</td><td align="center">5.46</td><td align="center"><b>4.882</b></td></tr> <tr><td align="center">2</td><td align="center">Qwen3-ASR</td><td align="center">0.6B</td><td align="center">2.18</td><td align="center">4.54</td><td align="center">8.94</td><td align="center">5.97</td><td align="center">6.88</td><td align="center">5.702</td></tr> <tr><td align="center">3</td><td align="center"><b>X-ASR-zh-en</b> (offline)</td><td align="center">0.16B</td><td align="center">2.56</td><td align="center">5.56</td><td align="center">9.17</td><td align="center">5.83</td><td align="center">7.06</td><td align="center">6.036</td></tr> <tr><td align="center">4</td><td align="center">SenseVoice-small</td><td align="center">234M</td><td align="center">3.16</td><td align="center">7.21</td><td align="center">11.24</td><td align="center">5.73</td><td align="center">6.47</td><td align="center">6.762</td></tr> <tr><td align="center">5</td><td align="center">VibeVoice-ASR</td><td align="center">9B</td><td align="center">2.18</td><td align="center">5.65</td><td align="center">9.49</td><td align="center">14.45</td><td align="center">17.19</td><td align="center">9.792</td></tr> </tbody> </table>

GigaSpeechBench Vertical Domain Evaluation

The following results report GigaSpeechBench vertical-domain performance for the current X-ASR-zh-en release. Values are WER/CER percentages; lower is better. Domain abbreviations follow the GigaSpeechBench vertical-domain labels.

CH

<table> <thead> <tr> <th align="center">Mode</th> <th align="center">Chunk size</th> <th align="center">ARG</th> <th align="center">AIT</th> <th align="center">ART</th> <th align="center">BIO</th> <th align="center">ECM</th> <th align="center">ENG</th> <th align="center">ENT</th> <th align="center">FIN</th> <th align="center">HUM</th> <th align="center">LAW</th> <th align="center">MED</th> <th align="center">MIL</th> </tr> </thead> <tbody> <tr><td align="center">Streaming</td><td align="center">160 ms</td><td align="center">9.88</td><td align="center">6.76</td><td align="center">4.39</td><td align="center">7.32</td><td align="center">4.13</td><td align="center">3.58</td><td align="center">8.45</td><td align="center">3.23</td><td align="center">10.42</td><td align="center">6.58</td><td align="center">4.25</td><td align="center">2.55</td></tr> <tr><td align="center">Streaming</td><td align="center">480 ms</td><td align="center">8.67</td><td align="center">6.17</td><td align="center">3.60</td><td align="center">6.22</td><td align="center">3.78</td><td align="center">3.04</td><td align="center">7.04</td><td align="center">2.78</td><td align="center">9.43</td><td align="center">5.84</td><td align="center">3.76</td><td align="center">2.11</td></tr> <tr><td align="center">Streaming</td><td align="center">960 ms</td><td align="center">8.00</td><td align="center">5.69</td><td align="center">3.44</td><td align="center">6.10</td><td align="center">3.69</td><td align="center">2.88</td><td align="center">6.71</td><td align="center">2.72</td><td align="center">9.07</td><td align="center">5.58</td><td align="center">3.69</td><td align="center">2.11</td></tr> <tr><td align="center">Streaming</td><td align="center">1920 ms</td><td align="center">7.24</td><td align="center">5.58</td><td align="center">3.27</td><td align="center">5.82</td><td align="center">3.48</td><td align="center">2.74</td><td align="center">6.55</td><td align="center">2.57</td><td align="center">8.59</td><td align="center">4.97</td><td align="center">3.53</td><td align="center">1.94</td></tr> <tr><td align="center">Offline</td><td align="center">-</td><td align="center"><b>6.56</b></td><td align="center"><b>4.54</b></td><td align="center"><b>2.77</b></td><td align="center"><b>5.04</b></td><td align="center"><b>2.99</b></td><td align="center"><b>2.32</b></td><td align="center"><b>6.02</b></td><td align="center"><b>1.94</b></td><td align="center"><b>7.64</b></td><td align="center"><b>4.20</b></td><td align="center"><b>2.90</b></td><td align="center"><b>1.68</b></td></tr> </tbody> </table>

EN

<table> <thead> <tr> <th align="center">Mode</th> <th align="center">Chunk size</th> <th align="center">ARG</th> <th align="center">AIT</th> <th align="center">ART</th> <th align="center">BIO</th> <th align="center">ECM</th> <th align="center">ENG</th> <th align="center">ENT</th> <th align="center">FIN</th> <th align="center">HUM</th> <th align="center">LAW</th> <th align="center">MED</th> <th align="center">MIL</th> </tr> </thead> <tbody> <tr><td align="center">Streaming</td><td align="center">160 ms</td><td align="center">5.29</td><td align="center">8.57</td><td align="center">8.55</td><td align="center">7.31</td><td align="center">4.33</td><td align="center">5.01</td><td align="center">16.25</td><td align="center">5.58</td><td align="center">7.36</td><td align="center">13.39</td><td align="center">6.03</td><td align="center">6.20</td></tr> <tr><td align="center">Streaming</td><td align="center">480 ms</td><td align="center">4.62</td><td align="center">8.40</td><td align="center">7.73</td><td align="center">6.12</td><td align="center">4.19</td><td align="center">4.65</td><td align="center">14.50</td><td align="center">5.21</td><td align="center">6.79</td><td align="center">11.51</td><td align="center">5.59</td><td align="center">6.02</td></tr> <tr><td align="center">Streaming</td><td align="center">960 ms</td><td align="center">4.58</td><td align="center">8.35</td><td align="center">7.45</td><td align="center">6.00</td><td align="center">4.13</td><td align="center">4.44</td><td align="center">13.99</td><td align="center">5.12</td><td align="center">6.58</td><td align="center">10.86</td><td align="center">5.52</td><td align="center">6.04</td></tr> <tr><td align="center">Streaming</td><td align="center">1920 ms</td><td align="center">4.33</td><td align="center">8.32</td><td align="center">6.90</td><td align="center">5.89</td><td align="center"><b>4.00</b></td><td align="center">4.37</td><td align="center">13.61</td><td align="center">4.98</td><td align="center">6.39</td><td align="center">10.52</td><td align="center">5.45</td><td align="center">5.78</td></tr> <tr><td align="center">Offline</td><td align="center">-</td><td align="center"><b>4.09</b></td><td align="center"><b>8.28</b></td><td align="center"><b>6.73</b></td><td align="center"><b>5.48</b></td><td align="center">4.12</td><td align="center"><b>4.30</b></td><td align="center"><b>12.30</b></td><td align="center"><b>4.94</b></td><td align="center"><b>6.17</b></td><td align="center"><b>10.41</b></td><td align="center"><b>5.35</b></td><td align="center"><b>5.61</b></td></tr> </tbody> </table>

🎧 Demo

A sherpa-onnx based online demo is available here:

Demo video:

<a href="demo/demo.mov"> <img src="figure/demo-preview.png" width="700" alt="X-ASR demo video preview"> </a>

Open demo video

<a id="download"></a>

⬇️ Download

GitHub

Use GitHub when you want the full project repository, bilingual documentation, training references, deployment examples, and issue-tracking context.

bash
git lfs install
git clone https://github.com/Gilgamesh-J/X-ASR.git
cd X-ASR
git lfs pull

Hugging Face

Use Hugging Face when you want the model artifact page and standard HF Hub download tooling.

bash
hf download GilgameshWind/X-ASR-zh-en \
  --local-dir ./X-ASR-zh-en

You can also clone the Hugging Face repository with Git LFS:

bash
git lfs install
git clone https://huggingface.co/GilgameshWind/X-ASR-zh-en
cd X-ASR-zh-en
git lfs pull

ModelScope

Use ModelScope when you prefer the ModelScope mirror or Git LFS clone from ModelScope.

bash
git lfs install
git clone https://www.modelscope.ai/Gilgamesh-J/X-ASR-zh-en.git
cd X-ASR-zh-en
git lfs pull

<a id="deployment"></a>

🚀 Deployment

The recommended runtime is sherpa-onnx. The shortest path is to use the deployment package in this repository.

bash
cd deployment
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Start a CPU streaming server with the 160 ms model:

bash
python infer_and_client/sherpa_streaming_server.py \
  --host 0.0.0.0 \
  --port 8766 \
  --tokens models/chunk-160ms-model/tokens.txt \
  --encoder models/chunk-160ms-model/encoder-160ms.onnx \
  --decoder models/chunk-160ms-model/decoder-160ms.onnx \
  --joiner models/chunk-160ms-model/joiner-160ms.onnx \
  --provider cpu \
  --sample-rate 16000 \
  --feature-dim 80 \
  --num-threads 1 \
  --decoding-method greedy_search \
  --model-type zipformer2 \
  --enable-endpoint-detection 0 \
  --text-format none

Optional interactive tail-probe mode:

bash
python infer_and_client/sherpa_streaming_server.py \
  --host 0.0.0.0 \
  --port 8766 \
  --tokens models/chunk-160ms-model/tokens.txt \
  --encoder models/chunk-160ms-model/encoder-160ms.onnx \
  --decoder models/chunk-160ms-model/decoder-160ms.onnx \
  --joiner models/chunk-160ms-model/joiner-160ms.onnx \
  --provider cpu \
  --sample-rate 16000 \
  --feature-dim 80 \
  --num-threads 1 \
  --decoding-method greedy_search \
  --model-type zipformer2 \
  --enable-endpoint-detection 0 \
  --text-format none \
  --enable-energy-tail-probe 1 \
  --low-energy-rms 0.003 \
  --speech-rms 0.010 \
  --min-speech-ms 200 \
  --min-silence-ms 500 \
  --tail-probe-ms 500 \
  --tail-probe-cooldown-ms 1000

The default mode keeps --enable-energy-tail-probe 0 and decodes only from client audio chunks. Tail-probe mode is useful for interactive voice-input demos where trailing partial results should refresh after the user pauses. Tune the RMS and silence thresholds according to microphone gain, background noise, and frontend chunking behavior.

Test it with a WAV file:

bash
python infer_and_client/sherpa_streaming_client.py \
  --server-uri ws://127.0.0.1:8766 \
  --wav /path/to/test.wav \
  --chunk-ms 100 \
  --simulate-realtime 1

For complete runtime options, see deployment/README.md. For the script-level server/client guide and full parameter reference, see deployment/infer_and_client/README.md.

⚠️ Intended Use and Limitations

  • —This release is intended for Chinese-English ASR research, evaluation, demos, and deployment experiments.
  • —The current release focuses on streaming and offline-streaming unified recognition.
  • —Production latency depends on hardware, concurrency, audio chunking, endpointing, and server configuration.
  • —The technical report with training details, evaluation protocol, ablations, and additional analysis is coming soon.

📄 Citation

The X-ASR-zh-en technical report is coming soon. Please cite the report once it is released. For now, refer to this model card and the GitHub project page.

📜 License

This model is released under the Apache-2.0 License.

🙏 Acknowledgements

This model is trained with icefall and deployed with sherpa-onnx.

  • —icefall: https://github.com/k2-fsa/icefall
  • —sherpa-onnx: https://github.com/k2-fsa/sherpa-onnx