Renesas/Qwen3-ASR-1.7B-GGUF

Qwen3-ASR-1.7B - Renesas X5H

Introduction

This repository contains Qwen3-ASR-1.7B model, which support language identification and speech recognition for 30 languages and 22 Chinese dialects, optimized for Renesas X5H platform for automatic speech recognition inference.

• —Model Architecture:

[image]

• —Model Summary:

Audio Encoder: | Parameter | Value | |:---:|:---:| | NUMLAYERS | 24 | | HIDDENSIZE | 1024 | | FFNDIM | 4096 | | NUMHEADS | 16 | | HEADDIM | 64 | | MELBINS | 128 | | OUTPUT_DIM | 2048 |

Text Decoder: | Parameter | Value | |:---:|:---:| | NUMLAYERS | 28 | | HIDDENSIZE | 2048 | | FFNDIM | 6144 | | NUMHEADS | 16 | | NUMKVHEADS | 8 | | HEADDIM | 128 | | GROUPSIZE | 2 | | VOCABSIZE | 151936 | | RMSNORMEPS | 1e-6 | | ROPETHETA | 1000000.0 |

• —Source Model: Qwen/Qwen3-ASR-1.7B

Performance

The following performance metrics were measured with a 15-second English audio sample. | Model | Precision | Device | Offloading | Prefill Rate (tokens/sec) Batch size=1 | Decode Rate (tokens/sec) | |:---:|:---:|:---:|:---:|:---:|:---:| | Qwen3-ASR-1.7B | W4A16 | X5H - Single Cluster NPX | AudioEncode [NPX] LLM Decoder [NPX] | 20.84 | 20.71 | | Qwen3-ASR-1.7B | FP16 | X5H - Single Cluster NPX | AudioEncode [NPX] LLM Decoder [NPX] | 12.28 | 12.23 |

Prerequisites

To run model, you need:

1. 1.Renesas X5H Board with SDK v4.34.0
2. 2.Hugging Face CLI: For downloading the model and installer.

Deployment

Qwen3-ASR-1.7B (W4A16)

1. 1.Download the installer qwen3-asr-w4a16-runner-0.1.0-Linux.sh from Files and version tab under w4a16/binaries/rcar-x5hv1/xOS-v4.34/ folder.
2. 2.Copy the installer to the X5H board and run the installer.

   bash ./qwen3-asr-w4a16-runner-0.1.0-Linux.sh --prefix=./ --exclude-subdir --skip-license

1. 1.Download GGUF files Qwen3-ASR-1.7B-FP16.gguf and mmproj-Qwen3-ASR-1.7b-FP16.gguf from Files and version tab under fp16 folder and copy to the installed directory on the X5H board.
2. 2.Download the W4A16 coefficients from w4a16/coefficients/ and copy to the installed directory.
3. 3.Expected directory structure on the X5H board.

    qwen3-asr-w4a16-runner
    ├── Qwen3-ASR-1.7B-FP16.gguf
    ├── mmproj-Qwen3-ASR-1.7b-FP16.gguf
    ├── firmwares
    ├── kernel_modules
    ├── qwen3-asr-w4a16
    ├── qwen3-asr-w4a16-runner
    ├── scripts
    ├── setup_npu.sh
    └── test_audio
        └── sample_en.wav

Inference - Qwen3-ASR-1.7B (W4A16)

bash ./setup_npu.sh
./qwen3-asr-w4a16-runner -m Qwen3-ASR-1.7B-FP16.gguf -v mmproj-Qwen3-ASR-1.7b-FP16.gguf -w test_audio/sample_en.wav -g ./ -s

Qwen3-ASR-1.7B (FP16)

1. 1.Download the installer qwen3-asr-runner-0.1.0-Linux.sh from Files and version tab under fp16/binaries/rcar-x5hv1/xOS-v4.34/ folder.
2. 2.Copy the installer to the X5H board and run the installer.

   bash ./qwen3-asr-runner-0.1.0-Linux.sh --prefix=./ --exclude-subdir --skip-license

1. 1.Download GGUF files Qwen3-ASR-1.7B-FP16.gguf and mmproj-Qwen3-ASR-1.7b-FP16.gguf from Files and version tab under fp16 folder and copy to the installed directory on the X5H board.
2. 2.Expected directory structure on the X5H board.

    qwen3-asr-runner
    ├── Qwen3-ASR-1.7B-FP16.gguf
    ├── mmproj-Qwen3-ASR-1.7b-FP16.gguf
    ├── firmwares
    ├── kernel_modules
    ├── qwen3-asr
    ├── qwen3-asr-runner
    ├── scripts
    ├── setup_npu.sh
    └── test_audio
        └── sample_en.wav

Inference - Qwen3-ASR-1.7B (FP16)

bash ./setup_npu.sh
./qwen3-asr-runner -m Qwen3-ASR-1.7B-FP16.gguf -v mmproj-Qwen3-ASR-1.7b-FP16.gguf -w test_audio/sample_en.wav -g ./ -s