Modelpublic

olafura/gemma-language-detection

sourceHugging Faceapache-2.0updated 8d agoView on Hugging Face

0likes

Model Card

Gemma 4 spoken language detector (work in progress)

Spoken language identification from one second of audio, built from the first five blocks of the Gemma 4 E2B audio tower with a small linear head on mean/std-pooled features. The tower slice is frozen Gemma weights (127M parameters); each head is about 100k parameters. Everything is run with Elixir/Nx from olafura/gemma-4-mic-transcribe (language_id detect | serve | validate), on CPU through Torchx or on an Nvidia GPU through EXLA.

This is a WIP snapshot, not a release: the weak languages below are weak because of training data, not by design.

Files

folder	languages	head fitted on
`sent49/`	49	Common Voice sentence onsets, up to 1500 per language, 47.6k clips
`mixed34/`	34	18 sentence-onset languages plus 16 single-word languages (incl. zh)

Each folder is one language_id artifact: parameters.safetensors (tower slice plus head, 254 MB) and manifest.etf (languages, depth, window, pooling, parameter paths). Load it with

bash

./language_id serve --artifact sent49 --port 7861
curl -s -X POST --data-binary @clip.wav 'localhost:7861/detect?languages=en,de,sv-SE'

Accuracy

Top-1 / top-3 on held-out speakers, one second of audio, all languages open unless stated:

head	test set	top-1	top-3
sent49	Common Voice sentence onsets, 49 languages, 1436 clips	48.7%	66.4%
sent49	same, 18 best-covered languages	70.0%	87.4%
sent49	same, 18 languages told as candidates	76.1%	90.9%
mixed34	Common Voice sentence onsets, its 18 sentence languages	75.2%	89.3%
sent49	fixie-ai/language_detection-audio, 21 candidates, first 1 s	46.0%	64.3%
sent49	same, 1 s windows summed over the whole clip	57.7%	74.7%

On the last set the 32B Ultravox (v0.6, Qwen3-32B) gets 45.0% from the same first second and 89.3% from whole clips. Model time per one-second clip: 93 ms on an A100, 143 ms on a Strix Halo CPU.

Languages with fewer than 1000 training onsets, where the head mostly answers a neighbour: te 8, oc 34, ko 47, ast 49, ml 158, sl 174, mk 211, sr 235, el 240, ha 241, fi 260, mn 272, mr 277, vi 288, et 395, sk 408, da 436, hi 587, bg 607, ro 643, ur 671, lt 907.

Training data

Common Voice 17 (fixie-ai/commonvoice17_0), train split, speakers disjoint from the test shards. Base model: google/gemma-4-E2B-it, subject to the Gemma terms of use; the head and code are Apache-2.0.