thomaseibner/whisper-large-v3-turbo-us-atc-v2

US ATC fine-tune for Whisper large-v3-turbo — LoRA adapter (v2)

A 13 MB LoRA adapter that fine-tunes OpenAI's `whisper-large-v3-turbo` for US air-traffic-control radio. It is not a full model: the stock turbo weights (~1.6 GB) come from Hugging Face and the adapter is applied on top. MIT licensed.

On 156 held-out ATC clips it takes word error rate from 56% to 32% and identifier recall (callsign digits, airline names, runway sides) from 53% to 83%.

Quick start

You need Python 3.10+ and ffmpeg on your PATH.

pip install -U huggingface_hub                  # for the `hf` command
hf download thomaseibner/whisper-large-v3-turbo-us-atc-v2 --local-dir whisper-atc   # or unpack the tarball
cd whisper-atc
pip install -r requirements.txt                 # torch, transformers, peft, numpy
python transcribe.py clip1.mp3 clip2.wav

The first run downloads the base model. The script uses CUDA, Apple MPS or the CPU, whichever it finds (--device overrides), and prints one file<TAB>transcript line per input.

Feed it one radio transmission per file, cut at the squelch or by a voice-activity detector. That is what it was trained on.

Applying the adapter in your own code

It is a standard PEFT adapter, so it is one extra line on top of ordinary Whisper code:

import torch
from peft import PeftModel
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from transcribe import load_audio        # from this repo: ffmpeg decode + the training audio chain

base = "openai/whisper-large-v3-turbo"
processor = WhisperProcessor.from_pretrained(base)
model = WhisperForConditionalGeneration.from_pretrained(base, torch_dtype=torch.float32)
model = PeftModel.from_pretrained(model, "thomaseibner/whisper-large-v3-turbo-us-atc-v2").merge_and_unload()   # <- the fine-tune
model = model.to("cuda", torch.float16).eval()

audio = load_audio("clip.mp3", radio_filter=True)      # 16 kHz mono float32
features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features
ids = model.generate(features.to("cuda", torch.float16), language="en", task="transcribe",
                     max_new_tokens=96, repetition_penalty=1.1)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

PeftModel.from_pretrained takes the repo id or a local directory holding adapter_config.json. Merge in float32 and cast afterwards: merging at half precision rounds every adapted weight twice.

Results

156 clips from days held out of training entirely, scored against human transcriptions:

The eval set deliberately over-samples clips our previous pipeline struggled with. By pool:

• —WER is word-level edit distance after normalising both sides: lower-cased, punctuation stripped (decimal points kept), 30L → 30 left, FL380 → flight level 380, ICAO airline codes → spoken names.
• —Identifier recall is the share of the reference's identifiers that appear in the transcript, counted as a bag (order ignored): digit groups, airline names, runway side (left/right/center) and wake category (heavy/super). These are the tokens you need to match a transmission to an aircraft, and on a seven-word clip WER is dominated by filler.

Getting good results

• —Match the audio chain. Every training and eval clip went through the same chain: 8 kHz AM airband audio resampled to 16 kHz mono, a 6th-order 300–3400 Hz bandpass, then EBU R128 loudness normalisation to −16 LUFS. transcribe.py applies it by default (--no-radio-filter skips it). To pre-filter files for another runtime:

  ffmpeg -i in.mp3 -af "aresample=16000,highpass=f=300:poles=2,highpass=f=300:poles=2,highpass=f=300:poles=2,lowpass=f=3400:poles=2,lowpass=f=3400:poles=2,lowpass=f=3400:poles=2,loudnorm=I=-16:TP=-1.5:LRA=11" -ar 16000 -ac 1 out.wav

Our own production pipeline feeds raw clips without the filter and does fine; it matters more the less your audio sounds like ours.

• —Force English, and do not prompt. Pass language="en", and no initial_prompt / prompt_ids. Every prompt we tested made results worse: a static ATC-phraseology prompt cost 16.7 WER points on stock Whisper, and on our first fine-tune a prompt listing the callsigns in the area took repetition loops from 1.7% to 13.3% of clips while recovering none of those callsigns.
• —Drop repetition loops. On silence or static the model can loop (Point Point Point…). transcribe.py flags these with the rule our production pipeline uses (8+ words, and either a run of 5 identical words or under 40% unique words). Discard whatever it flags.
• —Keep clips short. Whisper sees 30 s at a time. Split long recordings into transmissions rather than relying on long-form decoding.
• —fp16 is fine for inference; it is what we run in production. Avoid int8 quantisation (see faster-whisper below). bf16 weights produced repetition loops during training; bf16 inference is untested.

What the output looks like:

• —Numbers come out as digits, grouped the way they were spoken: Delta 2261, flight level 380, 12 right, and frequencies such as 124 85 for 124.85. Normalise downstream if you need canonical forms.
• —Waypoint and fix names come out as they sound (near Pagaz for PAGOZ). That is deliberate: the training transcripts wrote what was said rather than the resolved identifier, so the model does not learn Minneapolis-only names. Resolve names against your own fix data.

Other runtimes

The published numbers come from the transformers decode in transcribe.py. faster-whisper and mlx-whisper have their own decoders and feature extraction, so their transcripts differ slightly: mostly punctuation, and occasionally a word on the hardest clips (see Verification).

Both conversions start from a merged model:

python merge.py --out whisper-atc-turbo-v2-hf      # ~3.2 GB, float32

The merged directory can be deleted once converted.

faster-whisper (CTranslate2) — NVIDIA GPUs and CPUs

pip install ctranslate2 faster-whisper
ct2-transformers-converter --model whisper-atc-turbo-v2-hf --output_dir whisper-atc-turbo-v2-ct2 \
    --quantization float16 --copy_files tokenizer.json preprocessor_config.json

from faster_whisper import WhisperModel
from transcribe import load_audio

model = WhisperModel("whisper-atc-turbo-v2-ct2", device="cuda", compute_type="float16")
# on a CPU: WhisperModel(..., device="cpu", compute_type="float32")
segments, _ = model.transcribe(load_audio("clip.mp3", radio_filter=True), language="en", beam_size=5)
print(" ".join(s.text.strip() for s in segments))

--copy_files is not optional. faster-whisper reads the tokenizer and the feature-extractor settings from the model directory, and without them it quietly falls back to defaults that do not match this model.

Avoid compute_type="int8", even though it is the usual advice for CPUs. In our check it made errors that float32 did not: Alaska 216 came out as Alaska 266, and Clear to land as Slow land.

mlx-whisper — Apple Silicon

pip install mlx-whisper safetensors
python to_mlx.py --hf whisper-atc-turbo-v2-hf --out whisper-atc-turbo-v2-mlx
mlx_whisper --model whisper-atc-turbo-v2-mlx --language en clip.mp3

or from Python:

import mlx_whisper
from transcribe import load_audio

result = mlx_whisper.transcribe(load_audio("clip.mp3", radio_filter=True),
                                path_or_hf_repo="whisper-atc-turbo-v2-mlx", language="en")
print(result["text"])

to_mlx.py exists because mlx-whisper ships no converter for Hugging Face checkpoints. It checks every tensor against the stock mlx-community/whisper-large-v3-turbo and refuses to write a model that does not match.

Limitations

• —One area. All training audio came from receivers near Minneapolis–St Paul: KMSP tower, approach and departure, Minneapolis TRACON, St Paul tower, and about a dozen Minneapolis Center sectors. On the center frequencies it is mostly the pilot side, because the controllers' transmitters are out of range. It has not been scored anywhere else. The phraseology should carry over; local accents, the airline mix and fix names will not.
• —Small corpus. About 440 transcribed clips, 31 minutes of speech. Callsign digits are still wrong on a fair share of clips (Delta 766 came out as Delta 762).
• —Thin vocabulary. Terms that were rare in training come out as similar-sounding English: we have seen ILS as "island" and "honest", and VFR as "BFR". v2 added examples, but coverage of these, of squawk and of emergency phraseology is still thin.
• —Not for operational use.

Training details

Verification

Checked on 2026-09-10 against 8 of the held-out clips, whose transcripts from the evaluation run are on record:

Files

README.md                      this file
LICENSE                        MIT
adapter_config.json            PEFT config
adapter_model.safetensors      the weights (13 MB)
transcribe.py                  apply the adapter and transcribe
merge.py                       write a standalone merged model
to_mlx.py                      merged model -> mlx-whisper format
requirements.txt
SHA256SUMS                     tarball only; check with: shasum -a 256 -c SHA256SUMS

License

MIT — see LICENSE. The base model, openai/whisper-large-v3-turbo, is MIT-licensed by OpenAI.