DEEL-AI/STEAK

STEAK — Speech-to-Text for Error of Atc readbacK

STEAK is a synthetically generated dataset of ATCO–pilot radio exchanges — both the text and the audio are synthetic:

• —Text: assembled by formal rules, following an ontology of ATCO–pilot exchanges.
• —Audio: TTS → voice timbre / accent conversion (seed-vc) → noise addition (noise captured from real ATCO2 recordings).

One row = one audio (one ATCO controller utterance or one pilot readback). 2,519,694 audios. The ATCO↔pilot pair is always present: both roles of an exchange share the same pair_id. To rebuild the complete pairs directly:

from collections import defaultdict
from datasets import load_dataset, concatenate_datasets

subsets = ["STEAK-rare", "STEAK-medium", "STEAK-tough"]
ds = concatenate_datasets(
    [load_dataset("DEEL-AI/STEAK", s, split="train") for s in subsets]
)

pairs = defaultdict(dict)
for ex in ds:
    pairs[ex["pair_id"]][ex["role"]] = ex   # {'atco': ..., 'pilot': ...}

pair = pairs["0000001"]
atco, pilot = pair.get("atco"), pair.get("pilot")

Text generation (formal rules + ATC ontology)

82 commands in total can be generated (full table below). An utterance contains at most 7 (1 seed command + up to 6 added) and 2.2 on average (1 seed + ~1.2 added). The first (seed) command is drawn at random among the 82 with a weighting (not all equally likely): the weights come from EDA observations on the real ATC corpora ATCO2 and UWB-ATCC, so that the en-route core (climb/descend/heading/contact…) dominates and rare commands stay rare. Multiple phrasings per command: each command has several controller-side phrasings, and several ways to be read back on the pilot side — each phrasing is drawn at random with a weighting obtained from our EDA observations on ATCO2 and UWB-ATCC. The readback phrasing is not necessarily the same as the instruction (sometimes reduced to the value alone, “level 100”).

We also added incompatibility rules within a single utterance: at most one command per group (never two vertical commands, nor two heading commands, etc.), vertical-direction coherence (no “climb … rate of descent”), ground/flight exclusivity (no “taxi … descend FL90”) and phase conflicts (departure ⊥ arrival…).

The concrete values (waypoints, station names & frequencies, airports, airlines/callsigns) come from public sources:

The 82 commands and their value domains

The 82 commands are grouped by family (vertical, heading, speed…).

<details> <summary>Show the 82 commands and their value domains</summary>

</details>

A command may carry a condition — a temporal/spatial trigger attached to it.

<details> <summary>The 26 condition connectors (« … » = generated value)</summary>

</details>

At most 2 conditions per utterance. The probability of having a condition in an utterance is 0.08, and of having 2 conditions 0.08 × 0.18 = 0.0144. The command the condition is applied to is drawn at random, and on the pilot side the condition is read back with probability 0.55. The condition appears in the commandes column (atco_condition) and, if read back, in pilot_condition_readback.

Audio rendering: voice, accent, noise

The text is then synthesized and degraded to resemble a real ATC frequency:

• —Voice / timbre: each utterance is synthesized by Kokoro TTS, with a voice drawn at random among the Kokoro voices (column voice).
• —Accent (voice conversion): seed-vc then transfers the voice of a reference clip onto the synthesis. The reference clips (column accent_ref) are scraped from the Speech Accent Archive (≈ 3000 speakers of all native languages reading the same paragraph → varied accents). The pool is split by gender and the share of male voices is drawn per role (≈ 0.7 ATCO / ≈ 0.9 pilot).
• —Noise: a noise profile from real ATCO2 recordings is added (column atco2_noise_profile). No reverberation (small, absorptive ATC cabins → negligible T60).

<video controls width="100%" src="https://huggingface.co/datasets/DEEL-AI/STEAK/resolve/main/media/noisechain.mp4"> Your browser does not support the video tag — <a href="https://huggingface.co/datasets/DEEL-AI/STEAK/resolve/main/media/noisechain.mp4">download it</a>. </video>

Difficulty (measured-RMS tercile)

Difficulty = tercile of the RMS measured on the rendered WAV (column rms).

The RMS (root-mean-square) is the effective signal level: the average energy/volume of the audio over its whole duration. It is expressed in dBFS (decibels relative to full scale: 0 dBFS = full scale), so always ≤ 0; the closer to 0, the louder / more energetic the audio. Since the audio is peak-normalized at the channel output (peak fixed at 0 dBFS), the RMS — which includes the background noise and the soft-clip fold-back — is comparable across files. High RMS = noisy audio = harder.

Train / Test split

The train/test split (~50% test) is cut by pair (pair_id): a pair is always entirely in `train` OR entirely in `test`, never one role in train and the other in test — no atco↔pilot leakage between the two splits. This holds regardless of subset: the two roles may fall into different difficulty subsets (STEAK-rare/-medium/-tough, depending on their own noise), but they always share the same split.