syvai/danish-asr-verified

danish-asr-verified

ALL rows of `syvai/danish-asr-unified` transcribed by the syv-transcribe ensemble (hviske-v5.3 + hviske-v5, confidence-weighted ROVER), each annotated with:

• —verified — True when the ensemble independently reproduced the reference exactly (compared after lowercasing, punctuation-strip, whitespace-collapse). Two independent witnesses agree => near-certain label.
• —wer_teacher_vs_ref / cer_teacher_vs_ref — word/character error rate between normalized teacher output and normalized reference; use for curriculum staging (e.g. noisy rows first, verified last) or custom quality thresholds. CER is the better signal for Danish (compound words make WER harsh).

Audio is not duplicated here. Each row points at its source clip:

• —src_file — parquet path inside syvai/danish-asr-unified (revision pinned below)
• —src_row — row index inside that file
• —key — dau:<file_idx>:<row_idx> (same key joins the teacher K=128 logprob dumps used for distillation)

Source revision pin: 5a3a49ee981baab6e1e37ddd2c45f9943c27d08f

from huggingface_hub import hf_hub_download
import pyarrow.parquet as pq

def load_audio_bytes(row):
    p = hf_hub_download("syvai/danish-asr-unified", row["src_file"], repo_type="dataset", revision="5a3a49ee981baab6e1e37ddd2c45f9943c27d08f")
    return pq.read_table(p, columns=["audio"])["audio"][row["src_row"]].as_py()["bytes"]

Fields: text (reference), teacher_text (ensemble output, differs only in casing/punctuation), source (origin corpus), duration_s, teacher_avg_logprob (teacher confidence).