Datasetpublic

KBLab/rixvox-v2

RixVox-v2: A Swedish parliamentary speech dataset RixVox-v2 is a parliamentary speech dataset spanning nearly 23000 hours of speech. The dataset was built by matching and force aligning speeches in parliamentary protocols to media recordings of debates. Each observation contains metadata about the speaker's name, gender, district, role, party affiliation, and the date the speech was given. We include identifiers for protocols, speeches and speakers that allow linking… See the full description on the dataset page: https://huggingface.co/datasets/KBLab/rixvox-v2.

sourceHugging Faceodc-byupdated 1y agoView on Hugging Face

12likes7kdownloads

Dataset Card

RixVox-v2: A Swedish parliamentary speech dataset

RixVox-v2 is a parliamentary speech dataset spanning nearly 23000 hours of speech. The dataset was built by matching and force aligning speeches in parliamentary protocols to media recordings of debates. Each observation contains metadata about the speaker's name, gender, district, role, party affiliation, and the date the speech was given. We include identifiers for protocols, speeches and speakers that allow linking observations in our dataset to further metadata in SWERIK and the Riksdag's open data.

Variable	Description
text	Text from parliamentary protocols which have been force aligned with audio. Protocols from 1966-2002 sourced from SWERIK. Speeches from 2000-2024 sourced from the Riksdag's open data.
audio	16 kHz, mono audio segments, up to 30 seconds in duration.
name	Speaker name.
party	Party membership(s) of the speaker over their career.
gender	Gender.
role	Role(s) or position(s) the speaker has had over their career.
district	The district(s) the person has represented over their career (available only if person has been a member of parliament).
year	The year the speech was given according to parliamentary protocol metadata.
dates	Date(s) when speech was possibly given according to parsed protocol metadata. A source protocol document can contain several debates spanning multiples dates.
date_approx	Earliest possible date speech could have been given according to `dates`. Included to simplify date based filtering.
start	Start time of this audio segment in the debate's audio file (seconds). See `audio_file` for source media filename.
end	End time of this audio segment in debate audio file (seconds)
duration	Duration of audio segment in seconds.
audio_file	Filename of the source media file in the Riksdag's media archive.
text_normalized	Normalized version of original protocol `text` used for calculating eval metrics (lowercased, alphanumeric, converted numbers to spelled out letters, no punctuation).
text_timestamps	Text with timestamps inserted at sentence boundaries. Enables training Whisper with ability to output timestamps.
text_previous	Text of the previous audio segment. Enables training Whisper with previous context as prompt.
wav2vec_transcription	Audio transcribed by wav2vec2.
whisper_transcription	Audio transcribed by Whisper-large-v3.
bleu_whisper	BLEU score between `text_normalized` and normalized `whisper_transcription`.
bleu_wav2vec	BLEU score between `text_normalized` and normalized `wav2vec_transcription`.
wer_whisper	WER between `text_normalized` and normalized `whisper_transcription`.
wer_wav2vec	WER between `text_normalized` and normalized `wav2vec_transcription`.
cerwhisperfirst	CER of 10 first characters of `text_normalized` and normalized `whisper_transcription`.
cerwav2vecfirst	CER of 10 first characters of `text_normalized` and normalized `wav2vec_transcription`.
cerwhisperlast	CER of 10 last characters of `text_normalized` and normalized `whisper_transcription`.
cerwav2veclast	CER of 10 last characters of `text_normalized` and normalized `wav2vec_transcription`.
is_silence	Whether the segment contains no `text` and no detected speech by wav2vec
langprobsv	Probability of spoken language being Swedish per the utput of whisper-large-v3's language detection.
shard	Shard number. Corresponds to the shard numbering used in the naming of the uploaded parquet files.
speaker_id	`id` of the speaker/person in SWERIK or in the Riksdag's systems. SWERIK `id`s start with `i-`.
riksdagen_id	`id` of the person in the Riksdag's own system (if available).
protocol_id	`id` of protocol/debate in SWERIK or the Riksdag's open data. `id`s from SWERIK start with `prot-`.
speech_id	`id` of speech in SWERIK or the Riksdag's open data. `id`s from SWERIK start with `i-`

License

RixVox-v2 is made available under the Open Data Commons Attribution License: http://opendatacommons.org/licenses/by/1.0/

If you redistribute the dataset itself, or as part of another dataset/database, see section 4.2 of the license.

If you produce a work (e.g. finetuned model) from using the contents of the database, here is an example how to provide notice/attribution:

Contains information from the dataset RixVox-v2 which is made available under the ODC Attribution License. RixVox-v2 contains information sourced from The Swedish Parliament's open data.

You may adapt the phrasing "Contains information from" as appropriate to fit the use case of your produced work (e.g. "Model was trained using the dataset RixVox-v2 which...")

Citation

Coming soon