Ugiat/parlament_parla_v3_punctuated

ParlamentParla v3, punctuated and capitalized (train, short segments)

A derivative of [ParlamentParla v3](https://huggingface.co/datasets/projecte-aina/parlament_parla_v3), the speech corpus of Catalan parliamentary sessions published by the Language Technologies Unit of the Barcelona Supercomputing Center (BSC-LT) within the Aina project. ParlamentParla v3 distributes its transcriptions lowercased and without any punctuation. This dataset keeps that upstream text untouched in the text column and adds a second column, text_punctuated, with punctuation and capitalization restored automatically.

• —331,820 segments, 899.6 hours of Catalan parliamentary speech (16 kHz audio), one train split.
• —The same segments and identifiers as upstream (audio resampled to 16 kHz): every row can be traced back to the original corpus through identifier.
• —The punctuation is machine-generated and has not been reviewed by humans (see How the punctuation was produced and Limitations).

This derivative was prepared by Ugiat Technologies. It is an independent work and is not produced, reviewed or endorsed by BSC-LT, the Aina project or the Parliament of Catalonia, and it is not an official version of the Parliament's records.

Why this dataset

Most public speech corpora ship transcriptions that were normalized for classic ASR training: lowercase, no punctuation, numbers spelled out. ParlamentParla v3 is one of them. An ASR model trained only on such text can output nothing but an unpunctuated, lowercase stream of words.

Many applications need the final transcription to be readable as it comes out: subtitles, minutes of meetings and parliamentary sessions, dictation, voice assistants, search over transcribed audio. There are two ways to get there:

• —Two stages: run the ASR, then apply a separate punctuation-and-casing model to its text. It works, but it adds a second model, and that model cannot hear the audio: pauses and intonation carry a large part of what punctuation encodes.
• —One stage: train the ASR model to emit punctuated and capitalized text directly. That requires audio paired with punctuated, cased transcriptions.

Punctuated Catalan speech data tends to consist of read sentences (for example Common Voice). This dataset provides punctuated and capitalized targets for 899.6 hours of spontaneous Catalan parliamentary speech, and keeps the original normalized text next to them, so both kinds of model can be trained from the same audio and compared fairly.

It is a silver-standard resource: the punctuation is predicted, not written by people, which makes it well suited to training and less suited to being used as a gold reference (see Limitations).

Example

Dataset structure

Data fields

text deliberately keeps the meaning it has upstream, so anything that already reads that column keeps working. Use text_punctuated when you want punctuated and cased targets.

Names used in this card

ParlamentParla names each subcorpus `<quality>_<split>_<length>`:

• —quality is clean or other. These are the two quality partitions the authors of ParlamentParla distribute the corpus in (the older ParlamentParla card describes it as "211 hours of clean and 400 hours of other quality segments"). The upstream documentation does not state the criterion used to separate them, so this card does not either. other does not mean another language or other content: both partitions are Catalan parliamentary speech, and upstream says that segments of low quality or not in Catalan were removed from the corpus. The names follow the clean/other convention of LibriSpeech, where other is the second, less clean partition; treat that as a hint, not a definition.
• —split is train, dev or test. This dataset contains train only.
• —length is short (under 30 seconds) or long (over 30 seconds). This dataset contains short only.

What the difference looks like in this data (our measurements, not an upstream definition): other segments are somewhat longer (10.1 s on average against 9.3 s), more of them have an unusual speech rate (12.2% of the segments fall outside 1.5-4.5 words per second, against 8.4% in clean), and the punctuation model had to be corrected because it kept changing words for 3.96% of other segments against 0.29% of clean ones. This points to other being the noisier half, with a less exact match between audio and text, and punctuation that is somewhat less reliable. It is still most of the data (189,786 of 331,820 segments), and you can filter on the quality column.

So clean_train_short means "clean quality, train split, short segments", and other_train_short means "other quality, train split, short segments". Those two are what this dataset is made of, and the quality column tells which one each row comes from.

Data instance

{
  'identifier': '7230054490601ac2176d4b17882fae65',
  'audio': {'path': '7230054490601ac2176d4b17882fae65.flac', 'array': array([...], dtype=float32), 'sampling_rate': 16000},
  'text': 'en definitiva el que plantegem són mesures concretes fruit d’un treball amb moltes entitats del nostre país que el que intenten és representar ho repeteixo un canvi de paradigma en les relacions entre les administracions i el conjunt d’entitats',
  'text_punctuated': 'En definitiva, el que plantegem són mesures concretes, fruit d’un treball amb moltes entitats del nostre país, que el que intenten és representar, ho repeteixo, un canvi de paradigma en les relacions entre les administracions i el conjunt d’entitats.',
  'quality': 'clean',
  'duration': 17.657
}

Statistics

Segment duration: mean 9.76 s, median 8.03 s, maximum 42.7 s. Average length: 26.7 words per segment.

The hours are measured from the audio in this dataset: on a sample of 800 segments the duration values equal the lengths of the original upstream WAV files to within 1 ms. They are higher than the per-subcorpus figures in the upstream card table (that table lists 507:27:34 for other_train_short, while the same 189,786 segments add up to 534.5 hours here, about 5% more); we could not determine how the upstream figures were computed.

Punctuation in text_punctuated (per 100 words): 7.31 commas, 4.22 full stops, 0.22 question marks, 0.35 colons, 0.12 semicolons, 0.01 exclamation marks. 99.90% of segments start with a capital letter or an opening mark and 99.88% end with a closing mark.

Loading

from datasets import load_dataset

ds = load_dataset("Ugiat/parlament_parla_v3_punctuated", split="train")
row = ds[0]
print(row["text"])
print(row["text_punctuated"])
print(row["audio"]["sampling_rate"])   # 16000

# streaming, without downloading everything first
ds = load_dataset("Ugiat/parlament_parla_v3_punctuated", split="train", streaming=True)

Relationship to the original dataset

Please read this section before using the data: everything that differs from upstream is listed here.

1. 1.Subset. Only the train split, short segments (clean_train_short and other_train_short) are included. Dev, test and long segments are not part of this dataset; use the upstream repository for those. Upstream has 350,780 train-short segments, this dataset has 331,820 (94.6%): all of other_train_short and 142,034 of the 160,994 clean_train_short segments. The remaining 18,960 clean segments are not included: they could not be retrieved when the data was downloaded (at least one source archive on the Hub is truncated; we did not investigate further). No other selection was applied. None of the segments belongs to the upstream dev or test splits (checked by identifier).
2. 2.Audio. The source files are mono 32-bit float WAV at 22.05 kHz. Here the audio is resampled to 16 kHz with a polyphase filter, quantized to 16-bit PCM and stored as lossless FLAC inside Parquet. To our knowledge no other processing was applied (no normalization, denoising or trimming). The duration column matches the length of the stored audio.
3. 3.Identifiers. identifier is the upstream identifier. The upstream segment_path column is not included.
4. 4.`text` is upstream, unchanged. In every row it is byte-identical to the text of the upstream CSV.
5. 5.`text_punctuated` is new. Compared with text it differs in:
6. 6.punctuation and capitalization (the purpose of this dataset);
7. 7.removal of extraction debris that the upstream text carries: soft hyphens inside words (9,080 segments, for example execu­cions with an invisible hyphen) and stray middle dots inside words (9,894 segments, for example te·nim, recur·sos, elec· cions). The Catalan geminate l·l (col·legi) is left as it is, and a middle dot with a space on both sides (CUP · Alternativa) is kept because it is a separator in the source;
8. 8.hyphens and apostrophes, which the model may attach (posar li → posar-li);
9. 9.words split by a space or glued together in the upstream text, which the model may repair (ma nera → manera, dirho → dir ho). The word segmentation of text_punctuated differs from text in 19,836 segments (5.98%), for example: se ls po dria aplicar allò → se ls podria aplicar allò; senyores i se nyors diputats una → senyores i senyors diputats una; per exemple so bre la incorporació → per exemple sobre la incorporació;
10. 10.the sequence of letters is never changed: after lowercasing and ignoring punctuation, whitespace and hyphens, text_punctuated equals text in 100.00% of the segments (0 segments differ).

Words are not corrected, normalized or rewritten: dialectal, Aranese/Occitan and non-standard forms stay exactly as they are in the upstream text.

How the punctuation was produced

ParlamentParla v3 provides no punctuated text, so text_punctuated is a prediction, not a recovered original.

1. 1.Batches of segments were sent to Anthropic's Claude models with an instruction to restore capitalization and punctuation following standard Catalan orthography and to keep every word exactly as given. Most of the corpus was processed with Claude Haiku 4.5 (claude-haiku-4-5-20251001); the first 37 of the 83 shards (about 44.6% of the segments, in the order of this dataset) were processed in an earlier run, mostly with Claude Sonnet 5 (claude-sonnet-5). The instruction is reproduced at the end of this section.
2. 2.Every answer was checked automatically: it was accepted only if its letters were identical to the input (ignoring case, punctuation and spaces). A batch whose output did not parse or did not match in size was retried, and segments that were rejected were sent again in later passes (with Claude Haiku 4.5, for all shards).
3. 3.Segments whose answer changed words (the model tends to "correct" dialectal forms, Aranese words or spoken numbers) were not overwritten with the model's words. Instead the model's punctuation and capitals were transferred onto the original words by word-level alignment, and a segment where the model had changed more than half of the words was left unpunctuated. In this final stage 7,938 segments (about 2.4%) were still unpunctuated: 7,893 received transferred punctuation and 45 were left exactly as upstream.
4. 4.Extraction debris was removed and spacing normalized as described above.
5. 5.The final data was verified against the upstream text row by row (the numbers in the previous section).

124 segments (0.04%) contain no punctuation mark at all in text_punctuated: 76 of them are sentence fragments that start with a capital letter but end mid-sentence, and 48 are fully lowercase segments the procedure left exactly as upstream.

<details> <summary>The instruction given to the model</summary>

You are restoring proper Catalan orthography (capitalization and punctuation) to raw ASR transcripts that are currently all-lowercase and unpunctuated.

Rules:
- Preserve every word EXACTLY as given — do not add, remove, reorder, or paraphrase any word.
- Only add capitalization (proper nouns, sentence starts) and punctuation (periods, commas, question marks, etc.) as grammatically correct standard Catalan requires.
- Split run-on text into multiple correct sentences where grammatically appropriate, but do not invent sentence breaks around simple conjunctions like "i" unless truly grammatically warranted.
- The punt volat "·" is ONLY valid inside "l·l" (e.g. col·legi, novel·la) — never insert it anywhere else.
- Your ENTIRE response must be nothing but the JSON array itself — no preamble sentence, no explanation, no markdown code fences, no trailing commentary. The first character of your response must be "[".

Input (JSON array of {n} raw transcripts):
{input_json}

{n} and {input_json} were replaced by the number of segments in the batch and the batch itself as a JSON array (100 to 200 segments per call).

</details>

No human reviewed the punctuation, and there is no human-punctuated reference to score it against.

Uses

Intended for training and evaluating speech recognition that outputs punctuation and capitalization, and for training text models that restore them. It can also serve as a plain Catalan ASR training set through the text column, exactly as upstream.

Out of scope: using text_punctuated as a gold standard to evaluate punctuation quality (it is predicted); identifying speakers (see below).

Limitations

• —The punctuation is machine-generated. Expect errors: commas and sentence boundaries are partly a matter of judgement and the model can be wrong, especially on long, spontaneous, hesitant speech.
• —It is not the punctuation of the official parliamentary record. The upstream transcriptions were normalized (lowercased, punctuation removed, numbers spelled out) before release, so the original punctuation is not available. There are no digits or symbols: trenta dos stays trenta dos.
• —Segments are not complete sentences. Many start or end in the middle of one; the punctuation describes the segment, not the text around it.
• —Non-standard forms are kept as transcribed. The corpus contains Aranese/Occitan and other forms spoken in the Parliament. Example (upstream text in text, this dataset in text_punctuated):

bona hèsta entà toti i ara si em permet després de que vostè doncs acabi de fer la seva reflexió jo li contestaré → Bona hèsta entà toti i ara si em permet, després de que vostè doncs acabi de fer la seva reflexió, jo li contestaré.

Punctuation and capitalization are less reliable on such segments.

• —Proper-noun capitalization is inferred by the model (names of parties, institutions, places) and may be inconsistent.
• —Train split only, and incomplete (see the first point above).
• —Upstream transcriptions can contain errors; they are not corrected here.

Licensing, attribution and terms of use

• —License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), the same license as the upstream dataset. You may share and adapt this dataset, including commercially, as long as you give appropriate credit, link to the license and indicate if changes were made.
• —Source data and the Parliament's terms. The information comes from the plenary sessions of the Parliament of Catalonia (Parlament de Catalunya), which is the source and owner of the content. Upstream states that the data was released in accordance with the Parliament's legal notice. That notice (read on 2026-09-20) allows reuse without prior authorization or fee, on these conditions: (1) the meaning must not be altered or distorted; (2) it must not be presented as an official version of the documents, nor as a version made with the collaboration or endorsement of the Parliament; (3) the Parliament must be identified as the source; (4) the date of the data, or of its retrieval, must be given. How this dataset meets them: (1) the upstream text is kept unchanged in text, and text_punctuated only adds punctuation and capitalization without changing any letter (the punctuation is machine-generated and can misread a sentence, see Limitations); (2) this is not an official version and is not endorsed by the Parliament; (3) the Parliament is identified as the source here; (4) dates: the recordings are from sessions held between January 23, 2008 and October 24, 2023 and the upstream corpus was extracted on March 6, 2024 (both as stated by upstream); the files used to build this dataset were obtained from the upstream repository on July 17, 2026. No logos, marks or emblems of the Parliament are used. Please check that your own use is also compatible with the notice.
• —Speakers. The upstream card states that the recordings contain speech, which is personal data, that the speakers' voices were subjected to anonymization treatment in line with the GDPR, and that users agree not to attempt to determine the identity of the speakers. We ask users of this derivative to follow the same rule.
• —Changes made (required by the license): see Relationship to the original dataset.

Please cite the original corpus

If you use this dataset, please credit the original ParlamentParla v3 corpus (curated by the Language Technologies Unit at the Barcelona Supercomputing Center, funded by the Government of Catalonia through the Aina project) and mention this derivative:

@misc{bscib32024,
      title={ParlamentParla v3 - Speech Corpus of Catalan Parliamentary Sessions},
      author={Baybars, Kulebi},
      publisher={Barcelona Supercomputing Center},
      year={2024},
      url={},
}

Suggested reference for this derivative:

@misc{parlament_parla_v3_punctuated,
      title={ParlamentParla v3, punctuated and capitalized (train, short segments)},
      author={Ugiat},
      organization={Ugiat Technologies},
      year={2026},
      howpublished={\url{https://huggingface.co/datasets/Ugiat/parlament_parla_v3_punctuated}},
      note={Derivative of ParlamentParla v3 (BSC-LT), CC BY 4.0}
}

Contact

Ugiat Technologies, <ugiat@ugiat.com>.