pere/nb-asr-numerics-harvested

Norwegian Bokmål Numeric Expression Harvesting Dataset

This dataset contains cleaned, high-quality Norwegian Bokmål sentences containing numeric expressions harvested from both the Norwegian Colossal Corpus (NbAiLab/NCC) and the FineWeb-2 Norwegian Bokmål subset (HuggingFaceFW/fineweb-2 config nob_Latn). This is a combined high-volume intermediate dataset built for the first stage of a template-based Norwegian synthetic speech (TTS) generation pipeline to improve number/digit recognition in ASR systems.

Combined Dataset Statistics

• —Total Accepted Sentences: 3814821
• —NCC Contribution: 1,767,965 sentences
• —FineWeb-2 Contribution: 2,046,856 sentences (after cross-dataset deduplication)

Dataset Specifications

• —Source Datasets: NbAiLab/NCC and HuggingFaceFW/fineweb-2 (Bokmål subset).
• —License: cc-by-4.0 (inherits license requirements of the source corpora).
• —Language Filter: Norwegian Bokmål (filtered utilizing language label no in NCC and nob_Latn config in FineWeb-2).
• —Language Confidence Threshold: $\ge 0.90$.
• —Sentence length: 8–28 words, 45–190 characters.
• —Alphabetic Words: Minimum of 6 alphabetic words.
• —Digit Filters: 2–18 digit characters, max digit fraction of 0.20.
• —Numeric Span Count Limit: 1–3 spans.
• —Explicit Parentheses Rejection: Every sentence containing ( or ) is strictly rejected, as parentheses cause errors in downstream TTS and ASR components.
• —Character Restrictions: Only Norwegian letters (æøåÆØÅ), basic Latin letters, common accents (éèêòóôüäö), digits, ordinary spaces, and allowed symbols (. , : ; ? ! - – — / % + § ° and balanced quotes/apostrophes) are permitted.
• —Cleaning Operations:
• —Unicode normalization (NFKC).
• —Removal of control characters.
• —Typographic quotes and dashes normalization.
• —Whitespace collapse and stripping.
• —Collapsing spaces before punctuation and adding space after punctuation (where letters follow).
• —Deduplication Policy: Three-stage deduplication:
• —Exact text matches.
• —Case, whitespace, and punctuation-normalized hash matches.
• —Limit on extremely repetitive number-masked templates (e.g. Han ble født i <NUM>.).
• —Cross-dataset exact and normalized text deduplication.

Known Limitations

1. 1.No Semantic Categorisation: Semantic categorization (cardinal, ordinal, time, dates, phone numbers, etc.) has not yet been performed at this harvesting stage.
2. 2.Ambiguity: Sentences may still contain ambiguous or incorrectly detected numeric expressions.
3. 3.Draft Quality: This is an intermediate harvesting dataset for later categorization and template construction.