dataseek/ptbr-dou

Diário Oficial da União 2025-2026 (DO1 + DO1E)

Part of the MagTina350m pretrain corpus release by Dataseek under the Magestic.ai brand. This is one of nine silver-layer datasets that fed `dataseek/magtina350m-base`.

Summary

20 K articles from the Brazilian Official Federal Gazette covering 2025-2026 — sections DO1 (regular edition) and DO1E (extra edition). Rich structured metadata: publication date, edition, page, PDF URL, ementa (summary), full text. Small corpus but high signal for legal/admin PT-BR.

Source and collection method

Source: INLABS portal — https://inlabs.in.gov.br/ → daily ZIP downloads → XML parse → DO1 + DO1E section filter → dedup by id_materia.

ETL script (in the MagTina1B repository): `scripts/etl/09_dou_2026_v1.py` (public release of the ETL scripts is on the roadmap; until then the data card below documents the recipe in full).

Filters and deduplication

The following filters were applied before this dataset reached its silver (release-ready) state:

• —Sections DO1 (regular) and DO1E (extra) only
• —Dedup by id_materia

Global URL-normalised deduplication was applied across all web-derived corpora (webpages, news, blogs) so the same article does not appear twice across those three datasets.

Schema

Columns dropped at export (kept private as ETL internals): none

Size statistics

Used in MagTina350m pretrain: 0.017 B tokens (0.1 % of MagTina350m's 17.39 B-token pretrain budget).

How to load

from datasets import load_dataset

ds = load_dataset("dataseek/ptbr-dou", split="train", streaming=True)
for row in ds.take(5):
    print(row["text"][:200])

Streaming is recommended for the larger configs. For the smaller datasets (ptbr-dou, ptbr-books-publicos) eager loading is fine.

Licensing

CC0 1.0 / Public Domain. Official acts of the Brazilian federal government are not protected by copyright (Lei 9.610/98 art. 8 § I). Freely redistributable.

Upstream attribution: INLABS / Imprensa Nacional — https://inlabs.in.gov.br/ (Brazilian Federal Press Office)

Citation

If you use this dataset, please cite both the upstream source and MagTina350m:

@misc{magtina350m_pretrain_2026,
  title  = {MagTina350m pretrain corpus — Diário Oficial da União 2025-2026 (DO1 + DO1E)},
  author = {Frasson, Ricardo and {Dataseek Team}},
  year   = 2026,
  publisher = {Hugging Face},
  url    = {https://huggingface.co/datasets/dataseek/ptbr-dou}
}

Please also honour the upstream license terms — for CC-BY-derived data, attribution to the upstream creators is mandatory; for CC-BY-SA, downstream derivatives must remain CC-BY-SA-compatible.

Intended use

• —Pre-training, continued pre-training, or domain-adapting of Brazilian Portuguese language models.
• —PT-BR NLP research where statistically representative public-web / academic / legal / encyclopedic data is needed.
• —Reproducing or improving on the MagTina350m result.

Known limitations and PII statement

• —Text was NOT PII-scrubbed. URLs, emails, phone numbers and personal names that occurred in the source data may still be present. We strip zero-width characters and normalise Unicode but we do not run an NER pass.
• —Crawled data carries upstream biases of CommonCrawl, Wikipedia, news outlets and academic institutions present in the source. We have not audited these.
• —No safety filtering beyond langid and basic alpha-ratio gates. Hate-speech, spam and adult content present in the source remain unless caught incidentally.
• —Provenance preserved at row level. Every row has either a url, source or doc_id column that points back to upstream — this is intentional, so consumers can re-license, redact or filter.

Related releases

• —Model: `dataseek/magtina350m-base` (354.6 M params, pretrained on this corpus + 8 sibling datasets)
• —Instruct model: `dataseek/magtina350m-instruct`
• —Sibling datasets: see dataseek/ptbr-* for all nine corpora

License

cc0-1.0