dataseek/ptbr-gov-legal

PT-BR Legal & Government Documents

Part of the MagTina350m pretrain corpus release by Dataseek under the Magestic.ai brand. This is one of nine silver-layer datasets that fed `dataseek/magtina350m-base`.

Summary

935 K Brazilian legal and government documents: federal/state laws, court decisions, regulatory acts, official communications. Mixed corpus combining eduagarcia/LegalPT_dedup (HuggingFace) with a Kaggle Brazilian-legal-proceedings dump.

Source and collection method

Sources: eduagarcia/LegalPT_dedup (HuggingFace) + kaggle brazilian-legal-proceedings → NFKD → alpha-ratio filter → langid='pt' gate → SHA-1 dedup. Per-row source column lets you slice back to the upstream.

ETL script (in the MagTina1B repository): `scripts/etl/08_gov_legal_v1.py` (public release of the ETL scripts is on the roadmap; until then the data card below documents the recipe in full).

Filters and deduplication

The following filters were applied before this dataset reached its silver (release-ready) state:

• —alpha_ratio ≥ 0.65
• —len(text) ≥ 200 chars
• —FastText langid='pt'
• —SHA-1(text[:512]) dedup

Global URL-normalised deduplication was applied across all web-derived corpora (webpages, news, blogs) so the same article does not appear twice across those three datasets.

Schema

Columns dropped at export (kept private as ETL internals): none

Size statistics

Used in MagTina350m pretrain: 1.320 B tokens (7.6 % of MagTina350m's 17.39 B-token pretrain budget).

How to load

from datasets import load_dataset

ds = load_dataset("dataseek/ptbr-gov-legal", split="train", streaming=True)
for row in ds.take(5):
    print(row["text"][:200])

Streaming is recommended for the larger configs. For the smaller datasets (ptbr-dou, ptbr-books-publicos) eager loading is fine.

Licensing

CC0 1.0 for Brazilian federal government works (per Lei 9.610/98 art. 8, official acts of the State are not protected by copyright). The eduagarcia LegalPTdedup subset carries CC-BY-SA on aggregation; this corpus inherits that obligation where applicable. See `metajson` for the per-row source tag.

Upstream attribution: eduagarcia/LegalPTdedup — https://huggingface.co/datasets/eduagarcia/LegalPTdedup ; Kaggle brazilian-legal-proceedings

Citation

If you use this dataset, please cite both the upstream source and MagTina350m:

@misc{magtina350m_pretrain_2026,
  title  = {MagTina350m pretrain corpus — PT-BR Legal & Government Documents},
  author = {Frasson, Ricardo and {Dataseek Team}},
  year   = 2026,
  publisher = {Hugging Face},
  url    = {https://huggingface.co/datasets/dataseek/ptbr-gov-legal}
}

Please also honour the upstream license terms — for CC-BY-derived data, attribution to the upstream creators is mandatory; for CC-BY-SA, downstream derivatives must remain CC-BY-SA-compatible.

Intended use

• —Pre-training, continued pre-training, or domain-adapting of Brazilian Portuguese language models.
• —PT-BR NLP research where statistically representative public-web / academic / legal / encyclopedic data is needed.
• —Reproducing or improving on the MagTina350m result.

Known limitations and PII statement

• —Text was NOT PII-scrubbed. URLs, emails, phone numbers and personal names that occurred in the source data may still be present. We strip zero-width characters and normalise Unicode but we do not run an NER pass.
• —Crawled data carries upstream biases of CommonCrawl, Wikipedia, news outlets and academic institutions present in the source. We have not audited these.
• —No safety filtering beyond langid and basic alpha-ratio gates. Hate-speech, spam and adult content present in the source remain unless caught incidentally.
• —Provenance preserved at row level. Every row has either a url, source or doc_id column that points back to upstream — this is intentional, so consumers can re-license, redact or filter.

Related releases

• —Model: `dataseek/magtina350m-base` (354.6 M params, pretrained on this corpus + 8 sibling datasets)
• —Instruct model: `dataseek/magtina350m-instruct`
• —Sibling datasets: see dataseek/ptbr-* for all nine corpora

License

cc0-1.0