swiss-ai/apertus-pretrain-swiss
Swiss Pretrain Data This dataset provides a large collection of open-access and license-compliant Swiss data sources for language model training. The dataset includes the following sources: Name Internal ID Tokens (B) Description Curia Vista curiavista 0.5 Legal and administrative documents from the Swiss database of parliamentary proceedings. enscheidsuche enscheidsuche_html 4.5 Swiss court decisions, sampled at 50% for balance. FineWeb-2… See the full description on the dataset page: https://huggingface.co/datasets/swiss-ai/apertus-pretrain-swiss.
Swiss Pretrain Data
This dataset provides a large collection of open-access and license-compliant Swiss data sources for language model training.
The dataset includes the following sources: | Name | Internal ID | Tokens (B) | Description | |------|-------------|------------|-------------| | Curia Vista | curiavista | 0.5 | Legal and administrative documents from the Swiss database of parliamentary proceedings. | | enscheidsuche | enscheidsuche_html | 4.5 | Swiss court decisions, sampled at 50% for balance. | | FineWeb-2 | fineweb-2-swissfilter-quality_10-filterrobots | 1.8 | A Swiss-only subset of FineWeb-2, filtered for top 10% quality. | | Romansh Data | roh_data |0.1 | Various Romansh-language data sources, as detailed in the dataset documentation. |
All sampling, filtering and data-preparation scripts can be found in our dedicated GitHub repository.
Feel free to reach out for any questions or suggestions 😊