Datasetpublic

PotatoHD/ru-text-corpus

Description 798k deduplicated Russian documents (1.6B tokens) from FineWeb-2 (rus_Cyrl), ru-StackOverflow, Pikabu, Habr, Russian Wikipedia and news. Markdown- and code-bearing (StackOverflow/Habr/Pikabu are text_markdown). Filtered to >=200 chars and >=30% Cyrillic. Columns: text, src. Derived dataset. Source material retains its original per-item licence (see source/repo columns); treat as other / mixed. Provided as-is. Usage from datasets import load_dataset… See the full description on the dataset page: https://huggingface.co/datasets/PotatoHD/ru-text-corpus.

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes390downloads

Dataset Card

Description

~798k deduplicated Russian documents (~1.6B tokens) from FineWeb-2 (rusCyrl), ru-StackOverflow, Pikabu, Habr, Russian Wikipedia and news. Markdown- and code-bearing (StackOverflow/Habr/Pikabu are `textmarkdown). Filtered to >=200 chars and >=30% Cyrillic. Columns: text, src`.

Derived dataset. Source material retains its original per-item licence (see source/repo columns); treat as other / mixed. Provided as-is.

Usage

python

from datasets import load_dataset
ds = load_dataset("PotatoHD/ru-text-corpus")