stukenov/sozkz-corpus-dedup-kk-web-v1
sozkz-corpus-dedup-kk-web-v1 Deduplicated Kazakh web text corpus collected from 6 public HuggingFace datasets. Contains only texts not present in kz-transformers/multidomain-kazakh-dataset (12.4M texts were used as the dedup reference). Stats Field Value Total unique texts 9,475,089 Format Parquet (142 shards) Columns text, source Dedup method MD5 hash (exact match) Dedup reference kz-transformers/multidomain-kazakh-dataset (12.4M hashes) Date… See the full description on the dataset page: https://huggingface.co/datasets/stukenov/sozkz-corpus-dedup-kk-web-v1.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face