Helsinki-NLP/opus_paracrawl
Dataset Card for OpusParaCrawl Dataset Summary Parallel corpora from Web Crawls collected in the ParaCrawl project. Tha dataset contains: 42 languages, 43 bitexts total number of files: 59,996 total number of tokens: 56.11G total number of sentence fragments: 3.13G To load a language pair which isn't part of the config, all you need to do is specify the language code as pairs, e.g. dataset = load_dataset("opus_paracrawl", lang1="en", lang2="so") You can find… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/opus_paracrawl.
Convert dataset to Parquet (#2)
Delete legacy JSON metadata (#1)
rename configs to config_name
add dataset_info in dataset metadata
remove dummmy data
Update version of opus_paracrawl dataset (#4816)
Fix loading example in opus dataset cards (#4813)
Align more metadata with other repo types (models,spaces) (#4607)
Remove config names as yaml keys (#4367)
Update datasets task tags to align tags with models (#4067)
Update files from the datasets library (from 1.18.0)
Update files from the datasets library (from 1.7.0)
Update files from the datasets library (from 1.6.0)
Update files from the datasets library (from 1.3.0)
Update files from the datasets library (from 1.2.0)