Datasetpublic

Aamod37/Tokenizer-Testset

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes196downloads

rootForegin_Language

../

fileAlbanian-00000-of-00001.parquet2.4 MBdownload

fileAmharic-00000-of-00001.parquet3.7 MBdownload

fileArabic-00000-of-00001.parquet3.9 MBdownload

fileArmenian-00000-of-00001.parquet3.3 MBdownload

fileAzerbaijani-00000-of-00001.parquet1.5 MBdownload

fileBasque-00000-of-00001.parquet1.3 MBdownload

fileBelarusian-00000-of-00001.parquet2.2 MBdownload

fileBerber-00000-of-00001.parquet52 KBdownload

fileBulgarian-00000-of-00001.parquet4.2 MBdownload

fileCatalan-00000-of-00001.parquet1.9 MBdownload

fileChinese-00000-of-00001.parquet1.6 MBdownload

fileCroatian-00000-of-00001.parquet1.5 MBdownload

fileCzech-00000-of-00001.parquet3.2 MBdownload

fileDanish-00000-of-00001.parquet3.0 MBdownload

fileDhivehi-00000-of-00001.parquet3.2 MBdownload

fileDutch-00000-of-00001.parquet2.4 MBdownload

fileDzongkha-00000-of-00001.parquet133 KBdownload

fileEnglish-00000-of-00001.parquet1.5 MBdownload

fileEstonian-00000-of-00001.parquet2.1 MBdownload

fileFilipino-00000-of-00001.parquet1.9 MBdownload

fileFinnish-00000-of-00001.parquet3.4 MBdownload

fileFrench-00000-of-00001.parquet2.9 MBdownload

fileGeorgian-00000-of-00001.parquet2.1 MBdownload

fileGerman-00000-of-00001.parquet3.3 MBdownload

fileGreek-00000-of-00001.parquet2.3 MBdownload

fileHaitian_Creole-00000-of-00001.parquet977 KBdownload

fileHausa-00000-of-00001.parquet1.3 MBdownload

fileHebrew-00000-of-00001.parquet6.7 MBdownload

fileHungarian-00000-of-00001.parquet2.4 MBdownload

fileIgbo-00000-of-00001.parquet1.3 MBdownload

fileIndonesian-00000-of-00001.parquet2.2 MBdownload

fileItalian-00000-of-00001.parquet3.0 MBdownload

fileJapanese-00000-of-00001.parquet4.6 MBdownload

fileKazakh-00000-of-00001.parquet4.7 MBdownload

fileKorean-00000-of-00001.parquet3.2 MBdownload

fileKyrgyz-00000-of-00001.parquet1.6 MBdownload

fileLatvian-00000-of-00001.parquet2.2 MBdownload

fileLithuanian-00000-of-00001.parquet1.9 MBdownload

fileMacedonian-00000-of-00001.parquet2.2 MBdownload

fileMalagasy-00000-of-00001.parquet692 KBdownload

fileMalay-00000-of-00001.parquet358 KBdownload

fileMath-00000-of-00001.parquet331 KBdownload

fileNorwegian-00000-of-00001.parquet4.0 MBdownload

fileNyanja-00000-of-00001.parquet1.2 MBdownload

filePersian-00000-of-00001.parquet3.4 MBdownload

filePolish-00000-of-00001.parquet3.0 MBdownload

filePortuguese-00000-of-00001.parquet2.4 MBdownload

fileQuechua-00000-of-00001.parquet367 KBdownload

fileRomanian-00000-of-00001.parquet1.7 MBdownload

fileRussian-00000-of-00001.parquet4.7 MBdownload

fileSerbian-00000-of-00001.parquet3.2 MBdownload

fileShona-00000-of-00001.parquet957 KBdownload

fileSinhala-00000-of-00001.parquet236 KBdownload

fileSlovak-00000-of-00001.parquet1.8 MBdownload

fileSlovenian-00000-of-00001.parquet2.3 MBdownload

fileSomali-00000-of-00001.parquet2.1 MBdownload

fileSpanish-00000-of-00001.parquet2.9 MBdownload

fileSwahili-00000-of-00001.parquet1.5 MBdownload

fileSwedish-00000-of-00001.parquet1.9 MBdownload

fileTagalog-00000-of-00001.parquet2.3 MBdownload

fileThai-00000-of-00001.parquet4.0 MBdownload

fileTibetan-00000-of-00001.parquet1.6 MBdownload

fileTurkish-00000-of-00001.parquet2.7 MBdownload

fileUkrainian-00000-of-00001.parquet4.0 MBdownload

fileUzbek-00000-of-00001.parquet1.3 MBdownload

fileVietnamese-00000-of-00001.parquet3.3 MBdownload

fileXhosa-00000-of-00001.parquet200 KBdownload

fileYoruba-00000-of-00001.parquet1.1 MBdownload

fileZulu-00000-of-00001.parquet140 KBdownload

Aamod37/Tokenizer-Testset · main · files are served by the source, never re-hosted here