ce-lery/corpus-ja-11b
corpus-ja-11b Welcome to this repository.This dataset is japanese corpus that includes wiki, wikibooks, wikiversity, and cc100. Getting Started If you want to use this, please run as follows.This process takes about 3 hours. mkdir -p pretrain/input/ cd pretrain/input/ GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/ce-lery/corpus-ja-11b.git cd corpus-ja-11b git lfs pull bash merge_train.sh
015
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face