ce-lery/corpus-ja-11b
corpus-ja-11b Welcome to this repository.This dataset is japanese corpus that includes wiki, wikibooks, wikiversity, and cc100. Getting Started If you want to use this, please run as follows.This process takes about 3 hours. mkdir -p pretrain/input/ cd pretrain/input/ GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/ce-lery/corpus-ja-11b.git cd corpus-ja-11b git lfs pull bash merge_train.sh
015
This repository belongs to ce-lery on Hugging Face.
CoolFace never edits a repository it does not host. Visibility, licence, collaborators and gating are all managed at the source.
corpus-ja-11b
public
not set
no
ce-lery