inarikami
wikipedia-japanese
Japanese Wikipedia Dataset
This dataset is a comprehensive pull of all Japanese wikipedia article data as of 20220808.
Note: Right now its uploaded as a single cleaned gzip file (for faster usage), I'll update this in the future to include a huggingface datasets compatible class and better support for japanese than the existing wikipedia repo.
Example use case:
gunzip jawwiki20200808.json.gz
import pandas as pd
from datasets import load_dataset
df =… See the full description on the dataset page: https://huggingface.co/datasets/inarikami/wikipedia-japanese.RF-SamplesJACAR