EleutherAI/wikitext_document_level
Wikitext Document Level This is a modified version of https://huggingface.co/datasets/wikitext that returns Wiki pages instead of Wiki text line-by-line. The original readme is contained below. Dataset Card for "wikitext" Dataset Summary The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/wikitext_document_level.
Rename wikitext_document_level.py to wikitext_document_level.py.backup
Update README.md
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-test.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-validation.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-train.parquet with huggingface_hub
Upload wikitext-103-raw-v1/wikitext-103-raw-v1-test.parquet with huggingface_hub
Upload wikitext-103-raw-v1/wikitext-103-raw-v1-validation.parquet with huggingface_hub
Upload wikitext-103-raw-v1/wikitext-103-raw-v1-train.parquet with huggingface_hub
Upload wikitext-2-v1/wikitext-2-v1-test.parquet with huggingface_hub
Upload wikitext-2-v1/wikitext-2-v1-validation.parquet with huggingface_hub
Upload wikitext-2-v1/wikitext-2-v1-train.parquet with huggingface_hub
Upload wikitext-103-v1/wikitext-103-v1-test.parquet with huggingface_hub
Upload wikitext-103-v1/wikitext-103-v1-validation.parquet with huggingface_hub
Upload wikitext-103-v1/wikitext-103-v1-train.parquet with huggingface_hub
starting over
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-test.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-validation.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-train.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-test.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-validation.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-train.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-test.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-validation.parquet with huggingface_hub
Upload wikitext-2-raw-v1/wikitext-2-raw-v1-train.parquet with huggingface_hub
Update wikitext_document_level.py
remove data attributes - these are for line-level wt
use original wikitext reamde + note on document level
Rename wikitext.py to wikitext_document_level.py
innit commit
initial commit