wordpiece
the_pile_WordPiecex32768_2efdb9d060d1ae95faf952ec1a50f020
Dataset Card for "the_pile_WordPiecex32768_2efdb9d060d1ae95faf952ec1a50f020"
Dataset Summary
This is a preprocessed, tokenized dataset for the cramming-project.
Use only with the tokenizer uploaded here.
This version is 2efdb9d060d1ae95faf952ec1a50f020, which corresponds to a specific dataset construction setup, described below.
The raw data source is the Pile, a 825 GiB diverse, open source language modelling data set that consists of 22 smaller, high-quality
datasets… See the full description on the dataset page: https://huggingface.co/datasets/JonasGeiping/the_pile_WordPiecex32768_2efdb9d060d1ae95faf952ec1a50f020.the_pile_WordPiecex32768_97b8e776baafb99c3892e6572a9f51b3
Dataset Card for the_pile_WordPiecex32768_97b8e776baafb99c3892e6572a9f51b3
This is a preprocessed, tokenized dataset for the cramming-project.
Use only with the tokenizer uploaded here.
This version is 97b8e776baafb99c3892e6572a9f51b3, which corresponds to a specific dataset construction setup, described below.
The raw data source is the Pile, a 825 GiB diverse, open source language modelling data set that consists of 22 smaller, high-quality
datasets combined together.… See the full description on the dataset page: https://huggingface.co/datasets/JonasGeiping/the_pile_WordPiecex32768_97b8e776baafb99c3892e6572a9f51b3.the_pile_WordPiecex32768_8eb2d0ea9da707676c81314c4ea04507
Dataset Card for "the_pile_WordPiecex32768_8eb2d0ea9da707676c81314c4ea04507"
Dataset Summary
This is a preprocessed, tokenized dataset for the cramming-project.
Use only with the tokenizer uploaded here.
This version is 8eb2d0ea9da707676c81314c4ea04507, which corresponds to a specific dataset construction setup, described below.
The raw data source is the Pile, a 825 GiB diverse, open source language modelling data set that consists of 22 smaller, high-quality
datasets… See the full description on the dataset page: https://huggingface.co/datasets/JonasGeiping/the_pile_WordPiecex32768_8eb2d0ea9da707676c81314c4ea04507.the_pile_WordPiecex32768_e9f3c90fb38fb46185ad86ed3b69b9d5the_pile_WordPiecex32768_53b28db05413b6497e702f178268e1e2tolm-nexus-wordpiece-geometry-priors