ReactiveAI/Beta-Pre-Train-Corpus
Reactive AI / Beta Pre-Train Corpus Pre-training corpus for RxT-Beta models, created from public & open datasets. Includes high-quality english and polish web crawl data, mathematic and scientific subsets, and code in different programming languages. 2k subsets are filtered for 1024-2048 tokens, except MegaMath Web Pro and GitHub Code subsets, that were filtered for 512-2048 tokens Subsets & original datasets FineWeb-Edu fineweb-edu-s100 (51.3M examples) - 50% of… See the full description on the dataset page: https://huggingface.co/datasets/ReactiveAI/Beta-Pre-Train-Corpus.
021k
../
train-00000-of-00025.parquetdownload
train-00001-of-00025.parquetdownload
train-00002-of-00025.parquetdownload
train-00003-of-00025.parquetdownload
train-00004-of-00025.parquetdownload
train-00005-of-00025.parquetdownload
train-00006-of-00025.parquetdownload
train-00007-of-00025.parquetdownload
train-00008-of-00025.parquetdownload
train-00009-of-00025.parquetdownload
train-00010-of-00025.parquetdownload
train-00011-of-00025.parquetdownload
train-00012-of-00025.parquetdownload
train-00013-of-00025.parquetdownload
train-00014-of-00025.parquetdownload
train-00015-of-00025.parquetdownload
train-00016-of-00025.parquetdownload
train-00017-of-00025.parquetdownload
train-00018-of-00025.parquetdownload
train-00019-of-00025.parquetdownload
train-00020-of-00025.parquetdownload
train-00021-of-00025.parquetdownload
train-00022-of-00025.parquetdownload
train-00023-of-00025.parquetdownload
train-00024-of-00025.parquetdownload