togethercomputer/RedPajama-Data-V2

Add paper citation (#30)

Update README.md

rpv2-subsamples (#26)

Update README.md

Merge branch 'deduplicate_flag'

add list of missing urls

catch broken zips

fix missing urls

Revert "add dedupe flag"

add dedupe flag

fix writing

add blacklisted domain id->categories mapping

update README.md

add raw doc+token counts

update README.md

update README.md

update README.md

change config order

update table

add acknowledgments, citation, blog url

add default config name

remove _sample

Update README.md

Merge branch 'main' of https://huggingface.co/Datasets/togethercomputer/RedPajama-Data-V2

add snapshot ids

Update README.md

add minhash to table

add snippet, fix citation

remove snapshot file

add listings

fix README.md

fix README.md

todo: acknowledgements

update README.md

update README.md

update README.md

update README.md

update README.md

update README.md

update README.md

update README.md

simplify interface

bugfix

add snapshot ids

Merge branch 'main' of https://huggingface.co/Datasets/togethercomputer/RedPajama-Data-V2

Revert "add snapshot ids"

add snapshot ids

Update license information, fix typos

fix listings

fix listings