Datasetpublic

pt-sk/pretraining-dataset

Tokens are prepared using TikToken's "cl100k_base" tokenizer, which is used for GPT3.5 and GPT4.

sourceHugging Facemitupdated 2y agoView on Hugging Face

1likes104downloads

discussions and pull requests

Conversations for this repository live on Hugging Face.

CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.

Open discussions on Hugging Face