cs336
Datasets
All datasets matching “cs336”owt-sampleThese files were created with the following script:
from datasets import load_dataset
from tqdm import tqdm
import io
dataset = load_dataset("Skylion007/openwebtext")['train']
split_dataset = dataset.train_test_split(train_size=2400000, test_size=60000, seed=0)
with io.open('data/owt_train.txt','w') as fopen:
listout = []
for data in tqdm(split_dataset['train']):
listout.append(data['text']+'<|endoftext|>')
if len(listout) > 1000:
_ =… See the full description on the dataset page: https://huggingface.co/datasets/stanford-cs336/owt-sample.cs336-basics-collection
CS336 Assignment 1 — Pre-tokenized Data & BPE Tokenizers
This repository contains preprocessing artifacts produced for Stanford CS336: Language Modeling from Scratch, Spring 2025 — Assignment 1: Basics.
It includes:
pre-tokenized TinyStories train/validation data,
pre-tokenized OpenWebText (OWT sample) train/validation data,
byte-level BPE vocabularies and merge tables for both datasets.
The main purpose of this repository is to avoid repeating the relatively expensive… See the full description on the dataset page: https://huggingface.co/datasets/victorhu493/cs336-basics-collection.cs336-owt-32k-bpe
CS336 OWT 32k BPE tokenized
Pre-tokenized OpenWebText sample (Stanford CS336 assignment 1 source) using a custom 32k byte-level BPE tokenizer.
Files
owt_vocab_32k.pkl — vocab dict[int, bytes] (32000 entries; ID 0 = <|endoftext|>, 1..256 = raw bytes, 257..31999 = merges)
owt_merges_32k.pkl — list[tuple[bytes, bytes]] in merge order
owt_train.npy — uint16 token IDs of 11 GB owt_train.txt (2,729,574,028 tokens)
owt_valid.npy — uint16 token IDs of 277 MB owt_valid.txt (66… See the full description on the dataset page: https://huggingface.co/datasets/AndrewK101/cs336-owt-32k-bpe.sft-cs336-assign5-datasets
SFT CS336 Assignment 5 Datasets
This repository contains all training and evaluation datasets for Supervised Fine-Tuning (SFT) experiments (based on CS-336 Assignment-5)
Repository Structure
Folder
Description
sft-instruct/
Instruction-tuning datasets
sft-reason/
Reasoning SFT datasets
SFT Instruction finetuning
Datasets for instruction-following SFT is based on UltraChat-200K + SafetyLlama.
File
Size
Description… See the full description on the dataset page: https://huggingface.co/datasets/garg-aayush/sft-cs336-assign5-datasets.Stanford_CS336_25Spring_Datacs336-a4