sbordt/OLMo-2-2.7B-Exp-NoiseVectors

OLMo-2-2.7B-Exp Noise Vectors Gaussian noise vectors added to the input embeddings during pretraining of sbordt/OLMo-2-2.7B-Exp (a 2.7B-parameter OLMo-2-style model with d_model=2880). Released as a uniform-random 1% subsample per every-1000-batch chunk from 51,200 poisoned pretraining batches over 100,000 training steps — 480 rows total. How the noise was applied during training For each poisoned batch, Gaussian noise of shape (4096, 2880) was drawn and added to… See the full description on the dataset page: https://huggingface.co/datasets/sbordt/OLMo-2-2.7B-Exp-NoiseVectors.