open-sci/open-sci-ref-v0.02-1.7b-nemotron-hq-1T-4096-rope_theta-100k

open-sci-ref-v0.02-1.7b-nemotron-hq-1T-4096-rope_theta-100k

1.7B open-sci-ref model trained on Nemotron-CC HQ for 1T tokens (sequence length 4096, RoPE theta = 100000).

The main branch holds the final checkpoint (iter 238419). Intermediate checkpoints (iters 58000-238000, every 2000) are available as branches named iter_XXXXXXX.

Evaluation

Final checkpoint on the open-sci-0.01 suite (lm-eval-harness). Metrics collected with oellm collect-results.