ganscs/Olmo7b-olmo-a100-new-20260908-CPT-merged-step-750

Olmo7b-olmo-a100-new-20260908-CPT-merged-step-750

Standalone BF16 model combining allenai/Olmo-3-1025-7B with the trained LoRA adapter and full embedding/output weights from checkpoint 750. Load this repository directly with Transformers; a separate PEFT adapter is not required.

Provenance

• —Base revision: a81bae42db3975be1671e27b9c9a56da1a9f980f.
• —Adapter repository revision: edba4e91735a37b3c886961e707355f3e541ef61.
• —Selected step: 750, the best validation checkpoint in this training run.
• —Source checkpoint training-time validation loss: 1.3001196.
• —rsLoRA rank 128, alpha 64.
• —The fully trained embed_tokens and lm_head weights replace the original endpoints.
• —The output uses the checkpoint tokenizer and the original base architecture/configuration.

Adapters were trained with a 4-bit base. This export merges them into the original BF16 base revision using PEFT's safe merge operation. The training-time validation score above is not a new evaluation of this BF16 export.

Validation

Every adapter tensor was consumed; all output tensors are finite and match the Transformers architecture's names and shapes. The model loaded locally without PEFT adapters and produced finite logits and a short greedy generation. merge_manifest.json records source hashes, merge details and output checksums; validation.json records the smoke test. training_manifest.json preserves the source experiment settings. This is a weights-only export, without optimizer state.

Load

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ganscs/Olmo7b-olmo-a100-new-20260908-CPT-merged-step-750"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)