violetxi/qwen35-9b-harvey-v4-notes-ntp-100m

violetxi/qwen35-9b-harvey-v4-notes-ntp-100m

Full Qwen3.5-9B model from the September 22, 2026 Harvey notes-only training runs. This revision is epoch 2, step 1550 of a two-epoch run, job 1016041. The final checkpoint is on main and final; epoch 1 is on epoch1. Each saved checkpoint is also available by its checkpoint-<step> reference.

Training

• —Base: Qwen/Qwen3.5-9B, revision c202236235762e1c871ad0ccb60c8ee5ba337b9a.
• —Dataset: `violetxi/harvey-notes-v4`, revision 3540adb17977fc860e6aba1a9ccea069e9dda2a6.
• —Supervised text: training_text followed by two newline characters, with no chat template.
• —100,000,076 note labels/epoch before causal shift; 99,993,882 scored labels/epoch after shift.
• —223,681 notes, packed into 6,200 rows of 16,384 tokens with seed 731.
• —199,987,764 scored supervised labels seen through this checkpoint.
• —No supervised trajectory examples. These are the notes-only controls, separate from notes + trajectory mixture models.
• —Eight GH200 GPUs, effective batch 8, learning rate 5e-6, cosine schedule, 3% warmup, training seed 0.
• —W&B training metrics.

The objective is pure next-token prediction on notes; there is no KL term.

Validation

Held-out notes: 520 documents and 250,060 scored labels. At this checkpoint, notes NLL is 0.599234563 and perplexity is 1.820724616. Full diagnostic history through this checkpoint is in evaluation_summary.json. No downstream Harvey task-accuracy result or fresh export inference is claimed.

Loading

from transformers import AutoTokenizer, AutoModelForImageTextToText
model_id = "violetxi/qwen35-9b-harvey-v4-notes-ntp-100m"
tokenizer = AutoTokenizer.from_pretrained(model_id, revision="main")
model = AutoModelForImageTextToText.from_pretrained(
    model_id, revision="main", dtype="bfloat16", device_map="auto", use_safetensors=True,
)

This is a complete composite safetensors model with tokenizer, chat template, processors and generation settings. All 427 trained FP32 text tensors are cast to the pinned base BF16 dtype; 348 auxiliary/base tensors retain their original values and dtypes. Every exported tensor was checked against its source and for finiteness, and all expected Transformers tensor shapes were verified on CPU. Vision and auxiliary components were inherited and have not been evaluated here. This is a model-only checkpoint; exact optimizer/scheduler resume is unsupported.