Jack04810/spac-qwen3-4b-spatial-pe-8ep

output14bphase28ep0.008pe — SPAC Qwen3-14B Spatial-PE (merged fp32)

Qwen3-14B with the SPAC extended vocabulary (200,824 tokens: base BPE + spatial tokens), full fine-tuned for 3D mesh understanding & generation. Weights merged from the DeepSpeed ZeRO-3 checkpoint-18892 of output_14b_phase2_8ep_0.008pe into *13 fp32 `pytorch_model-.bin shards** (pytorch_model.bin.index.json`), i.e. the same layout as the previous release.

Relation to previous releases

• —Phase-2 run output_14b_phase2 (step 34268, spatial-PE scale 0.08) — the former root release.
• —This release: continuation from output_14b_phase2_full trained with spatial-PE scale 0.008, LR 1e-5 -> 1e-6 (cosine w/ min-LR, warmup 0.5%), 18,892 steps on bpe_160k_mix (160k samples) + ultrachat_8k (8k conversations), 8x A800, ZeRO-3 bf16, FA2, packing 8192.
• —Trainer-reported results: train loss 0.3348; eval loss 0.7694, eval accuracy 0.8356.

Spatial-PE parameters (spatialpemanifest_14b.json, this repo)

• —mode: absolute_anchor_plus_footprint_ipe (npe: false)
• —coordinate grid: [0, 31] (32 levels); coord_scale 32.0
• —base mesh vocab size: 8192; hidden size: 5120
• —training spatial-PE scale (spatial_pe_scale): 0.008 (manifest field spatial_scale: 0.08)
• —spatial tokens: <mesh_start> id 151669, <mesh_end> id 151670, mesh tokens 151672..168055 (16,384), morton tokens 168056..200823 (32,768); total vocab 200,824
• —merge table sha256: fb10dbe1f4b582a5c568169a167ccb98a3b49fd309c0a5cc386b17fc775705f8
• —requires the SPAC spatial-PE inference wrapper + hf_main_merge_table.json (this repo) to run with the same positional encoding used in training

Inference notes

• —Chat template: qwen3_nothink (no thinking). Mesh sequences tokenize as <mesh_start><morton_*><mesh_*> (see added_tokens.json).