CMSManhattan/JiRackNative_3b

JiRack Robotics - TernaryTransformer3B (Pre-training Phase)

JiRack Robotics has officially kicked off the multi-shard pre-training phase for its latest 3B parameter robotics model.

Running under the JiRackTrain pipeline on the enterprise infrastructure cluster (root@jirack1), the training initializes the next-generation TernaryTransformer3B architecture, tightly coupled with the advanced JiRack Pro Tokenizer.

Model Details

• —Model Name: TernaryTransformer3B (3 Billion Parameters)
• —Architecture: TernaryTransformer (custom ternary bit-response logic)
• —Tokenizer: JiRack Pro Tokenizer (128K) — 347 active language editions of Wikipedia + specialized robotic action tokens
• —Pre-training Dataset: JiRack-Pretrain-Dataset

Key Metrics (Step ~838/1000 on Shard 7) on Blackwell 96 Gb VRAM

• —Iteration Speed: 2.36s/it (network tensor loading)
• —Current Loss: 1.2665
• —Moving Average Loss: 1.6637
• —Perplexity (PPL): ppl=5.3

[Shard 8/100] jirack_pretrain_chunk_7.pt
Training jirack_pretrain_chunk_7.pt:  83%|█████▊ | 208/250 [08:09<01:38,  2.36s/it, loss=1.2665, avg_loss=1.6637, ppl=5.3, lr=2.00e-04]
[0] 0:python3*                                                                                  "mc [root@809e0ca1a59e" 19:45 04-Jul-26

Engineers anticipate a steep drop in the perplexity curve within the first 250 iterations of Shard 1 as the ternary weights align with the tokenizer's token distributions.

Intended Use

This foundational model is being developed specifically for robotics applications:

• —Real-time control policies
• —Multimodal reasoning (text + vision + action tokens)
• —Edge deployment with ternary efficiency
• —Low-latency physical interaction loops

Monitoring & Updates

Training is actively progressing across all 7 shards. Follow this repository or the linked tokenizer/dataset cards for checkpoint releases, evaluation results, and fine-tuned robotic variants.

Stay tuned — the ternary robotics revolution is just getting started! 🚀