jacepark12/LFM2.5-2.6B-Terminal-SFT

LFM2.5-2.6B Terminal SFT

Full-parameter supervised fine-tuning of LiquidAI/LFM2.5-2.6B for terminal-agent tasks.

The experiment reproduces the Phase 1 configuration in Liquid-CLI's train_unsloth_processed.py, changing only the base model and using native Transformers/TRL FlashAttention support for LFM2.5.

Training data

139,841 non-coding terminal-agent conversations prepared from the skill_based_easy, skill_based_medium, and skill_based_mixed configurations of nvidia/Nemotron-Terminal-Corpus using Liquid-CLI's prepare_data.py.

Training configuration

• —Full-parameter BF16 fine-tuning
• —Sequence length: 4,096
• —Packing: enabled
• —Assistant-only loss: enabled
• —Per-device batch size: 4
• —Gradient accumulation: 32 (effective batch size 128)
• —Epochs: 1
• —Learning rate: 2e-5
• —Optimizer: AdamW 8-bit
• —Warmup steps: 10
• —Weight decay: 0.01
• —Scheduler: linear
• —Seed: 3407
• —Attention: FlashAttention 2
• —Hardware: 1x NVIDIA H100 80GB

Training results

• —Optimizer steps: 1,092 / 1,092
• —Final epoch: 1.0
• —Training loss: 0.2867
• —Mean token accuracy: 0.8947
• —Tokens processed: 544,812,939
• —Training runtime: 34,353 seconds (about 9 hours 33 minutes)

Source

• —Experiment: https://github.com/gyunggyung/Liquid-CLI
• —Base model: https://huggingface.co/LiquidAI/LFM2.5-2.6B
• —Dataset: https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus