CoolFace
Modelpublic

osieosie/tmax-qwen3-8b-tmax-sft-full-20260513

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes22downloads
Model Card

sftqwen38btmaxfull05132node

Supervised fine-tune of [hamishivi/Qwen3-8B](https://huggingface.co/hamishivi/Qwen3-8B) on osieosie/tmax-sft-full-20260513-bad-tool-call-filtered, mixing all 8 subsets at weight 1.0 each.

Training data

  • —\allenai__Sera_4.6_Lite_47000\ (mixer weight \1.0\)
  • —\m_a_p__TerminalTraj\ (mixer weight \1.0\)
  • —\nvidia__Nemotron_Terminal_Corpus__dataset_adapters\ (mixer weight \1.0\)
  • —\nvidia__Nemotron_Terminal_Corpus__skill_based_easy\ (mixer weight \1.0\)
  • —\nvidia__Nemotron_Terminal_Corpus__skill_based_medium\ (mixer weight \1.0\)
  • —\nvidia__Nemotron_Terminal_Corpus__skill_based_mixed\ (mixer weight \1.0\)
  • —\open_thoughts__OpenThoughts_Agent_v1_SFT\ (mixer weight \1.0\)
  • —\skill_tax_20260505_2.2k_combined_balanced_thinking_all\ (mixer weight \1.0\)
  • —Split: train
  • —Transforms: sft_tulu_tokenize_and_truncate_v1, sft_tulu_filter_v1
  • —Dataset cache hash: 8cd5377c34

Hyperparameters

FieldValue
Base model`hamishivi/Qwen3-8B`
Local model path/gpfs/scrubbed/osey/tmax/models/hamishivi/Qwen3-8B
Max seq length32768
Sequence-parallel size1
Per-device batch size2
Gradient accumulation4
Effective batch size128
Packingtrue
Learning rate1e-5
LR schedulerlinear
Warmup ratio0.1
Epochs2
Checkpoint every250 steps
Seed123
Optimizer kernelLiger
Mixed precisionbf16
Gradient checkpointingtrue
DeepSpeed configstage2_no_offloading_accelerate.conf

Compute

  • —Nodes: 2
  • —GPUs per node: 8 (total 16)
  • —Slurm job: 117347 on g[007,021]
  • —Run timestamp: 20260516_120450

Tracking

  • —W&B project: tmax-full-sft (run name: sft_qwen3_8b_tmax_full_0513_2node)

Training command

bash
sbatch scripts/slurm/sft/qwen/sft_qwen3_8b_tmax_full_0513_2node.sh