Modelpublic

TAUR-dev/M-sft_exp_zayneV3_cd3arg_w_gpt4o_both-sft

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes7downloads

Model Card

M-sftexpzayneV3cd3argwgpt4oboth-sft

This model was created as part of the sft_exp_zayneV3_cd3arg_w_gpt4o_both experiment using the SkillFactory experiment management system.

Model Details

—Training Method: LLaMAFactory SFT (Supervised Fine-Tuning)
—Stage Name: sft
—Experiment: sftexpzayneV3cd3argwgpt4oboth

Training Configuration

{"modelnameorpath": "Qwen/Qwen2.5-1.5B-Instruct", "trustremotecode": true, "stage": "sft", "dotrain": true, "finetuningtype": "full", "deepspeed": "/scratch/10416/zaynesprague/skillfactorydir/skill-factory/thirdparty/LLaMA-Factory/examples/deepspeed/dsz2config.json", "dataset": "TAURdev_DSFTCsftexpzayneV3cd3argwgpt4obothsftdata_sfttrain", "template": "qwen", "cutofflen": 16384, "maxsamples": 1000000, "overwritecache": true, "preprocessingnumworkers": 1, "dataloadernumworkers": 0, "disabletqdm": false, "outputdir": "/scratch/10416/zaynesprague/skillinjectoutputs/sfexperiments/newsftv3911_zaynev3wgpt4oboth/llamafactory/checkpoints", "loggingsteps": 10, "savesteps": 100000, "plotloss": true, "overwriteoutputdir": true, "perdevicetrainbatchsize": 1, "gradientaccumulationsteps": 1, "learningrate": 1e-06, "numtrainepochs": 1, "lrschedulertype": "cosine", "warmupratio": 0.05, "weightdecay": 0.0001, "adambeta1": 0.9, "adambeta2": 0.95, "bf16": true, "ddptimeout": 180000000, "gradientcheckpointing": true, "saveonlymodel": true, "enablemaskedranges": false, "savestrategy": "steps", "savetotallimit": 5, "sftrackerdatasetid": "TAUR-dev/D-ExpTracker_sftexpzayneV3cd3argwgpt4obothv1", "sfevalbeforetraining": false, "sfwandbproject": "sftexpzayneV3cd3argwgpt4obothsft", "sfevalsteps": null, "runname": "sftexpzayneV3cd3argwgpt4oboth_sft"}

Experiment Tracking

🔗 View complete experiment details: Experiment Tracker Dataset

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TAUR-dev/M-sft_exp_zayneV3_cd3arg_w_gpt4o_both-sft")
model = AutoModelForCausalLM.from_pretrained("TAUR-dev/M-sft_exp_zayneV3_cd3arg_w_gpt4o_both-sft")