Modelpublic

SkillFactory/ablation-Qwen2.5-1.5B-Instruct-no_prompt_diversity-SFT

sourceHugging Faceupdated 10mo agoView on Hugging Face

0likes6downloads

Model Card

M-RC-absftourstructuresingle_sample-sft

This model was created as part of the RC-ab_sft_our_structure_single_sample experiment using the SkillFactory experiment management system.

Model Details

—Training Method: LLaMAFactory SFT (Supervised Fine-Tuning)
—Stage Name: sft
—Experiment: RC-absftourstructuresingle_sample

Training Configuration

{"modelnameorpath": "Qwen/Qwen2.5-1.5B-Instruct", "trustremotecode": true, "stage": "sft", "dotrain": true, "finetuningtype": "full", "deepspeed": "/scratch/10416/zaynesprague/skillfactorydir/skill-factory/thirdparty/LLaMA-Factory/examples/deepspeed/dsz2config.json", "dataset": "TAURdev_DSFTCRCabsftourstructuresinglesamplesftdata_sfttrain", "template": "qwen", "cutofflen": 16384, "maxsamples": 1000000, "overwritecache": true, "preprocessingnumworkers": 1, "dataloadernumworkers": 0, "disabletqdm": false, "outputdir": "/scratch/10416/zaynesprague/skillinjectoutputs/sfexperiments/RCabsftbonallsamples/llamafactory/checkpoints", "loggingsteps": 10, "savesteps": 100000, "plotloss": true, "overwriteoutputdir": true, "perdevicetrainbatchsize": 1, "gradientaccumulationsteps": 1, "learningrate": 1e-06, "numtrainepochs": 1, "lrschedulertype": "cosine", "warmupratio": 0.05, "weightdecay": 0.0001, "adambeta1": 0.9, "adambeta2": 0.95, "bf16": true, "ddptimeout": 180000000, "gradientcheckpointing": true, "saveonlymodel": true, "enablemaskedranges": false, "savestrategy": "steps", "savetotallimit": 5, "sftrackerdatasetid": "TAUR-dev/D-ExpTrackerRC-absftourstructuresinglesample_v1", "sfevalbeforetraining": false, "sfwandbproject": "RC-absftourstructuresinglesamplesft", "sfevalsteps": null, "runname": "RC-absftourstructuresinglesample_sft"}

Experiment Tracking

🔗 View complete experiment details: Experiment Tracker Dataset

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TAUR-dev/M-RC-ab_sft_our_structure_single_sample-sft")
model = AutoModelForCausalLM.from_pretrained("TAUR-dev/M-RC-ab_sft_our_structure_single_sample-sft")