deshanksuman/finetuned-Qwen2.5-3B-Instruct-WSD-Advanced-reasoning

Uploaded model

• —Developed by: deshanksuman
• —License: apache-2.0
• —Finetuned from model : Qwen/Qwen2.5-3B-Instruct

Dataset

Fews Training data arranged in the format of Instruction, Input and output deshanksuman/ReasoningWSDdataset

Hyperparameter for Training

• —perdevicetrainbatchsize=4,
• —gradientaccumulationsteps=8,
• —warmup_steps=50,
• —numtrainepochs=2,
• —learning_rate=2e-4,
• —fp16=not torch.cuda.isbf16supported(),
• —bf16=torch.cuda.isbf16supported(),
• —logging_steps=10,
• —optim="adamw_torch",
• —weight_decay=0.01,
• —lrschedulertype="linear",
• —seed=3407

This is developed by Deshan Sumanathilaka https://sumanathilaka.github.io

Acknowledgement

We acknowledge the support of the Supercomputing Wales project, which is part-funded by the European Regional Development Fund (ERDF) via Welsh Government.