Modelpublic

DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit_adapter

sourceHugging Facellama3.2updated 2y agoView on Hugging Face

1likes

Model Card

'Make knowledge free for everyone'

image/png

Eval

The fine tuned model (DevQuasar/analyticalreasoningr16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit) has gained performace over the base model (unsloth/Llama-3.2-3B-Instruct-bnb-4bit) in the following tasks.

Test	Base Model	Fine-Tuned Model	Performance Gain
leaderboardbbhlogicaldeductionseven_objects	0.2520	0.4360	0.1840
leaderboardbbhlogicaldeductionfive_objects	0.3560	0.4560	0.1000
leaderboardmusrteam_allocation	0.2200	0.3200	0.1000
leaderboardbbhdisambiguation_qa	0.3040	0.3760	0.0720
leaderboardgpqadiamond	0.2222	0.2727	0.0505
leaderboardbbhmovie_recommendation	0.5960	0.6360	0.0400
leaderboardbbhformal_fallacies	0.5080	0.5400	0.0320
leaderboardbbhtrackingshuffledobjectsthreeobjects	0.3160	0.3440	0.0280
leaderboardbbhcausal_judgement	0.5455	0.5668	0.0214
leaderboardbbhweboflies	0.4960	0.5160	0.0200
leaderboardmathgeometry_hard	0.0455	0.0606	0.0152
leaderboardmathnumtheoryhard	0.0519	0.0649	0.0130
leaderboardmusrmurder_mysteries	0.5280	0.5400	0.0120
leaderboardgpqaextended	0.2711	0.2802	0.0092
leaderboardbbhsports_understanding	0.5960	0.6040	0.0080
leaderboardmathintermediatealgebrahard	0.0107	0.0143	0.0036

Framework versions

—unsloth 2024.11.5
—trl 0.12.0

Training HW

—V100