Modelpublic

DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit

sourceHugging Facellama3.2updated 2y agoView on Hugging Face

2likes22downloads

Model Card

image/png

Eval

The fine tuned model (DevQuasar/analyticalreasoningr16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit) has gained performace over the base model (unsloth/Llama-3.2-3B-Instruct-bnb-4bit) in the following tasks.

Test	Base Model	Fine-Tuned Model	Performance Gain
leaderboardbbhlogicaldeductionseven_objects	0.2520	0.4360	0.1840
leaderboardbbhlogicaldeductionfive_objects	0.3560	0.4560	0.1000
leaderboardmusrteam_allocation	0.2200	0.3200	0.1000
leaderboardbbhdisambiguation_qa	0.3040	0.3760	0.0720
leaderboardgpqadiamond	0.2222	0.2727	0.0505
leaderboardbbhmovie_recommendation	0.5960	0.6360	0.0400
leaderboardbbhformal_fallacies	0.5080	0.5400	0.0320
leaderboardbbhtrackingshuffledobjectsthreeobjects	0.3160	0.3440	0.0280
leaderboardbbhcausal_judgement	0.5455	0.5668	0.0214
leaderboardbbhweboflies	0.4960	0.5160	0.0200
leaderboardmathgeometry_hard	0.0455	0.0606	0.0152
leaderboardmathnumtheoryhard	0.0519	0.0649	0.0130
leaderboardmusrmurder_mysteries	0.5280	0.5400	0.0120
leaderboardgpqaextended	0.2711	0.2802	0.0092
leaderboardbbhsports_understanding	0.5960	0.6040	0.0080
leaderboardmathintermediatealgebrahard	0.0107	0.0143	0.0036

Framework versions

—unsloth 2024.11.5
—trl 0.12.0

Training HW

—V100

I'm doing this to 'Make knowledge free for everyone', using my personal time and resources.

If you want to support my efforts please visit my ko-fi page: https://ko-fi.com/devquasar

Also feel free to visit my website https://devquasar.com/