Modelpublic

zhendrysiak/Bielik-1.5B-v3.0-Instruct-random-1.5-finetune-r-8-alpha-16-02022025_hq

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes16downloads

Model Card

Bielik-1.5B-v3.0-Instruct-random-1.5-finetune-r-8-alpha-16-02022025_hq

This model is a fine-tuned version of speakleash/Bielik-1.5B-v3.0-Instruct on the None dataset. It achieves the following results on the evaluation set:

—Loss: 5.8342

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 2.5e-05
—trainbatchsize: 2
—evalbatchsize: 2
—seed: 42
—optimizer: Use OptimizerNames.PAGEDADAMW8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
—lrschedulertype: linear
—lrschedulerwarmup_steps: 2
—num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0182	100	12.5344
No log	0.0364	200	10.1395
No log	0.0545	300	7.1765
No log	0.0727	400	6.1187
9.0468	0.0909	500	6.0085
9.0468	0.1091	600	5.9934
9.0468	0.1273	700	5.9969
9.0468	0.1455	800	5.9900
9.0468	0.1636	900	5.9763
5.7454	0.1818	1000	5.9654
5.7454	0.2	1100	5.9596
5.7454	0.2182	1200	5.9496
5.7454	0.2364	1300	5.9282
5.7454	0.2545	1400	5.9250
5.9191	0.2727	1500	5.9241
5.9191	0.2909	1600	5.9205
5.9191	0.3091	1700	5.9132
5.9191	0.3273	1800	5.9086
5.9191	0.3455	1900	5.9050
5.5413	0.3636	2000	5.9009
5.5413	0.3818	2100	5.8962
5.5413	0.4	2200	5.8965
5.5413	0.4182	2300	5.8930
5.5413	0.4364	2400	5.8916
5.6269	0.4545	2500	5.8862
5.6269	0.4727	2600	5.8860
5.6269	0.4909	2700	5.8825
5.6269	0.5091	2800	5.8797
5.6269	0.5273	2900	5.8796
5.4303	0.5455	3000	5.8805
5.4303	0.5636	3100	5.8684
5.4303	0.5818	3200	5.8646
5.4303	0.6	3300	5.8629
5.4303	0.6182	3400	5.8611
5.9092	0.6364	3500	5.8607
5.9092	0.6545	3600	5.8514
5.9092	0.6727	3700	5.8532
5.9092	0.6909	3800	5.8531
5.9092	0.7091	3900	5.8514
5.6308	0.7273	4000	5.8498
5.6308	0.7455	4100	5.8473
5.6308	0.7636	4200	5.8457
5.6308	0.7818	4300	5.8452
5.6308	0.8	4400	5.8424
5.7489	0.8182	4500	5.8399
5.7489	0.8364	4600	5.8385
5.7489	0.8545	4700	5.8377
5.7489	0.8727	4800	5.8370
5.7489	0.8909	4900	5.8357
5.7264	0.9091	5000	5.8350
5.7264	0.9273	5100	5.8351
5.7264	0.9455	5200	5.8352
5.7264	0.9636	5300	5.8343
5.7264	0.9818	5400	5.8343
5.6711	1.0	5500	5.8342

Framework versions

—PEFT 0.17.1
—Transformers 4.56.2
—Pytorch 2.8.0+cu126
—Datasets 4.0.0
—Tokenizers 0.22.1