Modelpublic

syarief-mulyadi/Llasa-edit-IWSE_QLoRA_Attn-Only

sourceHugging Facecc-by-nc-4.0updated 7mo agoView on Hugging Face

0likes8downloads

Model Card

Llasa-edit-IWSEQLoRAAttn-Only

This model is a fine-tuned version of HKUSTAudio/Llasa-1B on an unknown dataset. It achieves the following results on the evaluation set:

—Loss: 2.8392

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.0001
—trainbatchsize: 4
—evalbatchsize: 4
—seed: 42
—distributed_type: multi-GPU
—num_devices: 2
—gradientaccumulationsteps: 4
—totaltrainbatch_size: 32
—totalevalbatch_size: 8
—optimizer: Use OptimizerNames.ADAMW8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—training_steps: 1000
—mixedprecisiontraining: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
5.9720	7.1538	50	6.1841
4.6719	14.3077	100	5.7373
3.1021	21.4615	150	5.1832
1.9793	28.6154	200	4.6429
1.2189	35.7692	250	4.1411
0.8101	42.9231	300	3.8109
0.5745	50.0	350	3.5193
0.4165	57.1538	400	3.3177
0.3138	64.3077	450	3.1689
0.2551	71.4615	500	3.0233
0.2006	78.6154	550	2.9657
0.1784	85.7692	600	2.9367
0.1538	92.9231	650	2.8976
0.1430	100.0	700	2.8741
0.1335	107.1538	750	2.8598
0.1232	114.3077	800	2.8567
0.1214	121.4615	850	2.8443
0.1140	128.6154	900	2.8396
0.1144	135.7692	950	2.8381
0.1100	142.9231	1000	2.8392

Framework versions

—PEFT 0.18.1
—Transformers 5.2.0
—Pytorch 2.9.0+cu126
—Datasets 4.0.0
—Tokenizers 0.22.2