Modelpublic

leobianco/ragtruth-qa_RM_gemma-4-E4B-it_S130104_epo8_lr6_4e-04_r16_2609181710

sourceHugging Faceapache-2.0updated 9d agoView on Hugging Face

0likes139downloads

Model Card

ragtruth-qaRMgemma-4-E4B-itS130104epo8lr64e-04r162609181710

This model is a fine-tuned version of google/gemma-4-E4B-it on an unknown dataset. It achieves the following results on the evaluation set:

—Loss: 1.4386
—Roc Auc: 0.9083
—Best Threshold: 0.9893
—Tpr At Best Threshold: 0.8661
—Fpr At Best Threshold: 0.1429
—Accuracy At Best Threshold: 0.8649
—Avg Score True Positives: 0.9182
—Avg Score True Negatives: 0.3332

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.0006408489062977591
—trainbatchsize: 16
—evalbatchsize: 32
—seed: 130104
—distributed_type: multi-GPU
—num_devices: 2
—totaltrainbatch_size: 32
—totalevalbatch_size: 64
—optimizer: Use OptimizerNames.ADAMWTORCHFUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
—lrschedulertype: cosine
—lrschedulerwarmup_steps: 0.1
—num_epochs: 8.0

Training results

Training Loss	Epoch	Step	Validation Loss	Roc Auc	Best Threshold	Tpr At Best Threshold	Fpr At Best Threshold	Accuracy At Best Threshold	Avg Score True Positives	Avg Score True Negatives
No log	0	0	1.1265	0.4817	0.6065	0.5630	0.5238	0.5507	0.6132	0.6164
1.2617	0.2907	25	0.7961	0.6885	0.8176	0.4803	0.1667	0.5304	0.7952	0.7378
0.8691	0.5814	50	0.7304	0.7479	0.8706	0.5630	0.1429	0.6047	0.8448	0.7148
1.2773	0.8721	75	0.8366	0.8443	0.6796	0.7126	0.1667	0.7297	0.7618	0.3638
0.5078	1.1628	100	0.5977	0.9003	0.5465	0.9134	0.2619	0.8885	0.8462	0.3783
0.1870	1.4535	125	0.6073	0.9091	0.9795	0.8386	0.1190	0.8446	0.9721	0.6371
0.5757	1.7442	150	0.9846	0.8812	0.4739	0.8228	0.1667	0.8243	0.7434	0.2104
0.4380	2.0349	175	0.6051	0.8891	0.9522	0.8583	0.1905	0.8514	0.9481	0.5658
0.5752	2.3256	200	0.7669	0.8539	0.6245	0.8150	0.1905	0.8142	0.7824	0.3080
0.6396	2.6163	225	1.3900	0.8683	0.3771	0.8110	0.2143	0.8074	0.7164	0.2094
0.8691	2.9070	250	0.8335	0.8914	0.4158	0.9213	0.2143	0.9020	0.7531	0.2247
0.5664	3.1977	275	0.9697	0.9040	0.7957	0.7559	0.0476	0.7838	0.8123	0.1993
0.4810	3.4884	300	0.6069	0.8954	0.9511	0.8543	0.1667	0.8514	0.9526	0.5244
0.7480	3.7791	325	0.6620	0.8759	0.9781	0.7441	0.0952	0.7669	0.9310	0.4486
0.1305	4.0698	350	0.6583	0.9118	0.4886	0.9331	0.2143	0.9122	0.9087	0.2797
0.1034	4.3605	375	1.0164	0.8965	0.8544	0.8661	0.1905	0.8581	0.8960	0.3031
0.1956	4.6512	400	1.2204	0.8883	0.9966	0.8110	0.1190	0.8209	0.9364	0.4217
0.0081	4.9419	425	1.2378	0.8948	0.6327	0.8189	0.1667	0.8209	0.8284	0.2167
0.1104	5.2326	450	0.9881	0.9129	0.9546	0.8898	0.1190	0.8885	0.9235	0.3128
0.0093	5.5233	475	1.2518	0.9028	0.8239	0.8622	0.1429	0.8615	0.8878	0.2373
0.0071	5.8140	500	1.3686	0.9029	0.9161	0.8465	0.1667	0.8446	0.8970	0.2869
0.0005	6.1047	525	1.3951	0.9059	0.9872	0.8504	0.1667	0.8480	0.9164	0.3233
0.0006	6.3953	550	1.3897	0.9041	0.9937	0.8268	0.1429	0.8311	0.9152	0.3215
0.0145	6.6860	575	1.3913	0.9087	0.9928	0.8543	0.1429	0.8547	0.9224	0.3446
0.0000	6.9767	600	1.4198	0.9092	0.9921	0.8583	0.1429	0.8581	0.9203	0.3367
0.0003	7.2674	625	1.4378	0.9077	0.9887	0.8622	0.1429	0.8615	0.9166	0.3319
0.0002	7.5581	650	1.4389	0.9089	0.9892	0.8583	0.1429	0.8581	0.9183	0.3341
0.0002	7.8488	675	1.4407	0.9083	0.9910	0.8583	0.1429	0.8581	0.9179	0.3332
0.0001	8.0	688	1.4386	0.9083	0.9893	0.8661	0.1429	0.8649	0.9182	0.3332
0.0001	8.0	688	1.4386	0.9083	0.9893	0.8661	0.1429	0.8649	0.9182	0.3332

Framework versions

—PEFT 0.20.0
—Transformers 5.14.1
—Pytorch 2.11.0+cu130
—Datasets 5.0.1
—Tokenizers 0.22.2