Modelpublic

versae/snowflake-arctic-embed-m-edu-scorer-lr3e4-bs32

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes12downloads

Model Card

snowflake-arctic-embed-m-edu-scorer-lr3e4-bs32

This model is a fine-tuned version of Snowflake/snowflake-arctic-embed-m on an unknown dataset. It achieves the following results on the evaluation set:

—Loss: 1.5590
—Precision: 0.4789
—Recall: 0.2833
—F1 Macro: 0.2779
—Accuracy: 0.3073

Model description

More information needed

Intended uses & limitations

More information needed

Test results

Binary classification accuracy (threshold at label 3) ≈ 73.45%

Test Report:

              precision    recall  f1-score   support

           0       0.73      0.44      0.55       100
           1       0.25      0.30      0.28       100
           2       0.22      0.47      0.30       100
           3       0.26      0.33      0.29       100
           4       0.40      0.14      0.21       100
           5       1.00      0.02      0.04        50

    accuracy                           0.31       550
   macro avg       0.48      0.28      0.28       550
weighted avg       0.43      0.31      0.30       550

Confusion Matrix:

[[44 32 22  2  0  0]
 [13 30 47 10  0  0]
 [ 3 33 47 17  0  0]
 [ 0 14 44 33  9  0]
 [ 0  5 36 45 14  0]
 [ 0  4 14 19 12  1]]

Test metrics

  epoch                   =       20.0
  eval_accuracy           =     0.3073
  eval_f1_macro           =     0.2779
  eval_loss               =      1.559
  eval_precision          =     0.4789
  eval_recall             =     0.2833
  eval_runtime            = 0:00:04.86
  eval_samples_per_second =    113.008
  eval_steps_per_second   =      3.698

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.0003
—trainbatchsize: 32
—evalbatchsize: 32
—seed: 0
—optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—lrschedulerwarmup_ratio: 0.1
—num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 Macro	Accuracy
No log	0	0	3.1167	0.0587	0.1667	0.0869	0.3524
0.994	0.3368	1000	0.9856	0.2966	0.2691	0.2391	0.397
0.9708	0.6736	2000	0.9981	0.3297	0.2844	0.2609	0.4404
1.0021	1.0104	3000	0.9604	0.3580	0.2772	0.2524	0.391
0.9775	1.3473	4000	1.0404	0.3467	0.2686	0.2393	0.3408
0.9321	1.6841	5000	0.9524	0.3051	0.2840	0.2591	0.4184
0.9446	2.0209	6000	0.9473	0.3444	0.2819	0.2571	0.4018
0.9997	2.3577	7000	0.9371	0.3497	0.2870	0.2653	0.4198
0.9184	2.6945	8000	0.9295	0.3579	0.2989	0.2883	0.4292
0.9196	3.0313	9000	0.9193	0.3422	0.2972	0.2794	0.446
0.9115	3.3681	10000	0.9506	0.3245	0.2882	0.2697	0.4596
0.8851	3.7050	11000	0.8964	0.3725	0.2954	0.2810	0.4382
0.9021	4.0418	12000	0.8843	0.3722	0.3019	0.2890	0.4496
0.8914	4.3786	13000	0.8831	0.3764	0.2950	0.2811	0.4266
0.8962	4.7154	14000	0.8757	0.3609	0.3029	0.2864	0.4572
0.8414	5.0522	15000	0.8788	0.3725	0.3079	0.2922	0.4678
0.8621	5.3890	16000	0.8677	0.3684	0.3109	0.3012	0.4622
0.8172	5.7258	17000	0.8725	0.3877	0.3021	0.2884	0.4654
0.8282	6.0626	18000	0.8833	0.3651	0.3051	0.2951	0.4702
0.8652	6.3995	19000	0.8535	0.3712	0.3200	0.3126	0.4604
0.8123	6.7363	20000	0.8529	0.3664	0.3075	0.2931	0.4434
0.8402	7.0731	21000	0.8461	0.3816	0.3099	0.2969	0.466
0.8082	7.4099	22000	0.8515	0.3681	0.3193	0.3118	0.4636
0.815	7.7467	23000	0.8494	0.3692	0.3136	0.3027	0.453
0.7828	8.0835	24000	0.8476	0.3936	0.3187	0.3114	0.4756
0.806	8.4203	25000	0.8364	0.3875	0.3168	0.3075	0.4644
0.8356	8.7572	26000	0.8511	0.3869	0.3282	0.3223	0.4538
0.8418	9.0940	27000	0.8398	0.3679	0.3127	0.2988	0.4676
0.8087	9.4308	28000	0.8384	0.3795	0.3188	0.3101	0.4516
0.7822	9.7676	29000	0.8398	0.3667	0.3180	0.3082	0.4784
0.7911	10.1044	30000	0.8331	0.3796	0.3188	0.3123	0.4702
0.7887	10.4412	31000	0.8336	0.3659	0.3153	0.3041	0.473
0.7634	10.7780	32000	0.8347	0.3632	0.3149	0.3058	0.4722
0.802	11.1149	33000	0.8351	0.3834	0.3233	0.3190	0.4582
0.8075	11.4517	34000	0.8484	0.3694	0.3106	0.2990	0.4784
0.7871	11.7885	35000	0.8348	0.3683	0.3195	0.3143	0.4684
0.7465	12.1253	36000	0.8249	0.3773	0.3216	0.3130	0.4734
0.763	12.4621	37000	0.8263	0.3658	0.3160	0.3088	0.4746
0.7891	12.7989	38000	0.8283	0.3865	0.3272	0.3224	0.4596
0.7516	13.1357	39000	0.8275	0.3827	0.3269	0.3189	0.4812
0.7867	13.4725	40000	0.8269	0.3773	0.3272	0.3237	0.4652
0.7552	13.8094	41000	0.8386	0.3715	0.3132	0.3043	0.4742
0.7366	14.1462	42000	0.8219	0.3731	0.3219	0.3166	0.4654
0.7214	14.4830	43000	0.8243	0.3684	0.3210	0.3144	0.4688
0.7237	14.8198	44000	0.8205	0.3767	0.3234	0.3174	0.467
0.7562	15.1566	45000	0.8281	0.3741	0.3257	0.3200	0.4688
0.7582	15.4934	46000	0.8287	0.3692	0.3158	0.3090	0.4744
0.742	15.8302	47000	0.8264	0.3761	0.3189	0.3114	0.4774
0.7384	16.1671	48000	0.8295	0.3748	0.3269	0.3245	0.463
0.7315	16.5039	49000	0.8244	0.3722	0.3239	0.3200	0.4748
0.7006	16.8407	50000	0.8245	0.3673	0.3184	0.3116	0.4786
0.7676	17.1775	51000	0.8229	0.3698	0.3219	0.3148	0.4684
0.7172	17.5143	52000	0.8249	0.3677	0.3196	0.3130	0.4802
0.7331	17.8511	53000	0.8170	0.3761	0.3221	0.3167	0.4686
0.6957	18.1879	54000	0.8175	0.3779	0.3219	0.3177	0.4668
0.7378	18.5248	55000	0.8204	0.3725	0.3217	0.3181	0.4622
0.7196	18.8616	56000	0.8185	0.3780	0.3221	0.3160	0.4756
0.736	19.1984	57000	0.8158	0.3772	0.3212	0.3152	0.4714
0.7561	19.5352	58000	0.8161	0.3754	0.3225	0.3172	0.4706
0.7042	19.8720	59000	0.8181	0.3752	0.3218	0.3177	0.467

Framework versions

—Transformers 4.53.2
—Pytorch 2.7.1+cu126
—Datasets 4.0.0
—Tokenizers 0.21.2