Modelpublic

daryaZare/iris-olmo-2-1b-combined-multihop-k5-ep4

sourceHugging Faceapache-2.0updated 1d agoView on Hugging Face

0likes174downloads

Model Card

iris-olmo-2-1b-combined-multihop-k5-ep4

This model is a fine-tuned version of allenai/OLMo-2-0425-1B-Instruct on the None dataset. It achieves the following results on the evaluation set:

—Loss: 0.8649
—Model Preparation Time: 0.0151
—Soft Mae: 0.0936
—Soft Brier: 0.0261
—Student Prelevantmean: 0.3359
—Teacher Prelevantmean: 0.3342
—Bin F1: 0.8657
—Cal Ece: 0.0901
—Cal Brier: 0.0838
—Cal Auroc: 0.9611
—Info Ndcg@p8: 0.9738
—Info Pairwiseacc: 0.8498
—Num Questions: 110
—Num Pairs: 1071

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 5e-05
—trainbatchsize: 1
—evalbatchsize: 1
—seed: 42
—gradientaccumulationsteps: 16
—totaltrainbatch_size: 16
—optimizer: Use OptimizerNames.PAGEDADAMW8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
—lrschedulertype: constant
—lrschedulerwarmup_steps: 0.03
—num_epochs: 4.0

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Soft Mae	Soft Brier	Student Prelevantmean	Teacher Prelevantmean	Bin F1	Cal Ece	Cal Brier	Cal Auroc	Info Ndcg@p8	Info Pairwiseacc	Num Questions	Num Pairs
1.3153	0.0410	25	1.2375	0.0151	0.2609	0.0945	0.3080	0.3342	0.1776	0.0787	0.1893	0.7785	0.8496	0.6729	110	1071
0.8572	0.0820	50	1.2179	0.0151	0.1696	0.0725	0.2853	0.3342	0.7185	0.0753	0.1389	0.8845	0.9254	0.7512	110	1071
1.0151	0.1231	75	0.9505	0.0151	0.1832	0.0588	0.3463	0.3342	0.7197	0.0710	0.1365	0.8833	0.9369	0.7673	110	1071
0.9541	0.1641	100	0.8845	0.0151	0.1637	0.0457	0.3618	0.3342	0.8010	0.1057	0.1172	0.9233	0.9469	0.7963	110	1071
0.8827	0.2051	125	0.8358	0.0151	0.1595	0.0466	0.3533	0.3342	0.7851	0.0890	0.1209	0.9171	0.9532	0.8069	110	1071
0.8423	0.2461	150	0.8049	0.0151	0.1323	0.0401	0.3034	0.3342	0.7954	0.0664	0.1065	0.9324	0.9531	0.8126	110	1071
0.7262	0.2872	175	0.8554	0.0151	0.1411	0.0453	0.3975	0.3342	0.8047	0.0723	0.1091	0.9358	0.9555	0.8168	110	1071
0.7112	0.3282	200	0.8047	0.0151	0.1297	0.0414	0.2768	0.3342	0.7839	0.0770	0.1048	0.9396	0.9571	0.8160	110	1071
0.8105	0.3692	225	0.8056	0.0151	0.1478	0.0402	0.2949	0.3342	0.7308	0.1191	0.1126	0.9448	0.9586	0.8306	110	1071
0.6420	0.4102	250	0.7295	0.0151	0.1245	0.0364	0.3071	0.3342	0.8072	0.0579	0.1019	0.9387	0.9576	0.8293	110	1071
0.8122	0.4512	275	0.8429	0.0151	0.1437	0.0438	0.4022	0.3342	0.8146	0.1236	0.1140	0.9388	0.9657	0.8287	110	1071
0.6731	0.4923	300	0.6958	0.0151	0.1153	0.0301	0.3337	0.3342	0.8478	0.0909	0.0918	0.9523	0.9664	0.8303	110	1071
0.7049	0.5333	325	0.7645	0.0151	0.1288	0.0386	0.2776	0.3342	0.7672	0.0862	0.1058	0.9444	0.9648	0.8239	110	1071
0.7252	0.5743	350	0.6951	0.0151	0.1142	0.0316	0.3416	0.3342	0.8453	0.0758	0.0929	0.9508	0.9668	0.8258	110	1071
0.6421	0.6153	375	0.6898	0.0151	0.1097	0.0289	0.3428	0.3342	0.8586	0.0853	0.0894	0.9552	0.9683	0.8316	110	1071
0.6345	0.6563	400	0.6883	0.0151	0.1055	0.0292	0.3438	0.3342	0.8472	0.0614	0.0851	0.9556	0.9676	0.8294	110	1071
0.7143	0.6974	425	0.6949	0.0151	0.1110	0.0298	0.2886	0.3342	0.8444	0.0863	0.0913	0.9610	0.9670	0.8308	110	1071
0.6595	0.7384	450	0.6591	0.0151	0.1020	0.0271	0.3167	0.3342	0.8487	0.0695	0.0844	0.9630	0.9681	0.8405	110	1071
0.5604	0.7794	475	0.7032	0.0151	0.1182	0.0342	0.2769	0.3342	0.7908	0.0933	0.0947	0.9596	0.9644	0.8361	110	1071
0.6265	0.8204	500	0.6530	0.0151	0.1047	0.0280	0.2909	0.3342	0.8362	0.0788	0.0854	0.9635	0.9665	0.8434	110	1071
0.6794	0.8615	525	0.6205	0.0151	0.1023	0.0269	0.3195	0.3342	0.8458	0.0594	0.0819	0.9602	0.9678	0.8357	110	1071
0.5806	0.9025	550	0.6989	0.0151	0.1066	0.0298	0.3321	0.3342	0.8346	0.0738	0.0901	0.9520	0.9688	0.8330	110	1071
0.6811	0.9435	575	0.6919	0.0151	0.1176	0.0315	0.3423	0.3342	0.8362	0.0903	0.0966	0.9472	0.9698	0.8272	110	1071
0.5895	0.9845	600	0.6692	0.0151	0.1035	0.0308	0.2812	0.3342	0.8329	0.0822	0.0865	0.9605	0.9742	0.8346	110	1071
0.4926	1.0246	625	0.7009	0.0151	0.1014	0.0307	0.2862	0.3342	0.8411	0.0650	0.0871	0.9598	0.9685	0.8424	110	1071
0.4741	1.0656	650	0.6555	0.0151	0.0989	0.0289	0.3029	0.3342	0.8480	0.0562	0.0840	0.9589	0.9690	0.8410	110	1071
0.3791	1.1067	675	0.7708	0.0151	0.1012	0.0338	0.2949	0.3342	0.8310	0.0669	0.0878	0.9552	0.9707	0.8408	110	1071
0.4641	1.1477	700	0.6525	0.0151	0.0997	0.0277	0.3327	0.3342	0.8491	0.0682	0.0864	0.9582	0.9722	0.8471	110	1071
0.3932	1.1887	725	0.7194	0.0151	0.1110	0.0329	0.2746	0.3342	0.7827	0.0807	0.0920	0.9609	0.9692	0.8425	110	1071
0.4206	1.2297	750	0.6599	0.0151	0.1009	0.0269	0.3422	0.3342	0.8486	0.0859	0.0898	0.9592	0.9723	0.8485	110	1071
0.4216	1.2707	775	0.7033	0.0151	0.1083	0.0307	0.2925	0.3342	0.8081	0.0791	0.0929	0.9548	0.9711	0.8468	110	1071
0.4139	1.3118	800	0.7266	0.0151	0.1033	0.0301	0.3113	0.3342	0.8422	0.0622	0.0901	0.9530	0.9744	0.8428	110	1071
0.4448	1.3528	825	0.6580	0.0151	0.1009	0.0254	0.3379	0.3342	0.8617	0.0881	0.0841	0.9609	0.9736	0.8421	110	1071
0.5435	1.3938	850	0.7768	0.0151	0.1073	0.0267	0.3283	0.3342	0.8482	0.1023	0.0904	0.9607	0.9717	0.8443	110	1071
0.4878	1.4348	875	0.6614	0.0151	0.1085	0.0279	0.3109	0.3342	0.8273	0.0927	0.0918	0.9577	0.9736	0.8462	110	1071
0.4521	1.4758	900	0.7483	0.0151	0.1143	0.0337	0.2734	0.3342	0.7805	0.0841	0.0957	0.9559	0.9693	0.8446	110	1071
0.3263	1.5169	925	0.7516	0.0151	0.0967	0.0296	0.2982	0.3342	0.8587	0.0703	0.0859	0.9577	0.9726	0.8414	110	1071
0.5572	1.5579	950	0.6269	0.0151	0.1014	0.0256	0.3387	0.3342	0.8541	0.0784	0.0829	0.9600	0.9694	0.8343	110	1071
0.4654	1.5989	975	0.6782	0.0151	0.0971	0.0269	0.3109	0.3342	0.8484	0.0515	0.0795	0.9627	0.9781	0.8422	110	1071
0.5392	1.6399	1000	0.6656	0.0151	0.1053	0.0280	0.3083	0.3342	0.8252	0.0734	0.0881	0.9589	0.9720	0.8408	110	1071
0.5079	1.6810	1025	0.6198	0.0151	0.0971	0.0251	0.3069	0.3342	0.8495	0.0713	0.0819	0.9646	0.9740	0.8450	110	1071
0.4404	1.7220	1050	0.8601	0.0151	0.1023	0.0323	0.2696	0.3342	0.8260	0.0820	0.0891	0.9599	0.9719	0.8485	110	1071
0.4894	1.7630	1075	0.5924	0.0151	0.0950	0.0234	0.3302	0.3342	0.8668	0.0820	0.0822	0.9638	0.9722	0.8491	110	1071
0.5023	1.8040	1100	0.7622	0.0151	0.1113	0.0323	0.2786	0.3342	0.7821	0.0910	0.0967	0.9563	0.9696	0.8473	110	1071
0.3718	1.8450	1125	0.6609	0.0151	0.0964	0.0268	0.3057	0.3342	0.8524	0.0695	0.0850	0.9590	0.9670	0.8410	110	1071
0.3650	1.8861	1150	0.6278	0.0151	0.0939	0.0252	0.3042	0.3342	0.8596	0.0751	0.0815	0.9631	0.9724	0.8482	110	1071
0.4105	1.9271	1175	0.6757	0.0151	0.0996	0.0284	0.2926	0.3342	0.8379	0.0682	0.0861	0.9602	0.9705	0.8480	110	1071
0.4689	1.9681	1200	0.6176	0.0151	0.0953	0.0257	0.3075	0.3342	0.8567	0.0660	0.0812	0.9630	0.9730	0.8490	110	1071
0.4540	2.0082	1225	0.6413	0.0151	0.0909	0.0256	0.3107	0.3342	0.8641	0.0677	0.0759	0.9659	0.9761	0.8329	110	1071
0.2539	2.0492	1250	0.6758	0.0151	0.0884	0.0246	0.3302	0.3342	0.8663	0.0691	0.0787	0.9641	0.9740	0.8472	110	1071
0.2825	2.0902	1275	0.7186	0.0151	0.0885	0.0240	0.3307	0.3342	0.8689	0.0713	0.0800	0.9643	0.9764	0.8457	110	1071
0.3129	2.1313	1300	0.6613	0.0151	0.0892	0.0229	0.3493	0.3342	0.8735	0.0683	0.0740	0.9688	0.9760	0.8537	110	1071
0.2733	2.1723	1325	0.6736	0.0151	0.0910	0.0250	0.3108	0.3342	0.8650	0.0675	0.0801	0.9637	0.9741	0.8455	110	1071
0.2478	2.2133	1350	0.7575	0.0151	0.0921	0.0265	0.3117	0.3342	0.8414	0.0693	0.0845	0.9605	0.9755	0.8527	110	1071
0.2728	2.2543	1375	0.7379	0.0151	0.0913	0.0255	0.3033	0.3342	0.8678	0.0710	0.0784	0.9668	0.9737	0.8488	110	1071
0.3114	2.2954	1400	0.7740	0.0151	0.0949	0.0241	0.3216	0.3342	0.8471	0.0868	0.0831	0.9659	0.9760	0.8478	110	1071
0.3565	2.3364	1425	0.8107	0.0151	0.0933	0.0247	0.3223	0.3342	0.8678	0.0869	0.0834	0.9634	0.9735	0.8478	110	1071
0.3360	2.3774	1450	0.6883	0.0151	0.0966	0.0265	0.3022	0.3342	0.8436	0.0697	0.0843	0.9619	0.9734	0.8526	110	1071
0.2718	2.4184	1475	0.7667	0.0151	0.0957	0.0255	0.3305	0.3342	0.8564	0.0819	0.0851	0.9622	0.9803	0.8494	110	1071
0.1969	2.4594	1500	0.7403	0.0151	0.0897	0.0251	0.3405	0.3342	0.8732	0.0590	0.0766	0.9637	0.9777	0.8557	110	1071
0.1994	2.5005	1525	0.7676	0.0151	0.0874	0.0260	0.3041	0.3342	0.8634	0.0682	0.0787	0.9651	0.9796	0.8591	110	1071
0.2833	2.5415	1550	0.6596	0.0151	0.0879	0.0258	0.3034	0.3342	0.8623	0.0556	0.0766	0.9667	0.9783	0.8564	110	1071
0.3017	2.5825	1575	0.6968	0.0151	0.0905	0.0255	0.3423	0.3342	0.8564	0.0586	0.0781	0.9643	0.9784	0.8513	110	1071
0.2385	2.6235	1600	0.6811	0.0151	0.0918	0.0251	0.3244	0.3342	0.8517	0.0641	0.0827	0.9617	0.9813	0.8530	110	1071
0.2847	2.6645	1625	0.7099	0.0151	0.0866	0.0242	0.3342	0.3342	0.8552	0.0605	0.0789	0.9642	0.9720	0.8464	110	1071
0.2944	2.7056	1650	0.7387	0.0151	0.0925	0.0273	0.3216	0.3342	0.8418	0.0602	0.0814	0.9623	0.9736	0.8450	110	1071
0.3687	2.7466	1675	0.6983	0.0151	0.0951	0.0239	0.3317	0.3342	0.8583	0.0774	0.0835	0.9621	0.9812	0.8443	110	1071
0.1949	2.7876	1700	0.8009	0.0151	0.0922	0.0254	0.3399	0.3342	0.8583	0.0655	0.0815	0.9612	0.9776	0.8454	110	1071
0.2444	2.8286	1725	0.6966	0.0151	0.0937	0.0261	0.3177	0.3342	0.8595	0.0623	0.0820	0.9606	0.9777	0.8511	110	1071
0.3166	2.8697	1750	0.8596	0.0151	0.1007	0.0279	0.3046	0.3342	0.8239	0.0888	0.0895	0.9602	0.9757	0.8534	110	1071
0.2587	2.9107	1775	0.7817	0.0151	0.0914	0.0267	0.3362	0.3342	0.8530	0.0572	0.0827	0.9605	0.9726	0.8566	110	1071
0.3880	2.9517	1800	0.7786	0.0151	0.0991	0.0272	0.3015	0.3342	0.8234	0.0794	0.0879	0.9601	0.9731	0.8500	110	1071
0.2848	2.9927	1825	0.7029	0.0151	0.0922	0.0255	0.3131	0.3342	0.8451	0.0653	0.0833	0.9623	0.9786	0.8505	110	1071
0.2174	3.0328	1850	0.7859	0.0151	0.0902	0.0249	0.3305	0.3342	0.8560	0.0703	0.0824	0.9639	0.9811	0.8468	110	1071
0.2400	3.0738	1875	0.7584	0.0151	0.0903	0.0254	0.3145	0.3342	0.8540	0.0639	0.0830	0.9621	0.9794	0.8461	110	1071
0.2756	3.1149	1900	1.0203	0.0151	0.0966	0.0290	0.2813	0.3342	0.8253	0.0806	0.0893	0.9636	0.9734	0.8464	110	1071
0.3070	3.1559	1925	0.7490	0.0151	0.0920	0.0252	0.3062	0.3342	0.8524	0.0722	0.0843	0.9648	0.9721	0.8489	110	1071
0.2229	3.1969	1950	0.7509	0.0151	0.0869	0.0244	0.3254	0.3342	0.8641	0.0676	0.0799	0.9647	0.9719	0.8498	110	1071
0.2558	3.2379	1975	0.7839	0.0151	0.0924	0.0245	0.3328	0.3342	0.8598	0.0838	0.0835	0.9629	0.9756	0.8519	110	1071
0.1722	3.2789	2000	0.8377	0.0151	0.0900	0.0253	0.3216	0.3342	0.8610	0.0763	0.0822	0.9637	0.9760	0.8508	110	1071
0.1900	3.3200	2025	0.7288	0.0151	0.0872	0.0254	0.3141	0.3342	0.8583	0.0563	0.0799	0.9635	0.9729	0.8544	110	1071
0.2418	3.3610	2050	0.7686	0.0151	0.0927	0.0282	0.2930	0.3342	0.8369	0.0690	0.0837	0.9618	0.9723	0.8547	110	1071
0.2553	3.4020	2075	0.7686	0.0151	0.0864	0.0238	0.3309	0.3342	0.8644	0.0769	0.0805	0.9640	0.9692	0.8493	110	1071
0.1792	3.4430	2100	0.7822	0.0151	0.0882	0.0246	0.3461	0.3342	0.8627	0.0808	0.0808	0.9639	0.9750	0.8443	110	1071
0.1754	3.4841	2125	0.9027	0.0151	0.0920	0.0276	0.2976	0.3342	0.8482	0.0585	0.0828	0.9626	0.9715	0.8520	110	1071
0.1797	3.5251	2150	1.0702	0.0151	0.0949	0.0288	0.2938	0.3342	0.8178	0.0636	0.0867	0.9598	0.9689	0.8524	110	1071
0.2531	3.5661	2175	0.8017	0.0151	0.0949	0.0251	0.3215	0.3342	0.8458	0.0872	0.0836	0.9649	0.9734	0.8451	110	1071
0.2291	3.6071	2200	0.8493	0.0151	0.0867	0.0251	0.3521	0.3342	0.8605	0.0827	0.0815	0.9653	0.9761	0.8469	110	1071
0.2122	3.6481	2225	0.8643	0.0151	0.0905	0.0280	0.3224	0.3342	0.8459	0.0578	0.0803	0.9620	0.9715	0.8413	110	1071
0.2099	3.6892	2250	0.8294	0.0151	0.0860	0.0246	0.3280	0.3342	0.8667	0.0742	0.0801	0.9649	0.9768	0.8446	110	1071
0.2142	3.7302	2275	0.7437	0.0151	0.0863	0.0252	0.3213	0.3342	0.8587	0.0634	0.0796	0.9646	0.9717	0.8439	110	1071
0.2277	3.7712	2300	0.7762	0.0151	0.0904	0.0259	0.3156	0.3342	0.8532	0.0606	0.0824	0.9617	0.9737	0.8467	110	1071
0.1922	3.8122	2325	0.8241	0.0151	0.0957	0.0258	0.3131	0.3342	0.8382	0.0740	0.0835	0.9638	0.9736	0.8421	110	1071
0.2042	3.8532	2350	0.8476	0.0151	0.0925	0.0269	0.3048	0.3342	0.8421	0.0640	0.0845	0.9624	0.9716	0.8433	110	1071
0.2691	3.8943	2375	0.8728	0.0151	0.0890	0.0255	0.3274	0.3342	0.8703	0.0775	0.0807	0.9640	0.9702	0.8458	110	1071
0.1799	3.9353	2400	0.8481	0.0151	0.0905	0.0258	0.3179	0.3342	0.8583	0.0687	0.0822	0.9629	0.9723	0.8500	110	1071
0.2201	3.9763	2425	0.9206	0.0151	0.1004	0.0298	0.2888	0.3342	0.8090	0.0811	0.0897	0.9603	0.9725	0.8506	110	1071
0.2698	4.0	2440	0.8649	0.0151	0.0936	0.0261	0.3359	0.3342	0.8657	0.0901	0.0838	0.9611	0.9738	0.8498	110	1071

Framework versions

—PEFT 0.19.1
—Transformers 5.7.0
—Pytorch 2.11.0+cu130
—Datasets 4.8.5
—Tokenizers 0.22.2