Modelpublic

daryaZare/iris-olmo-2-1b-combined-multihop-k10-ep4

sourceHugging Faceapache-2.0updated 2d agoView on Hugging Face

0likes33downloads

Model Card

iris-olmo-2-1b-combined-multihop-k10-ep4

This model is a fine-tuned version of allenai/OLMo-2-0425-1B-Instruct on the None dataset. It achieves the following results on the evaluation set:

—Loss: 0.8972
—Model Preparation Time: 0.0156
—Soft Mae: 0.0784
—Soft Brier: 0.0179
—Student Prelevantmean: 0.2938
—Teacher Prelevantmean: 0.2965
—Bin F1: 0.8556
—Cal Ece: 0.0905
—Cal Brier: 0.0864
—Cal Auroc: 0.9672
—Info Ndcg@p8: 0.9703
—Info Pairwiseacc: 0.8578
—Num Questions: 110
—Num Pairs: 1071

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 5e-05
—trainbatchsize: 1
—evalbatchsize: 1
—seed: 42
—gradientaccumulationsteps: 16
—totaltrainbatch_size: 16
—optimizer: Use OptimizerNames.PAGEDADAMW8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
—lrschedulertype: constant
—lrschedulerwarmup_steps: 0.03
—num_epochs: 4.0

Training results

Training Loss	Epoch	Step	Bin F1	Cal Auroc	Cal Brier	Cal Ece	Info Pairwiseacc	Info Ndcg@p8	Soft Brier	Soft Mae	Student Prelevantmean	Teacher Prelevantmean	Validation Loss	Model Preparation Time	Num Pairs	Num Questions
1.5416	0.0410	25	0.0248	0.7675	0.2159	0.1492	0.6814	0.8596	0.0842	0.2322	0.2191	0.2965	1.4774	0.015	1071	110
1.0242	0.0820	50	0.6033	0.8783	0.1611	0.1338	0.7400	0.9273	0.0571	0.1625	0.2341	0.2965	1.3191	0.015	1071	110
1.1841	0.1231	75	0.6626	0.8823	0.1480	0.0823	0.7814	0.9448	0.0478	0.1680	0.3001	0.2965	1.1989	0.015	1071	110
1.0800	0.1641	100	0.7893	0.9255	0.1247	0.0932	0.8012	0.9538	0.0350	0.1319	0.2901	0.2965	1.0728	0.015	1071	110
1.0860	0.2051	125	0.7551	0.9264	0.1192	0.0834	0.8054	0.9529	0.0338	0.1221	0.2849	0.2965	1.0477	0.015	1071	110
1.0694	0.2461	150	0.7745	0.9238	0.1161	0.0732	0.8104	0.9524	0.0335	0.1240	0.3088	0.2965	1.0330	0.015	1071	110
0.9916	0.2872	175	0.8354	0.9378	0.1148	0.1112	0.8256	0.9610	0.0363	0.1316	0.3543	0.2965	1.1183	0.015	1071	110
0.9946	0.3282	200	0.8131	0.9414	0.1015	0.0745	0.8312	0.9660	0.0294	0.1057	0.2959	0.2965	0.9963	0.015	1071	110
1.0393	0.3692	225	0.6555	0.9429	0.1297	0.1304	0.8307	0.9619	0.0326	0.1232	0.2410	0.2965	1.0003	0.015	1071	110
0.8720	0.4102	250	0.8161	0.9438	0.1021	0.0759	0.8354	0.9616	0.0294	0.1025	0.2925	0.2965	0.9760	0.015	1071	110
1.0077	0.4512	275	0.8169	0.9490	0.1139	0.1248	0.8349	0.9622	0.0278	0.1186	0.3051	0.2965	0.9690	0.015	1071	110
0.9564	0.4923	300	0.8495	0.9554	0.0975	0.0994	0.8383	0.9665	0.0241	0.1020	0.3085	0.2965	0.8985	0.015	1071	110
0.8989	0.5333	325	0.8135	0.9585	0.1110	0.1381	0.8305	0.9630	0.0255	0.1167	0.3009	0.2965	0.9406	0.015	1071	110
0.8600	0.5743	350	0.7768	0.9579	0.1145	0.1265	0.8338	0.9641	0.0261	0.1102	0.2570	0.2965	0.9823	0.015	1071	110
0.9199	0.6153	375	0.8542	0.9559	0.1012	0.1053	0.8399	0.9689	0.0243	0.1030	0.3153	0.2965	0.8934	0.015	1071	110
0.9102	0.6563	400	0.8449	0.9587	0.1046	0.1046	0.8465	0.9711	0.0227	0.0924	0.2646	0.2965	0.9105	0.015	1071	110
0.8546	0.6974	425	0.8338	0.9584	0.0986	0.1066	0.8440	0.9712	0.0228	0.1034	0.3115	0.2965	0.8600	0.015	1071	110
0.7387	0.7384	450	0.8556	0.9608	0.0999	0.1028	0.8460	0.9697	0.0221	0.0902	0.2682	0.2965	0.8697	0.015	1071	110
0.8577	0.7794	475	0.8059	0.9649	0.1008	0.1107	0.8526	0.9705	0.0225	0.0947	0.2661	0.2965	0.8730	0.015	1071	110
0.8601	0.8204	500	0.7367	0.9613	0.1176	0.1414	0.8503	0.9734	0.0277	0.1060	0.2265	0.2965	1.0084	0.015	1071	110
0.8454	0.8615	525	0.7965	0.9556	0.1049	0.1036	0.8515	0.9698	0.0236	0.0965	0.2643	0.2965	0.8751	0.015	1071	110
0.8077	0.9025	550	0.8523	0.9585	0.0912	0.0781	0.8515	0.9716	0.0205	0.0924	0.3094	0.2965	0.8663	0.015	1071	110
0.7312	0.9435	575	0.8408	0.9532	0.1116	0.1145	0.8450	0.9716	0.0242	0.1067	0.2884	0.2965	0.9591	0.015	1071	110
0.7960	0.9845	600	0.8280	0.9644	0.0966	0.1061	0.8512	0.9728	0.0204	0.0861	0.2695	0.2965	0.8071	0.015	1071	110
0.6893	1.0246	625	0.8103	0.9581	0.1036	0.1136	0.8508	0.9728	0.0251	0.0882	0.2543	0.2965	0.9264	0.015	1071	110
0.5481	1.0656	650	0.8444	0.9596	0.0938	0.0853	0.8518	0.9735	0.0213	0.0843	0.2858	0.2965	0.8328	0.015	1071	110
0.5609	1.1067	675	0.8387	0.9523	0.1005	0.0933	0.8492	0.9735	0.0221	0.0884	0.2880	0.2965	0.9068	0.015	1071	110
0.6238	1.1477	700	0.8377	0.9599	0.1069	0.1110	0.8560	0.9726	0.0217	0.0907	0.2626	0.2965	0.9556	0.015	1071	110
0.4524	1.1887	725	0.8232	0.9635	0.0989	0.1081	0.8556	0.9724	0.0217	0.0863	0.2600	0.2965	0.8699	0.015	1071	110
0.6331	1.2297	750	0.8629	0.9636	0.0960	0.1061	0.8616	0.9725	0.0198	0.0890	0.3138	0.2965	0.8668	0.015	1071	110
0.6117	1.2707	775	0.7852	0.9576	0.1109	0.1198	0.8462	0.9632	0.0249	0.0977	0.2539	0.2965	0.9122	0.015	1071	110
0.4804	1.3118	800	0.8668	0.9625	0.0858	0.0768	0.8548	0.9722	0.0216	0.0871	0.3196	0.2965	0.8885	0.015	1071	110
0.6366	1.3528	825	0.8516	0.9649	0.0922	0.0969	0.8535	0.9666	0.0186	0.0825	0.2876	0.2965	0.8033	0.015	1071	110
0.6396	1.3938	850	0.8636	0.9667	0.0915	0.0945	0.8547	0.9697	0.0179	0.0816	0.2950	0.2965	0.7798	0.015	1071	110
0.6068	1.4348	875	0.8145	0.9584	0.1018	0.1090	0.8515	0.9680	0.0218	0.0855	0.2642	0.2965	0.8523	0.015	1071	110
0.5712	1.4758	900	0.8306	0.9620	0.0993	0.0975	0.8543	0.9711	0.0216	0.0882	0.2707	0.2965	0.8624	0.015	1071	110
0.4894	1.5169	925	0.8644	0.9617	0.1028	0.1261	0.8534	0.9689	0.0211	0.0843	0.2662	0.2965	0.9568	0.015	1071	110
0.6160	1.5579	950	0.8607	0.9667	0.0904	0.0873	0.8501	0.9734	0.0195	0.0816	0.2837	0.2965	0.7964	0.015	1071	110
0.6777	1.5989	975	0.8347	0.9624	0.0952	0.0971	0.8555	0.9732	0.0210	0.0830	0.2708	0.2965	0.8051	0.015	1071	110
0.5756	1.6399	1000	0.8352	0.9607	0.0924	0.0934	0.8530	0.9694	0.0235	0.0853	0.2745	0.2965	0.8804	0.015	1071	110
0.5165	1.6810	1025	0.8571	0.9627	0.0906	0.0852	0.8470	0.9701	0.0203	0.0828	0.2885	0.2965	0.8762	0.015	1071	110
0.5555	1.7220	1050	0.8282	0.9598	0.1032	0.1160	0.8569	0.9689	0.0235	0.0884	0.2519	0.2965	0.9036	0.015	1071	110
0.6647	1.7630	1075	0.8672	0.9646	0.0981	0.1053	0.8627	0.9750	0.0190	0.0846	0.2910	0.2965	0.8154	0.015	1071	110
0.6532	1.8040	1100	0.7898	0.9624	0.1036	0.1091	0.8509	0.9696	0.0224	0.0936	0.2587	0.2965	0.9010	0.015	1071	110
0.6027	1.8450	1125	0.8491	0.9675	0.0940	0.1144	0.8524	0.9699	0.0194	0.0804	0.2626	0.2965	0.8567	0.015	1071	110
0.5409	1.8861	1150	0.8367	0.9660	0.0939	0.1061	0.8581	0.9730	0.0211	0.0824	0.2618	0.2965	0.8948	0.015	1071	110
0.6275	1.9271	1175	0.8544	0.9641	0.0940	0.0963	0.8538	0.9701	0.0195	0.0837	0.2774	0.2965	0.8154	0.015	1071	110
0.6743	1.9681	1200	0.8422	0.9628	0.0925	0.1015	0.8540	0.9690	0.0213	0.0841	0.2741	0.2965	0.8445	0.015	1071	110
0.5237	2.0082	1225	0.8402	0.9656	0.0933	0.1096	0.8568	0.9714	0.0214	0.0808	0.2597	0.2965	0.9369	0.015	1071	110
0.4042	2.0492	1250	0.8675	0.9651	0.0927	0.0963	0.8521	0.9749	0.0186	0.0789	0.2765	0.2965	0.8832	0.015	1071	110
0.5259	2.0902	1275	0.8448	0.9679	0.0950	0.1096	0.8563	0.9720	0.0187	0.0789	0.2630	0.2965	0.8753	0.015	1071	110
0.3978	2.1313	1300	0.8471	0.9685	0.0988	0.1109	0.8580	0.9724	0.0199	0.0819	0.2570	0.2965	0.9676	0.015	1071	110
0.4026	2.1723	1325	0.8773	0.9672	0.0866	0.0900	0.8564	0.9729	0.0178	0.0767	0.3058	0.2965	0.8526	0.015	1071	110
0.4873	2.2133	1350	0.8532	0.9652	0.1013	0.1134	0.8554	0.9711	0.0204	0.0855	0.2588	0.2965	0.9566	0.015	1071	110
0.4678	2.2543	1375	0.8540	0.9664	0.0945	0.0999	0.8568	0.9737	0.0193	0.0800	0.2703	0.2965	0.9089	0.015	1071	110
0.4805	2.2954	1400	0.8713	0.9665	0.0934	0.1061	0.8558	0.9721	0.0175	0.0821	0.2932	0.2965	0.8750	0.015	1071	110
0.4692	2.3364	1425	0.8686	0.9688	0.0888	0.0982	0.8612	0.9753	0.0173	0.0756	0.2808	0.2965	0.8599	0.015	1071	110
0.4699	2.3774	1450	0.8492	0.9649	0.0930	0.1113	0.8539	0.9704	0.0215	0.0818	0.2565	0.2965	0.9591	0.015	1071	110
0.3308	2.4184	1475	0.8722	0.9657	0.0939	0.1153	0.8518	0.9754	0.0182	0.0769	0.2756	0.2965	0.9579	0.015	1071	110
0.3954	2.4594	1500	0.8755	0.9641	0.0913	0.0989	0.8577	0.9739	0.0198	0.0770	0.2783	0.2965	0.8967	0.015	1071	110
0.4075	2.5005	1525	0.8774	0.9654	0.0890	0.0955	0.8603	0.9727	0.0186	0.0764	0.2971	0.2965	0.8463	0.015	1071	110
0.4413	2.5415	1550	0.8627	0.9610	0.1003	0.1069	0.8595	0.9737	0.0209	0.0818	0.2644	0.2965	0.9913	0.015	1071	110
0.3978	2.5825	1575	0.8289	0.9615	0.0977	0.1033	0.8548	0.9732	0.0205	0.0816	0.2648	0.2965	0.9390	0.015	1071	110
0.4076	2.6235	1600	0.8540	0.9644	0.0973	0.1068	0.8562	0.9722	0.0196	0.0802	0.2699	0.2965	0.9082	0.015	1071	110
0.5393	2.6645	1625	0.8628	0.9636	0.0931	0.1018	0.8515	0.9690	0.0188	0.0782	0.2797	0.2965	0.8518	0.015	1071	110
0.3732	2.7056	1650	0.8528	0.9645	0.0959	0.1016	0.8596	0.9718	0.0193	0.0818	0.2697	0.2965	0.9028	0.015	1071	110
0.4996	2.7466	1675	0.8402	0.9602	0.0969	0.0976	0.8576	0.9738	0.0205	0.0807	0.2711	0.2965	0.8829	0.015	1071	110
0.3893	2.7876	1700	0.8607	0.9642	0.0909	0.0963	0.8615	0.9769	0.0210	0.0804	0.2716	0.2965	0.9179	0.015	1071	110
0.3717	2.8286	1725	0.8614	0.9635	0.0911	0.0994	0.8581	0.9716	0.0207	0.0796	0.2724	0.2965	0.9061	0.015	1071	110
0.5083	2.8697	1750	0.8024	0.9629	0.1066	0.1309	0.8539	0.9679	0.0238	0.0884	0.2370	0.2965	1.0656	0.015	1071	110
0.3859	2.9107	1775	0.8690	0.9647	0.0893	0.0891	0.8612	0.9726	0.0187	0.0775	0.2876	0.2965	0.9300	0.015	1071	110
0.4725	2.9517	1800	0.8408	0.9627	0.0938	0.0972	0.8520	0.9690	0.0214	0.0826	0.2707	0.2965	0.9237	0.015	1071	110
0.3645	2.9927	1825	0.8453	0.9609	0.0946	0.0987	0.8534	0.9693	0.0214	0.0827	0.2692	0.2965	0.9572	0.015	1071	110
0.3137	3.0328	1850	1.0085	0.0156	0.0816	0.0215	0.2581	0.2965	0.8340	0.1101	0.0989	0.9626	0.9712	0.8582	110	1071
0.3032	3.0738	1875	0.9212	0.0156	0.0792	0.0191	0.2865	0.2965	0.8704	0.0995	0.0939	0.9640	0.9738	0.8579	110	1071
0.3688	3.1149	1900	0.9974	0.0156	0.0879	0.0211	0.2617	0.2965	0.8312	0.1127	0.0993	0.9671	0.9712	0.8580	110	1071
0.3996	3.1559	1925	0.9795	0.0156	0.0836	0.0206	0.2745	0.2965	0.8579	0.1040	0.0958	0.9620	0.9710	0.8542	110	1071
0.3520	3.1969	1950	0.9515	0.0156	0.0786	0.0200	0.2775	0.2965	0.8556	0.0991	0.0947	0.9622	0.9751	0.8534	110	1071
0.3246	3.2379	1975	0.9734	0.0156	0.0801	0.0208	0.2821	0.2965	0.8571	0.0863	0.0872	0.9639	0.9747	0.8535	110	1071
0.3407	3.2789	2000	0.9257	0.0156	0.0826	0.0208	0.2657	0.2965	0.8324	0.1022	0.0948	0.9650	0.9727	0.8501	110	1071
0.3165	3.3200	2025	1.2010	0.0156	0.0817	0.0226	0.2497	0.2965	0.8439	0.1182	0.0975	0.9639	0.9751	0.8573	110	1071
0.2520	3.3610	2050	0.9372	0.0156	0.0782	0.0192	0.2697	0.2965	0.8532	0.1084	0.0962	0.9659	0.9729	0.8513	110	1071
0.2827	3.4020	2075	0.8823	0.0156	0.0797	0.0188	0.2890	0.2965	0.8519	0.0860	0.0932	0.9645	0.9730	0.8513	110	1071
0.3730	3.4430	2100	0.8886	0.0156	0.0798	0.0196	0.2716	0.2965	0.8383	0.0963	0.0945	0.9639	0.9733	0.8516	110	1071
0.2907	3.4841	2125	1.1510	0.0156	0.0842	0.0241	0.2546	0.2965	0.8441	0.1133	0.0976	0.9624	0.9717	0.8507	110	1071
0.3301	3.5251	2150	0.9969	0.0156	0.0805	0.0206	0.2647	0.2965	0.8407	0.1032	0.0965	0.9628	0.9767	0.8559	110	1071
0.3632	3.5661	2175	0.9523	0.0156	0.0790	0.0196	0.2639	0.2965	0.8532	0.1040	0.0962	0.9657	0.9694	0.8508	110	1071
0.3278	3.6071	2200	0.8928	0.0156	0.0773	0.0176	0.2860	0.2965	0.8606	0.0950	0.0918	0.9654	0.9758	0.8563	110	1071
0.3182	3.6481	2225	0.9513	0.0156	0.0850	0.0224	0.2578	0.2965	0.8420	0.1101	0.0985	0.9630	0.9715	0.8555	110	1071
0.3473	3.6892	2250	0.9202	0.0156	0.0772	0.0190	0.2846	0.2965	0.8621	0.0865	0.0906	0.9640	0.9724	0.8581	110	1071
0.3730	3.7302	2275	1.0620	0.0156	0.0856	0.0237	0.2495	0.2965	0.8362	0.1184	0.0997	0.9638	0.9698	0.8569	110	1071
0.3483	3.7712	2300	0.9395	0.0156	0.0784	0.0193	0.2917	0.2965	0.8590	0.0908	0.0918	0.9606	0.9722	0.8562	110	1071
0.3351	3.8122	2325	0.8791	0.0156	0.0798	0.0182	0.2721	0.2965	0.8487	0.1046	0.0951	0.9656	0.9780	0.8594	110	1071
0.2287	3.8532	2350	1.0657	0.0156	0.0833	0.0209	0.2654	0.2965	0.8452	0.1026	0.0984	0.9607	0.9695	0.8484	110	1071
0.4124	3.8943	2375	0.9188	0.0156	0.0821	0.0197	0.2738	0.2965	0.8496	0.0981	0.0966	0.9618	0.9745	0.8523	110	1071
0.3117	3.9353	2400	0.9499	0.0156	0.0769	0.0180	0.2844	0.2965	0.8598	0.0949	0.0918	0.9656	0.9722	0.8510	110	1071
0.3350	3.9763	2425	1.0307	0.0156	0.0805	0.0217	0.2675	0.2965	0.8434	0.1004	0.0949	0.9611	0.9752	0.8576	110	1071
0.3342	4.0	2440	0.8972	0.0156	0.0784	0.0179	0.2938	0.2965	0.8556	0.0905	0.0864	0.9672	0.9703	0.8578	110	1071

Framework versions

—PEFT 0.19.1
—Transformers 5.7.0
—Pytorch 2.11.0+cu130
—Datasets 4.8.5
—Tokenizers 0.22.2