autogluon/fev-bench

1model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,training_corpus_overlap,num_failures2Toto-2.0-1B,92.4731182795699,58.26022624502954,0.0,2.466042572241336,0.0,0.03Toto-2.0-2.5B,92.06989247311829,58.05945381980986,0.0,5.2229363491817695,0.0,0.04Toto-2.0-313m,89.91935483870968,57.694692238950076,0.0,1.0287665867857143,0.0,0.05TimesFM-3,85.75268817204302,57.21227788523926,0.0,10.094451361704877,0.0,0.06Toto-2.0-22m,82.25806451612904,56.07677943690712,0.0,0.20119827167874393,0.0,0.07Chronos-2,80.51075268817203,56.381305085766286,0.0,1.219865412061035,0.0,0.08t0-beta,77.68817204301077,55.94091660566616,0.0,4.521803010272157,0.0,0.09Toto-1.0,73.11827956989248,54.0349203442283,0.0,53.08930704428572,0.041666666666666664,0.010Toto-2.0-4m,71.63978494623656,53.83602437369355,0.0,0.16318737995341615,0.0,0.011TiRex-2,70.96774193548387,53.642114861828674,0.0,0.25031342818069136,0.0,0.012TiRex,65.05376344086022,52.710892944466025,0.0,0.534368017900975,0.041666666666666664,0.013TimesFM-2.5,64.11290322580645,53.090548120238765,0.0,8.086343701896517,0.041666666666666664,0.014TS-ICL,61.155913978494624,52.92655979852465,0.0,3.3554644721210938,0.0,0.015TabPFN-TS-3,55.107526881720425,51.42887909288382,0.0,469.67140080949105,0.0,0.016FlowState,54.03225806451613,47.894556376520924,0.0,10.897278185476189,0.041666666666666664,0.017Moirai-2.0,52.21774193548387,50.17769220777839,0.0,1.7491858470238095,0.25,0.018TFT,47.04301075268817,48.04926998698471,1154.7495615716732,1.2445208104761905,0.0,0.019PatchTST,45.49731182795699,47.5861909531335,1257.4287658338405,2.5367794131867605,0.0,0.020citras-fm,44.892473118279575,47.716360523829216,0.0,2.3104035928338638,0.0,0.021TabPFN-TS,44.22043010752687,47.148822968097335,0.0,300.46992049143205,0.0,0.022Chronos-Bolt,41.196236559139784,46.3687172213469,0.0,0.5042001652436238,0.0,0.023Sundial-Base,39.516129032258064,46.75728672813329,0.0,7.915062387440475,0.041666666666666664,0.024DeepAR,37.83602150537635,42.297561992708474,1872.6321609074305,2.1365429885578084,0.0,8.33333333333333225CatBoost,24.73118279569892,33.304508592545844,151.65138058258532,0.779223560829291,0.0,0.026AutoARIMA,21.63978494623656,25.693557606338423,0.0,16.966581296927224,0.0,16.66666666666666427Stat. Ensemble,19.489247311827956,13.462321560207968,0.0,107.0230329237857,0.0,16.66666666666666428LightGBM,19.22043010752688,28.327668712071517,8.723202911800314,0.5941772642042911,0.0,0.029Seasonal Naive,13.373655913978494,0.0,0.0,0.5226379387028302,0.0,0.030AutoTheta,12.5,-8.679221746369192,0.0,4.489498287272854,0.0,0.031AutoETS,12.029569892473118,-57.991659118752125,0.0,3.5860189728571426,0.0,12.532Naive,7.123655913978497,-93.34435531340436,0.0,0.5319220605854591,0.0,0.033Drift,1.6129032258064515,-103.16658810164463,0.0,0.5097226914596436,0.0,0.034