Datasetpublic

YuvrajSingh9886/jetson-non-reasoning-benchmark-ollama-15w

Tiny LLM Benchmark — Jetson Orin Nano Super 8GB Date: 2026-06-07 02:35Backends: ollamaSweep: prompt ∈ {128,512,1024,2048} tok × gen ∈ {64,128,256} tokArtifacts: /home/yuvrajsingh/Desktop/benchmark/smolbenchmark/non-reasoning-models/artifacts/blog-all-20260606-0139-15w Full Results — ollama Power = VDD_CPU_GPU_CV avg over aiperf window. Model Quant ISL OSL OSL mis% TTFT avg p50 p90 p99 T2T avg p50 p90 p99 ITL avg p50 p90 p99 Tok/s Req/s E2E avg p50 p90 p99… See the full description on the dataset page: https://huggingface.co/datasets/YuvrajSingh9886/jetson-non-reasoning-benchmark-ollama-15w.

sourceHugging Facecc-by-4.0updated 10d agoView on Hugging Face

0likes595downloads

Dataset Card

Tiny LLM Benchmark — Jetson Orin Nano Super 8GB

Date: 2026-06-07 02:35 Backends: ollama Sweep: prompt ∈ {128,512,1024,2048} tok × gen ∈ {64,128,256} tok Artifacts: /home/yuvrajsingh/Desktop/benchmark/smolbenchmark/non-reasoning-models/artifacts/blog-all-20260606-0139-15w

Full Results — ollama

Power = VDD\CPU\GPU\_CV avg over aiperf window.

Model	Quant	ISL	OSL	OSL mis%	TTFT avg	p50	p90	p99	T2T avg	p50	p90	p99	ITL avg	p50	p90	p99	Tok/s	Req/s	E2E avg	p50	p90	p99	RL avg	p50	p90	p99	Prefill avg	p50	p90	p99	Power (W)	Tok/J
gemma3-1b	Q4KM	128	629.8	883.98	397.6	398.8	405.0	408.8	42.39	42.75	43.01	44.08	40.86	40.85	40.95	41.07	24.48	0.038	23.93	24.15	24.22	24.25	26089.1	25763.9	35530.0	39290.4	322.0	321.0	329.3	331.8	4.64	5.277
gemma3-1b	Q4KM	512	795.4	1142.81	639.9	640.6	645.6	646.0	42.30	42.94	43.71	44.21	40.94	40.95	40.99	41.00	24.42	0.030	23.96	23.98	24.07	24.11	33166.9	32156.7	41279.9	46181.2	800.1	799.2	808.8	810.0	4.64	5.266
gemma3-1b	Q4KM	1024	927.9	1349.77	969.9	967.7	980.8	986.2	43.11	43.19	44.03	44.85	41.06	41.06	41.10	41.13	24.35	0.026	23.74	23.72	23.93	23.98	39024.7	37172.1	50520.2	59960.9	1055.8	1058.2	1065.6	1068.9	4.64	5.251
gemma3-1b	Q4KM	2048	884.5	1282.03	1630.0	1628.5	1639.7	1643.8	44.81	44.16	49.09	51.71	41.16	41.17	41.19	41.22	24.29	0.026	23.18	23.15	23.63	23.73	37997.4	33568.4	58395.1	68254.4	1256.4	1257.6	1262.3	1263.1	4.64	5.238
gemma3-1b	Q4KM	128	637.0	397.62	394.8	394.3	400.7	402.7	42.83	43.01	43.30	46.34	40.81	40.81	40.87	40.93	24.50	0.038	24.05	24.19	24.23	24.25	26358.5	28130.4	33908.5	38201.2	324.2	324.7	329.9	330.8	4.64	5.283
gemma3-1b	Q4KM	512	878.1	586.05	639.6	637.1	648.5	651.1	43.53	43.17	47.65	48.18	40.94	40.93	40.98	41.11	24.43	0.027	24.01	24.03	24.10	24.18	36547.9	35456.2	42243.2	57651.1	800.6	803.6	810.3	811.9	4.64	5.266
gemma3-1b	Q4KM	1024	895.0	599.18	968.8	969.7	973.5	982.6	43.48	43.33	44.00	49.21	41.04	41.03	41.10	41.12	24.37	0.027	23.75	23.77	23.88	23.91	37655.1	37089.3	46070.2	50588.0	1057.0	1056.0	1065.6	1066.6	4.64	5.254
gemma3-1b	Q4KM	2048	956.2	647.03	1631.1	1629.0	1641.9	1646.8	43.56	43.97	44.89	48.84	41.21	41.16	41.21	41.94	24.27	0.024	23.26	23.23	23.55	23.63	41032.0	37519.8	52972.9	71448.6	1255.6	1257.2	1261.3	1262.7	4.64	5.232
gemma3-1b	Q4KM	128	699.9	173.38	393.4	391.7	399.9	404.3	43.57	43.10	44.97	48.76	40.84	40.84	40.89	40.99	24.49	0.034	24.03	24.21	24.25	24.27	28936.5	29979.9	36635.1	39279.9	325.4	326.8	329.4	329.8	4.64	5.279
gemma3-1b	Q4KM	512	888.8	247.19	639.3	638.3	645.7	651.5	43.35	43.29	44.39	46.08	40.97	40.94	41.04	41.51	24.41	0.027	23.98	24.04	24.11	24.14	37017.8	38592.2	45373.4	53599.7	800.9	802.2	808.0	810.7	4.64	5.262
gemma3-1b	Q4KM	1024	833.0	225.41	964.5	963.7	971.1	977.9	44.07	43.51	44.91	50.30	41.07	41.04	41.18	41.23	24.35	0.028	23.69	23.70	23.81	23.92	35131.7	35095.6	40642.3	50427.5	1061.7	1062.5	1067.7	1068.8	4.64	5.250
gemma3-1b	Q4KM	2048	1280.4	400.16	1630.0	1628.8	1640.4	1641.8	44.40	44.27	45.10	49.93	41.18	41.14	41.22	41.63	24.29	0.018	23.33	23.22	23.93	24.06	54342.4	35885.9	111794.7	146432.7	1256.5	1257.4	1262.8	1264.0	4.64	5.236
gemma3-4b	Q4KM	128	1061.0	1557.81	1870.5	1870.5	1870.5	1870.5	130.89	130.89	130.89	130.89	166.00	166.00	166.00	166.00	6.02	0.006	5.97	5.97	5.97	5.97	177833.5	177833.5	177833.5	177833.5	68.4	68.4	68.4	68.4	3.09	1.952
gemma3-4b	Q4KM	512	893.0	1295.31	2409.3	2409.3	2409.3	2409.3	136.55	136.55	136.55	136.55	167.80	167.80	167.80	167.80	5.96	0.007	5.87	5.87	5.87	5.87	152088.7	152088.7	152088.7	152088.7	212.5	212.5	212.5	212.5	3.09	1.931
gemma3-4b	Q4KM	1024	872.0	1262.50	3403.7	3403.7	3403.7	3403.7	276.28	276.28	276.28	276.28	170.00	170.00	170.00	170.00	5.88	0.007	5.76	5.76	5.76	5.76	151473.0	151473.0	151473.0	151473.0	300.9	300.9	300.9	300.9	3.09	1.907
gemma3-4b	Q4KM	2048	956.0	1393.75	6007.3	6007.3	6007.3	6007.3	168.03	168.03	168.03	168.03	174.95	174.95	174.95	174.95	5.72	0.006	5.52	5.52	5.52	5.52	173080.3	173080.3	173080.3	173080.3	340.9	340.9	340.9	340.9	3.09	1.853
gemma3-4b	Q4KM	128	918.0	617.19	1312.9	1312.9	1312.9	1312.9	135.33	135.33	135.33	135.33	163.58	163.58	163.58	163.58	6.11	0.007	6.07	6.07	6.07	6.07	151320.3	151320.3	151320.3	151320.3	97.5	97.5	97.5	97.5	3.09	1.981
gemma3-4b	Q4KM	512	961.0	650.78	2217.3	2217.3	2217.3	2217.3	191.93	191.93	191.93	191.93	171.47	171.47	171.47	171.47	5.83	0.006	5.76	5.76	5.76	5.76	166830.1	166830.1	166830.1	166830.1	230.9	230.9	230.9	230.9	3.09	1.890
gemma3-4b	Q4KM	1024	963.0	652.34	3547.9	3547.9	3547.9	3547.9	378.55	378.55	378.55	378.55	174.22	174.22	174.22	174.22	5.74	0.006	5.63	5.63	5.63	5.63	171150.7	171150.7	171150.7	171150.7	288.6	288.6	288.6	288.6	3.09	1.860
gemma3-4b	Q4KM	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	3.09	—
gemma3-4b	Q4KM	128	1010.0	294.53	1336.4	1336.4	1336.4	1336.4	166.16	166.16	166.16	166.16	163.26	163.26	163.26	163.26	6.13	0.006	6.08	6.08	6.08	6.08	166065.9	166065.9	166065.9	166065.9	95.8	95.8	95.8	95.8	3.09	1.985
gemma3-4b	Q4KM	512	946.0	269.53	2389.0	2389.0	2389.0	2389.0	187.02	187.02	187.02	187.02	167.30	167.30	167.30	167.30	5.98	0.006	5.89	5.89	5.89	5.89	160486.9	160486.9	160486.9	160486.9	214.3	214.3	214.3	214.3	3.09	1.937
gemma3-4b	Q4KM	1024	1030.0	302.34	3477.8	3477.8	3477.8	3477.8	155.16	155.16	155.16	155.16	169.97	169.97	169.97	169.97	5.88	0.006	5.77	5.77	5.77	5.77	178376.2	178376.2	178376.2	178376.2	294.4	294.4	294.4	294.4	3.09	1.907
gemma3-4b	Q4KM	2048	757.0	195.70	6044.5	6044.5	6044.5	6044.5	140.76	140.76	140.76	140.76	172.93	172.93	172.93	172.93	5.78	0.007	5.53	5.53	5.53	5.53	136777.1	136777.1	136777.1	136777.1	338.8	338.8	338.8	338.8	3.09	1.874
lfm2.5-1.2b	Q4KM	128	441.4	589.61	242.1	240.4	259.1	276.1	48.89	49.39	60.71	63.52	51.37	52.12	52.88	52.91	19.60	0.044	19.40	19.05	19.23	25.96	22891.0	20911.5	32955.4	44503.0	530.8	532.5	562.6	594.7	4.53	4.322
lfm2.5-1.2b	Q4KM	512	624.8	876.17	504.5	504.8	515.3	525.8	52.05	52.15	59.89	66.29	51.81	52.52	53.06	53.90	19.43	0.030	19.11	18.75	18.92	25.54	32826.1	30635.6	47732.1	65314.2	1015.4	1014.4	1036.1	1065.2	4.53	4.285
lfm2.5-1.2b	Q4KM	1024	652.6	919.77	859.9	858.4	870.2	886.5	51.25	50.88	62.37	64.22	51.55	52.45	52.73	52.85	19.57	0.029	19.01	18.61	18.84	25.75	34716.6	35288.1	50046.9	53647.0	1191.1	1192.9	1208.8	1221.1	4.53	4.315
lfm2.5-1.2b	Q4KM	2048	667.7	943.28	1576.0	1576.8	1586.1	1588.4	50.37	51.55	55.81	57.78	51.95	52.83	53.04	53.40	19.37	0.028	18.49	18.15	18.49	24.02	36258.1	36261.7	44747.1	58129.9	1299.6	1298.8	1311.6	1314.4	4.53	4.270
lfm2.5-1.2b	Q4KM	128	480.3	275.23	242.9	246.0	257.3	262.9	51.91	50.14	62.59	63.21	51.46	52.30	52.88	53.08	19.61	0.040	19.43	18.95	19.13	27.21	24675.2	25913.9	37812.4	41163.0	528.5	520.4	557.7	596.0	4.53	4.323
lfm2.5-1.2b	Q4KM	512	511.4	299.57	507.0	508.1	520.9	523.0	50.08	51.10	59.23	68.19	51.46	52.46	52.94	53.26	19.58	0.037	18.68	18.68	18.96	25.47	27081.1	29717.0	48089.8	51941.0	1010.4	1007.7	1042.2	1051.6	4.53	4.318
lfm2.5-1.2b	Q4KM	1024	599.2	368.16	859.4	858.3	872.2	877.2	50.10	51.26	56.77	62.59	51.43	52.24	52.78	53.68	19.60	0.032	19.02	18.64	18.96	26.25	31477.5	31990.9	45818.7	49318.3	1191.7	1193.0	1209.5	1212.7	4.53	4.323
lfm2.5-1.2b	Q4KM	2048	634.8	395.94	1568.6	1569.5	1579.7	1590.0	51.98	51.73	60.91	63.13	51.83	52.60	53.09	53.28	19.43	0.029	18.44	18.21	18.47	24.51	34467.6	35395.3	49665.0	55372.2	1305.7	1304.8	1317.2	1321.6	4.53	4.285
lfm2.5-1.2b	Q4KM	128	448.4	75.14	238.5	241.9	249.0	251.5	51.67	51.04	61.67	64.26	51.39	52.22	52.73	52.85	19.59	0.044	19.41	19.00	19.28	26.13	22822.0	23046.3	35104.2	39668.9	537.5	529.1	561.9	600.8	4.53	4.321
lfm2.5-1.2b	Q4KM	512	634.6	147.89	505.3	507.5	517.7	525.2	52.26	51.66	62.05	65.90	51.60	52.42	52.89	53.36	19.53	0.030	19.22	18.81	18.95	26.15	33208.7	32556.4	47496.7	55489.0	1014.0	1008.9	1037.4	1073.6	4.53	4.307
lfm2.5-1.2b	Q4KM	1024	668.5	161.15	856.7	856.6	871.5	879.2	51.96	52.47	63.30	64.85	51.60	52.49	52.75	52.86	19.52	0.028	19.01	18.63	18.78	25.77	35167.6	35225.3	48077.8	53966.3	1195.5	1195.5	1213.9	1221.3	4.53	4.305
lfm2.5-1.2b	Q4KM	2048	711.7	178.01	1570.7	1571.2	1579.9	1594.7	49.71	50.04	58.93	65.66	51.68	52.45	53.16	53.36	19.51	0.026	18.63	18.25	18.59	25.37	38389.7	38067.8	54530.7	59245.5	1303.9	1303.5	1314.9	1319.8	4.53	4.303
lfm2.5-350m	Q4KM	128	209.6	227.50	168.2	168.1	180.2	191.2	40.69	39.33	48.40	54.23	40.91	41.96	42.54	43.64	24.77	0.113	23.36	23.36	24.01	36.03	8770.1	7287.8	18594.2	24100.4	764.5	761.6	822.7	872.9	3.65	6.794
lfm2.5-350m	Q4KM	512	384.8	501.17	268.4	269.1	278.1	282.5	41.26	40.80	52.31	55.75	40.63	41.80	42.23	42.37	25.20	0.062	24.28	23.63	23.99	41.93	16008.0	17326.3	23023.8	28917.8	1910.3	1902.8	1969.0	2134.7	3.65	6.910
lfm2.5-350m	Q4KM	1024	450.6	604.14	390.6	389.6	407.5	414.1	42.01	43.17	50.25	51.02	40.94	41.70	42.21	45.42	24.98	0.054	24.25	23.46	23.88	41.23	18487.1	16828.0	31762.2	34127.0	2623.6	2628.3	2709.2	2728.3	3.65	6.850
lfm2.5-350m	Q4KM	2048	481.6	652.42	661.9	660.8	671.1	680.5	39.53	39.30	48.97	55.96	41.20	42.16	43.17	43.88	24.83	0.050	23.16	22.73	23.35	41.17	19945.3	15885.4	32699.7	46055.0	3094.8	3099.4	3146.8	3156.1	3.65	6.810
lfm2.5-350m	Q4KM	128	250.3	95.55	165.4	167.2	173.2	183.6	39.01	38.74	48.44	54.21	41.31	41.82	42.72	47.77	24.62	0.096	23.77	23.57	24.12	37.82	10334.2	9272.0	17673.7	28057.8	775.9	765.5	836.3	853.1	3.65	6.751
lfm2.5-350m	Q4KM	512	409.3	219.77	265.3	266.5	277.5	280.9	43.10	43.13	49.22	57.21	41.36	42.03	43.28	43.58	24.56	0.057	24.07	23.49	23.77	36.71	17358.3	16321.4	28290.2	34180.9	1932.8	1920.9	2030.6	2044.6	3.65	6.736
lfm2.5-350m	Q4KM	1024	542.4	323.75	391.0	391.1	399.3	408.6	35.16	38.26	44.90	48.93	41.03	42.02	42.71	43.36	25.07	0.044	24.50	23.42	23.72	43.36	22783.8	20891.1	38551.0	51473.8	2619.8	2618.2	2681.2	2736.4	3.65	6.876
lfm2.5-350m	Q4KM	2048	420.1	228.24	660.9	660.4	674.0	676.8	39.55	38.92	47.64	51.22	41.42	42.45	43.05	43.86	24.70	0.055	23.25	22.73	23.22	38.48	18176.1	17118.7	35165.8	40398.4	3099.7	3101.3	3160.0	3212.1	3.65	6.774
lfm2.5-350m	Q4KM	128	240.9	-5.90	168.1	165.4	176.4	187.0	41.68	43.18	51.53	57.15	40.93	42.02	43.04	44.86	24.81	0.097	23.46	23.44	23.85	33.47	10254.3	10704.0	19700.8	24292.1	762.8	773.7	791.1	815.1	3.65	6.806
lfm2.5-350m	Q4KM	512	352.1	37.56	266.2	264.4	286.3	287.5	40.55	39.90	46.88	51.39	41.20	41.95	43.11	45.00	24.62	0.067	23.39	23.44	23.88	35.79	14855.5	14856.5	24871.8	39756.7	1927.7	1936.6	2033.6	2106.2	3.65	6.752
lfm2.5-350m	Q4KM	1024	422.2	64.92	394.4	391.7	404.4	426.8	40.97	42.60	50.72	54.19	40.65	41.73	42.55	42.73	25.03	0.057	23.39	23.39	23.59	38.15	17598.3	15567.6	31507.3	33377.4	2598.4	2614.3	2666.0	2713.4	3.65	6.864
lfm2.5-350m	Q4KM	2048	424.6	65.86	661.6	662.2	672.9	690.2	38.97	41.57	47.36	50.87	41.22	42.02	43.12	44.72	24.81	0.056	22.80	22.94	23.45	40.36	17878.0	17353.4	27074.9	45228.9	3096.2	3092.9	3150.1	3177.7	3.65	6.805
llama3.2-1b	Q4KM	128	315.4	392.81	374.5	359.0	378.7	602.2	32.11	32.24	32.74	33.31	30.85	30.81	31.00	31.08	32.41	0.099	31.11	31.32	31.69	31.69	10075.4	10566.1	13777.9	15660.7	347.9	356.5	364.6	366.8	5.49	5.904
llama3.2-1b	Q4KM	512	392.2	512.81	623.1	620.9	627.8	668.6	32.46	32.44	32.70	32.98	31.12	31.10	31.18	31.32	32.14	0.078	30.47	30.69	30.96	31.08	12801.4	12810.1	16842.0	19957.7	822.1	824.7	831.8	837.6	5.49	5.853
llama3.2-1b	Q4KM	1024	678.0	959.30	964.3	963.1	969.2	978.9	32.84	32.84	33.03	33.18	31.52	31.47	31.65	32.01	31.73	0.045	29.92	30.18	30.51	30.88	22386.4	18256.6	26015.3	88293.3	1062.0	1063.2	1067.2	1067.5	5.49	5.779
llama3.2-1b	Q4KM	2048	623.4	873.98	1663.8	1663.1	1667.8	1671.2	33.55	33.44	34.22	34.70	32.08	32.04	32.24	32.33	31.17	0.046	27.77	28.29	29.62	30.18	21661.3	17388.4	32933.3	68554.4	1230.9	1231.4	1233.4	1234.3	5.49	5.678
llama3.2-1b	Q4KM	128	289.8	126.41	356.0	356.1	359.2	371.1	32.37	32.33	32.95	33.30	30.83	30.83	30.98	30.99	32.43	0.108	30.89	31.26	31.67	31.78	9264.1	8992.5	13824.4	19808.1	359.6	359.5	364.7	366.2	5.49	5.907
llama3.2-1b	Q4KM	512	447.8	249.80	622.6	621.5	630.3	636.7	32.52	32.54	32.69	32.78	31.15	31.13	31.27	31.37	32.10	0.069	30.68	30.84	31.07	31.22	14540.9	15559.7	17663.5	22933.3	822.4	823.8	829.6	830.9	5.49	5.847
llama3.2-1b	Q4KM	1024	555.1	333.67	962.8	961.8	967.0	974.2	32.77	32.80	33.01	33.06	31.48	31.46	31.60	31.77	31.77	0.054	29.81	30.09	30.58	30.71	18429.7	17911.6	27545.4	34000.8	1063.5	1064.6	1067.3	1070.9	5.49	5.786
llama3.2-1b	Q4KM	2048	415.2	224.38	1664.7	1663.3	1670.3	1676.5	33.62	33.58	34.27	34.54	32.11	32.07	32.21	32.55	31.15	0.067	26.87	27.59	29.13	29.26	14968.7	14007.9	25192.8	27815.6	1230.3	1231.3	1233.6	1236.2	5.49	5.673
llama3.2-1b	Q4KM	128	250.6	-2.13	357.2	357.9	361.3	364.6	32.33	32.38	32.52	32.62	30.87	30.83	30.94	31.33	32.40	0.124	29.99	31.13	31.60	31.67	8052.3	8346.3	12759.5	14270.4	358.4	357.6	365.5	368.2	5.49	5.901
llama3.2-1b	Q4KM	512	392.2	53.22	621.9	621.9	624.5	628.5	32.54	32.53	32.92	33.23	31.13	31.12	31.23	31.24	32.12	0.078	29.94	30.68	31.21	31.24	12810.6	12835.1	22785.7	23787.1	823.4	823.3	826.7	830.9	5.49	5.850
llama3.2-1b	Q4KM	1024	467.8	82.71	963.5	963.3	967.9	972.7	32.71	32.73	33.00	33.18	31.44	31.43	31.58	31.66	31.81	0.064	29.04	29.66	30.54	30.84	15661.1	13738.2	24150.7	40437.4	1062.8	1063.0	1067.0	1070.6	5.49	5.794
llama3.2-1b	Q4KM	2048	564.7	120.59	1665.0	1664.4	1670.1	1675.8	33.50	33.43	34.33	34.37	32.06	32.05	32.13	32.31	31.19	0.050	27.61	27.69	29.63	30.07	19760.4	14476.0	36507.1	49882.0	1230.0	1230.4	1233.8	1234.8	5.49	5.681
qwen2.5-0.5b	Q4KM	128	567.2	786.33	278.2	276.4	289.1	292.1	26.31	26.38	26.80	26.96	24.25	24.17	24.85	25.04	41.25	0.070	38.58	39.47	40.42	40.49	14273.4	5667.4	16678.4	114880.3	460.4	463.1	470.7	472.7	4.38	9.413
qwen2.5-0.5b	Q4KM	512	368.2	475.37	423.5	419.2	434.0	438.7	26.72	26.72	26.95	27.23	24.48	24.42	24.66	24.89	40.85	0.024	38.08	38.76	39.66	40.10	9438.5	7596.3	15857.5	33289.8	1209.3	1221.4	1227.3	1230.0	4.38	9.322
qwen2.5-0.5b	Q4KM	1024	1343.1	1998.53	626.7	625.2	633.5	643.2	27.03	26.92	27.39	28.68	24.89	24.84	25.36	25.60	40.19	0.015	37.32	38.53	39.25	39.34	34566.8	12949.9	95931.3	164477.7	1634.2	1638.0	1648.5	1650.7	4.38	9.171
qwen2.5-0.5b	Q4KM	2048	1023.4	1499.01	1076.7	1076.9	1082.7	1085.1	27.73	27.61	28.24	28.56	25.41	25.42	25.51	25.53	39.35	0.027	35.88	37.36	38.63	38.70	27104.2	20769.5	63501.5	74164.0	1902.1	1901.7	1913.7	1917.3	4.38	8.979
qwen2.5-0.5b	Q4KM	128	374.2	192.34	275.2	274.9	279.7	283.8	26.66	26.71	27.04	27.92	24.15	24.05	24.33	24.87	41.41	0.106	38.75	39.71	40.63	40.75	9394.0	5966.3	12825.0	62247.3	465.3	465.5	474.4	475.7	4.38	9.450
qwen2.5-0.5b	Q4KM	512	544.6	325.49	420.7	420.2	428.5	430.3	26.67	26.58	27.00	27.29	24.49	24.40	24.76	25.14	40.84	0.043	36.78	38.75	39.60	40.17	13906.7	7160.0	20344.3	98575.1	1217.1	1218.4	1235.6	1240.9	4.38	9.320
qwen2.5-0.5b	Q4KM	1024	657.1	413.37	624.7	624.0	630.5	636.2	26.79	26.72	27.18	27.27	24.74	24.68	24.88	25.31	40.42	0.026	37.73	37.75	39.28	39.33	17010.7	8583.6	28888.8	88600.3	1639.3	1641.1	1656.2	1658.4	4.38	9.223
qwen2.5-0.5b	Q4KM	2048	1746.0	1264.06	1073.7	1073.0	1080.7	1084.3	27.60	27.70	27.92	28.45	25.43	25.45	25.52	25.56	39.33	0.022	36.93	38.04	38.82	38.93	45514.5	34829.7	99309.5	122642.5	1907.5	1908.7	1917.2	1928.3	4.38	8.974
qwen2.5-0.5b	Q4KM	128	204.6	-20.07	276.9	275.6	284.6	286.8	26.68	26.58	27.03	27.62	24.09	24.03	24.28	24.33	41.52	0.068	38.32	39.18	40.39	40.55	5181.1	4620.3	8844.1	9839.2	462.4	464.4	473.4	480.3	4.38	9.474
qwen2.5-0.5b	Q4KM	512	364.1	42.21	420.8	420.8	426.2	429.2	26.80	26.64	27.40	28.21	24.42	24.41	24.56	24.64	40.95	0.034	38.05	39.06	39.92	40.03	9300.5	8545.6	17381.1	24365.5	1217.0	1216.6	1236.9	1242.7	4.38	9.345
qwen2.5-0.5b	Q4KM	1024	1150.9	349.59	624.0	623.0	631.9	638.5	26.93	26.97	27.20	27.44	24.86	24.78	25.20	25.31	40.23	0.034	37.50	38.55	39.36	39.37	29502.2	11880.8	63483.2	148049.6	1641.1	1643.7	1661.7	1666.3	4.38	9.179
qwen2.5-0.5b	Q4KM	2048	755.7	195.20	1073.4	1072.7	1081.6	1087.1	27.41	27.74	28.04	28.17	25.37	25.36	25.48	25.51	39.41	0.049	34.99	36.03	38.42	38.74	20266.5	12640.1	51543.3	84557.3	1908.0	1909.2	1921.1	1925.5	4.38	8.993
qwen3-0.6b	Q8_0	128	651.2	917.50	270.5	266.4	274.8	327.6	27.90	28.16	28.88	29.16	26.69	26.68	26.87	27.10	37.47	0.057	36.92	36.91	37.07	37.11	17649.9	17442.5	22011.0	28743.7	474.5	480.4	486.7	488.6	4.68	8.005
qwen3-0.6b	Q8_0	512	886.2	1284.77	462.1	459.1	471.9	494.1	30.20	29.25	35.11	36.17	27.78	27.69	28.14	28.44	36.00	0.040	35.32	35.39	35.50	35.52	25119.5	23119.8	35962.9	39800.8	1108.4	1115.2	1127.4	1130.1	4.68	7.689
qwen3-0.6b	Q8_0	1024	940.5	1369.53	711.9	711.9	718.4	721.5	30.53	30.42	32.00	33.35	28.94	28.83	29.45	30.00	34.57	0.036	33.57	33.64	33.75	33.80	28089.3	23702.9	43662.9	69059.2	1438.4	1438.3	1450.5	1454.7	4.68	7.384
qwen3-0.6b	Q8_0	2048	1256.8	1863.75	1262.2	1262.4	1267.7	1270.9	33.25	33.21	33.76	34.72	30.79	30.78	30.89	30.92	32.48	0.025	31.00	30.88	31.92	31.99	39992.8	24722.5	84719.2	115309.8	1622.5	1622.3	1630.4	1634.0	4.68	6.939
qwen3-0.6b	Q8_0	128	744.5	481.60	268.6	268.0	274.5	282.9	28.58	28.52	29.09	29.67	26.79	26.80	27.07	27.21	37.33	0.049	36.84	36.88	37.01	37.12	20229.4	19639.8	27998.1	31192.3	476.8	477.5	485.6	490.2	4.68	7.974
qwen3-0.6b	Q8_0	512	965.6	654.41	460.9	459.6	467.9	477.1	29.29	29.46	30.01	30.85	27.92	27.82	28.37	29.20	35.82	0.036	35.15	35.29	35.49	35.51	27592.6	25777.4	37902.9	69704.2	1111.2	1114.1	1132.0	1132.5	4.68	7.652
qwen3-0.6b	Q8_0	1024	1072.8	738.16	715.0	714.6	720.1	725.5	31.06	30.88	31.40	35.56	28.97	28.92	29.47	30.16	34.53	0.031	33.55	33.65	33.73	33.77	32078.6	27278.5	45149.0	100046.6	1432.2	1433.0	1444.2	1447.6	4.68	7.376
qwen3-0.6b	Q8_0	2048	1225.0	857.07	1262.1	1261.8	1270.6	1272.9	33.68	33.40	34.34	37.04	30.76	30.78	30.87	30.88	32.51	0.026	31.05	31.11	31.96	32.02	38975.0	29104.6	96019.8	101146.5	1622.7	1623.1	1630.8	1632.1	4.68	6.944
qwen3-0.6b	Q8_0	128	771.5	201.39	267.4	264.6	275.1	277.9	28.90	28.40	29.97	32.91	26.80	26.77	27.08	27.33	37.31	0.048	36.85	36.92	37.01	37.06	20966.5	20302.9	28533.5	34798.9	478.9	483.8	489.4	489.9	4.68	7.970
qwen3-0.6b	Q8_0	512	904.0	253.12	459.1	457.7	464.9	469.9	29.77	29.57	30.77	38.46	27.81	27.77	28.21	28.56	35.97	0.039	35.27	35.34	35.48	35.50	25679.0	24323.5	36435.6	45491.4	1115.3	1118.7	1128.4	1132.0	4.68	7.683
qwen3-0.6b	Q8_0	1024	827.3	223.16	715.0	713.3	721.9	726.9	30.38	30.58	30.86	31.92	28.83	28.81	29.15	29.33	34.69	0.041	33.65	33.68	33.74	33.77	24590.1	23467.9	32414.7	39572.6	1432.3	1435.6	1444.0	1445.8	4.68	7.410
qwen3-0.6b	Q8_0	2048	1184.2	362.60	1264.3	1263.4	1269.6	1272.1	33.40	33.47	33.97	34.07	30.80	30.81	30.90	30.93	32.47	0.026	31.00	30.95	31.76	31.99	37752.7	26395.9	62638.9	117483.0	1620.0	1621.0	1628.0	1633.4	4.68	6.937
smollm2-135m	Q4KM	128	546.8	754.30	115.5	115.1	117.4	122.4	13.75	13.78	14.28	14.81	10.88	10.85	10.96	11.69	91.96	0.157	85.80	88.65	89.93	89.97	6345.8	2610.6	6236.8	59717.6	1109.1	1112.2	1129.7	1143.3	3.78	24.310
smollm2-135m	Q4KM	512	720.5	1025.78	206.6	205.9	209.2	214.9	13.78	13.71	14.29	14.90	11.14	11.10	11.27	11.80	89.79	0.117	80.96	84.43	86.78	87.16	8533.3	2919.3	11860.6	65215.7	2479.1	2487.2	2507.9	2523.4	3.78	23.737
smollm2-135m	Q4KM	1024	516.3	706.72	332.1	331.8	334.1	339.1	14.04	14.04	14.28	14.41	11.35	11.31	11.49	11.79	88.09	0.160	81.25	81.68	84.44	84.75	6243.7	4373.5	12349.4	23408.9	3083.2	3086.6	3104.9	3106.7	3.78	23.289
smollm2-135m	Q4KM	2048	427.6	568.12	623.7	624.0	625.8	630.6	14.75	14.77	15.01	15.35	11.87	11.87	11.99	12.02	84.24	0.175	72.17	71.85	78.66	80.04	5708.4	4210.0	11269.7	15383.0	3283.7	3282.0	3295.6	3312.0	3.78	22.270
smollm2-135m	Q4KM	128	608.8	375.59	114.7	114.4	116.3	122.3	13.90	13.93	14.33	14.66	10.87	10.82	10.98	11.64	92.04	0.142	87.72	89.05	90.19	90.87	7010.3	3694.0	6634.5	62118.2	1116.7	1119.3	1144.0	1145.2	3.78	24.333
smollm2-135m	Q4KM	512	873.1	582.11	206.3	205.2	209.6	212.9	13.87	13.99	14.14	14.38	11.15	11.11	11.26	11.77	89.73	0.097	84.05	85.43	87.27	87.44	10324.0	3774.5	6877.0	108058.4	2482.3	2495.2	2507.0	2530.0	3.78	23.721
smollm2-135m	Q4KM	1024	301.1	135.23	331.9	331.1	334.7	338.1	13.96	14.04	14.40	14.81	11.23	11.24	11.30	11.34	89.02	0.268	79.46	80.99	83.73	84.49	3707.6	3702.7	5742.6	7067.3	3085.0	3092.7	3101.9	3121.1	3.78	23.533
smollm2-135m	Q4KM	2048	527.5	312.15	624.3	624.1	628.1	630.7	14.82	14.80	15.06	15.41	11.89	11.89	11.96	12.04	84.13	0.144	72.78	74.26	78.32	81.25	6915.2	5190.1	10068.7	29733.9	3280.3	3281.3	3297.9	3306.9	3.78	22.242
smollm2-135m	Q4KM	128	573.8	124.14	114.3	113.6	116.2	121.3	14.01	13.93	14.30	15.61	10.82	10.77	10.88	11.60	92.45	0.151	88.39	88.57	90.28	90.40	6612.2	2701.3	6163.9	62615.3	1120.3	1126.6	1143.3	1146.5	3.78	24.440
smollm2-135m	Q4KM	512	661.4	158.34	207.9	207.9	213.2	215.1	13.99	13.94	14.30	14.62	11.07	11.02	11.15	11.83	90.33	0.127	82.08	85.05	86.59	87.65	7841.5	3412.7	7163.1	74437.4	2463.1	2462.8	2510.9	2523.5	3.78	23.880
smollm2-135m	Q4KM	1024	1036.8	305.00	331.9	331.0	335.1	339.2	14.05	14.06	14.32	14.57	11.31	11.24	11.57	11.91	88.47	0.080	79.34	80.82	84.37	84.76	12464.4	3568.1	18910.0	109209.4	3085.7	3093.6	3107.8	3113.1	3.78	23.389
smollm2-135m	Q4KM	2048	429.2	67.66	624.1	623.3	627.6	631.9	14.87	14.90	15.16	15.19	11.87	11.85	12.01	12.03	84.28	0.174	70.24	72.68	79.06	80.57	5725.8	4423.3	12356.0	16668.2	3281.8	3285.8	3295.1	3296.3	3.78	22.282
smollm2-360m	Q8_0	128	380.8	495.00	135.4	135.6	138.4	139.3	21.09	21.11	21.31	21.66	18.26	18.17	18.36	19.56	54.78	0.137	52.52	53.52	54.15	54.20	7285.2	4734.9	9314.5	48971.4	945.6	944.2	965.2	972.1	4.44	12.332
smollm2-360m	Q8_0	512	567.3	786.43	288.4	287.1	291.9	302.9	21.35	21.34	21.59	22.06	18.76	18.63	19.12	20.21	53.32	0.088	49.74	50.81	51.79	51.83	11325.9	5196.2	17666.5	80897.7	1775.7	1783.6	1794.9	1797.0	4.44	12.004
smollm2-360m	Q8_0	1024	569.1	789.14	494.0	493.2	497.2	501.5	22.18	22.20	22.51	23.10	19.65	19.48	20.19	20.63	50.90	0.084	43.85	47.87	49.00	49.16	11835.5	6914.6	23888.1	63118.1	2072.8	2076.2	2084.1	2091.7	4.44	11.459
smollm2-360m	Q8_0	2048	379.7	493.28	974.6	973.5	978.3	994.7	23.38	23.55	23.79	24.25	20.63	20.65	20.73	20.75	48.46	0.113	41.50	42.48	45.15	45.99	8801.7	7699.0	14969.9	20561.6	2101.6	2103.7	2110.4	2117.9	4.44	10.911
smollm2-360m	Q8_0	128	962.5	651.95	135.6	134.4	138.3	144.8	21.20	21.19	21.48	21.74	18.53	18.28	20.16	20.31	54.05	0.052	52.24	53.28	53.78	53.85	19212.8	4596.5	84136.0	121946.4	944.6	952.1	961.3	965.6	4.44	12.169
smollm2-360m	Q8_0	512	1495.9	1068.67	289.0	288.8	291.4	294.9	21.75	21.74	22.00	22.67	19.19	18.74	20.33	20.47	52.18	0.033	50.02	50.33	51.63	51.95	30230.8	8025.8	90864.8	132449.0	1771.9	1773.1	1789.0	1801.1	4.44	11.749
smollm2-360m	Q8_0	1024	711.9	456.21	493.6	493.8	495.7	497.2	22.24	22.33	22.65	22.78	19.61	19.50	20.20	20.44	51.00	0.068	47.08	48.42	49.17	49.41	14694.5	8303.1	34826.0	60782.4	2074.6	2073.5	2083.1	2096.2	4.44	11.483
smollm2-360m	Q8_0	2048	697.0	444.57	973.7	973.2	976.1	978.8	23.61	23.63	23.88	24.59	20.60	20.57	20.82	20.85	48.54	0.065	39.25	39.91	47.06	47.56	15418.4	5462.8	47733.3	92769.5	2103.3	2104.4	2106.9	2114.2	4.44	10.929
smollm2-360m	Q8_0	128	535.5	109.16	136.7	134.4	144.3	147.9	21.50	21.43	22.61	24.13	18.42	18.25	18.62	20.14	54.32	0.095	52.27	52.57	53.71	53.77	10525.7	3848.4	13459.0	85790.3	937.6	952.5	960.2	964.3	4.44	12.229
smollm2-360m	Q8_0	512	837.1	226.99	290.1	287.9	294.0	314.2	21.67	21.61	22.08	22.31	19.04	18.67	20.10	20.41	52.59	0.059	46.46	49.26	51.17	51.51	16992.5	4355.1	53181.2	90926.6	1765.8	1778.2	1791.5	1794.9	4.44	11.840
smollm2-360m	Q8_0	1024	825.6	222.49	494.9	493.7	497.7	512.0	22.23	22.43	23.07	23.21	19.63	19.39	20.42	20.64	50.98	0.058	43.38	47.26	48.82	49.39	17193.1	5670.9	44364.3	131869.9	2069.3	2073.9	2083.8	2086.5	4.44	11.477
smollm2-360m	Q8_0	2048	878.0	242.95	973.8	973.8	976.6	978.4	23.37	23.44	23.77	23.86	20.63	20.60	20.81	20.81	48.47	0.052	42.15	43.38	46.94	47.57	19171.1	8689.8	43549.5	93820.9	2103.2	2103.0	2108.6	2114.1	4.44	10.912

Best Output Tok/J per Model

Backend	Model	Quant	Best Tok/J	ISL	OSL	Tok/s	Power (W)
ollama	gemma3-1b	Q4KM	5.283	128	637.0	24.50	4.64
ollama	gemma3-4b	Q4KM	1.985	128	1010.0	6.13	3.09
ollama	lfm2.5-1.2b	Q4KM	4.323	128	480.3	19.61	4.53
ollama	lfm2.5-350m	Q4KM	6.910	512	384.8	25.20	3.65
ollama	llama3.2-1b	Q4KM	5.907	128	289.8	32.43	5.49
ollama	qwen2.5-0.5b	Q4KM	9.474	128	204.6	41.52	4.38
ollama	qwen3-0.6b	Q8_0	8.005	128	651.2	37.47	4.68
ollama	smollm2-135m	Q4KM	24.440	128	573.8	92.45	3.78
ollama	smollm2-360m	Q8_0	12.332	128	380.8	54.78	4.44

Thermal Summary

Backend	Model	Avg Power (W)	Avg CPU (°C)	Avg GPU (°C)	Peak TJ (°C)	Throttled
ollama	gemma3-1b	4.64	52.6	54.8	56.0	No
ollama	gemma3-4b	3.09	47.9	49.1	54.0	No
ollama	lfm2.5-1.2b	4.53	49.9	51.8	54.5	No
ollama	lfm2.5-350m	3.65	47.1	48.6	51.2	No
ollama	llama3.2-1b	5.49	52.8	55.2	57.4	No
ollama	qwen2.5-0.5b	4.38	50.6	52.8	54.4	No
ollama	qwen3-0.6b	4.68	53.0	55.1	56.7	No
ollama	smollm2-135m	3.78	48.3	50.4	53.3	No
ollama	smollm2-360m	4.44	50.5	52.7	54.8	No

Generated by `bench-non-reasoning.sh` on 2026-06-07 02:35:27 108 rows | 1 backends | 9 models

License & citation

—Benchmark results and artifacts (aiperf exports, server logs, tegrastats logs, generated reports): CC BY 4.0. Reuse and adaptation allowed, including commercially, provided you credit Yuvraj Singh, link the license, and indicate changes.
—Harness code that produced these results: Apache-2.0.

If you use these results, please cite:

bibtex

@misc{singh2026smolperfbenchmark,
      title={smolperfbenchmark: On-Device LLM Leaderboard},
      author={Yuvraj Singh},
      year={2026},
      howpublished={\url{https://github.com/YuvrajSingh-mist/smolperfbenchmark}},
}

---

Released freely — support more experiments like it:

![Support me on Ko-fi](https://ko-fi.com/O7W120DR8R) ![GitHub Sponsors](https://github.com/sponsors/YuvrajSingh-mist)

More: <https://huggingface.co/YuvrajSingh9886>