Datasetpublic

YuvrajSingh9886/jetson-non-reasoning-benchmark-ollama-maxn

Tiny LLM Benchmark — Jetson Orin Nano Super 8GB Date: 2026-06-22 01:58Backends: ollamaSweep: prompt ∈ {128,512,1024,2048} tok × gen ∈ {64,128,256} tokArtifacts: /home/yuvrajsingh/Desktop/benchmark/smolbenchmark/benchmark-jetson-nano-orin-super/non-reasoning-models/artifacts/blog-all-20260621-1401-maxn Full Results — ollama Power = VDD_CPU_GPU_CV avg over aiperf window. Model Quant ISL OSL OSL mis% TTFT avg p50 p90 p99 T2T avg p50 p90 p99 ITL avg p50 p90… See the full description on the dataset page: https://huggingface.co/datasets/YuvrajSingh9886/jetson-non-reasoning-benchmark-ollama-maxn.

sourceHugging Facecc-by-4.0updated 10d agoView on Hugging Face

0likes418downloads

Dataset Card

Tiny LLM Benchmark — Jetson Orin Nano Super 8GB

Date: 2026-06-22 01:58 Backends: ollama Sweep: prompt ∈ {128,512,1024,2048} tok × gen ∈ {64,128,256} tok Artifacts: /home/yuvrajsingh/Desktop/benchmark/smolbenchmark/benchmark-jetson-nano-orin-super/non-reasoning-models/artifacts/blog-all-20260621-1401-maxn

Full Results — ollama

Power = VDD\CPU\GPU\_CV avg over aiperf window.

Model	Quant	ISL	OSL	OSL mis%	TTFT avg	p50	p90	p99	T2T avg	p50	p90	p99	ITL avg	p50	p90	p99	Tok/s	Req/s	E2E avg	p50	p90	p99	RL avg	p50	p90	p99	Prefill avg	p50	p90	p99	Power (W)	Tok/J
gemma3-1b	Q4KM	128	601.2	839.38	304.8	304.2	312.0	315.9	27.71	27.86	28.55	28.57	26.13	26.13	26.20	26.22	38.27	0.062	37.34	37.60	37.76	37.84	15990.0	16608.7	20567.9	25400.8	420.0	420.8	427.3	429.1	7.85	4.877
gemma3-1b	Q4KM	512	777.5	1114.84	458.4	458.0	466.2	469.8	28.16	27.89	28.73	31.69	26.16	26.15	26.18	26.25	38.23	0.048	37.40	37.42	37.56	37.74	20769.0	20566.1	23731.2	33988.7	1117.0	1117.9	1130.8	1135.0	7.85	4.873
gemma3-1b	Q4KM	1024	899.2	1305.00	665.1	663.7	671.5	674.0	28.07	28.06	28.82	30.24	26.25	26.24	26.30	26.32	38.09	0.041	37.04	37.11	37.30	37.35	24245.9	23495.2	31064.8	33876.4	1539.6	1542.8	1550.2	1552.4	7.85	4.855
gemma3-1b	Q4KM	2048	946.0	1378.20	1078.6	1078.5	1084.1	1090.9	28.90	28.60	29.23	35.48	26.32	26.32	26.36	26.39	38.00	0.038	36.27	36.30	36.75	37.34	25952.0	23069.1	31716.9	67396.3	1898.8	1899.0	1910.8	1913.3	7.85	4.842
gemma3-1b	Q4KM	128	699.0	446.09	304.2	303.0	311.8	316.1	28.00	27.91	28.21	32.44	26.11	26.10	26.19	26.26	38.30	0.054	37.53	37.79	37.89	37.91	18529.7	19344.0	23701.9	25125.5	420.9	422.4	428.5	430.7	7.85	4.881
gemma3-1b	Q4KM	512	834.9	552.23	458.9	456.8	467.1	471.0	27.84	27.89	28.99	31.88	26.15	26.14	26.19	26.31	38.24	0.045	37.48	37.49	37.61	37.69	22268.7	21679.3	27335.0	28797.8	1115.8	1120.7	1130.7	1133.8	7.85	4.873
gemma3-1b	Q4KM	1024	893.7	598.20	666.2	666.0	672.8	678.5	27.54	27.97	28.61	31.61	27.05	27.05	27.88	27.95	37.00	0.040	36.01	35.85	37.19	37.47	24818.1	24515.2	29303.8	39085.2	1537.1	1537.6	1551.3	1557.7	7.85	4.716
gemma3-1b	Q4KM	2048	839.5	555.90	1080.8	1081.9	1087.7	1095.8	27.90	27.41	31.70	33.50	27.97	27.95	28.06	28.07	35.76	0.041	34.11	34.15	34.82	34.86	24526.1	23112.4	37185.9	41166.7	1895.0	1892.9	1909.1	1920.2	7.85	4.557
gemma3-1b	Q4KM	128	702.0	174.22	303.8	303.3	310.0	315.4	26.50	27.02	27.73	31.75	27.68	27.67	27.75	27.83	36.12	0.051	35.46	35.64	35.71	35.76	19717.3	20788.7	25512.3	31113.8	421.4	422.0	427.6	429.8	7.85	4.604
gemma3-1b	Q4KM	512	882.5	244.75	457.8	456.7	464.3	468.6	27.16	27.22	27.49	28.20	27.77	27.79	27.84	27.88	36.00	0.040	35.33	35.44	35.52	35.58	24938.1	26034.3	31051.8	32159.6	1118.5	1121.1	1133.0	1134.6	7.85	4.589
gemma3-1b	Q4KM	1024	874.1	241.46	665.6	663.8	674.7	692.1	27.17	27.38	28.04	31.82	27.86	27.85	27.93	28.00	35.89	0.040	34.91	34.94	35.21	35.41	24990.0	24515.3	32686.8	43777.1	1538.8	1542.6	1554.3	1560.3	7.85	4.575
gemma3-1b	Q4KM	2048	1016.6	297.13	1077.5	1075.7	1087.0	1093.6	27.22	27.36	28.30	29.63	27.92	27.91	27.97	28.03	35.81	0.034	34.35	34.28	34.99	35.26	29439.0	24849.7	47983.8	73757.5	1900.8	1903.8	1913.0	1918.8	7.85	4.564
lfm2.5-1.2b	Q4KM	128	410.9	542.03	182.2	181.9	197.6	204.2	40.13	39.70	49.09	52.73	42.20	43.20	44.10	44.44	24.32	0.057	24.06	22.91	23.36	42.11	17634.0	15860.2	29635.3	36402.2	705.0	703.9	756.2	795.9	6.82	3.566
lfm2.5-1.2b	Q4KM	512	614.1	859.53	344.3	343.9	356.7	362.7	39.35	40.04	46.57	50.37	42.49	43.52	43.97	44.55	23.99	0.038	23.67	22.72	23.02	38.61	26182.7	25803.2	37111.6	39550.1	1487.9	1488.9	1531.1	1561.9	6.82	3.517
lfm2.5-1.2b	Q4KM	1024	713.9	1015.39	561.6	562.2	571.2	583.1	43.36	46.60	51.59	55.83	42.51	43.48	44.15	44.27	23.98	0.032	23.45	22.69	22.75	38.04	30824.9	31732.9	46532.0	50334.8	1823.8	1821.4	1865.6	1884.0	6.82	3.515
lfm2.5-1.2b	Q4KM	2048	736.0	1049.92	1004.6	1004.5	1014.7	1029.7	41.62	41.90	49.73	55.15	42.51	43.50	44.07	44.12	23.98	0.031	23.09	22.32	22.67	37.46	32193.3	34386.6	39368.9	46206.4	2038.8	2038.7	2060.1	2084.3	6.82	3.516
lfm2.5-1.2b	Q4KM	128	366.5	186.33	182.2	181.3	197.1	200.4	41.98	42.50	49.58	54.81	42.49	43.25	44.19	44.41	23.93	0.063	23.65	22.87	23.28	37.19	15777.6	14481.5	25219.3	33315.0	705.0	706.1	746.8	808.9	6.82	3.509
lfm2.5-1.2b	Q4KM	512	637.5	398.09	345.2	348.0	353.8	357.2	42.94	44.01	51.99	65.22	42.54	43.40	43.98	45.11	24.00	0.037	23.70	22.74	22.98	39.57	27106.3	22869.2	39410.9	40052.7	1484.3	1471.4	1511.8	1624.0	6.82	3.519
lfm2.5-1.2b	Q4KM	1024	617.5	382.38	561.6	562.5	570.5	578.1	43.28	42.65	51.99	54.83	42.53	43.55	44.10	44.14	24.03	0.037	23.46	22.53	22.78	39.11	26869.4	27689.0	39217.9	45399.9	1823.9	1820.3	1859.5	1908.2	6.82	3.523
lfm2.5-1.2b	Q4KM	2048	706.4	451.84	1004.6	1006.6	1014.0	1017.7	39.80	40.16	47.84	49.70	42.38	43.40	43.67	43.83	24.02	0.032	23.07	22.41	22.66	36.07	31089.1	32192.7	42930.4	49953.1	2038.7	2034.6	2059.2	2074.5	6.82	3.522
lfm2.5-1.2b	Q4KM	128	425.5	66.21	176.4	178.5	183.5	189.6	41.67	42.59	50.13	50.92	42.39	43.57	43.81	43.93	24.11	0.056	23.89	22.79	23.18	40.51	17707.3	15855.4	25388.3	38572.2	727.5	717.2	756.6	835.8	6.82	3.535
lfm2.5-1.2b	Q4KM	512	644.9	151.91	347.7	347.7	355.5	367.7	39.74	40.68	46.49	49.32	42.40	43.46	43.84	44.47	24.13	0.036	23.83	22.75	23.19	40.59	27572.6	26081.2	36488.6	46966.3	1473.2	1472.5	1506.8	1517.2	6.82	3.538
lfm2.5-1.2b	Q4KM	1024	727.0	183.96	563.9	561.9	575.8	590.8	40.67	41.43	45.83	51.31	42.41	43.40	44.08	44.33	24.16	0.032	23.68	22.59	22.92	40.64	31439.1	28627.6	45341.8	49429.2	1816.8	1822.4	1853.6	1877.4	6.82	3.542
lfm2.5-1.2b	Q4KM	2048	735.1	187.17	1002.6	1000.8	1011.3	1028.8	42.09	43.71	49.26	50.82	42.47	43.47	44.16	44.64	24.10	0.031	23.29	22.33	22.58	39.40	32128.6	32238.6	41061.9	45902.2	2042.8	2046.3	2064.5	2074.0	6.82	3.534
lfm2.5-350m	Q4KM	128	260.6	307.27	126.8	126.1	137.6	141.8	31.29	29.73	42.17	45.78	31.20	32.72	33.40	34.11	33.49	0.120	29.90	29.87	30.90	63.43	8282.6	6302.2	16313.1	24352.9	1015.0	1015.4	1088.9	1207.2	5.37	6.236
lfm2.5-350m	Q4KM	512	341.7	433.91	196.6	196.7	208.5	210.0	33.46	32.65	39.66	43.58	32.19	33.20	34.00	34.94	32.51	0.090	30.89	29.68	30.40	66.56	11072.2	10972.9	18593.5	29490.1	2609.0	2602.9	2755.1	2785.7	5.37	6.053
lfm2.5-350m	Q4KM	1024	450.6	604.14	270.4	269.9	282.7	288.1	30.12	27.64	42.45	43.71	32.39	33.33	33.91	34.36	31.92	0.067	31.12	29.42	29.87	58.85	14893.6	12702.0	25146.1	31577.3	3791.7	3794.4	3894.7	4106.3	5.37	5.944
lfm2.5-350m	Q4KM	2048	483.2	655.00	439.2	441.5	449.0	453.6	32.28	32.61	40.03	46.65	32.67	33.65	34.44	35.85	31.87	0.063	29.57	28.94	29.81	62.38	15819.4	16326.1	25294.3	27768.4	4665.1	4638.4	4760.2	4851.7	5.37	5.934
lfm2.5-350m	Q4KM	128	221.2	72.85	129.8	130.5	138.1	144.0	34.91	34.22	45.27	46.80	32.60	33.31	35.32	36.30	32.16	0.134	30.42	29.54	30.21	62.38	7417.3	7803.0	12746.1	24552.5	989.9	981.2	1062.8	1142.2	5.37	5.989
lfm2.5-350m	Q4KM	512	320.2	150.20	193.0	192.1	202.1	207.3	29.32	29.61	36.68	40.35	30.91	32.54	33.43	33.67	33.71	0.096	29.21	29.76	30.49	61.06	10392.9	9772.0	20081.1	23085.1	2656.1	2664.9	2763.1	2847.8	5.37	6.277
lfm2.5-350m	Q4KM	1024	446.7	248.98	272.5	272.4	281.6	285.7	32.01	31.35	39.36	44.26	32.63	33.37	34.18	35.20	31.54	0.067	30.87	29.41	29.86	55.14	14908.2	13373.2	25467.0	28807.9	3760.3	3759.8	3866.4	4006.0	5.37	5.874
lfm2.5-350m	Q4KM	2048	507.5	296.48	442.4	440.8	453.8	460.0	31.93	31.63	43.07	53.15	33.14	33.98	34.98	35.18	31.22	0.060	30.20	28.42	29.41	58.54	16718.3	15653.1	28720.5	33748.1	4631.7	4645.9	4719.0	4795.3	5.37	5.814
lfm2.5-350m	Q4KM	128	263.5	2.93	125.9	124.5	132.8	142.7	33.10	33.27	43.51	51.42	32.66	33.53	34.35	35.12	31.78	0.115	30.84	29.30	30.16	61.15	8683.0	8158.0	16155.6	23964.2	1019.0	1028.4	1066.5	1083.2	5.37	5.919
lfm2.5-350m	Q4KM	512	361.8	41.33	194.9	194.4	207.9	214.2	29.36	30.79	34.71	48.64	31.78	32.87	33.40	34.25	32.64	0.086	31.01	29.76	30.36	62.33	11536.2	10756.8	22299.6	28498.0	2631.7	2633.6	2787.1	2817.2	5.37	6.079
lfm2.5-350m	Q4KM	1024	386.1	50.82	274.6	272.5	284.3	289.6	30.64	30.16	45.73	47.11	32.77	33.49	34.18	36.82	31.75	0.077	29.68	29.44	29.87	60.67	12866.9	14009.7	22231.1	25877.6	3732.0	3758.3	3806.6	3953.9	5.37	5.912
lfm2.5-350m	Q4KM	2048	392.7	53.40	439.2	439.0	446.1	452.3	33.69	32.64	39.64	41.86	32.86	33.59	34.27	35.65	31.56	0.077	30.12	28.81	29.39	59.62	12934.0	12364.8	19253.9	25409.9	4664.3	4665.3	4737.4	4791.8	5.37	5.877
llama3.2-1b	Q4KM	128	295.9	362.27	280.5	275.0	293.9	347.5	20.48	20.50	20.77	20.92	19.34	19.34	19.43	19.57	51.71	0.166	48.59	49.17	50.30	50.47	5988.1	5613.7	9001.3	11988.7	458.5	465.5	476.2	477.4	9.63	5.370
llama3.2-1b	Q4KM	512	448.0	600.00	440.6	438.3	440.7	484.1	20.61	20.63	20.86	21.00	19.50	19.50	19.56	19.64	51.27	0.109	48.53	48.84	49.57	49.80	9161.7	8537.4	13500.0	16115.0	1163.0	1168.1	1177.9	1181.7	9.63	5.326
llama3.2-1b	Q4KM	1024	513.5	702.27	648.6	648.0	652.2	655.0	20.80	20.80	21.01	21.35	19.79	19.69	19.79	21.05	50.56	0.093	45.75	47.50	48.87	49.07	10755.6	9297.9	17902.8	21986.5	1578.8	1580.3	1588.1	1589.9	9.63	5.251
llama3.2-1b	Q4KM	2048	505.2	689.45	1088.2	1088.4	1092.7	1093.8	21.22	21.19	21.58	21.74	20.07	20.06	20.14	20.15	49.82	0.089	43.65	44.36	45.57	48.26	11218.0	9761.5	13036.8	40263.9	1882.0	1881.6	1891.2	1894.1	9.63	5.174
llama3.2-1b	Q4KM	128	293.1	128.98	274.6	273.8	278.3	286.1	20.50	20.46	20.79	20.92	19.38	19.34	19.44	19.76	51.61	0.168	47.74	49.53	50.24	50.46	5927.6	6303.5	8993.9	12286.5	466.3	467.6	474.2	476.9	9.63	5.361
llama3.2-1b	Q4KM	512	453.4	254.18	438.8	438.4	442.4	452.9	20.66	20.69	20.78	20.94	19.53	19.52	19.58	19.64	51.20	0.107	48.62	48.67	49.47	49.83	9275.7	8586.1	12997.9	17313.4	1167.0	1167.8	1179.5	1185.5	9.63	5.318
llama3.2-1b	Q4KM	1024	384.1	200.08	649.5	649.0	653.1	656.5	20.80	20.81	20.97	21.03	19.73	19.70	19.77	20.20	50.68	0.121	45.12	46.60	48.25	48.51	8209.9	7847.5	13272.2	14934.2	1576.5	1577.8	1583.1	1590.8	9.63	5.264
llama3.2-1b	Q4KM	2048	562.2	339.26	1088.4	1088.7	1094.1	1095.9	21.29	21.29	21.46	21.58	20.10	20.10	20.15	20.20	49.76	0.081	44.17	44.62	47.11	47.64	12377.4	10317.9	21262.7	26391.1	1881.7	1881.2	1889.6	1898.2	9.63	5.168
llama3.2-1b	Q4KM	128	288.8	12.79	273.9	273.9	277.0	280.1	20.54	20.52	20.89	21.22	19.33	19.30	19.43	19.48	51.73	0.171	49.16	49.48	50.15	50.21	5839.7	5676.8	8128.0	10708.0	467.5	467.4	473.2	476.9	9.63	5.373
llama3.2-1b	Q4KM	512	468.6	83.05	438.5	437.4	442.8	448.4	20.56	20.56	20.68	20.77	19.51	19.49	19.63	19.73	51.25	0.104	48.78	48.88	49.59	49.99	9566.1	9434.6	12823.7	17784.0	1167.8	1170.5	1178.6	1187.3	9.63	5.323
llama3.2-1b	Q4KM	1024	432.4	68.89	648.7	648.7	653.0	657.1	20.78	20.74	20.99	21.35	19.70	19.67	19.77	19.95	50.76	0.109	46.75	47.18	48.12	48.30	9145.4	9117.2	12264.9	12645.5	1578.5	1578.5	1586.8	1597.1	9.63	5.272
llama3.2-1b	Q4KM	2048	446.4	74.39	1087.2	1086.2	1089.9	1098.4	21.19	21.14	21.47	21.76	20.06	20.05	20.13	20.24	49.85	0.099	43.05	43.93	45.64	47.51	10031.3	8820.6	13160.0	26350.1	1883.8	1885.5	1890.2	1892.1	9.63	5.178
qwen2.5-0.5b	Q4KM	128	245.8	283.98	215.4	214.6	223.6	224.2	17.51	17.54	17.84	17.86	15.56	15.52	15.69	15.87	64.27	0.247	59.40	60.58	62.44	62.76	4023.3	3412.5	7067.2	10476.0	594.4	596.4	605.9	612.4	7.49	8.584
qwen2.5-0.5b	Q4KM	512	1134.3	1672.37	307.4	306.5	315.3	317.6	17.53	17.47	17.81	17.89	15.80	15.71	16.13	16.21	63.28	0.036	55.92	59.92	61.73	63.99	18278.2	5347.6	53111.0	108768.4	1666.2	1670.6	1696.4	1698.3	7.49	8.452
qwen2.5-0.5b	Q4KM	1024	1111.9	1637.42	436.3	435.9	441.4	445.1	17.81	17.77	18.03	18.42	15.96	15.91	16.30	16.35	62.65	0.036	56.64	58.31	60.78	61.14	18453.1	6053.8	47011.3	131097.2	2347.2	2348.9	2371.8	2384.5	7.49	8.368
qwen2.5-0.5b	Q4KM	2048	1515.3	2267.66	719.4	718.3	722.9	728.0	17.95	17.95	18.38	19.07	16.33	16.35	16.40	16.43	61.22	0.039	56.73	56.86	60.18	60.67	25511.1	10335.3	47507.9	147148.6	2847.0	2851.0	2859.9	2871.9	7.49	8.177
qwen2.5-0.5b	Q4KM	128	232.8	81.88	214.0	212.1	217.1	228.9	17.49	17.51	17.81	17.92	15.59	15.58	15.76	15.77	64.14	0.259	57.32	59.64	62.06	62.70	3838.2	2833.9	6346.1	15835.7	598.3	603.4	607.4	611.3	7.49	8.566
qwen2.5-0.5b	Q4KM	512	1203.0	839.80	310.4	306.7	322.1	342.2	17.65	17.67	18.01	18.62	15.84	15.78	16.19	16.22	63.12	0.051	58.73	60.47	61.52	62.02	19670.7	6068.6	65846.3	136216.5	1651.2	1669.5	1700.6	1708.8	7.49	8.431
qwen2.5-0.5b	Q4KM	1024	3197.0	2397.66	437.9	437.3	444.1	446.7	17.68	17.73	18.13	18.17	16.09	16.13	16.27	16.30	62.16	0.016	58.36	61.00	61.31	61.36	52346.2	17808.7	150492.0	167213.2	2338.5	2341.4	2366.3	2380.9	7.49	8.303
qwen2.5-0.5b	Q4KM	2048	1002.9	683.48	721.1	718.9	729.2	739.9	18.09	18.13	18.33	18.35	16.35	16.32	16.50	16.58	61.17	0.058	54.30	55.60	59.97	60.31	17150.2	7534.5	36971.0	75910.5	2840.4	2848.7	2868.9	2873.8	7.49	8.170
qwen2.5-0.5b	Q4KM	128	784.6	206.48	216.1	215.1	223.4	225.4	17.61	17.55	17.88	18.50	15.69	15.62	16.06	16.19	63.75	0.078	57.56	58.81	62.03	62.49	12790.7	2414.7	36556.1	112925.4	592.4	595.1	603.5	609.3	7.49	8.514
qwen2.5-0.5b	Q4KM	512	1171.3	357.56	307.4	306.6	312.9	313.5	17.53	17.57	17.90	18.19	15.81	15.72	16.16	16.25	63.26	0.052	59.89	60.84	61.83	62.06	19122.9	6475.0	41477.8	128168.4	1666.0	1670.1	1691.1	1707.7	7.49	8.449
qwen2.5-0.5b	Q4KM	1024	1565.7	511.62	435.3	433.8	442.9	445.5	17.85	17.72	18.42	18.56	16.02	15.95	16.28	16.39	62.44	0.028	53.95	57.50	61.22	61.30	25800.0	4678.3	81076.2	145575.7	2353.0	2360.7	2388.7	2389.3	7.49	8.340
qwen2.5-0.5b	Q4KM	2048	951.0	271.48	720.1	719.3	726.9	730.7	18.28	18.25	18.70	19.00	16.34	16.34	16.38	16.67	61.20	0.061	55.24	58.38	59.58	60.13	16260.3	14983.9	27572.2	43287.2	2844.3	2847.2	2866.6	2875.9	7.49	8.175
qwen3-0.6b	Q8_0	128	740.6	1057.19	209.8	207.2	211.6	244.9	17.99	18.32	18.55	19.98	17.07	17.05	17.28	17.41	58.58	0.078	57.61	57.69	57.85	57.93	12870.6	12132.2	18708.6	22454.1	611.3	617.7	626.6	631.7	8.09	7.242
qwen3-0.6b	Q8_0	512	851.1	1229.84	331.4	329.5	334.3	367.4	19.46	19.01	20.15	24.61	17.67	17.68	17.83	18.41	56.59	0.065	55.26	55.37	55.50	55.55	15448.4	15241.0	19818.9	37885.9	1546.5	1554.0	1580.6	1583.4	8.09	6.997
qwen3-0.6b	Q8_0	1024	851.3	1230.16	487.9	486.1	493.6	501.5	20.32	19.68	21.74	25.37	18.35	18.33	18.46	18.76	54.50	0.062	52.82	52.86	53.00	53.04	16122.1	15355.4	19712.5	28523.3	2098.9	2106.4	2117.1	2121.4	8.09	6.739
qwen3-0.6b	Q8_0	2048	1589.2	2383.20	838.0	836.1	851.4	862.4	20.99	21.51	21.74	21.85	19.54	19.55	19.61	19.67	51.17	0.031	49.03	49.06	49.97	50.68	31933.5	19700.1	48127.3	136029.2	2444.2	2449.4	2466.7	2471.5	8.09	6.326
qwen3-0.6b	Q8_0	128	711.4	455.78	207.7	207.3	212.1	215.4	18.53	18.46	19.36	21.88	17.06	17.04	17.19	17.24	58.60	0.081	57.65	57.70	57.86	57.98	12343.1	12092.9	15733.4	18476.4	616.3	617.4	628.3	634.5	8.09	7.246
qwen3-0.6b	Q8_0	512	957.6	648.12	329.4	329.3	336.8	339.5	18.88	19.12	19.46	19.75	17.73	17.66	17.97	18.51	56.41	0.057	55.27	55.47	55.57	55.58	17392.5	15402.1	23826.2	43440.8	1554.5	1555.0	1579.4	1583.3	8.09	6.975
qwen3-0.6b	Q8_0	1024	807.3	530.70	488.4	487.8	493.6	498.2	20.02	19.90	20.74	21.18	18.33	18.35	18.46	18.63	54.57	0.065	52.75	52.86	52.93	52.95	15293.6	15395.3	18867.6	24578.0	2096.6	2099.3	2112.6	2119.1	8.09	6.747
qwen3-0.6b	Q8_0	2048	1639.7	1180.98	836.3	835.2	843.4	848.3	21.15	21.49	21.76	22.05	19.52	19.54	19.61	19.63	51.22	0.030	48.86	49.08	50.35	50.61	32918.8	20018.4	68737.8	104931.4	2449.1	2452.2	2467.2	2476.0	8.09	6.333
qwen3-0.6b	Q8_0	128	696.3	171.99	207.4	205.8	214.2	219.6	18.59	18.56	19.02	22.89	17.03	17.05	17.16	17.23	58.72	0.083	57.68	57.68	57.86	58.02	12074.5	12578.6	15344.1	17875.1	617.6	621.8	630.8	633.4	8.09	7.260
qwen3-0.6b	Q8_0	512	935.6	265.47	327.8	327.1	332.8	337.8	19.50	19.18	20.15	24.04	17.71	17.70	17.93	18.21	56.47	0.059	55.30	55.38	55.56	55.68	16941.9	16504.6	22254.3	31697.9	1562.3	1565.2	1580.4	1587.4	8.09	6.982
qwen3-0.6b	Q8_0	1024	821.5	220.88	488.5	487.2	495.4	499.2	19.44	19.80	20.23	20.87	18.30	18.33	18.46	18.81	54.64	0.064	52.72	52.92	53.02	53.06	15566.7	15797.5	19942.0	31648.2	2096.3	2102.0	2118.5	2119.5	8.09	6.756
qwen3-0.6b	Q8_0	2048	1710.2	568.05	837.1	836.7	844.8	845.5	21.76	21.73	22.19	25.82	19.52	19.51	19.59	19.67	51.23	0.029	49.13	48.90	50.58	50.63	34257.0	18922.5	81497.6	95139.8	2446.5	2447.8	2462.7	2465.4	8.09	6.335
smollm2-135m	Q4KM	128	1331.6	1980.59	87.4	86.6	92.7	94.3	9.44	9.40	9.87	10.07	6.97	6.91	7.28	7.51	143.55	0.051	133.29	135.95	140.63	142.73	9944.0	1659.8	19309.9	104666.8	1465.9	1478.2	1522.4	1528.6	6.38	22.492
smollm2-135m	Q4KM	512	1529.7	2290.08	145.6	145.2	147.7	152.0	9.25	9.22	9.49	9.55	7.08	7.01	7.49	7.58	141.28	0.086	132.51	134.37	137.48	137.94	11534.4	2757.2	42996.7	85006.3	3517.7	3526.9	3560.8	3610.0	6.38	22.137
smollm2-135m	Q4KM	1024	507.9	693.59	225.8	225.3	228.4	231.4	9.44	9.49	9.65	10.01	7.21	7.18	7.30	7.46	138.76	0.254	124.44	128.64	131.94	134.37	3918.3	2796.4	8353.6	16044.0	4536.2	4544.5	4585.3	4587.9	6.38	21.742
smollm2-135m	Q4KM	2048	524.4	719.30	411.6	411.8	414.0	417.9	9.93	9.90	10.36	11.02	7.57	7.56	7.66	7.70	132.15	0.227	110.23	117.09	123.72	127.43	4386.9	3495.7	7680.9	15165.2	4975.4	4973.7	5008.0	5024.3	6.38	20.706
smollm2-135m	Q4KM	128	797.8	523.28	87.0	86.3	90.0	92.8	9.35	9.34	9.60	10.34	6.93	6.89	7.01	7.45	144.35	0.169	136.14	137.50	141.12	141.51	5893.9	1751.9	7568.6	44317.8	1472.8	1482.6	1518.8	1521.5	6.38	22.618
smollm2-135m	Q4KM	512	285.9	123.40	145.4	144.1	149.4	153.3	9.33	9.33	9.54	9.71	7.04	7.04	7.08	7.12	141.96	0.461	129.59	132.74	135.08	137.11	2157.0	2005.2	2959.5	5962.1	3523.5	3553.2	3567.3	3581.8	6.38	22.244
smollm2-135m	Q4KM	1024	467.0	264.84	225.5	224.5	226.9	234.6	9.42	9.47	9.64	10.09	7.20	7.17	7.34	7.49	139.01	0.276	125.42	127.79	131.87	132.70	3612.6	2679.9	4322.8	17281.5	4541.5	4561.3	4586.5	4590.6	6.38	21.781
smollm2-135m	Q4KM	2048	367.6	187.23	411.3	410.3	415.4	418.9	9.72	9.76	10.07	10.35	7.53	7.54	7.58	7.60	132.79	0.313	109.88	114.66	122.95	124.96	3177.9	2960.5	5939.7	7650.8	4979.3	4991.3	5013.8	5021.2	6.38	20.806
smollm2-135m	Q4KM	128	473.6	85.02	85.8	85.5	87.4	90.1	9.39	9.40	9.64	10.20	6.92	6.87	6.97	7.37	144.67	0.287	134.04	138.54	140.69	141.16	3474.9	1787.9	2931.1	30765.9	1493.1	1497.8	1521.8	1529.1	6.38	22.667
smollm2-135m	Q4KM	512	738.1	188.32	147.3	147.1	151.6	157.0	9.28	9.35	9.68	9.88	7.11	7.06	7.22	7.58	140.66	0.179	126.19	132.49	135.94	137.10	5560.8	2168.4	7320.9	53066.7	3479.2	3479.8	3562.4	3602.1	6.38	22.039
smollm2-135m	Q4KM	1024	688.6	169.00	225.9	225.7	227.9	230.0	9.35	9.45	9.66	9.95	7.22	7.17	7.44	7.51	138.55	0.189	123.40	126.19	133.21	134.14	5285.3	2238.3	18362.5	22333.3	4533.3	4536.2	4592.4	4634.5	6.38	21.709
smollm2-135m	Q4KM	2048	461.0	80.08	411.4	410.9	413.6	418.1	9.88	9.90	10.06	10.41	7.56	7.56	7.62	7.62	132.36	0.255	110.29	114.23	125.05	126.51	3899.3	2858.2	8343.8	11107.4	4978.4	4984.5	5008.5	5018.1	6.38	20.738
smollm2-360m	Q8_0	128	207.0	223.44	98.4	98.3	99.8	102.4	13.77	13.77	14.12	14.47	11.57	11.55	11.65	11.95	86.42	0.401	80.01	83.57	84.60	85.02	2480.1	2369.1	3985.0	5375.5	1301.3	1302.6	1317.8	1335.9	7.51	11.508
smollm2-360m	Q8_0	512	385.6	502.50	197.8	196.5	201.1	210.0	14.22	14.22	14.43	14.57	11.96	11.91	12.01	12.60	83.65	0.204	76.38	78.69	80.98	82.11	4871.1	3345.8	7224.4	23297.7	2589.2	2606.1	2631.3	2636.9	7.51	11.140
smollm2-360m	Q8_0	1024	1073.5	1577.34	328.7	329.0	331.8	332.0	14.68	14.70	14.92	15.13	12.54	12.41	13.03	13.13	79.75	0.070	72.59	75.15	76.85	76.96	14177.7	5010.8	44194.6	85708.9	3115.3	3112.0	3151.3	3156.5	7.51	10.620
smollm2-360m	Q8_0	2048	356.4	456.88	631.5	631.8	634.5	636.7	15.40	15.41	15.70	15.99	13.09	13.09	13.15	13.20	76.38	0.188	63.70	65.81	70.42	72.90	5295.3	4482.9	8156.5	17274.3	3243.1	3241.6	3258.4	3259.7	7.51	10.171
smollm2-360m	Q8_0	128	775.5	505.82	99.5	98.4	102.4	106.2	14.04	14.09	14.38	14.66	11.70	11.63	11.71	12.81	85.53	0.101	81.54	83.23	84.04	84.18	9893.0	3091.8	4938.1	117298.9	1286.8	1300.2	1315.6	1322.0	7.51	11.389
smollm2-360m	Q8_0	512	1214.3	848.71	197.6	197.6	200.6	207.5	14.33	14.36	14.56	14.85	12.10	11.91	12.76	13.03	82.73	0.063	77.02	77.91	79.85	81.11	15720.3	2571.0	32158.7	143266.9	2591.3	2590.7	2633.9	2635.9	7.51	11.017
smollm2-360m	Q8_0	1024	1247.4	874.53	327.7	327.4	331.0	333.5	14.35	14.64	14.81	15.58	12.59	12.48	13.02	13.10	79.46	0.061	73.78	75.05	76.44	76.82	16457.0	4956.9	45156.4	108173.5	3125.1	3127.5	3148.5	3155.5	7.51	10.581
smollm2-360m	Q8_0	2048	610.8	377.19	629.5	629.4	632.3	633.4	15.45	15.50	15.72	15.89	13.13	13.13	13.18	13.24	76.17	0.115	64.90	69.09	72.56	74.15	8660.4	6575.7	14739.6	32312.6	3253.2	3253.8	3260.4	3265.5	7.51	10.143
smollm2-360m	Q8_0	128	377.2	47.34	99.4	98.4	102.5	106.3	14.26	14.07	14.69	15.79	11.70	11.65	11.78	12.52	85.54	0.214	80.88	82.12	83.87	85.14	4664.9	2310.2	4654.9	37041.2	1288.7	1300.4	1317.0	1327.7	7.51	11.391
smollm2-360m	Q8_0	512	1122.0	338.28	196.8	196.7	199.4	200.6	14.29	14.26	14.55	14.70	12.15	11.93	12.80	12.95	82.38	0.069	76.43	78.80	81.08	81.43	14380.3	4387.4	39101.2	64533.0	2601.6	2603.4	2634.3	2641.4	7.51	10.970
smollm2-360m	Q8_0	1024	1270.0	396.11	328.9	327.7	333.4	342.0	14.40	14.65	14.87	15.22	12.53	12.41	13.07	13.12	79.86	0.060	72.96	74.55	77.06	77.31	16750.8	4030.0	67487.3	97169.1	3113.7	3124.7	3148.1	3153.0	7.51	10.635
smollm2-360m	Q8_0	2048	379.6	48.26	629.8	629.3	632.5	635.2	15.44	15.48	15.65	15.75	13.09	13.09	13.17	13.21	76.42	0.178	62.21	64.42	72.49	73.00	5599.6	3824.8	13424.4	16858.7	3252.1	3254.5	3264.6	3274.1	7.51	10.176

Best Output Tok/J per Model

Backend	Model	Quant	Best Tok/J	ISL	OSL	Tok/s	Power (W)
ollama	gemma3-1b	Q4KM	4.881	128	699.0	38.30	7.85
ollama	lfm2.5-1.2b	Q4KM	3.566	128	410.9	24.32	6.82
ollama	lfm2.5-350m	Q4KM	6.277	512	320.2	33.71	5.37
ollama	llama3.2-1b	Q4KM	5.373	128	288.8	51.73	9.63
ollama	qwen2.5-0.5b	Q4KM	8.584	128	245.8	64.27	7.49
ollama	qwen3-0.6b	Q8_0	7.260	128	696.3	58.72	8.09
ollama	smollm2-135m	Q4KM	22.667	128	473.6	144.67	6.38
ollama	smollm2-360m	Q8_0	11.508	128	207.0	86.42	7.51

Thermal Summary

Backend	Model	Avg Power (W)	Avg CPU (°C)	Avg GPU (°C)	Peak TJ (°C)	Throttled
ollama	gemma3-1b	7.85	61.9	64.2	65.7	No
ollama	gemma3-4b	5.59	53.9	55.0	62.4	No
ollama	lfm2.5-1.2b	6.82	57.1	58.6	62.4	No
ollama	lfm2.5-350m	5.37	53.7	54.8	58.1	No
ollama	llama3.2-1b	9.63	65.1	67.9	70.2	No
ollama	qwen2.5-0.5b	7.49	60.3	62.6	64.0	No
ollama	qwen3-0.6b	8.09	63.5	65.8	67.6	No
ollama	smollm2-135m	6.38	57.9	59.9	62.0	No
ollama	smollm2-360m	7.51	61.1	63.4	65.4	No

Generated by `bench-non-reasoning.sh` on 2026-06-22 01:58:20 96 rows | 1 backends | 8 models

License & citation

—Benchmark results and artifacts (aiperf exports, server logs, tegrastats logs, generated reports): CC BY 4.0. Reuse and adaptation allowed, including commercially, provided you credit Yuvraj Singh, link the license, and indicate changes.
—Harness code that produced these results: Apache-2.0.

If you use these results, please cite:

bibtex

@misc{singh2026smolperfbenchmark,
      title={smolperfbenchmark: On-Device LLM Leaderboard},
      author={Yuvraj Singh},
      year={2026},
      howpublished={\url{https://github.com/YuvrajSingh-mist/smolperfbenchmark}},
}

---

Released freely — support more experiments like it:

![Support me on Ko-fi](https://ko-fi.com/O7W120DR8R) ![GitHub Sponsors](https://github.com/sponsors/YuvrajSingh-mist)

More: <https://huggingface.co/YuvrajSingh9886>