Controlled cannbot comparison

Fresh measurements of the existing cannbot source compiled by our TileLang. No cannbot agent was started. These results do not enter default coverage or the existing 150-row / 49-attained accounting.

D032 device-kernel interval union, 384 MiB eviction before each timed call, ABBA order. Five trials per case, 30 calls per trial. Ratio is the median of the five cannbot/ours ratios; the range shows their minimum and maximum. Individual latency medians need not divide to the median ratio.

Conv2d reuses the original workloads, input generator and tolerances through a separate timing driver because bench1 is correctness-only for that op. Conv2d SOL is unverified. Unsupported source contracts, including bias, are refused and preserved in tooltips and raw data.

N≥5: 13/13 ops · eager 71/95 cases · graph 71/95 cases.

Raw case statistics · Summary

Conv2dFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
bottleneck-expand-1x1	float16	eager	5/5	22.5	28.5	1.2667×	1.2444–1.2778
bottleneck-expand-1x1	float16	graph	5/5	35.25	35.5	1.0142×	1.0035–1.0216
bottleneck-expand-1x1-bias	float16	eager	0/5	— incomplete / refused
bottleneck-expand-1x1-bias	float16	graph	0/5	— incomplete / refused
bottleneck-reduce-1x1	float16	eager	5/5	33.625	28.75	0.85502×	0.85185–0.86245
bottleneck-reduce-1x1	float16	graph	5/5	39.75	39.625	0.99375×	0.98742–1.0063
bottleneck-reduce-1x1-bias	float16	eager	0/5	— incomplete / refused
bottleneck-reduce-1x1-bias	float16	graph	0/5	— incomplete / refused
classifier-1x1	float16	eager	5/5	28.125	76.125	2.6968×	2.303–2.7411
classifier-1x1	float16	graph	5/5	35	77.125	2.2036×	1.9968–2.2962
classifier-1x1-bias	float16	eager	0/5	— incomplete / refused
classifier-1x1-bias	float16	graph	0/5	— incomplete / refused
deeplabv3-aspp-3x3-rate12	float16	eager	0/5	— incomplete / refused
deeplabv3-aspp-3x3-rate12	float16	graph	0/5	— incomplete / refused
default-dense-cube-fp16	float16	eager	5/5	87.5	26	0.29714×	0.29429–0.32429
default-dense-cube-fp16	float16	graph	5/5	87.75	26.5	0.30199×	0.2963–0.34375
dense-cube-bf16	bfloat16	eager	5/5	53.875	39.75	0.73782×	0.6814–0.84028
dense-cube-bf16	bfloat16	graph	5/5	54	38.375	0.71065×	0.66667–0.83796
depthwise-direct-fp16	float16	eager	0/5	— incomplete / refused
depthwise-direct-fp16	float16	graph	0/5	— incomplete / refused
dilation-cube-fp16	float16	eager	0/5	— incomplete / refused
dilation-cube-fp16	float16	graph	0/5	— incomplete / refused
fp32-direct	float32	eager	0/5	— incomplete / refused
fp32-direct	float32	graph	0/5	— incomplete / refused
grouped-direct-fp16	float16	eager	0/5	— incomplete / refused
grouped-direct-fp16	float16	graph	0/5	— incomplete / refused
highres-3x3-s1	float16	eager	5/5	29079	3656.1	0.12573×	0.12567–0.12579
highres-3x3-s1	float16	graph	5/5	29085	3661.9	0.1259×	0.12577–0.12599
highres-3x3-s1-bias	float16	eager	0/5	— incomplete / refused
highres-3x3-s1-bias	float16	graph	0/5	— incomplete / refused
large-grid-depthwise-fp16	float16	eager	0/5	— incomplete / refused
large-grid-depthwise-fp16	float16	graph	0/5	— incomplete / refused
late-stage-1x1	float16	eager	5/5	12.5	20.75	1.66×	1.6–1.7292
late-stage-1x1	float16	graph	5/5	21.25	25.25	1.1792×	1.1647–1.2024
late-stage-1x1-bias	float16	eager	0/5	— incomplete / refused
late-stage-1x1-bias	float16	graph	0/5	— incomplete / refused
midres-5x5-s1	float16	eager	5/5	4193.6	794.62	0.18948×	0.18916–0.19091
midres-5x5-s1	float16	graph	5/5	4199.5	806.12	0.19185×	0.19154–0.1923
midres-5x5-s1-bias	float16	eager	0/5	— incomplete / refused
midres-5x5-s1-bias	float16	graph	0/5	— incomplete / refused
mobilenetv2-depthwise	float16	eager	0/5	— incomplete / refused
mobilenetv2-depthwise	float16	graph	0/5	— incomplete / refused
resnet-1x1	bfloat16	eager	5/5	15.25	18.25	1.1967×	1.1774–1.2131
resnet-1x1	bfloat16	graph	5/5	36.75	25.75	0.70068×	0.70068–0.70748
resnet-1x1	float16	eager	5/5	15.75	18.375	1.1746×	1.1429–1.184
resnet-1x1	float16	graph	5/5	37.75	25.875	0.68874×	0.68543–0.69
resnet-1x1-bias	bfloat16	eager	0/5	— incomplete / refused
resnet-1x1-bias	bfloat16	graph	0/5	— incomplete / refused
resnet-1x1-bias	float16	eager	0/5	— incomplete / refused
resnet-1x1-bias	float16	graph	0/5	— incomplete / refused
resnet-3x3	bfloat16	eager	5/5	1198.2	398	0.33184×	0.33045–0.33455
resnet-3x3	bfloat16	graph	5/5	1206	405.25	0.33648×	0.33392–0.33776
resnet-3x3	float16	eager	5/5	1198.9	398.5	0.33212×	0.33167–0.33337
resnet-3x3	float16	graph	5/5	1202.1	407.12	0.33881×	0.3368–0.33954
resnet-3x3-bias	bfloat16	eager	0/5	— incomplete / refused
resnet-3x3-bias	bfloat16	graph	0/5	— incomplete / refused
resnet-3x3-bias	float16	eager	0/5	— incomplete / refused
resnet-3x3-bias	float16	graph	0/5	— incomplete / refused
resnext-grouped-3x3	float16	eager	0/5	— incomplete / refused
resnext-grouped-3x3	float16	graph	0/5	— incomplete / refused
stage-transition-3x3-s2	float16	eager	5/5	8355.4	1514.9	0.18131×	0.1803–0.18339
stage-transition-3x3-s2	float16	graph	5/5	8365.8	1523.5	0.18211×	0.18187–0.18441
stage-transition-3x3-s2-bias	float16	eager	0/5	— incomplete / refused
stage-transition-3x3-s2-bias	float16	graph	0/5	— incomplete / refused
stage-transition-5x5-s2	float16	eager	5/5	24029	3927.2	0.16341×	0.16278–0.16401
stage-transition-5x5-s2	float16	graph	5/5	24036	3932.2	0.16359×	0.16295–0.16449
stage-transition-5x5-s2-bias	float16	eager	0/5	— incomplete / refused
stage-transition-5x5-s2-bias	float16	graph	0/5	— incomplete / refused
stem-3x3-s2	float16	eager	5/5	389.25	255.75	0.65598×	0.64067–0.66399
stem-3x3-s2	float16	graph	5/5	394.5	260.38	0.66043×	0.64227–0.66593
stem-3x3-s2-bias	float16	eager	0/5	— incomplete / refused
stem-3x3-s2-bias	float16	graph	0/5	— incomplete / refused
stride-padding-nondiv-cube-fp16	float16	eager	5/5	36.5	22	0.60274×	0.59589–0.66897
stride-padding-nondiv-cube-fp16	float16	graph	5/5	36.75	21.5	0.58503×	0.57823–0.65529
stride2	bfloat16	eager	5/5	1811.8	362.12	0.19983×	0.19829–0.20208
stride2	bfloat16	graph	5/5	1817	375	0.20618×	0.20516–0.20681
stride2-bias	bfloat16	eager	0/5	— incomplete / refused
stride2-bias	bfloat16	graph	0/5	— incomplete / refused

CountNonzeroFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
3d-multidim-reduce	float16	eager	5/5	114.75	10.75	0.094092×	0.092613–0.099352
3d-multidim-reduce	float16	graph	5/5	114.62	17.75	0.15536×	0.14642–0.15733
sparsity-hidden	bfloat16	eager	5/5	93.875	29	0.30892×	0.30441–0.31565
sparsity-hidden	bfloat16	graph	5/5	103.5	36.75	0.35663×	0.3382–0.36594
sparsity-hidden	float16	eager	5/5	93.5	28	0.29867×	0.29669–0.30831
sparsity-hidden	float16	graph	5/5	101.88	35.5	0.34847×	0.33661–0.35096
sparsity-seq	float16	eager	5/5	40.875	8.75	0.21407×	0.20732–0.21951
sparsity-seq	float16	graph	5/5	55.25	12.875	0.23333×	0.22624–0.24528

EluFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
mlp-hidden	bfloat16	eager	5/5	36	29.25	0.82909×	0.80903–0.84722
mlp-hidden	bfloat16	graph	5/5	43.75	36	0.81714×	0.7507–0.82286
mlp-hidden	float16	eager	5/5	35	29.25	0.83571×	0.81915–0.85357
mlp-hidden	float16	graph	5/5	43	34.75	0.81548×	0.7933–0.82759
mlp-hidden-wide	bfloat16	eager	5/5	68.25	57	0.83212×	0.82051–0.83883
mlp-hidden-wide	bfloat16	graph	5/5	78	62.125	0.79167×	0.77882–0.8013
mlp-hidden-wide	float16	eager	5/5	67.625	55.25	0.8209×	0.81331–0.83396
mlp-hidden-wide	float16	graph	5/5	76.125	61	0.80265×	0.79024–0.81773

GeluFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
llama-8b-ffn-decode	bfloat16	eager	5/5	3.5	1.75	0.5×	0.5–0.57143
llama-8b-ffn-decode	bfloat16	graph	5/5	3.5	3.75	1.0714×	1–1.1429
llama-8b-ffn-prefill	bfloat16	eager	5/5	184	128.25	0.69565×	0.69512–0.69837
llama-8b-ffn-prefill	bfloat16	graph	5/5	193.75	135.5	0.70305×	0.69231–0.71336
llama-8b-ffn-prefill	float16	eager	5/5	181.75	125.25	0.69003×	0.68913–0.69103
llama-8b-ffn-prefill	float16	graph	5/5	190.75	134.38	0.69855×	0.69709–0.70446

GemmFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
ds-v3-decode-down	bfloat16	eager	5/5	57.75	61.5	1.0563×	1.0377–1.0742
ds-v3-decode-down	bfloat16	graph	5/5	67.375	70.25	1.0427×	1.0337–1.0627
ds-v3-decode-gate-up	bfloat16	eager	5/5	71.5	77.375	1.0684×	1.0612–1.1014
ds-v3-decode-gate-up	bfloat16	graph	5/5	73.75	86.5	1.1611×	1.1508–1.2
ds-v3-prefill-attn-proj	bfloat16	eager	5/5	807	795.25	0.98528×	0.98344–0.98914
ds-v3-prefill-attn-proj	bfloat16	graph	5/5	816.5	804.62	0.98637×	0.982–0.98683
ds-v3-prefill-attn-proj	float16	eager	5/5	826.5	794.88	0.962×	0.96098–0.96531
ds-v3-prefill-attn-proj	float16	graph	5/5	835.5	804.12	0.96245×	0.96031–0.96711
ds-v3-prefill-down	bfloat16	eager	5/5	474.12	396.5	0.83628×	0.8336–0.83823
ds-v3-prefill-down	bfloat16	graph	5/5	483.88	404.12	0.83467×	0.83423–0.84105
ds-v3-prefill-gate-up	bfloat16	eager	5/5	484.38	455.5	0.94142×	0.93918–0.94273
ds-v3-prefill-gate-up	bfloat16	graph	5/5	493.5	464.62	0.94149×	0.93466–0.94634
k-dominant-7168x16384	bfloat16	eager	5/5	4800.6	4452.4	0.92159×	0.89613–0.95624
k-dominant-7168x16384	bfloat16	graph	5/5	4777.4	4445.5	0.92044×	0.91487–0.95147
mid-m16-attn	bfloat16	eager	5/5	110	107	0.97228×	0.96916–0.99511
mid-m16-attn	bfloat16	graph	5/5	116	114.12	0.98511×	0.98242–1.0235
mid-m32-attn	bfloat16	eager	5/5	103.25	101.62	0.98529×	0.97039–0.99031
mid-m32-attn	bfloat16	graph	5/5	108.88	110.25	1.0142×	1.0116–1.0195
mid-m64-down	bfloat16	eager	5/5	54	52.25	0.96774×	0.95305–0.98113
mid-m64-down	bfloat16	graph	5/5	63.5	60.75	0.95525×	0.94118–1.0122
mid-m96-gate-up	bfloat16	eager	5/5	68.25	68.5	1.0037×	0.97731–1.0282
mid-m96-gate-up	bfloat16	graph	5/5	77	77	0.9984×	0.97727–1.0201
square-1k-nn	bfloat16	eager	5/5	19.5	27.25	1.3846×	1.3797–1.3974
square-1k-nn	bfloat16	graph	5/5	28.75	37	1.2845×	1.2542–1.3028
square-1k-nn	float16	eager	5/5	19.75	27.5	1.3924×	1.3671–1.4103
square-1k-nn	float16	graph	5/5	29.5	36.75	1.2231×	1.2119–1.317
wide-n-24576	bfloat16	eager	5/5	1244.6	1014.5	0.8148×	0.80534–0.81914
wide-n-24576	bfloat16	graph	5/5	1253.5	1023.4	0.81681×	0.80676–0.8217

HardsigmoidFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
mbv3-se-gate	bfloat16	eager	5/5	1.75	1.5	0.85714×	0.85714–0.85714
mbv3-se-gate	bfloat16	graph	5/5	2	2.5	1.25×	1.25–1.25
mbv3-se-gate	float16	eager	5/5	1.75	1.5	0.78571×	0.75–0.85714
mbv3-se-gate	float16	graph	5/5	2	2.5	1.25×	1.25–1.375
mbv3-se-gate-deep	bfloat16	eager	5/5	2.25	2	0.88889×	0.88889–0.94444
mbv3-se-gate-deep	bfloat16	graph	5/5	2.75	2.75	1×	1–1.1429
mbv3-se-gate-deep	float16	eager	5/5	2.25	2	0.88889×	0.88889–0.88889
mbv3-se-gate-deep	float16	graph	5/5	2.625	2.75	1.0476×	1.0476–1.2

HardswishFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
mbv3-stage2	bfloat16	eager	5/5	37.5	35	0.92715×	0.91892–0.94079
mbv3-stage2	bfloat16	graph	5/5	45.75	40.75	0.90055×	0.84409–0.93646
mbv3-stage2	float16	eager	5/5	37.125	34.375	0.92833×	0.91892–0.93289
mbv3-stage2	float16	graph	5/5	44.5	39.5	0.8764×	0.84817–0.89205
mbv3-stage3	bfloat16	eager	5/5	24.25	22.5	0.92784×	0.91005–0.95833
mbv3-stage3	bfloat16	graph	5/5	32.75	27.5	0.83969×	0.81538–0.86992
mbv3-stage3	float16	eager	5/5	23.75	22.875	0.95812×	0.94737–0.97872
mbv3-stage3	float16	graph	5/5	31.5	27.5	0.89431×	0.87302–0.90299

LeakyReluFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
gan-feat	bfloat16	eager	5/5	62.75	54.625	0.87052×	0.85827–0.87327
gan-feat	bfloat16	graph	5/5	73.25	60.5	0.81229×	0.81208–0.91459
gan-feat	float16	eager	5/5	62.5	54.5	0.876×	0.86508–0.87903
gan-feat	float16	graph	5/5	73	60.75	0.84965×	0.81463–0.85274
gan-feat-deep	bfloat16	eager	5/5	33	28.25	0.87109×	0.84496–0.87879
gan-feat-deep	bfloat16	graph	5/5	42.25	34.75	0.82738×	0.81176–0.84302
gan-feat-deep	float16	eager	5/5	32	28.25	0.88281×	0.848–0.88462
gan-feat-deep	float16	graph	5/5	42.125	34	0.84472×	0.80712–0.86471

MaskedFillFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
elementwise-16M	bfloat16	eager	5/5	75.5	75.25	0.99833×	0.99013–1.0099
elementwise-16M	bfloat16	graph	5/5	80.5	80.75	1×	0.99048–1.0031
elementwise-16M	float16	eager	5/5	76.25	75.75	0.99494×	0.99007–1.013
elementwise-16M	float16	graph	5/5	80.5	80.5	1.0063×	0.99054–1.0262
elementwise-16M	float32	eager	5/5	129.62	129.5	1.0038×	0.99803–1.0086
elementwise-16M	float32	graph	5/5	136	136	1.0083×	0.97464–1.0303
elementwise-256M	bfloat16	eager	5/5	1115.2	1166.6	1.0454×	1.0444–1.0492
elementwise-256M	bfloat16	graph	5/5	1209.2	1179.5	0.9771×	0.97374–0.97975
elementwise-256M	float16	eager	5/5	1175.1	1135.2	0.9667×	0.96355–0.96778
elementwise-256M	float16	graph	5/5	1182	1140.8	0.96525×	0.96283–0.96604

MishFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
yolo-p3	bfloat16	eager	5/5	189.75	87.25	0.45982×	0.44921–0.47303
yolo-p3	bfloat16	graph	5/5	197.75	94.125	0.47398×	0.46617–0.48604
yolo-p3	float16	eager	5/5	187	88.25	0.47224×	0.45989–0.48131
yolo-p3	float16	graph	5/5	195.62	94.375	0.48051×	0.47519–0.50129
yolo-p4	bfloat16	eager	5/5	96.75	46.25	0.4768×	0.46641–0.50515
yolo-p4	bfloat16	graph	5/5	106	53.75	0.50708×	0.49406–0.52005
yolo-p4	float16	eager	5/5	95.5	47	0.49086×	0.47644–0.51436
yolo-p4	float16	graph	5/5	104.38	53.625	0.51377×	0.48588–0.5534

ReluFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
hidden-state-decode	bfloat16	eager	4/5	— incomplete / refused
hidden-state-decode	bfloat16	graph	4/5	— incomplete / refused
hidden-state-prefill	bfloat16	eager	5/5	30.25	27.875	0.92917×	0.90496–0.94531
hidden-state-prefill	bfloat16	graph	5/5	39.25	34.5	0.89634×	0.86624–0.94304
hidden-state-prefill	float16	eager	4/5	— incomplete / refused
hidden-state-prefill	float16	graph	4/5	— incomplete / refused

SiluFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
llama-8b-ffn-decode	bfloat16	eager	5/5	2.5	1.75	0.7×	0.63636–0.7
llama-8b-ffn-decode	bfloat16	graph	5/5	2.75	3.25	1.1818×	1.1818–1.3636
llama-8b-ffn-prefill	bfloat16	eager	5/5	134.5	103.38	0.76514×	0.75235–0.79368
llama-8b-ffn-prefill	bfloat16	graph	5/5	143.5	109.12	0.75259×	0.72903–0.78109
llama-8b-ffn-prefill	float16	eager	5/5	134.25	104.5	0.77623×	0.75282–0.79143
llama-8b-ffn-prefill	float16	graph	5/5	142.25	109.75	0.77153×	0.7551–0.77568

SoftplusFwdOp

Workload	Dtype	Regime	N / 5	Ours µs	cannbot µs	cannbot / ours	Range
mlp-hidden	bfloat16	eager	5/5	50.5	30.25	0.59901×	0.57711–0.6152
mlp-hidden	bfloat16	graph	5/5	59.25	37.75	0.63713×	0.61588–0.64807
mlp-hidden	float16	eager	5/5	50	31.25	0.625×	0.60101–0.6393
mlp-hidden	float16	graph	5/5	58.625	38	0.65714×	0.6087–0.68696
mlp-hidden-wide	bfloat16	eager	5/5	98	58.5	0.60154×	0.59439–0.61675
mlp-hidden-wide	bfloat16	graph	5/5	106.75	65.75	0.61466×	0.6046–0.62191
mlp-hidden-wide	float16	eager	5/5	95.875	57.25	0.59713×	0.59192–0.62338
mlp-hidden-wide	float16	graph	5/5	105.5	64.5	0.61137×	0.6019–0.63765