←back to thread

Speeding up PyTorch inference on Apple devices with AI-generated Metal kernels

(gimletlabs.ai)
186 points nserrino | 1 comments | 03 Sep 25 17:03 UTC | HN request time: 0.208s | source
Show context
Tiberium ◴[03 Sep 25 20:36 UTC] No.45120135[source]▶
It'd be curious to see how those AI generated kernels compare to kernels generated by https://github.com/tinygrad/tinygrad
replies(1): >>45123658 #
1. xiphias2 ◴[04 Sep 25 04:43 UTC] No.45123658[source]▶
As they wrote most of the wins are because of fusion and TimyGrad started to have fusion optimizations in the last few weeks.

GeoHot didn't want to make it only FlashAttention specific, he worked on FlashAttenrion being automatically generated by the optimizer. It's going well