Speeding up PyTorch inference on Apple devices with AI-generated Metal kernels

It'd be curious to see how those AI generated kernels compare to kernels generated by https://github.com/tinygrad/tinygrad

As they wrote most of the wins are because of fusion and TimyGrad started to have fusion optimizations in the last few weeks.

GeoHot didn't want to make it only FlashAttention specific, he worked on FlashAttenrion being automatically generated by the optimizer. It's going well