←back to thread

From multi-head to latent attention: The evolution of attention mechanisms

(vinithavn.medium.com)
169 points mgninad | 1 comments | 30 Aug 25 05:45 UTC | HN request time: 0.001s | source
Show context
attogram ◴[30 Aug 25 07:32 UTC] No.45072664[source]▶
"Attention Is All You Need" - I've always wondered if the authors of that paper used such a casual and catchy title because they knew it would be groundbreaking and massively cited in the future....
replies(9): >>45073018 #>>45073470 #>>45073494 #>>45073527 #>>45073545 #>>45074544 #>>45074862 #>>45075147 #>>45079506 #
1. hyperbovine ◴[30 Aug 25 10:37 UTC] No.45073545[source]▶
It sounds like a typical neurips paper to me. And no, they did know what a big deal it would be, else google never would have given the idea away.