←back to thread

From multi-head to latent attention: The evolution of attention mechanisms

(vinithavn.medium.com)
169 points mgninad | 1 comments | 30 Aug 25 05:45 UTC | HN request time: 0.203s | source
Show context
attogram ◴[30 Aug 25 07:32 UTC] No.45072664[source]▶
"Attention Is All You Need" - I've always wondered if the authors of that paper used such a casual and catchy title because they knew it would be groundbreaking and massively cited in the future....
replies(9): >>45073018 #>>45073470 #>>45073494 #>>45073527 #>>45073545 #>>45074544 #>>45074862 #>>45075147 #>>45079506 #
1. lucidrains ◴[30 Aug 25 13:33 UTC] No.45074544[source]▶
It is a reference to the beatles song, mainly because Noam Shazeer is a music lover