←back to thread

From multi-head to latent attention: The evolution of attention mechanisms

(vinithavn.medium.com)
169 points mgninad | 1 comments | 30 Aug 25 05:45 UTC | HN request time: 0s | source
Show context
mrtesthah ◴[30 Aug 25 07:05 UTC] No.45072533[source]▶
Do we know if any of these techniques are actually used in the so-called "frontier" models?
replies(3): >>45072588 #>>45073417 #>>45076391 #
1. vinithavn01 ◴[30 Aug 25 07:17 UTC] No.45072588[source]▶
The model names are mentioned under each type of attention mechanism