←back to thread

169 points mgninad | 1 comments | | HN request time: 0.206s | source
Show context
attogram ◴[] No.45072664[source]
"Attention Is All You Need" - I've always wondered if the authors of that paper used such a casual and catchy title because they knew it would be groundbreaking and massively cited in the future....
replies(9): >>45073018 #>>45073470 #>>45073494 #>>45073527 #>>45073545 #>>45074544 #>>45074862 #>>45075147 #>>45079506 #
1. jjaksic ◴[] No.45079506[source]
I was there. They had no idea. The purpose and expectation of the transformer architecture was to enable better translation at scale and without recurrence. This in itself would have been a big deal (and they probably expected some citations), but the actual impact of the architecture was a few orders of magnitude greater.