(fleetwood.dev)

213 points Philpax | 1 comments | 17 Nov 24 20:31 UTC | HN request time: 0.378s | source

Show context

1024core ◴[18 Nov 24 18:14 UTC] No.42175184[source]▶

I didn't get the sudden leap from "position encodings" to "QKV" magic.

What is the connection between the two? Where does "Q" come from? What are "K" and "V"? (I know they stand for "Query", "Key", "Value"; but what do they have to do with position embeddings?)

replies(2): >>42175364 #>>42176614 #

1. FL33TW00D ◴[18 Nov 24 20:30 UTC] No.42176614[source]▶

>>42175184 #

"This post intends to limit the mathematical knowledge required to follow along, but some basic linear algebra, trigonometry and understanding of self attention is expected."

If you're not sure on self attention, the post will be a little unclear

↑

You could have designed state of the art positional encoding