←back to thread

You could have designed state of the art positional encoding

(fleetwood.dev)
213 points Philpax | 1 comments | 17 Nov 24 20:31 UTC | HN request time: 0.209s | source
Show context
logicchains ◴[18 Nov 24 08:59 UTC] No.42170851[source]▶
Does anyone know why 2D rope implementations apply two separate 1D rotations to pairs, instead of applying a 2d rotation to triplets?
replies(1): >>42171935 #
1. rini17 ◴[18 Nov 24 12:57 UTC] No.42171935[source]▶
No they apply many rotations, same as the number of dimensions of the embedding space.