> Furthermore, by rotating the vector, we have absolutely zero impact on the norm of the vector, which encodes the semantic information of our token.
Doesn’t the angle encode semantic information? Cosine similarity works for embeddings after all.
Doesn’t the angle encode semantic information? Cosine similarity works for embeddings after all.