You could have designed state of the art positional encoding

Thanks to the author for clarifying something that's been a mystery to me for a few years. The positional encoding scheme in the "Attention Is All You Need" paper is only given half a page and the construction appears to come out of nowhere.

Thank you! Seemed like voodoo to me too, hence this post!