←back to thread

Important machine learning equations

(chizkidd.github.io)
302 points sebg | 1 comments | 28 Aug 25 11:38 UTC | HN request time: 0.201s | source
Show context
bee_rider ◴[28 Aug 25 12:46 UTC] No.45051496[source]▶
Are eigenvalues or singular values used much in the popular recent stuff, like LLMs?
replies(1): >>45051742 #
1. calebkaiser ◴[28 Aug 25 13:11 UTC] No.45051742[source]▶
LoRa uses singular value decomposition to get the low rank matrices. In different optimizers, you'll also see eigendecomposition or some approximation used (I think Shampoo does something like this, but it's been a while).