←back to thread

Understanding Transformers Using a Minimal Example

(rti.github.io)
295 points rttti | 1 comments | 03 Sep 25 15:30 UTC | HN request time: 0.302s | source
Show context
CGMthrowaway ◴[03 Sep 25 20:06 UTC] No.45119871[source]▶
Honest feedback - I was really excited when I read the opening. However, I did not come away from this without a greater understanding than I already had.

For reference, my initial understanding was somewhat low: basically I know a) what embedding is basically b) transformers work by matrix multiplication, and c) it's something like a multi-threaded Markov chain generator with the benefit of prior-trained embeddings

replies(8): >>45120114 #>>45120200 #>>45122565 #>>45123711 #>>45125243 #>>45128482 #>>45129469 #>>45134872 #
hunter2_ ◴[04 Sep 25 04:53 UTC] No.45123711[source]▶
Similarly, I was really excited when I read the headline here on HN and thought this would be about the electrical device. I wonder if the LLM meaning has eclipsed the electrical meaning at this point, as a default in the absence of other qualifiers, in communities like this.
replies(1): >>45124152 #
1. zxexz ◴[04 Sep 25 06:19 UTC] No.45124152[source]▶
It does seem to. I’ve been working on some personal projects where I’ve needed to look up and research transformers quite a bit (the kind that often has a ferrite core) and it has been frustrating. Frustrating not just trying to search for the wire datasheets, etc., but also because I often have to use the other transformer via service to find what I’m looking for because search is so enshittified by the newer definition.