←back to thread

Embeddings are underrated (2024)

(technicalwriting.dev)
484 points jxmorris12 | 2 comments | 12 May 25 15:05 UTC | HN request time: 0.644s | source
Show context
gnarlouse ◴[12 May 25 18:47 UTC] No.43966277[source]▶
The article did not go the direction that I imagined, which I loved. What if we could use embedding spaces to clean up documentation to be more explicit and direct? The biggest thing that stuck with me from my technical writing class was “get to the point”. Finding ways to narrow word choices in an almost 1984’esque ingSoc seems appropriate? Ish?
replies(1): >>43966787 #
1. energy123 ◴[12 May 25 19:44 UTC] No.43966787[source]▶
Neural nets aren't invertible so I don't see how embeddings can help reduce verbosity. It's not like we could take the embedding of a piece of verbose text and figure out less verbose phrasings that have an embedding with high similarity without using some expensive search. At which point it seems better to just use a standard LLM for that.
replies(1): >>43969088 #
2. kaycebasques ◴[13 May 25 02:00 UTC] No.43969088[source]▶
Yes, I have experimented with using ML for style concerns such as consistent capitalization and verbosity and can confirm: no need for embeddings here. You can get the job done with only a text generation model. I actually can't visualize how an embeddings-powered implementation would work here