←back to thread

549 points orcul | 1 comments | | HN request time: 0.202s | source
Show context
habitue ◴[] No.41889815[source]
Language may not be essential for thought, (most of us have the experience of an idea occurring to us that we struggle to put into words), but language acts as a regularization mechanism on thoughts.

Serializing much higher dimensional freeform thoughts into language is a very lossy process, and this kinda ensures that mostly only the core bits get translated. Think of times when someone gets an idea you're trying to convey, but you realize they're missing some critical context you forgot to share. It takes some activation energy to add that bit of context, so if it seems like they mostly get what you're saying, you skip it. Over time, transferring ideas from one person to the next, they tend towards a very compressed form because language is expensive.

This process also works on your own thoughts. Thinking out loud performs a similar role, it compresses the hell out of the thought or else it remains inexpressible. Now imagine repeated stages of compressing through language, allowing ideas to form from that compressed form, and then compressing those ideas in turn. It's a bit of a recursive process and language is in the middle of it.

replies(3): >>41890360 #>>41890458 #>>41890579 #
akomtu ◴[] No.41890579[source]
Imo, that's the essense of reasoning. Limited memory and slow communication channels force us to create compact, but expressive models of reality. LLMs, on the other hand, have all the memory in the world and their model of reality is a piece-wise interpolation of the huge training dataset. Why invent grammar rules if you can keep the entire dictionary in mind?
replies(1): >>41892332 #
1. mcswell ◴[] No.41892332[source]
Why do LLMs (or rather similar models that draw pictures) keep getting the number of fingers on the human hand wrong, or show two people's arms or legs merging? Or in computer-created videos, fail at object preservation? It seems to me they do not have a model of the world, only an imperfect model of pictures they've seen.