←back to thread

549 points orcul | 2 comments | | HN request time: 0.466s | source
Show context
Animats ◴[] No.41890003[source]
This is an important result.

The actual paper [1] says that functional MRI (which is measuring which parts of the brain are active by sensing blood flow) indicates that different brain hardware is used for non-language and language functions. This has been suspected for years, but now there's an experimental result.

What this tells us for AI is that we need something else besides LLMs. It's not clear what that something else is. But, as the paper mentions, the low-end mammals and the corvids lack language but have some substantial problem-solving capability. That's seen down at squirrel and crow size, where the brains are tiny. So if someone figures out to do this, it will probably take less hardware than an LLM.

This is the next big piece we need for AI. No idea how to do this, but it's the right question to work on.

[1] https://www.nature.com/articles/s41586-024-07522-w.epdf?shar...

replies(35): >>41890104 #>>41890470 #>>41891063 #>>41891228 #>>41891262 #>>41891383 #>>41891507 #>>41891639 #>>41891749 #>>41892068 #>>41892137 #>>41892518 #>>41892576 #>>41892603 #>>41892642 #>>41892738 #>>41893400 #>>41893534 #>>41893555 #>>41893732 #>>41893748 #>>41893960 #>>41894031 #>>41894713 #>>41895796 #>>41895908 #>>41896452 #>>41896476 #>>41896479 #>>41896512 #>>41897059 #>>41897270 #>>41897757 #>>41897835 #>>41905326 #
1. shepherdjerred ◴[] No.41892137[source]
> What this tells us for AI is that we need something else besides LLMs.

Humans not taking this approach doesn’t mean that AI cannot.

replies(1): >>41892429 #
2. earslap ◴[] No.41892429[source]
Not only that but also LLMs "think" in a latent representation that is several layers deep. Sure, the first and last layers make it look like it is doing token wrangling, but what is happening in the middle layers is mostly a mystery. First layer deals directly with the tokens because that is the data we are observing (a "shadow" of the world) and last layer also deals with tokens because we want to understand what the network is "thinking" so it is a human specific lossy decoder (we can and do remove that translator and plug the latent representations to other networks to train them in tandem). There is no reason to believe that the other layers are "thinking in language".