Thanks for pointing out the elephant in the room with LLMs.
The basic design is non-deterministic. Trying to extract "facts" or "truth" or "accuracy" is an exercise in futility.
You can't blame an LLM for getting the facts wrong, or hallucinating, when by design they don't even attempt to store facts in the first place. All they store are language statistics, boiling down to "with preceding context X, most statistically likely next words are A, B or C". The LLM wasn't designed to know or care that outputting "B" would represent a lie or hallucination, just that it's a statistically plausible potential next word.
Facts can be encoded as words. That's something we also do a lot for facts we learn, gather, and convey to other people. 99% of university is learning facts and theories and concept from reading and listening to words.
Also, even when directly observing the same fact, it can be interpreted by different people in different ways, whether this happens as raw "thought" or at the conscious verbal level. And that's before we even add value judgements to it.
>All they store are language statistics, boiling down to "with preceding context X, most statistically likely next words are A, B or C".
And how do we know we don't do something very similar with our facts - make a map of facts and concepts and weights between them for retrieving them and associating them? Even encoding in a similar way what we think of as our "analytic understanding".
LLMs are trained to auto-regressively predict text continuations. They are not concerned with the external world and any objective experimentally verifiable facts - they are just self-predicting "this is what I'm going to say next", having learnt that from the training data (i.e. "what would the training data say next").
Humans/animals are embodied, living in the real world, whose design has been honed by a "loss function" favoring survival. Animals are "designed" to learn facts about the real world, and react to those facts in a way that helps them survive.
What humans/animals are predicting is not some auto-regressive "what will I do next", but rather what will HAPPEN next, based largely on outward-looking sensory inputs, but also internal inputs.
Animals are predicting something EXTERNAL (facts) vs LLMs predicting something INTERNAL (what will I say next).
Yes - but LLMs also get this "embodied knowledge" passed down from human-generated training data. We are their sensory inputs in a way (which includes their training images, audio, and video too).
They do learn in a batch manner, and we learn many things not from books but from a more interactive direct being in the world. But after we distill our direct experiences and throughts derived from them as text, we pass them down to the LLMs.
Hey, there's even some kind of "loss function" in the LLM case - from the thumbs up/down feedback we are asked to give to their answers in Chat UIs, to $5/hour "mechanical turks" in Africa or something tasked with scoring their output, to rounds of optimization and pruning during training.
>Animals are predicting something EXTERNAL (facts) vs LLMs predicting something INTERNAL (what will I say next).
I don't think that matters much, in both cases it's information in, information out.
Human animals predict "what they will say/do next" all the time, just like they also predict what they will encounter next ("my house is round that corner", "that car is going to make a turn").
Our prompt to an LLM serves the same role as sensory input from the external world plays to our predictions.
It's not the same though. It's the difference between reading about something and, maybe having read the book and/or watched the video, learning to DO it yourself, acting based on the content of your own mind.
The LLM learns 2nd hand heresay, with no idea of what's true or false, what generalizations are valid, or what would be hallucinatory, etc, etc.
The human learns verifiable facts, uses curiosity to explore and fill the gaps, be creative etc.
I think it's pretty obvious why LLMs have all the limitations and deficiencies that they do.
If 2nd hand heresay (from 1000's of conflicting sources) really was good as 1st hand experience and real-world prediction, then we'd not be having this discussion - we'd be bowing to our AGI overlords (well, at least once the AI also got real-time incremental learning, internal memory, looping, some type of (virtual?) embodiment, autonomy ...).
The way that animals (maybe think apes and dogs, etc, not just humans) learn is by observing and interacting. If something is new or behaves in unexpected ways then "prediction failure", aka surprise, leads to them focusing on it and interacting with it, which is the way evolution has discovered for them to learn more about it.
Yes, an LLM has some agency via tool use, and via tool output it can learn/verify to some extent, although without continual learning this is only of ephemeral value.
This is all a bit off topic to my original point though, which is the distinction between trying to learn from 2nd hand conflicting heresay (he said, she said) vs having the ability to learn the truth for yourself, which starts with being built to predict the truth (external real-world) rather than being built to predict statistical "he said, she said" continuations. Sure, you can mitigate a few of an LLM's shortcomings by giving them tools etc, but fundamentally they are just doing the wrong thing (self-prediction) if you are hoping for them to become AGI rather than just being language models.
