←back to thread

A non-anthropomorphized view of LLMs

(addxorrol.blogspot.com)
475 points zdw | 7 comments | | HN request time: 0.94s | source | bottom
Show context
ants_everywhere ◴[] No.44485225[source]
> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human.

This is such a bizarre take.

The relation associating each human to the list of all words they will ever say is obviously a function.

> almost magical human-like powers to something that - in my mind - is just MatMul with interspersed nonlinearities.

There's a rich family of universal approximation theorems [0]. Combining layers of linear maps with nonlinear cutoffs can intuitively approximate any nonlinear function in ways that can be made rigorous.

The reason LLMs are big now is that transformers and large amounts of data made it economical to compute a family of reasonably good approximations.

> The following is uncomfortably philosophical, but: In my worldview, humans are dramatically different things than a function . For hundreds of millions of years, nature generated new versions, and only a small number of these versions survived.

This is just a way of generating certain kinds of functions.

Think of it this way: do you believe there's anything about humans that exists outside the mathematical laws of physics? If so that's essentially a religious position (or more literally, a belief in the supernatural). If not, then functions and approximations to functions are what the human experience boils down to.

[0] https://en.wikipedia.org/wiki/Universal_approximation_theore...

replies(5): >>44485574 #>>44486015 #>>44487960 #>>44488003 #>>44495590 #
1. cuttothechase ◴[] No.44486015[source]
>Think of it this way: do you believe there's anything about humans that exists outside the mathematical laws of physics? If so that's essentially a religious position (or more literally, a belief in the supernatural). If not, then functions and approximations to functions are what the human experience boils down to.

It seems like, we can at best, claim that we have modeled the human thought process for reasoning/analytic/quantitative through Linear Algebra, as the best case. Why should we expect the model to be anything more than a model ?

I understand that there is tons of vested interest, many industries, careers and lives literally on the line causing heavy bias to get to AGI. But what I don't understand is what about linear algebra that makes it so special that it creates a fully functioning life or aspects of a life?

Should we make an argument saying that Schroedinger's cat experiment can potentially create zombies then the underlying Applied probabilistic solutions should be treated as super-human and build guardrails against it building zombie cats?

replies(2): >>44486093 #>>44488791 #
2. ants_everywhere ◴[] No.44486093[source]
> It seems like, we can at best, claim that we have modeled the human thought process for reasoning/analytic/quantitative through Linear Algebra....I don't understand is what about linear algebra that makes it so special that it creates a fully functioning life or aspects of a life?

Not linear algebra. Artificial neural networks create arbitrarily non-linear functions. That's the point of non-linear activation functions and it's the subject of the universal approximation theorems I mentioned above.

replies(1): >>44486291 #
3. cuttothechase ◴[] No.44486291[source]
ANNs are just mathematical transformations, powered by linear algebra + non-linear functions. They simulate certain cognitive processes — but they are fundamentally math, not magic.
replies(3): >>44487409 #>>44487504 #>>44489911 #
4. delusional ◴[] No.44487409{3}[source]
I wouldn't say they "simulate cognitive processes". They do statistics. Advanced multivariadic statistics.

An LLM thinks in the same way excel thinks when you ask it to fit a curve.

5. ImHereToVote ◴[] No.44487504{3}[source]
Who invoked magic in this thread exactly?
6. hackinthebochs ◴[] No.44488791[source]
>Why should we expect the model to be anything more than a model ?

To model a process with perfect accuracy requires recovering the dynamics of that process. The question we must ask is what happens in the space between bad statistical model and perfect accuracy? What happens when the model begins to converge towards accurate reproduction. How far does generalization in the model take us towards capturing the dynamics involved in thought?

7. ants_everywhere ◴[] No.44489911{3}[source]
I think the point of mine that you're missing (or perhaps disagreeing with implicitly) is that *everything* is fundamentally math. Or, if you like, everything is fundamentally physics, and physics is fundamentally math.

So classes of functions (ANNs) that can approximate our desired function to arbitrary precision are what we should be expecting to be working with.