←back to thread

A non-anthropomorphized view of LLMs

(addxorrol.blogspot.com)
475 points zdw | 1 comments | | HN request time: 0.722s | source
Show context
ants_everywhere ◴[] No.44485225[source]
> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human.

This is such a bizarre take.

The relation associating each human to the list of all words they will ever say is obviously a function.

> almost magical human-like powers to something that - in my mind - is just MatMul with interspersed nonlinearities.

There's a rich family of universal approximation theorems [0]. Combining layers of linear maps with nonlinear cutoffs can intuitively approximate any nonlinear function in ways that can be made rigorous.

The reason LLMs are big now is that transformers and large amounts of data made it economical to compute a family of reasonably good approximations.

> The following is uncomfortably philosophical, but: In my worldview, humans are dramatically different things than a function . For hundreds of millions of years, nature generated new versions, and only a small number of these versions survived.

This is just a way of generating certain kinds of functions.

Think of it this way: do you believe there's anything about humans that exists outside the mathematical laws of physics? If so that's essentially a religious position (or more literally, a belief in the supernatural). If not, then functions and approximations to functions are what the human experience boils down to.

[0] https://en.wikipedia.org/wiki/Universal_approximation_theore...

replies(5): >>44485574 #>>44486015 #>>44487960 #>>44488003 #>>44495590 #
suddenlybananas ◴[] No.44488003[source]
>There's a rich family of universal approximation theorems

Wow, look-up tables can get increasingly good at approximating a function!

replies(1): >>44489946 #
1. ants_everywhere ◴[] No.44489946[source]
A function is by definition a lookup table.

The lookup table is just (x, f(x)).

So, yes, trivially if you could construct the lookup table for f then you'd approximate f. But to construct it you have to know f. And to approximate it you need to know f at a dense set of points.