A non-anthropomorphized view of LLMs

> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human.

This is such a bizarre take.

The relation associating each human to the list of all words they will ever say is obviously a function.

> almost magical human-like powers to something that - in my mind - is just MatMul with interspersed nonlinearities.

There's a rich family of universal approximation theorems [0]. Combining layers of linear maps with nonlinear cutoffs can intuitively approximate any nonlinear function in ways that can be made rigorous.

The reason LLMs are big now is that transformers and large amounts of data made it economical to compute a family of reasonably good approximations.

> The following is uncomfortably philosophical, but: In my worldview, humans are dramatically different things than a function . For hundreds of millions of years, nature generated new versions, and only a small number of these versions survived.

This is just a way of generating certain kinds of functions.

Think of it this way: do you believe there's anything about humans that exists outside the mathematical laws of physics? If so that's essentially a religious position (or more literally, a belief in the supernatural). If not, then functions and approximations to functions are what the human experience boils down to.

[0] https://en.wikipedia.org/wiki/Universal_approximation_theore...

> I am baffled that the AI discussions seem to never move away from treating a function to generate sequences of words as something that resembles a human.

You appear to be disagreeing with the author and others who suggest that there's some element of human consciousness that's beyond than what's observable from the outside, whether due to religion or philosophy or whatever, and suggesting that they just not do that.

In my experience, that's not a particularly effective tactic.

Rather, we can make progress by assuming their predicate: Sure, it's a room that translates Chinese into English without understanding, yes, it's a function that generates sequences of words that's not a human... but you and I are not "it" and it behaves rather an awful lot like a thing that understands Chinese or like a human using words. If we simply anthropomorphize the thing, acknowledging that this is technically incorrect, we can get a lot closer to predicting the behavior of the system and making effective use of it.

Conversely, when speaking with such a person about the nature of humans, we'll have to agree to dismiss the elements that are different from a function. The author says:

> In my worldview, humans are dramatically different things than a function... In contrast to an LLM, given a human and a sequence of words, I cannot begin putting a probability on "will this human generate this sequence".

Sure you can! If you address an American crowd of a certain age range with "We’ve got to hold on to what we’ve got. It doesn’t make a difference if..." I'd give a very high probability that someone will answer "... we make it or not". Maybe that human has a unique understanding of the nature of that particular piece of pop culture artwork, maybe it makes them feel things that an LLM cannot feel in a part of their consciousness that an LLM does not possess. But for the purposes of the question, we're merely concerned with whether a human or LLM will generate a particular sequence of words.

I see your point, and I like that you're thinking about this from the perspective of how to win hearts and minds.

I agree my approach is unlikely to win over the author or other skeptics. But after years of seeing scientists waste time trying to debate creationists and climate deniers I've kind of given up on trying to convince the skeptics. So I was talking more to HN in general.

> You appear to be disagreeing with the author and others who suggest that there's some element of human consciousness that's beyond than what's observable from the outside

I'm not sure what it means to be observable or not from the outside. I think this is at least partially because I don't know what it means to be inside either. My point was just that whatever consciousness is, it takes place in the physical world and the laws of physics apply to it. I mean that to be as weak a claim as possible: I'm not taking any position on what consciousness is or how it works etc.

Searle's Chinese room argument attacks attacks a particular theory about the mind based essentially turing machines or digital computers. This theory was popular when I was in grad school for psychology. Among other things, people holding the view that Searle was attacking didn't believe that non-symbolic computers like neural networks could be intelligent or even learn language. I thought this was total nonsense, so I side with Searle in my opposition to it. I'm not sure how I feel about the Chinese room argument in particular, though. For one thing it entirely depends on what it means to "understand" something, and I'm skeptical that humans ever "understand" anything.

> If we simply anthropomorphize the thing, acknowledging that this is technically incorrect, we can get a lot closer to predicting the behavior of the system and making effective use of it.

I see what you're saying: that a technically incorrect assumption can bring to bear tools that improve our analysis. My nitpick here is I agree with OP that we shouldn't anthropomorphize LLMs, any more than we should anthropomorphize dogs or cats. But OP's arguments weren't actually about anthropomorphizing IMO, they were about things like functions that are more fundamental than humans. I think artificial intelligence will be non-human intelligence just like we have many examples of non-human intelligence in animals. No attribution of human characteristics needed.

> If we simply anthropomorphize the thing, acknowledging that this is technically incorrect, we can get a lot closer to predicting the behavior of the system and making effective use of it.

Yes I agree with you about your lyrics example. But again here I think OP is incorrect to focus on the token generation argument. We all agree human speech generates tokens. Hopefully we all agree that token generation is not completely predictable. Therefore it's by definition a randomized algorithm and it needs to take an RNG. So pointing out that it takes an RNG is not a valid criticism of LLMs.

Unless one is a super-determinist then there's randomness at the most basic level of physics. And you should expect that any physical process we don't understand well yet (like consciousness or speech) likely involves randomness. If one *is* a super-determinist then there is no randomness, even in LLMs and so the whole point is moot.

>Think of it this way: do you believe there's anything about humans that exists outside the mathematical laws of physics? If so that's essentially a religious position (or more literally, a belief in the supernatural). If not, then functions and approximations to functions are what the human experience boils down to.

It seems like, we can at best, claim that we have modeled the human thought process for reasoning/analytic/quantitative through Linear Algebra, as the best case. Why should we expect the model to be anything more than a model ?

I understand that there is tons of vested interest, many industries, careers and lives literally on the line causing heavy bias to get to AGI. But what I don't understand is what about linear algebra that makes it so special that it creates a fully functioning life or aspects of a life?

Should we make an argument saying that Schroedinger's cat experiment can potentially create zombies then the underlying Applied probabilistic solutions should be treated as super-human and build guardrails against it building zombie cats?

> It seems like, we can at best, claim that we have modeled the human thought process for reasoning/analytic/quantitative through Linear Algebra....I don't understand is what about linear algebra that makes it so special that it creates a fully functioning life or aspects of a life?

Not linear algebra. Artificial neural networks create arbitrarily non-linear functions. That's the point of non-linear activation functions and it's the subject of the universal approximation theorems I mentioned above.

ANNs are just mathematical transformations, powered by linear algebra + non-linear functions. They simulate certain cognitive processes — but they are fundamentally math, not magic.

I wouldn't say they "simulate cognitive processes". They do statistics. Advanced multivariadic statistics.

An LLM thinks in the same way excel thinks when you ask it to fit a curve.

Who invoked magic in this thread exactly?

>> given a human and a sequence of words, I cannot begin putting a probability on "will this human generate this sequence".

> Sure you can! If you address an American crowd of a certain age range with "We’ve got to hold on to what we’ve got. It doesn’t make a difference if..." I'd give a very high probability that someone will answer "... we make it or not".

I think you may have this flipped compared to what the author intended. I believe the author is not talking about the probability of an output given an input, but the probability of a given output across all inputs.

Note that the paragraph starts with "In my worldview, humans are dramatically different things than a function, (R^n)^c -> (R^n)^c". To compute a probability of a given output, (which is a any given element in "(R^n)^n"), we can count how many mappings there are total and then how many of those mappings yield the given element.

The point I believe is to illustrate the complexity of inputs for humans. Namely for humans the input space is even more complex than "(R^n)^c".

In your example, we can compute how many input phrases into a LLM would produce the output "make it or not". We can than compute that ratio to all possible input phrases. Because "(R^n)^c)" is finite and countable, we can compute this probability.

For a human, how do you even start to assess the probability that a human would ever say "make it or not?" How do you even begin to define the inputs that a human uses, let alone enumerate them? Per the author, "We understand essentially nothing about it." In other words, the way humans create their outputs is (currently) incomparably complex compared to a LLM, hence the critique of the anthropomorphization.

Not that this is your main point, but I find this take representative, “do you believe there's anything about humans that exists outside the mathematical laws of physics?”There are things “about humans”, or at least things that our words denote, that are outside physic’s explanatory scope. For example, the experience of the colour red cannot be known, as an experience, by a person who only sees black and white. This is the case no matter what empirical propositions, or explanatory system, they understand.

>There's a rich family of universal approximation theorems

Wow, look-up tables can get increasingly good at approximating a function!

Perhaps. But I can't see a reason why they couldn't still write endless—and theoretically valuable—poems, dissertations, or blog posts, about all things red and the nature of redness itself. I imagine it would certainly take some studying for them, likely interviewing red-seers, or reading books about all things red. But I'm sure they could contribute to the larger red discourse eventually, their unique perspective might even help them draw conclusions the rest of us are blind to.

So perhaps the fact that they "cannot know red" is ultimately irrelevant for an LLM too?

>Why should we expect the model to be anything more than a model ?

To model a process with perfect accuracy requires recovering the dynamics of that process. The question we must ask is what happens in the space between bad statistical model and perfect accuracy? What happens when the model begins to converge towards accurate reproduction. How far does generalization in the model take us towards capturing the dynamics involved in thought?

I think the point of mine that you're missing (or perhaps disagreeing with implicitly) is that *everything* is fundamentally math. Or, if you like, everything is fundamentally physics, and physics is fundamentally math.

So classes of functions (ANNs) that can approximate our desired function to arbitrary precision are what we should be expecting to be working with.

A function is by definition a lookup table.

The lookup table is just (x, f(x)).

So, yes, trivially if you could construct the lookup table for f then you'd approximate f. But to construct it you have to know f. And to approximate it you need to know f at a dense set of points.

This idea is called qualia [0] for those unfamiliar.

I don't have any opinion on the qualia debates honestly. I suppose I don't know what it feels like for an ant to find a tasty bit of sugar syrup, but I believe it's something that can be described with physics (and by extension, things like chemistry).

But we do know some things about some qualia. Like we know how red light works, we have a good idea about how photoreceptors work, etc. We know some people are red-green colorblind, so their experience of red and green are mushed together. We can also have people make qualia judgments and watch their brains with fMRI or other tools.

I think maybe an interesting question here is: obviously it's pleasurable to animals to have their reward centers activated. Is it pleasurable or desirable for AIs to be rewarded? Especially if we tell them (as some prompters do) that they feel pleasure if they do things well and pain if they don't? You can ask this sort of question for both the current generation of AIs and future generations.

[0] https://en.wikipedia.org/wiki/Qualia

> do you believe there's anything about humans that exists outside the mathematical laws of physics?

I don't.

The point is not that we, humans, cannot arrange physical matter such that it have emergent properties just like the human brain.

The point is that we shouldn't.

Does responsibility mean anything to these people posing as Evolution?

Nobody's personally responsible for what we've evolved into; evolution has simply happened. Nobody's responsible for the evolutionary history that's carried in and by every single one of us. And our psychology too has been formed by (the pressures of) evolution, of course.

But if you create an artificial human, and create it from zero, then all of its emergent properties are on you. Can you take responsibility for that? If something goes wrong, can you correct it, or undo it?

I don't consider our current evolutionary state "scripture", so we certainly tweak, one way or another, aspects that we think deserve tweaking. To me, it boils down to our level of hubris. Some of our "mistaken tweaks" are now visible at an evolutionary scale, too; for a mild example, our jaws have been getting smaller (leaving less room for our teeth) due to our bad up diet (thanks, agriculture). But worse than that, humans have been breeding plants, animals, modifying DNA left and right, and so on -- and they've summarily failed to take responsibility for their atrocious mistakes.

Thus, I have zero trust in, and zero hope for, assholes who unabashedly aim to create artificial intelligence knowing full well that such properties might emerge that we'd have to call artificial psyche. Anyone taking this risk is criminally reckless, in my opinion.

It's not that humans are necessarily unable to create new sentient beings. Instead: they shouldn't even try! Because they will inevitably fuck it up, bringing about untold misery; and they won't be able to contain the damage.