Most active commenters
  • fragmede(3)

←back to thread

Grok 3: Another win for the bitter lesson

(www.thealgorithmicbridge.com)
129 points kiyanwang | 27 comments | | HN request time: 1.115s | source | bottom
Show context
ArtTimeInvestor ◴[] No.43112245[source]
It looks like the USA is bringing all technology in-house that is needed to build AI.

TSMC has a factory in the USA now, ASML too. OpenAI, Google, xAI and Nvidia are natively in the USA.

While no other country is even close to build AI on their own.

Is the USA going to "own" the world by becoming the keeper of AI? Or is there an alternative future that has a probability > 0?

replies(7): >>43112250 #>>43112266 #>>43112288 #>>43112313 #>>43113081 #>>43113084 #>>43113181 #
1. lompad ◴[] No.43112266[source]
You implicitly assume, LLMs are actually important enough to make a difference on the geopolitical level.

So far, I haven't seen any indication that this is the case. And I'd say, hyped up speculations by people financially incentivized to hype AI should be taken with an entire mine full of salt.

replies(5): >>43112290 #>>43112419 #>>43112691 #>>43112716 #>>43113043 #
2. ArtTimeInvestor ◴[] No.43112290[source]
First, its not just about LLMs. Its not an LLM that replaced human drivers in Waymo cars.

Second, how could AI not be the deciding geopolitical factor of the future? You expect progress to stop and AI not to achieve and surpass human intelligence?

replies(3): >>43112319 #>>43112382 #>>43112437 #
3. Eikon ◴[] No.43112319[source]
> You expect progress to stop and AI not to achieve and surpass human intelligence?

A word generator is not intelligence. There’s no “thinking” involved here.

To surpass human intelligence, you’d first need to actually develop intelligence, and llms will not be it.

replies(1): >>43112519 #
4. lompad ◴[] No.43112382[source]
>First, its not just about LLMs. Its not an LLM that replaced human drivers in Waymo cars.

As far as I know, Waymo is still not even remotely able to operate in any kind of difficult environment, even though insane amounts of money have been poured into it. You are vastly overstating its capabilities.

Is it cool tech? Sure. Is it safely going to replace all drivers? Doubt, very much so.

Secondly, this only works if progress in AI does not stagnate. And, again, you have no grounds to actually make that claim. It's all built on the fanciful imagination that we're close to AGI. I disagree heavily and think, it's much further away than people profiting financially from the hype tend to claim.

replies(1): >>43112804 #
5. OtherShrezzing ◴[] No.43112419[source]
I think ground-zero for that line of thought is with Leopold Aschenbrenner[0]. Who I believe now runs an AI focused hedge fund.

[0] https://situational-awareness.ai

6. ozornin ◴[] No.43112437[source]
> how could AI not be the deciding geopolitical factor of the future?

Easily. Natural resources, human talent, land and supply chains all are and will be more important factors than AI

> You expect progress to stop

no

> and AI not to achieve and surpass human intelligence

yes

7. willvarfar ◴[] No.43112519{3}[source]
I get that LLMs are just doing a probabilistic prediction etc. Its all Hutter Prize stuff.

But how are animals with nerve-centres or brains different? What do we think us humans do differently so we are not just very big probabilistic prediction systems?

A completely different tack: if we develop the technology to engineer animal-style nerves and form them into big lumps called 'brains', in what way is that not artificial and intelligence? And if we can do that, what is to stop that manufactured brain from not being twice or ten times larger than a humans?

replies(6): >>43112567 #>>43112751 #>>43112930 #>>43113206 #>>43113272 #>>43113546 #
8. dkjaudyeqooe ◴[] No.43112567{4}[source]
Human (and other animal) brains probably are probabilistic, but we don't understand their structure or mechanism in fine enough detail to replicate them, or simulate them.

People think LLMs are intelligent because intelligence is latent within the text they digest, process and regurgitate. Their performance reflects this trick.

9. fnordsensei ◴[] No.43112691[source]
They seem popular enough that they could be leveraged to influence opinion and twist perception, as has been done with social media.

Or, as is already being done, use them to influence opinion and twist perception within tools and services that people already use, such as social media.

replies(1): >>43112995 #
10. tankenmate ◴[] No.43112716[source]
It's an economic benefit. It's not a panacea but it does make some tasks much cheaper.

On the other hand if the economic benefit isn't shared across the whole of society it will become a destabilising factor and hence reduce the overall economic benefit it might have otherwise borne.

11. grumbel ◴[] No.43112751{4}[source]
I don't think the probabilistic prediction is a problem. The problem with current LLM is that they are limited to doing "System 1" thinking, only giving you a fast instinctive response to a question. While that works great for a lot of small problems, it completely falls apart on any larger task that requires multiple steps or backtracking. "System 2" thinking is completely missing as is the ability to just self-iterate on their own output.

Reasoning models are trying to address that now, but monologueing in token-space still feels more like a hack than a real solution, but it does improve their performance a good bit nonetheless.

In practical terms all this means is that current LLMs still need a hell of a lot of hand holding and fail at anything more complex, even if their "System 1" thinking is good enough for the task (e.g. they can write Tetris in 30sec no problem, but they can't write SuperMarioBros at all, since that has numerous levels that would blow the context window size).

replies(1): >>43114402 #
12. technocrat8080 ◴[] No.43112804{3}[source]
Vastly overstating its capabilities? SF is ~crawling~ with them 24/7 and I've yet to meet someone who's had a bad experience in one of them. They operate more than well enough to replace rideshare drivers, and they have been.
replies(2): >>43112863 #>>43113432 #
13. dash2 ◴[] No.43112863{4}[source]
But SF is a single US city built on a grid. Try London or Manila.
replies(2): >>43113192 #>>43113605 #
14. Eikon ◴[] No.43112930{4}[source]
> But how are animals with nerve-centres or brains different? What do we think us humans do differently so we are not just very big probabilistic prediction systems?

If you believe in free will, then we are not.

15. krainboltgreene ◴[] No.43112995[source]
So has Kendrick Lamar’s’ hit song, but no one is suggesting that it has geopolitical implications.
16. spacebanana7 ◴[] No.43113043[source]
The same stack is required for other AI stuff like diffusion models as well.
17. namaria ◴[] No.43113192{5}[source]
That's usually how it goes with 'AI'. It is very impressive on the golden path, but the world is 80% edge cases.
18. ◴[] No.43113206{4}[source]
19. sampo ◴[] No.43113272{4}[source]
> But how are animals with nerve-centres or brains different?

In current LLM neural networks, the signal proceeds in one direction, from input, through the layers, to output. To the extend that LLM's have memory and feedback loops, it's that they write the output of the process to text, and then read that text and process it again though their unidirectional calculations.

Animal brains have circular signals and feedback loops.

There are Recurrent Neural Network (RNN) architectures, but current LLM's are not these.

20. Y-bar ◴[] No.43113432{4}[source]
SF has pretty much the best weather there is to drive in. Try putting them on Minnesota winter roads, or muddy roads in Kansas for example.
replies(1): >>43114373 #
21. habinero ◴[] No.43113546{4}[source]
> But how are animals with nerve-centres or brains different? What do we think us humans do differently so we are not just very big probabilistic prediction systems?

I see this statement thrown around a lot and I don't understand why. We don't process information like computers do. We don't learn like they do, either. We have huge portions of our brains dedicated to communication and problem solving. Clearly we're not stochastic parrots.

> if we develop the technology to engineer animal-style nerves and form them into big lumps called 'brains'

I think y'all vastly underestimate how complex and difficult a task this is.

It's not even "draw a circle, draw the rest of the owl", it's "draw a circle, build the rest of the Dyson sphere".

It's easy to _say_ it, it's easy to picture it, but actually doing it? We're basically at zero.

replies(1): >>43114490 #
22. rafaelmn ◴[] No.43113605{5}[source]
With nicest weather on the planet probably
23. fragmede ◴[] No.43114373{5}[source]
How stupid of Google. Instead of getting their self driving car technology to work in a blizzard first, and then working on getting it working in a city, they choose to get it working in a city first, before getting it to work in inclement weather. What idiots!
replies(1): >>43114590 #
24. fragmede ◴[] No.43114402{5}[source]
give it a filesystem, like you can with Claude computer use, and you can have it make and forget memories to adapt for a limited context window size
25. fragmede ◴[] No.43114490{5}[source]
> Clearly we're not stochastic parrots

On Internet comment sections, that's not clear to me. Memes are incredibly infectious, and we can see by looking at, say, a thread about Nvidia. It's inevitable that someone is going to ask about a moat. In a thread about LLMs, the likelihood of stoichastic parrots getting a mention approaches one, as the thread gets longer. what does it all mean?

replies(1): >>43116732 #
26. Y-bar ◴[] No.43114590{6}[source]
I hope you are sarcastic! Because it is quite expected that they would test where it is easy first. The stupid ones are those who parrot the incorrect assumption that self-driving cars are comparable to humans at general driving where statistics on general driving includes lots of driving in suboptimal condition.
27. staticman2 ◴[] No.43116732{6}[source]
You seem to be confusing brain design with uniqueness.

If every single human on earth was an identical clone with the same cultural upbringing and similar language conversation choices and opinions and feelings, they still wouldn't work like an LLM and still wouldn't be stochastic parots.