←back to thread

OpenAI, Google and Anthropic are struggling to build more advanced AI

(www.bloomberg.com)
625 points lukebennett | 4 comments | 13 Nov 24 13:28 UTC | HN request time: 0.423s | source
Show context
iandanforth ◴[14 Nov 24 18:26 UTC] No.42139410[source]▶
A few important things to remember here:

The best engineering minds have been focused on scaling transformer pre and post training for the last three years because they had good reason to believe it would work, and it has up until now.

Progress has been measured against benchmarks which are / were largely solvable with scale.

There is another emerging paradigm which is still small(er) scale but showing remarkable results. That's full multi-modal training with embodied agents (aka robots). 1x, Figure, Physical Intelligence, Tesla are all making rapid progress on functionality which is definitely beyond frontier LLMs because it is distinctly different.

OpenAI/Google/Anthropic are not ignorant of this trend and are also reviving or investing in robots or robot-like research.

So while Orion and Claude 3.5 opus may not be another shocking giant leap forward, that does not mean that there arn't giant shocking leaps forward coming from slightly different directions.

replies(9): >>42139779 #>>42139984 #>>42140069 #>>42140194 #>>42140421 #>>42141563 #>>42142249 #>>42142983 #>>42143148 #
1. joe_the_user ◴[14 Nov 24 18:55 UTC] No.42139779[source]▶
Tesla are all making rapid progress on functionality which is definitely beyond frontier LLMs because it is distinctly different

Sure, that's tautologically true but that doesn't imply that beyondness will lead to significant leaps that offer notable utility like LLMs. Deep Learning overall has been a way around the problem that intelligent behavior is very hard to code and no wants to hire many, many coders needed to do this (and no one actually how to get a mass of programmers to actually be useful beyond a certain of project complexity, to boot). People take the "bitter lesson" to mean data can do anything but I'd say a second bitter lesson is that data-things are the low hanging fruit.

Moreover, robot behavior is especially to fake. Impressive robot demos have been happening for decades without said robots getting the ability to act effectively in the complex, ad-hoc environment that human live in, IE, work with people or even cheaply emulate human behavior (but they can do choreographed/puppeteered kung fu on stage).

replies(2): >>42139926 #>>42143654 #
2. hobs ◴[14 Nov 24 19:08 UTC] No.42139926[source]▶
And worth noting that Tesla faked a ton of its robot footage already, they might be making progress but their physical human robotics does not seem advanced at the moment.
replies(1): >>42140939 #
3. ben_w ◴[14 Nov 24 20:38 UTC] No.42140939[source]▶
Indeed.

Even assuming the recent robot demo was entirely AI, the only single thing they demonstrated that would have been noteworthy was isolating one voice in a noisy crowd well enough to respond; everything else I saw Optimus do, has already been demonstrated by others.

What makes the uncertainty extra sad, is that a remote controllable humanoid robot is already directly useful for work in hazardous environments, and we know they've got at least that… but Musk would rather it be about the AI.

4. hereme888 ◴[15 Nov 24 03:10 UTC] No.42143654[source]▶
Are we humans so different? Why do you wear what you wear? People emulate their older siblings, and so learn behavior. LLMs can create new programs, after having initially learned similar examples from others. Likewise for AI media.