←back to thread

OpenAI, Google and Anthropic are struggling to build more advanced AI

(www.bloomberg.com)
625 points lukebennett | 2 comments | 13 Nov 24 13:28 UTC | HN request time: 0.451s | source
Show context
osigurdson ◴[15 Nov 24 06:35 UTC] No.42144420[source]▶
This "running out of data" thing suggests that there is something fundamentally wrong with how things are working. A new driver does not need to experience 8000 different rabbit-on-road situations from all angles to know to slow down when we see one on the road. Similarly we don't need 10,000 addition examples to learn how to add. It is as though there is no generalization in the models - just fundamentally search.
replies(2): >>42144498 #>>42149778 #
1. slashdave ◴[15 Nov 24 18:59 UTC] No.42149778[source]▶
Deep learning is the very opposite of generalization.
replies(1): >>42170301 #
2. pas ◴[18 Nov 24 06:42 UTC] No.42170301[source]▶
it's not that simple

"""

Intuitively, an overparameterized model will generalize well if the model’s representations capture the essential information necessary for the best model in the model class to perform well

"""

https://iclr-blogposts.github.io/2024/blog/double-descent-de...