←back to thread

OpenAI, Google and Anthropic are struggling to build more advanced AI

(www.bloomberg.com)
625 points lukebennett | 1 comments | 13 Nov 24 13:28 UTC | HN request time: 0s | source
Show context
pluc ◴[14 Nov 24 18:23 UTC] No.42139375[source]▶
They've simply run out of data to use to fabricate legitimate-looking guesses. They can't create anything that doesn't already exist.
replies(7): >>42139490 #>>42140441 #>>42141114 #>>42141125 #>>42141590 #>>42141888 #>>42149715 #
whazor ◴[14 Nov 24 19:53 UTC] No.42140441[source]▶
But a LLM can certainly make up a lot information that never existed before.
replies(2): >>42141540 #>>42142063 #
bob1029 ◴[14 Nov 24 21:42 UTC] No.42141540[source]▶
I strongly believe this gets into an information theoretical constraint akin to why perpetual motion machines don't work.

In theory, yes you could generate an unlimited amount of data for the models, but how much of it is unique or valuable information? If you were to compress all this generated training data using a really good algorithm, how much actual information remains?

replies(3): >>42141792 #>>42141948 #>>42181780 #
1. moffkalast ◴[14 Nov 24 22:38 UTC] No.42141948{3}[source]▶
I make a lot of shitposts, how much of that is valuable information? Arguably not much. I doubt information value is a good way to estimate inteligence because most people's daily ramblings would grade them useless.