I don't think AGI is right around the corner

Anyone who claims that a poorly definined concept, AGI, is right around the corner is most likely:

- trying to sell something

- high on their own stories

- high on exogenous compounds

- all of the above

LLMs are good at language. They are OK summarizers of text by design but not good at logic. Very poor at spatial reasoning and as a result poor at connecting concepts together.

Just ask any of the crown jewel LLM models "What's the biggest unsolved problem in the [insert any] field".

The usual result is a pop-science-level article but with ton of subtle yet critical mistakes! Even worse, the answer sounds profound on the surface. In reality, it's just crap.

Interesting. I think the key to what you wrote is "poorly definined".

I find LLMs to be generally intelligent. So I feel like "we are already there" -- by some definition of AGI. At least how I think of it.

Maybe a lot of people think of AGI as "superhuman". And by that definition, we are not there -- and may not get there.

But, for me, we are already at the era of AGI.

They’re great at working with the lens on our reality that is our text output. They are not truth seekers, which is necessarily fundamental to every life form from worms to whales. If we get things wrong, we die. If they get them wrong, they earn 1000 generated tokens.

Where does Eric Schmidt fit? Selling something?

I would call them "generally applicable". "intelligence" definitely implies leaning - and I'm not sure RAG, fine-tuning, or 6monthly updates counts - to split hairs.

Where I will say we have a massive gap, which makes the average person not consider it AGI, is in context. I can give a person my very modest codebase, and ask for a change, and they'll deliver - mostly coherently - to that style, files in the right place etc. Still to today with AI, I get inconsistent design, files in random spots, etc.

that's the thing about language. we all kinda gotta agree on the meanings

I think he's generally optimistic which is a net positive.

I'll offer a definition of AGI:

An AI (a computer program) that is better at [almost] any task than 5% of the human specialists in that field has achieved AGI.

Or, stated another way, if 5% of humans are incapable of performing any intellectual job better than an AI can, then that AI has achieved AGI.

Note, I am not saying that an AI that is better than humans at one particular thing has achieved AGI, because it is not "general". I'm saying that if a single AI is better at all intellectual tasks than some humans, the AI has achieved AGI.

The 5th percentile of humans deserves the label of "intelligent", even if they are not the most intelligent, (I'd say all humans deserve the label "intelligent") and if an AI is able to perform all intellectual tasks better than such a person, the AI has achieved AGI.

Alright, let’s get this straight.

You’ve got people foaming at the mouth anytime someone mentions AGI, like it’s some kind of cult prophecy. “Oh it’s poorly defined, it’s not around the corner, everyone talking about it is selling snake oil.” Give me a break. You don’t need a perfect definition to recognize that something big is happening. You just need eyes, ears, and a functioning brain stem.

Who cares if AGI isn’t five minutes away. That’s not the point. The point is we’ve built the closest thing to a machine that actually gets what we’re saying. That alone is insane. You type in a paragraph about your childhood trauma and it gives you back something more coherent than your therapist. You ask it to summarize a court ruling and it doesn’t need to check Wikipedia first. It remembers context. It adjusts to tone. It knows when you’re being sarcastic. You think that’s just “autocomplete”? That’s not autocomplete, that’s comprehension.

And the logic complaints, yeah, it screws up sometimes. So do you. So does your GPS, your doctor, your brain when you’re tired. You want flawless logic? Go build a calculator and stay out of adult conversations. This thing is learning from trillions of words and still does better than half the blowhards on HN. It doesn’t need to be perfect. It needs to be useful, and it already is.

And don’t give me that “it sounds profound but it’s really just crap” line. That’s 90 percent of academia. That’s every selfhelp book, every political speech, every guy with a podcast and a ring light. If sounding smarter than you while being wrong disqualifies a thing, then we better shut down half the planet.

Look, you’re not mad because it’s dumb. You’re mad because it’s not that dumb. It’s close. Close enough to feel threatening. Close enough to replace people who’ve been coasting on sounding smart instead of actually being smart. That’s what this is really about. Ego. Fear. Control.

So yeah, maybe it’s not AGI yet. But it’s smarter than the guy next to you at work. And he’s got a pension.

I like where this is going.

However, it's not sufficient. The actual tasks have to be written down, tests constructed, and the specialists tested.

A subset of this has been done with some rigor and AI/computers have surpassed this threshold for some tests. Some have then responded by saying that it isn't AGI, and that the tasks aren't sufficiently measuring of "intelligence" or some other word, and that more tests are warranted.

You're saying we need to write down all intellectual tasks? How would that help?

If an AI is better at some tasks (that happen to be written down), it doesn't mean it is better at all tasks.

Actually, I'd lower my threshold even further--I originally said 50%, then 20%, then 5%--but now I'll say if an AI is better than 0.1% of people at all intellectual tasks, then it is AGI, because it is "general" (being able to do all intellectual tasks), and it is "intelligent" (a label we ascribe to all humans).

But the AGI has to be better at all (not just some) intellectual tasks.

Its right around the corner when you prove it as fact. Otherwise as suggested it is just hype to sell us on your LLM flavor.

I think your definition is flawed.

Take the Artificial out of AGI. What is GI, and do the majority of humans have it? If so, then why is your definition of AGI far stricter than the definition of Human GI?

This is a good summary of what LLM offer today.

My company is desperately trying to incorporate AI (to tell investors they are). The fact that LLM gets thing wrong is a huge problem since most work can’t be wrong and if if a human needs to carefully go through output to check it, it’s often just as much work as having that same human just create the output themselves.

But languages is one place LLMs shine. We often need to translate technical docs to layman language and LLMs work great. It quickly find words and phrases to describe complex topics. Then a human can do a final round of revisions.

But anything de novo? Or requiring logic? It works about as well as a high school student with no background knowledge.

Fundamentally, they are really powerful text transformers with some additional capability. The further away from that sweet spot and the closer to anthropomorphization the more unreliable the output

Why do you say that LLMs are not truth seekers? If I express an informational query not very well, the LLM will infer what I mean by it and address the possible well-posed information queries that I may have intended that I did not express well.

Can that not be considered truth-seeking, with the agent-environment boundary being the prompt box?

Already invested in the AI companies selling you something.

They are not intrinsically truth seekers, and any truth seeking behaviour is mostly tuned during the training process.

Unfortunately it also means it can be easily undone. E.g. just look at Grok in its current lobotomized version

There's definitely also people in the futurism and/or doom and gloom camps with absolutely no skin in the game that can't resist this topic.

I agree with the last part but I think that criticism applies to many humans too so I don't find it compelling at all.

I also think by original definition (better than median human at almost all task) it's close and I think in the next 5 years it will be competitive with professionals at all tasks which are nonphysical (physical could be 5-10 years idk). I could be high on my own stories but not the rest.

LLMs are good at language yes but I think to be good at language requires some level of intelligence. I find this notion that they are bad at spatial reasoning extremely flawed. They are much better than all previous models, some of which are designed for spatial reasoning. Are they worse than humans? Yes but just the fact that you can put newer models on robots and they just work means that they are quite good by AI standards and rapidly improving.

> An AI (a computer program) that is better at [almost] any task than 5% of the human specialists in that field has achieved AGI.

Let's say you have a candidate AI and assert that it indeed has passed the above benchmark. How do you prove that? Don't you have to say which tasks?

Thousands are being laid off, supposedly because they're "being replaced with AI," implying the AI is as good or better as humans at these jobs. Managers and execs are workers, too--so if the AI really is so good, surely they should recuse themselves and go live a peaceful life with the wealth they've accrued.

I don't know about you, but I can't imagine that ever happening. To me, that alone is a tip off that this tech, while amazing, can't live up to the hype in the long term.

> They are not intrinsically truth seekers

Is the average person a truth seeker in this sense that performs truth-seeking behavior? In my experience we prioritize sharing the same perspectives and getting along well with others a lot more than a critical examination of the world.

In the sense that I just expressed, of figuring out the intention of a user's information query, that really isn't a tuned thing, it's inherent in generative models from possessing a lossy, compressed representation of training data, and it is also truth-seeking practiced by people that want to communicate.

Every few weeks I give LLMs a chance to code something for me.

Friday I laid out a problem very cleanly. Take this datastructure and tranform it into this other datastructure in terraform. With examples of the data in both formats.

After the seventh round of back and forth where it would give me code that would not compile or code that gave me a totally different datastructure, giving it more examples and clarifications all the while I gave up. I gave the problem to a junior and they came back with the answer in about an hour.

Next time an AI bro tells you that AI can 'replace your juniors' tell him to go to hell.

They keep giving me incorrect answers to verifiable questions. They clearly don't 'seek' anything.

I don't think anyone is being laid off because of AI. People are being laid off because the market is bad for a myriad of reasons, and companies are blaming AI because it helps them deflect worry that might lower their stock price.

Companies say "we've laid people off because we're using AI,x but they mean "we had to lay people off, were hoping we can make up for them with AI."

> Thousands are being laid off, supposedly because they're "being replaced with AI," implying the AI is as good or better as humans at these jobs.

I don't think the "implying the AI is as good or better as humans" part is correct. While they may not be saying it loudly, I think most folks making these decisions around AI and staffing are quite clear that AI is not as good as human workers.

They do, however, think that in many cases it is "good enough". Just look at like 90%+ of the physical goods we buy these days. Most of them are almost designed to fall apart after a few years. I think it's almost exactly analogous to the situation with the Luddites (which is often falsely remembered as the Luddites being "anti-technology", when in reality they were just "pro-not-starving-to-death"). In that case, new mechanized looms greatly threatened the livelihood of skilled weavers. The quality of the fabric from these looms tended to be much worse than those of the skilled weavers. But it was still "good enough" for most people such that most consumers preferred the worse but much cheaper cloth.

It's the same thing with AI. It's not that execs think it's "as good as humans", it's that if AI costs X to do something, and the human costs 50X (which is a fair differential I think), execs think people will be willing to put up with a lot shittier quality if the can be delivered something much more cheaply.

One final note - in some cases people clearly do prefer the quality of AI. There was an article on HN recently discussing that folks preferred Waymo taxis, even though they're more expensive.

Some employees can be replaced by AI. That part is true. It's not revolutionary (at least not yet) — it's pretty much the same as other post-industrial technologies that have automated some types of work in the past. It also takes time for industries to adapt to these changes. Replacing workers couldn't possibly happen in one year, even if our AI models were more far more capable than they are in practice

I'm afraid that what we're seeing instead are layoffs that are purely oriented at the stock market. As long as layoffs and talk about AI are seen as a positive signal for investors and as long as corporate leadership is judged by the direction the stock price goes, we will see layoffs (as well as separate hiring sprees for "AI Engineers").

It's a telltale sign that we're seeing a large number of layoffs in the tech sector. It is true that tech companies are poised to adapt AI more quickly than others but that doesn't seem to be what's happening. What seem to be happening is that tech companies have been overhiring throughout the decade leading up to the end of COVID-19. At that time hiring was a positive signal — now firing is.

I don't think these massive layoffs are good for tech companies in the long term, but since they mostly affect things that don't touch direct revenue generating operations, they won't hurt in the near-term and by the time company starts feeling the pain, the cause would be too long in the past to be remembered.

> I don't think anyone is being laid off because of AI.

I think that's demonstratively false. While many business leaders may be overstating it, there are some pretty clear cut cases of people losing their jobs to AI. Here are 2 articles from the Washington Post from 2 years ago:

https://archive.vn/C5syl "ChatGPT took their jobs. Now they walk dogs and fix air conditioners."

https://archive.vn/cFWmX "ChatGPT provided better customer service than his staff. He fired them."

Well, to state it crudely, you just have to find a dumb person who is inferior to the AI at every single intellectual task. This is cruel, and I don't envy that dumb person, but who knows, I might end up being that dumb person--we all might.

The wave of layoffs started couple of years before the AI craze (ChatGPT).

My definition is a high-bar that is undeniably AGI. My personal opinion is that there are some lower-bars that are also AGI. I actually think it's fair to call LLMs from GPT3 onward AGI.

But, when it comes to the lower-bars, we can spend a lot of time arguing over the definition of a single term, which isn't especially helpful.

LLM are a compressed version of their training dataset with a text based interactive search function.

> Managers and execs are workers, too--so if the AI really is so good, surely they should recuse themselves and go live a peaceful life

One thing that doesn't get mentioned is AI capability for being held accountable. AI is fundamentally unaccountable. Like the genie from the lamp, it will grant you the 3 wishes but you bear the consequences.

So what can we do when the tasks are critically important, like deciding on an investment or spending much time and resources on a pursuit? We still need the managers. We need humans for all tasks of consequence where risks are taken. Not because humans are smarter, but because we have skin.

Even on the other side, that of goals, desires, choosing problems to be solved - AI has nothing to say. It has no desires of its own. It needs humans to expose the problem space inside which AI could generate value. It generates no value of its own.

This second observation means AI value will not concentrate in the hands of a few, but instead will be widespread. It's no different than Linux, yes, it has a high initial development cost, but then it generates value in the application layer which is as distributed as it gets. Each human using Linux exposes their own problems to the software to get help, and value is distributed across all problem contexts.

I have come to think that generating the opportunity for AI to provide value, and then incurring the outcomes, good or bad, of that work, are fundamentally human and distributed across society.

I think any task-based assessment of intelligence is missing the mark. Highly intelligent people are not considered smart just because they can accomplish tasks.

>Is the average person a truth seeker in this sense that performs truth-seeking behavior?

Absolutely

LLMs require the sum total of human knowledge to ape what you can find on google, meanwhile Ramanujan achieved brilliant discoveries in mathematics using nothing but a grade school education and a few math books.

Something big is definitely happening but it's not the intelligence explosion utopia that the AI companies are promising.

> Who cares if AGI isn’t five minutes away. That’s not the point. The point is we’ve built the closest thing to a machine that actually gets what we’re saying. That alone is insane. You type in a paragraph about your childhood trauma and it gives you back something more coherent than your therapist. You ask it to summarize a court ruling and it doesn’t need to check Wikipedia first. It remembers context. It adjusts to tone. It knows when you’re being sarcastic. You think that’s just “autocomplete”? That’s not autocomplete, that’s comprehension

My experience with LLMs have been all over the place. They're insanely good at comprehending language. As a side effect, they're also decent at comprehending complicated concepts like math or programming since most of human knowledge is embedded in language. This does not mean they have a thorough understanding of those concepts. It is very easy to trip them up. They also fail in ways that are not obvious to people who aren't experts on whatever is the subject of its output.

> And the logic complaints, yeah, it screws up sometimes. So do you. So does your GPS, your doctor, your brain when you’re tired. You want flawless logic? Go build a calculator and stay out of adult conversations. This thing is learning from trillions of words and still does better than half the blowhards on HN. It doesn’t need to be perfect. It needs to be useful, and it already is.

I feel like this is handwaving away the shortcomings a bit too much. It does not screw up in the same way humans do. Not even close. Besides, I think computers should rightfully be held up to a higher standard. We already have programs that can automate tasks that human brains would find challenging and tedious to do. Surely the next frontier is something with the speed and accuracy of a computer while also having the adaptability of human reasoning.

I don't feel threatened by LLMs. I definitely feel threatened by some of the absurd amount of money being put into them though. I think most of us here will be feeling some pain if a correction happens.

Most on HN are tech people and it is tiring to see they did not just spend a sunday morning doing a Karpathy llm implementation or so. Somehow, like believing in a deity, even smart folk seem to think 'there is more'. Stop. Go to youtube or whatever and watch a video of practically implementing a gpt like thing, and code along. It takes very little time and your hallucinations about agi with these models shall be exorcized.

There's a lot in here. I agree with a lot of it.

However, you've shifted the goal post from AGI to being useful in specific scenarios. I have no problem with that statement. It can write decent unit tests and even find hard-to-spot, trivial mistakes in code. But again, why can it do that? Because a version of that same mistake is in the enormous data set. It's a fantastic search engine!

Yet, it is not AGI.

Yes, but you're missing their ability to interpolate across that dataset at retrieval time, which is what makes them extremely useful. Also, people are willing to invest a lot of money to keep building those datasets, until nearly everything of economic value is in there.

not everything of economic value is data retrieval

Okay, but then its not so much a definition. It's more like a test.

LLMs are useful in that respect. As are media diffusion models. They've compressed the physics of light, the rules of composition, the structure of prose, the knowledge of the internet, etc. and made it infinitely remixable and accessible to laypersons.

AGI, on the other hand, should really stand for Aspirationally Grifting Investors.

Superintelligence is not around the corner. OpenAI knows this and is trying to become a hyperscaler / Mag7 company with the foothold they've established and the capital that they've raised. Despite that, they need a tremendous amount of additional capital to will themselves into becoming the next new Google. The best way to do that is to sell the idea of superintelligence.

AGI is a grift. We don't even have a definition for it.

You phrase it as a diss but "Yeah LLM suck, they aren't even as smart as Ramanujan" sounds like a high praise to me.

> Some employees can be replaced by AI.

Yes, but not lets pretend that there aren't a lot of middle and even upper management that couldn't also be replaced by AI.

Of course they won't be because they are the ones making the decisions.

I find it kind of funny that in order to talk to AI people, you need to preface your paragraph with "I find current AI amazing, but...". It's like, you guess it, pre-prompting them for better acceptance.

I don't know if you are indirectly referring to me, but I have done such an implementation, and those particular LLMs are very limited. Two things come to mind.

1. It is still correct that the limited "truth-seeking" that I expressed holds. With respect to the limited world model possessed by the limited training and limited dataset, such a model "seeks to understand" the approximate concept that I am imperfectly expressing that it has data for, and then generate responses based in that.

2. SotA models have access to external data, be it web search or RAG+vector database, etc.. They also have access to the Chain of Thought method. They are trained on datasets that enable them to exploit these tools, and will exploit these tools. The zero-to-hero sequence does not lead you to build such an LLM, and the one that you build has a very limited computational graph. So with respect to more... traditional notions of "truth seeking", these LLMs fundamentally lack the equipment to do that that SotA models have.

Most economic value is not data retrieval

In the sense that I expressed, has it not already then sought out an accurate meaning that you have asked? And then failed to give a satisfactory answer? I would also ask: is said model an advertised "reasoning" model? Also, does it have access to external facts via a tool like web search? I would not expect great ability to "arrive at truth" under certain limitations.

Now, you can't conclude that "they clearly don't 'seek' anything" just by the fact that they got an answer wrong. To use the broad notion of "seeking" like you do, a truth seeker with limited knowledge and equipment would arrive confidently at incorrect conclusions based on accurate reasoning. For example, without modern lenses to detect stellar parallax, one would confidently conclude that the stars in the sky are a different thing than the sun (and planets), since one travels across the sky, but the stars are fixed. Plato indeed thought so, and nobody would accuse him of not being a truth-seeker.

If this is what you had in mind, I hope that I have addressed it, otherwise I hope that you can communicate what you mean with an example.

Right now you’re putting in unrequested effort to get to an answer. Nobody is driving you to do this, you’re motivated to get the answer. At some point you’ll be satisfied, or you might give up because you have other things you want to do, more.

An LLM is primarily trying to generate content. It’ll throw the best tokens in there but it won’t lose any sleep if they’re suboptimal. It just doesn’t seek. It won’t come back an hour later and say “you know, I was thinking…”

I had one frustrating conversation with ChatGPT where I kept asking it to remove a tie from a picture it generated. It kept saying “done, here’s the picture without the tie”, but the tie was still there. Repeatedly. Or it’ll generate a reference or number that is untrue but looks approximately correct. If you did that you’d be absolutely mortified and you’d never do it again. You’d feel shame and a deep desire to be seen as someone who does it properly. It doesn’t have any such drive. Zero fucks given, training finished months ago.

The stock market is the root for the majority of all the world's economic value, and has almost-exclusively been data retrieval since 2001.

You say it's just a fancy search engine. Great. You know what else is a fancy search engine? Your brain. You think you're coming up with original thoughts every time you open your mouth? No. You're regurgitating every book, every conversation, every screw-up you've ever witnessed. The brain is pattern matching with hormones. That’s it.

Now you say I'm moving the goalposts. No, I’m knocking down the imaginary ones. Because this whole AGI debate has turned into a religion. “Oh it’s not AGI unless it can feel sadness, do backflips, and write a symphony from scratch.” Get over yourself. We don’t even agree on what intelligence is. Half the country thinks astrology is real and you’re here demanding philosophical purity from a machine that can debug code, explain calculus, and speak five languages at once? What are we doing?

You admit it’s useful. You admit it catches subtle bugs, writes code, gives explanations. But then you throw your hands up and go, “Yeah, but that’s just memorization.” You mean like literally how humans learn everything? You think Einstein invented relativity in a vacuum? No. He stood on Newton, who stood on Galileo, who probably stood on a guy who thought the stars were angry gods. It’s all remixing. Intelligence isn’t starting from zero. It’s doing something new with what you’ve seen.

So what if the model’s drawing from a giant dataset? That’s not a bug. That’s the point. It’s not pulling one answer like a Google search. It’s constructing patterns, responding in context, and holding a conversation that feels coherent. If a human did that, we’d say they’re smart. But if a model does it, suddenly it’s “just autocomplete.”

You know who moves the goalposts? The people who can’t stand that this thing is creeping into their lane. So yeah, maybe it's not AGI in your perfectly polished textbook sense. But it's the first thing that makes the question real. And if you don’t see that, maybe you’re not arguing from logic. Maybe you’re just pissed.

I an not an expert but I have a serious counterpoint.

While training LLMs to replicate the human output, the intelligence and understanding EMERGES in the internal layers.

It seems trivial to do unsupervised training on scientific data, for instance, such as star movements, and discover closed-form analytic models for their movements. Deriving Kepler’s laws and Newton’s equations should be fast and trivial, and by that afternoon you’d have much more profound models with 500+ variables which humans would struggle to understand but can explain the data.

AGI is what, Artificial General Intelligence? What exactly do we mean by general? Mark Twain said “we are all idiots, just on different subjects”. These LLMs are already better than 90% of humans at understanding any subject, in the sense of answering questions about that subject and carrying on meaningful and reasonable discussion. Yes occasionally they stumble or make a mistake, but overall it is very impressive.

And remember — if we care about practical outcomes - as soon as ONE model can do something, ALL COPIES OF IT CAN. So you can reliably get unlimited agents that are better than 90% of humans at understanding every subject. That is a very powerful baseline for replacing most jobs, isn’t it?

You say LLMs are “insanely good” at comprehending language, but then immediately pull back like it’s some kind of fluke. “Well yeah, it looks like it understands, but it doesn’t really understand.” What does that even mean? Do you think your average person walking around fully understands everything they say? Half of the people you know are just repeating crap they heard from someone else. You ask them to explain it and they fold like a cheap tent. But we still count them as sentient.

Then you say it’s easy to trip them up. Of course it is. You know what else is easy to trip up? People. Ask someone to do long division without a calculator. Ask a junior dev to write a recursive function that doesn’t melt the stack. Mistakes aren’t proof of stupidity. They’re proof of limits. And everything has limits. LLMs don’t need to be flawless. They need to be better than the tool they’re replacing. And in a lot of cases, they already are.

Now this part: “computers should be held to a higher standard.” Why? Says who? If your standard is perfection, then nothing makes the cut. Not the car, not your phone, not your microwave. We use tools because they’re better than doing it by hand, not because they’re infallible gods of logic. You want perfection? Go yell at the compiler, not the language model.

And then, this one really gets me, you say “surely the next frontier is a computer with the accuracy of a machine and the reasoning of a human.” No kidding. That’s the whole point. That’s literally the road we’re on. But instead of acknowledging that we’re halfway there, you’re throwing a tantrum because we didn’t teleport straight to the finish line. It’s like yelling at the Wright brothers because their plane couldn’t fly to Paris.

As for the money... of course there's a flood of it. That’s how innovation happens. Capital flows to power. If you’re worried about a correction, fine. But don’t confuse financial hype with technical stagnation. The tools are getting better. Fast. Whether the market overheats is a separate issue.

You say you're not threatened by LLMs. That’s cute. You’re writing paragraphs trying to prove why they’re not that smart while admitting they’re already better at language than most people. If you’re not threatened, you’re sure spending a lot of energy trying to make sure nobody else is impressed either.

Look, you don’t have to worship the thing. But pretending it's just a fancy parrot with a glitchy brain is getting old. It’s smart. It’s flawed. It’s changing everything. Deal with it.

Oh come on, it's not some secret code. People say “AI is amazing, but...” because it is amazing... and also flawed. That’s just called having a balanced take, not pre-prompting for approval. What do you want them to do, scream “THIS SUCKS” and ignore reality? It’s not a trick, it’s just how grown-ups talk when they’re not trying to win internet points.

You are completely missing the argument that was made to underline the claim.

If ChatGPT claims arsenic to be a tasty snack, nothing happens to it.

If I claim the same, and act upon it, I die.

Unfortunately LLMs fail even basic logic tests given to children so definitely not high praise. I'm just highlighting the absurd amount of data they need versus humans to highlight that they're just spitting out regressions on the training data. We're talking data that would take a human countless thousands of lifetimes to ingest. Yet a human can accomplish infinitely more with a basic grade school education.

If ChatGPT claims arsenic to be a tasty snack, OpenAI adds a p0 eval and snuffs that behavior out of all future generations of ChatGPT. Viewed vaguely in faux genetic terms, the "tasty arsenic gene" has been quickly wiped out of the population, never to return.

Evolution is much less brutal and efficient. To you death matters a lot more than being trained to avoid a response does to ChatGPT, but from the point of view of the "tasty arsenic" behavior, it's the same.

Come on. The stock market is not just data retrieval. The statement doesn’t even make sense.

I don't understand, you'll have to give an example.

What is the most non-task-like thing that highly intelligent people do as a sign of their intelligence?

It makes perfect sense, and I meant what I said.

60% of all US equity volume is pure high-frequency trading, and ETFs add roughly another 20% that’s literally just bots responding to market activity and bearish-bullish sentiment analysis on public(?) press releases. 2/3 of trading funds also rely on external data to price in decisions, and I think it was around 90% in 2021 use trading algorithms as their determining factor for their high-frequency trade strategies.

At its core, the movements that make up the market really IS data retrieval.

Of course, I have no original thoughts. Something is not created out of nothing. That would be Astrology, perhaps :).

But the difference between a human and an LLM is that humans can go out in the world and test their hypothesis. Literally every second is an interaction with a feedback loop. Even typing this response to you right now. LLMs currently have to wait for the next 6-month retraining cycle. I am not saying that AGI cannot be created. In theory it can be but we are definitely milking the crap out of a local maximum we've currently found which is definitely not the final answer.

PS Also, when I said "it can spot mistakes," I probably gave it too much credit. In one case, it presented several potential issues, and I happened to notice that one of them was a problem. In many cases, the LLM suggests issues that are either hypothetical or nonexistent.

Sure, the market, but HFT is relatively tiny as a market and the profit it brings. Not to mention, it's essentially a zero-sum game.

Brought to you by your favorite Google LLM search result:

"The global high-frequency trading (HFT) market was valued at USD 10.36 billion in 2024 and is projected to reach USD 16.03 billion by 2030"

(unverified by a human, use at your own risk).

>"The global high-frequency trading (HFT) market was valued at USD 10.36 billion in 2024 and is projected to reach USD 16.03 billion by 2030"

>

> (unverified by a human, use at your own risk).

Honorable for mentioning the lack of verification; doing so would have dissolved the AI's statement, but jury's out on how much EXACTLY:

Per https://www.sciencedirect.com/science/article/abs/pii/S03784...:

"While estimates vary due to the difficulty in ascertaining whether each trade is an HFT, recent estimates suggest HFT accounts for 50–70% of equity trades and around 50% of the futures market in the U.S., 40% in Canada, and 35% in London (Zhang, 2010, Grant, 2011, O’Reilly, 2012, Easley et al., 2012, Scholtus et al., 2014)"

In my original reply, I used the literal median of that spectrum @ 60%

Jane Street - who has recently found themselves in hot water from the India ban - disputes that AI summary ALONE. Per https://www.globaltrading.net/jane-street-took-10-of-of-us-e... , Jane Street booked 20.5B in trading revenue, primarily though HFT's, just in 2024.

Brought to you by someone who takes these market movements too seriously for their own good.

The stock market does not grow potatoes.

And potatoes don't grow nearly as much economic value within industrial societies - as they do in, say, agrarian ones. All to say, I don't understand your point.

The stock market does not make movies either.

Humans can achieve more within one (or two, or a few) narrowly scoped field(s), after a lot of hard work and effort. LLMs can display a basic level of competency (with some mistakes) in almost any topic known to mankind. No one reasonably expects a LLM to be able to do the former, and humans certainly cannot do the latter.

You're comparing apples and oranges.

Also, your comparison is unfair. You've chosen an exceptional high achiever as your example of a human to compare against LLMs. If you instead compare the average human, LLMs don't look so bad even when the human has the advantage of specialisation (e.g. medical diagnostics). A LLM can do reasonably well against an average (not exceptional) person with just a basic grade school education if asked to produce an essay on some topic.

Not surprising people like Waymos even though they are a bit more expensive. For a few more dollars you get:

- arguably a very nice, clean car

- same, ahem, Driver and driving style

With the basic UberX it’s a crapshoot. Good drivers, wild drivers, open windows, no air-con. UberX Comfort is better but there’s still a range.

You are right. I have ignored completely the context in the phrasing "truth seeker" was made, given my own wrong interpretation to the phrase, and I in fact agree with the comment I was responding to that they "work with the lens on our reality that is our text output".

>Humans can achieve more within one (or two, or a few) narrowly scoped field(s), after a lot of hard work and effort.

>No one reasonably expects a LLM to be able to do the former

I can feel Sam Altman's rage building ...

Anthropomorphization is doing a lot of heavy lifting in your comment.

> While training LLMs to replicate the human output, the intelligence and understanding EMERGES in the internal layers.

Is it intelligence and understanding that emerges, or is applying clever statistics on the sum of human knowledge capable of surfacing patterns in the data that humans have never considered?

If this were truly intelligence we would see groundbreaking advancements in all industries even at this early stage. We've seen a few, which is expected when the approach is to brute force these systems into finding actually valuable patterns in the data. The rest of the time they generate unusable garbage that passes for insightful because most humans are not domain experts, and verifying correctness is often labor intensive.

> These LLMs are already better than 90% of humans at understanding any subject, in the sense of answering questions about that subject and carrying on meaningful and reasonable discussion.

Again, exceptional pattern matching does not imply understanding. Just because these tools are able to generate patterns that mimic human-made patterns, doesn't mean they understand anything about what they're generating. In fact, they'll be able to tell you this if you ask them.

> Yes occasionally they stumble or make a mistake, but overall it is very impressive.

This can still be very impressive, no doubt, and can have profound impact on many industries and our society. But it's important to be realistic about what the technology is and does, and not repeat what some tech bros whose income depends on this narrative tell us it is and does.

Revenue is not profit

The percentage is irrelevant without knowing how they really work and how much profit they make. They could be at 95% with 0.1% of margin it wouldn’t mean much for the market.

At the end of the day talking about HFT this way is to not know what they do and what service they offer to the market. Overall they are not trending makers but trend followers.

Exactly I am so tired to hear about AI… And they are not even AI! I am also losing faith in this field when I see how much they all push so much hype and lies like this instead of being transparent. They are not AGIs not even AIs… For now they are only models and your definition is a good one

You sure spend a lot of energy and time living out this psychodrama.

If it's so self evidently revolutionary, why do you feel the need to argue about it?

Because hypetrain.

Yeah, I think many investors do expect that ...

> I find LLMs to be generally intelligent. So I feel like "we are already there" -- by some definition of AGI. At least how I think of it.

I don't disagree - they are useful in many cases and exhibit human like (or better) performance in many tasks. However they cannot simply be a "drop in white collar worker" yet, they are too jagged and unreliable, don't have a real memory etc. Their economic impact is still very much limited. I think this is what many people mean when they say AGI - something with a cognitive performance so good it equals or beats humans in the real world, at their jobs - not at some benchmark.

One could ask - does it matter ? Why can't we say the current tools are great task solvers and call it AGI even if they are bad agents? It's a lengthy discussion to have but I think that ultimately yes, agentic reliability really matters.

> Of course they won't be because they are the ones making the decisions.

That's not accurate at all

https://www.businessinsider.com/microsoft-amazon-google-embr...

Indeed 90% problems can be solved by googling and that's what LLMs do. AGI is expected to be something more than a talking encyclopedia.

I hate the "accessible to the layperson" argument.

People who couldn't do art before, still can't do art. Asking someone, or something else, to make a picture for you does not mean you created it.

And art was already accessible to anyone. If you couldn't draw something (because you never invested the time to learn the skill), then you could still pay someone else to paint it for you. We didn't call "commissioning a painting" as "being an artist", so what's different about "commissioning a painting from a robot?"

Why is that a net positive?

  > 60% of all US equity volume

Volume is not value.

I’m trying to fix humanity by smoothing out the sections that are deficient in awareness and IQ.

We need humanity at its best to prepare for the upcoming onslaught when something better tries to replace us. I do it for mankind.

I spent an hour on thrusday trying to get some code that would convert one data structure to another in terraform's HCL (which I only deal with once every few years and I find it's looping and eccentricities very annoying).

I opened my 'conversation' with a very clearly presented 'problem statement'. Given this datastructure (with code and an example with data) convert it to this datastructure (with code and the same example data transformed) in terraform.

I went through seven rounds of it presenting me either code that was not syntactically correct or produced a totally different datastructure. Every time it apologized for getting it wrong and then coming back with yet another wrong answer.

I stopped having the conversation when my junior who I also presented the problem to came back with a proper answer.

I'm not talking about it trying to prove to me that trump actually won the 2020 election or that vaccines don't cause autism or anything. Just actual 2+2=4 answers. Much like, in another reply to this post, the guy who had it try to find all the states that have w in their name.

The stock market is not the root of the value, it's where the value (and plans for generating more) is put to a popularity contest.

20 billion is a tiny fraction of the value represented in the stock exchange (and a tiny fraction of the profits made on it). HFT by its nature makes for a lot of volume but that's just a lot of shuffling of things around to peel a tiny fraction of value off the top, it's far from driving the market and the market isn't what makes the value in the first place.

> I hate the "accessible to the layperson" argument.

Accessible to a layperson also means lowering the gradient slope of learning.

Millions of people who would have never rented a camera from a rental house are now trying to work with these tools.

Those publishing "slop" on TikTok are learning the Hero's Journey and narrative structure. They're getting schooled on the 180-degree rule. They're figuring out how to tell stories.

> People who couldn't do art before, still can't do art. Asking someone, or something else, to make a picture for you does not mean you created it.

Speak for yourself.

I'm not an illustrator, but I'm a filmmaker in the photons-on-glass sense. Now I can use image and video models to make animation.

I agree that your average Joe isn't going to be able to make a Scorsese-inspired flick, but I know what I'm doing. And for me, these tools open an entire new universe.

Something like this still takes an entire week of work, even when using AI:

https://www.youtube.com/watch?v=tAAiiKteM-U

There's lots of editing, rotoscoping, compositing, grading, etc. and the AI models themselves are INSANELY finicky and take a lot of work to finesse.

But it would take months of work if you were posing the miniatures yourself.

With all the thought and intention and work that goes into something like this, would you still say it "does not mean you created it"? Do you still think this hasn't democratized access to a new form of expression for non-animators?

AI is a creative set of tools that make creation easier, faster, more approachable, and more affordable. They're accessible enough that every kid hustling on YouTube and TikTok can now supercharge their work. And they're going to have to use these tools to even stay treading water amongst their peers, because if they don't use them, their competition (for time and attention) will.

> Superintelligence is not around the corner. OpenAI knows this and is trying to become a hyperscaler / Mag7 company with the foothold they've established and the capital that they've raised.

+1 to this. I've often wondered why OpenAI is exploring so many different product ideas if they think AGI/ASI is less than a handful of years away. If you truly believe that, you would put all your resources behind that to increase the probability / pull-in the timelines even more. However, if you internally realized that AGI/ASI is much farther away, but that there is a technology overhang with lots of products possible on existing LLM tech, then you would build up a large applications effort with ambitions to join the Mag7.

Definitions around AI have been changing since the beginning, making it always farther in the future. In this system it can always be "right around the corner" but never arrive.

I stand corrected.

With Google I can demonstrate a wide breadth of knowledge too. LLM's aren't unique in that aspect.