There is also some moat in the refinement process (rlhf, model "safety" etc)
The lead is as strong as ever. They are 34 ELO above anyone else in blind testing, and 73 ELO above in coding [1]. They also seem to have artificially constrain the lead as they already have stronger model like o1 which they haven't released. Consistent to the past, they seem to release just <50 ELO above anyone else, and upgrades the model in weeks when someone gets closer.
[1]: https://lmarena.ai/
The moat as usual is extraordinary scale, resources, time. Nobody is putting $10 billion into the 7th OpenAI clone. Big tech isn't aggressively partnering with the 7th OpenAI clone. The door is already shut to that 7th OpenAI clone (they can never succeed or catch-up), there's just an enormous amount of naivety in tech circles about how things work in the real world: I can just spin up a ChatGPT competitor over the weekend on my 5090, therefore OpenAI have no barriers to entry, etc.
HN used to endlessly talk about how Uber could be cloned in a weekend. It's just people talking about something they don't actually understand. They might understand writing code (or similar) and their bias extends from the premise that their thing is the hard part of the equation (writing the code, building an app, is very far from the hardest part of the equation for an Uber).
Not even close to everything.
E.g. training on the NY Times and Wikipedia has zero meaningful AI. Training on books from reputable publishers similarly has zero meaningful AI. Any LLM usage was to polish prose or assist with research or whatever, but shouldn't affect the factual quality in any significant way.
The web hasn't been polluted with AI any more than e-mail has been polluted with spam. Which is to say it's there, but it's also entirely viable to separate. Nobody's worried that the group email chain with friends is being overrun with spam or with AI.
If these models will be trained on the outputs of themselves (and other models), then it's not so much a "flywheel", as it is a Perpetual Motion Machine.
The second idea being kicked around is synthetic data will create a new fountain of youth for data that will also fix its reasoning abilities.
This is clearly evident to anyone who spends any amount of time working on non-trivial projects with both models.
These companies will never admit it but AI is built on the back of piracy archives, easiest way and cheapest way to getting massive amounts of quality data.
The new O1 models by open ai are surprisingly good. So good that you can see intelligence at work through solutions when you go from:
GPT4 O1 mini O1 preview
Way better than the competition. And they have a mobile voice app with advanced voice which is phenomenal.
Even if the latter becomes commoditized (and we are far from that in practice), the former is a serious mote. Just like their is no secret to building a search engine or a social network platform (and that is not saying their are no technical challanges), operating it profitably requires massive aggregate user profile exploitation potential which requires huge upfront loss leaders.
I'm asking because I know that with some prompts it gets the answer correct, and in those cases nothing in the tokenization has changed.
OpenAI is not really leading the LLM world anyway ever since Claude sonnet 3.5 came out.
Every OpenAI thread focuses on the moat, but surely that was baked into their business dealings of the last 60 days.
Twenty-four years later I still regret not being able to raise money to enable us to keep working on that nascent startup. In most ways it was still too early. Google was still burning through VC money at that point and the midwestern investors we had some access to didn't get it. And, honestly they were probably correct. Compute power was still too expensive and quality data sources like published text were mostly locked up and generally not available to harvest.
But in general it’s hard to separate human from AI text.
Honestly the folks who don’t want to admit that it’s tokenization are just extremely salty that AI is actually good right now. Your “AI couldn’t tell me how many Rs in strawberry” stuff is extreme cope for your job prospects evaporating from a system that can’t spell correctly.
Ha. Really though, the entirety of venture capital could be summarized as “this probably won’t pay out but if it does it’s gonna be epic”. I wouldn’t read too much into America’s thirstiest capitalists spending their discretionary billions on the latest hype cycle.
The Wikipedia part at least, is incorrect. Currently Wikipedia mods/admins are dealing with AI generated articles being uploaded.
As for NYT - I am assuming that lots of those stories are already available in some blog or the other.
The e-mail and web forums are 100% polluted with spam, which takes constant effort to remove. For GenAI based content, it is far harder to identify and remove.
This example assumes the effort required to keep the web functional can deal with AI created content. Speaking from experience, our filters (human and otherwise) cannot. They fail to do so even now.
PS: Even given your example of closed email chains - the information in that depends on sources people read. Like plastic pollution in the food chain, this is inescapable.
It's not about a heuristic on text of unknown provenance -- it's about publishers that exert a certain level of editorial control and quality verification. Or social reputation mechanisms that achieve the same.
That's what is preventing your "model collapse". Reputations of provenance. Not pure-text heuristics.
And they've always dealt with spam and low-quality submissions before. The system is working.
> As for NYT - I am assuming that lots of those stories are already available in some blog or the other.
I don't know what relevance that has to what we're talking about. The point is, train on the NYT. Blogs don't change what's on the NYT.
> The e-mail and web forums are 100% polluted with spam, which takes constant effort to remove.
They've always been polluted with low-quality content. So yes, either don't train on them, or only train on highly upvoted solutions, etc.
AI pollution isn't fundamentally any different from previous low-quality content and spam. It's not terribly difficult to determine which parts of the internet are known to be high-quality and train only on those. LLM's can't spam the NY Times.
>The point is, train on the NYT. Blogs don't change what's on the NYT.
The counter point is that NYT content is already in the training data because its already replicated or copied into random blogs.
>So yes, either don't train on them, or only train on highly upvoted solutions, etc.
Highly upvoted messages on reddit are very regular bots copying older top comments. Mods already have issues with AI comments.
----
TLDR: Pollution is already happening. Verification does not scale, while generation scales.
That's not a counter point. My point is, train on things like the NYT, not random blogs. You can also whitelist the blogs you know are written by people, rather than randomly spidering the whole internet.
Also, no -- most of the NYT hasn't been copied into blogs. A small proportion of top articles, maybe.
> Highly upvoted messages on reddit are very regular bots copying older top comments.
What does that matter if the older top comment was written by a person? Also, Reddit is not somewhere you want to train in the first place if you're trying to generate a model where factual accuracy matters.
> Verification does not scale, while generation scales.
You don't need to verify everything -- you just need to verify enough stuff to train a model on. We're always going to have plenty of stuff that's sufficiently verified, whether from newspapers or Wikipedia or whitelisted blogs or books from verified publishers or whatever. It's not a problem.
You shouldn't be training on blogspam from random untrusted domains in the first place. So it doesn't matter if that junk is AI-generated or not.
> Also, Reddit is not somewhere you want to train in the first place if you're trying to generate a model where factual accuracy matters.
There is no model that can create facutal accuracy. This would basically contravene the laws of physics. LLMs predict the next token.
>You shouldn't be training on blogspam from random untrusted domains in the first place. So it doesn't matter if that junk is AI-generated or not
Afaik, all the current models are trained on this corpus. That is how they work.
Factual accuracy is not binary, it is a matter of degrees. Obviously training on content that is more factually correct will result in more factually correct next tokens. This is a pretty fundamental aspect of LLM's.
> Afaik, all the current models are trained on this corpus.
Then apologies for being so blunt, but you know wrong. There is a tremendous amount of work that goes on by the LLM companies in verifying, sanitizing, and structuring the training corpuses, using a wide array of techniques. The are absolutely not just throwing in blogspam and hoping for the best.
You are contradicting the papers and work that the people who make the models are saying. Alternatively, you are looking at the dataset curation process with rose tinted glasses. >There is a tremendous amount of work that goes on by the LLM companies in verifying, sanitizing, and structuring the training corpuses, using a wide array of techniques.
Common crawl is instrumental in building our models, 60% of GPT's training data was Common Crawl. (https://arxiv.org/pdf/2005.14165) pg 9.
CC in turn was never intended for LLM training, this misalignment in goals results in downstream issues like hate speech, NYT content, copyrighted content and more getting used to train models.
https://foundation.mozilla.org/en/research/library/generativ... (This article is to establish the issues with CC as a source of LLM training)
https://facctconference.org/static/papers24/facct24-148.pdf (this details those issues.)
Firms, such as the NYT are now stopping common crawl from archiving their pages. https://www.wired.com/story/the-fight-against-ai-comes-to-a-...
-----
TLDR: 'NYT' and other high quality content has largely been ingested by models. Reddit and other sources play a large part in training current models.
While I appreciate your being blunt, this also means not being sharp and incisive. Perhaps precision would be required here to clarify your point.
Finally -
>Factual accuracy is not binary, it is a matter of degrees. Obviously training on content that is more factually correct will result in more factually correct next tokens
What. Come on, I think you wouldnt agree with your own statement after reading it once more -Factual correctness is not a matter of degrees.
Furthermore, facts dont automatically create facts. Calcuation, processing, testing and verification create more facts. Just putting facts together creates content.
> What. Come on, I think you wouldnt agree with your own statement after reading it once more -Factual correctness is not a matter of degrees.
Of course it is. An LLM can be correct 30% of the time, 80% of the time, 95% of the time, 99% of the time. If that's not a matter of degrees, I don't know what is. If you're looking for 100% perfection, I think you'll find that not even humans can do that. ;)
Likely.
Re: > we have viable, working, scalable mechanisms to avoid the "pollution" you're worried about.
Do note - it’s the scalable mechanisms that I am looking at. I dont think the state of the art has shifted much since the last paper by OpenAI.
Can you link me to some new information or sources that lend credence to your claim.
> An LLM can be correct 30% of the time, 80% of the time, 95%…
That would be the error rate, which can be a matter of degrees.
However factual correctness largely cannot - the capital of Sweden today is Stockholm, with 0% variation in that answer.