ChatGPT Atlas

If the DeepSeek approach to training hyperscaler models "cheaply" after all the hype wears down works, we just need to follow in their footsteps and build open source alternatives to everything.

Frontier models take a lot of money and experimentation. But then people figure out how to train them and knowledge of those models and approaches leaks. Furthermore, we can make informed guesses. But best of all, we can exfiltrate the model's output and distill the model.

If we work together as an industry to open source everything, we can overcome this.

OpenAI has to 100x in five years or they're going to be in trouble.

Models are making it easy to replace SaaS, but also easy to replace other AI companies.

There may be no moat for any of this. The lead is only because they're a few generations ahead, running as fast as they can on the treadmill.

I don't think this hurts China at all.

Indeed the "no-moat" (weight exfiltration of any openly accessible model) is something that makes me optimistic. That, and the fact that most tasks have "capacity thresholds" which on-device models will increasingly be able to saturate. One example is SQL query generation from text (example: duckdb-text2sql https://motherduck.com/blog/duckdb-text2sql-llm/).

Curious as to what companies you think DO have a moat? I would say their moat is the same as Facebook's. Not permanent (see TikTok) but also super strong.

TikTok, Facebook, Instagram, etc. have network effect stickiness. YouTube has a vast library that grows in value over time and has been able to lose money for long enough to make the experience unassailable, etc. (Look at Vimeo.)

Maybe LLMs get that by knowing our entire past? But I find that creepier than useful. Right now ChatGPT is at the top of the world, but I don't see it becoming the new unrivaled Google Search. There's just too many people building it, and once OpenAI starts monetizing and "enshittifying" it, other offerings will become more compelling.

AI models put a large swath of mostly tech companies at risk. Including the old business models of titan products like Google Search.

Image/video/world models specifically do this more to legacy media incumbents than LLMs do to complex business processes. We see orders of magnitude savings with marketing, design, film, and possibly in the future game design. LLMs, on the other hand, aren't great at getting your taxes or complex business logic right.

> There may be no moat for any of this.

It's what I was thinking looking at the launch video. Is there a moat? No, there is none. The LLM itself is fungible, what matters is the agentic and memory layer on top. That can be reconstructed easily. All you need to do is export your data from old providers to bootstrap your system in another place. I actually did that, exported from reddit, hn, youtube, chatgpt, claude and gemini - about 15 years worth of content, now sitting on my laptop in a RAG system.

At minimum all you need is a config file, like CLAUDE.md containing the absolute minimum information you need to set your preferences and values. That would be even more portable, you can simply paste text in any LLM to configure it before use. Exporting all your data is the maximalist take on the problem of managing your online identity.

Very interesting and I would also love to export my own data! Care to share/elaborate a little further how you went into doing this, especially with platforms that do not offer the option to export your own data.