The AI Investment Boom

The question remains is when the bubble will crash. We could be in the 1995 equivalent of the dotcom boom and not 1999. If so, we have 4 more years of high growth and even after the crash, the market will still be much bigger in 2029 than in 2024. Cisco was still 4x bigger in 2001 than in 1995.

One thing that is slightly different from past bubbles is that the more compute you have, the smarter and more capable AI.

One gauge I use to determine if we are still at the beginning of the boom is this: Does Slack sell an LLM chatbot solution that is able to give me reliable answers to business/technical decisions made over the last 2 years in chat? We don't have this yet - most likely because it's probably still too expensive to do this much inference with such high context window. We still need a lot more compute and better models.

Because of the above, I'm in the camp that believe we are actually closer to the beginning of the bubble than at the end.

Another thing I would watch closely to see when the bubble might pop is if LLM scaling laws are quickly breaking down and that more compute no longer yields more intelligence in an economical way. If so, I think the bubble would pop. All eyes are on GPT5-class models for signs.

Glean.com does it for the enterprise I work at: It consumes all of our knowledge sources including Slack, Google docs, wiki, source code and provides answers to complex specific questions in a way that’s downright magical.

I was converted into a believer when I described an issue to it, pointers to a source file in online git repo and it pointed me to another repository that my team did not own that controlled DNS configs that we were not aware about. These configs were the reason our code did not behave as we expected.

I suppose one reason Slack doesn't have a solution yet is because they're having a hard time getting it to work for large companies.

AI code assistants have been a net neutral for me (they get enough idioms in C++ slightly incorrect that I have to spend a lot of time just reading the generated code thoroughly), but being able to say "tell me what the timeline for feature X is" and have it comb through a bunch of internal docs / tickets / git commit messages, etc, and give me a coherent answer with links is amazing.

They control the OS so they can combine and feed all your digital information to an LLM in a seamless way. However, in the very long term, I think their advantage will go away because at some point, LLMs could get so good that you don't need an OS like iOS anymore. An LLM could simply become standalone - and function without a traditional OS.

Therefore, I think the advantage for iOS, Android, Windows will increase in the next few years, but less powerful after that.

I think you may be confused about the length of the context window. These tools don't pull all of your Slack history into the context window. They use a RAG approach to index all of your content into a vector DB, then when you make a query only the relevant document snippets are pulled into the context window. It's similar for example to how Cursor implements repository-wide AI queries.

The point I'm trying to make is that increase context window will require more compute. Hence, we could still just be in the beginning of the compute/AI boom.

Well, this is taken on faith by OpenAI/etc, but obviously the curve has to flatten at some point, and appears to already be doing so. OpenAI are now experimenting with scaling inference-time compute (GPT-O1), but have said that it takes exponential increases in compute to produce linear gains in performance, so it remains to be seen if customers find this a worthwhile value.

If you run chain of thoughts on an 8B model, it becomes a lot smarter too.

GPT-o1 isn't GPT5 though. I think OpenAI will have a chain of thoughts model for GPT5-class models as well. They're separate from normal models.

GPT-5 and Claude-4 will be interesting, assuming these are both pure transformer models (not COT), as they will be a measure how much benefit remains to be had from training set scaling. I'd expect gains will be more against narrow benchmarks, than in the overall feel of intelligence (LLM arena score?) one gets from the model.

this is part of why I stay in python when doing ai-assisted programming; there's so much training information out there for python and I _generally_ don't care about if its slightly off-idiom, its still probably fine.

Has your barber/hairdresser recommended you buy NVDA?

Not only is it an extremely poor source of information, it has ruined the company's Slack culture as people are no longer willing to (for lack of a better term) shitpost knowing that their goofy sarcasm will now be presented to Glean users as fact.

They recently hired Jony Ive on a project - it could be this.

I think it'll be a long term goal - maybe in 3-4 years, a device similar to the Rabbit R1 would be viable. It's far too early right now.

I was thinking re the data in the tweet, that there were a lot of mentions of "soft landing" before the dot com crash, before the 2006 property crash and now, it is quite likely there was an easy money policy preceding them and then government policy mostly focuses on consumer price inflation and unemployment, so they relax when those are both low and then hit the brakes when inflation goes up and then it moderates and things look good similar to now. But that ignores that easy money can also inflate asset prices, eg dot com stocks, houses in 06, or money losing AI companies like now. And then at some point that ends and the speculative asset prices go down rather than up, leaving people thinking oh dear we've borrowed to put loads of money into that dotcom/house/ai thing and now it's not worth much and we still have the debts...

At least that's my guess.

Note that the presence of such a feature isn't the same as whether it's secure enough for normal use.

In particular, anything anyone said in the last 2 years in chat could poison the LLM into exfiltrating your data or giving false results chosen by the attacker, because of the fundamental problems of LLMs.

https://promptarmor.substack.com/p/data-exfiltration-from-sl...

Training costs are killing them, and it's obviously not sustainable to keep spending more on research and training than the revenue generated. Training costs are expected to keep growing fast, while revenue per token in/out is plummeting - they need massive inference volume to turn this into a profitable business, and need to pray that this doesn't turn into a commodity business where they are not the low cost producer.

https://x.com/ayooshveda/status/1847352974831489321

https://x.com/Gloraaa_/status/1847872986260341224

What it's really good at is taking my description of something and pointing me in the right direction to do my own research.

(two things that helped me with getting decent code were to describe the problem and desired solution, followed by a "Does that make sense?". This seems to get it to restate the problem itself and produce better solutions. The other thing was to copy the output into a fresh session, ask for a description of what the code does and what improvements could be made)

We've seen this before, with for example Amazon where they made a deliberate effort to avoid profitability by spending as much as possible on infrastructure until the revenue became some much they couldn't spend it.

Being in a position where you are highly cash-flow positive and it's strategic investment that is the cost seems like a good position.

When you need something fast for whatever reason sure, but later when you want to tweak or add something, you'll have to finally sit down and learn basically the whole thing or at least a major part of it to do so anyway. Imo it's better to do that from the start but sometimes that's not always ideal.

I'm guessing this is because the only training material was blogs from uber-nerdy CS researchers on a language where "mistakes" are basically impossible to write, and not a bunch of people flailing on forums asking about hello world-ish stuff and segfaulting examples.

At the end of the day, it depends where you are in the hierarchy. Having it write code for me on a hobby project in react that's bad but works is one thing. I'm having a lot of fun with that. Having it write bad code for me professionally is another thing though. Either way, there's no going back to before ChatGPT, just like there's no going back to before Stack Overflow or Google. Or the Internet.

    > 2006 property crash

There are a few other companies in this space (and it's not something that complex to DIY either); the issue is data quality. If your Google Docs and wikis contain obsolete information (because nobody updated them), it's just going to be shit in, shit out. Curating the input data is the challenging part.

And the point stands: you would not write a new OS, even to incorporate it into your LLM. You'd clone a BSD (or similar) and start there.

Underneath, it can be Linux, BSD, Unix, or nothing at all, whatever. It doesn't matter. That's not important.

OS was just a convenient phrase to describe this idea.

Amazon were reinvesting and building scale, breadth and efficiency that has become an effective moat. How do you compete with Amazon Prime free delivery without your own delivery fleet, and how do you build that without the scale of operations?

OpenAI don't appear to have any moat, don't own their own datacenters, and the datacenters they are using are running on expensive NVIDIA chips. Compare to Google with their own datacenters and TPUs, Amazon with own datacenters and chips (Graviton), Meta with own datacenters (providing value to their core business) and chips - and giving away the product for free despite spending billions on it ... If this turns into the commodity business that it appears it may (all frontier models converging in performance), then OpenAI would seem to be in trouble.

Of course OpenAI could stop training at any time, but to extent that there is further performance to be had from further scaling and training, then they will be left behind by the likes of Meta who have a thriving core business to fund continued investment and are not dependent on revenue directly from AI.

I don't think AI is ultimately even an application, it's a feature we will use in applications.

So, your problem there is 'reliable'. LLMs, fairly fundamentally, cannot do 'reliable'. If you're looking for reliable, you likely are looking at a different tech entirely.

There's a confirmation bias there, though. Economists, particularly pop economists, have predicted all 20 of the last two recessions; if you just say "the world is going to end" every year, then occasionally it kinda will, and certain people will think you're a visionary.

But I see your point. And yes, I think it has trickled down to the mainstream.

(By that metric, I guess Bitcoin crashed a few years ago.)

You're just burning money bothering to rewrite the rest of the stack when off the shelf will save you years.

This is akin to an analytic statement, eg, "all bachelors are married". The truth is completely within the definition of the statement. Compare this to a synthetic statement such as "it is raining outside". In this case the truth is contingent on facts outside of the statement itself.

When LLMs are faced with an analytic statement they are more reliable. When they are faced with a synthetic statement they are prone to confabulate and are unreliable.

Hrm. I've found LLM summaries to be of... dubious reliability. When someone posts an article on this here orange website, these days someone will sometimes 'helpfully' post a summary generated by a magic robot. Have a look at these, sometime. They _often_ leave out key details, and sometimes outright make shit up.

Interesting article on someones' experiences with this recently: https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actu...

No need to get to the technicals such as whether it's UNIX or Linux talking to the hardware.

Just from a pure user experience standpoint, OpenAI would become iOS.

But I think there will be a long period when people want both the traditional UI with buttons and sliders, and the AI that can do what you ask. (Analogy with phone keyboards where you can either speech-to-text, or slide to type, or type individual letters, or mix all three.)

Original article: https://osa1.net/posts/2024-10-09-oop-good.html

LLM (ChatGPT o1-preview) results: https://chatgpt.com/share/67166301-00d8-8013-9cf5-e8a980aca7...

LGTM!

I'd like to know which model is used in the article you've referenced as well as the prompt. I also suspect that 50 pages is pushing up to the limits of the context window and has an impact on the results.

---

As for the article itself... Use a language like F# or OCaml and you get a functional-first language that also supports OOP!

We will also get some nice things, like more intelligent IDE at affordable cost, think CursorAi costs $20/month(240/year), while whole JetBrain's package costs only 25/month(290/year).

However, I am a bit worried about all these data center and AI and energy use/scaling. While consumers are being pushed to more and more efficient energy usage and energy prices are definitely high(to what I would expect with massive renewable energy production), large corps and such will continue scaling energy usage higher and higher.

Also, the AI fad will eventually spook out a lot of free knowledge sharing on the open and everything will get behind paywall, so random poor kid in some poor country will no longer have access to some nice tutorial or documentations online to learn cool stuff because in some countries, what we call a price of "morning coffee" is actually could be a day's earning of an adult but not for non-privileged people. Without ability to pay for AI services, no more access to knowledge. Search engines will eventually drown in slop, I mean even google now frequently gives me "no resoult found" page and I need to use ddg/brave/bing to fish out some results still.

well, that's not a false statement. As much as I might dislike it, the raise of the web and web applications have made the OS themselves irrelevant for a significant number for tasks.

1996 - Cisco was 23.4B or 0.3% of US GDP

2000 - Cisco peaked at 536B or 5.2% of US GDP

2020 - Nvidia was 144B or 0.7% of US GDP

2024 - Nvidia is 3.4T or 11.9% of US GDP

Numbers very rough and from different sources, but I'd be surprised if Nvidia doesn't pop within 1-2 years at most.

There's a reason Microsoft just outright purchased the entire output of 3-mile island (a full sized nuclear power plant).

At some point, people will stop buying GPUs because we've simply run out of power.

My only hope is that we don't suffer some kind of ill effect (ex: double the cost of consumer electricity or something, or local municipalities going bankrupt due to rising energy costs). The AI boom has so much money in it we need to account for tail wags dog effects.

More importantly, Cisco's margins and PE were much higher than Nvidia's today.

You should use actual financial measures and not GDP national accounts which have zero bearing on business valuation.

There's a relation between P/E and future actual revenues of a company.

Imagine that a similar comparison would imply that it's projected that in a few years NVidia's revenues shall represent 10% of the US's GDP: do we really believe that's going to happen?

The Mag 7 + Broadcom have a market cap that is now 60% of the US's GDP. I know you think it's silly but... Doesn't that say something about the expect revenues of these companies in a few years?

Do we really think the Mag 7 + Broadcom (just an example) are really to represent the % of the actual US's GDP that that implies?

Just to be clear: I'm not saying it implies the percentage of the US GDP of these 8 companies alone is going to be 60% but there is a relation between the P/E of a company and its expected revenues. And revenues of companies do participate in the GDP computation.

I don't think it's as silly as several here think.

I also don't think GP should be downvoted: if we disagree, we can discuss it.