Tools: Code Is All You Need

Directionally I think this is right. Most LLM usage at scale tends to be filling the gaps between two hardened interfaces. The reliability comes not from the LLM inference and generation but the interfaces themselves only allowing certain configuration to work with them.

LLM output is often coerced back into something more deterministic such as types, or DB primary keys. The value of the LLM is determined by how well your existing code and tools model the data, logic, and actions of your domain.

In some ways I view LLMs today a bit like 3D printers, both in terms of hype and in terms of utility. They excel at quickly connecting parts similar to rapid prototyping with 3d printing parts. For reliability and scale you want either the LLM or an engineer to replace the printed/inferred connector with something durable and deterministic (metal/code) that is cheap and fast to run at scale.

Additionally, there was a minute during the 3D printer Gardner hype cycle where there were notions that we would all just print substantial amounts of consumer goods when the reality is the high utility use case are much more narrow. There is a corollary here to LLM usage. While LLMs are extremely useful we cannot rely on LLMs to generate or infer our entire operational reality or even engage meaningfully with it without some sort of pre-existing digital modeling as an anchor.

Hype cycle for drones and VR was similar -- at the peak, you have people claiming drones will take over package delivery and everyone will spend their day in VR. Reality is that the applicability is more narrow.

People claimed that we would spend most of our day on the internet in the mid-90s, and then the dotcom bubble burst. And then people claimed that by 2015 robo-taxis would be around all the major cities of the planet.

You can be right but too early. There was a hype wave for drones and VR (more than one for the latter one), but I wouldn't be so sure that it's peak of their real world usage yet.

Which is why I think there are two distinct kinds of perspective, and for one of them, AI hype is just about at the right levels - and being too early is not a problem, unless it delays things indefinitely.

I wrote about it recently here: https://news.ycombinator.com/item?id=44208831. Quoting myself (sorry):

> For me, one of the Beneficiaries, the hype seems totally warranted. The capability is there, the possibilities are enormous, pace of advancement is staggering, and achieving them is realistic. If it takes a few years longer than the Investor group thinks - that's fine with us; it's only a problem for them.

You're getting from it the one thing it can deliver. Dazzle and publicity.