LLM Inevitabilism

Most of them are a bunch of prompts and don't even have actual developers. For the good reason that there is no training system yet and the wording of how you call the people that build these system isn't even there or clearly defined. Local companies haven't even setup a proper internal LLM or at least a contract with a provider. I am in France so probably lagging behind USA a bit especially NY/SF but the word "LLM developer" is just arriving now and mostly under the pressure of isolated developers and companies like me. This feel really really early stage.

The smartest and most well funded people on the planet have been trying and failing to get value out of this technology for years and the best we've come up with so far is some statistically unreliable coding assistants. Hardly the revolution its proponents keep eagerly insisting we're seeing.

Between the ridiculously optimistic and the cynically nihilistic I personally believe there is some value that extremely talented people at huge companies can't really provide because they're not in the right environment (too big a scale) but neither can grifters packaging a prompt in a vibecoded app.

In the last few months the building blocks for something useful for small companies (think less than 100 employees) have appeared, now it's time for developers or catch-all IT at those companies and freelancers serving small local companies to "up-skill".

Why do I believe this? Well for a start OCR became much more accessible this year cutting down on manual data entry compared to tesseract of yesteryear.

is there a non-prompt way to interact with LLMs?

In an agentic setup the value is half the prompts half how you plug them together. I am opposing for instance a big prompt that is supposed to write a dissertation vs a smart web scraper that builds a knowledge graph out of sources and outputs a specialized search engine for your task. The former is a free funny intern, the latter is growth percentage visible in the economy.

They try to get value at their scale, which is tough. Your local SME definitely sees value in an embedding-based semantic search engine over their 20 years of weird unstructured data.

my company has already fired a bunch of people in favor of LLMs so they are realizing all kinds of value

I don’t know your company but this thinking doesn’t necessarily follow logically. In a large company the value of developers is not distributed evenly across people and time, and also has a strong dependency on market realities in front of them.

While it’s true that lots of companies are getting some value out of LLMs, a much larger number are using them as an excuse for layoffs they would have wanted to do anyway—LLMs are just a golden opportunity to tie in an unmitigated success narrative.

Only as much as replacing all your devs with a frog is "realizing value"

I imagine they HOPE they'll realize value. A lot of people are acting on what might be, rather than what is, which makes sense given that the AI "thought leaders" (CEOs with billions invested that need to start turning a profit) are all promising great things soon™.

The best they've come up with is the LLM chatbot, which both OpenAI and Anthropic have as their flagship product because many people find it extremely valuable. Many people I know routinely use ChatGPT to help them write things, even those who were already good at writing, and if you don't think that's true at your workplace I strongly suspect it's because people aren't telling you about it.

So has mine, and quite predictably our product has gone into the shitter and breaks constantly, requiring reverts almost daily. They've armed a couple of Juniors with Cursor and given them the workload of all those people they fired / have quit since the firings, some of which have been at the company for years and held a lot of institutional knowledge that is now biting them in the ass.

Now sure, "Just don't fire the useful people and get rid of the juniors and supercharge the good devs with AI tooling" or whatever, except the whole reason the C-level is obsessed with this AI shit is because they're sold on the idea of replacing their most expensive asset, devs, because they've been told by people who sell AI as a job that it can replace those pesky expensive devs and be replaced by any random person in the company prompting up a storm and vibecoding it all.

Churn rates are up, we're burning unfathomable amounts of money on the shitty AI tooling and the project has somehow regressed after we've finally managed to get a good foothold on it and start making real progress for once. Oh and the real funny part is they're starting to backpedal a bit and have tried to get some people back in.

I expect to hear a LOT more of this type of thing happening in the near future. As the idiots in charge start slowly realizing all the marketing sold to them on LinkedIn or wherever the fuck it is they get these moronic ideas from are literal, actual literal lies.

Yeah callousness does seem to be the leaking area of improvement.

Great, we've got mediocre writing from unprofitable companies that are subsidizing the cost of this technology.

> a much larger number are using them as an excuse for layoffs they would have wanted to do anyway

It's a simple formula. Layoffs because of market conditions or company health = stock price go down. Layoffs because "AI took the jobs" = stock price go up.

What specifically do you find to be mediocre? I feel like LLMs write better than most people I know, myself included.

There could be a mismatch on what the state of the art really is these days. In my experience, since the release of GPT-4 and especially 4o, ChatGPT has been able to do the vast majority of concrete things people tell me it can't do.