LLM Inevitabilism

1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them.

2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expenditure of training and hosting them, the majority of consumer usage is at the free tier, the industry is seeing the first signs of pulling back investments, and model capabilities are plateauing at a level where most people agree that the output is trite and unpleasant to consume.

There are many technologies that have seemed inevitable and seen retreats under the lack of commensurate business return (the supersonic jetliner), and several that seemed poised to displace both old tech and labor but have settled into specific use cases (the microwave oven). Given the lack of a sufficiently profitable business model, it feels as likely as not that LLMs settle somewhere a little less remarkable, and hopefully less annoying, than today's almost universally disliked attempts to cram it everywhere.

That is a such a wild claim. People like the output of LLMs so much that ChatGPT is the fastest growing app ever. It and other AI apps like Perplexity are now beginning to challenge Google’s search dominance.

Sure, probably not a lot of people would go out and buy a novel or collection of poetry written by ChatGPT. But that doesn’t mean the output is unpleasant to consume. It pretty undeniably produces clear and readable summaries and explanations.

The people using ChatGPT like its output enough when they're the ones reading it.

The people reading ChatGPT output that other people asked for generally don't like it. Especially if it's not disclosed up front.

While technically correct it came to the wrong conclusions about the best path forward and inevitably hamstrung the project.

I only discovered this later when attempting to fix the mess and having my own chat with an LLM and getting mysteriously similar responses.

The problem was that the assumptions made when asking the LLM were incorrect.

LLMs do not think independently and do not have the ability to challenge your assumptions or think laterally. (yet, possibly ever, one that does may be a different thing).

Unfortunately, this still makes them as good as or better than a very large portion of the population.

I get pissed off not because of the new technology or the use of the LLM, but the lack of understanding of the technology and the laziness with which many choose to deliver the results of these services.

I am more often mad at the person for not doing their job than I am at the use of a model, the model merely makes it easier to hide the lack of competence.

Yep.

More seriously, you described a great example of one of the challenges we haven't addressed. LLM output masquerades as thoughtful work products and wastes people's time (or worse tanks a project, hurts people, etc).

Now my job reviewing work is even harder because bad work has fewer warning signs to pick up on. Ugh.

I hope that your workplace developed a policy around LLM use that addressed the incident described. Unfortunately I think most places probably just ignore stuff like this in the faux scramble to "not be left behind".