What is going on right now?

I won't say too much, but I recently had an experience where it was clear that when talking with a colleague, I was getting back chat GPT output. I felt sick, like this just isn't how it should be. I'd rather have been ignored.

It didn't help that the LLM was confidently incorrect.

The smallest things can throw off an LLM, such as a difference in naming between configuration and implementation.

In the human world, you can with legacy stuff get in a situation where "everyone knows" that the foo setting is actually the setting for Frob, but with an LLM it'll happily try to configure Frob or worse, try to implement Foo from scratch.

I'd always rather deal with bad human code than bad LLM code, because you can get into the mind of the person who wrote the bad human code. You can try to understand their misunderstanding. You can reason their faulty reasoning.

With bad LLM code, you're dealing with a soul-crushing machine that cannot (yet) and will not (yet) learn from its mistakes, because it does not believe it makes mistakes ( no matter how apologetic it gets ).

>"It didn't help that the LLM was confidently incorrect."

Has anyone else ever dealt with a somewhat charismatic know-it-all who knows just enough to give authoritative answers? LLM output often reminds me of such people.

That’s a great question — and one that highlights a subtle misconception about how LLMs actually work.

At first glance, it’s easy to compare them to a charismatic “know-it-all” who sounds confident while being only half-right. After all, both can produce fluent, authoritative-sounding answers that sometimes miss the mark. But here’s where the comparison falls short — and where LLMs really shine:

(...ok ok, I can't go on.)

Perfect! You really got to the core of the matter! The only thing I noticed is that your use of the em-dash needs to not be bracketed with spaces on either end. LLMs—as recommended by most common style guides—stick to the integrated style that treats the em-dash as part of the surrounding words.

It bums me out that LLMs are ruining em dashes. I like em dashes and have used them for decades, but now I worry that when I do people will assume my writing is LLM output.

What's next—the interrobang‽