AI coding and the peanut butter and jelly problem

This is actually no different than for humans once you get past the familiar. It's like the famous project management tree story: https://pmac-agpc.ca/project-management-tree-swing-story

If anything, LLMs have surprised at much better they are than humans in understanding instructions for text based activities. But they are MUCH worse than humans when it comes to creating images/videos.

> If anything, LLMs have surprised at much better they are than humans in understanding instructions for text based activities.

That's demonstrateably false, as proven by both OpenAI's own research [1] and endless independent studies by now.

What is fascinating is how some people cling on false ideas about what LLM is and isnt.

Its a recurring fallacy that's bound to get it's own name any time soon.

1: https://news.ycombinator.com/item?id=43155825

You’re comparing an LLM to expert programmers. Compare an LLM on the same task versus the average college student. And try it for a math problem. A poetry problem. Ask it a more complex question about history or to do an analysis of an essay you wrote.

Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose?

> Put it this way — I’m going to give you a text based question to solve and you have a choice to get another human to solve it (randomly selected from adults in the US) or ChatGPT, and both will be given 30 minutes to read and solve the problem — which would you choose?

You wouldn't randomly selected an arbitrary adult from the USA to do a brain surgery on you, so this argument is rabulistic.

Brain surgery requires a license.

But I do expect an arbitrary adult to be able to follow instructions.

Ok. How about you give me a text based task where you would pick the random adult over the LLM?