Most likely its training data included countless Space Invaders in various programming languages.
replies(6):
We went from chatgpt's "oh, look, it looks like python code but everything is wrong" to "here's a full stack boilerplate app that does what you asked and works in 0-shot" inside 2 years. That's the kicker. And the sauce isn't just in the training set, models now do post-training and RL and a bunch of other stuff to get to where we are. Not to mention the insane abilities with extended context (first models were 2/4k max), agentic stuff, and so on.
These kinds of comments are really missing the point.
Even then, when you start to build up complexity within a codebase - the results have often been worse than "I'll start generating it all from scratch again, and include this as an addition to the initial longtail specification prompt as well", and even then... it's been a crapshoot.
I _want_ to like it. The times where it initially "just worked" felt magical and inspired me with the possibilities. That's what prompted me to get more engaged and use it more. The reality of doing so is just frustrating and wishing things _actually worked_ anywhere close to expectations.
I am definitely at a point where I am more productive with it, but it took a bunch of effort.
If I didn't have an LLM to figure that out for me I wouldn't have done it at all.
That's what I've done with my ffmpeg LLM queries, anyway - can't speak for simonw!
Meanwhile I've spent the past two years constantly building and implementing things I never would have done because of the reduction in friction LLM assistance gives me.
I wrote about this first two years ago - AI-enhanced development makes me more ambitious with my projects - https://simonwillison.net/2023/Mar/27/ai-enhanced-developmen... - when I realized I was hacking on things with tech like AppleScript and jq that I'd previously avoided.
It's hard to measure the productivity boost you get from "wouldn't have built that thing" to "actually built that thing".
• https://stackoverflow.com/questions/10957412/fastest-way-to-...
• https://superuser.com/questions/984850/linux-how-to-extract-...
• https://www.aleksandrhovhannisyan.com/notes/video-cli-cheat-...
• https://www.baeldung.com/linux/ffmpeg-extract-video-frames
• https://ottverse.com/extract-frames-using-ffmpeg-a-comprehen...
Search engines have been able to translate "vague natural language queries" into search results for a decade, now. This pre-existing infrastructure accounts for the vast majority of ChatGPT's apparent ability to find answers.
Not comparable and I fail to see why going through Google's ads/results would be better?
I did not suggest using Google Search (the company's on record as deliberately making Google Search worse), but there are other search engines. My preferred search engines don't do the fancy "interpret natural language queries" pre-processing, because I'm quite good at doing that in my head and often want to research niche stuff, but there are many still-decent search engines that do, and don't have ads in the results.
Heck, you can even pay for a good search engine! And you can have it redirect you to the relevant section of the top search result automatically: Google used to call this "I'm feeling lucky!" (although it was before URI text fragments, so it would just send you to the top of the page). All the properties you're after, much more cheaply, and you keep the information about provenance, and your answer is more-reliably accurate.
Agreed on all fronts. jq and AppleScript are a total syntax mystery to me, but now I use them all the times since claude code has figured them out.
It's so powerful knowing the shape of a solution on not having to care about the details.