Some thoughts on LLMs and software development

There are only 3 things that I have strong empirical evidence for with respect to LLMs

1. Routinely some task or domain of work that some expert claims that LLM’s will able to do, LLM’s start being able to reliably perform that task within 6 months to a year, if they haven’t already

2. Whenever AI gets better, people move the goalposts regarding what “intelligence” counts as

3. Still, LLM’s reveal that there is an element to intelligence that is not orthogonal to the ability to do well on tests or benchmarks