These are valuable call-outs. They are evidence that the definition of intelligence is at best, a set of ever-moving goalposts and at worst, vague enough to be useless.
However. It seems the ML community has centered around the idea that LLMs are zero or few shot learners, meaning that despite only being trained on the task of predicting the next token, they do well on other specific, highly niche, tasks. This is surprising and important.
Predictions of immediate profession replacement seem silly. As noted in TFA, there's more to a job than the license test.