I think Yann Lecun was right about LLMs (but perhaps only by accident)

> Yann Lecun ... argued that because language models generate outputs token-by-token, and each token introduces a new probability of error, if we generate outputs that are too long, this per-token error will compound to inevitable failure.

That seems like a poor argument. Each word a human utters also has a chance of being wrong, yet somehow we have been successful overall.

Human errors don't compound the same way. Arguably.

Right, and neither do LLM errors. They can go off course, and then get back on course, just like humans do.

I mostly agree with you. Just pointing out what the argument was.

I think our agreement ends if we consider long-running tasks. A human can work a long time on a task like "find a way to make money". An AI, on the other hand, as it gets further and further from human input into autoregressive territory, is more and more likely to become "stuck" on a road to nowhere and needs human intervention to get unstuck.