I think Yann Lecun was right about LLMs (but perhaps only by accident)

I think Yann is right if all you do is output a token, which is dependent on the previous token. If it's a simple Markov chain, sure, errors will eventually compound. But with Attention mechanism, the output token depends not only on the previous one, but all 1 million previous ones (assuming a 1M context window). This gives the model plenty of opportunity to fix its errors (and hence the "aha moment"s).

No this isn't right. The probabilistic formulation for autoregressive language models looks like this

     p(x_n | x_1 ... x_{n-1})

which means that each token depends on all the previous tokens. Attention is one way to parameterize this. Yann's not talking about Markov chains, he's talking about all autoregressive models.

Current token depends on _all_ previous tokens, but only indirectly for the ones before the previous one, no?