I don't completely disagree but I believe it's a bit more fuzzy than that. From what I understand, the models learn a very compressed version of what they receive as input and produce as output. While not sufficient to generalize, you could say they memorize some very high-dimensional function to cause the expected text to be produced, and they can turn on and combine multiple of these functions (multiply by non-zero, sum, etc). So on some level an LLM can kind of perform logic on the input, even if it has a slightly novel pattern. But at the same time, no model is shown to completely generalize the way a human would.
And let's also be fair, it would take a lot of effort for a human to generalize to a previously unseen pattern as well, so I always wonder just how useful it is to try to make such binary statements as "models don't reason" or they're "stochastic parrots". But maybe it's to counterweigh the statements that they are sentient, AGI is here, etc?