That's how I've always characterized them. But if you think about it, it's not really true.
The LLM is "lossily" containing things an encyclopedia would never contain. An encyclopedia, no matter how large, would never contain the entire text of every textbook it deems worth of inclusion. It would always contain a summary and/or discussion of the contents. The LLM does, though it "compresses" over it, so that it, too, only has the gist at whatever granularity it's big enough to contain.
So in that sense, an encyclopedia is also a lossy encyclopedia.