That's also a concern. Many day-to-day tasks for employees are repetitive to the point even a less-than-AGI system could potentially disrupt those jobs (and there are people actively working right now to make this happen).
The best case scenario would be the employees taking advantage of their increased productivity to make themselves more valuable to their employer (and if they are lucky, gain increased compensation).
However, it's also possible employers decide they don't need many of their lower level workforce anymore because the remaining ones are more productive. It wouldn't take much of this to drive unemployment levels way up. Perhaps not to the level of the Great Depression, at least not for a while, but it is certainly a potential outcome of the ongoing, long-term process in our economic system of increasingly automating repetitive, low skill tasks.
IOW, it doesn't take AGI to throw a lot of people out of work. It's happened many times with other technologies in the past, and when it happens, things can get pretty bad for a large number of people even if the majority are still doing okay (or even great, for those at the top).