If we're talking about someone who has received a binary copy of software, then isn't this obvious?
The MIT license permits the distributor to close the source of what they've redistributed, in original or modified form. Potentially depriving the end user of the freedom to view/modify/distribute the source.
Permissive licenses prioritize rights of the software redistributors at the expense of the end users.
Edit: and to your point of a distributor withholding the source: yeah, so? If there ever came a point where the current maintainer closed its source (unlikely), somebody with a copy of it can step in with a fork. Or the project can die a deserved death for closing its source. At this point the benefits of open source are pretty much obvious to anyone with a brain, and closing the source of an open-source project is practically suicide.