If consumers paid out of pocket for their phones then they would be more picky about upgrading and plan prices. It would also make upselling shitty plan features harder so the carriers would loose a lot of money.
I guess the carriers still make money because once habituated, especially if they've never done the port-number-to-new-carrier thing, people stay in the high priced plan longer than necessary. Like the three years until they've truly paid the above-market price for the phone, and are now eligible for another "free" phone which they may not even take advantage of.
For what it's worth, carrier locking phones has been illegal here for some years (and any phone from the locked era had to be unlocked for free for the asking after the law was changed) and it hasn't changed anything in terms of these rent-to-buy type carrier plans. So I don't know what the fuss is about. A contract is a contract.
A similar psychological thing is going on in both cases I feel - some minority will manage to emerge winners due to a mixture of persistence, intelligence, luck, greed, etc. And the majority will get squeezed, or else how would such large percentages of marketing budgets continue to be pumped into club card and subscription schemes.