As semi-autonomous and autonomous cars become the norm, I would adore to see obtaining a drivers license ratchet up in difficulty in order to remove dangerous human drivers from the road.
Driving in American cities is the opposite of freedom. The necessity of regulating apes piloting heavy machinery in close proximity to each other and society is a major source of our modern police state.
It is a privilege licensed by the State and regularly revoked through due process or expiry.
While your concern about mobility and privacy are valid, I would contend that public safety is what it’s to be weighed against. Some people really are better riders than drivers.
Of course people who really don't like that freedom will play up the privilige vs right angle as they see fit to advance their goals.