The problem with rail isn't just labor, it's land acquisition. For the old freight lines that was done centuries ago, now that virtually all land has been claimed by someone it's much more expensive by default. On top of that, California got Musk disrupting everything with Hyperloop.
You need to use eminent domain on straight lines as much as possible for HSR, both to keep costs low and to allow for actually high speeds, but that's risky for legal challenges and even then, horribly expensive at US scales.
Yes, China has larger scales and still gets it done, but they a) just throw money at the problem and b) just do what the CCP wants.
> Have crazy people screaming on the street corner in the city and people will retreat to the suburbs and order from Amazon instead of going shopping, order a private taxi for their burrito instead of going to a restaurant.
That's not made easier by the fact that many cities just hand one way bus tickets to local homeless and nutjobs that bus them off to somewhere else [1], often to Democrat-run cities. In addition to that, there are almost no asylums left to take care of the nutjobs because a lot of them had been forced to shut down for sometimes atrocious violations of human rights many decades ago. Some areas now (ab)use jails and prisons to punish homeless people for being homeless, a practice that has also come under fire for creating the same abusive conditions, on top of scandals like "Kids for cash" [2].
The obvious solution to a lot of the problems with nutjobs, homeless and drug addicts would be a sensible drug policy combined with a "housing first" policy. Both of that has been tried in the US and in other countries worldwide to a sometimes massively positive effect, the problem is it has to be done federally - otherwise you end up like Frankfurt here in Germany, where Frankfurt pays the bill for drug addiction treatments and somewhat safe consumption facilities, but ended up having to pay that for people from almost across the whole of Europe.
> If the public sector were efficient, responsive and pleasant people would be voting for more of it.
It could be at least pleasant and responsive, the problem is you need (a lot) of money to pay for it, and no one likes paying taxes. It's a chicken and egg problem across Western countries - ever since up to the 80s, when neoliberal politics, trickle-down and lean-state ideology took over, public service has been cut and cut and cut. People don't believe any more that paying higher taxes would yield a net benefit because they lost all trust in politicians, and I don't see any way of fixing that - not without a stint of a good-willing dictator at least, and I don't see that on the horizon at all.
[1] https://awards.journalists.org/entries/bussed-out-how-americ...
[2] https://en.wikipedia.org/wiki/Kids_for_cash_scandal