Car infrastructure takes up a huge amount of space and is incredibly hostile to any kind of mixed use. Having near-zero cars means there is suddenly space available for an order of magnitude more pedestrians. It's why reducing car traffic almost always results in a significant increase in revenue for local shops and restaurants - which means more taxes are being paid.
Converting all of NYC into a huge pedestrian-only zone obviously isn't going to work, but having a few pedestrianized superblocks could greatly improve the quality-of-life.
I have spent over a decade without owning a car in multiple cities. It's definitely possible but I've been fortunate enough to have friends and family with personal vehicles I can use.
Alternatively, you rent a car from a car sharing service for the few times a year you want to do this.
Philadelphia is also an odd choice to bring up since there's a train from the center of Manhattan to Philly, and it's even a common commute for people to go back and forth.
Small errands also completely defeats the point of Costco. Running to a corner store to buy 4 rolls of toilet paper ends up being way more expensive than going to Costco and picking up 30+ at a time.