Granted, NYC is the biggest city in the US, so maybe that sort of reaction is more reasonable there than when people in Dallas or Boston do it.
Your grid system is far less of a challenge than the amount of hills, twists, narrow streets and low visibility back streets in California.
I genuinely think the most complicated challenge for Waymo in NYC will be…winter snow and ice.
And if the car reduces speed when appropriate and some assholes start tailgating it, it won't suffer the anxiety of holding up 10 cars that want to drive beyond the safe, reasonable speed for the snowy/icy conditions.
Right now, most self-driving software will refuse to activate in conditions of poor visibility. I've had that happen with Tesla's FSD, though in that case it was snowing so much that the road should have been closed. Also when the snow is deep enough that your front bumper becomes a plow, it will refuse to activate.
In ice none of these really stop overcorrection, or at least they don't in my 2020 truck on icy hill/mountain roads in Maine. And I've seen nice recent Volvos and BMWs with presumably the best safety tech in ditches up in the ski towns. The correct safe speed to drive on icy roads is not to drive at all of course, but people have to get places and people make mistakes. IME the assistive technology defaults don't do great on ice roads on some kind of up/down grade.
AFAIK drivers can still steer and brake themselves into a loss of control situation on ice regardless of safety features. So I guess I'm hoping once you take those two variables out of their hands, the FSD vehicles will be safer. Who knows though.
I went many years without a loss of control and the one time it did happen (logging roads with ice pack) was enough for me to buy Nokian studded winter tires to minimize the effect of ice as much as possible.