Indeed, people really under(over?)estimate how small a loss of attention has to be to become catastrophic.
I once wanted to know the name of a track that was playing while driving on the highway. I looked right to the stereo display and read it, that probably took a tenth of a second, but it happened right at the moment when something came into my lane and I had to veer off not to hit it, I did not hit it but also almost drove the car out of the road.
When you're distracted, even if you're looking straight ahead, coming back to reality, assessing the situation, reacting, ... takes at least a couple seconds and that's a lot of time in these scenarios.