In practice of course, my phone is 100% permanently in "do not disturb" mode and does not ring at all unless I've added you to my contact list. Which means the scammer, already pretending to live in small town rural USA (where they most certainly are not) has to correctly guess the number of one of my relatives before my pocket actually rings. It also means I'm unreachable for anything actually important that isn't in my contact list. That's an annoying price.
I'm not sure what the correct end solution is, but the current solution seems to be very broken.