←back to thread

US probes Tesla's Full Self-Driving software after fatal crash

(www.reuters.com)
410 points jjulius | 10 comments | 18 Oct 24 16:01 UTC | HN request time: 0.001s | source | bottom
Show context
AlchemistCamp ◴[19 Oct 24 17:17 UTC] No.41889077[source]▶
The interesting question is how good self-driving has to be before people tolerate it.

It's clear that having half the casualty rate per distance traveled of the median human driver isn't acceptable. How about a quarter? Or a tenth? Accidents caused by human drivers are one of the largest causes of injury and death, but they're not newsworthy the way an accident involving automated driving is. It's all too easy to see a potential future where many people die needlessly because technology that could save lives is regulated into a greatly reduced role.

replies(20): >>41889114 #>>41889120 #>>41889122 #>>41889128 #>>41889176 #>>41889205 #>>41889210 #>>41889249 #>>41889307 #>>41889331 #>>41889686 #>>41889898 #>>41890057 #>>41890101 #>>41890451 #>>41893035 #>>41894281 #>>41894476 #>>41895039 #>>41900280 #
gambiting ◴[19 Oct 24 17:31 UTC] No.41889176[source]▶
>>. How about a quarter? Or a tenth?

The answer is zero. An airplane autopilot has increased the overall safety of airplanes by several orders of magnitude compared to human pilots, but literally no errors in its operation are tolerated, whether they are deadly or not. The exact same standard has to apply to cars or any automated machine for that matter. If there is any issue discovered in any car with this tech then it should be disabled worldwide until the root cause is found and eliminated.

>> It's all too easy to see a potential future where many people die needlessly because technology that could save lives is regulated into a greatly reduced role.

I really don't like this argument, because we could already prevent literally all automotive deaths tomorrow through existing technology and legislation and yet we are choosing not to do this for economic and social reasons.

replies(6): >>41889247 #>>41889255 #>>41890925 #>>41891202 #>>41891217 #>>41893571 #
1. travem ◴[19 Oct 24 17:41 UTC] No.41889255[source]▶
> The answer is zero

If autopilot is 10x safer then preventing its use would lead to more preventable deaths and injuries than allowing it.

I agree that it should be regulated and incidents thoroughly investigated, however letting perfect be the enemy of good leads to stagnation and lack of practical improvement and greater injury to the population as a whole.

replies(2): >>41889357 #>>41889900 #
2. gambiting ◴[19 Oct 24 17:54 UTC] No.41889357[source]▶
>>If autopilot is 10x safer then preventing its use would lead to more preventable deaths and injuries than allowing it.

And yet whenever there is a problem with any plane autopilot it's preemptively disabled fleet wide and pilots have to fly manually even though we absolutely beyond a shadow of a doubt know that it's less safe.

If an automated system makes a wrong decision and it contributes to harm/death then it cannot be allowed on public roads full stop, no matter how many lives it saves otherwise.

replies(3): >>41889557 #>>41891095 #>>41891568 #
3. exe34 ◴[19 Oct 24 18:22 UTC] No.41889557[source]▶
> And yet whenever there is a problem with any plane autopilot it's preemptively disabled fleet wide and pilots have to fly manually even though we absolutely beyond a shadow of a doubt know that it's less safe.

just because we do something dumb in one scenario isn't a very persuasive reason to do the same in another.

> then it cannot be allowed on public roads full stop, no matter how many lives it saves otherwise.

ambulances sometimes get into accidents - we should ban all ambulances, no matter how many lives they save otherwise.

replies(1): >>41894362 #
4. penjelly ◴[19 Oct 24 19:09 UTC] No.41889900[source]▶
I'd challenge the legitimacy of the claim that it's 10x safer, or even safer at all. The safety data provided isn't compelling to me, it can be games or misrepresented in various ways, as pointed out by others.
replies(1): >>41890184 #
5. yCombLinks ◴[19 Oct 24 19:42 UTC] No.41890184[source]▶
That claim wasn't made. It was a hypothetical, what if it was 10x safer? Then would people tolerate it.
replies(1): >>41896727 #
6. CrimsonRain ◴[19 Oct 24 22:00 UTC] No.41891095[source]▶
So your only concern is, when something goes wrong, need someone to blame. Who cares about lives saved. Vaccines can cause adverse effects. Let's ban all of them.

If people like you were in charge of anything, we'd still be hitting rocks for fire in caves.

replies(1): >>41899449 #
7. Aloisius ◴[19 Oct 24 23:18 UTC] No.41891568[source]▶
Depends on what one considers a "problem." As long as the autopilot's failures conditions and mitigation procedures are documented, the burden is largely shifted to the operator.

Autopilot didn't prevent slamming into a mountain? Not a problem as long as it wasn't designed to.

Crashed on landing? No problem, the manual says not to operate it below 500 feet.

Runaway pitch trim? The manual says you must constantly be monitoring the autopilot and disengage it when it's not operating as expected and to pull the autopilot and pitch trim circuit breakers. Clearly insufficient operator training is to blame.

8. ◴[20 Oct 24 10:19 UTC] No.41894362{3}[source]▶
9. penjelly ◴[20 Oct 24 16:54 UTC] No.41896727{3}[source]▶
yes people would, if we had a reliable metric for safety of these systems besides engaged/disengaged. We don't, and 10x safer with the current metrics is not satisfactory.
10. gambiting ◴[20 Oct 24 23:38 UTC] No.41899449{3}[source]▶
Ok, consider this for a second. You're a director of a hospital that owns a Therac radiotherapy machine for treating cancer. The machine is without any shadow of a doubt saving lives. People without access to it would die or have their prognosis worsen. Yet one day you get a report saying that the machine might sometimes, extremely rarely, accidentally deliver a lethal dose of radiation instead of the therapeutic one.

Do you decide to keep using the machine, or do you order it turned off until that defect can be fixed? Why yes or why not? Why does the same argument apply/not apply in the discussion about self driving cars?

(And in case you haven't heard about it - the Therac radiotherapy machine fault was a real thing, it's being used as a cautionary tell for software development but I sometimes wonder if it should be used in philosophy classes too)