←back to thread

US probes Tesla's Full Self-Driving software after fatal crash

(www.reuters.com)
410 points jjulius | 1 comments | 18 Oct 24 16:01 UTC | HN request time: 0s | source
Show context
lrvick ◴[19 Oct 24 23:16 UTC] No.41891555[source]▶
All these self driving car companies are competing to see whose proprietary firmware and sensors kill the fewest people. This is insane.

I will -never- own a self driving car unless the firmware is open source, reproducible, remotely attestable, and built/audited by several security research firms and any interested security researchers from the public before all new updates ship.

It is the only way to avoid greedy execs from cutting corners to up profit margins like VW did with faking emissions tests.

Proprietary safety tech is evil, and must be made illegal. Compete with nicer looking more comfortable cars with better miles-to-charge, not peoples lives.

replies(1): >>41891758 #
1. boshalfoshal ◴[19 Oct 24 23:53 UTC] No.41891758[source]▶
You are conflating two seperate problems (security vs functionality).

"Firmware" can be open source and secure, but how does this translate to driving performance at all? Why does it matter if the firmware is validated by security researchers, who presumably don't know anything about motion planning, perception, etc? And this is even assuming that the code can be reasonably verified statically. You probably need to to run that code on a car for millions of miles (maybe in simulation) in an uncoutable number of scenarios to run through every edge case.

The other main problem with what you're asking is that most of the "alpha" of these self driving companies is in proprietary _models_, not software. No one is giving up their models. That is a business edge.

As someone who has been at multiple AV companies, no one is cutting corners on "firmware" or "sensors" (apart from making it reasonably cost effective so normal people can buy their cars). Its just that AV is a really really really difficult problem with no closed form solution.

Your normal car has all the same pitfalls of "unverified software running on a safety critical system," except that its easier to verify that straightforward device firmware works vs a very complex engine whose job is to ingest sensor data and output a trajectory.