<story><title>San Francisco cops pull over a Cruise driverless car for no lights on</title><url>https://www.theverge.com/2022/4/10/23019303/heres-what-happens-cops-pull-over-a-driverless-cruise-vehicle-general-motors</url></story><parent_chain><item><author>carabiner</author><text>You didn&amp;#x27;t design the aircraft that you fly on, which are incredibly computerized. Also the drugs that your doctor prescribes. Etc.&lt;p&gt;You put your faith in myriad bureaucracies and trust networks that you have no hope of understanding. Driverless cars are just one more added to the list.</text></item><item><author>jwr</author><text>I am really worried by the fact that &lt;i&gt;I&lt;/i&gt; am the unwilling tester in the Great Driverless Car Experiment.&lt;p&gt;Tradition has it that when you load-test a new bridge, you put the architect underneath. I feel like this, except I didn&amp;#x27;t design those driverless cars, somebody else did. Being an experienced software engineer, my trust in the software in these cars is pretty low. And yet they are testing them on me, because &lt;i&gt;I&lt;/i&gt; can be the one getting killed.&lt;p&gt;I think we should set a much higher bar for allowing those cars on the streets, rather than &amp;quot;it kinda works, so let&amp;#x27;s roll with it&amp;quot;.</text></item></parent_chain><comment><author>Veserv</author><text>The difference is that airplane software standards are by far the most rigorous in the world and so rigorous that most software developers are shocked at the level of rigor employed where as driverless cars have exactly &lt;i&gt;zero&lt;/i&gt; software standards with respect to their safety or fitness for purpose. Comparing the processes in play is a outrageously fundamental category error; the systems have standards that are like 6 orders of magnitude apart.&lt;p&gt;As a example, just look at the 737 MAX, a airplane viewed as a literal deathtrap and a complete indictment of airplane software development, which had &lt;i&gt;2&lt;/i&gt; crashes in 400,000 flights. At a average flight distance of ~500 miles, the 737 MAX, a airplane literally multiple orders of magnitude worse than any other and a complete failure of the approval process, only resulted in 1 fatality per 100 million person-miles. That is safer than the driving average of 1.1 fatalities per 100 million person-miles. What is viewed as by far the most atrocious outcome in the airplane sector, a system literally hundreds or thousands of times more dangerous than the average, is above average in the car sector; that is how far the gap is.&lt;p&gt;And this is ignoring driverless car software which is being developed to even lower standards than regular car software and which has no actual independent quality requirements. In contrast, far more lives are at risk with respect to driverless cars, so they should be developed to standards probably around 1000x better than airplane software which puts them around 0.0001% of the way there to a acceptable level of quality even if the software was already as reliable as regular car software.</text></comment>
<story><title>San Francisco cops pull over a Cruise driverless car for no lights on</title><url>https://www.theverge.com/2022/4/10/23019303/heres-what-happens-cops-pull-over-a-driverless-cruise-vehicle-general-motors</url></story><parent_chain><item><author>carabiner</author><text>You didn&amp;#x27;t design the aircraft that you fly on, which are incredibly computerized. Also the drugs that your doctor prescribes. Etc.&lt;p&gt;You put your faith in myriad bureaucracies and trust networks that you have no hope of understanding. Driverless cars are just one more added to the list.</text></item><item><author>jwr</author><text>I am really worried by the fact that &lt;i&gt;I&lt;/i&gt; am the unwilling tester in the Great Driverless Car Experiment.&lt;p&gt;Tradition has it that when you load-test a new bridge, you put the architect underneath. I feel like this, except I didn&amp;#x27;t design those driverless cars, somebody else did. Being an experienced software engineer, my trust in the software in these cars is pretty low. And yet they are testing them on me, because &lt;i&gt;I&lt;/i&gt; can be the one getting killed.&lt;p&gt;I think we should set a much higher bar for allowing those cars on the streets, rather than &amp;quot;it kinda works, so let&amp;#x27;s roll with it&amp;quot;.</text></item></parent_chain><comment><author>slg</author><text>Furthermore, at this stage the driverless cars are unlikely to be creating demand.  These are replacing human drivers.  Most of us have no hand in verifying the safety of other drivers on the road.  I happen to believe it is bonkers that we generally test drivers once at the age of 16 and somehow that earns a person driving privileges for at least 50 years before being retested if not their entire life.</text></comment>