<story><title>Study: Tesla Autopilot misleading, overestimated more than similar technology</title><url>https://www.mercurynews.com/2019/06/21/study-tesla-autopilot-misleading-overestimated-more-than-similar-technology/</url></story><parent_chain><item><author>Cookingboy</author><text>If you go to certain sub-Reddits or other similar forums, you will see that there is a tremendous amount of misinformation associated with Tesla, Tesla Auto Pilot, driver assist technology and Full Self Driving technology in general. I largely blame the absurdly irresponsible misinformation campaign spearheaded by Tesla and Elon personally. It&amp;#x27;s awful to get the public&amp;#x27;s understand of a nascent and potential game changing technology to start off on such a terrible note.&lt;p&gt;Some of the most common mis-conceptions I&amp;#x27;ve seen include:&lt;p&gt;1. Level 5 Full Self Driving tech is right around the corner, with government regulation being the biggest show stopper. Elon promised &amp;quot;feature complete&amp;quot; of FSD by end of this year, and claims that within 6-12 months of that full level 5 self driving will be ready to ship to consumers.&lt;p&gt;2. Tesla is unique in their usage of computer vision in their FSD stack, and the rest of the industry &lt;i&gt;only&lt;/i&gt; uses LIDAR. In reality almost &lt;i&gt;everything&lt;/i&gt; Tesla does Waymo and others also do and very likely do it better, and on top of that they use LIDAR.&lt;p&gt;3. Since companies like Waymo employs pre-mapping and LIDAR, what they are building is a completely hardcoded solution to driving with very low degree of versatility and flexibility. I&amp;#x27;ve seen Tesla fan suggesting that Waymo handles Stop Signs by hard coding the location of each stop sign in the map.&lt;p&gt;4. Because Auto Pilot, a level 2 driver assist system is being shipped in production cars and are racking up millions of miles, Tesla does not need any actual comprehensive testing of their FSD stack. I&amp;#x27;ve seen many Tesla fans arguing this is why Tesla has done more test and gathers more &amp;quot;data&amp;quot; than companies like Waymo, when in reality Tesla&amp;#x27;s FSD system has seen almost &lt;i&gt;ZERO&lt;/i&gt; actual testing anywhere, and puts them far behind competitors.&lt;p&gt;5. OTA updates makes Auto Pilot safer than competitors. In reality OTA very often enables less stringent QA and less rigorous regression testing. The Mountain View driver who was killed at the 85&amp;#x2F;101 on-ramp was the result of a terrible AP regression that failed to handle concrete barriers that it used to handle.&lt;p&gt;6. Auto Pilot is already safer than human driver because it has lower accidents&amp;#x2F;mile than the national average. This one is especially egregious since Tesla used active AP safety data compared against all motor vehicle accident data, which includes &lt;i&gt;motorcycles&lt;/i&gt; and cars of all ages, driven by drivers of all ages, employing safety techs (or lack there of) of all levels, driving in all conditions. Where as active AP data is gathered by Tesla cars that&amp;#x27;s newer than 5 years old, driven by older, wealthier drivers, employs crucial safety tech such as automatic emergency braking, and only engaged during favorable conditions (mostly highway driving, clear weather, etc). When compared against data from cars with AEB, Auto Pilot has very comparative safety record.&lt;p&gt;Elon always had a tendency to be fast and loose with facts&amp;#x2F;predictions&amp;#x2F;promises, and usually I view them as entertaining at best and disappointing at worst, but with safety critical features like this what he&amp;#x27;s doing makes me feel borderline angry. A few major mistakes will set back the industry by years and as of now, Tesla is literally the &lt;i&gt;only&lt;/i&gt; driver assist system that has a fatality count. I honestly cannot understand how any engineers with good conscious can still work on AP&amp;#x2F;FSD at Tesla and sleep well at night.</text></item><item><author>neotek</author><text>&amp;gt;Tesla is making progress, and regardless of whether they survive these various courts of public opinion, they&amp;#x27;re helping to lay the ground work for safer roads by forcing the industry to integrate more and more automated safety systems.&lt;p&gt;They&amp;#x27;re also wilfully misrepresenting the quality and safety of their technology to their customers, presenting intentionally misleading statistics to the media about it, and attempting to suppress all criticism by insisting it&amp;#x27;s part of some big oil &amp;#x2F; short seller conspiracy.</text></item><item><author>AcerbicZero</author><text>&amp;quot;Cruise Control&amp;quot; doesn&amp;#x27;t mean I can ignore the ~3200lbs of sports car I drive, just like &amp;quot;Tesla Autopilot&amp;quot; doesn&amp;#x27;t mean drivers can ignore the ~5000lbs of battery under them.&lt;p&gt;As an American who spends considerable time driving cars in various performance events (even some fun wheel to wheel endurance stuff) I think I can safely estimate at least ~25% of drivers on the road today, probably shouldn&amp;#x27;t be. Tesla is making progress, and regardless of whether they survive these various courts of public opinion, they&amp;#x27;re helping to lay the ground work for safer roads by forcing the industry to integrate more and more automated safety systems.&lt;p&gt;There will be problems. Computers will fail, mistakes will be made, and people will die, but I&amp;#x27;m confident the break even point on those human costs is already pretty far behind us.*&lt;p&gt;*That said, we already have onerous and terrible auto regulations in this country. If we (as a society) want to force companies to be more cautious in this area, well, its entirely within our regulatory structures power to do so.</text></item></parent_chain><comment><author>Faark</author><text>Your statements seem to have quite a slant as well.&lt;p&gt;&amp;gt; 3. [..] I&amp;#x27;ve seen Tesla fan suggesting that Waymo handles Stop Signs by hard coding the location of each stop sign in the map.&lt;p&gt;I&amp;#x27;m sure you&amp;#x27;ll see uninformed individuals behaving uninformed. But in my experience does this not represent the average of e.g. the r&amp;#x2F;tesla community. It&amp;#x27;s surprising to have you specifically mention Waymo, since it was mostly others (especially German car makers) who usually bring up how important high precision maps (or more recently &amp;quot;5G&amp;quot;, while we&amp;#x27;re mentioning buzzwords) is for self driving. My first google result right now was from Lyft [0]. I can totally understand how people less informed on the topic can get that wrong.&lt;p&gt;&amp;gt; 4. [..] when in reality Tesla&amp;#x27;s FSD system has seen almost ZERO actual testing anywhere&lt;p&gt;What? The usual argument I remember is Tesla outsourcing the testing to its customers and the many good reasons against it. But how can you in good faith represent the tesla AP as untested after it has driven millions of miles? Or are you specifically talking about whatever final L5 FSD solution Tesla might come up with one day? Since this obviously cannot be tested yet, since it doesn&amp;#x27;t exist.&lt;p&gt;&amp;gt; 5. The Mountain View driver who was killed at the 85&amp;#x2F;101 on-ramp was the result of a terrible AP regression that failed to handle concrete barriers that it used to handle.&lt;p&gt;Can you source that being caused by a regression? The best I&amp;#x27;ve found [1] mentions someone else saying it got worse in a comparable spot, but I also remember reading the killed driver having specifically complained to Tesla about the location the crash happened, but not if AP behavior there had gotten better in between.&lt;p&gt;But yeah, a lot of Tesla&amp;#x27;s communication around that case was horrible and I also don&amp;#x27;t feel they do enough to stop their fans and customers from being aware of the technologies current limits.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;medium.com&amp;#x2F;@LyftLevel5&amp;#x2F;https-medium-com-lyftlevel5-rethinking-maps-for-self-driving-a147c24758d6&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;medium.com&amp;#x2F;@LyftLevel5&amp;#x2F;https-medium-com-lyftlevel5-r...&lt;/a&gt;
[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arstechnica.com&amp;#x2F;cars&amp;#x2F;2019&amp;#x2F;03&amp;#x2F;dashcam-video-shows-tesla-steering-toward-lane-divider-again&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arstechnica.com&amp;#x2F;cars&amp;#x2F;2019&amp;#x2F;03&amp;#x2F;dashcam-video-shows-tes...&lt;/a&gt;</text></comment>
<story><title>Study: Tesla Autopilot misleading, overestimated more than similar technology</title><url>https://www.mercurynews.com/2019/06/21/study-tesla-autopilot-misleading-overestimated-more-than-similar-technology/</url></story><parent_chain><item><author>Cookingboy</author><text>If you go to certain sub-Reddits or other similar forums, you will see that there is a tremendous amount of misinformation associated with Tesla, Tesla Auto Pilot, driver assist technology and Full Self Driving technology in general. I largely blame the absurdly irresponsible misinformation campaign spearheaded by Tesla and Elon personally. It&amp;#x27;s awful to get the public&amp;#x27;s understand of a nascent and potential game changing technology to start off on such a terrible note.&lt;p&gt;Some of the most common mis-conceptions I&amp;#x27;ve seen include:&lt;p&gt;1. Level 5 Full Self Driving tech is right around the corner, with government regulation being the biggest show stopper. Elon promised &amp;quot;feature complete&amp;quot; of FSD by end of this year, and claims that within 6-12 months of that full level 5 self driving will be ready to ship to consumers.&lt;p&gt;2. Tesla is unique in their usage of computer vision in their FSD stack, and the rest of the industry &lt;i&gt;only&lt;/i&gt; uses LIDAR. In reality almost &lt;i&gt;everything&lt;/i&gt; Tesla does Waymo and others also do and very likely do it better, and on top of that they use LIDAR.&lt;p&gt;3. Since companies like Waymo employs pre-mapping and LIDAR, what they are building is a completely hardcoded solution to driving with very low degree of versatility and flexibility. I&amp;#x27;ve seen Tesla fan suggesting that Waymo handles Stop Signs by hard coding the location of each stop sign in the map.&lt;p&gt;4. Because Auto Pilot, a level 2 driver assist system is being shipped in production cars and are racking up millions of miles, Tesla does not need any actual comprehensive testing of their FSD stack. I&amp;#x27;ve seen many Tesla fans arguing this is why Tesla has done more test and gathers more &amp;quot;data&amp;quot; than companies like Waymo, when in reality Tesla&amp;#x27;s FSD system has seen almost &lt;i&gt;ZERO&lt;/i&gt; actual testing anywhere, and puts them far behind competitors.&lt;p&gt;5. OTA updates makes Auto Pilot safer than competitors. In reality OTA very often enables less stringent QA and less rigorous regression testing. The Mountain View driver who was killed at the 85&amp;#x2F;101 on-ramp was the result of a terrible AP regression that failed to handle concrete barriers that it used to handle.&lt;p&gt;6. Auto Pilot is already safer than human driver because it has lower accidents&amp;#x2F;mile than the national average. This one is especially egregious since Tesla used active AP safety data compared against all motor vehicle accident data, which includes &lt;i&gt;motorcycles&lt;/i&gt; and cars of all ages, driven by drivers of all ages, employing safety techs (or lack there of) of all levels, driving in all conditions. Where as active AP data is gathered by Tesla cars that&amp;#x27;s newer than 5 years old, driven by older, wealthier drivers, employs crucial safety tech such as automatic emergency braking, and only engaged during favorable conditions (mostly highway driving, clear weather, etc). When compared against data from cars with AEB, Auto Pilot has very comparative safety record.&lt;p&gt;Elon always had a tendency to be fast and loose with facts&amp;#x2F;predictions&amp;#x2F;promises, and usually I view them as entertaining at best and disappointing at worst, but with safety critical features like this what he&amp;#x27;s doing makes me feel borderline angry. A few major mistakes will set back the industry by years and as of now, Tesla is literally the &lt;i&gt;only&lt;/i&gt; driver assist system that has a fatality count. I honestly cannot understand how any engineers with good conscious can still work on AP&amp;#x2F;FSD at Tesla and sleep well at night.</text></item><item><author>neotek</author><text>&amp;gt;Tesla is making progress, and regardless of whether they survive these various courts of public opinion, they&amp;#x27;re helping to lay the ground work for safer roads by forcing the industry to integrate more and more automated safety systems.&lt;p&gt;They&amp;#x27;re also wilfully misrepresenting the quality and safety of their technology to their customers, presenting intentionally misleading statistics to the media about it, and attempting to suppress all criticism by insisting it&amp;#x27;s part of some big oil &amp;#x2F; short seller conspiracy.</text></item><item><author>AcerbicZero</author><text>&amp;quot;Cruise Control&amp;quot; doesn&amp;#x27;t mean I can ignore the ~3200lbs of sports car I drive, just like &amp;quot;Tesla Autopilot&amp;quot; doesn&amp;#x27;t mean drivers can ignore the ~5000lbs of battery under them.&lt;p&gt;As an American who spends considerable time driving cars in various performance events (even some fun wheel to wheel endurance stuff) I think I can safely estimate at least ~25% of drivers on the road today, probably shouldn&amp;#x27;t be. Tesla is making progress, and regardless of whether they survive these various courts of public opinion, they&amp;#x27;re helping to lay the ground work for safer roads by forcing the industry to integrate more and more automated safety systems.&lt;p&gt;There will be problems. Computers will fail, mistakes will be made, and people will die, but I&amp;#x27;m confident the break even point on those human costs is already pretty far behind us.*&lt;p&gt;*That said, we already have onerous and terrible auto regulations in this country. If we (as a society) want to force companies to be more cautious in this area, well, its entirely within our regulatory structures power to do so.</text></item></parent_chain><comment><author>wcoenen</author><text>&amp;gt; &lt;i&gt;Tesla is literally the only driver assist system that has a fatality count&lt;/i&gt;&lt;p&gt;Uber had a fatal self-driving accident.</text></comment>