<story><title>Court rules drug dog barely more accurate than a coin flip is good enough</title><url>https://www.washingtonpost.com/news/the-watch/wp/2015/08/04/federal-appeals-court-drug-dog-thats-barely-more-accurate-than-a-coin-flip-is-good-enough/</url></story><parent_chain><item><author>gburt</author><text>This isn&amp;#x27;t being fair. I don&amp;#x27;t really think the post title is fair either.  That 93% isn&amp;#x27;t comparable to the 50% you&amp;#x27;re probably thinking for the coin flip, or even the 0&amp;#x2F;100% mentioned in your edit.&lt;p&gt;---&lt;p&gt;93% alert rate means:&lt;p&gt;93% = false positive + true positive&lt;p&gt;7% = true negative + false negative&lt;p&gt;---&lt;p&gt;We&amp;#x27;re given &amp;quot;His success rate was much lower, at 59 percent.&amp;quot; Assuming that means that 59% are true positives, we know the false positive rate is 34%.&lt;p&gt;Without data on how many passengers are carrying drugs, we can&amp;#x27;t say anything about the rest of that result. A plausible world where the drug dog may be valuable is when he unquestionably rules out that 7% of the population that he doesn&amp;#x27;t alert on, as in false negative = 0.&lt;p&gt;I believe the reason this number is so high (59%) is that the officer has already used his suspicion to determine filter out most people. This makes the value of the &amp;quot;ruling out&amp;quot; plausibility even more valuable: the drug dog reduces searches compared to trusting the cop.&lt;p&gt;Ideally, they&amp;#x27;d provide the receiver operating characteristics curve [0] or the confusion matrix [1].&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Receiver_operating_characteristic&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Receiver_operating_characteris...&lt;/a&gt;
[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Confusion_matrix&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Confusion_matrix&lt;/a&gt;</text></item><item><author>fiatmoney</author><text>The dog isn&amp;#x27;t even &amp;quot;as accurate as a coin flip&amp;quot;.  Given a 93% alert rate, regardless of the subsequent true positive rate, information gain from an alert is approximately zero.&lt;p&gt;Edit: &amp;quot;accurate as a coin flip&amp;quot; is a bad construction, but I wouldn&amp;#x27;t exactly expect better from a newspaper.  The information gain from a device that activates randomly 50% of the time, 0% of the time, and 100% of the time are all the same, which is to say zero.</text></item></parent_chain><comment><author>nickff</author><text>The real question is whether the dog is better than the officer at guessing who has drugs.  If the dog is much better than the officer, it would suggest that a dog &amp;#x27;alert&amp;#x27; should be given more weight than the officer&amp;#x27;s guess.  If the dog is no better than the officer, the dog is useless.&lt;p&gt;I say this because it is possible (and indeed likely) that the officer is also better at guessing who has drugs than a coin toss.&lt;p&gt;Full Disclosure: I have never used or trafficked contraband, but I do not support random searches or the use of drug dogs in public without a warrant, regardless of their effectiveness.</text></comment>
<story><title>Court rules drug dog barely more accurate than a coin flip is good enough</title><url>https://www.washingtonpost.com/news/the-watch/wp/2015/08/04/federal-appeals-court-drug-dog-thats-barely-more-accurate-than-a-coin-flip-is-good-enough/</url></story><parent_chain><item><author>gburt</author><text>This isn&amp;#x27;t being fair. I don&amp;#x27;t really think the post title is fair either.  That 93% isn&amp;#x27;t comparable to the 50% you&amp;#x27;re probably thinking for the coin flip, or even the 0&amp;#x2F;100% mentioned in your edit.&lt;p&gt;---&lt;p&gt;93% alert rate means:&lt;p&gt;93% = false positive + true positive&lt;p&gt;7% = true negative + false negative&lt;p&gt;---&lt;p&gt;We&amp;#x27;re given &amp;quot;His success rate was much lower, at 59 percent.&amp;quot; Assuming that means that 59% are true positives, we know the false positive rate is 34%.&lt;p&gt;Without data on how many passengers are carrying drugs, we can&amp;#x27;t say anything about the rest of that result. A plausible world where the drug dog may be valuable is when he unquestionably rules out that 7% of the population that he doesn&amp;#x27;t alert on, as in false negative = 0.&lt;p&gt;I believe the reason this number is so high (59%) is that the officer has already used his suspicion to determine filter out most people. This makes the value of the &amp;quot;ruling out&amp;quot; plausibility even more valuable: the drug dog reduces searches compared to trusting the cop.&lt;p&gt;Ideally, they&amp;#x27;d provide the receiver operating characteristics curve [0] or the confusion matrix [1].&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Receiver_operating_characteristic&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Receiver_operating_characteris...&lt;/a&gt;
[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Confusion_matrix&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Confusion_matrix&lt;/a&gt;</text></item><item><author>fiatmoney</author><text>The dog isn&amp;#x27;t even &amp;quot;as accurate as a coin flip&amp;quot;.  Given a 93% alert rate, regardless of the subsequent true positive rate, information gain from an alert is approximately zero.&lt;p&gt;Edit: &amp;quot;accurate as a coin flip&amp;quot; is a bad construction, but I wouldn&amp;#x27;t exactly expect better from a newspaper.  The information gain from a device that activates randomly 50% of the time, 0% of the time, and 100% of the time are all the same, which is to say zero.</text></item></parent_chain><comment><author>brobdingnagian</author><text>The dog has low precision and high recall. The dogs are capable of high precision and high recall, but aren&amp;#x27;t being trained to do that because other incentives are at play.&lt;p&gt;I found it impressive that the dog that alerts 93% of the time was correct 40% of the time. That means the police are doing what seems like a good job of identifying targets that actually have drugs. With those kind of odds, you&amp;#x27;d probably want to take a closer look at everyone just to make sure you aren&amp;#x27;t letting people with drugs get away (false negatives).</text></comment>