<story><title>I found a bug in SQLite</title><url>https://www.philipotoole.com/how-i-found-a-bug-in-sqlite/</url></story><parent_chain><item><author>adius</author><text>Just a few days ago I found a serious security issue in SQLite: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;sqlite.org&amp;#x2F;forum&amp;#x2F;forumpost&amp;#x2F;07beac8056151b2f&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;sqlite.org&amp;#x2F;forum&amp;#x2F;forumpost&amp;#x2F;07beac8056151b2f&lt;/a&gt;&lt;p&gt;It was also promptly fixed, but it makes me feel like the millions of tests sound better than they are in reality …</text></item></parent_chain><comment><author>ygra</author><text>Tests usually cannot prove &lt;i&gt;absence&lt;/i&gt; of defects. But they can show &lt;i&gt;presence&lt;/i&gt; of defects. So yes, a test suite that runs without errors tells you “nothing” in a way. But it&amp;#x27;s a very good safeguard to ensure that code changes don&amp;#x27;t inadvertently introduce unexpected changes.&lt;p&gt;For me it&amp;#x27;s also the case that I think much more thorough about what inputs could be possible and potentially problematic, so there&amp;#x27;s often an extra set of test cases around boundaries of input values that would have never been tried when just quickly throwing together a demo application to showcase and experiment with a new feature.&lt;p&gt;But the fundamental problem that lots of bugs don&amp;#x27;t appear in testing since those code paths are never tested also means that testing alone isn&amp;#x27;t sufficient. But I guess we all know that by now, and combining different kinds of tests with other approaches like code reviews (actual proofs are probably beyond the scope for the vast majority of software projects) is being done all the time to not bet everything on a misguided 100 % code coverage unit test approach that&amp;#x27;s both expensive and fairly useless.</text></comment>
<story><title>I found a bug in SQLite</title><url>https://www.philipotoole.com/how-i-found-a-bug-in-sqlite/</url></story><parent_chain><item><author>adius</author><text>Just a few days ago I found a serious security issue in SQLite: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;sqlite.org&amp;#x2F;forum&amp;#x2F;forumpost&amp;#x2F;07beac8056151b2f&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;sqlite.org&amp;#x2F;forum&amp;#x2F;forumpost&amp;#x2F;07beac8056151b2f&lt;/a&gt;&lt;p&gt;It was also promptly fixed, but it makes me feel like the millions of tests sound better than they are in reality …</text></item></parent_chain><comment><author>komali2</author><text>I&amp;#x27;m watching engineers re-implement error handling code I wrote line by line over the weeks after they &amp;quot;optimized&amp;quot; my error handling into a oneliner. And as the old bugs crop up when API error responses are weirdly formed, they find themselves adding the edge case code I had added previously.&lt;p&gt;If I had been granted time to add unit tests, those would just function as a source of truth: &amp;quot;sometimes the API returns this kinda weird error, so we handle it. Sometimes this one, so we handle it.&amp;quot; Unit tests are nice for that, all things this given program (the UI in this case) needs to worry about from the various things it talks to (it could talk to a couple different APIs who all have different quirks).&lt;p&gt;I wasn&amp;#x27;t granted time because the API quirks are considered bugs that are being fixed... one day... hence why the oneliner &amp;quot;refactor&amp;quot; was allowed, but regardless, it has been my go to object lesson in why I finally find unit tests useful.</text></comment>