<story><title>Alcoholics Anonymous vs. other approaches: the evidence is now in</title><url>https://www.nytimes.com/2020/03/11/upshot/alcoholics-anonymous-new-evidence.html</url></story><parent_chain><item><author>notafraudster</author><text>Do you think you could elaborate why you believe this is &amp;quot;heavy-duty&amp;quot; new research (or even new research at all, rather than a new meta-analytic synthesis of old research)?&lt;p&gt;My quick read of the study suggests that they examined several outcomes, got null or low powered results on most of them, and chose to highlight the single result which did show a significantly positive effect on recovery; that result was based on a meta-analytic estimate of the pooled sample of just two studies. In particular, my understanding is that this study found a null on &amp;quot;percentage days abstinent&amp;quot;, a null on &amp;quot;longest period of abstinence&amp;quot;, a null on &amp;quot;drinks per drinking day&amp;quot;, a null on &amp;quot;alcohol-related consequences&amp;quot;. For alcohol addiction severity they report the results of one study without doing any original analysis. And for &amp;quot;rates of continuous abstinence&amp;quot; they found a positive effect. In most cases these are graded as &amp;quot;low certainty evidence&amp;quot;. The positive effect is claimed to be &amp;quot;high certainty evidence&amp;quot;, but the CI &amp;#x2F; p-value for the positive effect is not adjusted for multiple comparisons -- I don&amp;#x27;t think that&amp;#x27;s fatal, but it does speak to the fragility of the outcome.&lt;p&gt;The other concern I have is that they separate &amp;quot;manualized&amp;quot; treatments (e.g. AA treatments administered according to a protocol) from &amp;quot;non-manualized&amp;quot; (e.g. ad hoc administration of the AA treatment) treatments. This is fine -- we would expect to see that if something about the protocol was useful, the non-manualized treatments would display an attenuated version of the same effect. Instead we get a null on the original positive effect, and a positive effect on one of the original nulls&lt;p&gt;Finally, the authors seem a little loose with what the comparison groups are. They claim they recruited studies which compared AA to no treatment, but the results are all motivated as being AA versus CBT. What happened to the no treatment studies? Perhaps these are buried in the full text. The rate of spontaneous remission of addiction is believed to be fairly high and most of the past criticism of the efficacy of AA has been motivated by comparison to spontaneous remission.&lt;p&gt;It strikes me that there is significant distortion of the study findings from the Cochrane write-up to the Stanford press release, and from the Stanford press release to the NYT piece, and that in both cases the distortion is in favour of claiming the study has found affirmative evidence that AA works better, rather than just no evidence to claim it works worse. For instance, one of the thrusts of the NYT piece is that the past Cochrane review was based on a limited number of studies... but the operative finding in this review is based on an even more limited number of studies, even if the pool of studies from which they drew has become larger over the last 15 years.&lt;p&gt;All of this is from a quick read of the review -- I am off campus right now and don&amp;#x27;t feel like VPNing or pirating the full text to deep dive the analysis.</text></item><item><author>dang</author><text>So far the comments have been about people&amp;#x27;s prior experiences with the topic (which is great), or prior opinions about the topic (which is ok). But the more interesting story is being overlooked: this is heavy-duty new research that overturns previous conclusions, including the prior expectations of at least one of the lead authors. It would be good to discuss the specifics of the article and the new study.</text></item></parent_chain><comment><author>runarberg</author><text>&amp;gt; And for &amp;quot;rates of continuous abstinence&amp;quot; they found a positive effect.&lt;p&gt;It’s been a while since I reviewed the literature for substance abuse disorder, but if I remember correctly it was highly disputed whether complete abstinence was necessary (or even beneficial) for treatment. Has this changed?&lt;p&gt;The argument was that by focusing on abstinence you are setting a significant part of your patience up for failure with all the psychological harm involved in knowing that they’ve failed, which might result in severe and frequent relapses.&lt;p&gt;EDIT: To conclude. I find it hard to draw conclusions if success is measured in “days of abstinence”. That means that a patient that has recovered from cognitive behavior therapy (CBT) that uses occasionally and non-detrimentally is not considered a success! Further AA or other 12-step programs do often stress abstinence while CBT does not, this creates a counting bias in favor of 12-step programs.</text></comment>
<story><title>Alcoholics Anonymous vs. other approaches: the evidence is now in</title><url>https://www.nytimes.com/2020/03/11/upshot/alcoholics-anonymous-new-evidence.html</url></story><parent_chain><item><author>notafraudster</author><text>Do you think you could elaborate why you believe this is &amp;quot;heavy-duty&amp;quot; new research (or even new research at all, rather than a new meta-analytic synthesis of old research)?&lt;p&gt;My quick read of the study suggests that they examined several outcomes, got null or low powered results on most of them, and chose to highlight the single result which did show a significantly positive effect on recovery; that result was based on a meta-analytic estimate of the pooled sample of just two studies. In particular, my understanding is that this study found a null on &amp;quot;percentage days abstinent&amp;quot;, a null on &amp;quot;longest period of abstinence&amp;quot;, a null on &amp;quot;drinks per drinking day&amp;quot;, a null on &amp;quot;alcohol-related consequences&amp;quot;. For alcohol addiction severity they report the results of one study without doing any original analysis. And for &amp;quot;rates of continuous abstinence&amp;quot; they found a positive effect. In most cases these are graded as &amp;quot;low certainty evidence&amp;quot;. The positive effect is claimed to be &amp;quot;high certainty evidence&amp;quot;, but the CI &amp;#x2F; p-value for the positive effect is not adjusted for multiple comparisons -- I don&amp;#x27;t think that&amp;#x27;s fatal, but it does speak to the fragility of the outcome.&lt;p&gt;The other concern I have is that they separate &amp;quot;manualized&amp;quot; treatments (e.g. AA treatments administered according to a protocol) from &amp;quot;non-manualized&amp;quot; (e.g. ad hoc administration of the AA treatment) treatments. This is fine -- we would expect to see that if something about the protocol was useful, the non-manualized treatments would display an attenuated version of the same effect. Instead we get a null on the original positive effect, and a positive effect on one of the original nulls&lt;p&gt;Finally, the authors seem a little loose with what the comparison groups are. They claim they recruited studies which compared AA to no treatment, but the results are all motivated as being AA versus CBT. What happened to the no treatment studies? Perhaps these are buried in the full text. The rate of spontaneous remission of addiction is believed to be fairly high and most of the past criticism of the efficacy of AA has been motivated by comparison to spontaneous remission.&lt;p&gt;It strikes me that there is significant distortion of the study findings from the Cochrane write-up to the Stanford press release, and from the Stanford press release to the NYT piece, and that in both cases the distortion is in favour of claiming the study has found affirmative evidence that AA works better, rather than just no evidence to claim it works worse. For instance, one of the thrusts of the NYT piece is that the past Cochrane review was based on a limited number of studies... but the operative finding in this review is based on an even more limited number of studies, even if the pool of studies from which they drew has become larger over the last 15 years.&lt;p&gt;All of this is from a quick read of the review -- I am off campus right now and don&amp;#x27;t feel like VPNing or pirating the full text to deep dive the analysis.</text></item><item><author>dang</author><text>So far the comments have been about people&amp;#x27;s prior experiences with the topic (which is great), or prior opinions about the topic (which is ok). But the more interesting story is being overlooked: this is heavy-duty new research that overturns previous conclusions, including the prior expectations of at least one of the lead authors. It would be good to discuss the specifics of the article and the new study.</text></item></parent_chain><comment><author>dang</author><text>Thanks! That&amp;#x27;s a fabulous comment. By &amp;quot;heavy duty new research&amp;quot; I was just referring to this from the article:&lt;p&gt;&lt;i&gt;In the last decade or so, researchers have published a number of very high-quality randomized trials and quasi-experiments. Of the 27 studies in the new review, 21 have randomized designs. Together, these flip the conclusion.&lt;/i&gt;&lt;p&gt;That sounds impressive. If it turns out not to be so impressive, that&amp;#x27;s definitely on topic.</text></comment>