Statistical analysis plan giving away some of my results, Reviewer 2 Context: At the end of the methodology section, I explain that I will apply an ANOVA to check if at least one condition is different, then I will investigate using multiple comparisons.
The reviewers argue that I am giving away some of my results. Since I mention that I will do multiple comparisons, I am indicating that indeed at least one of the conditions differs.
How can I reformulate my analysis plan (or change my write-up) to accommodate the reviewers? Alternately, what could I answer the proverbial reviewer #2 if I want to keep the same write-up?
This is a ridiculous request (edit: eh, maybe not quite so ridiculous; comments suggest that my 'however' paragraph below applies in this case, but I'll leave the rest of this answer as-is). For one, even before you have run any tests you should have a plan for post hoc testing.
I would respond that it's not possible to correct this without moving statistical methodology to the results, and that the authors feel strongly about keeping this methodology to one section. I can't imagine an editor having a problem with this response, and if the reviewer feels strongly that the boundaries between sections of a paper are sacred then this speaks to their preference directly as well.
However, make sure you aren't saying things you don't need to say if it will help please the reviewer. For example, if you write something equivalent to &quot;we do ANOVA on apple oranges and banana models&quot; and then say &quot;for oranges, we then did post hoc test xyz corrected for multiple comparisons&quot;, you can drop oranges from the second section, writing &quot;if significant, we then did post hoc test xyz corrected for multiple comparisons&quot;. It doesn't matter that you ran these tests on oranges, it matters that you planned to do this post hoc testing for any significant omnibus ANOVA results, no matter which class of fruit.