Publishing a paper with questionable statistics I've submitted a paper to a reasonably good journal in my field. One of the reviewers questioned the statistical methods and recommended major revisions. Without getting into the nitty-gritty of the analysis, I sought help from a statistics expert and whilst they said that my analysis wasn't strictly &quot;wrong&quot; they suggested that I correct my p-values for multiple comparisons in order to pacify the reviewers. The issue is, when I applied the correction none of my findings came out to be significant. I am fine with that, for most of the paper we're describing general trends anyway, so I think that removing mention of significance isn't a huge deal, and everyone talks about the need to publish negative findings, so why would this be so bad?
My advisor however disagrees, they think that the adjustment is too stringent and we're penalizing the findings unnecessarily (I am using false-discovery rate, so the most lenient you could get). They also think that if we remove the mentions of significance the paper is likely to be rejected because then we don't have much to back up the trends we're presenting. My advisor wants me to go ahead and resubmit the paper by making slight adjustments. This would be the first paper I've published as first author, and I am struggling to get onboard with blatantly ignoring a the statistical error I would be committing for the sake of publishing it. My advisor has agreed for the correction of the p-values to be applied and discussed in my thesis.
My dilemma lies in whether I try to fight my advisor on this (I don't think I will win), or do as they ask and hope the paper gets rejected, or maybe I'm blowing this out of proportion and my advisor is right. I am concerned that I will regret publishing something I know to be questionable for the sake of publishing.
Well, if you're having second thoughts is because deep down you have a solid reason for it.
And, you do, as you said in your text, the statistical data is lacking additional background support. If possible add more data to the existing data, and make changes to the text accordingly.
Is frequently found on research papers, some shortcuts when presenting experimental data that is lacking some explanation/support. For instance, changing the axis scales, present data in a normalized axis scale, among many others. These data presentation techniques will force the reviewers to spend more time analyzing your data looking for clues on lack of validity. In such cases, even if it gets published, later on, it will be reflected on the number of citations (for instance). To simply put, I don't recommend doing it nowadays.