How to argue against questionable research practices such as P-hacking and Harking? I have come into conflict with co-authors when being asked to do things that I consider to be questionable. 


Once I was told to try every possible specification of a dependent variable (count, proportion, binary indicator, you name it) in a regression until I find a significant relationship. That is it, no justification for choosing one specification over another besides finding significance. The famous fishing expedition for starfish (also known as P-Hacking).
In another occasion I was asked to re-write a theory section of a paper to reflect an incidental finding from our analysis, so that it shows up as if we were asking a question about the incidental finding and had come up with the supported hypothesis a priori. The famous hypothesising after results are known (Harking). 


In both cases I refused to comply and explained my reasoning, what led to conflicts with the other party. I tried my best to not sound accusatory (not to give the impression that I doubt the ethics of the other party), but it nonetheless led to attrition and a worsening of the working relationship. In the long argument that followed, I was told that 'social science is not done as the natural sciences,' and that I was 'too inflexible,' 'too positivist,' and that everybody does these things that I was being asked to do. The argument culminated with me being asked to 'stop obstructing the progress of the paper,' what made me feel very frustrated.

Since then I have seen several cases of what I suspect to be this type of research practice. For example, PhD students coming to me to ask about what they should change in their models so that their results come out significant, and people working at the same computer lab as me asking me for the same type of help. 

I do consider these things to be seriously questionable from an ethical point of view, and would like to be able to argue against them effectively. However, the other parties are usually experienced researchers or students under the supervision of an experienced researcher. As a young researcher, I feel that I'm at a disadvantage when arguing against. It is often the case that I'm arguing against the instructions of someone who has more experience, publications, and, supposedly, knowledge than I do. 

Is this one of those cases where we can't do much but try to be the 'change that we want to bring about,' shud it, and just make sure that we are doing the right things ourselves? Should we speak up more often? If so, any good strategies to be more effective and convincing?

p.s. The tag is social sciences because of my field, but I reckon that this happens in other areas as well, and I welcome input from other fields.



EDIT 1: In example 2), at no moment anyone suggested that we would confirm the new hypothesis in a new set of data. The intention was to pretend that we got it right from the onset, which is why I objected.

EDIT 2: Just to make clear. I am aware of the right way of doing these things (i.e. cross validation, confirmatory analysis in a new dataset, penalising for multiple statistical tests, etc.). This is a question about how to argue that p-hacking and harking are not the way to go.

EDIT 3: I was unaware of the strong connotations of the word misconduct. I have edited and replaced it with 'questionable research practices'
This sort of thing happens in both the social sciences AND physical sciences.  For instance, often a scientist will collect data to test a theory but will also collect lots of extraneous data.  Analyses on these extraneous data often should be considered exploratory and labeled as such (because significant results could be due to the multiple tests)  [As another example, you don't want to know how often chemists repeat an experiment until they get a good yield, then stop and report that yield without mentioning that that was the best in 20 experiments!]

The fastest solution is to agree to do the multiple analyses, but then tell what you did in the methodology section.  If you say that you analyzed it several ways and one way showed significance, readers can decide whether or not to believe the result.  Just tell your co-authors that not mentioning that you did multiple analyses is leaving the research improperly described.

However, you can (occasionally) save the day.  If, for instance, you did 10 different analyses and picked the best one, you'll be ok if the result would hold under a Bonferroni correction (i.e. instead of requiring significance at the 0.05 level, you require significance at the 0.05/#tests level).  So it the final test shows a p-value such as 0.000001, you probably are on safe grounds.

Another approach is to a priori decide that some tests are obvious (confirmatory) and some are just searching around the data (exploratory).  Then you can demonstrate the confirmatory results, while labeling anything interesting among the 'exploratory' results as 'needs further research'.  That is, you can mix well-founded tests with 'data dredging' as long as you acknowledge the difference between the two sets of tests.

But if it isn't possible to rescue the result, I'd go with insisting that they describe what they did, with the comment that if they are embarrassed to describe it, they shouldn't have done it.  :)

You might also add that it is often obvious (at least to statisticians) that a researcher has pulled this trick.  When we see a test in isolation that would not occur to us to be the obvious approach, or a hypothesis that we'd not choose a priori, it looks suspicious.  For instance, I recently read a paper that claimed that a certain group of people tend to commit suicide more often if they were BORN in the Spring.  It was clear that JUST testing the effect of birth in Springtime was not something that would occur to anyone, without testing the effect of birth in other seasons.  So they probably had a spurious result due to multiple comparisons.