I have egregiously sloppy (possibly falsified) data that I need to correct. How should I go about this? Several years ago, I did a summer project as an undergrad with a PI not at my undergrad institution (I'm in grad school now). Now, the project is finished and he wants to publish it with data that I've gathered.
The problem is, after checking my data, that the spreadsheets that I used to calculate the data are very inaccurate, to say the least. I can't remember exactly what I did, but there do seem to be some egregious inaccuracies (as in, values from the wrong dataset) in my spreadsheet compared to the raw data from the instrument either that I was colossally bad at copying data over correctly (all of the data in my spreadsheet do appear in the raw data someplace, but not necessarily in the right location, the right dataset, etc.), or worse, I'm terrified that I may have done it deliberately at the time. (How would someone be able to tell?) I did analysis with the faulty data back then and produced graphs.
Obviously, I will have to supply the correct data analysis before it gets published. But is it ok to explain it as &quot;I majorly screwed up the analysis and made a lot of data entry errors, here's the corrected version&quot;? I'm hoping that my PI will just use my corrected data and analysis and just focus on that instead trying to pore over my wrong spreadsheets to individually compare all the changes, many of which are embarrassing at best and look suspicious at worst. I am not under any suspicion of wrongdoing thus far.
I redid my data analysis with correct data entry and the results look messier on most of the things I messed up, though they do not affect the underlying trends. Not sure what this means - how likely is it for honest error to have the effect of making data look cleaner than it actually is? Also, my god, I was awful - mishmashing data from one trial with another, copying data from the wrong wavelengths and/or from the wrong trial, wrong normalization numbers, etc. etc. etc.
If you were a professor and you were approached by a student who says that made egregious errors on past data analysis and the corrected data looks not nearly as good, what would you think? Would you think they were honestly trying to fix a mistake, or trying to save face and cover for past misconduct?
I infer from the question that you still have the raw data. In that case, I think you need to re-do all the steps in which errors could be introduced (such as data entry, spread-sheet calculations, analysis of trends). Then see what effect the corrections have on the conclusions of your paper, and rewrite as necessary. I realize that this sounds like a lot of work, essentially repeating most of what you had done earlier, but since you apparently did it wrong the first time, you now need to do it right.
Meanwhile, you should tell the PI about the problem and your plans for correcting it. If I correctly understand the situation, the raw data are the only reliable information currently available. You should make sure he knows that, so that he (and others working with him) won't build further research on a foundation of wrong results.