How to proceed when the baseline (state-of-the-art) published results claim much better performance than I can reproduce? I am graduate student, to finish my degree I need to build methods outperform what is already there. An issue that I came across with, is that two papers reported way (I mean more than 20%) more than what resulted from my reimplementation. This could be due to two reasons:


I missed something during the implementation. Which is what I have been telling myself. For months, I tried all possible combinations and possible paths. One of the methods is straightforward. Still, I could not reach their claimed performance.

I contacted the corresponding authors, and no one replied. So I tried to contact the other authors.

The first paper, the author replied and sent me the code. He/she told me to keep all details ”confidential”. Well, it turns out they they are not using the data they claim in their the paper, of course their results are different than my reimplementation. And my implementation was correct.

The second paper author also replied and they didn’t send me the code because they say it is easy to implement, but confirmed that what I did is correct still I couldn’t understand why such difference.

Both papers are published in &lt;2 impact factor journals. Their web servers are not working.
They are not honest.


Now I am stuck, my method does outperform my reimplementation of their methods but not what they claim. The first paper I can’t say anything because “it is confidential” the second paper I can only confirm that I correctly implemented their method for the most part  (based on my chat with the authors)

I know that I probably could not publish on this part of my work, because who is going to believe a young scientist who just started her way?
 But not sure how the committee are going to believe me. What can I say or do? Please help me
There is absolutely no reason that you can't publish a paper that says "We compared our method to methods X and Y. Since code the original code was not available for X and Y, we reimplemented the methods to the best of our ability. The code for these reimplementations is available in supplementary files A and B. Our new method out performed the reimplementations of X and Y by z%. However, it should be noted that it was not possible to reproduce the reported results for X and Y. "

People who want to know will have to look at your re-implementations and decide themselves if they think you have correctly re-implemented. 

Seniority has nothing to do with it - be transparent, and the world will judge if they believe you or the people that won't release their code.