How to deal with low quality software project that needs to be evaluated I have evaluated a software project which has been created by a consortium of top scientists in the field. However, the project itself it doesn't really work, and has only been developed as a proof-of-concept rather than an end product (i.e., it works only with 2-3 scenarios).

This software application should perform 4 steps in order to successfully execute. Each step takes an input file, and produces an output file. The output file from the previous step is used as input in the current step. 
Initially, you start with 1 file. This file is used as input for step 1. After step 1, another file is produced. Let’s call the input file general_input_file, and the output file general_output_file. When general_input_file is loaded into the application, general_output_file should be produced. Now, I have an input file which I will call my_input_file. I expect the application to produce my_output_file. However, the application only accepts specific_input_file and will produce a specific_output_file. This means that it only works with 2 files which has been previously generated. Both files exist in the project. When I look at the part of the project which should process the general_input_file, there is a statement which looks like this: if the name of the given input file equals specific_input_file, then return specific_output_file. This is a file inside the project. Otherwise, try to process the generate_input_file and generate general_output_file. At this point the software breaks. A number of exceptions are thrown, and debugging and fixing this is beyond of the work that I’m doing.

The question is: in the paper, how do you approach this issue? And how do you argue, in the paper, that the reason you're unable to evaluate the software in a different scenario is due to the limitations of the software? What is the best wording to be used, without being offensive to the authors?
When I look at the part of the project which should process the
  general_input_file, there is a statement which looks like this: if the
  name of the given input file equals specific_input_file, then return
  specific_output_file. [...]
  Otherwise, try to process the generate_input_file and generate 
  general_output_file. At this point the software breaks.


By your description, it sounds like the code you are looking at is cheating by returning a precomputed output known to be correct if the input matches a single sample input the authors included with their code (presumably to demonstrate its correctness and because including a sample input was required to get their paper accepted). For all input instances other than this one sample input, the code does something entirely different, which doesn't produce correct results.

This sort of behavior would without question merit a failing grade in a programming course homework assignment or exam. If done in the context of a commercial product, similar behavior would justify the resignation of the CEO and a multibillion dollar scandal. So, saying that the code is "of low quality" seems to me so euphemistic as to be itself a dishonest statement. The way I would describe it is: if the authors are claiming that their code is correct, then they are lying.

Now, I could discuss your actual question of how to discuss this situation in your follow-up paper "without being offensive to the authors," but honestly, I don't see the point. I would ask instead, why would you want to not be offensive to the authors? It's not just that the authors' algorithm may be wrong and that their code is of poor quality; they are seemingly committing academic fraud by submitting an incorrect algorithm with code dishonestly tweaked to make it seem like the algorithm is working. I'm willing to leave a 3% chance that some other more innocent explanation can be proposed, but given the description you've provided I really can't think of one. I'll be happy to reconsider if you provide a bit more detail on how the authors' claims are presented in the paper.