Is the evaluation section of a paper a "contribution," if it only describes the performance of the proposed system with no novel ideas? Recently I was reading a paper where the authors include their evaluation as a part of their contribution. In general the paragraph I am referring to was something like this:


  Our contributions are a,b,c,d and e


Where e describes the evaluation. 

In evaluation section, is about the performance evaluation of the proposed system. So the are running some simulations and benchmarks to show in which of the cases they perform better than the state of the art tools.

The evaluation (as a process) is not containing something novel, except from some modifications they did to adapt the benchmark / evaluation framework to their needs.

So, is this considered a 'contribution'? Is it 'good' to list the evaluation in the list of contributions?
There are no official standards for what constitutes a valid use of the term "contribution".  It's intrinsically a vague word, and authors can use it in many ways.  It would be inappropriate to list something as a research contribution if there's absolutely nothing at all original about it, but a small amount of originality could be enough (and opinions can differ as to where the threshold is).  Of course it would be foolish to highlight the less original parts of a paper as contributions when this might distract readers from the more original parts, but that's a different issue.

You haven't given enough details to say much about your particular case, but making small modifications to an evaluation framework could be considered a small contribution.