How much is the relevance of a synthetic benchmark? Often researchers verify their proposed algorithm or techniques using standard benchmarks. My question is related to performance of a benchmark application. 

If a researcher propose a new performance optimization or analysis technique optimizing based on a particular factor, is it enough to only use a synthetic benchmark for verification. If instead a standard benchmark application is used its performance may be subjected to multiple factors. It will be difficult to argue that improvement is only due to the particular factor, the researcher optimized for.

Let me give an example. Often a synthetic benchmark is used in synchronization lock related research. Psuedocode of benchmark:

In every thread

For N times // where N is in millions
       //do something in the non-critical section part
       acquire a lock
          // do something in the critical section
       release a lock


Such a benchmark has been used in some notable paper's of the field. But now most of the research use some applications that uses the lock. 

The problem is, it is difficult to individually identify and quantify each factor that affect a benchmark lock-sensitive application performance. Despite the application being lock-sensitive there are a host of other factor that affect the application performance. It would be easier to do for a synthetic benchmark, where a researcher can limit the code.

How convincing an argument in a research paper is if it is solely based on analysis on synthetic benchmarks?
Having just a synthetic benchmark is generally weaker than having a realistic benchmark, but better than nothing. 

There some more considerations to be taken into account.

First, quality considerations: Is the design rationale of the benchmark clearly communicated? Was there any effort made to reflect realistic cases? Was the benchmark designed with input from multiple, independent experts (ideally different authors than the author of the paper, due to researcher bias)? Have independent experts been consulted to validate the benchmark? Is the benchmark publicly available and well-documented?

Second, historical considerations: Has the benchmark recently been used in other research? The more widely a benchmark is used, the better, since it will allow an easy comparison between different works.