Does an engineering paper have to improve on the state of the art quantitatively in order to be publishable? In most engineering papers, I see that the authors justify their research by displaying how well it performs against the state-of-the-art methods or algorithms, in a quantitative sense. To put it loosely, this is usually stated along the lines of "Our method is faster/better/cheaper than existing methods, and here are the charts and graphs to prove it".

However, what about papers that are putting forward a more visionary approach that would take time to justify using quantitative means? For example, it is conceivable that there is an idea that performs moderately well, but needs work by the entire community in order to reach its potential. It is also conceivable that this idea is a more fruitful one than existing lines of research.

Does a piece of research have to beat others in a measurable way in order to be publishable, or is there also a qualitative, holistic way that papers are evaluated?
In order to be publishable, a scientific work needs to provide evidence that it advances the state of human knowledge.

Direct quantitative comparison to the state of the art is a relatively straightforward way of doing this, but there are many other alternatives.  For example, one might also provide mathematical proof of a difference in expected scaling or demonstrate a qualitative property that is not present in current systems.

The key in every case, however, is that there should be clear evidence provided of some sort of value that other investigators are likely to find to be of interest.