Should I publish a linear programming method despite my advisor's advice? TL;DR: I developed a way of solving systems of equations which is up to 25,000x faster than base methods. I want to publish but my advisor doesn't think it's worth it.



I work as a data scientist with hyperspectral data. In this area, we treat every single pixel as its own system of equations. Because of this, we need to solve potentially thousands of systems of equations. Obviously, if we use traditional tools this is exceptionally slow and potentially not worth our time. However, each problem is relatively small, think 1 variable with ~100 non-linear equations. 

I came up with 2 separate systems to solve these equations much faster. The first one adapts MATLAB's fminbnd so that every single one of these problems can be solved at once. This allows the function evaluation to take advantage of vectorized processing which in many languages, particularly research based languages gives a drastic speedup. See the MATLAB codebase. In addition, this allowed us to use GPUs to solve the problem resulting in a much larger performance increase. All told, this allows us to have a ~300-1000x performance speedup.

The second system is meant for solving much larger systems of equations. This uses the same idea but with a Levenburg-Marquardt implementation. This utilizes MATLAB's pagefun and with an aggressive implementation, this beats out the best global solver I could find by a factor of 25,000 for the same results (this method also allowed me to obtain an order of magnitude improvement for my results with a 4000x speedup).

While these results are impressive, my advisor wants me to focus on my primary research and wants me to present those results. I'd like to present both results as I believe they are both valuable to the body of knowledge but my advisor has always been honest with me and I feel like I should follow his lead.

Should I publish this linear programming method despite my advisor's advice?
It is good to have publications. It is also good to follow the advice of your advisor for many reasons. Advisors have some say over your future, of course, so caution is indicated, at least, in every case. 

Note first that it has been many years since I was current with the algorithms of integer and linear programming. I suspect you are more current and your advisor more so, so be a bit careful in analyzing what I say below. 

The field in general is quite mature and the existing algorithms are very sophisticated. This was true 30 year ago, in fact. If the field has settled down since then there is little urgency in publishing now, as it is unlikely that you would be beaten to publication if you wait. You can make your advisor happy and, indeed, make progress on your main work without abandoning what you have. 

However, I think that a real advance should be questioned and you should be the one to question it. In particular, it may be that the reason for such good results is that your specific problem has characteristics that make this possible and your algorithm is nicely tailored to a special case. The possibility of that actually opens a door for your future work. If it is true, then knowing and describing that special case would be very useful, along with the algorithm. 

Also, if it is, indeed, a special case and you have tested only with the confines of this case then your results may not apply generally. It would be good to know, before you publish, whether this is true or not. And it would be especially good for you if you are the one to discover this. Fortunately, you can test your algorithm on a wider class of problems found in the literature. Thus, you can, perhaps, rule out the situation in which you have an anomaly. But your primary work should likely come first.