If I think I have a publishable idea on a certain problem, how do I publish and proceed? I'm new to research.  I'm working on a certain problem that has sort of been abandoned by all researchers (as far as publications go).  Anyway, I like it for its simplicity and easiness to grasp.  However, it's a new algorithm for a problem and a new way of possibly answering theoretical questions about the problem.  

There is so little research out there on the certain problem, that it would literally take you a day or less to view it all and ascertain that indeed the idea hasn't been looked at yet.  I'm applying the well-studied field of integer programming / linear algebra to model each instance of the problem.  

Anyhow, regardless of whether I answer a major open problem, I think the approach is valuable in itself, if for no other reason that it makes answering basic questions about the problem a lot more easy, sort of like how algebraic geometry answers questions about geometric objects using algebra.  

How do I proceed?  Do I just spend a month perfecting the paper on my own and put it on arXiv?  Is that all?
You are on the cusp between math and CS, actually. The better way to "publish" your work is to write it up formally, with references/citations and everything and submit it to either a journal or a conference. Choose one that matches the content of your paper. 

But also make sure it is a reputable journal and not one that just publishes anything for the money you pay them.

However, you also have to be prepared, with journals, for publication fees, even with reputable ones. Sometimes they may waive the fees, but you may need to find a way to pay them. And, for conferences, make sure you have some way of attending if your paper is accepted for presentation. In CS, conferences are usually preferred. In math, usually journals.