What are the factors one needs to take into account in order to choose an appropriate mathematical journal? Several months ago I submitted a paper to a journal. The area covered by the journal is the same as the one covered in the paper. However, I waited several months and finally the paper was rejected by the journal, and they explained to me that it was not of sufficient interest for them. What can I do to avoid having to go through this again if I submit my paper to another journal?
There is no easy solution to this problem. 

I recently went through a situation where a paper was rejected from a journal without review; the same paper, with only minor edits to the abstract and the introduction, was accepted "as is" for publication in another journal of similar quality and impact factor.

The main issue for getting your work accepted may not be that it isn't of sufficient interest—it may be that the way you have presented it does not convince the editors and referees that the work is suitable. Have you hit on the "hot-button areas" that tend to come up again and again in the journal? Have you made it clear from your abstract and your leadoff paragraphs why this article will be of interest to the target audience of the journal? 

Small differences—these changes don't require much work—can make a huge difference in the acceptance of a paper.