Registering for a math conference with no publication record in this field I’m pretty sure that I’ve solved a certain open problem in mathematics. My solution is currently in its third week of peer-review at some journal that is known for rejecting cranky papers within five days of submission. 

I’m currently searching for the next relevant conferences where I can present my paper. The thing is I’m not yet a famous person in the math community, hence it might not be easy to convince the conference organisers. Can someone kindly advice me on how I can go about it?
I’ll assume the open problem you solved is in pure mathematics. What you should know is that in most areas of pure mathematics, conferences do not have a formal paper submission mechanism, nor do they publish a proceedings volume as they typically do in other areas. One does not go to a conference to “present a paper” but rather one will get invited to “give a talk”, whose contents may be somewhat correlated with the contents of one’s most recent paper, but could also be about a less recent paper or about a still unfinished work, or some combination of such things; that’s entirely up to the speaker.

My suggestion is that you try to find a conference in an area of math related to the open problem.  Once you find one, you’ll have the option to register to attend (not to give a talk, which is by invitation only). Except for very large events, there is usually no registration fee, and although they may ask about your affiliation, it’s likely that most conferences will not use that as a basis for deciding whether to let you register, so don’t worry too much if you don’t work at a university.

Now, as I said in order to give a talk you’ll need to be invited to speak by the organizers. They usually select speakers with a proven record of publishing interesting results in the field of the conference. But one thing you could do is email them in parallel with registering, tell them about your paper with the solution of the open problem and ask if you could give a talk about it. I don’t think your chances are too high, but it’s worth a try. Your chances will be higher if your paper is available to download online from a place like arXiv, and if it’s written in a professional way that makes it clear it comes from a mathematician with a good level of experience and formal training. If you don’t want to make your paper publicly available just yet, you can send it to the organizers directly, but it will likely be viewed with suspicion and lower your chances of being invited to speak. (And if you don’t make the paper viewable at all, no one will take you seriously and the chances are essentially zero.)

Even if you are not invited to give a talk at the conference you found, it may still be a good idea to attend it, since there you will have a chance to talk to experts informally, tell them about your solution and get some valuable feedback about your approach. 

Good luck!