Careless uninterested potential advisor A professor, call X, who is with strong probability about to be my future MS Thesis supervisor/advisor, seems very interested to work with me. Mostly on lectures, X seems interested in my ideas and is very encouraging.

However, out of the class, X is totally unreachable, uninterested in my problems. e.g. very slow to respond questions, once I needed an important paperwork to be done and X totally slacked it, although a ceratin mouthful "sure I will do it" promise. I had really hardtime to fix related issue with much frustations, because of X's attitude. And X did not give me any apologies for this situation. 

I understand that profs are busy beings, managing tons of stuff at the same time, but I expect not to break a given promise or not to give promise at all to do something. I am really fed up and tired of this repeating behavior.    

I have started to think that, it is not a good idea to work with X. I seriously doubt that professor would be reachable in my thesis study, not to mention to be helpful in my future academic pursuits, such as a strong letter for a Ph.D. application.

Am I expecting a lot ? Also, I want to kindly inform X that, I do not want to involve any future studies with X. What kind of wording should I use ?
Professors are busy and are always juggling multiple commitments.

This does not mean that you are unimportant to them but maybe some judicious memory jogging is required for them to remember what X has promised you.

Another excellent strategy which I have adopted is to show up to my committee members' open office hours with issues when I need them solved instead of bothering them over email. Usually I need 5-10 minutes of their time to explain my issue and solicit future directions. That works really well. I am not a big fan of long meetings.

In response to your specific question, if you want to not work with X just tell X that you do not feel that you can work with them in the future. It might be helpful to have someone else lined up before you do so though.