How to tell a faculty member that I don't need him on my thesis committee anymore? Out of five needed members for my doctoral thesis committee, four are fixed. For the fifth member, I approached Mr.A. He has no real connection to my research or topic but I contacted him because he was just as good a fit as anybody else in the department to be the fifth member. I have never worked with him or taken any classes but we did talk a lot in the first few years where he would just ask about my progress by the copy machine, etc. We don't even have a "rapport". The relationship is absolutely neutral, nothing positive/friendly, and nothing negative. I asked him to be on my committee and he said yes. I said I'll get back to you with dates, etc. He is a senior tenured professor.

Then about a month later, I was looking at the CV of some of the new hired (junior) faculty in the department Mr.B (this is his first year) and I see that we have a collaborator in common. Mr.B has published multiple papers with him and Mr.B is working with the same code and the code managing team that we are working with. He is definitely a good person to have on my committee. I finally see Mr.B after a few more months (because of busy schedule/travel/etc.) and he agrees to be on my committee.

The question is, how can I tell Mr.A, as tactfully as possible, after about four months of being quiet that he is basically rejected from my committee because I didn't know about Mr.B but now that I know, Mr.B is a much better match than he is? That I don't need him but I do appreciate his offer? I don't want to antagonize Mr.A or burn any bridges. Should this be in email or in person? Should I explain or just keep it terse? Should I keep it vague that I don't need him and thanks for his time? Or should I tell him about Mr.B and why he is a better match?

FYI, this was before anything official/paperwork was done so there is no problem with that.
I think you're making a bigger deal about this than necessary.  You enlisted Professor A to serve on your committee because you needed a fifth man, not because of any close research connection.  More recently, but still before signing any paperwork, you found Professor B, who does have a close research connection.

Just tell Professor A that!  Problem solved.  You don't have to dither about how to present this information: send it in an email or send it in person.  The information of the previous paragraph is sufficient: you don't need to dwell on your connection with Professor A as much as you did in your question.  (But by the way: asking about your progress at the copy machine is what I would call "friendly".  At least he knows who you are and something about your progress in the program, and he cares enough to ask about it sometimes.  That puts him ahead of the curve in many academic departments at many universities.)  

What you do not seem to realize is that if Professor A has no close connection with your work, it is overwhelmingly likely that he was being friendly indeed by volunteering to serve on your committee anyway, and he'll be equally happy or, more probably, a little happier not to serve.  Since he is a senior tenured professor he has lots of stuff to do.  Serving as the fifth man on a student's committee is not nearly such a prestige job that his layoff needs to be sugarcoated.  He'll understand, and he'll be especially happy that someone with more relevant expertise will be taking his place.