Trying to finish a paper with a professor, but also needing to let them know I switched advisors I need a bit of advice. I am currently in my third year of a math PhD program.

I have been working with Prof. X for about 2.5 years now. He thinks he is my advisor but I have also been working with Prof. Z who is my chosen advisor for many reasons. In short, Prof. Z a much better advisor for me (nicer to me, better at helping me learn, more available, supportive for me to attend summer research programs, etc.) and I want to work in the research area of Z and not X. I know that I will be successful with Prof Z.

Last semester I was traveling most of the time doing research related to Prof. X's work. In that time, I solidified that X's area is not the area I want to study. I am trying to finish a paper with Prof. X though. 

I need to tell Prof. X that I want to finish the paper but then work with only Prof. Z. 

I am thinking that telling Prof. X in person is best; however, I am also worried that Prof. X will have a bad reaction (due to his personality). Maybe it is better for me to tell X over email so that the initial reaction doesn't happen with me there. 

Any advice would be really helpful on how I should tell him.

Thanks!

Update: Hey all. Just as an update. I told him today. I thought he would be mad but it was more of a sad reaction. I felt bad, but I know I’m making the right choice. Thank you all for your time and help with this :)
Having been in a similar situation before, I can say this: Prof. X may not want to finish a paper with you if you're changing advisors, and Prof. Z may not want you to continue spending time on unrelated research.  Does Prof. Z even know of your intentions?  You should make sure Z actually wants to be your advisor first.  Does Prof. X know you've already been working with Z?  He will probably feel betrayed, and there's nothing you can do to avoid that.  You will have to have in-person conversations with both professors, maybe even both at the same time, if the two are on friendly terms.  (If they aren't on friendly terms, then things will get more complicated, and you may need a mediator.)

In any case, I would not explain the situation via e-mail, but I would e-mail to schedule an appointment and say that you need to discuss something important.  This will allow Prof. X to mentally prepare and also ensure that everyone has set aside time to focus on the issue.  I would spend some time writing out what you want to say, maybe even bring the notes with you, but also anticipate that you may need to have multiple conversations.  If Prof. X is notoriously belligerent, see if you can arrange for a mediator to be present.  Finally, try to think of what you really want out the interaction-- working on a project with someone you've just told you don't want to work with can be quite miserable for both parties, and it may be better to cut your losses and move on.