Correct!  Were he an honorable person, he would have recused himself regardless if there is an actual conflict or even if only the "appearance" of a conflict.  I know, he won't get to sit at the big-boy table but, that's the way it goes, or that's the way it should go.  Speaking of going... have you left yet?