With his mother dead and father rotting in prison, there was nothing tying him to the wizarding world. Taking up with Granger would give him an opportunity to free himself from the constraints of a polite society that he no longer fit into. 