Four - he questioned their relationship often. It wasn’t about loyalty or lack of love, but about deserving. He was convinced he didn’t deserve Hermione, but she passionately disagreed and threatened to murder him herself if he ever tried to leave her in hopes that she’d be "better off without him". Not to mention he had a daughter on the way, and he didn’t want to be anything like his father had been. Detached and apathetic was one thing, absent was an entirely different thing altogether.