Though he is powerfully drawn to the woman, he does not know how to fight for her or even that he IS to fight for her.
He finds her mostly a mystery that he knows he cannot solve and so at a soul level he keeps his distance.