he remembered the woman who crept back to her family , leaving him to deal with the merchants and face the neighbors , leaving him with that soul-withered husk of a man who spent his days rereading beloved books and never going beyond the garden gate , leaving him to share in his father 's shame for no other reason than because he was male .