He prayed for Hermione's safety and cared not otherwise. He didn't mean that; he'd never mean it. But he was very upset with Hagrid. And he didn't think it an unjustified anger. But he refused to fight to justify himself right now. So he chose not to care about anything else. 