Voldemort had said that, the man who had murdered Harry's parents who had murder many people, a mad man, the most powerful Dark Lord Britain had seen in a generation and here Harry was, agreeing with him. Voldemort, Harry agreed with him. The idea filled him with dread, it's horrible, he's horrible he's agreeing with that monster.