  Petunia hated his mother so much that she beat her nephew as pay back. He was confused by that, after all Lily had left Harry as well but he still got beaten for it. Shouldn't they be working together, shouldn't they both hate Lily together, rant about her. She ruined both their lives after all. Then again Petunia and Vernon would beat him just because of his magic.