It didn't mean he was not enjoying them. This was the first Christmas holiday he had ever enjoyed. Normally when he was back at the Dursleys, Christmas tended to be the time. Harry was busiest with housework since in Vernon's eyes since he wasn't at school, he should be using his free time to better himself. Harry thought that the betterment Vernon was suggesting was just the betterment of his cooking skills.