His parents don't expect much from him, so he had no idea why he felt so close to Albus without even knowing him. It's painfully obvious that Albus had an abusive childhood, and at first Avery mistook his affection for an expression of pity - one of the reasons he didn't let Albus bite him away.