It is not the fact that Avery knows such things that puzzles Scorpius, but his particular choice of words. The carefree tone. Is that the way a person who just picked up their information somewhere would talk about these relationships?