He is not saying that some votes are not counted. He is saying that they do not count toward electing someone. There are people who have voted in elections for 50 years, and never cast a ballot for someone who became a representative.

All ballots are counted. Many do not count toward representation.