You make the assumption that people change their vote based on their age. There's another possibility: if you remember 'the good old days' before the UK joined the EU in 1973 you might be more likely to vote for exit. In that case, it doesn't matter if the older demographic groups are growing faster than younger ones.