While it makes no sense to worry that you are living well at your grandchildren's expense, you might legitimately worry that someone else is living well at your grandchildren's expense.
They have children of their own.