Why should the population be lower if we can support more people? The resources a human actually needs compared to the available resources of our solar system let alone galaxy are, for all intensive purposes, infinite. The issue is our systems are grossly inefficient and as you said cause environmental damage because of that inefficiency