I am looking for credible references stating how much resources supercomputers spend on coordinating versus doing actual task-related work. Resources could be available processing power but even Watts seem like a valid unit.I believe one of my professors or text books once said that in massively parallel systems, up to half of the available processing power is spent on coordinating the task and message passing. Unfortunately, I can't seem to find this reference or any other material about this proportion. I realize this will differ a lot depending on the supercomputer architecture and modern implementations probably are more efficient in this regard, so an overview of this metric across multiple architectures or evolutions (before and after dedicated message passing hardware) would be even better.
Communication overhead in supercomputing
hpc;mpi