I am running some parallel programs on a heterogeneous cluster: 2 CentOS virtual nodes (node 1 & node 2) on a Mac host and 2 others (node 3 & node 4) on a PC. When I run a parallel program (which consists of searching a specific value in a big interval & waiting till all processes have finished to print the result & total execution time), I noticed that processes from node 1 and node 2 always finish their loop before nodes 3 & 4 and wait for them for too long (doing nothing) to finally display the result. I would like to know if there's any way to make the processes from nodes 2 & 1 to go help the other processes (3 & 4) by re-dividing their loops to accelerate the parallel program, instead of waiting doing nothing. Thanks a lot for your help.
Improving MPI performances on a heterogeneous cluster
performance;cluster;parallelism;mpi