I am running my below shell script from machineA which is copying the files machineB and machineC into machineA. If the files are not there in machineB, then it should be there in machineC.The below shell script will copy the files into TEST1 and TEST2 directory in machineA..#!/bin/bashset -ereadonly TEST1=/data01/test1readonly TEST2=/data02/test2readonly SERVER_LOCATION=(machineB machineC)readonly FILE_LOCATION=/data/snapshotdir1=$(ssh -o StrictHostKeyChecking no david@${SERVER_LOCATION[0]} ls -dt1 $FILE_LOCATION/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -n1)dir2=$(ssh -o StrictHostKeyChecking no david@${SERVER_LOCATION[1]} ls -dt1 $FILE_LOCATION/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -n1)echo $dir1echo $dir2if [ $dir1 = $dir2 ]then    rm -rf $TEST1/*    rm -rf $TEST2/*    for el in $test1_partition    do        scp david@${SERVER_LOCATION[0]}:$dir1/pp_monthly_9800_$el_200003_5.data $TEST1/. || scp david@${SERVER_LOCATION[1]}:$dir2/pp_monthly_9800_$el_200003_5.data $TEST1/.    done    for sl in $test2_partition    do            scp david@${SERVER_LOCATION[0]}:$dir1/pp_monthly_9800_$sl_200003_5.data $TEST2/. || scp david@${SERVER_LOCATION[1]}:$dir2/pp_monthly_9800_$sl_200003_5.data $TEST2/.    donefiIs there a way to run process parallelly in the loop of a bash scriptCurrently it copies the file from machineB and machineC into machineA TEST1 directory first, and if it is done, then only it will go and copy the files from machineB and machineC into machineA TEST2 directory.. Is there any way I transfer the files both in TEST1 and TEST2 directory simultaneously?I am running Ubuntu 12.04
How to parallelize the for loop while scp the files?
linux;ubuntu;ssh;scp;parallel
In addition to sending them to the background, use the wait built in to wait for all background processes to finish before continuing.for el in $test1_partitiondo    (scp david@${SERVER_LOCATION[0]}:$dir1/pp_monthly_9800_$el_200003_5.data $TEST1/. || scp david@${SERVER_LOCATION[1]}:$dir2/pp_monthly_9800_$el_200003_5.data $TEST1/.) &    WAITPID=$WAITPID $!donefor sl in $test2_partitiondo        (scp david@${SERVER_LOCATION[0]}:$dir1/pp_monthly_9800_$sl_200003_5.data $TEST2/. || scp david@${SERVER_LOCATION[1]}:$dir2/pp_monthly_9800_$sl_200003_5.data $TEST2/.) &    WAITPID=$WAITPID $!donewait $WAITPIDecho All files done copying.