I have a command text file that issues a data file download command on each    line.   I send the command file to bash.  However, a small percentage of the downloads fail.  Here is the algorithm I use to find out what's missing:After downloading, I go back through the command file and check if each download file exists.If the download doesn't exist, I copy the command line into a new command file.I am left with a new command file for the remaining downloads.Here is the bash script I implemented the algorithm with:  1 #!/bin/bash  2 while read line  3 do  4         for item in $line  5         do  6                 if [[ $item == *out_fname* ]]; then   7                         splitline=(${item//=/ })     8                         target_file=${splitline[1]}  9                         if [ ! -f $target_file ]; then 10                                 echo $line >> stillneed.txt 11                         fi 12                 fi 13         done 14 done < $@Question:  This works well, but is there a better algorithm or implementation (maybe using something other than bash)?  What I did was just have bash do what a human would have to do.  But it seems Unix always has a better way of doing things...
Checking for the existence of files against a list
bash;files
It looks like you are looking for 'out_fname=', not just 'out_fname'.I'd either do this in a mix of awk and shell or in python. In awk/shell:awk '{for(i=0;i<NF;i++) {if (index($i, 'out_fname=')) {split($i,A,/=/);print A[i]}}' $@ |    while read filename; do        if [ ! -f $filename ]; then echo $filename; fi    done > stillneed.txtIn python:import fileinput, osstillneed = open(stillneed.txt, w)for line in fileinput.input():    for filename in [l.split('=')[1] for l in line.split() if l.find('out_fname=')!=-1]:        if not os.path.exists(filename):            print >>stillneed, filename