I have some experience with Inix terminals from science internships I've been part of, mainly using a few utilities like grep, awk, and sed but there is one thing I've been trying to figure out for a while that would really make me far more efficient with the number crunching I have to do. I have a script run.awk that performs some manipulations on a large of collection of massive text files. As it is it will take the file chloride.out, extract data from it and write chloride.cm . Is there anyway I can make this script take in *.out and write *.cm files based off the initial wildcard phrase in shell?The amount of scripts I've written for processing large amounts of data that I've had to do a hundred+ iterations of is just annoying.Ideally I would like to know if there is a way that I can do this for all my scripts with something via the shell. If it can't be automated in the shell or an equivalent can I least automate my awk scripts in a similar manner as I've described?
Way to write new filename on wildcard input?
shell;awk;filenames;wildcards
You most certainly can make awk deal with multiple files via wildcards. One suggestion would be to leave the run.awk as a generic function that takes a single file in and produces a single output file, and then call it from another script which could then take care of assimilating the input and output files.ExampleThis would be a Bash script, we can call it, awk_runner.bash.#!/bin/bashfor ifname in *.out; do   ofname=${ifname/.out/.cm}  printf IN: %s, OUT: %s\n $ifname $ofname  printf running run.awk with %s & %s\n\n $ifname $ofname  run.awk $ifname $ofnamedoneSample runI made a example directory with some test files in it.$ touch file{1..4}.outThis resulted in 4 files being made:$ ls -1file1.outfile2.outfile3.outfile4.outNow we run our script:$ ./awk_runner.bashIN: file1.out, OUT: file1.cmrunning run.awk with file1.out & file1.cmIN: file2.out, OUT: file2.cmrunning run.awk with file2.out & file2.cmIN: file3.out, OUT: file3.cmrunning run.awk with file3.out & file3.cmIN: file4.out, OUT: file4.cmrunning run.awk with file4.out & file4.cmAfter each line that starts with, running... our script could run from here.Files in a listSay instead of using the wildcard, *.out we instead had a file with a list of filenames in it, say:$ cat filelist.txt file1.outfile2.outfile3.outfile4.outWe could use this modified version of our script which would use a while loop instead of a for loop. Now let's call this variant of the script, awk_file_runner.bash:#!/bin/bashwhile read ifname; do   ofname=${ifname/.out/.cm}  printf IN: %s, OUT: %s\n $ifname $ofname  printf running run.awk with %s & %s\n\n $ifname $ofname  run.awk $ifname $ofnamedone < filelist.txtThis version of the script reads the input from the file, filelist.txt:done < filelist.txtThen for each turn of the while loop, we're using the read command to read in a line from the input file.while read ifname; doIt then performs everything in the same way as the first script where it will run the awk script run.awk as it loops through each line of the file.