I am trying to work with awk     datetime=`date +\%Y\%m\%d`    logdatetime=`date +\%Y\%m\%d`    foldermonth=`date +\%B_\%Y`    folderday=`date +\%d`    inputdir=~/sboper/Standalone/fsplit/GLMS    outputdir=~/sboper/StandAlone/input/$foldermonth  outputdirday=~/sboper/StandAlone/input/$foldermonth/GLMS_Daily/$folderday    awk -v outdir=$outputdirday 'BEGIN{ FS = ~ } ;{if( $12 == A ) filename=outdir/Customer_Create_Records.dat ; print >> filename;close(filename)}' $inputdir/$sourcefile    awk -v outdir=$outputdirday 'BEGIN{ FS = ~ } ;{if( $12 == M ) filename=outdir/Customer_Modify_Records.dat ; print >> filename;close(filename)}' $inputdir/$sourcefilehowever the variable outdir=$outputdirday is not expanding to the value of path which is my output directory.I have declared the awk variable and did not use shell variable inside awk script. But its still giving me error: cannot open Customer_Create_Records.dat.Where am i going wrong with this script? When i copy complete path instead of 'outdir', it created new files and moved the record but doesnt work right when used in a variable. How can i create new files in the directory i want through awk?
Creating a new file inside awk but in different directory based on input field value
awk;variable
Wow. Where do I begin?You need to learn how to debug. How to debug a bash script?is a highly upvoted Stack Exchange reference on the topic,with contributions from several of our most prolific members.You can find other information on this site, and lots more on the web.For instance,Simplify.Get rid of folderday, foldermonthand the `date` assignments;just construct outputdirday with constant values.Avoid confusingly similar variable names.For example, outdir is equal to outputdirday,but outputdir is something different.Eliminate datetime, logdatetime, and outputdir,which are not even used in the script.Eliminate redundancy.You show two awk commands that are identicalexcept for a trivial difference.Does one of them work and one of them fail?Do they fail in different ways?I dont think so.Then dont show them both.Dont use long, eight-level pathnames(e.g., ~/sboper/StandAlone/input/$foldermonth/GLMS_Daily/$folderday/Customer_Create_Records.dat)unless you really really need to.For presentation purposes(i.e., when presenting your problem to others),avoid confusing naming conventions,like naming your output directory /input/.For presentation purposes,try to avoid lines that are wider than the screen(so people have to scroll to read them in their entirety).Most systems have ways to let you split commands and programsinto multiple lines.Do you really have two directories one called Standalone and one called StandAlone in your ~/sboper directory?If the end result isnt what you want,find out whats happening in the middle of the process.Display intermediate values. For example,outputdirday.When I run your script and do echo$outputdirday before the awk, Iget~/sboper/StandAlone/input/June_2016/GLMS_Daily/08I.e., because the ~ was in quotes,it didnt get expanded to my home directory.I had to change the assignment tooutputdirday=$HOME/sboper/StandAlone/input/$foldermonth/GLMS_Daily/$folderdayto get it to work.outdir.You say it isnt get set correctly, but have you tried to print it?When I do, I get the same value as I do for outputdirday.filename.This is where (part of) the problem is; see below.When youre asking for help, you shouldSimplify the problem, as discussed above.Describe what your command is supposed to be doing.Show what your input file looks like.Show what you expect your output to look like.Identify the versions of the software you are using.While I believe that I understand whats going wrong with your script,I cant reproduce your errors exactly(so, I guess Im running a different version of awk).It probably isnt really important,but you dont even say what operating system youre on.Heres the important part:If, as I suggested above,you had broken the main action block in your awk scriptdown into shorter lines, you might have realized that it was equivalent to{      if ( $12 == A )            filename=outdir/Customer_Create_Records.dat      print >> filename      close(filename)}As Michael Vehrs pointed out in his answer,this sets filename conditionally,and then writes every input line to filename unconditionally.As he suggests, you need to do something like{      if ( $12 == A ) {            filename=outdir/Customer_Create_Records.dat            print >> filename            close(filename)      }}If, as I suggested above,you had printed the value of filename after setting it,you would have seen that it wasoutdir/Customer_Create_Records.datbecause outdir (in quotes)means the string outdir,and not the value of the variable outdir.You need to dofilename = outdir /Customer_Create_Records.datAlmost as important:You should always quote your shell variable references(e.g., $outputdirday, $inputdir, and $sourcefile)unless you have a good reason not to,and youre sure you know what youre doing.That doesnt appear to be the cause of this problem,but eventually it will get you into trouble.Just for clarity, you might want to change `` to $() see this, this, and this.While you should always quote your shell variable referencesunless you have a good reason not to,the same does not apply to backslashes.You should use them only when you have a reason to.For example, date+%Y%m%d and date+%B_%Y, etc., work just fine.As I suggested earlier, for what you appear to be doing,you dont need two awk invocations.You can do the whole thing withawk -v outdir=$outputdirday '    BEGIN { FS = ~            filenameA = outdir /Customer_Create_Records.dat            filenameM = outdir /Customer_Modify_Records.dat          }          {            if ($12 == A) print >> filenameA            if ($12 == M) print >> filenameM          }    ' $inputdir/$sourcefileorawk -v outdir=$outputdirday '    BEGIN { FS = ~            filenameA = outdir /Customer_Create_Records.dat            filenameM = outdir /Customer_Modify_Records.dat          }    $12 == A { print >> filenameA}    $12 == M { print >> filenameM}    '    $inputdir/$sourcefileAs far as I can tell, you dont need to close the file(s) at all;files are typically closed automatically when a program exits.