I want to create a large test file with lines containg dates listed by the second, but my method is taking inordinately long... (or at least, that's how it feels :) ... 43 minutes to create only 1051201 lines. 20.1 MB file.... I want to crate a much bigger file, with each line's date being unique..Is there a faster way than how I've approached it?:    # # BEGIN CREATE TEST DATA  ============ # # Create some dummy data.  file=/tmp/$USER/junk  ((secY2 =s3600*24*365*2))  cnt=0  secBeg=$(date --date=2010-01-01 00:00:00 +%s)  secEnd=$((secBeg+secY2))  ((sec=secBeg))  while ((sec<=secEnd)) ; do      date -d '1970-01-01 UTC '$sec' seconds' '+%Y-%m-%d %H:%M:%S' >>$file       ((sec+=1))      ((cnt+=1))  done  ls -l $file  echo Lines written: $cnt# END CREATE TEST DATA  ============
Fast way to build a test file with every second listed in YYYY-mm-dd HH:MM:SS format
bash;text processing;performance;date
I haven't made any benchmark, but I see a few potential improvements.You open and close the file for each call to date. This is a waste: just put the redirection around the whole loop.while ; do ; done >$fileYou're making separate calls to date for each line. Unix is good at calling external programs quickly, but internal is still better. GNU date has a batch option: feed it dates on standard input, and it pretty-prints them. Furthermore, to enumerate a range of integers, use seq, it's likely to be faster than interpreting the loop in the shell.seq -f @%12.0f $secBeg $secEnd | date -f - '+%Y-%m-%d %H:%M:%S' >$filecnt=$(($secY2 + 1))Generally speaking, if your shell script is too slow, try to have the inner loop executed in a dedicated utility  here seq and date, but often sed or awk. If you can't manage that, switch to a more advanced scripting language such as Perl or Python (but the dedicated utilities are typically faster, if you fit their use cases).