I know we can concat  files by cat file [file] [[file] ...] > joined-file. I have directory which contains lots of files. I want to concat all of them. I want to concat 1000 files irrespective of their name and order. This what I have triedfor i in /var/abc/*.csv; do $i > file1.csv ; rm -rf $i;donebut to keep track of count another variable. What can be efficient method? So that I cant directly concat 1000 files and move them.EditWhy 1000 because the directory contains lakhs of files. So size of file matters.I have tried this with your answeres.cd /var/abc for file in $(ls -p | grep -v / | tail -1000);do cat $file >>/var/abcd/xigzag$tick.csv && rm -rf $file;done
Concat 1000 files within a directory
bash;shell;sed;ls;cat
You don't need to loop, you can tell cat to read all the files:cat /var/abc/*.csv > file1.csv && rm /var/abc/*.csvas long as there aren't too many files (but the limit is huge).Using && between the two commands ensures the files are only deleted if they were successfully copied.There are a couple of caveats though:you mustn't run this in the same folder as the original files you're concatenating, otherwise the rm will delete the agregate and you'll lose everything;if new CSV files appear between the start of the cat and the expansion of rm's arguments, they'll be deleted without being copied.To concatenate files 1000 at a time (so one resulting CSV per 1000 original CSV), with any number of files you'd proceed as follows, in the target directory:find /var/abc -maxdepth 1 -type f -name \*.csv | split -d -l 1000 - csvlistsfor file in csvlists*; do cat $(cat $file) > concat${file##csvlists}.csv && rm $(cat $file); doneThis will find all the files in /var/abc named *.csv, and list them 1000 at a time in files starting with csvlists (csvlists00, csvlists01...). Then the for loop reads each file list and concatenates the listed CSV files in a file named concat00.csv etc. to match the list. Once each set of files is copied, the original files are deleted.This version assumes that the CSV files' names don't contain spaces, newlines and so on.