I'm copying large files over LAN, and I've noticed that with doing one ssh pipe, I can get around 10MB/s transfer speed. However, if I shoot off four of those at the same time, I get around 40MB/s transfer speeds in total. So I'd like to split up a large file (eg. 1GB) into parts, and pipe each part on its own, then reassemble on other side. Need to tar/zip before transfer also, and untar/unzip on other side.How can I do this?
How to tar, split, pipe, untar over ssh
networking;ssh;tar
Ended up with HPN-SSHand pigz.tar -cf - -C [RELATIVEFOLDER] [FILENAME] | pigz | ssh px pigz -d | tar xf - -C [REMOTEFOLDER]Improvement by power of ten.For reference, installing HPN-SSH and pigz on Ubuntu 14.04 is easy as:# hpn-patched ssh from ppasudo apt-get install python-software-propertiessudo add-apt-repository ppa:w-rouesnel/openssh-hpnsudo apt-get update -y  sudo apt-get install openssh-serverssh -V # should have 'hpn' in it somewhere# pigzsudo apt-get install pigz