<commit_before>pyzfsautosnap
=============

ZFS Automatic Snapshots for Linux and Solaris

This is a Python implementation of ZFS autosnap which was written for
OpenSolaris.

TODO
----
* Add snapshot syncronization as client/server
** use mbuffer for the sync stuff, SSH wrapper or something for setup of stream

mbuffer
-------

http://everycity.co.uk/alasdair/2010/07/using-mbuffer-to-speed-up-slow-zfs-send-zfs-receive/

The receiver does a giant read per TXG. Between these reads, the standard TCP
network buffers fill and flow control off. When the disks and the network are
similar bandwidths, you will see alternate network, then disk activity. By
having a buffer on the receive side which can hold 5 seconds worth of data at
the rate the disk and network can stream, you will then see the dataflow stream
continuously. I didn’t see any significant benefit of a buffer at the sender,
but a small one can’t do any harm.

If you are using mbuffer on a local replication it’s both tedious, complicated
and unnecessary to use two copies running on the same machine. Try this syntax:

    zfs send ... | mbuffer -s 128k -m 2G -o - | zfs receive ...

the “-o -” puts mbuffer into file output mode, but then sends to stdout
instead. Fancy!<commit_msg>Add paramiko notes<commit_after>pyzfsautosnap
=============

ZFS Automatic Snapshots for Linux and Solaris

This is a Python implementation of ZFS autosnap which was written for
OpenSolaris.

TODO
----
* Add snapshot syncronization as client/server
** use mbuffer for the sync stuff, SSH wrapper or something for setup of stream

mbuffer
-------

http://everycity.co.uk/alasdair/2010/07/using-mbuffer-to-speed-up-slow-zfs-send-zfs-receive/

The receiver does a giant read per TXG. Between these reads, the standard TCP
network buffers fill and flow control off. When the disks and the network are
similar bandwidths, you will see alternate network, then disk activity. By
having a buffer on the receive side which can hold 5 seconds worth of data at
the rate the disk and network can stream, you will then see the dataflow stream
continuously. I didn’t see any significant benefit of a buffer at the sender,
but a small one can’t do any harm.

If you are using mbuffer on a local replication it’s both tedious, complicated
and unnecessary to use two copies running on the same machine. Try this syntax:

    zfs send ... | mbuffer -s 128k -m 2G -o - | zfs receive ...

the “-o -” puts mbuffer into file output mode, but then sends to stdout
instead. Fancy!

paramiko
--------

http://hackerific.net/2009/02/06/paramiko-scripting-ssh-with-python/
http://www.paramiko.org/
{'fraction_non_alphanumeric': 0.07668828691339183, 'fraction_numerical': 0.011446012972148036, 'mean_word_length': 3.75, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '273458', 'n_tokens_mistral': 790, 'n_tokens_neox': 729, 'n_words': 376}