<commit_before># gtfc
git for cluster sync, naive synchronization (pull only) over ssh every minute (with crontab)

# How it works

Using cron to schedule a task that: in a git repo, for each git remote, perform a git pull.

# Properties

Property                                   | Description
-------------------------------------------|------------------
Centralized aspect                         | Only the first node with unique/latest data will be central until another completes a pull.
Decentralized/distributed storage          | Kind of. When all nodes are up to date, there is no central store because git repositories are mirrors. Akin to RAID1 mirror, but not to RAID5 span.
Decentralized/distributed transfers        | Kind of. Nodes can pull from any other nodes that are ahead.
Chunked data transfers from multiple hosts | No.
Optimal use                                | Good for small files or text.
Limitations                                | Bad for large files or binary. Sync based on git commits, not files.

# Quick Usage

Assuming you can ssh forward

/etc/hosts
```
192.168.0.2 alpha
192.168.0.3 betasaurus
192.168.0.4 caviar
192.168.0.5 delta
8.8.x.x public
```

Example session from deployment machine that can ssh into all hosts:

```
ssh    urep@alpha      gtfc init  alpha      gitsync

ssh    urep@alpha      gtfc add   betasaurus gitsync
ssh -A urep@betasaurus gtfc clone alpha      gitsync
ssh    urep@alpha      sh gitsync/gitpullall.sh -f

ssh    urep@alpha      gtfc add   caviar gitsync
ssh -A urep@caviar     gtfc clone alpha  gitsync
ssh    urep@alpha      sh gitsync/gitpullall.sh -f

ssh    urep@alpha      gtfc add   delta gitsync
ssh -A urep@delta      gtfc clone alpha gitsync<commit_msg>-A takes care of add<commit_after># gtfc
git for cluster sync, naive synchronization (pull only) over ssh every minute (with crontab)

# How it works

Using cron to schedule a task that: in a git repo, for each git remote, perform a git pull.

# Properties

Property                                   | Description
-------------------------------------------|------------------
Centralized aspect                         | Only the first node with unique/latest data will be central until another completes a pull.
Decentralized/distributed storage          | Kind of. When all nodes are up to date, there is no central store because git repositories are mirrors. Akin to RAID1 mirror, but not to RAID5 span.
Decentralized/distributed transfers        | Kind of. Nodes can pull from any other nodes that are ahead.
Chunked data transfers from multiple hosts | No.
Optimal use                                | Good for small files or text.
Limitations                                | Bad for large files or binary. Sync based on git commits, not files.

# Quick Usage

Assuming you can ssh forward

/etc/hosts
```
192.168.0.2 alpha
192.168.0.3 betasaurus
192.168.0.4 caviar
192.168.0.5 delta
8.8.x.x public
```

Example session from deployment machine that can ssh into all hosts:

```
ssh    urep@alpha      gtfc init  alpha      gitsync

ssh -A urep@betasaurus gtfc clone alpha      gitsync
ssh    urep@alpha      sh gitsync/gitpullall.sh -f

ssh -A urep@caviar     gtfc clone alpha  gitsync
ssh    urep@alpha      sh gitsync/gitpullall.sh -f

ssh -A urep@delta      gtfc clone alpha gitsync
{'fraction_non_alphanumeric': 0.0871132472213878, 'fraction_numerical': 0.021628116551516974, 'mean_word_length': 2.135593220338983, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 8, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '277894', 'n_tokens_mistral': 1021, 'n_tokens_neox': 903, 'n_words': 415}