<commit_before>Bigitr
======

Bigitr is a tool that copies content bidirectionally between Git
and CVS repositories in an asynchronous and asymmetric fashion.
(You can configure only CVS->Git, only Git->CVS, or bidirectional
copying; separately for each repository you are synchronizing.)
The name *Bigitr* is intended to convey at least bidirectionality,
an emphasis on Git over CVS, and that it helps to *beget* Git
repositories from CVS repositories.

Unlike Git's cvsimport, Bigitr does not attempt to assemble
changesets out of CVS into separate Git commits.  It merely takes
snapshots of the current (at the time of the snapshot) state of CVS,
using the `cvs export` command.  Similarly, it does not export Git
commits separately into CVS; it writes only the current (at the time)
latest content from the specified Git branch into CVS.

The flow is conceptually similar to Git's flow for synchronizing
with a remote repository.  For example, in Git, the local `master`
branch is pushed to the `master` branch on the `origin` repository,
pulled from the `origin` remote's `master` branch to the local
`origin/master` reference, and then the local `origin/master`
reference is merged onto the local `master` branch.  Similarly, in
Bigitr, the Git `master` branch might be exported to a CVS branch
named `current-dev`, which is then imported into a Git branch named
`cvs-current-dev`, which then might be merged back into the Git
`master` branch.  A remote `export-master` branch tracks what has
been exported from the `master` branch to CVS.

![dataflow](doc/dataflow.svg)

The bidirectional synchronization is not intended to allow developers
to develop in their choice of CVS or Git indefinitely.  Bigitr should
be used in order to move primary development of a branch from CVS
to Git.  After that, the purpose of merging changes from CVS into Git
is to preserve the content of changes that have to be made in CVS,
without preserving context (changeset or commit messages).

Bigitr is a power tool, and like all power tools, it is dangerous.
It commits to CVS and pushes to remote Git repositories.
Wrong configuration or bugs in Bigitr can be very destructive to
your source code, both in CVS and in Git.  Do backups, and test
configuration initially with throwaway copies of CVS and Git
repositories, before you put this tool into production use.


Reserved Branch Names
---------------------

Branches in the Git repositories with names starting with `cvs-`
and `export-` are reserved to be written only by Bigitr.
These branches are pushed to the origin repository so that local
state does not need to be maintained.  This means that multiple
users can run this tool using the same repository configuration
but different application configuration (local directories).
However, no locking is attempted, so if two users run it at the
same time, the results are undefined.


Bigitr imports branches from CVS into special import branches (with
names starting with "cvs-") in Git.  It also exports branches in
Git into branches in CVS.  It will not touch the CVS trunk (at
least at this time).

Note: It will not synchronize only parts of a CVS `sub/module`.  It is
all or nothing: for any particular `sub/module`, any files not in
the Git branch being exported to CVS will be **deleted** from that
branch in that CVS `sub/module`.

It records which content has been pushed from a Git branch into
CVS by using another branch on the origin with a name that
starts with `export-`.  If you export the Git `master` branch
to CVS, Bigitr will track what has been exported using the
`export-master` branch.  It uses this to attach Git commit
messages to CVS commits.  It makes no attempt to attach CVS
commit messages to Git commits.


Merging
-------

After committing to branches (not including fast-forwarding the
export- branches), Bigitr will automatically attempt to merge changes
into other explicitly-specified branches.  If any merges are
successful, those merges themselves may trigger other merge
operations.

It is reasonable to configure no automatic merging at all.  Be,
aware that if you have configured the Git "master" branch to be
exported to an existing CVS branch, then you should populate the
Git "master" branch before exporting.

Any branch that you export from Git to CVS, including master,
must be populated before you export it.  You may populated it
by merging from a branch imported from CVS, either manually or
by configuring an automatic merge.

Files in CVS ignored by .gitignore will not be committed to the
corresponding Git branches, and if those .gitignore changes are
represented on a branch being exported from Git into CVS, those
files will be deleted from the CVS branch when the Git branch
is exported to CVS.

Files with names starting with ".git" will not be commited to CVS;
that file namespace is reserved for Git, even for Git metadata file
types that have not yet been created as of the time that this tool
was written.

Bigitr tries to notice entirely empty branches and should refuse to
delete entire CVS `sub/modules`.  An attempt to export an empty Git
branch to a CVS branch will raise an exception rather than deleting
all the files on the CVS branch.  However, even a single file is
enough to override this security check.)  (If you really want to
delete all the files on the CVS branch, do it using CVS directly.)

It is recommended that only Git metadata files with names starting
with .git be included in "skeletons" used to populate empty
branches, because the protections against accidentally exporting
empty branches to CVS apply only to Git metadata files.  Putting
any other files in a skeleton may cause you to delete a CVS
`sub/module` due to misconfiguring a repository in Bigitr.


Hooks
-----

There are pre-commit and post-commit hooks for both Git and CVS,
in order to implement normalization rules, notification, and so
forth.  These commands are run with the current working directory
set to the Git working directory or CVS checkout.  These can be
specified for all repositories or for a single repository, and
for all branches or only individual branches.  Any arguments are
included in the configuration using shell syntax; no extra
arguments are added by Bigitr at run time.  Hooks that exit with
an error cause the current Bigitr repository operation to halt
immediately, though additional repositories will still be processed.


Configuration
-------------

There are two kinds of configuration files.

* Application configuration: general behavior

* Repository configuration: specific to a Git repository

Both of these kinds of configuration files use "ini" syntax.
This is intentionally similar to Git configuration.  Any syntax
differences are due to using the Python ConfigFile parser to
implement the configuration file handling.

These files support using `#` characters for comments.  It is normal
to *comment out* configuration lines that do not currently apply
but might apply later.

The values in these files can include environment variables,
which must be specified using the `${VARNAME}` syntax.

In addition, there are skeleton directories containing initial
files for populating new branches, hooks that are called before
and after commits, and working directories in which CVS and
Git operations are done and log files are stored.

### Application configuration ###

A bigitr configuration file (by default, the file named by
the `BIGITR_APP_CONFIG` environment variable, or `~/.bigitr`
if it is not set, or provided via the `--appconfig` option)
has three sections:<commit_msg>add daemon configuration to list of configuration file types<commit_after>Bigitr
======

Bigitr is a tool that copies content bidirectionally between Git
and CVS repositories in an asynchronous and asymmetric fashion.
(You can configure only CVS->Git, only Git->CVS, or bidirectional
copying; separately for each repository you are synchronizing.)
The name *Bigitr* is intended to convey at least bidirectionality,
an emphasis on Git over CVS, and that it helps to *beget* Git
repositories from CVS repositories.

Unlike Git's cvsimport, Bigitr does not attempt to assemble
changesets out of CVS into separate Git commits.  It merely takes
snapshots of the current (at the time of the snapshot) state of CVS,
using the `cvs export` command.  Similarly, it does not export Git
commits separately into CVS; it writes only the current (at the time)
latest content from the specified Git branch into CVS.

The flow is conceptually similar to Git's flow for synchronizing
with a remote repository.  For example, in Git, the local `master`
branch is pushed to the `master` branch on the `origin` repository,
pulled from the `origin` remote's `master` branch to the local
`origin/master` reference, and then the local `origin/master`
reference is merged onto the local `master` branch.  Similarly, in
Bigitr, the Git `master` branch might be exported to a CVS branch
named `current-dev`, which is then imported into a Git branch named
`cvs-current-dev`, which then might be merged back into the Git
`master` branch.  A remote `export-master` branch tracks what has
been exported from the `master` branch to CVS.

![dataflow](doc/dataflow.svg)

The bidirectional synchronization is not intended to allow developers
to develop in their choice of CVS or Git indefinitely.  Bigitr should
be used in order to move primary development of a branch from CVS
to Git.  After that, the purpose of merging changes from CVS into Git
is to preserve the content of changes that have to be made in CVS,
without preserving context (changeset or commit messages).

Bigitr is a power tool, and like all power tools, it is dangerous.
It commits to CVS and pushes to remote Git repositories.
Wrong configuration or bugs in Bigitr can be very destructive to
your source code, both in CVS and in Git.  Do backups, and test
configuration initially with throwaway copies of CVS and Git
repositories, before you put this tool into production use.


Reserved Branch Names
---------------------

Branches in the Git repositories with names starting with `cvs-`
and `export-` are reserved to be written only by Bigitr.
These branches are pushed to the origin repository so that local
state does not need to be maintained.  This means that multiple
users can run this tool using the same repository configuration
but different application configuration (local directories).
However, no locking is attempted, so if two users run it at the
same time, the results are undefined.


Bigitr imports branches from CVS into special import branches (with
names starting with "cvs-") in Git.  It also exports branches in
Git into branches in CVS.  It will not touch the CVS trunk (at
least at this time).

Note: It will not synchronize only parts of a CVS `sub/module`.  It is
all or nothing: for any particular `sub/module`, any files not in
the Git branch being exported to CVS will be **deleted** from that
branch in that CVS `sub/module`.

It records which content has been pushed from a Git branch into
CVS by using another branch on the origin with a name that
starts with `export-`.  If you export the Git `master` branch
to CVS, Bigitr will track what has been exported using the
`export-master` branch.  It uses this to attach Git commit
messages to CVS commits.  It makes no attempt to attach CVS
commit messages to Git commits.


Merging
-------

After committing to branches (not including fast-forwarding the
export- branches), Bigitr will automatically attempt to merge changes
into other explicitly-specified branches.  If any merges are
successful, those merges themselves may trigger other merge
operations.

It is reasonable to configure no automatic merging at all.  Be,
aware that if you have configured the Git "master" branch to be
exported to an existing CVS branch, then you should populate the
Git "master" branch before exporting.

Any branch that you export from Git to CVS, including master,
must be populated before you export it.  You may populated it
by merging from a branch imported from CVS, either manually or
by configuring an automatic merge.

Files in CVS ignored by .gitignore will not be committed to the
corresponding Git branches, and if those .gitignore changes are
represented on a branch being exported from Git into CVS, those
files will be deleted from the CVS branch when the Git branch
is exported to CVS.

Files with names starting with ".git" will not be commited to CVS;
that file namespace is reserved for Git, even for Git metadata file
types that have not yet been created as of the time that this tool
was written.

Bigitr tries to notice entirely empty branches and should refuse to
delete entire CVS `sub/modules`.  An attempt to export an empty Git
branch to a CVS branch will raise an exception rather than deleting
all the files on the CVS branch.  However, even a single file is
enough to override this security check.)  (If you really want to
delete all the files on the CVS branch, do it using CVS directly.)

It is recommended that only Git metadata files with names starting
with .git be included in "skeletons" used to populate empty
branches, because the protections against accidentally exporting
empty branches to CVS apply only to Git metadata files.  Putting
any other files in a skeleton may cause you to delete a CVS
`sub/module` due to misconfiguring a repository in Bigitr.


Hooks
-----

There are pre-commit and post-commit hooks for both Git and CVS,
in order to implement normalization rules, notification, and so
forth.  These commands are run with the current working directory
set to the Git working directory or CVS checkout.  These can be
specified for all repositories or for a single repository, and
for all branches or only individual branches.  Any arguments are
included in the configuration using shell syntax; no extra
arguments are added by Bigitr at run time.  Hooks that exit with
an error cause the current Bigitr repository operation to halt
immediately, though additional repositories will still be processed.


Configuration
-------------

There are three kinds of bigitr configuration files.

* Application configuration: general behavior

* Repository configuration: specific to a Git repository

* Daemon configuration: behavior of the bigitrd daemon

Each of these kinds of configuration files use "ini" syntax.
This is intentionally similar to Git configuration.  Any syntax
differences are due to using the Python ConfigFile parser to
implement the configuration file handling.

These files support using `#` characters for comments.  It is normal
to *comment out* configuration lines that do not currently apply
but might apply later.

The values in these files can include environment variables,
which must be specified using the `${VARNAME}` syntax.

In addition, there are skeleton directories containing initial
files for populating new branches, hooks that are called before
and after commits, and working directories in which CVS and
Git operations are done and log files are stored.

### Application configuration ###

A bigitr configuration file (by default, the file named by
the `BIGITR_APP_CONFIG` environment variable, or `~/.bigitr`
if it is not set, or provided via the `--appconfig` option)
has three sections:
{'fraction_non_alphanumeric': 0.043654955427372834, 'fraction_numerical': 0.0, 'mean_word_length': 4.389261744966443, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6377314', 'n_tokens_mistral': 3794, 'n_tokens_neox': 3675, 'n_words': 2409}