<commit_before>3rd-Party Dependencies in Fubsy
===============================

Fubsy depends on:

  * Go (the gc compiler and associated tools)
  * Kyoto Cabinet (a database library)
  * some third-party tools and libraries, all written in Go

Of these, the following must be installed on your system in order to
build Fubsy:

  * Go
  * Kyoto Cabinet

The other dependencies are included directly in Fubsy's source control
repository, e.g. golex is in src/github.com/cznic/golex.

Describing dependencies
-----------------------

$BASE is the "package" path that specifies the upstream source
repository. $PKG is the path to actual buildable Go code. These are
often are the same, e.g. ::

    BASE=github.com/cznic/golex
    PKG=github.com/cznic/golex

But sometimes they are different::

    BASE=code.google.com/p/go-bit
    PKG=code.google.com/p/go-bit/bit

Adding a new dependency
-----------------------

To add a new third-party dependency ``$PKG``::

    export GOPATH=$PWD
    go get -v $BASE
    go install -v $PKG
    go test $PKG

One or both of these steps will fail if the package is not compatible
with ``go get``. Once that's good, commit the new dependency:

    rm -rf `find src/$BASE -name .hg -o -name .git -o -name .bzr`
    hg add src/$BASE
    hg commit -m"add new dependency $BASE"

Modify some Fubsy code to use the new package, and make sure the unit
tests touch that code. Then commit that change:

    hg commit -m"<topic>: use $BASE to make everything spiffy"

It's OK to fold those two commits together if the change to Fubsy code
is small and well contained.

Updating a dependency
---------------------

The process is quite similar::

    export GOPATH=$PWD
    rm -rf src/$BASE
    go get -v $BASE
    go install -v $PKG
    go test $PKG

Make sure Fubsy still builds and passes its tests::

    ./build.sh

If any code changes are required, make them now. Commit everything
together:

    rm -rf `find src/$BASE -name .hg -o -name .git -o -name .bzr`
    hg commit -m"update to latest version of $BASE" src/$BASE

Bonus points for including the upstream commit ID/changeset
ID/revision number in the commit message.

Dependencies that come in a closely related cluster -- like golex and
friends -- should be added/updated en masse.

Modifying a dependency
----------------------

Do NOT modify any of Fubsy's dependencies in place. Instead, fork the
upstream repository, make your required changes, and send them a
patch/pull request/whatever. If upstream is unresponsive or refuses
the change, and you absolutely cannot find another way to do the same
thing, then change Fubsy to depend on your fork instead. That means 1)
rename the existing dependency to match your repository location, and
2) follow the "update" procedure.<commit_msg>docs: describe how to get upstream rev number when updating dependencies<commit_after>3rd-Party Dependencies in Fubsy
===============================

Fubsy depends on:

  * Go (the gc compiler and associated tools)
  * Kyoto Cabinet (a database library)
  * some third-party tools and libraries, all written in Go

Of these, the following must be installed on your system in order to
build Fubsy:

  * Go
  * Kyoto Cabinet

The other dependencies are included directly in Fubsy's source control
repository, e.g. golex is in src/github.com/cznic/golex.

Describing dependencies
-----------------------

$BASE is the "package" path that specifies the upstream source
repository. $PKG is the path to actual buildable Go code. These are
often are the same, e.g. ::

    BASE=github.com/cznic/golex
    PKG=github.com/cznic/golex

But sometimes they are different::

    BASE=code.google.com/p/go-bit
    PKG=code.google.com/p/go-bit/bit

Adding a new dependency
-----------------------

To add a new third-party dependency ``$PKG``::

    export GOPATH=$PWD
    go get -v $BASE
    go install -v $PKG
    go test $PKG

One or both of these steps will fail if the package is not compatible
with ``go get``. Once that's good, commit the new dependency:

    rm -rf `find src/$BASE -name .hg -o -name .git -o -name .bzr`
    hg add src/$BASE
    hg commit -m"add new dependency $BASE"

Modify some Fubsy code to use the new package, and make sure the unit
tests touch that code. Then commit that change:

    hg commit -m"<topic>: use $BASE to make everything spiffy"

It's OK to fold those two commits together if the change to Fubsy code
is small and well contained.

Updating a dependency
---------------------

The process is quite similar::

    export GOPATH=$PWD
    rm -rf src/$BASE
    go get -v $BASE
    go install -v $PKG
    go test $PKG

Make sure Fubsy still builds and passes its tests::

    ./build.sh

If any code changes are required, make them now. Commit everything
together:

    cd src/$BASE
    if git:
        rev=`git log -n1 --pretty=oneline | awk '{print $1}'`
    else if hg:
        rev=`hg parents --template '{node|short}'`

    rm -rf `find . -name .hg -o -name .git -o -name .bzr`
    hg addremove -s 75 .
    hg commit -m"update to latest version of $BASE (upstream rev $rev)" .

Dependencies that come in a closely related cluster -- like golex and
friends -- should be added/updated en masse.

Modifying a dependency
----------------------

Do NOT modify any of Fubsy's dependencies in place. Instead, fork the
upstream repository, make your required changes, and send them a
patch/pull request/whatever. If upstream is unresponsive or refuses
the change, and you absolutely cannot find another way to do the same
thing, then change Fubsy to depend on your fork instead. That means 1)
rename the existing dependency to match your repository location, and
2) follow the "update" procedure.
{'fraction_non_alphanumeric': 0.11025280898876405, 'fraction_numerical': 0.0017556179775280898, 'mean_word_length': 2.9262577532736045, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1637493', 'n_tokens_mistral': 1707, 'n_tokens_neox': 1604, 'n_words': 802}