<commit_before># Release Management & Docker Images

## GitHub Actions Required Secrets

<!-- markdown-link-check-disable-next-line -->
The following “Repository secret” environment variables must be added to <https://github.com/drud/ddev/settings/secrets/actions>:

* `AUR_SSH_PRIVATE_KEY`: Private ssh key for the `ddev-releaser` user. This must be processed into a single line, for example, `perl -p -e 's/\n/<SPLIT>/' ~/.ssh/id_rsa_ddev_releaser| pbcopy`.
* `CHOCOLATEY_API_KEY`: API key for Chocolatey.
* `DDEV_GITHUB_TOKEN`: GitHub token that gives access to create releases and push to the Homebrew repositories.
* `DDEV_MACOS_APP_PASSWORD`: Password used for notarization, see [signing_tools](https://github.com/drud/signing_tools).
* `DDEV_MACOS_SIGNING_PASSWORD`: Password for the macOS signing key, see [signing_tools](https://github.com/drud/signing_tools).
* `DDEV_WINDOWS_SIGNING_PASSWORD`: Windows signing password.
* `SegmentKey`: Key that enables Segment reporting.
* `FURY_ACCOUNT`: [Gemfury](https://gemfury.com) account that receives package pushes.
* `FURY_TOKEN`: Push token assigned to the above Gemfury account.
* `AUR_STABLE_GIT_URL`: The Git URL for AUR stable (normally `ddev-bin`), for example `ssh://aur@aur.archlinux.org/ddev-bin.git`.
* `AUR_EDGE_GIT_URL`: The Git URL for AUR edge (normally `ddev-edge-bin`), for example `ssh://aur@aur.archlinux.org/ddev-edge-bin.git`.

## Creating a Release

!!!tip "This is completely automated now, so nothing needs to be done unless something goes wrong."

### Prerelease Tasks

* Make sure [`version-history.md`](https://github.com/drud/ddev/blob/master/version-history.md) is up to date.
* Make sure the Docker images are all tagged and pushed.
* Make sure [`pkg/version/version.go`](https://github.com/drud/ddev/blob/master/pkg/version/version.go) is all set to point to the new images and tests have been run.

### Actual Release Creation

1. Create a release for the new version using the GitHub UI. It should be “prerelease” if it’s an edge release.
2. Use the “Auto-generate release notes” option to get the commit list, then edit to add all the other necessary info.
3. Verify that Homebrew (Linux and macOS) and Chocolatey and AUR are working correctly with the right versions.

## Pushing Docker Images with the GitHub Actions Workflow

The easiest way to push Docker images is to use the GitHub Actions workflow, especially if the code for the image is already in the [drud/ddev](https://github.com/drud/ddev) repository.

<!-- markdown-link-check-disable-next-line -->
You can push an image at <https://github.com/drud/ddev/actions/workflows/push-tagged-image.yml>

<!-- markdown-link-check-disable-next-line -->
If you need to push from a forked PR, you’ll have to do this from your fork (for example, `https://github.com/drud/rfay/actions/workflows/push-tagged-image.yml`), and you’ll have to specify the branch on the fork. This requires setting the `DOCKERHUB_TOKEN` and `DOCKERHUB_USERNAME` secrets on the forked PR, for example `https://github.com/rfay/ddev/settings/secrets/actions`.

* Visit `https://github.com/drud/ddev/actions/workflows/push-tagged-image.yml`.
* Click the “Push tagged image” workflow on the left side of the page.
* Click the “Run workflow” button in the blue section above the workflow runs.
* Choose the branch to build from (usually `master`).
* Enter the image (`ddev-webserver`, `ddev-dbserver`, `ddev-php-base`, etc.).
* Enter the tag that will be used in `pkg/version/version.go`.

## Pushing Docker Images Manually

While it’s more error-prone, images can be pushed from the command line:

1. `docker login` with a user that has push privileges.
2. `docker buildx create --name ddev-builder-multi --use` or if it already exists, `docker buildx use ddev-builder-multi`.
3. `cd containers/<image>`.
4. Before pushing `ddev-webserver`, make sure you’ve pushed a version of `ddev-php-base` and updated `ddev-webserver`’s Dockerfile to use that as a base.
5. `make push VERSION=<release_version> DOCKER_ARGS=--no-cache` for most of the images. For `ddev-dbserver` it’s `make PUSH=true VERSION=<release_version> DOCKER_ARGS=--no-cache`. There’s a [push-all.sh](https://github.com/drud/ddev/blob/master/containers/push-all.sh) script to update all of them, but it takes forever.

## Maintaining `ddev-dbserver` MySQL 5.7 & 8.0 ARM64 Images

Sadly, there are no ARM64 Docker images for MySQL 5.7 and 8.0, so we have our own process to maintain [drud/mysql-arm64-images](https://github.com/drud/mysql-arm64-images) and [drud/xtrabackup-build](https://github.com/drud/xtrabackup-build) images for DDEV.

* `drud/mysql:5.7` uses Ubuntu 18.04 as the base image, and Ubuntu 18.04 ARM64 has `mysql-server` 5.7 in it, so we can install.
* `drud/mysql:8.0` uses Ubuntu 20.04 as the base image, and Ubuntu 20.04 ARM64 has `mysql-server` 8.0 in it, so we can install it from packages.
* Unfortunately, the `ddev snapshot` feature depends on `xtrabackup` 8.0 being installed for `mysql:8.0`. There are no ARM64 packages or binaries provided by Percona for `xtrabackup`, so we build it from source with [drud/xtrabackup-build](https://github.com/drud/xtrabackup-build). **There’s a catch, however:** `xtrabackup`’s development cycle lags behind `mysql:8.0`’s development cycle, so you can’t build a usable `drud/mysql:8.0` image until there’s an `xtrabackup` version released. Further, when Ubuntu bumps `mysql-server-8.0` to a new version, there’s no way to use the old one. So the only time that you can maintain `drud/mysql:8.0` is when Ubuntu 20.04 has the same version that’s released for `percona-xtrabackup`. (In the case at this writeup, I was finally able to build `percona-xtrabackup` 8.0.28, and the same day Ubuntu bumped its packages to 8.0.29, meaning that it was unusable.)
* To build percona-xtrabackup, follow the instructions on [drud/xtrabackup-build](https://github.com/drud/xtrabackup-build). You just create a release with the release of Percona xtrabackup, for example `8.0.29-21`. When that succeeds, then there is an upstream xtrabackup to be used in the drud/mysql:8.0 build.
* To build `drud/mysql` (both 5.7 and 8.0) ARM64 images, follow the instructions on [drud/mysql-arm64-images](https://github.com/drud/mysql-arm64-images). After the various files are updated, you can push a new release and the proper images will be pushed.
* After building a new set of `drud/mysql` images, you’ll need to push `drud/ddev-dbserver` with new tags. Make sure to update the [drud/`ddev-dbserver` Makefile](https://github.com/drud/ddev/blob/master/containers/ddev-dbserver/Makefile) to set the explicit version of the upstream `mysql:8.0` (for example, 8.0.29, if you’ve succeed in getting 8.0.29 for `percona-xtrabackup` and `mysql:8.0`).

## Actual Release Docker Image Updates

We don’t actually build every image for every point release. If there have been no changes to `ddev-router` or `ddev-ssh-agent`, for example, we only usually push those and update `pkg/version/version.go` on major releases.

But here are the steps for building:

1. The `drud/ddev-php-base` image must be updated as necessary with a new tag before pushing `ddev-webserver`. You can do this using the [process above](#pushing-docker-images-with-the-github-actions-workflow).
2. The `drud/ddev-webserver` Dockerfile must `FROM drud/ddev-php-base:<tag>` before building/pushing `ddev-webserver`. But then it can be pushed using either the GitHub Actions or the manual technique.
3. If you’re bumping `ddev-dbserver` 8.0 minor release, follow the upstream [Maintaining ddev-dbserver MySQL 5.7 & 8.0 ARM64 Images](#maintaining-ddev-dbserver-mysql-57--80-arm64-images) instructions.
4. Update `pkg/version/version.go` with the correct versions for the new images, and run all the tests.

## Manually Updating Homebrew Formulas

Homebrew formulas normally update with the release process, so nothing needs to be done.

If you have to temporarily update the Homebrew formulas, you can do that with a commit to <https://github.com/drud/homebrew-ddev> and <https://github.com/drud/homebrew-ddev-edge>. The bottles and checksums for macOS (High Sierra) and x86_64_linux are built and pushed to the release page automatically by the release build process (see [bump_homebrew.sh](https://github.com/drud/ddev/blob/master/.ci-scripts/bump_homebrew.sh). Test `brew upgrade ddev` both on macOS and Linux and make sure DDEV is the right version and behaves well.

## Manually Updating Chocolatey

Normally the release process does okay with pushing to Chocolatey, but at times a failure can happen and it’s not worth doing the whole release process again.

* Open up Gitpod, <https://gitpod.io/#https://github.com/drud/ddev> and

```bash
cd /workspace/ddev
sudo apt-get update && sudo apt-get install -y nsis
sudo .ci-scripts/nsis_setup.sh /usr/share/nsis
make chocolatey
cd .gotmp/bin/windows_amd64/chocolatey
```

* Edit the checksum in `tools/chocolateyinstall.ps1` to match the released checksum of the `ddev-windows-installer` (not the choco package; for v1.19.2 this was <https://github.com/drud/ddev/releases/download/v1.19.2/ddev_windows_installer.v1.19.2.exe.sha256.txt>)

```bash
rm .gotmp/bin/windows_amd64/chocolatey/*.nupkg
export CHOCOLATEY_API_KEY=key33333
docker run --rm -v "/$PWD:/tmp/chocolatey" -w "//tmp/chocolatey" linuturk/mono-choco pack ddev.nuspec;
docker run --rm -v $PWD:/tmp/chocolatey -w /tmp/chocolatey linuturk/mono-choco push -s [https://push.chocolatey.org/](https://push.chocolatey.org/) --api-key "${CHOCOLATEY_API_KEY}"

```

## Manually Updating AUR Repository

The AUR repository normally updates with the release process, so nothing needs to be done.

However, you can manually publish the release to [the DDEV AUR repository](https://aur.archlinux.org/packages/ddev-bin/). The README.md in the AUR git repo (`ssh://aur@aur.archlinux.org/ddev-bin.git` or `https://aur.archlinux.org/ddev-bin.git`) has instructions on how to update, including how to do it with a Docker container, so it doesn’t have to be done on an ArchLinux or Manjaro VM.

## Manually Signing the Windows Installer

!!!tip "This is done by the release process, but the manual process is documented here."

This is done automatically by the release build on a dedicated Windows test runner (GitHub Actions runner) named `testbot-asus-win10pro`. You would need to do this process manually on that build machine or install the fob on another machine.

**After rebooting this machine, sometimes an automated reboot, the password for the security fob has to be re-entered or Windows signing will fail. We do this by opening up `tb-win11-06` using Chrome Remote Desktop (or manually physically opening it), opening Git Bash, and `cd ~/tmp && signtool sign gsudo.exe`. There happens to be a `gsudo.exe` there but it doesn’t matter what you sign—the idea is to pop up the GUI where you enter the password (which is in LastPass).**

### Basic Instructions

1. Install the suggested [Windows SDK](https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/). Only the signing component is required.
2. Add the path of the kit binaries to the Windows system PATH, `C:/Program Files (x86)/Windows Kits/10/bin/10.0.22621.0/x64/`.
3. The keyfob and Safenet Authentication Client must be installed. The best documentation for the Safenet software is at <https://support.globalsign.com/ssl/ssl-certificates-installation/safenet-drivers>. You must configure the advanced client settings to “Enable single logon” or it will require the password on each run.
4. After `make windows_install` the `ddev-windows-installer.exe` will be in `.ddev/bin/windows_amd64/ddev_windows_installer.exe` and you can sign it with `signtool sign ddev-windows-installer.exe`.
5. If you need to install the GitHub self-hosted Windows runner, do it with the instructions in project settings → Actions → Runners.
6. Currently the `actions/cache` runner does not work out of the box on Windows, so you have to install tar and zstd as described in [this issue](https://github.com/actions/cache/issues/580#issuecomment-1165839728).

!!!tip "We shouldn’t use this high-security keyfob approach to signing on the next go-around with the certs."
    It’s way too difficult to manage, and the Safenet software is atrocious.
<commit_msg>[docs] Remove inappropriate link from chocolatey deploy (#4316)<commit_after># Release Management & Docker Images

## GitHub Actions Required Secrets

<!-- markdown-link-check-disable-next-line -->
The following “Repository secret” environment variables must be added to <https://github.com/drud/ddev/settings/secrets/actions>:

* `AUR_SSH_PRIVATE_KEY`: Private ssh key for the `ddev-releaser` user. This must be processed into a single line, for example, `perl -p -e 's/\n/<SPLIT>/' ~/.ssh/id_rsa_ddev_releaser| pbcopy`.
* `CHOCOLATEY_API_KEY`: API key for Chocolatey.
* `DDEV_GITHUB_TOKEN`: GitHub token that gives access to create releases and push to the Homebrew repositories.
* `DDEV_MACOS_APP_PASSWORD`: Password used for notarization, see [signing_tools](https://github.com/drud/signing_tools).
* `DDEV_MACOS_SIGNING_PASSWORD`: Password for the macOS signing key, see [signing_tools](https://github.com/drud/signing_tools).
* `DDEV_WINDOWS_SIGNING_PASSWORD`: Windows signing password.
* `SegmentKey`: Key that enables Segment reporting.
* `FURY_ACCOUNT`: [Gemfury](https://gemfury.com) account that receives package pushes.
* `FURY_TOKEN`: Push token assigned to the above Gemfury account.
* `AUR_STABLE_GIT_URL`: The Git URL for AUR stable (normally `ddev-bin`), for example `ssh://aur@aur.archlinux.org/ddev-bin.git`.
* `AUR_EDGE_GIT_URL`: The Git URL for AUR edge (normally `ddev-edge-bin`), for example `ssh://aur@aur.archlinux.org/ddev-edge-bin.git`.

## Creating a Release

!!!tip "This is completely automated now, so nothing needs to be done unless something goes wrong."

### Prerelease Tasks

* Make sure [`version-history.md`](https://github.com/drud/ddev/blob/master/version-history.md) is up to date.
* Make sure the Docker images are all tagged and pushed.
* Make sure [`pkg/version/version.go`](https://github.com/drud/ddev/blob/master/pkg/version/version.go) is all set to point to the new images and tests have been run.

### Actual Release Creation

1. Create a release for the new version using the GitHub UI. It should be “prerelease” if it’s an edge release.
2. Use the “Auto-generate release notes” option to get the commit list, then edit to add all the other necessary info.
3. Verify that Homebrew (Linux and macOS) and Chocolatey and AUR are working correctly with the right versions.

## Pushing Docker Images with the GitHub Actions Workflow

The easiest way to push Docker images is to use the GitHub Actions workflow, especially if the code for the image is already in the [drud/ddev](https://github.com/drud/ddev) repository.

<!-- markdown-link-check-disable-next-line -->
You can push an image at <https://github.com/drud/ddev/actions/workflows/push-tagged-image.yml>

<!-- markdown-link-check-disable-next-line -->
If you need to push from a forked PR, you’ll have to do this from your fork (for example, `https://github.com/drud/rfay/actions/workflows/push-tagged-image.yml`), and you’ll have to specify the branch on the fork. This requires setting the `DOCKERHUB_TOKEN` and `DOCKERHUB_USERNAME` secrets on the forked PR, for example `https://github.com/rfay/ddev/settings/secrets/actions`.

* Visit `https://github.com/drud/ddev/actions/workflows/push-tagged-image.yml`.
* Click the “Push tagged image” workflow on the left side of the page.
* Click the “Run workflow” button in the blue section above the workflow runs.
* Choose the branch to build from (usually `master`).
* Enter the image (`ddev-webserver`, `ddev-dbserver`, `ddev-php-base`, etc.).
* Enter the tag that will be used in `pkg/version/version.go`.

## Pushing Docker Images Manually

While it’s more error-prone, images can be pushed from the command line:

1. `docker login` with a user that has push privileges.
2. `docker buildx create --name ddev-builder-multi --use` or if it already exists, `docker buildx use ddev-builder-multi`.
3. `cd containers/<image>`.
4. Before pushing `ddev-webserver`, make sure you’ve pushed a version of `ddev-php-base` and updated `ddev-webserver`’s Dockerfile to use that as a base.
5. `make push VERSION=<release_version> DOCKER_ARGS=--no-cache` for most of the images. For `ddev-dbserver` it’s `make PUSH=true VERSION=<release_version> DOCKER_ARGS=--no-cache`. There’s a [push-all.sh](https://github.com/drud/ddev/blob/master/containers/push-all.sh) script to update all of them, but it takes forever.

## Maintaining `ddev-dbserver` MySQL 5.7 & 8.0 ARM64 Images

Sadly, there are no ARM64 Docker images for MySQL 5.7 and 8.0, so we have our own process to maintain [drud/mysql-arm64-images](https://github.com/drud/mysql-arm64-images) and [drud/xtrabackup-build](https://github.com/drud/xtrabackup-build) images for DDEV.

* `drud/mysql:5.7` uses Ubuntu 18.04 as the base image, and Ubuntu 18.04 ARM64 has `mysql-server` 5.7 in it, so we can install.
* `drud/mysql:8.0` uses Ubuntu 20.04 as the base image, and Ubuntu 20.04 ARM64 has `mysql-server` 8.0 in it, so we can install it from packages.
* Unfortunately, the `ddev snapshot` feature depends on `xtrabackup` 8.0 being installed for `mysql:8.0`. There are no ARM64 packages or binaries provided by Percona for `xtrabackup`, so we build it from source with [drud/xtrabackup-build](https://github.com/drud/xtrabackup-build). **There’s a catch, however:** `xtrabackup`’s development cycle lags behind `mysql:8.0`’s development cycle, so you can’t build a usable `drud/mysql:8.0` image until there’s an `xtrabackup` version released. Further, when Ubuntu bumps `mysql-server-8.0` to a new version, there’s no way to use the old one. So the only time that you can maintain `drud/mysql:8.0` is when Ubuntu 20.04 has the same version that’s released for `percona-xtrabackup`. (In the case at this writeup, I was finally able to build `percona-xtrabackup` 8.0.28, and the same day Ubuntu bumped its packages to 8.0.29, meaning that it was unusable.)
* To build percona-xtrabackup, follow the instructions on [drud/xtrabackup-build](https://github.com/drud/xtrabackup-build). You just create a release with the release of Percona xtrabackup, for example `8.0.29-21`. When that succeeds, then there is an upstream xtrabackup to be used in the drud/mysql:8.0 build.
* To build `drud/mysql` (both 5.7 and 8.0) ARM64 images, follow the instructions on [drud/mysql-arm64-images](https://github.com/drud/mysql-arm64-images). After the various files are updated, you can push a new release and the proper images will be pushed.
* After building a new set of `drud/mysql` images, you’ll need to push `drud/ddev-dbserver` with new tags. Make sure to update the [drud/`ddev-dbserver` Makefile](https://github.com/drud/ddev/blob/master/containers/ddev-dbserver/Makefile) to set the explicit version of the upstream `mysql:8.0` (for example, 8.0.29, if you’ve succeed in getting 8.0.29 for `percona-xtrabackup` and `mysql:8.0`).

## Actual Release Docker Image Updates

We don’t actually build every image for every point release. If there have been no changes to `ddev-router` or `ddev-ssh-agent`, for example, we only usually push those and update `pkg/version/version.go` on major releases.

But here are the steps for building:

1. The `drud/ddev-php-base` image must be updated as necessary with a new tag before pushing `ddev-webserver`. You can do this using the [process above](#pushing-docker-images-with-the-github-actions-workflow).
2. The `drud/ddev-webserver` Dockerfile must `FROM drud/ddev-php-base:<tag>` before building/pushing `ddev-webserver`. But then it can be pushed using either the GitHub Actions or the manual technique.
3. If you’re bumping `ddev-dbserver` 8.0 minor release, follow the upstream [Maintaining ddev-dbserver MySQL 5.7 & 8.0 ARM64 Images](#maintaining-ddev-dbserver-mysql-57--80-arm64-images) instructions.
4. Update `pkg/version/version.go` with the correct versions for the new images, and run all the tests.

## Manually Updating Homebrew Formulas

Homebrew formulas normally update with the release process, so nothing needs to be done.

If you have to temporarily update the Homebrew formulas, you can do that with a commit to <https://github.com/drud/homebrew-ddev> and <https://github.com/drud/homebrew-ddev-edge>. The bottles and checksums for macOS (High Sierra) and x86_64_linux are built and pushed to the release page automatically by the release build process (see [bump_homebrew.sh](https://github.com/drud/ddev/blob/master/.ci-scripts/bump_homebrew.sh). Test `brew upgrade ddev` both on macOS and Linux and make sure DDEV is the right version and behaves well.

## Manually Updating Chocolatey

Normally the release process does okay with pushing to Chocolatey, but at times a failure can happen and it’s not worth doing the whole release process again.

* Open up Gitpod, <https://gitpod.io/#https://github.com/drud/ddev> and

```bash
cd /workspace/ddev
sudo apt-get update && sudo apt-get install -y nsis
sudo .ci-scripts/nsis_setup.sh /usr/share/nsis
make chocolatey
cd .gotmp/bin/windows_amd64/chocolatey
```

* Edit the checksum in `tools/chocolateyinstall.ps1` to match the released checksum of the `ddev-windows-installer` (not the choco package; for v1.19.2 this was <https://github.com/drud/ddev/releases/download/v1.19.2/ddev_windows_installer.v1.19.2.exe.sha256.txt>)

```bash
rm .gotmp/bin/windows_amd64/chocolatey/*.nupkg
export CHOCOLATEY_API_KEY=key33333
docker run --rm -v "/$PWD:/tmp/chocolatey" -w "//tmp/chocolatey" linuturk/mono-choco pack ddev.nuspec;
docker run --rm -v $PWD:/tmp/chocolatey -w /tmp/chocolatey linuturk/mono-choco push -s https://push.chocolatey.org/ --api-key "${CHOCOLATEY_API_KEY}"

```

## Manually Updating AUR Repository

The AUR repository normally updates with the release process, so nothing needs to be done.

However, you can manually publish the release to [the DDEV AUR repository](https://aur.archlinux.org/packages/ddev-bin/). The README.md in the AUR git repo (`ssh://aur@aur.archlinux.org/ddev-bin.git` or `https://aur.archlinux.org/ddev-bin.git`) has instructions on how to update, including how to do it with a Docker container, so it doesn’t have to be done on an ArchLinux or Manjaro VM.

## Manually Signing the Windows Installer

!!!tip "This is done by the release process, but the manual process is documented here."

This is done automatically by the release build on a dedicated Windows test runner (GitHub Actions runner) named `testbot-asus-win10pro`. You would need to do this process manually on that build machine or install the fob on another machine.

**After rebooting this machine, sometimes an automated reboot, the password for the security fob has to be re-entered or Windows signing will fail. We do this by opening up `tb-win11-06` using Chrome Remote Desktop (or manually physically opening it), opening Git Bash, and `cd ~/tmp && signtool sign gsudo.exe`. There happens to be a `gsudo.exe` there but it doesn’t matter what you sign—the idea is to pop up the GUI where you enter the password (which is in LastPass).**

### Basic Instructions

1. Install the suggested [Windows SDK](https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/). Only the signing component is required.
2. Add the path of the kit binaries to the Windows system PATH, `C:/Program Files (x86)/Windows Kits/10/bin/10.0.22621.0/x64/`.
3. The keyfob and Safenet Authentication Client must be installed. The best documentation for the Safenet software is at <https://support.globalsign.com/ssl/ssl-certificates-installation/safenet-drivers>. You must configure the advanced client settings to “Enable single logon” or it will require the password on each run.
4. After `make windows_install` the `ddev-windows-installer.exe` will be in `.ddev/bin/windows_amd64/ddev_windows_installer.exe` and you can sign it with `signtool sign ddev-windows-installer.exe`.
5. If you need to install the GitHub self-hosted Windows runner, do it with the instructions in project settings → Actions → Runners.
6. Currently the `actions/cache` runner does not work out of the box on Windows, so you have to install tar and zstd as described in [this issue](https://github.com/actions/cache/issues/580#issuecomment-1165839728).

!!!tip "We shouldn’t use this high-security keyfob approach to signing on the next go-around with the certs."
    It’s way too difficult to manage, and the Safenet software is atrocious.

{'fraction_non_alphanumeric': 0.10318880663792403, 'fraction_numerical': 0.016188074513951028, 'mean_word_length': 4.469855394883203, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 61, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 6, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2880934', 'n_tokens_mistral': 8075, 'n_tokens_neox': 7566, 'n_words': 3001}