<commit_before>About this repo
===============

The cli-perf repo contains tools to monitor the performance of the [dotnet/cli
repo](https://github.com/dotnet/cli).

Setup Steps
===========

0. Install Visual Studio 2015 (required for the CLI build; the Community Edition
   is good enough)

1. Create a base directory (example here; you can put it anywhere)

    ```batch
    mkdir C:\DotNet
    cd /d C:\DotNet
    ```

2. Clone this repo

    ```batch
    git clone https://github.com/dpodder/cli-perf.git
    ```

3. Install chocolatey (must run elevated):

    ```batch
    @powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin
    ```

4. Install required tools (must run elevated):

    ```batch
    choco install git -y
    choco install python2 -y
    choco install microsoft-build-tools -y
    choco install cmake.portable -y
    setx /m PATH "%PATH%;C:\Program Files (x86)\MSBuild\14.0\Bin"
    ```

5. Launch the rolling script (assuming `C:\DotNet` is the base directory):

    ```batch
    cd /d C:\DotNet\cli-perf\rolling-perf
    py rolling-perf.py --branch rel/1.0.0 --dir C:\DotNet\working-dir
    ```<commit_msg>README fixes - reorder setup steps to fix dependency issues - add python3 as well as python2 - add newline at EOF<commit_after>About this repo
===============

The cli-perf repo contains tools to monitor the performance of the [dotnet/cli
repo](https://github.com/dotnet/cli).

Setup Steps
===========

0. Install Visual Studio 2015 (required for the CLI build; the Community Edition
   is good enough)

1. Install chocolatey (must run elevated):

    ```batch
    cd /d %USERPROFILE%
    @powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin
    ```

2. Install required tools (must run elevated; best to start a clean shell):

    ```batch
    cd /d %USERPROFILE%
    choco install git python2 python microsoft-build-tools cmake.portable -y
    ```

3. Set up dependencies (must run elevated; best to start a clean shell):

    ```batch
    cd /d %USERPROFILE%
    setx /m PATH "%PATH%;C:\Program Files (x86)\MSBuild\14.0\Bin"
    ```

4. Create a base directory--example only here, `C:\DotNet` can be replaced with
   any other path on the system. (Best to start this from a clean,
   *non*-elevated shell.)

    ```batch
    mkdir C:\DotNet
    cd /d C:\DotNet
    ```

5. Clone this repo

    ```batch
    git clone https://github.com/dpodder/cli-perf.git
    ```

Running the Script
==================

```batch
cd /d C:\DotNet\cli-perf\rolling-perf
py rolling-perf.py --branch rel/1.0.0 --dir C:\DotNet\working-dir
```
{'fraction_non_alphanumeric': 0.14471830985915493, 'fraction_numerical': 0.014788732394366197, 'mean_word_length': 3.252994011976048, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '846136', 'n_tokens_mistral': 928, 'n_tokens_neox': 874, 'n_words': 302}