<commit_before>Plugin Compatibility Tester (PCT)
------

Generates a compatibility matrix for plugins against Jenkins core.

See https://wiki.jenkins-ci.org/display/JENKINS/Plugin+Compatibility+Tester for background.

## Running PCT 

### Running PCT in Docker

It is recommended to run PCT in the prepared Docker image.
You can find it [here](https://hub.docker.com/r/jenkins/pct/).

#### Examples

This command will clone of a repository and run PCT against the latest Jenkins Core:

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -e ARTIFACT_ID=ssh-slaves -e VERSION=ssh-slaves-1.24 jenkins/pct
```

This command will run PCT for the latest plugin version against the specified Jenkins WAR.
PCT supports running against SNAPSHOT builds, but PCT will have to install local Maven artifacts in such case.

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -v my/jenkins.war:/pct/jenkins.war:ro -e ARTIFACT_ID=ssh-slaves jenkins/pct
```

This command will run PCT against custom versions of Jenkins and the plugin specified by volumes.

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -v my/jenkins.war:/pct/jenkins.war:ro -v my/plugin:/pct/plugin-src:ro jenkins/pct
```

The command below will run PCT for a branch in a custom repository.

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -e CHECKOUT_SRC=https://github.com/oleg-nenashev/job-restrictions-plugin.git -e VERSION=JENKINS-26374 jenkins/pct
```

#### Configuration

Environment variables:

* `ARTIFACT_ID` - ID of the artifact to be tested.
The image will be able to determine this ID automatically if `CHECKOUT_SRC` or `/pct/plugin-src` are defined.
* `VERSION` - tag/commit/branch to be checked out and tested. `master` by default
* `CHECKOUT_SRC` - Custom Git clone source (e.g. `https://github.com/oleg-nenashev/job-restrictions-plugin.git`). `https://github.com/jenkinsci/${ARTIFACT_ID}-plugin.git` by default
* `JAVA_OPTS` - Java options to be passed to the PCT CLI
* `DEBUG` - Boolean flag, which enables the Remote Debug mode (port == 5000)
* `M2_SETTINGS_FILE` -  If set indicates the path of the custom maven settings file to use (see volumes below)
* `INSTALL_BUNDLED_SNAPSHOTS` - Install JAR and plugin snapshots to local repository.
        `true` by default. WAR snapshots will be always installed.

Volumes:

* `/pct/plugin-src` - Plugin sources to be used for the PCT run. Sources will be checked out if not specified
* `/pct/jenkins.war` - Jenkins WAR file to be used for the PCT run
* `/pct/m2-settings.xml` - Custom Maven Settings (optional) if `M2_SETTINGS_FILE` environment variable exists `run-pct` will ignore this location and use the one specified in the variable
* `/pct/out` - Output directory for PCT. All reports will be stored there
* `/pct/tmp` - Temporary directory. Can be exposed to analyze run failures
* `/root/.m2` - Maven repository. It can be used to pass settings.xml or to cache artifacts

### Running PCT manually

PCT offers the CLI interface which can be used to run PCT locally.

* Download `PCT` and execute `mvn clean install`
* Download plugin sources to a `PLUGIN_SRC` directory. 
* Checkout the plugin repo to a tag/commit/branch you want to test
* Run `mvn clean install -DskipTests` to initialize artifacts
* Go to `PCT` folder and run the CLI (make sure to modify paths according to your system). Example:

```shell
java -jar plugins-compat-tester-cli/target/plugins-compat-tester-cli-${PCT_VERSION}.jar \
  -reportFile $(pwd)/out/report.xml \
  -workDirectory $(pwd)/tmp/work \
  -includePlugins ${PLUGIN_ARTIFACT_ID} \
  -war jenkins.war -localCheckoutDir ${PLUGIN_SRC} \
  -skipTestCache true \
  -mvn ${PATH_TO_MAVEN}
```

You can run the CLI with the `-help` argument to get a full list of supported options.

:exclamation: For the moment testing more than one plugin at once requires plugins to be released, so for testing SNAPSHOTS you need to execute the last step for every plugin you want to test*

### Running PCT with custom Java versions

Plugin compat tester supports running Test Suites with a Java version different 
from the one being used to run PCT and build the plugins.
For example, such mode can be used to run tests with JDK11 while
the plugin build flow is not adapted to it.

Two options can be passed to PCT CLI for such purpose:

* `testJDKHome` - A path to JDK HOME to be used for running tests in plugins
* `testJavaArgs` - Java test arguments to be used for test runs.
                   For JDK 11 this option may be used to pass modules and the classpath

You can run the example by running the following command:

    make demo-jdk11 -e TEST_JDK_HOME=${YOUR_JDK11_HOME}

Full list of options for JDK11 can be found [here](./Makefile).

When using the docker image, it is possible to use `JDK_VERSION` variable to select 
the version to use. The version needs to be bundled in the docker image. By specifying 
`11`, the modules and jaxb libraries are also added to the command line. 

### Running PCT with different version of dependencies

Plugin Compat Tester supports overriding the plugin dependency version. <commit_msg>[JENKINS-54862] Add notice in README<commit_after>Plugin Compatibility Tester (PCT)
------

Generates a compatibility matrix for plugins against Jenkins core.

See https://wiki.jenkins-ci.org/display/JENKINS/Plugin+Compatibility+Tester for background.

## Running PCT 

### Running PCT in Docker

It is recommended to run PCT in the prepared Docker image.
You can find it [here](https://hub.docker.com/r/jenkins/pct/).

#### Examples

This command will clone of a repository and run PCT against the latest Jenkins Core:

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -e ARTIFACT_ID=ssh-slaves -e VERSION=ssh-slaves-1.24 jenkins/pct
```

This command will run PCT for the latest plugin version against the specified Jenkins WAR.
PCT supports running against SNAPSHOT builds, but PCT will have to install local Maven artifacts in such case.

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -v my/jenkins.war:/pct/jenkins.war:ro -e ARTIFACT_ID=ssh-slaves jenkins/pct
```

This command will run PCT against custom versions of Jenkins and the plugin specified by volumes.

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -v my/jenkins.war:/pct/jenkins.war:ro -v my/plugin:/pct/plugin-src:ro jenkins/pct
```

The command below will run PCT for a branch in a custom repository.

```shell
docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -e CHECKOUT_SRC=https://github.com/oleg-nenashev/job-restrictions-plugin.git -e VERSION=JENKINS-26374 jenkins/pct
```

#### Configuration

Environment variables:

* `ARTIFACT_ID` - ID of the artifact to be tested.
The image will be able to determine this ID automatically if `CHECKOUT_SRC` or `/pct/plugin-src` are defined.
* `VERSION` - tag/commit/branch to be checked out and tested. `master` by default
* `CHECKOUT_SRC` - Custom Git clone source (e.g. `https://github.com/oleg-nenashev/job-restrictions-plugin.git`). `https://github.com/jenkinsci/${ARTIFACT_ID}-plugin.git` by default
* `JAVA_OPTS` - Java options to be passed to the PCT CLI
* `DEBUG` - Boolean flag, which enables the Remote Debug mode (port == 5000)
* `M2_SETTINGS_FILE` -  If set indicates the path of the custom maven settings file to use (see volumes below)
* `INSTALL_BUNDLED_SNAPSHOTS` - Install JAR and plugin snapshots to local repository.
        `true` by default. WAR snapshots will be always installed.

Volumes:

* `/pct/plugin-src` - Plugin sources to be used for the PCT run. Sources will be checked out if not specified
* `/pct/jenkins.war` - Jenkins WAR file to be used for the PCT run
* `/pct/m2-settings.xml` - Custom Maven Settings (optional) if `M2_SETTINGS_FILE` environment variable exists `run-pct` will ignore this location and use the one specified in the variable
* `/pct/out` - Output directory for PCT. All reports will be stored there
* `/pct/tmp` - Temporary directory. Can be exposed to analyze run failures
* `/root/.m2` - Maven repository. It can be used to pass settings.xml or to cache artifacts

### Running PCT manually

PCT offers the CLI interface which can be used to run PCT locally.

* Download `PCT` and execute `mvn clean install`
* Download plugin sources to a `PLUGIN_SRC` directory. 
* Checkout the plugin repo to a tag/commit/branch you want to test
* Run `mvn clean install -DskipTests` to initialize artifacts
* Go to `PCT` folder and run the CLI (make sure to modify paths according to your system). Example:

```shell
java -jar plugins-compat-tester-cli/target/plugins-compat-tester-cli-${PCT_VERSION}.jar \
  -reportFile $(pwd)/out/report.xml \
  -workDirectory $(pwd)/tmp/work \
  -includePlugins ${PLUGIN_ARTIFACT_ID} \
  -war jenkins.war -localCheckoutDir ${PLUGIN_SRC} \
  -skipTestCache true \
  -mvn ${PATH_TO_MAVEN}
```

You can run the CLI with the `-help` argument to get a full list of supported options.

:exclamation: For the moment testing more than one plugin at once requires plugins to be released, so for testing SNAPSHOTS you need to execute the last step for every plugin you want to test*

### Running PCT with custom Java versions

Plugin compat tester supports running Test Suites with a Java version different 
from the one being used to run PCT and build the plugins.
For example, such mode can be used to run tests with JDK11 while
the plugin build flow is not adapted to it.

Two options can be passed to PCT CLI for such purpose:

* `testJDKHome` - A path to JDK HOME to be used for running tests in plugins
* `testJavaArgs` - Java test arguments to be used for test runs.
                   For JDK 11 this option may be used to pass modules and the classpath

You can run the example by running the following command:

    make demo-jdk11 -e TEST_JDK_HOME=${YOUR_JDK11_HOME}

Full list of options for JDK11 can be found [here](./Makefile).

When using the docker image, it is possible to use `JDK_VERSION` variable to select 
the version to use. The version needs to be bundled in the docker image. By specifying 
`11`, the modules and jaxb libraries are also added to the command line. 

### Running PCT with different version of dependencies

IMPORTANT: At this moment, the replacement of a dependency will not update the transitive dependencies.
Because of this, you might have to provide on the same parameter the new versions of the transitive dependencies.

Plugin Compat Tester supports overriding the plugin dependency version. 
{'fraction_non_alphanumeric': 0.0896538679780594, 'fraction_numerical': 0.006525439757896728, 'mean_word_length': 3.916317991631799, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 6}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '435871', 'n_tokens_mistral': 3377, 'n_tokens_neox': 3171, 'n_words': 1400}