<commit_before>Integration Testing Framework
=============

This code builds an image which serves as a framework to run basic integration tests for a language runtime image. It relies on the existence of a sample application which fulfills a spec (detailed below), that will be built and deployed on top of the newly built runtime image. These tests are intended to be run as part of a post-push verification step in a continuous deployment system. The test driver can be run directly via any build host, or as a build step in a [Google Cloud Container Build](https://cloud.google.com/container-builder/docs/overview)).

To run these tests through a cloudbuild, add the following build step to the **end** of your build config (cloudbuild.yaml or cloudbuild.json):

	name: gcr.io/gcp-runtimes/integration_test
	args: [
		'-i', <target_staging_image>,
		'-d', <sample_application_directory>
	]

**It's crucial that this step appears after your image under test has been built**; without a built image, there will be nothing to test, and your build will fail!


The sample application directory should contain the application fulfilling the integration test spec, as well as the necessary build artifacts to deploy the application via gcloud, which means at minimum:

	* a templated Dockerfile, with the first line being
		` FROM ${STAGING_IMAGE} `

	* an app.yaml config

Alternatively, the application can be manually deployed *before* running the tests; in this scenario, the '--no-deploy' flag can be passed to the build step to opt out of deploying, in tandem with the URL at which the deployed application can be accessed:

	args: [
		...,
		'--no-deploy',
		'--url', '<application_url>'
	]

In addition, each test (detailed below) can be skipped by passing the corresponding flag to the build step. For example, to opt out of the monitoring test in the run, pass the `--no-monitoring` flag to the build step.

## Tests

### Serving (Root)
#####` - GET http://<application_url>/`

*Response*

- If successful, the application should respond with the string 'Hello World!'

This is the most basic integration test. The driver performs a GET request to the root endpoint at the deployed application's URL. It retrieves data from the application, and verifies it matches the expected output (the text ‘Hello World!’).<commit_msg>update docs to specify Dockerfile.in (#137)<commit_after>Integration Testing Framework
=============

This code builds an image which serves as a framework to run basic integration tests for a language runtime image. It relies on the existence of a sample application which fulfills a spec (detailed below), that will be built and deployed on top of the newly built runtime image. These tests are intended to be run as part of a post-push verification step in a continuous deployment system. The test driver can be run directly via any build host, or as a build step in a [Google Cloud Container Build](https://cloud.google.com/container-builder/docs/overview)).

To run these tests through a cloudbuild, add the following build step to the **end** of your build config (cloudbuild.yaml or cloudbuild.json):

	name: gcr.io/gcp-runtimes/integration_test
	args: [
		'-i', <target_staging_image>,
		'-d', <sample_application_directory>
	]

**It's crucial that this step appears after your image under test has been built**; without a built image, there will be nothing to test, and your build will fail!


The sample application directory should contain the application fulfilling the integration test spec, as well as the necessary build artifacts to deploy the application via gcloud, which means at minimum:

* a templated `Dockerfile.in`, with the first line being
	` FROM ${STAGING_IMAGE} `

* an `app.yaml` config

Alternatively, the application can be manually deployed *before* running the tests; in this scenario, the '--no-deploy' flag can be passed to the build step to opt out of deploying, in tandem with the URL at which the deployed application can be accessed:

	args: [
		...,
		'--no-deploy',
		'--url', '<application_url>'
	]

In addition, each test (detailed below) can be skipped by passing the corresponding flag to the build step. For example, to opt out of the monitoring test in the run, pass the `--no-monitoring` flag to the build step.

## Tests

### Serving (Root)
#####` - GET http://<application_url>/`

*Response*

- If successful, the application should respond with the string 'Hello World!'

This is the most basic integration test. The driver performs a GET request to the root endpoint at the deployed application's URL. It retrieves data from the application, and verifies it matches the expected output (the text ‘Hello World!’).
{'fraction_non_alphanumeric': 0.07644804088586031, 'fraction_numerical': 0.0006388415672913117, 'mean_word_length': 4.442641946697567, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7290948', 'n_tokens_mistral': 1222, 'n_tokens_neox': 1153, 'n_words': 672}