<commit_before>= DevSecOps Example
v${project.version}
ifdef::backend-pdf[]
:title-logo-image: image:icf-logo.png[500, 500, align="center"]
endif::backend-pdf[]

ifdef::backend-html5[]
image:icf-logo.png[ICF Logo, 150, 150, float="left"]
endif::backend-html5[]

== Overview

This project demonstrates an automated process to deploy an environment to AWS that will push updates into
production in response to code updates.

== Diagram

image::architecture.png[link=images/architecture.png]

== Deployment

=== Prerequisites

* An AWS account with administrative privileges as the solution creates identity roles for the service
* https://git-scm.com/[Git^]: Download the git version control system or use a packaged version for your OS
* A https://help.github.com/articles/signing-up-for-a-new-github-account/[GitHub Account^]
* A https://github.com/settings/tokens[GitHub Access Token^] is required to set the repository hook that starts the
    build and deploy process immediately after new code commits have been pushed. The token needs to have `repo` and `admin:repo_hook` scopes authorized
    icon:camera-retro[link=images/token-scopes.png, size=2x, title=screenshot]. More information about tokens is
    available from https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/[GitHub Help^]

=== Steps to Deploy

* *Fork the code repository:* Go the https://github.com/ICFI/raptor-devops[GitHub repository web page^] for this project and click the
    Fork button in the top right-hand side of the page. This will create a copy of the code in your own account and will be needed to
    test the automation hook. After the deployment has completed we'll push a new code commit to observe the automated build
    and deployment cycle icon:camera-retro[link=images/fork.png, size=2x, title=screenshot].
* *Clone your fork:* Run the command `git clone https://github.com/YOUR_GITHUB_USERNAME/raptor-devops` and change into
    the raptor-devops directory. This is the workspace from which you will make changes to see the automated process
    run shortly after you push your commits to GitHub
* *Launch the CloudFormation Stack:* Log into the AWS account and then use one of the links below to launch the CloudFormation
    stack.

.CloudFormation Launch Links
|===
|Region |Launch Stack in Region

|N. Virginia (us-east-1)
a|image::launch.png[link="https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=raptor-devops&templateURL=https://s3.amazonaws.com/raptor-devops-deployment-us-east-1/raptor-devops-template.yaml"^]

|Ohio (us-east-2)
a|image::launch.png[link="https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/new?stackName=raptor-devops&templateURL=https://s3.amazonaws.com/raptor-devops-deployment-us-east-2/raptor-devops-template.yaml"^]

|Oregon (us-west-2)
a|image::launch.png[link="https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=raptor-devops&templateURL=https://s3.amazonaws.com/raptor-devops-deployment-us-east-1/raptor-devops-template.yaml"^]

|===
* *CloudFormation Launch Parameters* After clicking the Launch Stack link in the region of your choice you'll be taken through a series
    of screens that capture any additional information prior to launching the solution stack.
    ** *Select Template Screen* The correct template has been entered by the launch link so press the Next button to continue to the next screen
    ** *Specify Details Screen* icon:camera-retro[link=images/cloudformation-params.png, size=2x, title=screenshot]
        *** *Restrict Access* Enter the allowed IP addresses that will be able to access the solution. Use http://checkip.amazonaws.com[this link^],
            to find your IP address, remember to add the /32 at the end to limit to just your address example: 10.10.123.123/32
        *** *Repo* This is the name of your forked repo and can be left with the default value
        *** *Branch* The master branch can be left with the default value
        *** *User:* Enter your GitHub username
        *** *Personal Access Token* The access token you created authorizing Code Pipeline to receive notification when
            new code is pushed. https://github.com/settings/tokens[Token creation^] is accomplished from within your GitHub account
            and needs to have `repo` and `admin:repo_hook` scopes authorized icon:camera-retro[link=images/token-scopes.png, size=2x, title=screenshot].
            Once the token has been created you will have a chance to copy the code and it will then be visible within a list of
            active tokens within your account icon:camera-retro[link=images/token-screenshot.png, size=2x, title=screenshot]

    ** *Options Screen* Defaults are all fine here, press the Next button to continue to the next screen.
    ** *Review Screen* All of the captured parameters are listed for review. At the bottom of the screen you'll need to tick the checkbox that
        acknowledges that CloudFormation will create IAM resources and finally press the Create button to launch the stack.

== Review Deployment

Once the initial five CloudFormation infrastructure stacks have completed deployment navigate to the Outputs tab of the parent
stack icon:camera-retro[link=images/outputs.png, size=2x, title=screenshot] and open the *PipelineURL* link to watch the CI/CD process in action. Code Pipeline performs a
three-step process to build and deploy the project.

. Code Pipeline checks out the code from your forked GitHub project and sets a hook so when future commits are pushed the pipeline will
    be notified.
. Code Pipeline next uses Code Build to build and package the api application artifact
    ** Compiles the code into a war file using a Docker container
    ** Creates a Docker image containing the application for use when deploying to an (ECS) EC2 Container Service cluster
    ** Pushes the application to ECR, which we're using to host our private container images

        pre_build:
          commands:
            - $(aws ecr get-login)
            - TAG="$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)"
        build:
          commands:
            - docker run --rm -v "$(pwd)":/tmp/api -w /tmp/api maven:3.5-jdk-8 mvn -pl app/api verify
            - docker build -t api app/api
            - docker tag api:latest "${REPOSITORY_URI}:${TAG}"
        post_build:
          commands:
            - docker push "${REPOSITORY_URI}:${TAG}"

. The last step is Code Pipeline pulls the newly created application container image from ECR and either deploys
    the CloudFormation service stack that launches the api or if already deployed performs an update to launch
    the version that corresponds with the latest code commit.
. Once the Code Pipeline has completed the first deployment you'll be able to use the other two links listed in the
    Outputs tab to view the deployed application.
    ** *GreetingUrl:* This link is to the `/greet` endpoint of the api application deployed as a container within ECS.
        The output is in json format as the application function is that as an api providing information to possibly
        many different front-end UIs.
    ** *VersionUrl:* When the application is built timestamp and other information is captured and written into the
        final artifact to allow for version information to be discoverable. Notice that the timestamp shows that
        the application was just built on demand when you launched the stack.
. Make a change, commit and push in the local repo we cloned right after forking the project. We've authorized
    GitHub to notify CodePipeline when new code is pushed so this should trigger a redeployment of the api application
    which can be followed from the *PipelineURL* link. Verify the redeployment by checking the timestamp of the api
    application again after the pipeline completes.

== Steps to Undeploy

When finished with the solution follow the steps below to remove all resources provisioned.

. *Delete the Service Stack* From the CloudFormation console locate the stack that ends in -Service. This is the running
    application and must be deleted prior to the rest of the stack. Tick the checkbox and then use Actions->Delete Stack
    to remove icon:camera-retro[link=images/outputs.png, size=2x, title=screenshot]
. *Delete the Stacks* Once the web service stack has been deleted tick the checkbox next to the parent stack and use Actions->Delete Stack to remove
. *Delete the S3 bucket* The bucket containing the build artifacts produced by the CI/CD process
. *Delete the container repo* The ECR contains the Docker images created by the CI/CD process

== Tools

* Build:
    ** https://maven.apache.org/[Maven^] is a software project management tool used to build and package the project
    ** https://www.sonarqube.org/[SonarQube^] provides the capability to not only show health of an application but also
        to highlight issues newly introduced. Used to scan the example application in this project.
* AWS Provisioning and Deployment:
    ** https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer/[Application Load Balancer^] is a load
        balancing option for the Elastic Load Balancing service that operates at the application layer and allows you
        to define routing rules based on content across multiple services or containers running on one or more Amazon
        Elastic Compute Cloud (Amazon EC2) instances.<commit_msg>fix screenshot links in pdf<commit_after>= DevSecOps Example
v${project.version}
ifdef::backend-pdf[]
:title-logo-image: image:icf-logo.png[500, 500, align="center"]
endif::backend-pdf[]

ifdef::backend-html5[]
image:icf-logo.png[ICF Logo, 150, 150, float="left"]
endif::backend-html5[]

== Overview

This project demonstrates an automated process to deploy an environment to AWS that will push updates into
production in response to code updates.

== Diagram

image::architecture.png[link=images/architecture.png]

== Deployment

=== Prerequisites

* An AWS account with administrative privileges as the solution creates identity roles for the service
* https://git-scm.com/[Git^]: Download the git version control system or use a packaged version for your OS
* A https://help.github.com/articles/signing-up-for-a-new-github-account/[GitHub Account^]
* A https://github.com/settings/tokens[GitHub Access Token^] is required to set the repository hook that starts the
    build and deploy process immediately after new code commits have been pushed. The token needs to have `repo` and
    `admin:repo_hook` scopes authorized. More information about creating an access token is available from
    https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/[GitHub Help^]
    https://github.com/ICFI/raptor-devops/raw/master/docs/src/docs/images/token-scopes.png[(screenshot icon:camera-retro[link=images/token-scopes.png, size=2x, title=screenshot] )]

=== Steps to Deploy

* *Fork the code repository:* Go the https://github.com/ICFI/raptor-devops[GitHub repository web page^] for this project and click the
    Fork button in the top right-hand side of the page. This will create a copy of the code in your own account and will be needed to
    test the automation hook. After the deployment has completed we'll push a new code commit to observe the automated build
    and deployment cycle https://github.com/ICFI/raptor-devops/raw/master/docs/src/docs/images/fork.png[(screenshot icon:camera-retro[link=images/fork.png, size=2x, title=screenshot] )].
* *Clone your fork:* Run the command `git clone https://github.com/YOUR_GITHUB_USERNAME/raptor-devops` and change into
    the raptor-devops directory. This is the workspace from which you will make changes to see the automated process
    run shortly after you push your commits to GitHub
* *Launch the CloudFormation Stack:* Log into the AWS account and then use one of the links below to launch the CloudFormation
    stack.

.CloudFormation Launch Links
|===
|Region |Launch Stack in Region

|N. Virginia (us-east-1)
a|image::launch.png[link="https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=raptor-devops&templateURL=https://s3.amazonaws.com/raptor-devops-deployment-us-east-1/raptor-devops-template.yaml"^]

|Ohio (us-east-2)
a|image::launch.png[link="https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/new?stackName=raptor-devops&templateURL=https://s3.amazonaws.com/raptor-devops-deployment-us-east-2/raptor-devops-template.yaml"^]

|Oregon (us-west-2)
a|image::launch.png[link="https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=raptor-devops&templateURL=https://s3.amazonaws.com/raptor-devops-deployment-us-east-1/raptor-devops-template.yaml"^]

|===
* *CloudFormation Launch Parameters* After clicking the Launch Stack link in the region of your choice you'll be taken through a series
    of screens that capture any additional information prior to launching the solution stack.
    ** *Select Template Screen* The correct template has been entered by the launch link so press the Next button to continue to the next screen
    ** *Specify Details Screen* https://github.com/ICFI/raptor-devops/raw/master/docs/src/docs/images/cloudformation-params.png[(screenshot icon:camera-retro[link=images/cloudformation-params.png, size=2x, title=screenshot] )]
        *** *Restrict Access* Enter the allowed IP addresses that will be able to access the solution. Use http://checkip.amazonaws.com[this link^],
            to find your IP address, remember to add the /32 at the end to limit to just your address example: 10.10.123.123/32
        *** *Repo* This is the name of your forked repo and can be left with the default value
        *** *Branch* The master branch can be left with the default value
        *** *User:* Enter your GitHub username
        *** *Personal Access Token* The access token you created authorizing Code Pipeline to receive notification when
            new code is pushed. https://github.com/settings/tokens[Token creation^] is accomplished from within your GitHub account
            and needs to have `repo` and `admin:repo_hook` scopes authorized https://github.com/ICFI/raptor-devops/raw/master/docs/src/docs/images/token-scopes.png[(screenshot icon:camera-retro[link=images/token-scopes.png, size=2x, title=screenshot] )].
            Once the token has been created you will have a chance to copy the code and it will then be visible within a list of
            active tokens within your account https://github.com/ICFI/raptor-devops/raw/master/docs/src/docs/images/token-screenshot.png[(screenshot icon:camera-retro[link=images/token-screenshot.png, size=2x, title=screenshot] )]

    ** *Options Screen* Defaults are all fine here, press the Next button to continue to the next screen.
    ** *Review Screen* All of the captured parameters are listed for review. At the bottom of the screen you'll need to tick the checkbox that
        acknowledges that CloudFormation will create IAM resources and finally press the Create button to launch the stack.

== Review Deployment

Once the initial five CloudFormation infrastructure stacks have completed deployment navigate to the Outputs tab of the parent
stack and open the *PipelineURL* link to watch the CI/CD process in action. Code Pipeline performs a
three-step process to build and deploy the project. https://github.com/ICFI/raptor-devops/raw/master/docs/src/docs/images/outputs.png[(screenshot icon:camera-retro[link=images/outputs.png, size=2x, title=screenshot] )]

. Code Pipeline checks out the code from your forked GitHub project and sets a hook so when future commits are pushed the pipeline will
    be notified.
. Code Pipeline next uses Code Build to build and package the api application artifact
    ** Compiles the code into a war file using a Docker container
    ** Creates a Docker image containing the application for use when deploying to an (ECS) EC2 Container Service cluster
    ** Pushes the application to ECR, which we're using to host our private container images

        pre_build:
          commands:
            - $(aws ecr get-login)
            - TAG="$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)"
        build:
          commands:
            - docker run --rm -v "$(pwd)":/tmp/api -w /tmp/api maven:3.5-jdk-8 mvn -pl app/api verify
            - docker build -t api app/api
            - docker tag api:latest "${REPOSITORY_URI}:${TAG}"
        post_build:
          commands:
            - docker push "${REPOSITORY_URI}:${TAG}"

. The last step is Code Pipeline pulls the newly created application container image from ECR and either deploys
    the CloudFormation service stack that launches the api or if already deployed performs an update to launch
    the version that corresponds with the latest code commit.
. Once the Code Pipeline has completed the first deployment you'll be able to use the other two links listed in the
    Outputs tab to view the deployed application.
    ** *GreetingUrl:* This link is to the `/greet` endpoint of the api application deployed as a container within ECS.
        The output is in json format as the application function is that as an api providing information to possibly
        many different front-end UIs.
    ** *VersionUrl:* When the application is built timestamp and other information is captured and written into the
        final artifact to allow for version information to be discoverable. Notice that the timestamp shows that
        the application was just built on demand when you launched the stack.
. Make a change, commit and push in the local repo we cloned right after forking the project. We've authorized
    GitHub to notify CodePipeline when new code is pushed so this should trigger a redeployment of the api application
    which can be followed from the *PipelineURL* link. Verify the redeployment by checking the timestamp of the api
    application again after the pipeline completes.

== Steps to Undeploy

When finished with the solution follow the steps below to remove all resources provisioned.

. *Delete the Service Stack* From the CloudFormation console locate the stack that ends in -Service. This is the running
    application and must be deleted prior to the rest of the stack. Tick the checkbox and then use Actions->Delete Stack
    to remove https://github.com/ICFI/raptor-devops/raw/master/docs/src/docs/images/outputs.png[(screenshot icon:camera-retro[link=images/outputs.png, size=2x, title=screenshot] )]
. *Delete the Stacks* Once the web service stack has been deleted tick the checkbox next to the parent stack and use Actions->Delete Stack to remove
. *Delete the S3 bucket* The bucket containing the build artifacts produced by the CI/CD process
. *Delete the container repo* The ECR contains the Docker images created by the CI/CD process

== Tools

* Build:
    ** https://maven.apache.org/[Maven^] is a software project management tool used to build and package the project
    ** https://www.sonarqube.org/[SonarQube^] provides the capability to not only show health of an application but also
        to highlight issues newly introduced. Used to scan the example application in this project.
* AWS Provisioning and Deployment:
    ** https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer/[Application Load Balancer^] is a load
        balancing option for the Elastic Load Balancing service that operates at the application layer and allows you
        to define routing rules based on content across multiple services or containers running on one or more Amazon
        Elastic Compute Cloud (Amazon EC2) instances.
{'fraction_non_alphanumeric': 0.07715628315901167, 'fraction_numerical': 0.005457025920873124, 'mean_word_length': 3.967871485943775, 'pattern_counts': {'":': 2, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 39, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4129652', 'n_tokens_mistral': 5363, 'n_tokens_neox': 5191, 'n_words': 2299}