<commit_before># 5. 3 Go Agent

## 5.3.1 Overview

Go Agent is the component of Go which runs the jobs given to it by the Go Server. An Agent comprises of 3 parts, the Agent Bootstrapper, the Agent Launcher and the Agent itself.

**Photo needs to be inserted here.** 

## 5.3.2 Bootstrapper

The bootstrapper is the only component of an agent that requires a manual upgrade, therefore its code is kept to a bare minimum to avoid this manual upgrade.
 
The purpose of the bootstrapper is to launch the agent launcher and keep the process running. 
A new installation of an agent has only the agent-bootstrapper.jar. 

When you start an agent, it essentially starts the agent-bootstrapper.jar, which does the following: 

1. The agent-bootstrapper.jar copies over the agent-launcher.jar, bundled along with it, to a [UUID]agent-launcher.jar and starts it. 
2. If at any point the launcher process is killed, the bootstrapper re-launches it as long as the boostrapper process is still running. 

## 5.3.3 Launcher

Once the launcher is launched, it does the following:

1. Creates a lockfile. The purpose of the lockfile is to prevent multiple instances of the bootstrapper from being launched using the same agent-bootstrapper.jar, i.e., if you want to run two agents on the same box, they should not be started in the same location as it would interfere with the working of the agents. 
2. Checks to see if a new version of the agent-launcher.jar is available. This is done by comparing the `md5` checksum of the agent-launcher.jar present locally with the agent-launcher.jar on the Go Server. If the launcher is upgraded, it is restarted by the bootstrapper.
3. Downloads the agent.jar and agent-plugins.zip from the Go server. It then starts the agent.

## 5.3.4 Agent

The agent is responsible for running a job given to it by the server. Agents are uniquely identified by the server based on their UUID. 

At any given time, an agent has to be in one of the following states:

1. Pending - until an adminitrator enables it for the very first time.
2. Idle - when it is not running any jobs.
3. Preparing - when it is preparing to run the job given to it by the server, for example - checking out the source code.
4. Building - when it is running the job given to it by the server.
5. Lost Contact - if it does not ping the server for more than 5 minutes.
6. Missing - if it is restarted when an agent is in the lost contact state.
7. Disabled - when an agent is disabled manually by an administrator.

An agent has 3 threads running as part of the agent process:

1. Ping Thread - runs every 5 seconds. to inform the server that the agent is alive. 
2. Loop/Worker Thread - The first time an agent comes up the loop thread, which runs every 10 seconds, registers it with the server. Once this is done, its job is to retrieve work from the server.
3. Instruction Execution Thread - which runs every 10 seconds to check if the job has been cancelled.

### 5.3.4.1 Initialization

Agent initialization happens through spring. The first time the AgentController bean is initialized, it creates the following:

1. Pipelines folder - the location where the agent will check out the source code to run the jobs on.
2. Secure Socket Layer (SSL) - communication infrastructure that an agent uses to communicate with the server. 
3. Global Unique Identifier (GUID) - with which the agent identifies itself with the server. As part of this procces the GUID of the agent is stored in the guid.txt, present under the config folder of the agent working directory.

The initial communication, up until registration, between the server and agent happens over HTTP. Once the registration is complete, the communication between the server and agent happens over HTTPS. 

### 5.3.4.2 Registration

Registration of the agent with the server happens when the loop thread is first initialized. Since, a number of agents can contact the server at the same time, the registration process might not be successful the very first time itself. So, the agent keeps trying to register itself every 5 seconds until it is successful.

**Agent Side**

    loop_thread()
        agent_registration()
        setup_ssl_infrastructure()
        retrieve_agent_cookie()
    end

Registration of the agent involves setting up the following:

1. Agent Certificate.
2. Agent Trust Store.
3. Agent Cookie.
4. SSL infratructure between the server and agent.

Once the agent requests registration form the server, the server does the following:

**Server Side**

    registration_request_from_agent()
        auto_register_agent_if_required()
        allAgentInstances.registerNewAgent()
        assign_certification()
    end
    
    assign_certification()
        assign_agent_x509_certificate
        assign_agent_trust_store
    end
    
When an agent is initially registered with the server, it is in the pending state until a Go administrator manually enables the agent. This is mainly done to prevent malicious agents from automatically enabling themselves on the server.

In the case where you still want to enable an agent automatically with the server, you can do so by creating the agent-registration.properties file. Through this file you can specify which resources and environments the agent belongs to. When the agent initially tries to register itself with the server, the server checks to see if the incoming request has an agent-registration.properties file "auto_register_agent_if_required()", if so then it sets up the agent with the resources and environments specified in this file.

### 5.3.4.3 Retrieve Work

Once the agent is enabled on the server, it is by default in the "idle" state, indicating that the agent is not doing any work. The loop thread then pings the server requesting for work. 

**Server Side**
    
    getWork()
        if_agent_has_not_been_assigned_work_yet_then
            idleAgentsTopic.post(agent_idle_details)
            return_no_work
        else_if_agent_is_assigned_work
            return_work
    end

A listener registered on the IdleAgentsTopic queue, so when an idle agent detail is added to this queue the following happens:

**Server Side**

    assignWork()
        buildAssignmentWorkService.assignWork(agent_details)

<commit_msg>Made some more changes to the agent documentation.<commit_after># 5. 3 Go Agent

## 5.3.1 Overview

Go Agent is the component of Go which runs the jobs given to it by the Go Server. An Agent comprises of 3 parts, the Agent Bootstrapper, the Agent Launcher and the Agent itself.

**Photo needs to be inserted here.** 

## 5.3.2 Bootstrapper

The bootstrapper is the only component of an agent that requires a manual upgrade, therefore its code is kept to a bare minimum to avoid this manual upgrade.
 
The purpose of the bootstrapper is to launch the agent launcher and keep the process running. It facilitates the automatic upgrade of the agent-launcher.

A new installation of an agent has only the agent-bootstrapper.jar. When you start an agent, it essentially starts the agent-bootstrapper.jar, which does the following: 

1. The agent-bootstrapper.jar copies over the agent-launcher.jar, bundled along with it, to a [UUID]agent-launcher.jar and starts it. 
2. If at any point the launcher process is killed, the bootstrapper re-launches it as long as the boostrapper process is still running. 

## 5.3.3 Launcher

Once the launcher is launched, it does the following:

1. Creates a lockfile. The purpose of the lockfile is to prevent multiple instances of the bootstrapper from being launched using the same agent-bootstrapper.jar, i.e., if you want to run two agents on the same box, they should not be started in the same location as it would interfere with the working of the agents. 
2. Checks to see if a new version of the agent-launcher.jar is available. This is done by comparing the `md5` checksum of the agent-launcher.jar present locally with the agent-launcher.jar on the Go Server. If the launcher is upgraded, it is restarted by the bootstrapper.
3. Downloads the agent.jar and agent-plugins.zip from the Go server. It then starts the agent.

The launcher allows for the automatic upgradation of the agent without manual intervention.

## 5.3.4 Agent

The agent is responsible for running a job given to it by the server. Agents are uniquely identified by the server based on their UUID. 

At any given time, an agent has to be in one of the following states:

1. Pending - until an adminitrator enables it for the very first time.
2. Idle - when it is not running any jobs.
3. Preparing - when it is preparing to run the job given to it by the server, for example - checking out the source code.
4. Building - when it is running the job given to it by the server.
5. Lost Contact - if it does not ping the server for more than 5 minutes.
6. Missing - if it is restarted when an agent is in the lost contact state.
7. Disabled - when an agent is disabled manually by an administrator.

An agent has 3 threads running as part of the agent process:

1. Ping Thread - runs every 5 seconds, to inform the server that the agent is alive. It also informs the server of the agents runtime information. 
2. Loop/Worker Thread - The first time an agent comes up the loop thread, which runs every 10 seconds, registers it with the server. Once this is done, its job is to retrieve work from the server.
3. Instruction Execution Thread - which runs every 10 seconds to check if the job has been cancelled.

### 5.3.4.1 Initialization

Agent initialization happens through spring. The first time the AgentController bean is initialized, it creates the following:

1. Pipelines folder - the location where the agent will check out the source code to run the jobs on.
2. Secure Socket Layer (SSL) - communication infrastructure that an agent uses to communicate with the server. 
3. Global Unique Identifier (GUID) - with which the agent identifies itself with the server. As part of this procces the GUID of the agent is stored in the guid.txt, present under the config folder of the agent working directory.

The initial communication, up until registration, between the server and agent happens over HTTP. Once the registration is complete, the communication between the server and agent happens over HTTPS. 

### 5.3.4.2 Registration

Registration of the agent with the server happens when the loop thread is first initialized. Since, a number of agents can contact the server at the same time, the registration process might not be successful the very first time itself. So, the agent keeps trying to register itself every 5 seconds until it is successful.

*Agent Side*

    loop_thread()
        agent_registration()
        setup_ssl_infrastructure()
        retrieve_agent_cookie()
    end

Registration of the agent involves setting up the following:

1. Agent Certificate.
2. Agent Trust Store.
3. Agent Cookie.
4. SSL infratructure between the server and agent.

Once the agent requests registration form the server, the server does the following:

*Server Side*

    registration_request_from_agent()
        auto_register_agent_if_required()
        allAgentInstances.registerNewAgent()
        assign_certification()
    end
    
    assign_certification()
        assign_agent_x509_certificate
        assign_agent_trust_store
    end
    
When an agent is initially registered with the server, it is in the pending state until a Go administrator manually enables the agent. This is mainly done to prevent malicious agents from automatically enabling themselves on the server.

In the case where you still want to enable an agent automatically with the server, you can do so by creating the agent-registration.properties file. Through this file you can specify which resources and environments the agent belongs to. When the agent initially tries to register itself with the server, the server checks to see if the incoming request has an agent-registration.properties file "auto_register_agent_if_required()", if so then it sets up the agent with the resources and environments specified in this file.

### 172.16.17.32 Retrieve & Build Work

Once the agent is enabled on the server, it is by default in the "idle" state, indicating that the agent is not doing any work. The loop thread then requests the server for work. 

*Server Side*
    
    getWork()
        if_agent_has_not_been_assigned_work_yet_then
            idleAgentsTopic.post(agent_idle_details)
            return_no_work
        else_if_agent_is_assigned_work
            return_work
    end

A listener, i.e., the `WorkFinder`, is registered on the IdleAgentsTopic queue, so when an idle agent detail is added to this queue the following happens:

*Server Side*

    assignWork()
        buildAssignmentWorkService.assignWork(agent_details)

Once the server assigns work to the agent, this work is picked up by the agent the next time the loop thread contacts the server, which is 10 seconds later.  The agent then goes into the "preparing" stage while it is preparing to run the job. It then goes into the "building" stage once it begins to run the job, which happens through `BuildWork`

The agent communicates its entire transition, from being assigned a job to completing the job, constantly with the server through the `CruisePublisher`. The output of the task being run is contantly updated in the go-agent.log through the `CruiseControlLog`, and this is the information that is available on the console log of the server for that particular task.  


{'fraction_non_alphanumeric': 0.04049452176488007, 'fraction_numerical': 0.009549896357713947, 'mean_word_length': 3.8057630736392745, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 6, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2152126', 'n_tokens_mistral': 3437, 'n_tokens_neox': 3307, 'n_words': 2025}