<commit_before>
[[batch]]
= Batch

[[partintro]]
--
This section goes into more detail about Spring Cloud Task's integrations with Spring
Batch.  Tracking the association between a job execution and the task it was executed
within as well as remote partitioning via Spring Cloud Deployer are all covered within
this section.
--

[[batch-association]]
== Associating A Job Execution To The Task In Which It Was Executed

Spring Boot provides facilities for the execution of batch jobs easily within an über-jar.
Spring Boot's support of this functionality allows for a developer to execute multiple
batch jobs within that execution.  Spring Cloud Task provides the ability to associate the
execution of a job (a job execution) with a task's execution so that one can be traced
back to the other.

This functionality is accomplished by using the `TaskBatchExecutionListener`.  By default,
this listener is auto configured in any context that has both a Spring Batch Job configured
(via having a bean of type `Job` defined in the context) and the spring-cloud-task-batch jar
is available within the classpath.  The listener will be injected into all jobs.

[[batch-association-override]]
=== Overriding the TaskBatchExecutionListener

To prevent the listener from being injected into any batch jobs within the current context,
the autoconfiguration can be disabled via standard Spring Boot mechanisms.

To only have the listener injected into particular jobs within the context, the
`batchTaskExecutionListenerBeanPostProcessor` may be overridden and a list of job bean ids
can be provided:

```
public TaskBatchExecutionListenerBeanPostProcessor batchTaskExecutionListenerBeanPostProcessor() {
	TaskBatchExecutionListenerBeanPostProcessor postProcessor =
		new TaskBatchExecutionListenerBeanPostProcessor();

	postProcessor.setJobNames(Arrays.asList(new String[] {"job1", "job2"}));

	return postProcessor;
}
```

[[batch-partitioning]]
== Remote Partitioning

Spring Cloud Deployer provides facilities for launching Spring Boot based applications on
most cloud infrastructures.  The `DeployerPartitionHandler` and
`DeployerStepExecutionHandler` delegate the launching of worker step executions to Spring
Cloud Deployer.

To configure the `DeployerStepExecutionHandler`, a `Resource` representing the Spring Boot
über-jar to be executed, a `TaskLauncher`, and a `JobExplorer` are all required.  You can
configure any environment properties as well as the max number of workers to be executing
at once, the interval to poll for the results (defaults to 10 seconds), and a timeout
(defaults to -1 or no timeout).  An example of configuring this `PartitionHandler` would
look like the following:

```
@Bean
public PartitionHandler partitionHandler(TaskLauncher taskLauncher,
		JobExplorer jobExplorer) throws Exception {

	MavenProperties mavenProperties = new MavenProperties();
	mavenProperties.setRemoteRepositories(new HashMap<>(Collections.singletonMap("springRepo",
		new MavenProperties.RemoteRepository(repository))));

 	MavenResource resource =
		MavenResource.parse(String.format("%s:%s:%s",
				"io.spring.cloud",
				"partitioned-batch-job",
				"1.0.0.RELEASE"), mavenProperties);

	DeployerPartitionHandler partitionHandler =
		new DeployerPartitionHandler(taskLauncher, jobExplorer, resource, "workerStep");

	Map<String, String> environmentProperties = new HashMap<>();
	environmentProperties.put("spring.profiles.active", "worker");
	SimpleEnvironmentVariablesProvider environmentVariablesProvider = new SimpleEnvironmentVariablesProvider(this.environment);
	environmentVariablesProvider.setEnvironmentProperties(environmentProperties);
	partitionHandler.setEnvironmentVariablesProvider(environmentVariablesProvider);

	partitionHandler.setMaxWorkers(2);

	return partitionHandler;
}
```

The `Resource` to be executed is expected to be a Spring Boot über-jar with a
`DeployerStepExecutionHandler` configured as a `CommandLineRunner` in the current context.
The repository enumerated in the example above should be the location of the remote repository
from which the über-jar is located. Both the master and slave are expected to have
visibility into the same data store being used as the job repository and task repository.
Once the underlying infrastructure has bootstrapped the Spring Boot jar and Spring Boot
has launched the `DeployerStepExecutionHandler`, the step handler will execute the Step
requested.  An example of configuring the `DefaultStepExecutionHandler` is show below:

```
@Bean
public DeployerStepExecutionHandler stepExecutionHandler(JobExplorer jobExplorer) {
	DeployerStepExecutionHandler handler =
		new DeployerStepExecutionHandler(this.context, jobExplorer, this.jobRepository);

	return handler;
}
```

=== Notes on developing a batch partitioned app for the Yarn platform

* When deploying partitioned apps on the Yarn platform be sure use the following dependency for the
Spring Cloud Yarn Deployer:
```
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-deployer-yarn</artifactId>
		</dependency>
```
* Also add the following property to your application.properties: `spring.yarn.container.keepContextAlive=false`.<commit_msg>Updated adding guava dependency<commit_after>
[[batch]]
= Batch

[[partintro]]
--
This section goes into more detail about Spring Cloud Task's integrations with Spring
Batch.  Tracking the association between a job execution and the task it was executed
within as well as remote partitioning via Spring Cloud Deployer are all covered within
this section.
--

[[batch-association]]
== Associating A Job Execution To The Task In Which It Was Executed

Spring Boot provides facilities for the execution of batch jobs easily within an über-jar.
Spring Boot's support of this functionality allows for a developer to execute multiple
batch jobs within that execution.  Spring Cloud Task provides the ability to associate the
execution of a job (a job execution) with a task's execution so that one can be traced
back to the other.

This functionality is accomplished by using the `TaskBatchExecutionListener`.  By default,
this listener is auto configured in any context that has both a Spring Batch Job configured
(via having a bean of type `Job` defined in the context) and the spring-cloud-task-batch jar
is available within the classpath.  The listener will be injected into all jobs.

[[batch-association-override]]
=== Overriding the TaskBatchExecutionListener

To prevent the listener from being injected into any batch jobs within the current context,
the autoconfiguration can be disabled via standard Spring Boot mechanisms.

To only have the listener injected into particular jobs within the context, the
`batchTaskExecutionListenerBeanPostProcessor` may be overridden and a list of job bean ids
can be provided:

```
public TaskBatchExecutionListenerBeanPostProcessor batchTaskExecutionListenerBeanPostProcessor() {
	TaskBatchExecutionListenerBeanPostProcessor postProcessor =
		new TaskBatchExecutionListenerBeanPostProcessor();

	postProcessor.setJobNames(Arrays.asList(new String[] {"job1", "job2"}));

	return postProcessor;
}
```

[[batch-partitioning]]
== Remote Partitioning

Spring Cloud Deployer provides facilities for launching Spring Boot based applications on
most cloud infrastructures.  The `DeployerPartitionHandler` and
`DeployerStepExecutionHandler` delegate the launching of worker step executions to Spring
Cloud Deployer.

To configure the `DeployerStepExecutionHandler`, a `Resource` representing the Spring Boot
über-jar to be executed, a `TaskLauncher`, and a `JobExplorer` are all required.  You can
configure any environment properties as well as the max number of workers to be executing
at once, the interval to poll for the results (defaults to 10 seconds), and a timeout
(defaults to -1 or no timeout).  An example of configuring this `PartitionHandler` would
look like the following:

```
@Bean
public PartitionHandler partitionHandler(TaskLauncher taskLauncher,
		JobExplorer jobExplorer) throws Exception {

	MavenProperties mavenProperties = new MavenProperties();
	mavenProperties.setRemoteRepositories(new HashMap<>(Collections.singletonMap("springRepo",
		new MavenProperties.RemoteRepository(repository))));

 	MavenResource resource =
		MavenResource.parse(String.format("%s:%s:%s",
				"io.spring.cloud",
				"partitioned-batch-job",
				"1.0.0.RELEASE"), mavenProperties);

	DeployerPartitionHandler partitionHandler =
		new DeployerPartitionHandler(taskLauncher, jobExplorer, resource, "workerStep");

	Map<String, String> environmentProperties = new HashMap<>();
	environmentProperties.put("spring.profiles.active", "worker");
	SimpleEnvironmentVariablesProvider environmentVariablesProvider = new SimpleEnvironmentVariablesProvider(this.environment);
	environmentVariablesProvider.setEnvironmentProperties(environmentProperties);
	partitionHandler.setEnvironmentVariablesProvider(environmentVariablesProvider);

	partitionHandler.setMaxWorkers(2);

	return partitionHandler;
}
```

The `Resource` to be executed is expected to be a Spring Boot über-jar with a
`DeployerStepExecutionHandler` configured as a `CommandLineRunner` in the current context.
The repository enumerated in the example above should be the location of the remote repository
from which the über-jar is located. Both the master and slave are expected to have
visibility into the same data store being used as the job repository and task repository.
Once the underlying infrastructure has bootstrapped the Spring Boot jar and Spring Boot
has launched the `DeployerStepExecutionHandler`, the step handler will execute the Step
requested.  An example of configuring the `DefaultStepExecutionHandler` is show below:

```
@Bean
public DeployerStepExecutionHandler stepExecutionHandler(JobExplorer jobExplorer) {
	DeployerStepExecutionHandler handler =
		new DeployerStepExecutionHandler(this.context, jobExplorer, this.jobRepository);

	return handler;
}
```

=== Notes on developing a batch partitioned app for the Yarn platform

* When deploying partitioned apps on the Yarn platform be sure use the following dependency for the
Spring Cloud Yarn Deployer:
```
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-deployer-yarn</artifactId>
		</dependency>
```
* Add the following dependency to the dependency management for a transient dependency required by Yarn:
```
	<dependencyManagement>
		<dependencies>
...
			<dependency>
				<groupId>com.google.guava</groupId>
				<artifactId>guava</artifactId>
				<version>18.0</version>
			</dependency>
		</dependencies>
...
	</dependencyManagement>
```
* Also add the following property to your application.properties: `spring.yarn.container.keepContextAlive=false`.
{'fraction_non_alphanumeric': 0.06399925905344077, 'fraction_numerical': 0.001944984717977216, 'mean_word_length': 5.957474226804123, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1365339', 'n_tokens_mistral': 2766, 'n_tokens_neox': 2584, 'n_words': 1140}