<commit_before>[[getting-started-kubernetes]]
== Getting Started - Kubernetes

http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] is a toolkit for building data integration and real-time data-processing pipelines.

Pipelines consist of Spring Boot apps, built with the Spring Cloud Stream or Spring Cloud Task microservice frameworks.
This makes Spring Cloud Data Flow suitable for a range of data-processing use cases, from import-export to event-streaming and predictive analytics.

This project provides support for using Spring Cloud Data Flow with Kubernetes as the runtime for these pipelines, with applications packaged as Docker images.

=== Installation

This section covers how to install the Spring Cloud Data Flow Server on a Kubernetes cluster.
Spring Cloud Data Flow depends on a few services and their availability.
For example, we need an RDBMS service for the application registry, stream and task repositories, and task management.
For streaming pipelines, we also need link:http://cloud.spring.io/spring-cloud-skipper/[Skipper] server for lifecycle management and a messaging middleware option, such as Apache Kafka or RabbitMQ.
In addition, if the analytics features are in use, we need a Redis service.

IMPORTANT: This guide describes setting up an environment for testing Spring Cloud Data Flow on Google Kubernetes Engine and is not meant to be a definitive guide for setting up a production environment. Feel free to adjust the suggestions to fit your test setup. Remember that a production environment requires much more consideration for persistent storage of message queues, high availability, security, and other concerns.

[NOTE]
=====
Currently, only applications registered with a `--uri` property pointing to a Docker resource are supported by the Data Flow Server for Kubernetes. However, we do support Maven resources for the `--metadata-uri` property, which is used to list the properties supported by each application. For example, the following application registration is valid:

====
[source,console,subs=attributes]
----
dataflow:>app register --type source --name time --uri docker://springcloudstream/time-source-rabbit:{docker-time-source-rabbit-version} --metadata-uri maven://org.springframework.cloud.stream.app:time-source-rabbit:jar:metadata:{docker-time-source-rabbit-version}
----
====

However, any application registered with a Maven, HTTP, or File resource for the executable jar (by using a `--uri` property prefixed with `maven://`, `http://` or `file://`) is *_not supported_*.
=====

==== Kubernetes Compatibility

The Spring Cloud Data Flow implementation for Kubernetes uses the link:https://github.com/spring-cloud/spring-cloud-deployer-kubernetes[Spring Cloud Deployer Kubernetes] library for orchestration.
Before you begin setting up a Kubernetes cluster, refer to the link:https://github.com/spring-cloud/spring-cloud-deployer-kubernetes#kubernetes-compatibility[compatibility matrix] to learn more about deployer and server compatibility against Kubernetes release versions.

The following outlines the compatibility between Spring Cloud Data Flow for Kubernetes Server and Kubernetes versions:

[source,console]
----
| Versions: SCDF K8S Server - K8S Deployer \ Kubernetes | 1.8.x | 1.9.x | 1.10.x |
|-------------------------------------------------------|-------|-------|--------|
| Server: 1.4.x - Deployer: 1.3.2                       | ✓     | ✓     | ✓     |
| Server: 1.5.x - Deployer: 1.3.6                       | ✓     | ✓     | ✓     |
| Server: 1.6.x - Deployer: 1.3.7                       | ✓     | ✓     | ✓     |
| Server: 1.7.x - Deployer: 1.3.9                       | ✓     | ✓     | ✓     |
|--------------------------------------------------------------------------------|
----

==== Create a Kubernetes Cluster
<commit_msg>Drop k8s from compat chart, add 1.11<commit_after>[[getting-started-kubernetes]]
== Getting Started - Kubernetes

http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] is a toolkit for building data integration and real-time data-processing pipelines.

Pipelines consist of Spring Boot apps, built with the Spring Cloud Stream or Spring Cloud Task microservice frameworks.
This makes Spring Cloud Data Flow suitable for a range of data-processing use cases, from import-export to event-streaming and predictive analytics.

This project provides support for using Spring Cloud Data Flow with Kubernetes as the runtime for these pipelines, with applications packaged as Docker images.

=== Installation

This section covers how to install the Spring Cloud Data Flow Server on a Kubernetes cluster.
Spring Cloud Data Flow depends on a few services and their availability.
For example, we need an RDBMS service for the application registry, stream and task repositories, and task management.
For streaming pipelines, we also need link:http://cloud.spring.io/spring-cloud-skipper/[Skipper] server for lifecycle management and a messaging middleware option, such as Apache Kafka or RabbitMQ.
In addition, if the analytics features are in use, we need a Redis service.

IMPORTANT: This guide describes setting up an environment for testing Spring Cloud Data Flow on Google Kubernetes Engine and is not meant to be a definitive guide for setting up a production environment. Feel free to adjust the suggestions to fit your test setup. Remember that a production environment requires much more consideration for persistent storage of message queues, high availability, security, and other concerns.

[NOTE]
=====
Currently, only applications registered with a `--uri` property pointing to a Docker resource are supported by the Data Flow Server for Kubernetes. However, we do support Maven resources for the `--metadata-uri` property, which is used to list the properties supported by each application. For example, the following application registration is valid:

====
[source,console,subs=attributes]
----
dataflow:>app register --type source --name time --uri docker://springcloudstream/time-source-rabbit:{docker-time-source-rabbit-version} --metadata-uri maven://org.springframework.cloud.stream.app:time-source-rabbit:jar:metadata:{docker-time-source-rabbit-version}
----
====

However, any application registered with a Maven, HTTP, or File resource for the executable jar (by using a `--uri` property prefixed with `maven://`, `http://` or `file://`) is *_not supported_*.
=====

==== Kubernetes Compatibility

The Spring Cloud Data Flow implementation for Kubernetes uses the link:https://github.com/spring-cloud/spring-cloud-deployer-kubernetes[Spring Cloud Deployer Kubernetes] library for orchestration.
Before you begin setting up a Kubernetes cluster, refer to the link:https://github.com/spring-cloud/spring-cloud-deployer-kubernetes#kubernetes-compatibility[compatibility matrix] to learn more about deployer and server compatibility against Kubernetes release versions.

The following outlines the compatibility between Spring Cloud Data Flow for Kubernetes Server and Kubernetes versions:

[source,console]
----
| Versions: SCDF K8S Server - K8S Deployer \ Kubernetes | 1.9.x | 1.10.x | 1.11.x |
|-------------------------------------------------------|-------|--------|--------|
| Server: 1.4.x - Deployer: 1.3.2                       | ✓     | ✓      | ✓     |
| Server: 1.5.x - Deployer: 1.3.6                       | ✓     | ✓      | ✓     |
| Server: 1.6.x - Deployer: 1.3.7                       | ✓     | ✓      | ✓     |
| Server: 1.7.x - Deployer: 1.3.9                       | ✓     | ✓      | ✓     |
|---------------------------------------------------------------------------------|
----

==== Create a Kubernetes Cluster

{'fraction_non_alphanumeric': 0.12480499219968799, 'fraction_numerical': 0.008190327613104524, 'mean_word_length': 3.1271459227467813, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7662828', 'n_tokens_mistral': 1973, 'n_tokens_neox': 1803, 'n_words': 856}