<commit_before>[id="ocp-4-1-release-notes"]
= {product-title} 4.1 release notes
include::modules/common-attributes.adoc[]
:context: release-notes

toc::[]

Red Hat {product-title} provides developers and IT organizations with a hybrid
cloud application platform for deploying both new and existing applications on
secure, scalable resources with minimal configuration and management overhead.
{product-title} supports a wide selection of programming languages and
frameworks, such as Java, JavaScript, Python, Ruby, and PHP.

Built on Red Hat Enterprise Linux and Kubernetes, {product-title}
provides a more secure and scalable multi-tenant operating system for today’s
enterprise-class applications, while delivering integrated application runtimes
and libraries. {product-title} enables organizations to meet security, privacy,
compliance, and governance requirements.

[id="ocp-4-1-about-this-release"]
== About this release

Red Hat {product-title}
(link:https://access.redhat.com/errata/RHBA-2019:0758[RHBA-2019:0758]) is now
available. This release uses Kubernetes 1.13. New features, changes, and known
issues that pertain to {product-title} {product-version} are included in this
topic.

Red Hat did not publicly release {product-title} 4.0 and, instead, is releasing
{product-title} {product-version} directly after version 3.11.

{product-title} {product-version} clusters are available at
https://cloud.openshift.com. The Unified Hybrid Cloud (UHC) application for
{product-title} allows you to deploy OpenShift clusters to either on-premise or
cloud environments.

{product-title} {product-version} is supported on Red Hat Enterprise Linux 7.6 and
later, as well as Red Hat Enterprise Linux CoreOS 4.1.

You must use {op-system-first} for the control plane, or master, machines and
can use either {op-system} or Red Hat Enterprise Linux 7.6 for compute, or worker,
machines.

[IMPORTANT]
====
Because only Red Hat Enterprise Linux version 7.6 is supported for compute
machines, you must not upgrade the Red Hat Enterprise Linux compute machines to
version 8.
====

You can install {product-title} {product-version} with installer-provisioned
infrastructure on Amazon Web Services (AWS) or user-provided infrastructure on
AWS, bare metal, or VMware vSphere hosts. If you use the installer-provisioned
infrastructure installation, the cluster provisions and manages all of the
cluster infrastructure for you.

{product-title} requires all machines, including the computer that you run the
installation process on, to have direct internet access to pull images for
platform containers and provide telemetry data to Red Hat. You cannot specify a
proxy server for {product-title}.

[id="ocp-4-1-release-acknowledgments"]
=== Acknowledgments

Red Hat Global Support Services would like to recognize <NAME>, <NAME>, and <NAME> for their outstanding contributions in evaluating and
testing {product-title} 4.1.

[id="ocp-41-deprecated-features"]
=== Deprecated features

Large changes to the underlying architecture and installation process are
applied in {product-title} 4.1, and many features from {product-title} 3.x are
now deprecated.

.Features Deprecated in Version 4.1
[cols="2",options="header"]
|====
|Feature |Justification

|Hawkular
|Replaced by cluster monitoring.

|Cassandra
|Replaced by cluster monitoring.

|Heapster
|Replaced by Prometheus adapter.

|Atomic Host
|Replaced by Red Hat Enterprise Linux CoreOS.

|System containers
|Replaced by Red Hat Enterprise Linux CoreOS.

|`projectatomic/docker-1.13` additional search registries
|CRI-O is the default container runtime for {product-title} 4.x on RHCOS and Red
Hat Enterprise Linux.

|`oc adm diagnostics`
|Operator-based diagnostics.

|`oc adm registry`
|Replaced by the Image Registry Operator.

|Custom strategy builds using Docker
|If you want to continue using custom builds, you should replace your Docker
invocations with Podman or Buildah. The custom build strategy will not be
removed, but the functionality changed significantly in {product-title} 4.1.

|Cockpit
|Improved {product-title} 4.1 web console.

|Stand-alone registry installations
|Quay is Red Hat's enterprise container image registry.

|DNSmasq
|CoreDNS is the default.

|External etcd nodes
|etcd is always on the cluster in {product-title} 4.1.

|CloudForms OpenShift Provider and Podified CloudForms
|Replaced by built-in management tooling.

|Volume Provisioning via installer
|Replaced by dynamic volumes or, if NFS is required, NFS provisioner.

|Blue-green installation method
|Ease of upgrade is a core value of {product-title} 4.1.

|OpenShift Service Broker and Service Catalog
|The Service Catalog and the OpenShift service brokers are being replaced over
the course of several future OpenShift 4 releases. Reference the Operator
Framework and Operator Lifecycle Manager (OLM) to continue providing your
applications to OpenShift 4 clusters. These new technologies provide many
benefits around complete management of the lifecycle of your application.

|`oc adm ca`
|Certificates are managed by Operators internally.

|`oc adm create-api-client-config`
.2+.^|Functions are managed by Operators internally.

|`oc adm create-bootstrap-policy-file`

|Web console
|The web console from {product-title} 3.11 has been replaced by a new web
console in {product-title} 4.1.

|====

[id="ocp-4-1-new-features-and-enhancements"]
== New features and enhancements

This release adds improvements related to the following components and concepts.

[id="ocp-4-1-operators"]
=== Operators

xref:../applications/operators/olm-what-operators-are.adoc#olm-what-operators-are[Operators]
are pieces of software that ease the operational complexity of running another
piece of software. They act like an extension of the software vendor’s
engineering team, watching over a Kubernetes environment (such as
{product-title}) and using its current state to make decisions in real time.
Advanced Operators are designed to handle upgrades seamlessly, react to failures
automatically, and not take shortcuts, like skipping a software backup process
to save time.

[id="ocp-4-1-operator-lifecycle-manager"]
==== Operator Lifecycle Manager (OLM)

This feature is now fully supported in {product-title} 4.1.

The OLM aids cluster administrators in installing, upgrading, and granting
access to Operators running on their cluster:

* Includes a catalog of curated Operators, with the ability to load other Operators into the cluster
* Handles rolling updates of all Operators to new versions
* Supports role-based access control (RBAC) for certain teams to use certain Operators

See
xref:../applications/operators/olm-understanding-olm.adoc#olm-understanding-olm[Understanding the Operator Lifecycle Manager (OLM)] for more information.

[id="ocp-4-1-installation-and-upgrade"]
=== Installation and upgrade

Red Hat {product-title} 4.1 has an installer-provisioned infrastructure, where
the installation program controls all areas of the installation process.
Installer-provisioned infrastructure also provides an opinionated best practices
deployment of {product-title} 4.1 for AWS instances only. This provides a
slimmer default installation, with incremental feature buy-in through
OperatorHub.

You can also install with a user-provided infrastructure on
AWS, bare metal, or vSphere hosts. If you use the installer-provisioned
infrastructure installation, the cluster provisions and manages all of the
cluster infrastructure for you.

Upgrading from 3.x to 4.1 is currently not available. You must perform a new
installation of {product-title} 4.1.

Easy, over-the-air upgrades for asynchronous z-stream releases of
{product-title} 4.x is available. Cluster administrators can upgrade using the
*Cluster Settings* tab in the web console.
See
xref:../updating/updating-cluster.adoc#updating-cluster[Updating a cluster]
for more information.

[id="ocp-4-1-operator-hub"]
==== OperatorHub

OperatorHub is available to administrators and helps with easy discovery and
installation of all optional components and applications. It includes offerings
from Red Hat products, Red Hat partners, and the community.

.Features provided with base installation and OperatorHub
[cols="3",options="header"]
|===
|Feature |New installer |OperatorHub

|Console and authentication
|* [x]
| -

|Prometheus cluster monitoring
|* [x]
| -

|Over-the-air updates
|* [x]
| -

|Machine management
|* [x]
| -

|Optional service brokers
| -
|* [x]

|Optional {product-title} components
| -
|* [x]

|Red Hat product Operators
| -
|* [x]

|Red Hat partner Operators
| -
|* [x]

|Community Operators
| -
|* [x]

|===

See
xref:../applications/operators/olm-understanding-operatorhub.adoc#olm-understanding-operatorhub[Understanding the OperatorHub] for more information.

[id="ocp-4-1-storage"]
=== Storage

Storage support in {product-title} 4.1 is the same as {product-title} 3.11 with
the exception of the following available in Technology Preview: EFS (CSI Driver
handled via Amazon), Manila provisioner/operator, and Snapshot.

[id="ocp-4-1-scale"]
=== Scale

[id="ocp-4-1-scale-cluster-limits"]
==== Cluster limits

Updated guidance around
xref:../scalability_and_performance/planning-your-environment-according-to-object-limits.adoc[Cluster
Limits] for {product-title} 4.1 is now available.

Use the link:https://access.redhat.com/labs/ocplimitscalculator/[{product-title}
Limit Calculator] to estimate cluster limits for your environment.

[id="ocp-4-1-node-tuning-operator"]
==== Node Tuning Operator

The
xref:../scalability_and_performance/using-node-tuning-operator.adoc#using-node-tuning-operator[Node
Tuning Operator] is now part of a standard {product-title} installation in
version 4.1 and later.

The Node Tuning Operator helps you manage node-level tuning by orchestrating the
tuned daemon. The majority of high-performance applications require some level
of kernel tuning. The Node Tuning Operator provides a unified management
interface to users of node-level sysctls and more flexibility to add custom
tuning, which is currently a Technology Preview feature, specified by user
needs. The Operator manages the containerized tuned daemon for OpenShift
Container Platform as a Kubernetes DaemonSet. It ensures the custom tuning
specification is passed to all containerized tuned daemons running in the
cluster in the format that the daemons understand. The daemons run on all nodes
in the cluster, one per node.

[id="ocp-4-1-cluster-monitoring"]
=== Cluster monitoring

[id="ocp-4-1-autoscale-pods-horizontally-based-on-custom-metrics-api"]
==== Autoscale pods horizontally based on the custom metrics API (Technology Preview)

This feature, currently in Technology Preview, enables you to configure
horizontal pod autoscaling (HPA) based on the custom metrics API. As part of
this Technology Preview, a Prometheus Adapter component can be deployed to
provide any app metrics for the custom metrics API.

Limitations:

* The adapter only connects to a single Prometheus instance (or a set of
load-balanced replicas, using Kubernetes services).
* Manually deploying adapter and configuring it to use Prometheus.
* Syntax for the Prometheus Adapter configuration could change in the future.
* The `APIService` configuration to wire Kubernetes' API aggregation to the
instance of the custom metrics adapter will be overwritten in future releases,
if {product-title} ships an out-of-the-box custom metrics adapter.

[id="ocp-4-1-cluster-monitoring-alerting-UI"]
==== New alerting user interface

An alerting UI is now natively integrated into the {product-title} web console.
You can now view cluster-level alerts and alerting rules from a single place, as
well as configure silences.

[id="ocp-4-1-cluster-monitoring-telemeter"]
==== Telemeter

Telemeter collects anonymized cluster-related metrics to proactively help
customers with their {product-title} clusters. This helps:

* Gather crucial health metrics of {product-title} installations.
* Enable a viable feedback loop of {product-title} upgrades.
* Gather the cluster's number of nodes per cluster and their size (CPU cores and
RAM).
* Gather the size of etcd.
* Gather details about the health condition and status for any OpenShift framework
component installed on an OpenShift cluster.

[id="ocp-4-1-cluster-monitoring-autoscale"]
==== Autoscale pods horizontally based on the resource metrics API

By default, OpenShift Cluster Monitoring exposes CPU and Memory utilization
through the Kubernetes resource metrics API. There is no longer a requirement to
install a separate metrics server.

[id="ocp-4-1-developer-experience"]
=== Developer experience

[id="ocp-4-1-code-ready-containers"]
==== Code ready containers

A local desktop instance of {product-title} 4.1 replaces the functions of `oc
cluster` commands, Minishift, and CDK. {product-title} 4.1 focuses on ease of
access and native experience, with a native installation program on macOS and Microsoft
Windows, native hypervisor support, and tray icon integration.

[id="ocp-4-1-multistage-builds"]
==== Multi-stage Dockerfile Builds Generally Available

Multi-stage Dockerfiles are now supported in all `Docker` strategy builds.

[id="ocp-4-1-registry"]
=== Registry

[id="ocp-4-1-registry-managed-by-operator"]
==== The registry is now managed by an Operator

The registry is now managed by an Operator instead of `oc adm registry`.

[id="ocp-4-1-networking"]
=== Networking

[id="ocp-4-1-cno"]
==== Cluster Network Operator (CNO)

The cluster network is now configured and managed by an Operator. The Operator
upgrades and monitors the cluster network.

[id="ocp-4-1-openshift-sdn"]
==== OpenShift SDN

The default mode is now `NetworkPolicy`.

[id="ocp-4-1-multus"]
==== Multus

Multus is a meta plug-in for Kubernetes Container Network Interface (CNI), which
enables a user to create multiple network interfaces per pod.

[id="ocp-4-1-web-console"]
=== Web console

[id="ocp-4-1-developer-catalog"]
==== Developer Catalog

{product-title} 4.1 features a redesigned Developer Catalog that brings all of
the new Operators and existing broker services together, with new ways to
discover, sort, and understand how to best use each type of offering. The
Developer Catalog is the entry point for a developer to access all services
available to them. It merges all capabilities from Operators, the Service
Catalog, brokers, and Source-to-Image (S2I).

[id="ocp-4-1-new-management-screens"]
==== New management screens

New management screens in {product-title} 4.1 support automated operations.
Examples include the management of machine sets and machines, taints,
tolerations, and cluster settings.

[id="ocp-4-1-security"]
=== Security

In {product-title} 4.1, an Operator is utilized to install, configure, and manage
the certificate signing server. Certificates are managed as secrets, and are
stored and encrypted in etcd.

[id="ocp-4-1-notable-technical-changes"]
== Notable technical changes

{product-title} 4.1 introduces the following notable technical changes.

[discrete]
[id="ocp-4-1-buildah"]
==== Builds powered by buildah

Source and Docker strategy builds are now performed by buildah instead of the
docker daemon.

[discrete]
[id="ocp-4-1-SecurityContextConstraints"]
==== SecurityContextConstraints

`SecurityContextConstraints` now only exist in the `security.openshift.io` group.

[discrete]
[id="ocp-4-1-Service-CA-bundle"]
==== Service CA bundle changes

Pods can trust cluster-created certificates, which are only signed for internal
DNS names, by using a CA bundle that is automatically injected into any
configMap annotated with `service.beta.openshift.io/inject-cabundle=true`. The
CA bundle will be made available as PEM-encoded data under the key
`service-ca.crt`. This annotation results in wiping out existing content in the
configMap.

Pods that currently consume the service-serving CA bundle from
`/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt` should migrate to
obtaining the CA bundle from a configMap annotated with
`service.beta.openshift.io/inject-cabundle=true`.

The `/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt` file is now
deprecated and will be removed in a future release.

[discrete]
[id="ocp-4-1-service-broker-service-catalog-deprecation"]
==== OpenShift Service Broker and Service Catalog deprecation

The Service Catalog and the OpenShift service brokers are being replaced over
the course of several future OpenShift 4 releases. Red Hat will be deprecating
the Template Service Broker and OpenShift Ansible Broker once important
dependent content is ported to new Operator-driven solutions. Users are
encouraged to look at the Operator Framework and Operator Lifecycle Manager
(OLM) to continue providing their applications to OpenShift 4 clusters. These
new technologies provide many benefits around complete management of the
lifecycle of your application.

[discrete]
[id="ocp-4-1-service-catalog-no-longer-installed-by-default"]
==== Service Catalog no longer installed by default

The Service Catalog is not installed by default in {product-title} 4.1. You must
install it if you plan on using any of the services from the OpenShift Ansible
broker or template service broker. In {product-title} 4.1, the Service Catalog
API server is installed into the `openshift-service-catalog-apiserver` namespace
and the Service Catalog controller manager is installed into the
`openshift-service-catalog-controller-manager` namespace. In {product-title}
3.11, both of these components were installed into the `kube-service-catalog`
namespace.

[discrete]
[id="ocp-4-1-template-service-broker-no-longer-installed-by-default"]
==== Template Service Broker no longer installed by default

The Template Service Broker is not installed by default in {product-title} 4.1.
Cluster administrators can install the Template Service Broker if users will
need access to template applications from the web console.

[discrete]
[id="ocp-4-1-openshift-ansible-service-broker-no-longer-installed-by-default"]
==== OpenShift Ansible Service Broker no longer installed by default

The OpenShift Ansible Service Broker is not installed by default in
{product-title} 4.1.

[discrete]
[id="ocp-4-1-oc-adm-commands-deprecated"]
==== Several `oc adm` commands are now deprecated

Deprecated `oc adm` commands include:

* `oc adm create-master-certs` - Create keys and certificates
* `oc adm create-key-pair` - Create an RSA key pair.
* `oc adm create-server-cert` - Create a key and server certificate.
* `oc adm create-signer-cert` - Create a self-signed CA.

[discrete]
[id="ocp-4-1-configurability-of-imagepolicyadmission"]
==== The configurability of the imagepolicyadmission plug-in is not present

The configurability of the `imagepolicyadmission` plug-in is not present in
{product-title} 4.1. The plug-in runs, but currently only with default
configuration. Configuring it requires using the unsupported overrides
mechanism.

[id="ocp-4-1-technology-preview"]
== Technology Preview features

Some features in this release are currently in Technology Preview. These
experimental features are not intended for production use. Note the
following scope of support on the Red Hat Customer Portal for these features:

link:https://access.redhat.com/support/offerings/techpreview[Technology Preview
Features Support Scope]

In the table below, features marked *TP* indicate _Technology Preview_ and
features marked *GA* indicate _General Availability_.

.Technology Preview Tracker
[cols="3",options="header"]
|====
|Feature |OCP 3.11 |OCP 4.1

|Prometheus Cluster Monitoring
|GA
|GA

|Local Storage Persistent Volumes
|TP
|TP

|CRI-O for runtime pods
|GA* footnoteref:[disclaimer, Features marked with `*` indicate delivery in a z-stream patch.]
|GA

|Tenant Driven Snapshotting
|TP
|TP

|`oc` CLI Plug-ins
|TP
|TP

|Service Catalog
|GA
|GA

|Template Service Broker
|GA
|GA

|OpenShift Ansible Service Broker
|GA
|GA

|Network Policy
|GA
|GA

|Multus
|-
|GA

|Service Catalog Initial Experience
|GA
|GA

|New Add Project Flow
|GA
|GA

|Search Catalog
|GA
|GA

|Cron Jobs
|GA
|GA

|Kubernetes Deployments
|GA
|GA

|StatefulSets
|GA
|GA

|Explicit Quota
|GA
|GA

|Mount Options
|GA
|GA

|System Containers for Docker, CRI-O
|-
|-

|Hawkular Agent
|-
|-

|Pod PreSets
|-
|-

|experimental-qos-reserved
|TP
|TP

|Pod sysctls
|GA
|GA. See xref:ocp-4-1-known-issues[Known issues] for current limitations.

|Central Audit
|GA
|GA

|Static IPs for External Project Traffic
|GA
|GA

|Template Completion Detection
|GA
|GA

|`replicaSet`
|GA
|GA

|Fluentd Mux
|TP
|TP

|Clustered MongoDB Template
|-
|-

|Clustered MySQL Template
|-
|-

|Image Streams with Kubernetes Resources
|GA
|GA

|Device Manager
|GA
|GA

|Persistent Volume Resize
|GA
|GA

|Huge Pages
|GA
|GA

|CPU Pinning
|GA
|GA

|Admission Webhooks
|TP
|GA

|External provisoner for AWS EFS
|TP
|TP

|Pod Unidler
|TP
|TP

|Node Problem Detector
|TP
|TP

|Ephemeral Storage Limit/Requests
|TP
|TP

|Descheduler
|TP
|TP

|CephFS
|TP
|TP

|Podman
|TP
|TP

|Kuryr CNI Plug-in
|TP
|TP

|Istio
|TP
|TP

|Sharing Control of the PID Namespace
|TP
|TP

|Manila Provisioner
|TP
|TP

|Cluster Administrator console
|GA
|GA

|Cluster Autoscaling (AWS Only)
|GA
|GA

|Container Storage Interface (CSI)
|TP
|TP

|Operator Lifecycle Manager
|TP
|GA

|Red Hat OpenShift Service Mesh
|TP
|TP

|"Fully Automatic" Egress IPs
|GA
|GA

|Pod Priority and Preemption
|GA
|GA

|Multi-stage builds in Dockerfiles
|TP
|GA

|OVN
|
|TP

|HPA custom metrics adapter based on Prometheus
|
|TP

|Machine health checks
|
|TP
|====

[id="ocp-4-1-known-issues"]
== Known issues

* Unsafe sysctl cannot be used in {product-title} 4.1.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1690754[*BZ#1690754*])

* If an instance is removed from the cloud provider (either via a user deletion,
or cloud-provider event of some kind), and the machine-object is reconciled
again for some reason, the machine-controller might determine the instance no
longer exists and attempt to create the instance. This is undocumented behavior
and should not be relied upon for workflows. This operation might interfere with
current or future components, such as the node-health-checker.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712068[*BZ#1712068*])

* Builds which use shell substitution to populate an environment variable may fail.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712245[*BZ#1712245*])

* When deleting a machine-object, either directly or by scaling down the owning
machine-set, if the associated node has already been deleted somehow (possibly
by a cluster administrator), the machine-controller will fail to successfully
delete the backing cloud instance, and the machine-object will be stuck in
`deleting` status.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713061[*BZ#1713061*])

* Querying `Jolokia` on `JBoss EAP` images fails as the result of empty
certificates presented to the client. The `Jolokia` SSL client authentication
will fail and may require a username and password challenge if enabled.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1708640[*BZ#1708640*])

* The `TokenRequest` API is not available in {product-title} 4.1. Requesting a
`ServiceAccountTokenVolumeProjection` volume is not available in {product-title}
4.1. The kubelet will present an error if a
`ServiceAccountTokenVolumeProjection` is used.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1695196[*BZ#1695196*])

* The `es nodeCount` in ElasticSearch CRD instances can not be scaled up if
deployed with three nodes. Scaling works correctly if deployed with one, two,
four, five or six ElasticSearch CRD instances.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712955[*BZ#1712955*])

* ElasticSearch instances created from OperatorHub deploy with a one CPU limit,
even though no limits are specified.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1710657[*BZ#1710657*])

* After an AWS installation, the `openshiftClustID` tag is not present.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1685089[*BZ#1685089*])

* scc(CRD) resources can not be upgraded by using the `oc patch` and `oc edit`
commands. As a result, strategic merges also fail.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1707679[*BZ#1707679*])

* The {product-title} 4.1 registry service utilizes port 5000 instead of port 443.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1701422[*BZ#1701422*])

* Machineset scaling in AWS environments may fail if the resources requested are
unavailable in the chosen Availability Zone.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713157[*BZ#1713157*])

* Using `OAuth` endpoints after configuring ingress wildcard certificates from
custom PKIs result in login errors.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712525[*BZ#1712525*])

* Source-to-Image (S2I) builds in {product-title} 4.1 may take longer to complete.
This is because {product-title} 4.1 does not utilize a shared image cache for
building images like in previous versions of {product-title}.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1685352[*BZ#1685352*])

* The `cloud-credential-operator` may crash on clusters with large numbers of
projects or namespaces due to memory limitations.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1711402[*BZ#1711402*])

* The Marketplace can not detect `opsrc` after a cluster upgrade is performed. As
a result, the `csc` packages are empty and can not download `packagemanifests`.
Marketplace can repair this problem approximately one hour later when it syncs
again.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1695550[*BZ#1695550*])

* In {product-title} 4.1, `oc` and `openshift-install` version may have a dirty
`GitTreeState:` when checking the `oc version`.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1715001[*BZ#1715001*])

* In AWS environments, if a master node is stopped, the `kubeapiserver` cannot be
deployed due a pod stuck in a `Pending` status.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713292[*BZ#1713292*])

* All m4 instances on AWS fail to verify
(https://access.redhat.com/security/cve/cve-2019-11091[*CVE-2019-1109*]) using
Broadwell CPU model 79 (type m4) because the `microcode_ctl` will not update.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1710981[*BZ#1710981*])

* New ElasticSearch deployments can not be created if another ElasticSearch
deployment is stuck in a deleting state.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1711044[*BZ#1711044*])

* There is no _Open Java Console_ link available in the {product-title} 4.1 web
console.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713656[*BZ#1713656*])

* The `openshift-cluster-node-tuning-operator` may generate a large number of
secrets after several days of uptime.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1714484[*BZ#1714484*])

* Autoscaling for Memory Utilization is not working as expected. Creating HPA for
memory-based autoscaling is failing while looking for resources.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1707785[*BZ#1707785*])

* After successfully performing updates, `oc clusterversion` may report that the
update could not be applied.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1711964[*BZ#1711964*])

* The installer may have a `0` return code when hitting a FATAL event.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712409[*BZ#1712409*])

[id="ocp-4-1-asynchronous-errata-updates"]
== Asynchronous errata updates

Security, bug fix, and enhancement updates for {product-title} 4.1 are released
as asynchronous errata through the Red Hat Network. All {product-title} 4.1
errata is https://access.redhat.com/downloads/content/290/[available on the Red
Hat Customer Portal]. See the
https://access.redhat.com/support/policy/updates/openshift[{product-title}
Life Cycle] for more information about asynchronous errata.

Red Hat Customer Portal users can enable errata notifications in the account
settings for Red Hat Subscription Management (RHSM). When errata notifications
are enabled, users are notified via email whenever new errata relevant to their
registered systems are released.

[NOTE]
====
Red Hat Customer Portal user accounts must have systems registered and consuming
{product-title} entitlements for {product-title} errata notification
emails to generate.
====

This section will continue to be updated over time to provide notes on
enhancements and bug fixes for future asynchronous errata releases of
{product-title} 4.1. Versioned asynchronous releases, for example with the form
{product-title} 4.1.z, will be detailed in subsections. In addition, releases
in which the errata text cannot fit in the space provided by the advisory will
be detailed in subsections that follow.

[IMPORTANT]
====
For any {product-title} release, always review the instructions on
xref:../updating/updating-cluster.adoc#updating-cluster[updating your cluster]
properly.
====

[[ocp-4-1]]
=== RHBA-2019:0758 - {product-title} 4.1 Image Release advisory

Issued: 2019-06-04

{product-title} release 4.1 is now available. The list of packages
included in the update are documented in the
link:https://access.redhat.com/errata/RHBA-2019:1173[RHBA-2019:1173] advisory.
The container images and bug fixes included in the update are provided by the
link:https://access.redhat.com/errata/RHBA-2019:0758[RHBA-2019:0758] advisory.

Space precluded documenting all of the container images for this release in the
advisory. See the following sections for notes on the container images in this
release.

[[ocp-4-1-container-images]]
==== Images

This release updates the Red Hat Container Registry
(_registry.access.redhat.com_) with the following images:

----
openshift4/apb-base
openshift4/apb-tools
openshift4/mariadb-apb
openshift4/mediawiki-apb
openshift4/mediawiki
openshift4/mysql-apb
openshift4/ose-ansible-operator
openshift4/ose-ansible-service-broker
openshift4/ose-ansible-service-broker-operator
openshift4/ose-aws-machine-controllers
openshift4/ose-cli-artifacts
openshift4/ose-cli
openshift4/ose-cloud-credential-operator
openshift4/ose-cluster-authentication-operator
openshift4/ose-cluster-autoscaler
openshift4/ose-cluster-autoscaler-operator
openshift4/ose-cluster-bootstrap
openshift4/ose-cluster-capacity
openshift4/ose-cluster-config-operator
openshift4/ose-cluster-dns-operator
openshift4/ose-cluster-image-registry-operator
openshift4/ose-cluster-ingress-operator
openshift4/ose-cluster-kube-apiserver-operator
openshift4/ose-cluster-kube-controller-manager-operator
openshift4/ose-cluster-kube-scheduler-operator
openshift4/ose-cluster-logging-operator
openshift4/ose-cluster-machine-approver
openshift4/ose-cluster-monitoring-operator
openshift4/ose-cluster-network-operator
openshift4/ose-cluster-node-tuned
openshift4/ose-cluster-node-tuning-operator
openshift4/ose-cluster-openshift-apiserver-operator
openshift4/ose-cluster-openshift-controller-manager-operator
openshift4/ose-cluster-samples-operator
openshift4/ose-cluster-storage-operator
openshift4/ose-cluster-svcat-apiserver-operator
openshift4/ose-cluster-svcat-controller-manager-operator
openshift4/ose-cluster-version-operator
openshift4/ose-configmap-reloader
openshift4/ose-console
openshift4/ose-console-operator
openshift4/ose-container-networking-plugins-supported
openshift4/ose-container-networking-plugins-unsupported
openshift4/ose-coredns
openshift4/ose-deployer
openshift4/ose-descheduler
openshift4/ose-descheduler-operator
openshift4/ose-docker-builder
openshift4/ose-docker-registry
openshift4/ose-egress-dns-proxy
openshift4/ose-egress-http-proxy
openshift4/ose-egress-router
openshift4/ose-elasticsearch-operator
openshift4/ose-etcd
openshift4/ose-grafana
openshift4/ose-haproxy-router
openshift4/ose-hyperkube
openshift4/ose-hypershift
openshift4/ose-installer-artifacts
openshift4/ose-installer
openshift4/ose-jenkins-agent-base
openshift4/ose-jenkins-agent-maven
openshift4/ose-jenkins-agent-nodejs
openshift4/ose-jenkins
openshift4/ose-k8s-prometheus-adapter
openshift4/ose-keepalived-ipfailover
openshift4/ose-kube-client-agent
openshift4/ose-kube-etcd-signer-server
openshift4/ose-kube-proxy
openshift4/ose-kube-rbac-proxy
openshift4/ose-kube-state-metrics
openshift4/ose-libvirt-machine-controllers
openshift4/ose-logging-curator5
openshift4/ose-logging-elasticsearch5
openshift4/ose-logging-eventrouter
openshift4/ose-logging-fluentd
openshift4/ose-logging-kibana5
openshift4/ose-machine-api-operator
openshift4/ose-machine-config-controller
openshift4/ose-machine-config-daemon
openshift4/ose-machine-config-operator
openshift4/ose-machine-config-server
openshift4/ose-multus-cni
openshift4/ose-must-gather
openshift4/ose-node
openshift4/ose-oauth-proxy
openshift4/ose-operator-lifecycle-manager
openshift4/ose-operator-marketplace
openshift4/ose-operator-registry
openshift4/ose-ovn-kubernetes
openshift4/ose-pod
openshift4/ose-prometheus-alertmanager
openshift4/ose-prometheus-config-reloader
openshift4/ose-prometheus
openshift4/ose-prometheus-node-exporter
openshift4/ose-prometheus-operator
openshift4/ose-prom-label-proxy
openshift4/ose-recycler
openshift4/ose-service-ca-operator
openshift4/ose-service-catalog
openshift4/ose-setup-etcd-environment
openshift4/ose-sriov-cni
openshift4/ose-sriov-network-device-plugin
openshift4/ose-telemeter
openshift4/ose-template-service-broker-operator
openshift4/ose-template-service-broker
openshift4/ose-tests
openshift4/postgresql-apb
openshift4/ose-haproxy-router-base
----
<commit_msg>Add oc adm reconcile-scc to deprecated list (#15313)<commit_after>[id="ocp-4-1-release-notes"]
= {product-title} 4.1 release notes
include::modules/common-attributes.adoc[]
:context: release-notes

toc::[]

Red Hat {product-title} provides developers and IT organizations with a hybrid
cloud application platform for deploying both new and existing applications on
secure, scalable resources with minimal configuration and management overhead.
{product-title} supports a wide selection of programming languages and
frameworks, such as Java, JavaScript, Python, Ruby, and PHP.

Built on Red Hat Enterprise Linux and Kubernetes, {product-title}
provides a more secure and scalable multi-tenant operating system for today’s
enterprise-class applications, while delivering integrated application runtimes
and libraries. {product-title} enables organizations to meet security, privacy,
compliance, and governance requirements.

[id="ocp-4-1-about-this-release"]
== About this release

Red Hat {product-title}
(link:https://access.redhat.com/errata/RHBA-2019:0758[RHBA-2019:0758]) is now
available. This release uses Kubernetes 1.13. New features, changes, and known
issues that pertain to {product-title} {product-version} are included in this
topic.

Red Hat did not publicly release {product-title} 4.0 and, instead, is releasing
{product-title} {product-version} directly after version 3.11.

{product-title} {product-version} clusters are available at
https://cloud.openshift.com. The Unified Hybrid Cloud (UHC) application for
{product-title} allows you to deploy OpenShift clusters to either on-premise or
cloud environments.

{product-title} {product-version} is supported on Red Hat Enterprise Linux 7.6 and
later, as well as Red Hat Enterprise Linux CoreOS 4.1.

You must use {op-system-first} for the control plane, or master, machines and
can use either {op-system} or Red Hat Enterprise Linux 7.6 for compute, or worker,
machines.

[IMPORTANT]
====
Because only Red Hat Enterprise Linux version 7.6 is supported for compute
machines, you must not upgrade the Red Hat Enterprise Linux compute machines to
version 8.
====

You can install {product-title} {product-version} with installer-provisioned
infrastructure on Amazon Web Services (AWS) or user-provided infrastructure on
AWS, bare metal, or VMware vSphere hosts. If you use the installer-provisioned
infrastructure installation, the cluster provisions and manages all of the
cluster infrastructure for you.

{product-title} requires all machines, including the computer that you run the
installation process on, to have direct internet access to pull images for
platform containers and provide telemetry data to Red Hat. You cannot specify a
proxy server for {product-title}.

[id="ocp-4-1-release-acknowledgments"]
=== Acknowledgments

Red Hat Global Support Services would like to recognize <NAME>, JooHo
Lee, and <NAME> for their outstanding contributions in evaluating and
testing {product-title} 4.1.

[id="ocp-41-deprecated-features"]
=== Deprecated features

Large changes to the underlying architecture and installation process are
applied in {product-title} 4.1, and many features from {product-title} 3.x are
now deprecated.

.Features Deprecated in Version 4.1
[cols="2",options="header"]
|====
|Feature |Justification

|Hawkular
|Replaced by cluster monitoring.

|Cassandra
|Replaced by cluster monitoring.

|Heapster
|Replaced by Prometheus adapter.

|Atomic Host
|Replaced by Red Hat Enterprise Linux CoreOS.

|System containers
|Replaced by Red Hat Enterprise Linux CoreOS.

|`projectatomic/docker-1.13` additional search registries
|CRI-O is the default container runtime for {product-title} 4.x on RHCOS and Red
Hat Enterprise Linux.

|`oc adm diagnostics`
|Operator-based diagnostics.

|`oc adm registry`
|Replaced by the Image Registry Operator.

|Custom strategy builds using Docker
|If you want to continue using custom builds, you should replace your Docker
invocations with Podman or Buildah. The custom build strategy will not be
removed, but the functionality changed significantly in {product-title} 4.1.

|Cockpit
|Improved {product-title} 4.1 web console.

|Stand-alone registry installations
|Quay is Red Hat's enterprise container image registry.

|DNSmasq
|CoreDNS is the default.

|External etcd nodes
|etcd is always on the cluster in {product-title} 4.1.

|CloudForms OpenShift Provider and Podified CloudForms
|Replaced by built-in management tooling.

|Volume Provisioning via installer
|Replaced by dynamic volumes or, if NFS is required, NFS provisioner.

|Blue-green installation method
|Ease of upgrade is a core value of {product-title} 4.1.

|OpenShift Service Broker and Service Catalog
|The Service Catalog and the OpenShift service brokers are being replaced over
the course of several future OpenShift 4 releases. Reference the Operator
Framework and Operator Lifecycle Manager (OLM) to continue providing your
applications to OpenShift 4 clusters. These new technologies provide many
benefits around complete management of the lifecycle of your application.

|`oc adm ca`
|Certificates are managed by Operators internally.

|`oc adm create-api-client-config`
.2+.^|Functions are managed by Operators internally.

|`oc adm create-bootstrap-policy-file`

|`oc adm policy reconcile-sccs`
|Functions are managed by `openshift-apiserver` internally.

|Web console
|The web console from {product-title} 3.11 has been replaced by a new web
console in {product-title} 4.1.

|====

[id="ocp-4-1-new-features-and-enhancements"]
== New features and enhancements

This release adds improvements related to the following components and concepts.

[id="ocp-4-1-operators"]
=== Operators

xref:../applications/operators/olm-what-operators-are.adoc#olm-what-operators-are[Operators]
are pieces of software that ease the operational complexity of running another
piece of software. They act like an extension of the software vendor’s
engineering team, watching over a Kubernetes environment (such as
{product-title}) and using its current state to make decisions in real time.
Advanced Operators are designed to handle upgrades seamlessly, react to failures
automatically, and not take shortcuts, like skipping a software backup process
to save time.

[id="ocp-4-1-operator-lifecycle-manager"]
==== Operator Lifecycle Manager (OLM)

This feature is now fully supported in {product-title} 4.1.

The OLM aids cluster administrators in installing, upgrading, and granting
access to Operators running on their cluster:

* Includes a catalog of curated Operators, with the ability to load other Operators into the cluster
* Handles rolling updates of all Operators to new versions
* Supports role-based access control (RBAC) for certain teams to use certain Operators

See
xref:../applications/operators/olm-understanding-olm.adoc#olm-understanding-olm[Understanding the Operator Lifecycle Manager (OLM)] for more information.

[id="ocp-4-1-installation-and-upgrade"]
=== Installation and upgrade

Red Hat {product-title} 4.1 has an installer-provisioned infrastructure, where
the installation program controls all areas of the installation process.
Installer-provisioned infrastructure also provides an opinionated best practices
deployment of {product-title} 4.1 for AWS instances only. This provides a
slimmer default installation, with incremental feature buy-in through
OperatorHub.

You can also install with a user-provided infrastructure on
AWS, bare metal, or vSphere hosts. If you use the installer-provisioned
infrastructure installation, the cluster provisions and manages all of the
cluster infrastructure for you.

Upgrading from 3.x to 4.1 is currently not available. You must perform a new
installation of {product-title} 4.1.

Easy, over-the-air upgrades for asynchronous z-stream releases of
{product-title} 4.x is available. Cluster administrators can upgrade using the
*Cluster Settings* tab in the web console.
See
xref:../updating/updating-cluster.adoc#updating-cluster[Updating a cluster]
for more information.

[id="ocp-4-1-operator-hub"]
==== OperatorHub

OperatorHub is available to administrators and helps with easy discovery and
installation of all optional components and applications. It includes offerings
from Red Hat products, Red Hat partners, and the community.

.Features provided with base installation and OperatorHub
[cols="3",options="header"]
|===
|Feature |New installer |OperatorHub

|Console and authentication
|* [x]
| -

|Prometheus cluster monitoring
|* [x]
| -

|Over-the-air updates
|* [x]
| -

|Machine management
|* [x]
| -

|Optional service brokers
| -
|* [x]

|Optional {product-title} components
| -
|* [x]

|Red Hat product Operators
| -
|* [x]

|Red Hat partner Operators
| -
|* [x]

|Community Operators
| -
|* [x]

|===

See
xref:../applications/operators/olm-understanding-operatorhub.adoc#olm-understanding-operatorhub[Understanding the OperatorHub] for more information.

[id="ocp-4-1-storage"]
=== Storage

Storage support in {product-title} 4.1 is the same as {product-title} 3.11 with
the exception of the following available in Technology Preview: EFS (CSI Driver
handled via Amazon), Manila provisioner/operator, and Snapshot.

[id="ocp-4-1-scale"]
=== Scale

[id="ocp-4-1-scale-cluster-limits"]
==== Cluster limits

Updated guidance around
xref:../scalability_and_performance/planning-your-environment-according-to-object-limits.adoc[Cluster
Limits] for {product-title} 4.1 is now available.

Use the link:https://access.redhat.com/labs/ocplimitscalculator/[{product-title}
Limit Calculator] to estimate cluster limits for your environment.

[id="ocp-4-1-node-tuning-operator"]
==== Node Tuning Operator

The
xref:../scalability_and_performance/using-node-tuning-operator.adoc#using-node-tuning-operator[Node
Tuning Operator] is now part of a standard {product-title} installation in
version 4.1 and later.

The Node Tuning Operator helps you manage node-level tuning by orchestrating the
tuned daemon. The majority of high-performance applications require some level
of kernel tuning. The Node Tuning Operator provides a unified management
interface to users of node-level sysctls and more flexibility to add custom
tuning, which is currently a Technology Preview feature, specified by user
needs. The Operator manages the containerized tuned daemon for OpenShift
Container Platform as a Kubernetes DaemonSet. It ensures the custom tuning
specification is passed to all containerized tuned daemons running in the
cluster in the format that the daemons understand. The daemons run on all nodes
in the cluster, one per node.

[id="ocp-4-1-cluster-monitoring"]
=== Cluster monitoring

[id="ocp-4-1-autoscale-pods-horizontally-based-on-custom-metrics-api"]
==== Autoscale pods horizontally based on the custom metrics API (Technology Preview)

This feature, currently in Technology Preview, enables you to configure
horizontal pod autoscaling (HPA) based on the custom metrics API. As part of
this Technology Preview, a Prometheus Adapter component can be deployed to
provide any app metrics for the custom metrics API.

Limitations:

* The adapter only connects to a single Prometheus instance (or a set of
load-balanced replicas, using Kubernetes services).
* Manually deploying adapter and configuring it to use Prometheus.
* Syntax for the Prometheus Adapter configuration could change in the future.
* The `APIService` configuration to wire Kubernetes' API aggregation to the
instance of the custom metrics adapter will be overwritten in future releases,
if {product-title} ships an out-of-the-box custom metrics adapter.

[id="ocp-4-1-cluster-monitoring-alerting-UI"]
==== New alerting user interface

An alerting UI is now natively integrated into the {product-title} web console.
You can now view cluster-level alerts and alerting rules from a single place, as
well as configure silences.

[id="ocp-4-1-cluster-monitoring-telemeter"]
==== Telemeter

Telemeter collects anonymized cluster-related metrics to proactively help
customers with their {product-title} clusters. This helps:

* Gather crucial health metrics of {product-title} installations.
* Enable a viable feedback loop of {product-title} upgrades.
* Gather the cluster's number of nodes per cluster and their size (CPU cores and
RAM).
* Gather the size of etcd.
* Gather details about the health condition and status for any OpenShift framework
component installed on an OpenShift cluster.

[id="ocp-4-1-cluster-monitoring-autoscale"]
==== Autoscale pods horizontally based on the resource metrics API

By default, OpenShift Cluster Monitoring exposes CPU and Memory utilization
through the Kubernetes resource metrics API. There is no longer a requirement to
install a separate metrics server.

[id="ocp-4-1-developer-experience"]
=== Developer experience

[id="ocp-4-1-code-ready-containers"]
==== Code ready containers

A local desktop instance of {product-title} 4.1 replaces the functions of `oc
cluster` commands, Minishift, and CDK. {product-title} 4.1 focuses on ease of
access and native experience, with a native installation program on macOS and Microsoft
Windows, native hypervisor support, and tray icon integration.

[id="ocp-4-1-multistage-builds"]
==== Multi-stage Dockerfile Builds Generally Available

Multi-stage Dockerfiles are now supported in all `Docker` strategy builds.

[id="ocp-4-1-registry"]
=== Registry

[id="ocp-4-1-registry-managed-by-operator"]
==== The registry is now managed by an Operator

The registry is now managed by an Operator instead of `oc adm registry`.

[id="ocp-4-1-networking"]
=== Networking

[id="ocp-4-1-cno"]
==== Cluster Network Operator (CNO)

The cluster network is now configured and managed by an Operator. The Operator
upgrades and monitors the cluster network.

[id="ocp-4-1-openshift-sdn"]
==== OpenShift SDN

The default mode is now `NetworkPolicy`.

[id="ocp-4-1-multus"]
==== Multus

Multus is a meta plug-in for Kubernetes Container Network Interface (CNI), which
enables a user to create multiple network interfaces per pod.

[id="ocp-4-1-web-console"]
=== Web console

[id="ocp-4-1-developer-catalog"]
==== Developer Catalog

{product-title} 4.1 features a redesigned Developer Catalog that brings all of
the new Operators and existing broker services together, with new ways to
discover, sort, and understand how to best use each type of offering. The
Developer Catalog is the entry point for a developer to access all services
available to them. It merges all capabilities from Operators, the Service
Catalog, brokers, and Source-to-Image (S2I).

[id="ocp-4-1-new-management-screens"]
==== New management screens

New management screens in {product-title} 4.1 support automated operations.
Examples include the management of machine sets and machines, taints,
tolerations, and cluster settings.

[id="ocp-4-1-security"]
=== Security

In {product-title} 4.1, an Operator is utilized to install, configure, and manage
the certificate signing server. Certificates are managed as secrets, and are
stored and encrypted in etcd.

[id="ocp-4-1-notable-technical-changes"]
== Notable technical changes

{product-title} 4.1 introduces the following notable technical changes.

[discrete]
[id="ocp-4-1-buildah"]
==== Builds powered by buildah

Source and Docker strategy builds are now performed by buildah instead of the
docker daemon.

[discrete]
[id="ocp-4-1-SecurityContextConstraints"]
==== SecurityContextConstraints

`SecurityContextConstraints` now only exist in the `security.openshift.io` group.

[discrete]
[id="ocp-4-1-Service-CA-bundle"]
==== Service CA bundle changes

Pods can trust cluster-created certificates, which are only signed for internal
DNS names, by using a CA bundle that is automatically injected into any
configMap annotated with `service.beta.openshift.io/inject-cabundle=true`. The
CA bundle will be made available as PEM-encoded data under the key
`service-ca.crt`. This annotation results in wiping out existing content in the
configMap.

Pods that currently consume the service-serving CA bundle from
`/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt` should migrate to
obtaining the CA bundle from a configMap annotated with
`service.beta.openshift.io/inject-cabundle=true`.

The `/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt` file is now
deprecated and will be removed in a future release.

[discrete]
[id="ocp-4-1-service-broker-service-catalog-deprecation"]
==== OpenShift Service Broker and Service Catalog deprecation

The Service Catalog and the OpenShift service brokers are being replaced over
the course of several future OpenShift 4 releases. Red Hat will be deprecating
the Template Service Broker and OpenShift Ansible Broker once important
dependent content is ported to new Operator-driven solutions. Users are
encouraged to look at the Operator Framework and Operator Lifecycle Manager
(OLM) to continue providing their applications to OpenShift 4 clusters. These
new technologies provide many benefits around complete management of the
lifecycle of your application.

[discrete]
[id="ocp-4-1-service-catalog-no-longer-installed-by-default"]
==== Service Catalog no longer installed by default

The Service Catalog is not installed by default in {product-title} 4.1. You must
install it if you plan on using any of the services from the OpenShift Ansible
broker or template service broker. In {product-title} 4.1, the Service Catalog
API server is installed into the `openshift-service-catalog-apiserver` namespace
and the Service Catalog controller manager is installed into the
`openshift-service-catalog-controller-manager` namespace. In {product-title}
3.11, both of these components were installed into the `kube-service-catalog`
namespace.

[discrete]
[id="ocp-4-1-template-service-broker-no-longer-installed-by-default"]
==== Template Service Broker no longer installed by default

The Template Service Broker is not installed by default in {product-title} 4.1.
Cluster administrators can install the Template Service Broker if users will
need access to template applications from the web console.

[discrete]
[id="ocp-4-1-openshift-ansible-service-broker-no-longer-installed-by-default"]
==== OpenShift Ansible Service Broker no longer installed by default

The OpenShift Ansible Service Broker is not installed by default in
{product-title} 4.1.

[discrete]
[id="ocp-4-1-oc-adm-commands-deprecated"]
==== Several `oc adm` commands are now deprecated

Deprecated `oc adm` commands include:

* `oc adm create-master-certs` - Create keys and certificates
* `oc adm create-key-pair` - Create an RSA key pair.
* `oc adm create-server-cert` - Create a key and server certificate.
* `oc adm create-signer-cert` - Create a self-signed CA.

[discrete]
[id="ocp-4-1-configurability-of-imagepolicyadmission"]
==== The configurability of the imagepolicyadmission plug-in is not present

The configurability of the `imagepolicyadmission` plug-in is not present in
{product-title} 4.1. The plug-in runs, but currently only with default
configuration. Configuring it requires using the unsupported overrides
mechanism.

[id="ocp-4-1-technology-preview"]
== Technology Preview features

Some features in this release are currently in Technology Preview. These
experimental features are not intended for production use. Note the
following scope of support on the Red Hat Customer Portal for these features:

link:https://access.redhat.com/support/offerings/techpreview[Technology Preview
Features Support Scope]

In the table below, features marked *TP* indicate _Technology Preview_ and
features marked *GA* indicate _General Availability_.

.Technology Preview Tracker
[cols="3",options="header"]
|====
|Feature |OCP 3.11 |OCP 4.1

|Prometheus Cluster Monitoring
|GA
|GA

|Local Storage Persistent Volumes
|TP
|TP

|CRI-O for runtime pods
|GA* footnoteref:[disclaimer, Features marked with `*` indicate delivery in a z-stream patch.]
|GA

|Tenant Driven Snapshotting
|TP
|TP

|`oc` CLI Plug-ins
|TP
|TP

|Service Catalog
|GA
|GA

|Template Service Broker
|GA
|GA

|OpenShift Ansible Service Broker
|GA
|GA

|Network Policy
|GA
|GA

|Multus
|-
|GA

|Service Catalog Initial Experience
|GA
|GA

|New Add Project Flow
|GA
|GA

|Search Catalog
|GA
|GA

|Cron Jobs
|GA
|GA

|Kubernetes Deployments
|GA
|GA

|StatefulSets
|GA
|GA

|Explicit Quota
|GA
|GA

|Mount Options
|GA
|GA

|System Containers for Docker, CRI-O
|-
|-

|Hawkular Agent
|-
|-

|Pod PreSets
|-
|-

|experimental-qos-reserved
|TP
|TP

|Pod sysctls
|GA
|GA. See xref:ocp-4-1-known-issues[Known issues] for current limitations.

|Central Audit
|GA
|GA

|Static IPs for External Project Traffic
|GA
|GA

|Template Completion Detection
|GA
|GA

|`replicaSet`
|GA
|GA

|Fluentd Mux
|TP
|TP

|Clustered MongoDB Template
|-
|-

|Clustered MySQL Template
|-
|-

|Image Streams with Kubernetes Resources
|GA
|GA

|Device Manager
|GA
|GA

|Persistent Volume Resize
|GA
|GA

|Huge Pages
|GA
|GA

|CPU Pinning
|GA
|GA

|Admission Webhooks
|TP
|GA

|External provisoner for AWS EFS
|TP
|TP

|Pod Unidler
|TP
|TP

|Node Problem Detector
|TP
|TP

|Ephemeral Storage Limit/Requests
|TP
|TP

|Descheduler
|TP
|TP

|CephFS
|TP
|TP

|Podman
|TP
|TP

|Kuryr CNI Plug-in
|TP
|TP

|Istio
|TP
|TP

|Sharing Control of the PID Namespace
|TP
|TP

|Manila Provisioner
|TP
|TP

|Cluster Administrator console
|GA
|GA

|Cluster Autoscaling (AWS Only)
|GA
|GA

|Container Storage Interface (CSI)
|TP
|TP

|Operator Lifecycle Manager
|TP
|GA

|Red Hat OpenShift Service Mesh
|TP
|TP

|"Fully Automatic" Egress IPs
|GA
|GA

|Pod Priority and Preemption
|GA
|GA

|Multi-stage builds in Dockerfiles
|TP
|GA

|OVN
|
|TP

|HPA custom metrics adapter based on Prometheus
|
|TP

|Machine health checks
|
|TP
|====

[id="ocp-4-1-known-issues"]
== Known issues

* Unsafe sysctl cannot be used in {product-title} 4.1.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1690754[*BZ#1690754*])

* If an instance is removed from the cloud provider (either via a user deletion,
or cloud-provider event of some kind), and the machine-object is reconciled
again for some reason, the machine-controller might determine the instance no
longer exists and attempt to create the instance. This is undocumented behavior
and should not be relied upon for workflows. This operation might interfere with
current or future components, such as the node-health-checker.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712068[*BZ#1712068*])

* Builds which use shell substitution to populate an environment variable may fail.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712245[*BZ#1712245*])

* When deleting a machine-object, either directly or by scaling down the owning
machine-set, if the associated node has already been deleted somehow (possibly
by a cluster administrator), the machine-controller will fail to successfully
delete the backing cloud instance, and the machine-object will be stuck in
`deleting` status.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713061[*BZ#1713061*])

* Querying `Jolokia` on `JBoss EAP` images fails as the result of empty
certificates presented to the client. The `Jolokia` SSL client authentication
will fail and may require a username and password challenge if enabled.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1708640[*BZ#1708640*])

* The `TokenRequest` API is not available in {product-title} 4.1. Requesting a
`ServiceAccountTokenVolumeProjection` volume is not available in {product-title}
4.1. The kubelet will present an error if a
`ServiceAccountTokenVolumeProjection` is used.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1695196[*BZ#1695196*])

* The `es nodeCount` in ElasticSearch CRD instances can not be scaled up if
deployed with three nodes. Scaling works correctly if deployed with one, two,
four, five or six ElasticSearch CRD instances.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712955[*BZ#1712955*])

* ElasticSearch instances created from OperatorHub deploy with a one CPU limit,
even though no limits are specified.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1710657[*BZ#1710657*])

* After an AWS installation, the `openshiftClustID` tag is not present.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1685089[*BZ#1685089*])

* scc(CRD) resources can not be upgraded by using the `oc patch` and `oc edit`
commands. As a result, strategic merges also fail.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1707679[*BZ#1707679*])

* The {product-title} 4.1 registry service utilizes port 5000 instead of port 443.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1701422[*BZ#1701422*])

* Machineset scaling in AWS environments may fail if the resources requested are
unavailable in the chosen Availability Zone.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713157[*BZ#1713157*])

* Using `OAuth` endpoints after configuring ingress wildcard certificates from
custom PKIs result in login errors.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712525[*BZ#1712525*])

* Source-to-Image (S2I) builds in {product-title} 4.1 may take longer to complete.
This is because {product-title} 4.1 does not utilize a shared image cache for
building images like in previous versions of {product-title}.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1685352[*BZ#1685352*])

* The `cloud-credential-operator` may crash on clusters with large numbers of
projects or namespaces due to memory limitations.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1711402[*BZ#1711402*])

* The Marketplace can not detect `opsrc` after a cluster upgrade is performed. As
a result, the `csc` packages are empty and can not download `packagemanifests`.
Marketplace can repair this problem approximately one hour later when it syncs
again.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1695550[*BZ#1695550*])

* In {product-title} 4.1, `oc` and `openshift-install` version may have a dirty
`GitTreeState:` when checking the `oc version`.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1715001[*BZ#1715001*])

* In AWS environments, if a master node is stopped, the `kubeapiserver` cannot be
deployed due a pod stuck in a `Pending` status.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713292[*BZ#1713292*])

* All m4 instances on AWS fail to verify
(https://access.redhat.com/security/cve/cve-2019-11091[*CVE-2019-1109*]) using
Broadwell CPU model 79 (type m4) because the `microcode_ctl` will not update.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1710981[*BZ#1710981*])

* New ElasticSearch deployments can not be created if another ElasticSearch
deployment is stuck in a deleting state.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1711044[*BZ#1711044*])

* There is no _Open Java Console_ link available in the {product-title} 4.1 web
console.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1713656[*BZ#1713656*])

* The `openshift-cluster-node-tuning-operator` may generate a large number of
secrets after several days of uptime.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1714484[*BZ#1714484*])

* Autoscaling for Memory Utilization is not working as expected. Creating HPA for
memory-based autoscaling is failing while looking for resources.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1707785[*BZ#1707785*])

* After successfully performing updates, `oc clusterversion` may report that the
update could not be applied.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1711964[*BZ#1711964*])

* The installer may have a `0` return code when hitting a FATAL event.
(link:https://bugzilla.redhat.com/show_bug.cgi?id=1712409[*BZ#1712409*])

[id="ocp-4-1-asynchronous-errata-updates"]
== Asynchronous errata updates

Security, bug fix, and enhancement updates for {product-title} 4.1 are released
as asynchronous errata through the Red Hat Network. All {product-title} 4.1
errata is https://access.redhat.com/downloads/content/290/[available on the Red
Hat Customer Portal]. See the
https://access.redhat.com/support/policy/updates/openshift[{product-title}
Life Cycle] for more information about asynchronous errata.

Red Hat Customer Portal users can enable errata notifications in the account
settings for Red Hat Subscription Management (RHSM). When errata notifications
are enabled, users are notified via email whenever new errata relevant to their
registered systems are released.

[NOTE]
====
Red Hat Customer Portal user accounts must have systems registered and consuming
{product-title} entitlements for {product-title} errata notification
emails to generate.
====

This section will continue to be updated over time to provide notes on
enhancements and bug fixes for future asynchronous errata releases of
{product-title} 4.1. Versioned asynchronous releases, for example with the form
{product-title} 4.1.z, will be detailed in subsections. In addition, releases
in which the errata text cannot fit in the space provided by the advisory will
be detailed in subsections that follow.

[IMPORTANT]
====
For any {product-title} release, always review the instructions on
xref:../updating/updating-cluster.adoc#updating-cluster[updating your cluster]
properly.
====

[[ocp-4-1]]
=== RHBA-2019:0758 - {product-title} 4.1 Image Release advisory

Issued: 2019-06-04

{product-title} release 4.1 is now available. The list of packages
included in the update are documented in the
link:https://access.redhat.com/errata/RHBA-2019:1173[RHBA-2019:1173] advisory.
The container images and bug fixes included in the update are provided by the
link:https://access.redhat.com/errata/RHBA-2019:0758[RHBA-2019:0758] advisory.

Space precluded documenting all of the container images for this release in the
advisory. See the following sections for notes on the container images in this
release.

[[ocp-4-1-container-images]]
==== Images

This release updates the Red Hat Container Registry
(_registry.access.redhat.com_) with the following images:

----
openshift4/apb-base
openshift4/apb-tools
openshift4/mariadb-apb
openshift4/mediawiki-apb
openshift4/mediawiki
openshift4/mysql-apb
openshift4/ose-ansible-operator
openshift4/ose-ansible-service-broker
openshift4/ose-ansible-service-broker-operator
openshift4/ose-aws-machine-controllers
openshift4/ose-cli-artifacts
openshift4/ose-cli
openshift4/ose-cloud-credential-operator
openshift4/ose-cluster-authentication-operator
openshift4/ose-cluster-autoscaler
openshift4/ose-cluster-autoscaler-operator
openshift4/ose-cluster-bootstrap
openshift4/ose-cluster-capacity
openshift4/ose-cluster-config-operator
openshift4/ose-cluster-dns-operator
openshift4/ose-cluster-image-registry-operator
openshift4/ose-cluster-ingress-operator
openshift4/ose-cluster-kube-apiserver-operator
openshift4/ose-cluster-kube-controller-manager-operator
openshift4/ose-cluster-kube-scheduler-operator
openshift4/ose-cluster-logging-operator
openshift4/ose-cluster-machine-approver
openshift4/ose-cluster-monitoring-operator
openshift4/ose-cluster-network-operator
openshift4/ose-cluster-node-tuned
openshift4/ose-cluster-node-tuning-operator
openshift4/ose-cluster-openshift-apiserver-operator
openshift4/ose-cluster-openshift-controller-manager-operator
openshift4/ose-cluster-samples-operator
openshift4/ose-cluster-storage-operator
openshift4/ose-cluster-svcat-apiserver-operator
openshift4/ose-cluster-svcat-controller-manager-operator
openshift4/ose-cluster-version-operator
openshift4/ose-configmap-reloader
openshift4/ose-console
openshift4/ose-console-operator
openshift4/ose-container-networking-plugins-supported
openshift4/ose-container-networking-plugins-unsupported
openshift4/ose-coredns
openshift4/ose-deployer
openshift4/ose-descheduler
openshift4/ose-descheduler-operator
openshift4/ose-docker-builder
openshift4/ose-docker-registry
openshift4/ose-egress-dns-proxy
openshift4/ose-egress-http-proxy
openshift4/ose-egress-router
openshift4/ose-elasticsearch-operator
openshift4/ose-etcd
openshift4/ose-grafana
openshift4/ose-haproxy-router
openshift4/ose-hyperkube
openshift4/ose-hypershift
openshift4/ose-installer-artifacts
openshift4/ose-installer
openshift4/ose-jenkins-agent-base
openshift4/ose-jenkins-agent-maven
openshift4/ose-jenkins-agent-nodejs
openshift4/ose-jenkins
openshift4/ose-k8s-prometheus-adapter
openshift4/ose-keepalived-ipfailover
openshift4/ose-kube-client-agent
openshift4/ose-kube-etcd-signer-server
openshift4/ose-kube-proxy
openshift4/ose-kube-rbac-proxy
openshift4/ose-kube-state-metrics
openshift4/ose-libvirt-machine-controllers
openshift4/ose-logging-curator5
openshift4/ose-logging-elasticsearch5
openshift4/ose-logging-eventrouter
openshift4/ose-logging-fluentd
openshift4/ose-logging-kibana5
openshift4/ose-machine-api-operator
openshift4/ose-machine-config-controller
openshift4/ose-machine-config-daemon
openshift4/ose-machine-config-operator
openshift4/ose-machine-config-server
openshift4/ose-multus-cni
openshift4/ose-must-gather
openshift4/ose-node
openshift4/ose-oauth-proxy
openshift4/ose-operator-lifecycle-manager
openshift4/ose-operator-marketplace
openshift4/ose-operator-registry
openshift4/ose-ovn-kubernetes
openshift4/ose-pod
openshift4/ose-prometheus-alertmanager
openshift4/ose-prometheus-config-reloader
openshift4/ose-prometheus
openshift4/ose-prometheus-node-exporter
openshift4/ose-prometheus-operator
openshift4/ose-prom-label-proxy
openshift4/ose-recycler
openshift4/ose-service-ca-operator
openshift4/ose-service-catalog
openshift4/ose-setup-etcd-environment
openshift4/ose-sriov-cni
openshift4/ose-sriov-network-device-plugin
openshift4/ose-telemeter
openshift4/ose-template-service-broker-operator
openshift4/ose-template-service-broker
openshift4/ose-tests
openshift4/postgresql-apb
openshift4/ose-haproxy-router-base
----

{'fraction_non_alphanumeric': 0.09036458721337366, 'fraction_numerical': 0.023466483901247077, 'mean_word_length': 5.248184695587414, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 68, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7646628', 'n_tokens_mistral': 21751, 'n_tokens_neox': 19928, 'n_words': 7204}