<commit_before>---
title: SLATE Cluster Upgrade Guide - K8s v1.24
overview: Update SLATE clusters to K8s v1.24
published: true
permalink: blog/slate-cluster-upgrade-guide-1_24.html
attribution: The SLATE Team
layout: post
type: markdown
tag: draft
---

The team has been busy preparing SLATE for Kubernetes (K8s) `v1.24.x` and today we are happy to announce that this work is now live. As cluster administrators there are several upgrade tasks you must perform to continue using SLATE with this new version of K8s.

<!--end_excerpt-->

## Table of Contents

This post will walk you, the cluster administrator, through the following tasks:

### Kubernetes Tasks

1. Upgrade all `v1beta1` Ingress objects to `v1` (<a href="#update-ingress-objects">see below</a>)
2. Upgrade your SLATE Kubernetes Cluster from K8s `v1.x` to `v1.24.x` using `kubeadm` (<a href="#upgrade-k8s">see below</a>)
3. (Recommended) Update the Calico CNI to `>= v3.24.1` (<a href="#update-calico-cni">see below</a>)
4. (Recommended) Update MetalLB to `>= v0.13.5` (<a href="#update-metallb">see below</a>)

### SLATE Tasks

1. Upgrade the SLATE Federation Controller roles (<a href="#update-fed-ctrl-role">see below</a>)
2. Update the SLATE Federation Controller itself (<a href="#update-fed-ctrl">see below</a>)
3. Upgrade the SLATE Ingress Controller (<a href="#update-ingress-ctrl">see below</a>)

## Kubernetes Tasks

<span id="update-ingress-objects"></span>
### Update Ingress Objects

{% include alert/note.html content="If you encounter an error while performing these steps contact [the SLATE team](/community/) for further assistance." %}
{% include alert/note.html content="If your cluster is already at `v1.22` (or newer) skip this section." %}

Support for `v1beta1` Ingress objects have been deprecated and completely removed by Kubernetes `v1.22`. Let's walk through an example to illustrate the changes you may need to apply to any existing Ingress objects.

#### Example

Below is a sample `v1beta1` Ingress object for the fictitious `hello-app-example`:

```yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: hello-app-example
  labels:
     app: hello-app-example
     chart: hello-app-example-chart
     release: hello-app-example-release
     instance: hello-app-example-instance
  annotations:
    kubernetes.io/ingress.class: slate
spec:
  rules:
  - host: hello.some-cluster.slateci.net
    http:
      paths:
        - path: /
          backend:
            serviceName: hello-app-example
            servicePort: 80
```

To update the Ingress object to `v1` you need to:
1. Change the `apiVersion` to `networking.k8s.io/v1`
2. Move the `ingress.class` annotation to `spec.ingressClassName`
3. Add a `pathType` to each specified `path`
4. Expand the `backend` structure from:
   ```yaml
   backend:
     serviceName: hello-app-example
     servicePort: 80
   ```
   to:
   ```yaml
   backend:
     service:
       name: hello-app-example
       port:
         number: 80
   ```   

Putting everything together gives us the new `v1` Ingress object:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-app-example
  labels:
     app: hello-app-example
     chart: hello-app-example-chart
     release: hello-app-example-release
     instance: hello-app-example-instance
spec:
  ingressClassName: slate
  rules:
  - host: hello.some-cluster.slateci.net
    http:
      paths:
        - path: /
          pathType: ImplementationSpecific
          backend:
            service:
              name: hello-app-example
              port:
                number: 80
```

For more information on Ingress objects see the [Kubernetes Ingress documentation](https://kubernetes.io/docs/concepts/services-networking/ingress).

<span id="upgrade-k8s"></span>
### Upgrade to K8s `v1.24.x`

{% include alert/note.html content="If you encounter an error while performing these steps contact [the SLATE team](/community/) for further assistance." %}

#### Check the status of the cluster

Start by SSH-ing to your control plane and switching to the `root` user.

```shell
[you@controlplane] $ sudo su -
```
{:data-add-copy-button='true'}

Configure `kubectl`/`kubeadm` and check the state of the Kubernetes nodes.

```shell
[root@controlplane] # export KUBECONFIG=/etc/kubernetes/admin.conf
```
{:data-add-copy-button='true'}

```shell
[root@controlplane ~]# kubectl get nodes
NAME                    STATUS   ROLES           AGE     VERSION
<worker>                Ready    <none>          2y68d   v1.22.1
<controlplane>          Ready    control-plane   2y68d   v1.22.1
```
{:data-add-copy-button='true'}

#### Determine the upgrade path

Best practice is to upgrade from one Kubernetes minor release to the next and so forth down the line all the way to `v1.24.x`. For example, if you are starting at `v1.21.x` the upgrade path should resemble:
* `v1.21.x` --> `v1.22.13`
* `v1.22.13` --> `v1.23.10`
* `v1.23.10` --> `v1.24.x`

#### Upgrade the control plane

Let's assume that like the example above, we are beginning with Kubernetes `v1.21.x`. Install the related packages for Kubernetes `v1.22.13`, making sure the `kubernetes` YUM repo is enabled.

```shell
yum update --enablerepo=kubernetes kubelet-1.22.13 kubeadm-1.22.13 kubectl-1.22.13
```
{:data-add-copy-button='true'}

Check that your cluster can be upgraded from `v1.21.x` --> `v1.22.13`.

```shell
kubeadm upgrade plan
```
{:data-add-copy-button='true'}

If there aren't any issues proceed with the upgrade. 

```shell
[root@controlplane ~]# kubeadm upgrade apply v1.22.13
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.22.13". Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
```
{:data-add-copy-button='true'}

Restart `kubelet` and check its status.

```shell
systemctl daemon-reload && \
systemctl restart kubelet && \
systemctl status kubelet 
```
{:data-add-copy-button='true'}

#### Upgrade the worker nodes one at a time

Leaving a terminal connected to the control plane, SSH to your first worker node in a fresh terminal, and switch to the `root` user.

```shell
[you@workernode1] $ sudo su -
```
{:data-add-copy-button='true'}

Install the related packages for Kubernetes `v1.22.13`, making sure the `kubernetes` YUM repo is enabled.

```shell
yum update --enablerepo=kubernetes kubelet-1.22.13 kubeadm-1.22.13 kubectl-1.22.13
```
{:data-add-copy-button='true'}

Back in the control plane terminal window apply the upgrade and prepare the worker node for maintenance.

```shell
kubeadm upgrade node && \
kubectl drain <workernode1> --ignore-daemonsets
```
{:data-add-copy-button='true'}

In the worker node terminal window restart `kubelet` and check its status.

```shell
systemctl daemon-reload && \
systemctl restart kubelet && \
systemctl status kubelet 
```
{:data-add-copy-button='true'}

If everything looks good, finish up by uncordoning the node in the control plane terminal window.

```shell
kubectl uncordon <workernode1>
```
{:data-add-copy-button='true'}

Log out of your worker node terminal window and rinse-repeat for your remaining worker nodes.

#### Verify the status of the cluster

Now that the `kubelet` has been upgraded on the control plane and worker nodes, once more SSH to your control plane and switch to the `root` user.

```shell
[you@controlplane] $ sudo su -
```
{:data-add-copy-button='true'}

Configure `kubectl`/`kubeadm` and check the state of the Kubernetes nodes.

```shell
[root@controlplane] # export KUBECONFIG=/etc/kubernetes/admin.conf
```
{:data-add-copy-button='true'}

```shell
[root@controlplane ~]# kubectl get nodes
NAME                    STATUS   ROLES           AGE     VERSION
<worker>                Ready    <none>          2y68d   v1.22.13 
<controlplane>          Ready    control-plane   2y68d   v1.22.13 
```
{:data-add-copy-button='true'}

If everything was successful the control plane and workers should all report as `v1.22.13`.

#### Next steps: `v1.22.13` to `v1.23.10`

At this point in the example your cluster should be running `v1.22.13`. Repeat the steps described above to upgrade from `v1.22.13` to `v1.23.10`, adjusting the K8s versions described in the commands accordingly.

#### Next steps: `v1.23.10` to `v1.24.x`

Once your cluster is running `v1.23.10` you are nearly are ready to make the final jump to `v1.24.x`. Before repeating the steps above, switch from `dockershim` to `containerd` as the default container runtime for your cluster.

SSH to your control plane and switching to the `root` user.

```shell
[you@controlplane] $ sudo su -
```
{:data-add-copy-button='true'}

Configure `kubectl`/`kubeadm`.

```shell
[root@controlplane] # export KUBECONFIG=/etc/kubernetes/admin.conf
```
{:data-add-copy-button='true'}

Stop any existing `containerd` service, set it the default runtime, and enable the service.

```shell
systemctl stop containerd && \
containerd config default | sudo tee /etc/containerd/config.toml && \
systemctl enable --now containerd
```
{:data-add-copy-button='true'}

Stop the `kubelet` service, configure `containerd` as the new runtime endpoint, and restart the service.

```shell
systemctl stop kubelet
```
{:data-add-copy-button='true'}

```shell
echo 'KUBELET_KUBEADM_ARGS="--pod-infra-container-image=k8s.gcr.io/pause:3.4.1 --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock"' > /var/lib/kubelet/kubeadm-flags.env
```
{:data-add-copy-button='true'}

```shell
systemctl daemon-reload && \
systemctl restart kubelet
```
{:data-add-copy-button='true'}

Next, for each worker node edit its manifest.

```shell
kubectl edit node <workernode>
```
{:data-add-copy-button='true'}

In the file content replace:

```yaml
...
metadata:
   annotations:
      ...
      kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
      ...
...
```
{:data-add-copy-button='true'}

with:

```yaml
...
metadata:
   annotations:
      ...
      kubeadm.alpha.kubernetes.io/cri-socket: unix:///run/containerd/containerd.sock
      ...
...
```
{:data-add-copy-button='true'}

Save the file and exit the editor.

Finally, complete the final upgrade from `v1.23.10` to `v1.24.x` using the steps described above, adjusting the K8s versions described in the commands accordingly.

#### Additional information

See the [Kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/) for complete instructions on updating a Kubernetes cluster from `v1.x` to `v1.24.x` using `kubeadm`.








<span id="update-calico-cni"></span>
### (Recommended) Update Calico CNI

{% include alert/note.html content="If you encounter an error while performing these steps contact [the SLATE team](/community/) for further assistance." %}

Update the Calico CNI to `>= v3.24.1`.
* If you followed our [Manual Cluster Installation](https://slateci.io/docs/cluster/manual/slate-master-node.html#pod-network) instructions when initially setting up your cluster, use the example below to update your Tigera operators and custom resources files.
* If you chose a different route for initially installing and configuring Calico, please refer directly to the [Calico documentation](https://projectcalico.docs.tigera.io/maintenance/kubernetes-upgrade) for update procedures.

#### Example
<commit_msg>Make ingress object updates optional and move section (#162)<commit_after>---
title: SLATE Cluster Upgrade Guide - K8s v1.24
overview: Update SLATE clusters to K8s v1.24
published: true
permalink: blog/slate-cluster-upgrade-guide-1_24.html
attribution: The SLATE Team
layout: post
type: markdown
tag: draft
---

The team has been busy preparing SLATE for Kubernetes (K8s) `v1.24.x` and today we are happy to announce that this work is now live. As cluster administrators there are several upgrade tasks you must perform to continue using SLATE with this new version of K8s.

<!--end_excerpt-->

## Table of Contents

This post will walk you, the cluster administrator, through the following tasks:

### Kubernetes Tasks

1. Upgrade your SLATE Kubernetes Cluster from K8s `v1.x` to `v1.24.x` using `kubeadm` (<a href="#upgrade-k8s">see below</a>)
2. (Optional) Upgrade manifests with  `v1beta1` Ingress objects to `v1` (<a href="#update-ingress-objects">see below</a>)
3. (Recommended) Update the Calico CNI to `>= v3.24.1` (<a href="#update-calico-cni">see below</a>)
4. (Recommended) Update MetalLB to `>= v0.13.5` (<a href="#update-metallb">see below</a>)

### SLATE Tasks

1. Upgrade the SLATE Federation Controller roles (<a href="#update-fed-ctrl-role">see below</a>)
2. Update the SLATE Federation Controller itself (<a href="#update-fed-ctrl">see below</a>)
3. Upgrade the SLATE Ingress Controller (<a href="#update-ingress-ctrl">see below</a>)

## Kubernetes Tasks

<span id="upgrade-k8s"></span>
### Upgrade to K8s `v1.24.x`

{% include alert/note.html content="If you encounter an error while performing these steps contact [the SLATE team](/community/) for further assistance." %}

#### Check the status of the cluster

Start by SSH-ing to your control plane and switching to the `root` user.

```shell
[you@controlplane] $ sudo su -
```
{:data-add-copy-button='true'}

Configure `kubectl`/`kubeadm` and check the state of the Kubernetes nodes.

```shell
[root@controlplane] # export KUBECONFIG=/etc/kubernetes/admin.conf
```
{:data-add-copy-button='true'}

```shell
[root@controlplane ~]# kubectl get nodes
NAME                    STATUS   ROLES           AGE     VERSION
<worker>                Ready    <none>          2y68d   v1.22.1
<controlplane>          Ready    control-plane   2y68d   v1.22.1
```
{:data-add-copy-button='true'}

#### Determine the upgrade path

Best practice is to upgrade from one Kubernetes minor release to the next and so forth down the line all the way to `v1.24.x`. For example, if you are starting at `v1.21.x` the upgrade path should resemble:

  * `v1.21.x` --> `v1.22.13`
  * `v1.22.13` --> `v1.23.10`
  * `v1.23.10` --> `v1.24.x`

#### Upgrade the control plane

Let's assume that like the example above, we are beginning with Kubernetes `v1.21.x`. Install the related packages for Kubernetes `v1.22.13`, making sure the `kubernetes` YUM repo is enabled.

```shell
yum update --enablerepo=kubernetes kubelet-1.22.13 kubeadm-1.22.13 kubectl-1.22.13
```
{:data-add-copy-button='true'}

Check that your cluster can be upgraded from `v1.21.x` --> `v1.22.13`.

```shell
kubeadm upgrade plan
```
{:data-add-copy-button='true'}

If there aren't any issues proceed with the upgrade. 

```shell
[root@controlplane ~]# kubeadm upgrade apply v1.22.13
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.22.13". Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
```
{:data-add-copy-button='true'}

Restart `kubelet` and check its status.

```shell
systemctl daemon-reload && \
systemctl restart kubelet && \
systemctl status kubelet 
```
{:data-add-copy-button='true'}

#### Upgrade the worker nodes one at a time

Leaving a terminal connected to the control plane, SSH to your first worker node in a fresh terminal, and switch to the `root` user.

```shell
[you@workernode1] $ sudo su -
```
{:data-add-copy-button='true'}

Install the related packages for Kubernetes `v1.22.13`, making sure the `kubernetes` YUM repo is enabled.

```shell
yum update --enablerepo=kubernetes kubelet-1.22.13 kubeadm-1.22.13 kubectl-1.22.13
```
{:data-add-copy-button='true'}

Back in the control plane terminal window apply the upgrade and prepare the worker node for maintenance.

```shell
kubeadm upgrade node && \
kubectl drain <workernode1> --ignore-daemonsets
```
{:data-add-copy-button='true'}

In the worker node terminal window restart `kubelet` and check its status.

```shell
systemctl daemon-reload && \
systemctl restart kubelet && \
systemctl status kubelet 
```
{:data-add-copy-button='true'}

If everything looks good, finish up by uncordoning the node in the control plane terminal window.

```shell
kubectl uncordon <workernode1>
```
{:data-add-copy-button='true'}

Log out of your worker node terminal window and rinse-repeat for your remaining worker nodes.

#### Verify the status of the cluster

Now that the `kubelet` has been upgraded on the control plane and worker nodes, once more SSH to your control plane and switch to the `root` user.

```shell
[you@controlplane] $ sudo su -
```
{:data-add-copy-button='true'}

Configure `kubectl`/`kubeadm` and check the state of the Kubernetes nodes.

```shell
[root@controlplane] # export KUBECONFIG=/etc/kubernetes/admin.conf
```
{:data-add-copy-button='true'}

```shell
[root@controlplane ~]# kubectl get nodes
NAME                    STATUS   ROLES           AGE     VERSION
<worker>                Ready    <none>          2y68d   v1.22.13 
<controlplane>          Ready    control-plane   2y68d   v1.22.13 
```
{:data-add-copy-button='true'}

If everything was successful the control plane and workers should all report as `v1.22.13`.

#### Next steps: `v1.22.13` to `v1.23.10`

At this point in the example your cluster should be running `v1.22.13`. Repeat the steps described above to upgrade from `v1.22.13` to `v1.23.10`, adjusting the K8s versions described in the commands accordingly.

#### Next steps: `v1.23.10` to `v1.24.x`

Once your cluster is running `v1.23.10` you are nearly are ready to make the final jump to `v1.24.x`. Before repeating the steps above, switch from `dockershim` to `containerd` as the default container runtime for your cluster.

SSH to your control plane and switching to the `root` user.

```shell
[you@controlplane] $ sudo su -
```
{:data-add-copy-button='true'}

Configure `kubectl`/`kubeadm`.

```shell
[root@controlplane] # export KUBECONFIG=/etc/kubernetes/admin.conf
```
{:data-add-copy-button='true'}

Stop any existing `containerd` service, set it the default runtime, and enable the service.

```shell
systemctl stop containerd && \
containerd config default | sudo tee /etc/containerd/config.toml && \
systemctl enable --now containerd
```
{:data-add-copy-button='true'}

Stop the `kubelet` service, configure `containerd` as the new runtime endpoint, and restart the service.

```shell
systemctl stop kubelet
```
{:data-add-copy-button='true'}

```shell
echo 'KUBELET_KUBEADM_ARGS="--pod-infra-container-image=k8s.gcr.io/pause:3.4.1 --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock"' > /var/lib/kubelet/kubeadm-flags.env
```
{:data-add-copy-button='true'}

```shell
systemctl daemon-reload && \
systemctl restart kubelet
```
{:data-add-copy-button='true'}

Next, for each worker node edit its manifest.

```shell
kubectl edit node <workernode>
```
{:data-add-copy-button='true'}

In the file content replace:

```yaml
...
metadata:
   annotations:
      ...
      kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
      ...
...
```
{:data-add-copy-button='true'}

with:

```yaml
...
metadata:
   annotations:
      ...
      kubeadm.alpha.kubernetes.io/cri-socket: unix:///run/containerd/containerd.sock
      ...
...
```
{:data-add-copy-button='true'}

Save the file and exit the editor.

Finally, complete the final upgrade from `v1.23.10` to `v1.24.x` using the steps described above, adjusting the K8s versions described in the commands accordingly.

#### Additional information

See the [Kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/) for complete instructions on updating a Kubernetes cluster from `v1.x` to `v1.24.x` using `kubeadm`.


<span id="update-ingress-objects"></span>
### (Optional) Update Ingress Objects

{% include alert/note.html content="If you encounter an error while performing these steps contact [the SLATE team](/community/) for further assistance." %}
{% include alert/note.html content="If your cluster is already at `v1.22` (or newer) skip this section." %}

Support for `v1beta1` Ingress objects have been deprecated and completely removed by Kubernetes `v1.22`.  You will only
need to update manifests to use the new Ingress objects.  Any Ingress objects currently active on your cluster will 
automatically get updated when Kubernetes is upgraded to `v1.22`.  See the 
[Kubernetes deprecation guide](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122) for
more details on the deprecation.

If you need to update a manifest, [this article](https://awstip.com/upgrading-kubernetes-ingresses-from-v1beta1-to-v1-7f9235765332) 
gives step by step instructions on what is needed.  

If you need further assistance, please contact the [the SLATE team](/community/).



<span id="update-calico-cni"></span>
### (Recommended) Update Calico CNI

{% include alert/note.html content="If you encounter an error while performing these steps contact [the SLATE team](/community/) for further assistance." %}

Update the Calico CNI to `>= v3.24.1`.
* If you followed our [Manual Cluster Installation](https://slateci.io/docs/cluster/manual/slate-master-node.html#pod-network) instructions when initially setting up your cluster, use the example below to update your Tigera operators and custom resources files.
* If you chose a different route for initially installing and configuring Calico, please refer directly to the [Calico documentation](https://projectcalico.docs.tigera.io/maintenance/kubernetes-upgrade) for update procedures.

#### Example

{'fraction_non_alphanumeric': 0.12563280850866193, 'fraction_numerical': 0.02508011704054619, 'mean_word_length': 3.904783599088838, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 79, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '525877', 'n_tokens_mistral': 7286, 'n_tokens_neox': 6732, 'n_words': 2309}