Skip to main content

Upgrade EKS addons

We have 3 addons managed through cloud-platform-terraform-eks-add-ons module.

Before every EKS major version upgrade, check and upgrade if the addons versions don’t match the EKS major version the cluster is currently on.

After every EKS major versions upgrade, check and upgrade if the addons don’t match the EKS major version the cluster you just upgraded to.

The following addons are managed through cloud-platform-terraform-eks-add-ons module.

managing-kube-proxy

managing-coredns

managing-vpc-cni

Listing available EKS upgrades

eksctl Install

brew install eksctl

Run the following command to get a list of supported addon version for the Kubernetes version.

eksctl utils describe-addon-versions --kubernetes-version [k8s-version] --name [addon name: kube-proxy/vpc-cni/coredns] | grep AddonVersion

Error: error creating EKS Add-On (cluster-name:addon-name): InvalidParameterException: Addon version specified is not supported

If you come across this error, it is possible that the aws ui has incorrectly informed you about the supported version. To figure out which version is supported run the following command:

aws eks describe-addon-versions --kubernetes-version=$K8S_VERSION | jq '.addons[] | select(.addonName==$ADDON_NAME) | .addonVersions[] | select(.compatibilities[] | .defaultVersion==true)'

this will pull out the default compatible value for the k8s version for your addon.

You can use the helper script to get the most up-to-date available addon versions for each kubernetes cluster version script here

Preparing for upgrade

Check the changelog for each of the addons and determine if there are any breaking changes.

Create a thread in #cloud-platform notifying the team that upgrades are starting and the pipelines will be paused.

Starting the upgrade

  1. Run the helper script before
  2. Bump the version of the addon
  3. Commit changes on a new branch and create a pull request
  4. Request review from someone on the team
  5. Check the terraform plan in concourse and pause the following pipelines:
    • bootstrap
    • infrastructure-live
    • infrastructure-manager
    • infrastructure-live-2
  6. Create an output of the configuration of a pod before the upgrade. kubectl -n kube-system get pod $addon -oyaml there is also a helper .
  7. Merge the pull request 8 Unpause an infrastructure pipeline and wait for it to complete
  8. While running:
    • Keep an eye on pods recycling watch -n 1 "kubectl -n kube-system get pods"
    • Keep an eye on events watch -n 1 "kubectl -n kube-system get events"
  9. Run the helper script after
  10. Run the reporting pipeline on the infrastructure environment
  11. If everything is green repeat steps 8 - 11 on each environment.

Finish the upgrade

Finish up communications and close the thread.

Finally, ensure that all add-on underlying image versions are updated in the Container Images used by Cluster Components runbook

This page was last reviewed on 24 December 2024. It needs to be reviewed again on 24 June 2025 by the page owner #cloud-platform .
This page was set to be reviewed before 24 June 2025 by the page owner #cloud-platform. This might mean the content is out of date.