It's time to upgrade your EKS clusters! Following up on my previous blog about upgrading an EKS cluster from version 1.30 to 1.31, I'm excited to discuss the highly anticipated Kubernetes 1.32 release, nicknamed Penelope. This release brings some improvements with it. If you're concerned about navigating the upgrade process on your own, there's no need to worry. This blog will guide you through a smooth and efficient upgrade using Terraform, ensuring that your workloads transition seamlessly into the Penelope era.
Overview of EKS 1.32 release
As per Kubernetes official notes: "If Kubernetes is Ancient Greek for "pilot," in this release, we start from that origin and reflect on the last 10 years of Kubernetes and our accomplishments: each release cycle is a journey, and just like Penelope, in "The Odyssey," weaved for 10 years—each night removing parts of what she had done during the day—so does each release add new features and remove others."
What are the changes in the EKS 1.32 release?
Anonymous authentication changes:
Starting with Amazon EKS 1.32, anonymous authentication is restricted to the following API server health check endpoints:
/healthz
/livez
/readyz
Requests to any other endpoint using the system:unauthenticated users will receive a 401 Unauthorized HTTP response. This security enhancement helps prevent unintended cluster access due to misconfigured RBAC policies.
What are the changes in Kubernetes 1.32 release?
You can find a complete list of changes and updates in Kubernetes version 1.32 here. These are the updates that caught my eye.
- Custom Resource field selectors: Custom resource field selectors enable developers to apply filtering to custom resources, similar to the functionality available for built-in Kubernetes objects. This enhances the efficiency and precision of filtering custom resources, encouraging better API design practices.
- Support to size memory backed volumes: This feature enables dynamic sizing of memory-backed volumes based on Pod resource limits, enhancing workload portability and overall resource utilization on nodes.
- Bound service account token improvement: Incorporating the node name into the token claims allows users to utilize this information during authorization and admission (ValidatingAdmissionPolicy). Additionally, this enhancement prevents service account credentials from serving as a privilege escalation path for nodes.
- Structured authorization configuration: Multiple authorizers can be configured in the API server to enable structured authorization decisions and support CEL match conditions in webhooks.
- Auto remove PVCs created by StatefulSet: PersistentVolumeClaims (PVCs) generated by StatefulSets are automatically removed when they are no longer necessary, ensuring data persistence during updates of StatefulSets and during node maintenance. This feature streamlines storage management for StatefulSets and lessens the risk of orphaned PVCs.
This release includes a total of 13 enhancements promoted to stable:
- Structured Authorization Configuration
- Bound service account token improvements
- Custom Resource Field Selectors
- Retry Generate Name
- Make Kubernetes aware of the LoadBalancer behavior
- Field status.hostIPs added for Pod
- Custom profile in kubectl debug
- Memory Manager
- Support to size memory-backed volumes
- Improved multi-numa alignment in Topology Manager
- Add job creation timestamp to job annotations
- Add Pod Index Label for StatefulSets and Indexed Jobs
- Auto-remove PVCs created by StatefulSet
Deprecation and Removal in Kubernetes 1.32 release:
- Amazon Linux 2 AMI deprecation: AWS EKS will not provide pre-built Amazon Linux 2 (AL2) Amazon Machine Images (AMIs) in Kubernetes versions 1.33 and later. AWS suggests migrating to Amazon Linux 2023 (AL2023) or Bottlerocket.
- Withdrawal of the old DRA implementation: DRA will undergo significant changes, with the removal of original implementation code. This will enable Kubernetes to manage new hardware requirements and resource claims more predictably, eliminating the complexities of back-and-forth API calls to the kube-apiserver.
Deprecated API versions in Kubernetes 1.32 release:
- Flow control resources: The flowcontrol.apiserver.k8s.io/v1beta3 API version of FlowSchema and PriorityLevelConfiguration is deprecated. Migrate manifests and API clients to use the flowcontrol.apiserver.k8s.io/v1beta3 API version to flowcontrol.apiserver.k8s.io/v1
Steps to upgrade EKS from 1.31 to 1.32 with Terraform
I deployed an EKS cluster in the AWS Cloud using Terraform version 1.31. Here's a screenshot of the cluster.
We successfully deployed the NGINX application in the cluster and verified its functionality by port forwarding to port 9999.
kubectl get po -n nginx NAME READY STATUS RESTARTS AGE nginx-deployment-7dbfbc79cf-4zt2z 1/1 Running 0 6m27s kubectl port-forward -n nginx nginx-deployment-7dbfbc79cf-4zt2z 9999:80 Forwarding from 127.0.0.1:9999 -> 80 Forwarding from [::1]:9999 -> 80 Handling connection for 9999 Handling connection for 9999
We can access the NGINX application here: http://localhost:9999
Prerequisites to upgrade from 1.31 – 1.32
Before upgrading to Kubernetes v1.32 in Amazon EKS, update the API version of FlowSchema and PriorityLevelConfiguration from flowcontrol.apiserver.k8s.io/v1beta3 to flowcontrol.apiserver.k8s.io/v1
Upgrade EKS cluster:
Updates Terraform script to upgrade EKS cluster to 1.32 version.
terraform {
backend "s3" {
encrypt = true
bucket = "terraform-state-bucket"
key = "infra"
region = "ap-south-1"
}
}
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
region = "ap-south-1"
}
module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "~> 20.0"
cluster_name = "eks-cluster"
cluster_version = "1.32"
cluster_endpoint_public_access = true
cluster_addons = {
coredns = {
resolve_conflicts = "OVERWRITE"
version = "v1.11.4-eksbuild.2"
}
kube-proxy = {
resolve_conflicts = "OVERWRITE"
version = " v1.31.3-eksbuild.2"
}
vpc-cni = {
resolve_conflicts = "OVERWRITE"
version = " v1.19.2-eksbuild.1"
}
aws-ebs-csi-driver = {
resolve_conflicts = "OVERWRITE"
version = " v1.39.0-eksbuild.1"
}
}
vpc_id = "*******"
subnet_ids = ["*******", " *******", "******* "]
control_plane_subnet_ids = ["*******", "*******", "*******"]
eks_managed_node_groups = {
group = {
ami_type = "AL2023_x86_64_STANDARD"
instance_types = ["t3a.medium"]
min_size = 1
max_size = 2
desired_size = 1
}
}
enable_cluster_creator_admin_permissions = true
tags = {
Environment = "dev"
Terraform = "true"
}
}
To see the changes after modifying the Terraform script, run Terraform plan.
terraform plan
To apply the changes to upgrade the EKS cluster to 1.31 version, run Terraform apply.
terraform apply
Upgrading the cluster will take 8-10 minutes and 25-30 minutes to do the node group and AWS EKS add-ons. Here's the screenshot of the cluster after upgradation.
Now, let's verify the NGINX application, which was already deployed. I verified the nginx application by port forwarding to 9999 port.
kubectl get po -n nginx NAME READY STATUS RESTARTS AGE nginx-deployment-7dbfbc79cf-j9vxm 1/1 Running 0 6m37s kubectl port-forward -n nginx nginx-deployment-7dbfbc79cf-j9vxm 9999:80 Forwarding from 127.0.0.1:9999 -> 80 Forwarding from [::1]:9999 -> 80 Handling connection for 9999 Handling connection for 9999
From above, the NGINX app is redeployed after the node group gets upgraded to 1.31 version. Now let's access the NGINX application by http://localhost:9999
Verify all other components after upgrading the EKS cluster to check whether they could include:
- Load Balancer Controller
- EBS CSI Driver
- calico-node
- Cluster Autoscaler or Karpenter
- External Secrets Operator
- Kube State Metrics
- Metrics Server
- csi-secrets-store
- Keda (event driven autoscaler)
Conclusion
Upgrading an EKS cluster is much quicker; upgrading the control plane took only eight minutes. I utilized Terraform to seamlessly upgrade the EKS cluster and node group to version 1.32, making the process significantly easier. Fortunately, there were no significant issues during the upgrade, and all workloads ran without problems.
My next blog will talk about migrating from EKS from 1.32 to 1.33. Stay tuned!
