Hello DevOps enthusiasts 😃 !
Today Lets learn the importance of knowing kubernetes upgrading process and steps.
Kubernetes comes with new version for every 3 months. We as a DevOps engineers, we should know how to upgrade cluster with zero downtime as this activity is a frequent one.
In an organization, DevOps engineers maintains different clusters like Unit, Test, Stage, Pre, Prod etc.
As users uses prod environment, we should make sure prod is not impacted with our upgradation. Hence we test prior with different lower stage clusters.
Without further delay, lets jump into further steps.
Upgrading cluster correctly is very important to know and if any mistake is done it cant be irreversible.
Here are pre-requisites that you should know before directly upgrading:
Prerequisites
Cordon Nodes
In Kubernetes, cordoning a node refers to the action of marking a node as unschedulable, meaning no new pods will be scheduled on that node, but the existing pods continue to run without disruptions.
Understanding change logs and release notes
It is crucial for ensuring smooth system upgrades and maintaining stability. These documents track updates, detailing new features, bug fixes, and improvements, which helps teams stay informed about changes. For example: If a new change uses new feature, you should not get confused why it is not working with previous one. 🤔
Irreversible
For Example : Your current cluster is 1.30 and you upgraded to 1.31. If you want to undo the process, it is not possible as this is irreversible process. You should do fresh start in that case which impacts the product deployment.
Hence it is better to test first in lower environments, and then go with prod.
Start with low level env —> Stage env —> pre prod —> Prod ( Bteer to maintain gap of one week between lower envs to prod to understand smooth upgradation )
Maintain Same Versions ( Control plane - Nodes - Autoscalers - Kubelet )
Control plane and Nodes should be on same versions.
Example:
Control plane ( 1.30 ) Nodes (1.29) —> To upgrade cluster to 1.31 from 1.30 make sure nodes are with 1.30.
If you are using Cluster Auto Scaler ..it also should align with same version before upgrading. Also kubelet should be aligned with same version.
Five IP addresses available
Make sure you have atlease 5 ipaddress available in a subnet you provide in cluster.
Actual Process of Upgrading an EKS Cluster
We will take EKS as an example to upgrade.
- EKS is a managed Kubernetes service provided by AWS. When we say it is a managed service, it doesnt manage cluster upgrades but manages the following:
✍🏻 Availability of Control Plane
✍🏻 Disaster Recovery
✍🏻 Security
✍🏻 Scaling API server
- Upgrade Node Groups / Nodes / Fargate
Has different approaches. ( You will use rolling / force update strategy —> rolling update ( Zero downtime strategy) )
Upgrade Add-Ons
Understand your k8s cluster well
Test the Upgrade