How to Upgrade PMK Cluster in Batches

Problem

How to Upgrade PMK Cluster in Batches?

Environment

Platform9 Managed Kubernetes - v5.4 and Higher
Qbert v4 API

Procedure

Fetch Keystone authentication token in order to access the PMK REST API by following Keystone Identity API.
Check if the PMK Cluster is available for an upgrade.

GET Cluster API
    
xxxxxxxxxx
 
curl --header "X-Auth-Token: $TOKEN" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/clusters/<CLUSTER_UUID>
Copy

Example GET Cluster API
    
​x
 
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90 | jq | grep -e name -e uuid -e projectId -e kubeRoleVersion -e canUpgrade -e canMinorUpgrade -e minorUpgradeRoleVersion -e canPatchUpgrade -e patchUpgradeRoleVersion   "name": "batch-upgrade-test",  "uuid": "41596ba7-b6e5-4008-85e4-42080f43eb90",  "canUpgrade": true,  "projectId": "86f43d7fc7ce4287a6be1d861a9ac8b4",  "kubeRoleVersion": "1.20.11-pmk.2064",  "canMinorUpgrade": 1,  "minorUpgradeRoleVersion": "1.21.3-pmk.72",  "canPatchUpgrade": 0,  "patchUpgradeRoleVersion": ""
Copy

Cluster is currently on version 1.20.11-pmk.2064 and there is a minor upgrade available to version 1.21.3-pmk.72.

Cluster Node Information

Master Node

GET Master Node API
    
 
curl --header "X-Auth-Token: $TOKEN" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/nodes | jq '.[] | select(.clusterUuid=="<CLUSTER_UUID>" and .isMaster=='1')'
Copy

Example GET Master Node API
    
 
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='1')' | grep -e uuid   "uuid": "ba53cb28-14ab-4406-b352-9dd6a1314320",
Copy

Worker Node

GET Worker Node API
    
 
curl --header "X-Auth-Token: $TOKEN" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/nodes | jq '.[] | select(.clusterUuid=="<CLUSTER_UUID>" and .isMaster=='0')'
Copy

Example GET Worker Node API
    
 
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='0')' | grep -e uuid   "uuid": "522b7a45-b98a-4990-9482-826beab2dfbf",  "uuid": "64e08702-041a-43fb-a708-00cb02112f19",  "uuid": "f3b0cbb0-efb9-407e-939a-21809421c376",
Copy

Performing Batch Cluster Upgrade

As part of the Upgrade API with Batch Option, only Worker Nodes can/should be specified in the batchUpgradeNodes parameter.

The Master Nodes will always be upgraded first in a sequential manner followed by the worker nodes specified in the batchUpgradeNodes argument.

Upgrade API with Batch Option
    
xxxxxxxxxx
 
curl --request POST \ --url https://platform9.io/qbert/v4/{project_uuid}/clusters/{uuid}/upgrade \ --header "X-Auth-Token: {X-Auth-Token}" \ --header "content-type: application/json" \ --data '{ "batchUpgradeNodes": [  "{array[string]...}" ],}'
Copy

Example Upgrade API with Batch Option for Minor Type
    
 
curl --header "X-Auth-Token: $TOKEN" --header "content-type: application/json" --request POST https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90/upgrade/\?type\="minor" --data '{ "batchUpgradeNodes": ["522b7a45-b98a-4990-9482-826beab2dfbf"] }' OK
Copy

If Patch type cluster upgrade is available and that is desired, the same needs to be specified in the API call as upgrade/\?type\="patch"

--header "content-type: application/json" should be specified as shown above, otherwise the API call will upgrade all nodes in a sequential manner.

The above example API upgraded the master nodes within the cluster first followed by the provided single worker node in batch.

Example Master Node Status
    
 
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='1')' | grep -e uuid -e actualKubeRoleVersion   "uuid": "ba53cb28-14ab-4406-b352-9dd6a1314320",  "actualKubeRoleVersion": "1.21.3-pmk.72"
Copy

Example Worker Node Status
    
 
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='0')' | grep -e uuid -e actualKubeRoleVersion   "uuid": "522b7a45-b98a-4990-9482-826beab2dfbf",  "actualKubeRoleVersion": "1.21.3-pmk.72",   "uuid": "64e08702-041a-43fb-a708-00cb02112f19",  "actualKubeRoleVersion": "1.20.11-pmk.2064",   "uuid": "f3b0cbb0-efb9-407e-939a-21809421c376",  "actualKubeRoleVersion": "1.20.11-pmk.2064",
Copy

Upgrade the remaining worker nodes as part of the next batch.

Example Upgrade API with Batch Option for Minor Type on Remaining Worker Nodes
    
 
curl --header "X-Auth-Token: $TOKEN" --header "content-type: application/json" --request POST https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90/upgrade/\?type\="minor" --data '{ "batchUpgradeNodes": ["64e08702-041a-43fb-a708-00cb02112f19", "f3b0cbb0-efb9-407e-939a-21809421c376"] }' OK
Copy

Post Upgrade Status

Example Cluster Status Post Upgrade
    
 
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90 | jq | grep -e name -e uuid -e projectId -e kubeRoleVersion -e canUpgrade -e canMinorUpgrade -e minorUpgradeRoleVersion -e canPatchUpgrade -e patchUpgradeRoleVersion   "name": "batch-upgrade-test",  "uuid": "41596ba7-b6e5-4008-85e4-42080f43eb90",  "canUpgrade": false,  "projectId": "86f43d7fc7ce4287a6be1d861a9ac8b4",  "kubeRoleVersion": "1.21.3-pmk.72",  "canMinorUpgrade": 0,  "minorUpgradeRoleVersion": "",  "canPatchUpgrade": 0,  "patchUpgradeRoleVersion": "",
Copy

Example Node Status Post Upgrade
    
 
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90")' | grep -e uuid -e actualKubeRoleVersion   "uuid": "522b7a45-b98a-4990-9482-826beab2dfbf",  "actualKubeRoleVersion": "1.21.3-pmk.72",   "uuid": "ba53cb28-14ab-4406-b352-9dd6a1314320",  "actualKubeRoleVersion": "1.21.3-pmk.72",   "uuid": "64e08702-041a-43fb-a708-00cb02112f19",  "actualKubeRoleVersion": "1.21.3-pmk.72",   "uuid": "f3b0cbb0-efb9-407e-939a-21809421c376",  "actualKubeRoleVersion": "1.21.3-pmk.72",
Copy

Last updated on

Was this page helpful?