Nodelet Phase failing at Cert Generation Phase.

Problem

• The Kubernetes workload cluster was impacted and inaccessible.
• Unable to execute any cluster operations as a result of the kubectl command failing with the message "error: You must be logged in to the server (Unauthorized)".

Environment

• Platform9 Managed Kubernetes - v5.6 and Higher

Cause

• As per the current implementation of TCP Cert certificate rotation in PMK, TCP cert secret holds the old, intermediate and current CA certificate.
• In a recent incident, Platform9 observed that an old certificate from the certificate chain has expired, due to which comms service is failing to communicate with KDU, resulting in the host disconnection from KDU.

Resolution

• Platform9 is aware of this issue and has created a Bugzilla request PMK-6444
• ETA: Support of Automation CA certificate rotation will included in PMK v5.11.

Additional Information

• The progress of this feature request can be tracked by opening a support ticket mentioning the Jira ID PMK-6444.