Disabling pf9-managed cert-manager

Problem

After upgrading the cluster to version 1.28.6, cert-manager is getting installed in the luigi-system namespace is having conflict with the existing custom cert manager in the cert-manager namespace. As a result, the cert-manager pods are going into an error state.

% kubectl get pods -A | grep cert-manager
cert-manager cert-manager-89b545d6d-zstl8 1/1 Running 2 (3h4m ago) 9h
cert-manager cert-manager-cainjector-646bf69b85-xhbxp 0/1 CrashLoopBackOff 64 (78s ago) 9h
cert-manager cert-manager-webhook-796478777-qzzfs 1/1 Running 0 9h
luigi-system cert-manager-5dcbbc765c-hpbql 1/1 Running 2 (5h3m ago) 8h
luigi-system cert-manager-cainjector-6db486b6b7-d8btt 1/1 Running 2 (5h3m ago) 8h
luigi-system cert-manager-webhook-57876b9fd-j4f6l 1/1 Running 0 8h

Environment

  • Platform9 Managed Kubernetes 5.9.4

  • Kubernetes version 1.28.6

Procedure

To completely disable pf9 managed cert-manager and continue using custom cert-manager:

  1. Patch the pf9-addon-operator image to the custom private image which doesn't install/uninstall pf9-managed cert-manager.<Need to add the repo details here>

  2. Apply the below script, which will point all the CRB from luigi-system to cert-manager system.

#!/bin/bash

# List of ClusterRoleBindings to update
CRBS=(
cert-manager-cainjector
cert-manager-controller-issuers
cert-manager-controller-clusterissuers
cert-manager-controller-certificates
cert-manager-controller-orders
cert-manager-controller-challenges
cert-manager-controller-ingress-shim
cert-manager-controller-approve:cert-manager-io
cert-manager-controller-certificatesigningrequests
cert-manager-webhook:subjectaccessreviews
)

# New namespace value
NEW_NAMESPACE="cert-manager"

echo "Updating ClusterRoleBinding subjects to use namespace: $NEW_NAMESPACE"

for crb in "${CRBS[@]}"; do
echo "Patching $crb..."
kubectl patch clusterrolebinding "$crb" \
--type=json \
  1. Edit the webhooks to point to the cert-manager namespace instead of luigi-system namespace.

kubectl edit ValidatingWebhookConfiguration cert-manager-webhook
kubectl edit MutatingWebhookConfiguration cert-manager-webhook

And delete all the 3 cert-manager deployments from luigi-system.

kubectl delete deploy cert-manager-webhook -n luigi-system
kubectl delete deploy cert-manager-cainjector -n luigi-system
kubectl delete deploy cert-manager -n luigi-system

Once this is done all the pf9-managed cert-manager will be completely cleaned and wont be applied again.