Replacing Cert-Manager in Kubernetes

How to replace the cert-manager in Kubernetes or update it.

Long time ago I installed cert-manager in Kubernetes using old approach described in this article.

Having the version of cert-manager 1.15.3 working very well for long time I noticed that there is an issue in cluster caused by changing the API on Cloudflare side.

E0430 20:27:39.763346       1 controller.go:162] "re-queuing item due to error processing" err=< while querying the Cloudflare API for DELETE "/zones//dns_records/6a3b8cf5..."    Error: 7003: Could not route to /client/v4/zones/dns_records/6a3b8cf5..., perhaps your object identifier is invalid? > logger="cert-manager.controller" key="gateway-infra/...-3230772443-4213816010"user@server:~$     kubectl get deployment -n cert-manager -l app=cert-manager -o jsonpath='{.spec.template.metadata.labels["cert-manager.io/version"]}'error: error parsing jsonpath {.spec.template.metadata.labels["cert-manager.io/version"]}, invalid array index "cert-manager.io/version"

By searching the issue in Google I found this PR.

Cloudflare - DNS Challenge Broken · Issue #7540 · cert-manager/cert-manager

As we can see, the problem was solved in 1.16.4+ version and we need to upgrade.

Since, I did not install the cert-manager through the helm, I am having the challenge to update it.

Let’s correct the installation of cert-manager.

Optional step to download manifests

user@kubenode1:~/k8s$ wget https://github.com/cert-manager/cert-manager/releases/download/v1.16.4/cert-manager.crds.yamluser@kubenode1:~/k8s$ wget https://github.com/cert-manager/cert-manager/releases/download/v1.16.4/cert-manager.yaml

Using this locally to examine all of them.

I would also change the original article to use 1.16.4 version of the cert-manager.

Now let’s upgrade the manifests

user@kubenode1:~/k8s$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.4/cert-manager.yaml

Wait some period of time and get status

kubectl get all -n cert-managerNAME                                           READY   STATUS    RESTARTS   AGEpod/cert-manager-8c879665-wsjwq                1/1     Running   0          25spod/cert-manager-cainjector-7bfd55d9f7-cpxcp   1/1     Running   0          25spod/cert-manager-webhook-6c8564598b-dbx4c      1/1     Running   0          24sNAME                              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)            AGEservice/cert-manager              ClusterIP   10.96.69.221    <none>        9402/TCP           240dservice/cert-manager-cainjector   ClusterIP   10.96.176.33    <none>        9402/TCP           25sservice/cert-manager-webhook      ClusterIP   10.96.196.235   <none>        443/TCP,9402/TCP   240dNAME                                      READY   UP-TO-DATE   AVAILABLE   AGEdeployment.apps/cert-manager              1/1     1            1           240ddeployment.apps/cert-manager-cainjector   1/1     1            1           240ddeployment.apps/cert-manager-webhook      1/1     1            1           240dNAME                                                 DESIRED   CURRENT   READY   AGEreplicaset.apps/cert-manager-7fbbc65b49              0         0         0       240dreplicaset.apps/cert-manager-8c879665                1         1         1       25sreplicaset.apps/cert-manager-cainjector-6664fc84f6   0         0         0       240dreplicaset.apps/cert-manager-cainjector-7bfd55d9f7   1         1         1       25sreplicaset.apps/cert-manager-webhook-59598898fd      0         0         0       240dreplicaset.apps/cert-manager-webhook-6c8564598b      1         1         1       24sashvid@karagatan:~/cert-manager$

Checking logs in the cert-manager

kubectl logs pod/cert-manager-8c879665-wsjwq -n cert-manager

Everything looks good, no more deletion of DNS error.