Kubelet Error "Search Line limits were exceeded, some search paths have been omitted" is Causing Pods to Fail.

Problem

• Pods going in to CrashLoopBackOff state due to the DNS configuration error mentioned below.

DNSConfigForming  2m58s (x5957 over 5d4h)  kubelet  Search Line limits were exceeded, some search paths have been omitted, the applied search line is

Environment

• Platform9 Managed Kubernetes

Cause

• Kubernetes allows for a maximum of six search domains.
• CoreDNS creates three search domains at run-time and not in the /etc/resolv.conf file, which leaves a maximum of three domains that can be defined in the /etc/resolv.conf file.
• If there are more than three user-defined search domains are included in the /etc/resolv.conf file on the cluster host then The kubelet agent writes excessive error messages to the kubelet log file.

Resolution

• Need to make sure that there are no more than three search domains included in the /etc/resolv.conf file on the cluster host for which the error is logged in the kubelet logs.