How to Allow Unsafe Sysctls on a PMK Cluster

Problem

How to Allow Unsafe Sysctls on a PMK Cluster?

Environment

Platform9 Managed Kubernetes - All Versions
Kubelet

Procedure

Depending on whether the change is needed to be done on a single node or all nodes part of the cluster, refer to

How To Change Configuration for Kubelet Service on a Single Worker or Master Node in a PMK Cluster

How To Change Configuration for Kubelet Service on All Worker or Master Nodes in a PMK Cluster

Stop the pf9-hostagent & pf9-nodeletd services on ALL the worker/master nodes depending on which configmap is being edited.

sudo systemctl stop pf9-{hostagent,nodeletd}

Edit the ConfigMap as needed and save it. Here as an example, we are editing it master-default-kubelet-config ConfigMap for the single master node cluster.

# kubectl get configmap master-default-kubelet-config -n kube-system -o yaml

...

allowedUnsafeSysctls:

- "kernel.msg*"

- "net.core.somaxconn"

Start the pf9-hostagent service ALL the worker/master nodes where it was initially stopped.

sudo systemctl start pf9-hostagent

# less /var/log/pf9/kubelet/kubelet.INFO | grep -i unsafe

I0209 01:40:12.118508 10415 flags.go:59] FLAG: --allowed-unsafe-sysctls="[]"

...

I0209 01:40:12.161606 10415 controller.go:267] kubelet config controller: ensuring filesystem is set up correctly

I0209 01:40:12.161624 10415 fsstore.go:59] kubelet config controller: initializing config checkpoints directory "/var/opt/pf9/kube/kubelet-config/dynamic-config/store"

I0209 01:40:12.163696 10415 fsstore.go:116] kubelet config controller: loading Kubelet configuration checkpoint for source /api/v1/namespaces/kube-system/configmaps/master-default-kubelet-config, UID: ae560d51-931a-4a1c-ae90-0ea9fee5d5b7, ResourceVersion: 1187451

Deployed a pod with security context.

# cat pod.yaml

apiVersion: v1

kind: Pod

metadata:

name: nginx

spec:

securityContext:

sysctls:

- name: kernel.shm_rmid_forced

value: "0"

- name: net.core.somaxconn

value: "1024"

- name: kernel.msgmax

value: "65536"

containers:

- name: nginx

image: nginx

imagePullPolicy: IfNotPresent

Pod has been created and is in running state.

# kubectl get pods

NAME READY STATUS RESTARTS AGE

nginx 1/1 Running 0 67s

If we deploy the same pod on a different cluster which does not have the Kubelet setting for allowedUnsafeSysctls added.

# kubectl get pods

NAME READY STATUS RESTARTS AGE

nginx 0/1 SysctlForbidden 0 13s

# kubectl describe pod nginx

Events:

Type Reason Age From Message

---- ------ ---- ---- -------

Normal Scheduled <unknown> Successfully assigned default/nginx to 10.128.146.164

Warning SysctlForbidden 27s kubelet, 10.128.146.164 forbidden sysctl: "net.core.somaxconn" not whitelisted

Additional Information

Using sysctls in a Kubernetes Cluster

Last updated on

Was this page helpful?