Updating API Audit Logging Parameters using Qbert API Leads to Duplicate Entries.

Problem

While using the below Qbert API call with few sample API Audit Logging parameters;

Bash
    
 
curl -kv --request PUT -H "X-Auth-Token: <MASKED>" -H "Content-Type: application/merge-patch+json" -H "Accept: application/json" --data '{"apiServerFlags": "--audit-policy-file=/var/opt/pf9/kube/apiserver-config/audit-policy.yaml,--audit-log-path=/var/opt/pf9/kube/audit/audit.log,--audit-log-maxage=60,--audit-log-maxsize=200,--audit-log-maxbackup=20"}' https://<DU-FQDN>/qbert/v4/<PROJECT-ID>/clusters/<CLUSTER-UUID
Copy

We could see duplicate entries in the /opt/pf9/pf9-kube/conf/master.yaml

Bash
    
xxxxxxxxxx
 
$ grep -i audit /opt/pf9/pf9-kube/conf/pod-manifests/master.yaml---    - --audit-log-path=/var/opt/pf9/kube/audit/audit.log    - --audit-log-maxage=30    - --audit-log-maxbackup=20    - --audit-log-maxsize=50    - --audit-policy-file=/var/opt/pf9/kube/apiserver-config/audit-policy.yaml    - --audit-log-path=/var/opt/pf9/kube/audit/audit.log    - --audit-log-maxage=60    - --audit-log-maxsize=200    - --audit-log-maxbackup=20    - mountPath: /var/opt/pf9/kube/audit      name: apiserver-audit      path: /var/opt/pf9/kube/audit    name: apiserver-audit---
Copy

Environment

Platform9 Managed Kubernetes - v5.6.8
- PF9-Kube - 1.22.9-pmk.384
- PF9-Kube - 1.23.8-pmk.373
Platform9 Edge Cloud - LTS2 #4

Cause

Starting with the above mentioned releases, below parameters are by default baked with PF9-Kube package. This was introduced as part of a vulnerability scan.

Bash
    
 
--audit-log-path=/var/opt/pf9/kube/audit/audit.log --audit-log-maxage=30 --audit-log-maxbackup=20 --audit-log-maxsize=50
Copy

Using the Qbert API to update the Audit logging API server arguments may help the values persist even after cluster upgrades, but, is currently not recommended as it adds new entries instead of overriding the existing ones as seen in the Problem section.
This is currently tracked under JIRA AIR-1101 and PMK-5901.

Workaround

The current workaround is to manually update the _/opt/pf9/pf9-kube/conf/masterconfig/base/centos/master.yaml_ file on each master node followed by a PMK stack restart.

This method doesn't persist the values after cluster upgrades and needs to be manually updated after every upgrade.

Modify/Edit the below parameters on each master node:

Bash
    
xxxxxxxxxx
 
# vi /opt/pf9/pf9-kube/conf/masterconfig/base/centos/master.yaml---    - --audit-policy-file=/var/opt/pf9/kube/apiserver-config/audit-policy.yaml    - --audit-log-path=/var/opt/pf9/kube/audit/audit.log    - --audit-log-maxage=60    - --audit-log-maxsize=200    - --audit-log-maxbackup=20---
Copy

Restart the PMK stack one by one on all the master nodes:

Bash
    
xxxxxxxxxx
 
# systemctl stop pf9-hostagent pf9-nodeletd# /opt/pf9/nodelet/nodeletd phases restart# systemctl start pf9-hostagent
Copy

Verify the content of _/opt/pf9/pf9-kube/conf/pod-manifests/master.yaml_ to make sure the above values are reflected in the actual master configuration.

Bash
    
xxxxxxxxxx
 
# grep -i audit /opt/pf9/pf9-kube/conf/pod-manifests/master.yaml---    - --audit-policy-file=/var/opt/pf9/kube/apiserver-config/audit-policy.yaml    - --audit-log-path=/var/opt/pf9/kube/audit/audit.log    - --audit-log-maxage=60    - --audit-log-maxsize=200    - --audit-log-maxbackup=20---
Copy

Last updated on

Was this page helpful?