Fixing PodSecurity Admission warnings for deployments

Environment

• Red Hat OpenShift Container Platform (RHOCP)
  • 4.11+

Issue

• Why I am receiving warnings related to Pod Security even though the deployment is running with restricted-v2 SCC?

  $ oc  create deployment hello-node --image=k8s.gcr.io/e2e-test-images/agnhost:2.33 -- /agnhost serve-hostname
  Warning: would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "agnhost" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "agnhost" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "agnhost" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "agnhost" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
  deployment.apps/hello-node created
  

  $ oc describe pod hello-node-855787d74c-h99pg | grep scc
               openshift.io/scc: restricted-v2
  

• Such warnings could be seen for pods running inside openshift-operators namespace as well.

Resolution

• This warning wouldn't prevent the pod creation, so it can be ignored however, if one still wants to fix the warning then it's essential to specify security parameters reported in the warning inside the spec.containers[*].securityContext.

  spec:
    containers:
    - name: $NAME
      image: ${IMAGE}
      securityContext:
        allowPrivilegeEscalation: false
        seccompProfile:
          type: RuntimeDefault
        capabilities:
          drop:
          - ALL
  

• Please note that not all deployments may require these parameters in securityContext, depending on the application requirements user may run containers with extra privileges and capabilities.

• For workloads running inside openshift-operators there's no need to tweak the containers.spec, but one needs to make sure below label is set in the namespace.
  security.openshift.io/scc.podSecurityLabelSync=true

  • If there's isn't such label then one could set it manually as well:

  $ oc label namespace openshift-operators security.openshift.io/scc.podSecurityLabelSync=true
  

Root Cause

Disclaimer: Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or its entities, products, or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

• Openshift 4.11 includes the Kubernetes pod security admission plugin. Globally, the privileged profile is enforced, violations to the Content from kubernetes.io is not included.restricted profile generate a warning on the command line and an entry in the audit log.

• You can check the global default pod security admission configuration via jq "" $(oc extract cm/config -n openshift-kube-apiserver --confirm) | jq '.admission.pluginConfig.PodSecurity' but it is not supported to change the global configuration.

• If we look at the restricted pod security standard, the pod security admission plugin expects a strong isolation like a SecComp profile turned on and all Linux capabilities dropped.

• When Pod Security Admission doesn't find the desired parameters in the pod/container's securityContext then it produces a warning because at the namespace level we have this label:

  pod-security.kubernetes.io/warn: restricted

• Check Understanding and managing pod security admission for more details around PSA that were introduced in OCP 4.11+.

Diagnostic Steps

• For warnings related to pods running inside openshift-operators namesapce, ensure the namespace has podSecurity label sync as true.

  $ oc describe ns/openshift-operators
  

• Use below command to check all the pod-security warnings recorded as audit-violation

  $ oc adm node-logs --role=master --path=kube-apiserver/audit.log | cut -d " " -f 2- | jq -r 'select(.annotations["pod-security.kubernetes.io/audit-violations"] != null) | (.objectRef | .namespace + " " + .name + " " + .resource) + " " + .annotations["pod-security.kubernetes.io/audit-violations"]' | sort | uniq -c
  

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.