How to apply Skip SELinux relabeling workaround with spc_t automatically on workloads running on OpenShift Container Platform 4?
Environment
- OpenShift Container Platform 4.10 - 4.13
- Red Hat Advanced Cluster Manager for Kubernetes 2.6 and above
Issue
- How to apply Skip SELinux relabeling workaround with spc_t automatically on workloads running on OpenShift Container Platform 4?
Resolution
This article explores an alternative way using Red Hat Advanced cluster Manager for Kubernetes to automatically add the necessary statements to a pod for skipping the SELinux relabeling.
As documented in the Kb article 6221251, there are two workarounds to skip SELinux relabeling
1. Skip SELinux Relabeling with spc_t
2. Skip SELinux Relabeling if already done with an annotation
For the second workaround, Skip SELinux Relabeling if already done with an annotation, refer the kbase article
Prerequisite
1\. Red Hat Advanced Cluster Manager for Kubernetes(RHACM)
2\. Clusters that needs to be managed by RHACM are already imported
Important Note
1. Namespaces controlled by an operator should be skipped and should not be included in the list of namespaces where the SELinux relabeling workaround has to be applied
2. SELinux Relabeling workaround will be applied to all pods even if they are not using any PVC's.
3. Add the required namespaces to the RHACM policy at the appropriate section in the policy as shown below. For example, policy has thedemo-gknamespace mentioned. Replace it appropriately. Modify the Assign mutation policies to add the required namespaces
Example
namespaces:
- demo-gk
- namespace1
location: "spec.securityContext.fsGroupChangePolicy"
parameters:
assign:
value: "OnRootMismatch"
Applying Skip SELinux Relabeling with spc_t using Red Hat Advanced Cluster Manager for Kubernetes
As per the primary kbase article, to skip SELinux relabeling, pod should include the following in their respective definitions. To prevent the manual addition to the pod, this article explores an automated method.
securityContext:
seLinuxOptions:
type: spc_t
Additionally, to prevent delays due to file ownership update, it is required to add fsGroupChangePolicy: OnRootMismatch
Custom scc
As described in the kbase article, create a custom scc custom which sets runAsUser to MustRunAsRange and has a priority of 5. Use the below definition to create the scc
allowHostDirVolumePlugin: false
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegeEscalation: true
allowPrivilegedContainer: false
allowedCapabilities: null
apiVersion: security.openshift.io/v1
defaultAddCapabilities: null
fsGroup:
type: MustRunAs
groups:
- system:authenticated
kind: SecurityContextConstraints
metadata:
name: custom
priority: 5
readOnlyRootFilesystem: false
requiredDropCapabilities:
- KILL
- MKNOD
- SETUID
- SETGID
runAsUser:
type: MustRunAsRange
seLinuxContext:
type: RunAsAny
supplementalGroups:
type: RunAsAny
users: []
volumes:
- configMap
- downwardAPI
- emptyDir
- persistentVolumeClaim
- projected
- secret
Create the SCC
# oc create -f custom-scc.yaml
securitycontextconstraints.security.openshift.io/custom created
Apply the scc to the default serviceaccount or the serviceaccount used in the namespace.
Example
# oc adm policy add-scc-to-user custom -z default -n demo-gk
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:custom added: "default"
Creating Governance Policy
Login to RHACM -> Select Governance TAB -> Create policy under a policy set so that it will be applied to clusters that are managed by RHACM. Copy the below policy definition to the YAML section
The below policy performs the following on the managed cluster
- Installs Gatekeeper Operator
- Sets up two Gatekeeper Mutation rules
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: policy-gatekeeper-operator
namespace: default
annotations:
policy.open-cluster-management.io/categories: CM Configuration Management
policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
policy.open-cluster-management.io/standards: NIST SP 800-53
spec:
disabled: false
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-operator-subscription
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: gatekeeper-operator-product
namespace: openshift-operators
spec:
channel: stable
installPlanApproval: Automatic
name: gatekeeper-operator-product
source: redhat-operators
sourceNamespace: openshift-marketplace
startingCSV: gatekeeper-operator-product.v0.2.5-0.1683051284.p
remediationAction: enforce
severity: high
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operator.gatekeeper.sh/v1alpha1
kind: Gatekeeper
metadata:
name: gatekeeper
spec:
audit:
auditChunkSize: 500
logLevel: INFO
replicas: 1
mutatingWebhook: Enabled
validatingWebhook: Enabled
webhook:
emitAdmissionEvents: Enabled
logLevel: INFO
replicas: 2
remediationAction: enforce
severity: high
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-spc-t
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: Assign
metadata:
name: spc-t
spec:
applyTo:
- groups:
- ""
kinds:
- Pod
versions:
- v1
location: spec.securityContext.seLinuxOptions.type
match:
kinds:
- apiGroups:
- "*"
kinds:
- Pod
namespaces:
- demo-gk
scope: Namespaced
parameters:
assign:
value: spc_t
remediationAction: enforce
severity: high
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-fsgrouppolicy
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: Assign
metadata:
name: fsgrouppolicy
spec:
applyTo:
- groups: [""]
kinds: ["Pod"]
versions: ["v1"]
match:
scope: Namespaced
kinds:
- apiGroups: ["*"]
kinds: ["Pod"]
namespaces:
- demo-gk
location: "spec.securityContext.fsGroupChangePolicy"
parameters:
assign:
value: "OnRootMismatch"
remediationAction: enforce
severity: high
remediationAction: enforce
Verifying whether the policy has been created
On the managed cluster, execute the following command to verify if the gatekeeper policies are installed
# oc get assign
NAME AGE
fsgrouppolicy 33s
spc-t 33s
Verification
Verify whether the SELinux relabeling workaround has been applied automatically when a new pod/deployment is created
# oc get pod selinux-relabel-openshift1 -o yaml |egrep -i 'spc_t|fsgroupchange'
fsGroupChangePolicy: OnRootMismatch
type: spc_t
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.