How to apply Skip SELinux Relabeling if already done with an annotation automatically on workloads running on OpenShift Container Platform 4?
Environment
- OpenShift Container Platform 4.10 - 4.13
- Red Hat Advanced Cluster Manager for Kubernetes 2.6 and above
Issue
- How to apply
Skip SELinux Relabeling if already done with an annotationdocumented in Kb article 6221251 automatically on a namespace in OpenShift container Platform 4 ?
Resolution
This article explores an alternative way using Red Hat Advanced cluster Manager for Kubernetes to automatically add the necessary statements to a pod for skipping the SELinux relabeling.
As documented in the Kb article 6221251, there are two workarounds to skip SELinux relabeling
1. Skip SELinux Relabeling with spc_t
2. Skip SELinux Relabeling if already done with an annotation
For the first workaround, Skip SELinux Relabeling with spc_t, refer the kbase article
Prerequisite
1\. Red Hat Advanced Cluster Manager for Kubernetes(RHACM)
2\. Clusters that needs to be managed by RHACM are already imported
Important Note
1. Namespaces controlled by an operator should be skipped and should not be included in the list of namespaces where the SELinux relabeling workaround has to be applied
2. SELinux Relabeling workaround will be applied to all pods even if they are not using any PVC's.
3. Add the required namespaces to the RHACM policy at the appropriate section in the policy as shown below. For example, the policy has thedemo-gknamespace mentioned. Replace it appropriately. Modify the Assign/AssignMetadata mutation policies to add the required namespaces
4. Machineconfig for enabling runtimeclass and selinux runtimeclass API object should be created. Steps are included in the Kb article 6221251 under headingSkip SELinux Relabeling if already done with an annotation
Example
namespaces:
- demo-gk
- namespace1
location: "spec.securityContext.fsGroupChangePolicy"
parameters:
assign:
value: "OnRootMismatch"
Applying Skip SELinux Relabeling if already done with an annotation using Red Hat Advanced Cluster Manager for Kubernetes
As per the primary kbase article, to skip SELinux relabeling, pod should include the following in their respective definitions. To prevent the manual addition to the pod, this article explores an automated method.
apiVersion: v1
kind: Pod
metadata:
name: sandbox
annotations:
io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel: "true"
...
spec:
runtimeClassName: selinux
Additionally, to prevent delays due to file ownership update, it is required to add fsGroupChangePolicy: OnRootMismatch
Creating Governance Policy
Login to RHACM -> Select Governance TAB -> Create policy under a policy set so that it will be applied to clusters that are managed by RHACM. Copy the below policy definition to the YAML section
The below policy performs the following on the managed cluster
- Installs Gatekeeper Operator
- Sets up three Gatekeeper Mutation rules
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: policy-gatekeeper-operator
namespace: default
annotations:
policy.open-cluster-management.io/categories: CM Configuration Management
policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
policy.open-cluster-management.io/standards: NIST SP 800-53
spec:
disabled: false
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-operator-subscription
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: gatekeeper-operator-product
namespace: openshift-operators
spec:
name: gatekeeper-operator-product
channel: stable
installPlanApproval: Automatic
source: redhat-operators
sourceNamespace: openshift-marketplace
startingCSV: gatekeeper-operator-product.v0.2.5-0.1683051284.p
remediationAction: enforce
severity: high
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operator.gatekeeper.sh/v1alpha1
kind: Gatekeeper
metadata:
name: gatekeeper
spec:
audit:
auditChunkSize: 500
logLevel: INFO
replicas: 1
mutatingWebhook: Enabled
validatingWebhook: Enabled
webhook:
emitAdmissionEvents: Enabled
logLevel: INFO
replicas: 2
remediationAction: enforce
severity: high
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-applyruntimeclass
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: Assign
metadata:
name: applyruntimeclass
spec:
applyTo:
- groups:
- ""
kinds:
- Pod
versions:
- v1
location: spec.runtimeClassName
match:
kinds:
- apiGroups:
- "*"
kinds:
- Pod
namespaces:
- demo-gk
scope: Namespaced
parameters:
assign:
value: selinux
remediationAction: enforce
severity: high
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-fsgrouppolicy
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: Assign
metadata:
name: apply-fsgrouppolicy
spec:
applyTo:
- groups:
- ""
kinds:
- Pod
versions:
- v1
location: spec.securityContext.fsGroupChangePolicy
match:
kinds:
- apiGroups:
- "*"
kinds:
- Pod
namespaces:
- demo-gk
scope: Namespaced
parameters:
assign:
value: OnRootMismatch
remediationAction: enforce
severity: high
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-applyannotation
spec:
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: AssignMetadata
metadata:
name: apply-annotation
spec:
location: metadata.annotations."io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel"
match:
kinds:
- apiGroups:
- "*"
kinds:
- Pod
namespaces:
- demo-gk
scope: Namespaced
parameters:
assign:
value: "true"
remediationAction: enforce
severity: high
remediationAction: enforce
Verification
On the managed cluster, execute the following command to verify if the gatekeeper policies are installed
# oc get assign
NAME AGE
apply-fsgrouppolicy 17h
applyruntimeclass 18h
# oc get assignmetadata
NAME AGE
apply-annotation 4m32s
Verify whether the SELinux relabeling workaround has been applied on a newly created pod
# oc get pod httpd1 -o yaml | egrep -i 'runtimeclassname|TrySkipVolumeSELinuxLabel|fsgroup'
io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel: "true"
runtimeClassName: selinux
fsGroupChangePolicy: OnRootMismatch
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.