OpenShift Virtualization: OADP or Cohesity backup freezes VM during backup

Solution Verified - Updated

Environment

  • OpenShift
    • 4.18.21
  • OADP
    • 1.5
  • Third Party: Cohesity (managing the Velero Backup CR)

Issue

During any regular VM Backup process, OADP freezes the VM for a short time to take a backup which will cause real time applications to fail.

  • Customer reported PartiallyFailed status on VM back-ups
  • Cohesity is reporting fully failed back-ups, resulting in no back-up of metadata.

Resolution

The permanent solution is to remove the virt-freezer hooks from being executed in the virt-launcher pods before a VM backup. This effort is being tracked via bug This content is not included.CNV-75370, but it’s not expected to be resolved in the near term.

As a temporary workaround, Kyverno policies can be used to automatically remove the virt-freezer hooks. Instructions are provided in this Content from github.com is not included.github page. Installation and basic verification instructions are copied below.

Please review the github page for additional tests, debugging procedures, and common issues.

Installation

1. Install Kyverno

Add the Kyverno Helm repository:

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update

Install Kyverno:

helm install kyverno kyverno/kyverno \
  --namespace kyverno \
  --create-namespace \
  --set admissionController.replicas=3 \
  --set backgroundController.replicas=2

Wait for Kyverno to be ready:

kubectl wait --for=condition=Ready pods -l app.kubernetes.io/instance=kyverno -n kyverno --timeout=300s

2. Apply RBAC

The background controller needs permission to update pods:

kubectl apply -f kyverno-rbac.yaml

3. Apply the Policies

kubectl apply -f kyverno-policy.yaml

Verification

Check policies are installed

kubectl get clusterpolicies

Expected output:

NAME                                    ADMISSION   BACKGROUND   READY   AGE   MESSAGE
kubevirt-strip-velero-hooks-admission   true        true         True    1m    Ready
kubevirt-strip-velero-hooks-reconcile   true        true         True    1m    Ready

Please review the github page for additional tests, debugging procedures, and common issues.

Root Cause

As part of the OADP backup process of a VM, OpenShift freezes the VM via virt-freezer hooks in the virt-launcher pods. These VM freezes are done to maintain data integrity during the backup process, and are done unconditionally, including metadata-only backups. Cohesity is not expecting the VM to be frozen for metadata backups, and as a result, their backup operation fails.

Diagnostic Steps

Look for error messages like these in the OADP Velero pods in the Cohesity namespace:

$ oc -n cohesity-8563618369352193 logs velero-66fbc56b5b-4t7x4 | grep cohesity | grep virt-freezer

2026-01-30T13:25:42.104005814Z time="2026-01-30T13:25:42Z" level=info msg="stdout: " backup=cohesity-8563618369352193/8563618369352193-26755373 hookCommand="[/usr/bin/virt-freezer --freeze --name tpolwsccm00080 --namespace apm0100323]" hookContainer=compute hookName="<from-annotation>" hookOnError=Fail hookPhase=pre hookSource=annotation hookTimeout="{30s}" hookType=exec logSource="/remote-source/velero/app/pkg/podexec/pod_command_executor.go:180" name=virt-launcher-tpolwsccm00080-6j2mh namespace=apm0100323 resource=pods

2026-01-30T13:25:42.104152819Z time="2026-01-30T13:25:42Z" level=info msg="stderr: {\"component\":\"freezer\",\"level\":\"info\",\"msg\":\"Guest agent version is 109.1.0\",\"pos\":\"virt-freezer.go:114\",\"timestamp\":\"2026-01-30T13:25:35.663379Z\"}\n{\"component\":\"freezer\",\"level\":\"error\",\"msg\":\"Freezing VMI failed, please make sure guest agent and VSS are running and try again\",\"pos\":\"virt-freezer.go:131\",\"reason\":\"server error. command Freeze failed: \\\"LibvirtError(Code=86, Domain=10, Message='Guest agent is not responding: Guest agent not available for now')\\\"\",\"timestamp\":\"2026-01-30T13:25:42.099993Z\"}\n" backup=cohesity-8563618369352193/8563618369352193-26755373 hookCommand="[/usr/bin/virt-freezer --freeze --name tpolwsccm00080 --namespace apm0100323]" hookContainer=compute hookName="<from-annotation>" hookOnError=Fail hookPhase=pre hookSource=annotation hookTimeout="{30s}" hookType=exec logSource="/remote-source/velero/app/pkg/podexec/pod_command_executor.go:181" name=virt-launcher-tpolwsccm00080-6j2mh namespace=apm0100323 resource=pods
SBR
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.