RHV: Cannot upgrade to RHV 4.4 with "A VM running the engine ("hosted engine") cannot be set to highly available as it has its own HA mechanism."

Environment

• Red Hat Virtualization 4.4
  • ovirt-engine-4.4.1.10-0.1.el8e
  • ovirt-hosted-engine-setup-2.4.5-1.el8ev.noarch
  • ovirt-ansible-hosted-engine-setup-1.1.7-1.el8ev.noarch
  • ansible-2.9.11-1.el8ae.noarch

Issue

• Cannot upgrade to RHV 4.4
• Cannot restore Hosted-Engine from Backup

Resolution

• Edit the Blank template and disable High Availability, then attempt to upgrade again.

Root Cause

• The Blank template being set with HA makes the newly created HostedEngine VM inherit the HA setting, which is invalid for it. Then the UpdateVm Validation on the next deployment steps fail.
• This is being investigated in This content is not included.Bugzilla 1866956 - Hosted-Engine restore from backup and 4.4 upgrade fail if Blank template is set as HA.

Diagnostic Steps

• The deployment fails with:

[ INFO  ] TASK [ovirt.hosted_engine_setup : Trigger hosted engine OVF update and enable the serial console]
[ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot edit VM. A VM running the engine ("hosted engine") cannot be set to highly available as it has its own HA mechanism.]". HTTP response code is 409.
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Cannot edit VM. A VM running the engine (\"hosted engine\") cannot be set to highly available as it has its own HA mechanism.]\". HTTP response code is 409."}

• Due to the engine command validation:

2020-08-07 10:26:06,550+10 WARN  [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [7d6bebfa-d31b-4f03-962b-f9140e00376c] Validation of action 'UpdateVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_CANNOT_BE_HIGHLY_AVAILABLE_AND_HOSTED_ENGINE

• The Blank Template has High Availability set.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.