Virt-who failed with Error"consumer no longer exists" during hypervisor checkin

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 8
  • Red Hat Subscription Management(RHSM)
  • Red Hat Customer Portal
  • Virtual Data Center(VDC) subscription

Issue

  • Virt-who failed with the following error: 
2021-02-03 18:19:32,531 [virtwho.destination_xxxxxxxxxxx ERROR] MainProcess(27718):Thread-8 @virt.py:_send_data:702 - Error during hypervisor checkin: Communication with subscription manager failed: consumer no longer exists
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/virtwho/virt/virt.py", line 685, in _send_data
    options=self.options)
  File "/usr/lib/python2.7/site-packages/virtwho/manager/subscriptionmanager/subscriptionmanager.py", line 238, in hypervisorCheckIn
    raise ManagerError("Communication with subscription manager failed: consumer no longer exists")
ManagerError: Communication with subscription manager failed: consumer no longer exists
  • Virt-who failing at hypervisor checking, therefore, unable to register new virtual machines.
  • No new Hypervisors reported on the customer portal and getting "Communication with subscription manager failed" Error

Resolution

The issue is caused because the virt-who is unable to identify the consumer and failed with the output "consumer no longer exists".

Run the below commands to unregister the system.

   # subscription-manager unregister
   # subscription-manager remove --all
   # subscription-manager clean

Run the below command to register the system, so the virt-who agent identifies the consumer and starts the Host-Guest mapping.

subscription-manager register

Note: Run the commands on the virt-who configured server.

For more KB articles/solutions related to Virt-who and Virtual Datacenter (VDC) Subscriptions Issues, please refer to the Consolidated Troubleshooting Article for Virt-who and Virtual Datacenter (VDC) Subscriptions Issues

Root Cause

  • The issue is caused because the system failed to check subscription and identity data from the local system which is checked every 4 hours by the subscription management (rhsmcertd) service.
  • rhsmcertd failed to capture the required information due to an intermittent communication failure between the system and the Red Hat Network.

Diagnostic Steps

  • The status will have the error message "ManagerError: Communication with subscription manager failed: consumer no lo...".
[]# systemctl status virt-who.service
? virt-who.service - Daemon for reporting virtual guest IDs to subscription-manager
   Loaded: loaded (/usr/lib/systemd/system/virt-who.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

Feb 03 17:47:57 xxxxxxxxxx /usr/bin/virt-who[27534]: 2021-02-03 17:47:57,154 [virtwho.destination_xxxxxxxxxx INFO] M...found
Feb 03 17:47:57 xxxxxxxxxx /usr/bin/virt-who[27534]: 2021-02-03 17:47:57,156 [virtwho.destination_xxxxxxxxxx INFO] M...found
Feb 03 17:47:57 xxxxxxxxxx/usr/bin/virt-who[27534]: 2021-02-03 17:47:57,157 [virtwho.destination_xxxxxxxxxx INFO] M...found
Feb 03 17:47:57 xxxxxxxxxx /usr/bin/virt-who[27534]: 2021-02-03 17:47:57,158 [virtwho.destination_xxxxxxxxxx INFO] M...found
Feb 03 17:47:57 xxxxxxxxxx /usr/bin/virt-who[27534]: 2021-02-03 17:47:57,159 [virtwho.destination_xxxxxxxxxx INFO] M...found
Feb 03 17:47:57 xxxxxxxxxx /usr/bin/virt-who[27534]: 2021-02-03 17:47:57,159 [virtwho.destination_xxxxxxxxxx INFO] M...uests
Feb 03 17:47:57 xxxxxxxxxx /usr/bin/virt-who[27534]: 2021-02-03 17:47:57,159 [virtwho.destination_xxxxxxxxxx INFO] M...xxxx
Feb 03 17:48:01 xxxxxxxxxx /usr/bin/virt-who[27534]: 2021-02-03 17:48:01,672 [virtwho.destination_xxxxxxxxxx ERROR] ...xists
                                                         ManagerError: Communication with subscription manager failed: consumer no lo...
Feb 03 17:49:02 xxxxxxxxxx systemd[1]: Stopping Daemon for reporting virtual guest IDs to subscription-manager...
Feb 03 17:49:02 xxxxxxxxxx systemd[1]: Stopped Daemon for reporting virtual guest IDs to subscription-manager.
Hint: Some lines were ellipsized, use -l to show in full.
  • rhsm.log (/var/log/rhsm/rhsm.log) have a detailed error message.
[]# tail -15 /var/log/rhsm/rhsm.log
.
.
.
2021-02-04 10:55:00,270 [ERROR] @virt.py:609 - Error during heartbeat: Communication with subscription manager failed: consumer no longer exists
ManagerError: Communication with subscription manager failed: consumer no longer exists
2021-02-04 10:55:00,270 [INFO] @virt.py:651 - Hosts-to-guests mapping for config "xxxxx": 6 hypervisors and 0 guests found
2021-02-04 10:55:00,271 [INFO] @virt.py:651 - Hosts-to-guests mapping for config "xxxxx": 3 hypervisors and 0 guests found
2021-02-04 10:55:00,271 [INFO] @virt.py:651 - Hosts-to-guests mapping for config "xxxxx": 2 hypervisors and 0 guests found
2021-02-04 10:55:00,271 [INFO] @virt.py:651 - Hosts-to-guests mapping for config "xxxxx": 2 hypervisors and 0 guests found
2021-02-04 10:55:00,272 [INFO] @virt.py:651 - Hosts-to-guests mapping for config "xxxxx": 3 hypervisors and 1 guests found
2021-02-04 10:55:00,272 [INFO] @virt.py:651 - Hosts-to-guests mapping for config "xxxxx": 2 hypervisors and 0 guests found
2021-02-04 10:55:00,272 [INFO] @virt.py:682 - Sending updated Host-to-guest mapping to "xxxxx" including 18 hypervisors and 1 guests
2021-02-04 10:55:00,274 [INFO] @subscriptionmanager.py:156 - X-Correlation-ID: xxxxxxxxxxxxxxxxxxxxxxxx
2021-02-04 10:55:04,939 [ERROR] @virt.py:702 - Error during hypervisor checkin: Communication with subscription manager failed: consumer no longer exists
ManagerError: Communication with subscription manager failed: consumer no longer exists
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.