[Satellite6] After changing Content View of a Host and deleting the old one, candlepin raises 410 Gone error

Environment

• Satellite 6.1

Issue

• migrating some managed Hosts from one Content View to another
  • doing so via Hosts -> particular host -> Content View change in WebUI
• deleting the old Content View after the migration
• since then, candlepin raises 410 Gone errors

Resolution

The underlying bug is tracked in This content is not included.this bugzilla.

As a workaround, whenever updating a Host that way, update the Content Host via hammer the same way:

hammer> content-host update --content-view-id 6 --organization-id 1 --name content-host.example.com

For more KB articles/solutions related to Red Hat Satellite 6.x Content View Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Content View Issues

Root Cause

Updating a Content View of a (managed) Host triggers these actions:

• this updates katello_systems in foreman DB but leaves candlepin consumers in old environment (candlepin'`s environment is a pair of content view and lifecycle environment the content host belongs to) - this is subject of This content is not included.that bugzilla
• after the migration, deleting old content view:
  • passes some preventive check "has katello some Content Host in the Content View", since katello has been properly updated
  • Content View is thus deleted from katello
  • candlepin environemnt is also deleted - including all consumers it has
    • due to This content is not included.this bugzilla, candlepin logs dont contain much details about this removal of Content Hosts / candlepin's consumers
• now one has systems with a record in katello but not in candlepin
• starting to get 410 Gone errors
• An attempt to call clean_backend_objects rake script then would delete the existing systems also from katello

The workaround in updating Content Host as an extra action triggers an explicit update of candlepin, causing candlepin and katello databases are aligned again.

Diagnostic Steps

As a result, production.log prints:

2016-02-03 12:30:45 [E] RestClient::Gone: Katello::Resources::Candlepin::Consumer: 410 Gone {"displayMessage":"Unit 6f46f97d-76ec-4175-926d-23af3b71895e has been deleted","requestUuid":"cce87f5d-9303-41f8-9947-b822c8232f68","deletedId":"6f46f97d-76ec-4175-926d-23af3b71895e"} (GET /candlepin/consumers/6f46f97d-76ec-4175-926d-23af3b71895e)

Note however, there can be different scenarios leading to the above error.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.