[Satellite6] foreman-rake katello:reindex (or reimport) fails on Katello::Resources::Candlepin::Consumer: 410 Gone

Solution Verified - Updated

Environment

  • Red Hat Satellite 6

Issue

  • foreman-rake katello:reindex (or katello:reimport since 6.2.7) fails after the line "Re-indexing Katello::System" with the message:
..
Re-indexing Katello::System
rake aborted!
Katello::Resources::Candlepin::Consumer: 410 Gone {"displayMessage":"Unit 66fdf601-764e-4cf8-a5cf-ecf0ebe5a0af has been deleted","requestUuid":"46423f10-6661-4517-9531-38123a3f5da6","deletedId":"66fdf601-764e-4cf8-a5cf-ecf0ebe5a0af"}
  • Can't delete the host machine from Satellite and getting the following error:
Error: Katello::Resources::Candlepin::CandlepinResource: 404 Resource Not Found {"displayMessage":"Runtime Error RESTEASY001185: Could not find resource for relative : /consumers//entitlements of full path: https://localhost:8443/candlepin/consumers//entitlements at org.jboss.resteasy.core.registry.PathParamSegment.matchPattern:209","requestUuid":"3c1d0f37-227f-56a6-9537-88d3bf6b11e9"} (GET /candlepin/consumers//entitlements)

Resolution

  • Remove the system consumer using the following command:
# foreman-rake console
Loading production environment (Rails 3.2.8)
irb(main):001:0> Katello::System.find_by_uuid("66fdf601-764e-4cf8-a5cf-ecf0ebe5a0af").destroy
  • After that re-run the "foreman-rake katello:reindex" command.
# foreman-rake katello:reindex

(for Satellite 6.2.7 or newer, use foreman-rake katello:reimport instead)

Notice: If you know or suspect there are multiple systems that might be partially missing, you can generate their list per this comment and run Katello::System.find_by_uuid("<uuid>").destroy for all of them one after another.

Important: It is possible the deleted record was for a valid Satellite's consumer / Content Host (that has a record in just katello database but not in candlepin one). Check if that Content Host exists and if so, re-register via subscription-manager it to let Satellite to know it.

For more KB articles/solutions related to Red Hat Satellite 6.x Candlepin Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Candlepin Issues

Root Cause

Katello and candlepin got misaligned. One known possible scenario is described This content is not included.in this bugzilla.

Diagnostic Steps

# foreman-rake katello:reindex
Elasticsearch Indices cleared.
Re-indexing User
Re-indexing Organization
Re-indexing Katello::ActivationKey
Re-indexing Katello::TaskStatus
Re-indexing Katello::ContentView
Re-indexing Katello::ContentViewFilter
Re-indexing Katello::Repository
Re-indexing Katello::ContentViewErratumFilterRule
Re-indexing Katello::ContentViewHistory
Re-indexing Katello::ContentViewPackageFilterRule
Re-indexing Katello::SyncPlan
Re-indexing Katello::ContentViewPackageGroupFilterRule
Re-indexing Katello::ContentViewPuppetEnvironment
Re-indexing Katello::ContentViewPuppetModule
Re-indexing Katello::Distributor
Re-indexing Katello::GpgKey
Re-indexing Katello::HostCollection
Re-indexing Katello::System
rake aborted!
Katello::Resources::Candlepin::Consumer: 410 Gone {"displayMessage":"Unit 66fdf601-764e-4cf8-a5cf-ecf0ebe5a0af has been deleted","requestUuid":"46423f10-6661-4517-9531-38123a3f5da6","deletedId":"66fdf601-764e-4cf8-a5cf-ecf0ebe5a0af"}
  • Delete the host from satellite GUI:
Error: Katello::Resources::Candlepin::CandlepinResource: 404 Resource Not Found {"displayMessage":"Runtime Error RESTEASY001185: Could not find resource for relative : /consumers//entitlements of full path: https://localhost:8443/candlepin/consumers//entitlements at org.jboss.resteasy.core.registry.PathParamSegment.matchPattern:209","requestUuid":"3c1d0f37-227f-56a6-9537-88d3bf6b11e9"} (GET /candlepin/consumers//entitlements)
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.