[Satellite 6.7] regenerate applicability is empty for random Content Hosts

Solution Verified - Updated

Environment

Red Hat Satellite 6.7

Issue

  • yum on a Content Host shows some package or errata is installable
  • Satellite shows no content is applicable or installable to that Content Host
  • random Content Hosts are affected at a time, and the set of affected systems is bit varying over time

Resolution

For a final resolution, upgrade to Satellite 6.8 or later in which the issue is fixed. See this Bugzilla and Errata for more details:

Potential workaround that might but not need to help: forcefully regenerate applicability for affected hosts as follows:

  • Find out Host's identity - either in WebUI (Content Hosts -> specific host -> UUID field), or by running subscription-manager identity | grep identity)

  • regenerate applicability for that host, by running on Satellite:

uuid=<the-host's-UUID>
pulpAdminPassword=$(grep ^default_password /etc/pulp/server.conf | cut -d' ' -f2)
curl -u admin:$pulpAdminPassword -X POST https://$(hostname -f)/pulp/api/v2/consumers/${uuid}/actions/content/regenerate_applicability/

The curl command should return something like:
{"spawned_tasks": [{"_href": "/pulp/api/v2/tasks/82e287fd-ca10-4d60-9e65-9faf6586db37/", "task_id": "82e287fd-ca10-4d60-9e65-9faf6586db37"}], "result": null, "error": null}

  • Check status of the regenerate applicability task (use proper task_id):
curl -u admin:$pulpAdminPassword https://$(hostname -f)/pulp/api/v2/tasks/82e287fd-ca10-4d60-9e65-9faf6586db37/ | python -m json.tool
  • once you see there:
    "state": "finished",

then the calculation is done. In case WebUI does not yet show an installable package, run further:

foreman-rake katello:import_applicability

In case the above does not help, one could try manually applying the fix (not supported until released in an official hotfix/errata), as follows:

  • Fix code:
wget https://github.com/pulp/pulp/commit/d7a37903bf81ea8ad6974b5101499b2fc97ded92.patch -O /tmp/bz1835241.patch
cd /usr/lib/python2.7/site-packages
cat /tmp/bz1835241.patch | patch -p2
  • Apply the change (ideally when Satellite is idle):
for i in pulp_celerybeat pulp_resource_manager pulp_streamer pulp_workers httpd; do service $i restart; done

  • Since now, new batch regenerate applicability for repos should work well. I.e. syncing a repo with new content synced, applicability should be properly calculated for it.

  • To fix currently missing applicability calculations:

    • identify pulp repo_id for such repository. In WebUI, go to Content -> Products -> select the product -> select the repository. Copy "Backend identifier" (some hash-like UUID).
    • have file /tmp/repo_regen_request.json (update the repo_idhash by your backend identifier):
{
 "repo_criteria": {
  "filters": {"id": {"$in": ["6ad7b30a-be50-4892-ba77-313341114c2e"]}}
 },
 "parallel": true
}
  • request regenerate applicability task:
pulpAdminPassword=$(grep ^default_password /etc/pulp/server.conf | cut -d' ' -f2)
curl -H "Content-Type: application/json" -u admin:$pulpAdminPassword -d @/tmp/repo_regen_request.json https://$(hostname -f)/pulp/api/v2/repositories/actions/content/regenerate_applicability/
  • output will have group_id like:
{"group_id": "1ed066cb-6eaa-4be0-beab-06e93889d5eb", "_href": "/pulp/api/v2/task_groups/1ed066cb-6eaa-4be0-beab-06e93889d5eb/"}
  • use that group_id in query:
curl -u admin:$pulpAdminPassword https://$(hostname -f)/pulp/api/v2/task_groups/1ed066cb-6eaa-4be0-beab-06e93889d5eb/state_summary/
  • once the response shows "finished" = "total" counts, the tasks triggered on background are complete, like:
{"accepted": 0, "finished": 4, "running": 0, "canceled": 0, "waiting": 0, "skipped": 0, "suspended": 0, "error": 0, "total": 4}
  • then, pulp should have the applicability, that must be propagated to katello, via:
foreman-rake katello:import_applicability
  • then you should be able to see it in WebUI as well

For more KB articles/solutions related to Red Hat Satellite 6.x Pulp 2.0 Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Pulp 2.0-related Issues

Root Cause

There is a bug in pulp that skips regenerating applicability for random Content Hosts. See the This content is not included.underlying bugzilla for details.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.