Unable to fix broken repository metadata even after performing a force full re-sync.

Environment

• Red Hat Enterprise Linux
• Satellite 6.2.9+

Issue

• Not able to apply errata to some systems registered to a content view.
• Errata is marked as installable in satellite content hosts, but 'yum update' does not list errata in client.
• Repository has missing rpms.

Resolution

• There is a bug in Satellite 6.2. For more information about this bug please refer to the Root Cause section.

• Before proceeding, please refer to the "Diagnostic Steps" section to check whether the patch is applicable to you or not.

• This bug can be fixed by applying the following patch to the Satellite server and all Capsule servers.

  • Download the attached "pulp_rpm_publish.txt" patch file.

  • SSH to the Satellite server or Capsule server.

  • Run the command to apply the download patch.

      patch /usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py /<PATH>/<TO>/pulp_rpm_publish.txt
    

  • Restart Pulp workers. Make sure you are not in the middle of regenerating applicability or syncing repos before restarting the workers.

      systemctl restart pulp_workers pulp_resource_manager pulp_celerybeat
    

  • Repeat the above steps for each of the Capsule server.

  • NOTE: The patch will be removed after upgrading your Satellite to the newer 6.2.z version so you need to re-apply the patch.

• To fix broken Red Hat repository, you can now perform a full sync from Red Hat CDN using hammer command. For example:

# hammer repository synchronize --id <repo id> --skip-metadata-check true

• To fix broken content view repositories, you can now perform a full publish/promote using hammer command. For example:

# hammer content-view publish --name <cv name> --organization <org> --force-yum-metadata-regeneration true
# hammer content-view version promote --organization <org> --to-lifecycle-environment <to LCE> --content-view <cv_name> --version <cv version> --force-yum-metadata-regeneration true

• To fix all broken repositories in the Capsule, you can initiate a full Capsule content sync using hammer command. For example:

# hammer capsule content synchronize --id <capsule id> --skip-metadata-check true

For more KB articles/solutions related to Red Hat Satellite 6.x Repository Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Repository Issues.

Root Cause

• Satellite didn't perform a full repository synchronization or full metadata regeneration even "--skip-metadata-check" or " --force-yum-metadata-regeneration" option has been specified in the hammer command. For more information please refer to This content is not included.force_full publish a repository doesn't work

Diagnostic Steps

• To check whether the provided patch is applicable to your Satellite or Capsule. You need to SSH to your Satellite or Capsule and run the below command.

# grep -n --color "force_full = False" /usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py
283:        force_full = False  <=========== If you see this output that means you have not applied the patch.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.