Capsule sync fails with 'A file located at the url https://satellite.example.com/pulp/content/${ORG}/${LCE}/${CV}/content/dist/rhel8/8.X/x86_64/baseos/kickstart/images/boot.iso failed" error

Solution Verified - Updated

Environment

  • Red Hat Satellite 6.10+

Issue

  • Satellite Capsule fails to download image files when syncing a kickstart repository.

    A file located at the url https://satellite.example.com/pulp/content/${ORG}/${LCE}/${CV}/content/dist/rhel8/8.1/x86_64/baseos/kickstart/images/boot.iso failed validation due to checksum.Could not lookup a publication_href for repo XXXX
A file located at the url https://satellite.example.com/pulp/content/${ORG}/${LCE}/${CV}/content/dist/rhel8/8.4/x86_64/baseos/kickstart/images/efiboot.img failed validation due to checksum.Could not lookup a publication_href for repo XXXX

Resolution

  • Firstly, we need to get the relative_path of the kickstart repository. Let says, you are getting the following error:

    A file located at the url https://satellite.example.com/pulp/content/${ORG}/${LCE}/${CV}/content/dist/rhel8/8.6/x86_64/baseos/kickstart/images/boot.iso failed validation due to checksum.

  • In this case, the relative_path of the repository should be content/dist/rhel8/8.6/x86_64/baseos/kickstart.

  • Use the relative_path to get the repository ID as below. Replace content/dist/rhel8/8.6/x86_64/baseos/kickstart with your own relative_path.

    # su - postgres -c "psql foreman -c \"select id, relative_path from katello_repositories where library_instance_id is null and relative_path ilike '%content/dist/rhel8/8.6/x86_64/baseos/kickstart%'\""

  • You should get a similar result listing the repository ID as below:

     id  |                         relative_path                         
-----+---------------------------------------------------------------
 348 | ${Org}/Library/content/dist/rhel8/8.6/x86_64/baseos/kickstart

  • Ensure the download policy of the repository is set to Immediate and then trigger a Complete repository sync. Wait for the task to finish.

    hammer repository update --download-policy immediate --id <REPO ID>
hammer repository synchronize --skip-metadata-check true --id <REPO ID>

  • Repair the kickstart repository in case of any corrupted contents. Wait for the task to finish.

    hammer repository synchronize --validate-contents true --id <REPO ID>

  • Repeat the steps above for other kickstart repositories hitting the same error.

  • Publish and Promote new version of the content views that are using the kickstart repositories.

  • Publish and Promote new version of the Composite content views related to the content views.

  • This should trigger multiple Capsule sync tasks automatically.

  • Monitor the Capsule sync tasks in Web UI -> Monitor -> Tasks page and make sure they are finished successfully.

  • Optionally trigger an Optimized Capsule Sync to make sure the Capsule is fully synced.

For more KB articles/solutions related to Red Hat Satellite 6.x Capsule Sync Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Capsule Sync Issues

Root Cause

  • Image files were not synchronized properly in Pulp 2 or they have been corrupted in Pulp 2. They were later migrated to Pulp 3. While performing Capsule sync, Pulp 3 raises an error while trying to verify the checksum of the corrupted image files.

Diagnostic Steps

  • Run the following script to download the images and treeinfo. NOTE: Replace the relative paths with those you want to check.

    SATELLITE="<REPLACE WITH YOUR SATELLITE FQDN>"
CURL_OPT="--cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key -v -k -O"
ORG="<REPLACE WITH YOUR ORG LABEL>"
cd /tmp
curl ${CURL_OPT} https://${SATELLITE}/pulp/content/${ORG}/Library/content/dist/rhel/server/7/7.9/x86_64/kickstart/treeinfo
curl ${CURL_OPT} https://${SATELLITE}/pulp/content/${ORG}/Library/content/dist/rhel/server/7/7.9/x86_64/kickstart/images/pxeboot/upgrade.img
curl ${CURL_OPT} https://${SATELLITE}/pulp/content/${ORG}/Library/content/dist/rhel/server/7/7.9/x86_64/kickstart/images/pxeboot/initrd.img
curl ${CURL_OPT} https://${SATELLITE}/pulp/content/${ORG}/Library/content/dist/rhel/server/7/7.9/x86_64/kickstart/LiveOS/squashfs.img
curl ${CURL_OPT} https://${SATELLITE}/pulp/content/${ORG}/Library/content/dist/rhel/server/7/7.9/x86_64/kickstart/images/boot.iso

  • After that, print the checksum of those downloaded image files with the commands below:

    cd /tmp
cat treeinfo
for FILE in upgrade.img initrd.img squashfs.img boot.iso; do echo "FILE: ${FILE}"; ls -la ${FILE} ; sha256sum ${FILE}; done

  • Compare the checksums listed in the treeinfo

        For example:
    The checksum of those downloaded image files are not matching those checksums listed in treeinfo.

    Treeinfo:
    LiveOS/squashfs.img = sha256:9bee5b17815b077847ae0f5b49f9dc681548855ca947903ef010182916654fd7
    images/boot.iso = sha256:0f2002c201ac5d8565dd8891e56be3711f45a0ac482f58294bb6d3654ac621d7
    images/pxeboot/initrd.img = sha256:4589ec17474707cc17e5c8112987a04e8ac9b0428db6fac96f1f3511ca6c98ba
    images/pxeboot/upgrade.img = sha256:874f51d3546fca5dbc79c2bca0818dbd009b0567c1c7ed6d4d2ee3a2f5467836
    images/pxeboot/vmlinuz = sha256:9f67343147f945a5e75f14e9012315d8300d78fb67b1a35db1e561b0b261c06c

    FILE: upgrade.img
    0046cecf9390b5e23ade5b444a58bf21c877f2d3011535f883741c44aaa39aa1  upgrade.img
    FILE: initrd.img
    db96c089a37e71c91fb52cf0f6e0f08e6ca151db35f1aa152dfa0aa55ffb0f00  initrd.img
    FILE: squashfs.img
    135c4c0ec6704fdc750db6791a25d5ac05d26b4743c2c1ebd654cf4449854e3d  squashfs.img
    FILE: boot.iso
    196c0e2eec615d680efcd8025db057a80b4247550fdf84d3c5330860b5dd4130  boot.iso
SBR
Components
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.