[pulp-3] How to mimic fetching a package or metadata for a repository on Satellite/Capsule/RHUA?

Solution Verified - Updated

Environment

  • Red Hat Satellite or Capsule 6.10+
  • Red Hat Update Infrastructure 4.*

Issue

  • Satellite or Capsule or RHUA is failing to fetch a package or metadata from its upstream (RHUA and Satellite from CDN, Capsule from Satellite or some Alternate Content Source)
  • how to mimic the failing request on command line?
  • how to mimic synchronizing some whole repository?
  • is there a way to generate a curl command for that?

Resolution

Upload the attached mimic-package-fetch.py script to the Satellite/Capsule/RHUI, and run it in either way:

cat mimic-package-fetch.py | sudo -u pulp REMOTEURL="https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2/os" PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager shell

cat mimic-package-fetch.py | sudo -u pulp DPATH="RedHat/Library/content/dist/layered/rhel8/x86_64/ansible/2/os" PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager shell

cat mimic-package-fetch.py | sudo -u pulp REPONAME="Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs-3175746" PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager shell

cat mimic-package-fetch.py | sudo -u pulp REPOID="ba35fc38-81b9-4777-86d3-270c9dfb5724" PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager shell

where one should use either of REMOTEURL or DPATH or REPONAME or REPOID variables, with meaning:

  • REMOTEURL should be the full URL of the repository.
  • DPATH should be a (prefix of) base_path where clients of the Satellite/Capsule consume the content from. The path can contain also Content View and Lifecycle Environment, as the script should be clever enough to find out the root Repository behind it.
  • REPONAME must be Pulp's repository name - but only the "Root" one (not a Content View clone). Use only when you are certain of the value.
  • REPOID must the the UUID of Pulp's repository, again the "Root" one, not from a Content View.

If unsure of possible values, provide a dummy one and the script prints all available options, like:

cat mimic-package-fetch.py | sudo -u pulp REMOTEURL="blah" PULP_SETTINGS='/etc/pulp/settings.py' \ 
DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager shell
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
No Remote found for given URL 'blah'. Available values:
---------------------------------
https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2/os
https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/sat-client/6/os
https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/sat-maintenance/6.13/os
https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/sat-utils/6.13/os
https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/sat-client/6/os
https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os
https://cdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os
https://cdn.redhat.com/content/dist/rhel9/9/x86_64/appstream/os
https://cdn.redhat.com/content/dist/rhel9/9/x86_64/baseos/os
https://cdn.redhat.com/content/dist/rhel/server/7/7.8/x86_64/optional/os
https://cdn.redhat.com/content/dist/rhel/server/7/7.9/x86_64/optional/os
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/optional/os

Expected output of the script is curl command with certificates to use, like:

cat mimic-package-fetch.py | sudo -u pulp REMOTEURL="https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2/os"  PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager shell

curl -vv --cacert /var/tmp/Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs-3178556.ca_cert.pem --cert /var/tmp/Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs-3178556.client_cert.pem --key /var/tmp/Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs-3178556.client_key.pem  https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2/os/

Run that curl command.

Satellite/Capsule 6.9 or older

For such old Satellite/Capsule that runs pulp-2, use the attached script mimic-pulp2-repo-sync.sh with an argument of pulp repository id (seen in the failing step of sync task). Expected output is something like:

# ./mimic-pulp2-repo-sync.sh fe06288f-326c-48b9-9111-c14cf2e29901
command to run:   curl --cacert ./ssl_ca_cert.fe06288f-326c-48b9-9111-c14cf2e29901.crt --cert ./ssl_client_cert.fe06288f-326c-48b9-9111-c14cf2e29901.crt --key ./ssl_client_key.fe06288f-326c-48b9-9111-c14cf2e29901.crt --proxy https://proxy.example.com:8080 --proxy-user testuser:testpassword https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os

* Could not resolve proxy: proxy.example.com; Unknown error
* Closing connection 0
curl: (5) Could not resolve proxy: proxy.example.com; Unknown error

certificates used:  ssl_ca_cert.fe06288f-326c-48b9-9111-c14cf2e29901.crt ssl_client_cert.fe06288f-326c-48b9-9111-c14cf2e29901.crt ssl_client_key.fe06288f-326c-48b9-9111-c14cf2e29901.crt
                    we recommend to remove them after inspection
#
  • One can provide additional arguments like:
./mimic-pulp2-repo-sync.sh repoID importerType testRepo repoData packages
       .. where importerType=yum, testRepo=yes, repoData=no and packages=no by default

where the arguments stand for:

  • repository ID like before

  • type of the repository (yum, iso,..)

  • if to test fetching repository directory like before

  • if to download repository metadata

  • if to download each and every package and verify its checksum

  • One can use the curl command in the script output for further troubleshooting (until the *crt files for accessing CDN are deleted).

  • E.g. to mimic synchronizing whole repository, one can use attached mimic-pulp2-repo-sync.sh script with arguments yum no yes yes. Test it carefully, first on some small repository. Be aware, downloading packages of a bigger repository requires lots of disk space (and time).

For more KB articles/solutions related to Red Hat Satellite 6.x Repository Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Repository Issues.

For more KB articles/solutions related to Red Hat Satellite 6.x Pulp 2.0 Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Pulp 2.0-related Issues

For more KB articles/solutions related to Red Hat Satellite 6.x Pulp 3.0 Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Pulp 3.0-related Issues

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.