Certain repositories fail to sync, failing with pulp nectar.downloader errors due to proxy connection on Red Hat Satellite.

Solution Verified - Updated

Environment

  • Red Hat Satellite 6
  • An environment configured such that Satellite 6 syncs to Red Hat CDN through a proxy.

Issue

  • All repositories sync to the Red Hat Content Delivery Network (CDN) as expected through the environment proxy except for single repositories. Attempts to sync these repositories fail with the below error seen in /var/log/messages (where 123.456.789.111 is the IP of the proxy in the environment):

      pulp: nectar.downloaders.threaded:ERROR: Skipping requests to cdn.redhat.com due to repeated connection failures: 
  HTTPSConnectionPool(host='cdn.redhat.com', 
  port=443): Max retries exceeded with url: /content/dist/rhel/server/6/6Server/x86_64/sat-tools/6.2/os/repodata/repomd.xml 
  (Caused by 
  ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x2ddccd0>, u'Connection to 
  123.456.789.111 timed out. 
  (connect 
  timeout=6.05)'))

Resolution

  • Ensure that the Satellite itself is properly configured to connect to cdn.redhat.com via the correct proxy server as per the following article and firewall\proxy is properly configured as per this article.

  • If the repository is determined to be configured with the incorrect proxy information (see Diagnostic Steps), the following steps may be taken to correct the issue:

  • Install pulp-admin per this article: How to deploy and use pulp-admin

  • Update the repo metadata, replacing the REPO_ID with the repo id defined in repo_importers.txt (from Diagnostic Steps):

      # pulp-admin -u admin -p $(grep ^default_password /etc/pulp/server.conf | cut -d' ' -f2) rpm repo update --repo-id="REPO_ID" --proxy-host="" --proxy-port=""

  • Attempt to synchronize the repository once again, it will use the proxy definitions outlined in /etc/pulp/server/plugins.conf.d/yum_importer.json

For more KB articles/solutions related to Red Hat Satellite 6.x Repository Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Repository Issues.

Root Cause

  • The importers for the affected repositories have been configured with incorrect proxy information in the pulp database.

Diagnostic Steps

  • Check /var/log/messages for errors:

      pulp: nectar.downloaders.threaded:ERROR: Skipping requests to cdn.redhat.com due to repeated connection failures: HTTPSConnectionPool(host='cdn.redhat.com', port=443): 
  Max retries exceeded with url: /content/dist/rhel/server/6/6Server/x86_64/sat-tools/6.2/os/repodata/repomd.xml (Caused by 
  ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x2ddccd0>, u'Connection to 123.456.789.111 timed out. (connect 
  timeout=6.05)'))

  • Export the importer information configured in mongo for the affected repository:

      # mongo pulp_database --eval "DBQuery.shellBatchSize = 10000000; db.repo_importers.find().shellPrint()" > repo_importers.txt

  • View the generated repo_importers.txt file for a line such as the below, checking that the proxy IP and port are defined correctly:

      { "_id" : ObjectId("57dc7b09332e734f543a2c52"), "_ns" : "repo_importers", "config" : { "feed" : "https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/sat- 
  tools/6.2/os", [...SNIP...], "remove_missing" : true, "proxy_host" : "http://123.456.789.111", "ssl_client_key" [...SNIP...] "proxy_port" : 1234, "download_policy" : "immediate" }, 
  "importer_type_id" : "yum_importer", "last_sync" : "2016-11-10T08:08:00Z", "repo_id" : "SMI-Red_Hat_Enterprise_Linux_Server- 
  Red_Hat_Satellite_Tools_6_2_for_RHEL_6_Server_RPMs_x86_64", "scratchpad" : { "repomd_revision" : 1475563406, "previous_skip_list" : [ ] } }

  • In the above example, the repo id, proxy IP, and ports are defined as:

      "repo_id" : "SMI-Red_Hat_Enterprise_Linux_Server-Red_Hat_Satellite_Tools_6_2_for_RHEL_6_Server_RPMs_x86_64"
  "proxy_host" : "http://123.456.789.111"
  "proxy_port" : 1234
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.