[Red Hat Satellite 6.2] RestClient::InternalServerError: 500 Internal Server Error while syncing repositories

Solution Verified - Updated

Environment

  • Red Hat Satellite or Proxy

Issue

  • Synchronizing repository on Red Hat Satellite Server shows RestClient::InternalServerError: 500 Internal Server Error.
Action:
Actions::Pulp::Repository::Create
Input:
{"content_type"=>"yum",
 "pulp_id"=>
  "STTCONNECT-Red_Hat_Enterprise_Linux_Server-Red_Hat_Satellite_Tools_6_2_for_RHEL_7_Server_RPMs_x86_64",
 "name"=>"Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64",
 "docker_upstream_name"=>nil,
 "feed"=>
  "https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os",
 "ssl_ca_cert"=>
  "-----BEGIN CERTIFICATE-----\nMIIG/TCCBOWgAwIBAgIBNzANBgkqhkiG9w0BAQUFADCBsTELMAkGA1UEBhMCVVMx\nFzAVBgNVBAgMDk5vcnRoIENhcm9saW5hMRYwFAYDVQQKDA1SZWQgSGF0LCBJbmMu\nMRgwFgYDVQQLDA9SZWQgSGF0IE5ldHdvcmsxMTAvBgNVBAMMKFJlZCBIYXQgRW50\naXRsZW1lbnQgT3BlcmF0aW9ucyBBdXRob3JpdHkxJDAiBgkqhkiG9w0BCQEWFWNh\nLXN1cHBvcnRAcmVkaGF0LmNvbTAeFw0xMDEwMDQxMzI3NDhaFw0zMDA5MjkxMzI3\nNDhaMIGuMQswCQYDVQQGEwJVUzEXMBUGA1UECAwOTm9ydGggQ2Fyb2xpbmExFjAU\nBgNVBAoMDVJlZCBIYXQsIEluYy4xGDAWBgNVBAsMD1JlZCBIYXQgTmV0d29yazEu\nMCwGA1UEAwwlUmVkIEhhdCBFbnRpdGxlbWVudCBQcm9kdWN0IEF1dGhvcml0eTEk\nMCIGCSqGSIb3DQEJARYVY2Etc3VwcG9ydEByZWRoYXQuY29tMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEA2QurMeAVnCHVsuZNQzciWMdpd4LAVk2eGugN\n0cxmBpzoVI8lIsJOmJkpOAuFOQMX9CBr8RuQyg4r1/OH/rfhm6FgGIw8TGKZoWC/\n1B9teZqTiM85k6/1GRNxdk6dUK77HVO0PMIKtNBHRxIsXcRzJ1q+u5WPBes9pEVG\nnbidTNUkknrSIdynTJcqAI/I0VAsqLqX87XJSzXKvRilE+p/fLHmVTAffl1Cn/Dy\nKULxna7ooyrKKnfqeQ5dK8aMr1ASQ1wphWohLjegly9V0amEi+HHWnOL8toxJy8v\nWUTUzzAvZ4ZTtTV26xGetZZWEaNyv7YCv2AexjcBQ2x+ejrFJrVNo9jizHS06HK8\nUgHVDKhmVcAe2/5yrJCjKDLwg1FJfjKwhzhLYdNVCejpy8CHQndwO0EX1hHv/AfP\nRTAmr5qPhHFD+uuIrYrSLUpgMLmWa9dinJcGeKlA1KJvG5emGMM3k64Xr7dJToXo\n5loGyZ6lvKPIKLmfeXMRW/4+BqyzwbO1i4aIHAZcSPDFGKWwuvF0iVUYUUVxw0nv\nqPZA4roq5+j/YSz0q5XGVgiIt34htlvunLp/ICGYJBR6zEHcB9aZGJdDcJvoYZjw\n7Gphw6lFF6Ta4imoyhGECWKjd1ips3opcN+DlU0yCUrcIXVIXAnkTwu5ocOgAkxr\nf/6FjqcCAwEAAaOCAR8wggEbMB0GA1UdDgQWBBSW/bscQED/QIStsh8LJsHDam/W\nfDCB5QYDVR0jBIHdMIHagBTESXhWRZ0eLGFgw2ZLWAU3LwMie6GBtqSBszCBsDEL\nMAkGA1UEBhMCVVMxFzAVBgNVBAgMDk5vcnRoIENhcm9saW5hMRAwDgYDVQQHDAdS\nYWxlaWdoMRYwFAYDVQQKDA1SZWQgSGF0LCBJbmMuMRgwFgYDVQQLDA9SZWQgSGF0\nIE5ldHdvcmsxHjAcBgNVBAMMFUVudGl0bGVtZW50IE1hc3RlciBDQTEkMCIGCSqG\nSIb3DQEJARYVY2Etc3VwcG9ydEByZWRoYXQuY29tggkAkYrPyoUAAAAwEgYDVR0T\nAQH/BAgwBgEB/wIBADANBgkqhkiG9w0BAQUFAAOCAgEArWBznYWKpY4LqAzhOSop\nt30D2/UlCSr50l33uUCNYD4D4nTr/pyX3AR6P3JcOCz0t22pVCg8D3DZc5VlzY7y\nP5RD3KbLxFNJTloclMG0n6aIN7baA4b8zwkduMQvKZnA/YNR5xE7V7J2WJHCEBBB\nZ+ZFwGpGsoZpPZP4hHLVke3xHm6A5F5SzP1Ug0T9W80VLK4jtgyGs8l1R7rXiOIt\nNik8317KGq7DU8TI2Rw/9Gc8FKNfUYcVD7uC/MMQXJTRvkADmNLtZM63nhzpg1Hr\nhA6U5YcDCBKsPA43/wsPOONYtrAlToD5hJhU+1Rhmwcw3qvWBO3NkdilqGFOTc2K\n50PQrqoRTCZFS41nv2WqZFfbvSq4dZRJl8xpB4LAHSspsMrbr9WZHX5fbggf6ixw\nS9KDqQbM7asP0FEKBFXJV1rE8P/oSK6yVWQyigTsNcdGR4AUzDsTO9udcwoM2Ed4\nXdakVkF+dXm9ZBwv5UBf5ITSyMXL3qlusIOblJVGUQizumoq0LiSnjwbkxh2XHhd\nXD/B/qax7FnaNg+TfujR/kk3kF1OpqWx/wC/qPR+zho1+35Al31gZOfNIn/sReoM\ntcci9LFHGvijIy4VUDQK8HmGjIxJPrIIe1nB5BkiGyjwn00D5q+BwYVst1C68Rwx\niRZpyzOZmeineJvhrJZ4Tvs=\n-----\nBmi1YX1eXVIZHlguFxprM3RvjaXap9I9nPzg+lwleNOtzh442ql2XiC7CUFh9ZmG\nfB3vITRGagvJyBbDQlaaPdjd1fk3EXdNzK6YVhDgIL9+eRFoQbkUk9KFMMArXz+m\nUJBydEfocl7SkYTDTkFr2oqBVgre9D4yI50VXpuC7b4GqcQHvk12d2jy/3qtC0r1\n6hGqVL5svuhi/4Zzt3YiJY/5nY83c70rMxDyvFO6h9vbeY4fakS8ZV9X+DPBwhLu\nbXoQUjEbGkUiiVxeekS60hs7+utP5ZvLLZkIFj1FMSaUApvsfHAlLlHcsrjLiifu\n3G2Z995Lspqw62biieGEEdLv+EN/XMEzYUKu+ZmukCg=\n-----END RSA SIGNATURE-----\n",
 "ssl_client_key"=>
  "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAng4nNY9p+wXlVmcqHRmKrNATaSH8bsVeTMUjHjcEapMvPa4b\n5NRxVoO+jGdiL9K3cxmbxMoAnWd1+6XmYLbjFuAJidI9PRrLzDOd6leJHJhFluBB\n3b/+U9cvecos0veNRH8rCESRlkS4IBgyNxI89ZYq9xOcWJ6/Ym\n-----END RSA PRIVATE KEY-----\n",
 "unprotected"=>false,
 "checksum_type"=>nil,
 "path"=>
  "STTCONNECT/Library/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.2/os",
 "download_policy"=>"immediate",
 "with_importer"=>true,
 "mirror_on_sync"=>true,
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin"}

Output:
{}

Exception:
RestClient::InternalServerError: 500 Internal Server Error


Backtrace:
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!'
/opt/theforeman/tfm/root/usr/share/gems/gems/runcible-1.7.0/lib/runcible/base.rb:79:in `block in get_response'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:228:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:228:in `process_result'
/opt/theforeman/tfm/root/usr/share/gems/gems/rbovirt-0.0.37/lib/restclient_ext/request.rb:50:in `block in transmit'

Resolution

  • Check permission for below directory on Satellite Server.
    ls -ltr /etc/pki/pulp/content/pulp-protected-repos.
  • Re-run satellite-installer --scenario satellite

For more KB articles/solutions related to Red Hat Satellite 6.x Repository Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Repository Issues.

Diagnostic Steps

  • /var/log/messages on Satellite shows ERROR's related to pulp permissions for Satellite 6.2 Repos.
Mar 30 15:47:49 satellite pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Mar 30 15:47:49 satellite pulp: py.warnings:WARNING: (12885-67360) /usr/lib/python2.7/site-packages/mongoengine/document.py:324: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
Mar 30 15:47:49 satellite pulp: py.warnings:WARNING: (12885-67360)   object_id = collection.save(doc, **write_concern)
Mar 30 15:47:49 satellite pulp: py.warnings:WARNING: (12885-67360)
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360) Exception adding distributor to repo [STTCONNECT-Red_Hat_Enterprise_Linux_Server-Red_Hat_Satellite_Tools_6_2_for_RHEL_7_Server_RPMs_x86_64]; the repo will be deleted
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360) Traceback (most recent call last):
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 426, in create_repo
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)     dist_controller.add_distributor(repo_id, type_id, plugin_config, auto_publish, dist_id)
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/distributor.py", line 67, in add_distributor
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)     result = distributor_instance.validate_config(transfer_repo, call_config, config_conduit)
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 85, in validate_config
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)     return configuration.validate_config(repo, config, config_conduit)
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/configuration.py", line 135, in validate_config
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)     process_cert_based_auth(repo, config)
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/configuration.py", line 158, in process_cert_based_auth
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)     protected_repo_utils_instance.delete_protected_repo(relative_path)
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)   File "/usr/lib/python2.7/site-packages/pulp/repoauth/protected_repo_utils.py", line 63, in delete_protected_repo
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)     f.save()
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)   File "/usr/lib/python2.7/site-packages/pulp/repoauth/protected_repo_utils.py", line 143, in save
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360)     f = open(self.filename, 'w')
Mar 30 15:47:49 satellite pulp: pulp.server.controllers.repository:ERROR: (12885-67360) IOError: [Errno 13] Permission denied: '/etc/pki/pulp/content/pulp-protected-repos'
Mar 30 15:47:49 satellite pulp: py.warnings:WARNING: (12885-67360) /usr/lib/python2.7/site-packages/mongoengine/queryset/base.py:420: DeprecationWarning: remove is deprecated. Use delete_one or delete_many instead.
Mar 30 15:47:49 satellite pulp: py.warnings:WARNING: (12885-67360)   result = queryset._collection.remove(queryset._query, **write_concern)
Mar 30 15:47:49 satellite pulp: py.warnings:WARNING: (12885-67360)
Mar 30 15:47:49 satellite pulp: pulp.server.webservices.middleware.exception:ERROR: Unhandled Exception
Mar 30 15:47:49 satellite pulp: pulp.server.webservices.middleware.exception:ERROR: (12885-67360) [Errno 13] Permission denied: '/etc/pki/pulp/content/pulp-protected-repos'
SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.