Any remote execution job on RHEL 7.x fails with "Error initializing command: RuntimeError - Could not establish connection to remote host using any available authentication method, tried publickey" in Red Hat Satellite 6

Environment

• Red Hat Satellite 6
• Red Hat Capsule 6
• Red Hat Enterprise Linux 7

Issue

• On RHEL 7.x host registered to Red Hat Satellite/Capsule 6, execution of any remote job fails and displays the following error:

   Error initializing command: RuntimeError - Could not establish connection to remote host using any available   authentication method, tried publickey
   Authentication method 'publickey' failed with:
   root@client.example.com Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
   Exit status: EXCEPTION
   StandardError: Job execution failed
  

Resolution

• Verify from the Diagnostic Steps, ensure that both OpenSSL and OpenSSH packages are updated to compatible ELS versions on the RHEL 7 host, which are openssl-1.0.2k-26.el7_9.x86_64 and openssh-7.4p1-23.el7_9.x86_64 respectively.

  • NOTE: From Red Hat Satellite 6.16 onwards, Red Hat Enterprise Linux 7 with the ELS Add-On is the supported operating system as a client. Upgrade the operating system to ELS if not done, following the steps articulated in article 7081103.

• Reach out to the This content is not included.Red Hat Technical Support, if the issue still persists and further investigation is required.

For more KB articles/solutions related to Red Hat Satellite 6.x Remote Execution Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Remote Execution Issues

Root Cause

• An outdated/older OpenSSH package on the RHEL 7 host is incompatible with the updated OpenSSL libraries, preventing Red Hat Satellite from establishing an SSH connection using the foreman-proxy user's public ssh-key.

Diagnostic Steps

• In this scenario, the Openssl package is updated to the latest version on the affected RHEL 7.3 host; however, the Openssh package is one version older.

    $ rpm -qa | grep -e openssl -e openssh
   openssl-1.0.2k-26.el7_9.x86_64                             Mon Dec 11 12:12:21 2023
   openssl-libs-1.0.2k-26.el7_9.x86_64                        Mon Dec 11 12:12:19 2023
   openssh-6.6.1p1-31.el7.x86_64                              Thu May  4 09:33:54 2017
   openssh-clients-6.6.1p1-31.el7.x86_64                      Thu May  4 09:34:29 2017
   openssh-server-6.6.1p1-31.el7.x86_64                       Thu May  4 09:34:26 2017
  

• The following error can be observed while trying to SSH from Red Hat Satellite 6 to the affected host via foreman-proxy user's ssh-key:

   [root@satellite.example.com ~]# ssh -o PubkeyAcceptedAlgorithms=+ssh-rsa \
   -o HostkeyAlgorithms=+ssh-rsa \
   -i ~foreman-proxy/.ssh/id_rsa_foreman_proxy \
   root@client.example.com 'sudo hostname'
   ...
   sign_and_send_pubkey: signing failed for RSA "/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy": error in libcrypto
  

  [root@satellite.example.com ~]# ssh -vvv -i ~foreman-proxy/.ssh/id_rsa_foreman_proxy  roo@client.example.com 'sudo hostname'
   debug1: Reading configuration data /etc/ssh/ssh_config
   ...
   debug1: Offering public key: /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy RSA SHA256:pXYZ explicit
   debug1: send_pubkey_test: no mutual signature algorithm
  

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.