Running leapp upgrade on Content Hosts registered with Red Hat Satellite 6 fails with Message: Unable to use yum successfully

Solution Verified - Updated 13 Jun 2024

Environment

Red Hat Satellite 6
Red Hat Enterprise Linux 7
Red Hat Enterprise Linux 8

Issue

Upgrading RHEL-7.9 Content Hosts registered with Red Hat Satellite 6 to RHEL-8.2 using leapp upgrade fails with the below error.

  ============================================================
                             ERRORS                                                    
  ============================================================

  2021-02-03 15:50:55.193514 [ERROR] Actor: target_userspace_creator
  Message: Unable to use yum successfully
  Summary:
      Details: Command ['systemd-nspawn', '--register=no', '--quiet', '-D', '/var/lib/leapp/scratch/mounts/root_/system_overlay', '--bind=/etc/hosts:/etc/hosts', '--                setenv=LEAPP_NO_RHSM=0', '--setenv=LEAPP_EXPERIMENTAL=0', '--setenv=LEAPP_COMMON_TOOLS=:/etc/leapp/repos.d/system_upgrade/el7toel8/tools', '--setenv=LEAPP_COMMON_FILES=:/etc/leapp/repos.d/system_upgrade/el7toel8/files', '--setenv=LEAPP_UNSUPPORTED=0', '--setenv=LEAPP_EXECUTION_ID=4e1b074b-5cba-467c-aaa6-6f45ac9b2939', '--setenv=LEAPP_HOSTNAME=client1.example.com', 'yum', 'clean', 'all'] failed with exit code 1.
      Stderr: Host and machine ids are equal (189dd10294c44c12a412d75395b32192): refusing to link journals
              There are no enabled repos.
               Run "yum repolist all" to see the repos you have.
               To enable Red Hat Subscription Management repositories:
           subscription-manager repos --enable <repo>
               To enable custom repositories:
                   yum-config-manager --enable <repo>

  ============================================================
                         END OF ERRORS                        
  ============================================================

Resolution

IMPORTANT NOTE
In case these resolution steps suggested not works, please consider also this alternative kcs RHEL7: leapp preupgrade fails with message Unable to use yum successfully

Refer to the RHEL 7 to RHEL 8 LEAP Upgrade Failing When Using Satellite and make sure that all the checkpoints are validated.
If you have configured your Red Hat Satellite and the RHEL 7 content host environment as per the above guide and still facing the mentioned error then follow the steps mentioned in the Diagnostic Steps section.

If the execution of the steps mentioned in the Diagnostic Steps section shows that you have incorrect entitlement keys and certs then just remove the incorrect keys and certs using the below command:

  # ls -l /etc/pki/entitlement/
  total 16
  -rw-r--r--. 1 root root  3243 Feb  3 14:15 5139900205724790719-key.pem
  -rw-r--r--. 1 root root  2883 Feb  3 14:15 5139900205724790719.pem
  -rw-r--r--. 1 root root  3243 Feb  3 14:15 5312053190752041761-key.pem
  -rw-r--r--. 1 root root  2801 Feb  3 14:15 5312053190752041761.pem
  -rw-r--r--. 1 root root  3243 Feb  3 14:15 8112770996627031619-key.pem
  -rw-r--r--. 1 root root  6296 Feb  3 14:15 8112770996627031619.pem
  -rw-r--r--. 1 root root  3243 Feb 18 11:06 1028450780616877016-key.pem
  -rw-r--r--. 1 root root  3255 Feb 18 11:06 1028450780616877016.pem

From the results of the Diagnostic Steps, the correct/valid key and cert file is /etc/pki/entitlement/1028450780616877016-key.pem and /etc/pki/entitlement/1028450780616877016.pem.
So we will proceed to remove the remaining certs and key files as below:

  # rm -f /etc/pki/entitlement/5139900205724790719-key.pem
  # rm -f /etc/pki/entitlement/5139900205724790719.pem
  # rm -f /etc/pki/entitlement/5312053190752041761-key.pem
  # rm -f /etc/pki/entitlement/5312053190752041761.pem
  # rm -f /etc/pki/entitlement/8112770996627031619-key.pem
  # rm -f /etc/pki/entitlement/8112770996627031619.pem

Now proceed to initiate the leapp preupgrade command again and you should not be able to see the same error.

For more KB articles/solutions related to Red Hat Satellite 6.x LEAPP Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x LEAPP Issues

Root Cause

There were multiple certs and keys in /etc/pki/entitlement directory on the affected RHEL 7 server and some of them were not the correct certs and keys.
As the affected RHEL 7 server was still using the incorrect certs and keys, it was denied access to the required RHEL repos for purpose of leapp uprgade.

Diagnostic Steps

List all the available entitlement certs and keys on the affected RHEL 7 server:

  # ls -l /etc/pki/entitlement/
  total 16
  -rw-r--r--. 1 root root  3243 Feb  3 14:15 5139900205724790719-key.pem
  -rw-r--r--. 1 root root  2883 Feb  3 14:15 5139900205724790719.pem
  -rw-r--r--. 1 root root  3243 Feb  3 14:15 5312053190752041761-key.pem
  -rw-r--r--. 1 root root  2801 Feb  3 14:15 5312053190752041761.pem
  -rw-r--r--. 1 root root  3243 Feb  3 14:15 8112770996627031619-key.pem
  -rw-r--r--. 1 root root  6296 Feb  3 14:15 8112770996627031619.pem
  -rw-r--r--. 1 root root  3243 Feb 18 11:06 1028450780616877016-key.pem
  -rw-r--r--. 1 root root  3255 Feb 18 11:06 1028450780616877016.pem

Now, execute the below curl command to validate if all the entitlement certs are granting access to RHEL 8 repos on RedHat Satellite or not:

  # curl -v --key /etc/pki/entitlement/8112770996627031619-key.pem --cert /etc/pki/entitlement/8112770996627031619.pem --cacert /etc/rhsm/ca/katello-server-ca.pem https://sat1.example.com/pulp/repos/Default_Organization/Library/CV1/content/dist/rhel8/8.6/x86_64/baseos/os/repodata/repomd.xml

Execute the above command for all the available keys and certs.

If the above curl command responds with a HTTP 200 OK response then the entitlement cert and key are correct and are allowing the affected RHEL 7 server to access the RHEL 8 content.

  # curl -v --key /etc/pki/entitlement/1028450780616877016-key.pem --cert /etc/pki/entitlement/1028450780616877016.pem --cacert /etc/rhsm/ca/katello-server-ca.pem https://sat1.example.com/pulp/repos/Default_Organization/Library/CV1/content/dist/rhel8/8.6/x86_64/baseos/os/repodata/repomd.xml
  * About to connect() to sat1.example.com port 443 (#0)
  *   Trying 10.74.251.224...
  * Connected to sat1.example.com (10.74.251.224) port 443 (#0)
  * Initializing NSS with certpath: sql:/etc/pki/nssdb
  *   CAfile: /etc/rhsm/ca/katello-server-ca.pem
    CApath: none
  <------Truncated Output------>
  > GET /pulp/repos/Default_Organization/Library/CV1/content/dist/rhel8/8.6/x86_64/baseos/os/repodata/repomd.xml HTTP/1.1
  > User-Agent: curl/7.29.0
  > Host: sat1.example.com
  > Accept: */*
  > 
  <------Truncated Output------>
  < HTTP/1.1 200 OK
  < Date: Tue, 09 Feb 2021 16:00:55 GMT
  < Server: Apache
  < ETag: "d41d8cd98f00b204e9800998ecf8427e"
  < Last-Modified: Tue, 02 Feb 2021 18:58:49 GMT
  < Content-Length: 2413
  < Content-Type: text/xml
  <------Truncated Output------>

If the curl command responds with a HTTP 403 Forbidden response then the entitlement cert and key are incorrect and are not allowing the affected RHEL 7 server to access the RHEL 8 content.

  # curl -v --key /etc/pki/entitlement/8112770996627031619-key.pem --cert /etc/pki/entitlement/8112770996627031619.pem --cacert /etc/rhsm/ca/katello-server-ca.pem https://sat1.example.com/pulp/repos/Default_Organization/Library/CV1/content/dist/rhel8/8.6/x86_64/baseos/os/repodata/repomd.xml
  * About to connect() to sat1.example.com port 443 (#0)
  *   Trying 10.74.251.224...
  * Connected to sat1.example.com (10.74.251.224) port 443 (#0)
  * Initializing NSS with certpath: sql:/etc/pki/nssdb
  *   CAfile: /etc/rhsm/ca/katello-server-ca.pem
    CApath: none
  <------Truncated Output------>
  > GET /pulp/repos/Default_Organization/Library/CV1/content/dist/rhel8/8.6/x86_64/baseos/os/repodata/repomd.xml HTTP/1.1
  > User-Agent: curl/7.29.0
  > Host: sat1.example.com
  > Accept: */*
  > 
  <------Truncated Output------>
  < HTTP/1.1 403 Forbidden
  < Date: Tue, 09 Feb 2021 16:02:20 GMT
  < Server: Apache
  < Content-Length: 0
  < ETag: "d41d8cd98f00b204e9800998ecf8427e"
  < Content-Type: text/html; charset=utf-8
  <------Truncated Output------>

Note the details of the entitlement certs and keys with their respective responses and refer to the Resolution section for further steps.

SBR

Product(s)

Components

Category

Troubleshoot

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.