The Pulp 2 to Pulp 3 migration fails with error "NoMethodError: undefined method `link?' for nil:NilClass" in Red Hat Satellite 6.9

Solution Verified - Updated

Environment

  • Red Hat Satellite 6.9

Issue

  • The attempt to perform Pulp 2 to Pulp 3 migration via the satellite-maintain content prepare command fails with the following error:

       Prepare content for Pulp 3:
   Checking for valid Katello configuration.
   Starting task.

   2022-04-01 08:11:23 -0400: Content migration starting. These steps may take a while to complete. Refer to `foreman-maintain content migration-stats` for an estimate.Migration failed, You will want to investigate: https://satellite.example.com/foreman_tasks/tasks/fa9130c3-5660-46cb-82ea-a03389e27968
   rake aborted!
   ForemanTasks::TaskError: Task fa9130c3-5660-46cb-82ea-a03389e27968: NoMethodError: undefined method `link?' for nil:NilClass
   /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/lib/katello/tasks/pulp3_migration.rake:43:in `block (2 levels) in <top (required)>'
   /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
   Tasks: TOP => katello:pulp3_migration
   (See full trace by running task with --trace)
                                                                         [FAIL]
   Failed executing preserve_output=true foreman-rake katello:pulp3_migration, exit status 1

Resolution

This issue has been reported to the Red Hat Engineering team via This content is not included.Bugzilla 2073481 and is currently under investigation.

WORKAROUND:

  • Copy the following echo command in the Satellite server and execute it in one go. 

    # echo -e '
Katello::ContentView.non_default.each do |cv|
  Katello::Repository.in_default_view.each do |library_instance|
    cv.versions.with_library_repo(library_instance).each do |version|
      if version.archived_repos.find_by(:library_instance_id => library_instance.id).nil?
        found = version.repositories.find_by(:library_instance_id => library_instance.id)
        puts "#{cv.name} #{version.version}: #{library_instance.name} : env repo: #{found.id} in #{found.environment&.name}"    
      end
    end
  end
end ; nil
' | foreman-rake console

    • It should produce an output similar to the following if there are any issues present with any repositories inside any content-views.

      RHEL6 24.0: Red Hat Enterprise Linux 6 Server RPMs x86_64 6.10 : env repo: 48 in Library

    • From this example :

      • The concerned repository is Red Hat Enterprise Linux 6 Server RPMs x86_64 6.10
      • The concerned content-view is RHEL6 and the affected version of the same CV is 24.0 which is promoted to the Library environment.

  • Considering the example mentioned above, the next action plan to solve the issue would be as mentioned below:

    • Ensure that the affected repository is properly synchronized i.e. Red Hat Enterprise Linux 6 Server RPMs x86_64 6.10 in this example.
    • Republish a new version of the affected content-view i.e. RHEL6 in this example and promote the affected environment to that newly published version.
    • Delete the affected version of the affected content-view i.e. version 24.0 of RHEL6 CV.
    • Re-execute the echo script from the very first step on the Satellite server. It should not show any outputs this time.
    • If everything looks good, re-attempt the content migration process and observe the progress.

 

Root Cause

There is a CV version present that has a repo in it but the repo itself in that version does not have an archived copy. So deletion of the affected version of the CV gets rid of the repo from that version itself and allows the content migration to happen properly.

Diagnostic Steps

  • The /var/log/foreman-maintain/foreman-maintain.log file captures the exact error message as mentioned in the Issue section in this solution article.

  • From /var/log/foreman/production.log following error traceback was observed when the content migration failed :

    2022-03-30T14:00:29 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: planned 
2022-03-30T14:00:30 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: running 
2022-03-30T14:00:30 [E|bac|] undefined method `link?' for nil:NilClass (NoMethodError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:168:in `block in content_view_migration'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:166:in `each'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:166:in `content_view_migration'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:123:in `block in content_view_migrations_for'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:117:in `each'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:117:in `content_view_migrations_for'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:43:in `block in repository_migrations'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/delegation.rb:87:in `each'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:40:in `repository_migrations'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:24:in `block in generate_plugins'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:21:in `map'	
..
..
2022-03-30T14:00:30 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: stopped  result: warning
2022-03-30T14:00:30 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: stopped  result: warning
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.