Publishing CV with depsolving enabled on RHEL8 BaseOS repo and including all except new errata fails on package dependencies

Environment

Red Hat Satellite 6.10 or newer

Issue

Having a Content View with:

• Red Hat Enterprise Linux 8 BaseOS repository (optionally other repos)
• dependency solving enabled
• include filters to include:
  • all packages without errata
  • all module streams without an errata
  • all errata released before some date

Then publishing this Content View fails in Actions::Pulp3::Repository::MultiCopyContent dynflow step.

Resolution

• open the dynflow console of the CV publish task
• expand the failed dynflow step Actions::Pulp3::Repository::MultiCopyContent
• wait until all pulp tasks are completed (no task has state: waiting or running) - this might require manual refreshing the page
• only when all tasks arecompleted, it is safe to skip the "failed" dynflow step:
  • click to Skip link on the right from the failed Actions::Pulp3::Repository::MultiCopyContent dynflow step
  • click to Resume link on the top of the page
• wait until the publish task completes

For more KB articles/solutions related to Red Hat Satellite 6.x Content View Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Content View Issues

Root Cause

The dynflow step is failed because one "copy content following dependencies" hit unresolvable dependencies. All expected content was copied, just the warning put the underlying pulp task to cancelled state. dynflow just propagated that warning as an error further.

The unresolvable dependencies normally should not happen (when using filters properly). In this particular case:

• perl and npm packages have known dependency issues in several modules
• lmdb-libs .. ?
• elfutils-debuginfod-client-devel .. ?
• python2-backports-* .. ?

Diagnostic Steps

• Actions::Pulp3::Repository::MultiCopyContent step of the CV publish has a cancelled pulp task:

- pulp_href: "/pulp/api/v3/tasks/5552552d-97e8-433f-964f-4f68cca21fe5/"
  pulp_created: '2021-08-17T00:31:02.484+00:00'
  state: canceled
  name: pulp_rpm.app.tasks.copy.copy_content
  logging_cid: 21b7f226-d3f8-4f05-a164-131cc21a601c
  started_at: '2021-08-17T00:47:37.336+00:00'
  worker: "/pulp/api/v3/workers/c6eda7d0-4082-481e-b63b-3f6f688ef0dd/"
  child_tasks: []
  progress_reports: []
  created_resources: []
  reserved_resources_record:
  - "/pulp/api/v3/repositories/rpm/rpm/566db2d9-b7d1-43e1-8a32-7d3c9657fd84/"
  - "/pulp/api/v3/repositories/rpm/rpm/41a0f380-8a61-4806-a7a2-34072a620264/"
  - "/pulp/api/v3/repositories/rpm/rpm/98b8c06d-0fc2-49ac-ad70-31a54b6aafe7/"
  - "/pulp/api/v3/repositories/rpm/rpm/2fb4d5b8-7c1c-4870-8fea-4719b3c8df00/"

• grepping for the logging_cid, multiple broken dependency warnings raised:

# grep 21b7f226-d3f8-4f05-a164-131cc21a601c /var/log/messages | grep -v "GET /pulp/api/v3/"
..
Aug 17 03:07:52 pmoravec-sat610-beta-2 pulpcore-worker-4: pulp [21b7f226-d3f8-4f05-a164-131cc21a601c]: pulp_rpm.app.depsolving:WARNING: Encountered problems solving dependencies, copy may be incomplete: package perl-LWP-MediaTypes-0:6.02-15.module+el8.3.0+6498+31ea7642.noarch requires perl(:MODULE_COMPAT_5.24.4), but none of the providers can be installed
..
Aug 17 03:14:56 pmoravec-sat610-beta-2 pulpcore-worker-4: pulp [21b7f226-d3f8-4f05-a164-131cc21a601c]: pulp_rpm.app.depsolving:WARNING: Encountered problems solving dependencies, copy may be incomplete: cannot install both lmdb-libs-0:0.9.23-5.el8.x86_64 and lmdb-libs-0:0.9.24-1.el8.x86_64
..
Aug 17 03:29:04 pmoravec-sat610-beta-2 pulpcore-worker-4: pulp [21b7f226-d3f8-4f05-a164-131cc21a601c]: pulp_rpm.app.depsolving:WARNING: Encountered problems solving dependencies, copy may be incomplete: package elfutils-debuginfod-client-devel-0:0.180-1.el8.i686 requires elfutils-debuginfod-client(x86-32) = 0:0.180-1.el8, but none of the providers can be installed
..
Aug 17 08:17:57 pmoravec-sat610-beta-2 pulpcore-worker-3: pulp [21b7f226-d3f8-4f05-a164-131cc21a601c]: pulp_rpm.app.depsolving:WARNING: Encountered problems solving dependencies, copy may be incomplete: cannot install both npm-1:6.14.11-1.10.24.0.1.module+el8.3.0+10166+b07ac28e.x86_64 and npm-1:6.14.11-1.14.16.0.2.module+el8.3.0+10180+b92e1eb6.x86_64, package npm-1:6.14.11-1.12.21.0.1.module+el8.3.0+10191+34fb5a07.x86_64 requires nodejs = 1:12.21.0-1.module+el8.3.0+10191+34fb5a07, but none of the providers can be installed
..
Aug 17 08:18:52 pmoravec-sat610-beta-2 pulpcore-worker-3: pulp [21b7f226-d3f8-4f05-a164-131cc21a601c]: pulp_rpm.app.depsolving:WARNING: Encountered problems solving dependencies, copy may be incomplete: cannot install both python2-backports-ssl_match_hostname-0:3.5.0.1-11.module+el8.1.0+3111+de3f2d8e.noarch and python2-backports-ssl_match_hostname-0:3.5.0.1-11.module+el8.0.0+2961+596d0223.noarch, cannot install both python2-backports-ssl_match_hostname-0:3.5.0.1-11.module+el8.0.0.z+3358+99b46920.noarch and python2-backports-ssl_match_hostname-0:3.5.0.1-11.module+el8.0.0+4028+a686efca.noarch, cannot install both python2-backports-ssl_match_hostname-0:3.5.0.1-12.module+el8.4.0+9193+f3daf6ef.noarch and python2-backports-ssl_match_hostname-0:3.5.0.1-10.module+el8+2540+b19c9b35.noarch
..

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.