[Satellite 6] upgrade fails on`foreman-rake db:migrate failed`

Solution Verified - Updated

Environment

Red Hat Satellite 6.4
Red Hat Satellite 6.5

Issue

Running foreman-maintain upgrade run --target-version 6.4.z fails on error foreman-rake db:migrate failed!:

Execute db:migrate
== 20190205142619 AddContentConstraints: migrating ============================
-- change_column(:katello_product_contents, :content_id, :integer, {:null=>false})
   -> 0.0038s
-- change_column(:katello_product_contents, :product_id, :integer, {:null=>false})
   -> 0.0018s
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:
PG::ForeignKeyViolation: ERROR:  update or delete on table "katello_contents" violates foreign key constraint "katello_product_content_content_id_fk" on table "katello_product_contents" 
DETAIL:  Key (id)=(2403) is still referenced from table "katello_product_contents".
: DELETE FROM "katello_contents" WHERE "katello_contents"."id" = $1

Resolution

This is a bug tracked in Red Hat This content is not included.bugzilla. As a workaround, apply the patch following below steps:

  1. Make sure latest tfm-rubygem-katello package is installed on Satellite server.

For Satellite 6.4.3 :

# rpm -qf /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.58/db/migrate/20190205142619_add_content_constraints.rb

Expected result is tfm-rubygem-katello-3.7.0.58-1.el7sat.noarch

For Satellite 6.5

# rpm -qf /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.48/db/migrate/20190205142619_add_content_constraints.rb

Expected result is tfm-rubygem-katello-3.10.0.48.el7sat.noarch

  1. Download the patch and apply it:

For Satellite 6.4.3

# wget https://patch-diff.githubusercontent.com/raw/Katello/katello/pull/8096.patch
# cd /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.58/db/migrate/
# cat /root/8096.patch | patch -p3

For Satellite 6.5

# wget https://patch-diff.githubusercontent.com/raw/Katello/katello/pull/8096.patch
# cd /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.48/db/migrate/
# cat /root/8096.patch | patch -p3
  1. Verify if the issue is resolved:
# foreman-rake db:migrate --trace

For more KB articles/solutions related to Red Hat Satellite 6.x Installation/Upgrade/Update Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Installation/Upgrade/Update Issues.

Root Cause

There are some bad data in the db which are not handled properly during the upgrade.

Diagnostic Steps

foreman-rake db:migrate fails with below error:

# foreman-rake db:migrate
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.56/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::ERRATA_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.56/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: previous definition of ERRATA_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.56/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::TRACE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.56/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: previous definition of TRACE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.56/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::SUBSCRIPTION_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.56/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: previous definition of SUBSCRIPTION_STATUS_MAP was here
== 20190205142619 AddContentConstraints: migrating ============================
-- change_column(:katello_product_contents, :content_id, :integer, {:null=>false})
   -> 0.0035s
-- change_column(:katello_product_contents, :product_id, :integer, {:null=>false})
   -> 0.0017s
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

PG::ForeignKeyViolation: ERROR:  update or delete on table "katello_contents" violates foreign key constraint "katello_product_content_content_id_fk" on table "katello_product_contents"
DETAIL:  Key (id)=(1721) is still referenced from table "katello_product_contents".
: DELETE FROM "katello_contents" WHERE "katello_contents"."id" = $1
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activerecord-5.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:616:in `async_exec'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activerecord-5.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:616:in `block (2 levels) in exec_no_cache'
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.