ERROR:nextval: reached maximum value of sequence "container_blobmanifest_id_seq" while synchronizing the container images in Red Hat Satellite 6

Solution Verified - Updated

Environment

  • Red Hat Satellite
    • 6.15
    • 6.16
    • 6.17

Issue

  • Synchronizing the large number of container images fails with the following error:

        2025-09-29T05:45:04 [I|app|82d35371] ForemanWebhooks::EventSubscriber: actions.katello.repository.sync_failed.event.foreman event received
    2025-09-29T05:45:09 [E|bac|82d35371] nextval: reached maximum value of sequence "container_blobmanifest_id_seq" (2147483647) (Katello::Errors::Pulp3Error)
    

Resolution

  • This issue has been reported to the Red Hat Satellite Engineering team via This content is not included.SAT-38887 bug and is being actively investigated.

  • To fix the issue manually on the affected Red Hat Satellite server, proceed with the following steps:

    1. Ensure that a backup or VM snapshot exists for the affected Red Hat Satellite server, before proceeding further.

    2. Clear any paused tasks that might be present:

      # foreman-rake foreman_tasks:cleanup TASK_SEARCH='label ~ *' STATES='paused' VERBOSE=true
      
    3. Check the max_value and last_value of container_blobmanifest_id_seq sequence in pulpcore database. They are expected to be the same at this point:

      # echo "select * from pg_sequences where sequencename='container_blobmanifest_id_seq';" | su - postgres -c "psql -x pulpcore"
      
    4. Fix the issue:

              # echo "ALTER SEQUENCE container_blobmanifest_id_seq AS bigint;" | su - postgres -c "psql pulpcore"
      
              # echo "ALTER TABLE container_blobmanifest ALTER COLUMN id TYPE BIGINT;" | su - postgres -c "psql pulpcore"
      
    5. Verify that max_value is now much higher than last_value for container_blobmanifest_id_seq sequence and its data_type is set as bigint:

      # echo "select * from pg_sequences where sequencename='container_blobmanifest_id_seq';" | su - postgres -c "psql -x pulpcore"
      
    6. Restart the services.

      # satellite-maintain service restart
      # sleep 20 & hammer ping
      
  • Once done with all the steps try to synchronize the container images and observed the progress.

For more KB articles/solutions related to Red Hat Satellite 6.x PostgreSQL-related Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x PostgreSQL-related Issues

Root Cause

  • The container_blobmanifest_id_seq itself also has the data type set to integer due to which the value of the max_value field is limited to 2147483647 and the last_value field has reached that specific limit.

Diagnostic Steps

  • The following logs observed in the /var/log/foreman/production.log:

     2025-09-29T05:43:43 [I|bac|82d35371] Task {label: Actions::Katello::Repository::Sync, id: 8e0c75ce-d5e2-4556-9420-6df613a7aec5, execution_plan_id: 8da9dbc1-8af2-4fbc-aafb-994472f87532} state changed: stopped  result: warning
     2025-09-29T05:43:43 [I|app|82d35371] ForemanWebhooks::EventSubscriber: actions.katello.repository.sync_failed.event.foreman event received
     2025-09-29T05:43:46 [E|bac|82d35371] nextval: reached maximum value of sequence "container_blobmanifest_id_seq" (2147483647) (Katello::Errors::Pulp3Error)
     82d35371 | /usr/share/gems/gems/katello-4.11.0.16/app/lib/actions/pulp3/abstract_async_task.rb:108:in `block in check_for_errors'
     82d35371 | /usr/share/gems/gems/katello-4.11.0.16/app/lib/actions/pulp3/abstract_async_task.rb:106:in `each'
     82d35371 | /usr/share/gems/gems/katello-4.11.0.16/app/lib/actions/pulp3/abstract_async_task.rb:106:in `check_for_errors'
     82d35371 | /usr/share/gems/gems/katello-4.11.0.16/app/lib/actions/pulp3/abstract_async_task.rb:162:in `poll_external_task'
    
  • Inspecting the sequence and table in concern, the following can be found in the pulpcore database.

    # echo "select * from pg_sequences where sequencename='container_blobmanifest_id_seq';" | su - postgres -c "psql -x pulpcore"
    -[ RECORD 1 ]-+------------------------------
    schemaname    | public
    sequencename  | container_blobmanifest_id_seq
    sequenceowner | pulp
    data_type     | integer
    start_value   | 1
    min_value     | 1
    max_value     | 2147483647   ------------------> Not large enough value
    increment_by  | 1
    cycle         | f
    cache_size    | 1
    last_value    | 2147483647   ------------------> same as max_value which means it had reached its limit
    
SBR
Product(s)
Components

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.