How to cancel all waiting Pulp-3 tasks in Red Hat Satellite 6.10 or newer?

Solution Verified - Updated

Environment

  • Red Hat Satellite 6.10 or newer
  • Red Hat Capsule 6.10 or newer

Issue

  • What are the commands to clean all Pulp-3/pulpcore tasks on Red Hat Satellite/Capsule 6.10 or newer?

Resolution

  • Below command will work for Satellite only:

    for taskid in $(su - postgres -c "psql pulpcore -c \"COPY (SELECT pulp_id FROM core_task WHERE state = 'waiting' OR state = 'running') TO STDOUT;\""); do
        curl -H "content-type: application/json" --cacert /etc/pki/katello/certs/katello-server-ca.crt --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem -X PATCH -d "{\"state\": \"canceled\"}" https://$(hostname -f)/pulp/api/v3/tasks/${taskid}/
     echo
    done
    
  • Follow below steps for a Capsule:

    • Run below steps from Satellite, providing proper CAPSULE.FQDN in the first command:
       capsule=CAPSULE.FQDN
    
       certs="--cacert /etc/pki/katello/certs/katello-server-ca.crt --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem"
    
       for task_href in $(curl -s $certs "https://${capsule}/pulp/api/v3/tasks/?state__in=running,waiting&limit=10000" | json_reformat | grep pulp_href | grep /pulp/api/v3/tasks/ | cut -d\" -f4); do
           curl -s -H "content-type: application/json" -X PATCH $certs -d "{\"state\": \"canceled\"}" "https://${capsule}${task_href}"
       done
    

For more KB articles/solutions related to Red Hat Satellite 6.x Pulp 3.0 Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Pulp 3.0-related Issues

Diagnostic Steps

  • SQL query to check the stuck/waiting tasks in pulpcore:
su - postgres -c "psql pulpcore -c \"select name, started_at, pulp_created, pulp_last_updated, state from core_task where state in ('waiting', 'canceling', 'running');\""
  • Sample output shows multiple pulpcore thats in waiting:
                     name                     |          started_at           |         pulp_created          |       pulp_last_updated       |  state  
----------------------------------------------+-------------------------------+-------------------------------+-------------------------------+---------
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:53.371028+10 | 2024-06-16 04:36:53.37105+10  | waiting
 pulp_rpm.app.tasks.copy.copy_content         | 2024-07-02 16:11:02.927733+10 | 2024-06-16 04:36:52.633232+10 | 2024-06-16 04:36:52.633256+10 | running
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:54.995468+10 | 2024-06-16 04:36:54.99549+10  | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:58.522238+10 | 2024-06-16 04:36:58.522261+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:59.315672+10 | 2024-06-16 04:36:59.315696+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:52.812748+10 | 2024-06-16 04:36:52.812772+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:55.326793+10 | 2024-06-16 04:36:55.326817+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:57.616239+10 | 2024-06-16 04:36:57.61626+10  | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:57.934772+10 | 2024-06-16 04:36:57.934793+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:58.813649+10 | 2024-06-16 04:36:58.813673+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:36:59.559019+10 | 2024-06-16 04:36:59.559042+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:00.054326+10 | 2024-06-16 04:37:00.054349+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:05.030709+10 | 2024-06-16 04:37:05.030731+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:09.814259+10 | 2024-06-16 04:37:09.814281+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:12.918062+10 | 2024-06-16 04:37:12.918084+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:02.548955+10 | 2024-06-16 04:37:02.548978+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:07.393433+10 | 2024-06-16 04:37:07.393454+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:10.320747+10 | 2024-06-16 04:37:10.320771+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:14.89711+10  | 2024-06-16 04:37:14.897132+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:24.416366+10 | 2024-06-16 04:37:24.416387+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:19.538958+10 | 2024-06-16 04:37:19.538979+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:17.205375+10 | 2024-06-16 04:37:17.205399+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:21.93497+10  | 2024-06-16 04:37:21.934992+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:30.141219+10 | 2024-06-16 04:37:30.14124+10  | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:29.209738+10 | 2024-06-16 04:37:29.209758+10 | waiting
 pulp_rpm.app.tasks.copy.copy_content         |                               | 2024-06-16 04:37:26.774149+10 | 2024-06-16 04:37:26.774171+10 | waiting
 pulpcore.app.tasks.repository.delete_version |                               | 2024-06-16 22:01:22.929219+10 | 2024-06-16 22:01:22.929248+10 | waiting
 pulpcore.app.tasks.repository.delete_version |                               | 2024-06-16 22:01:22.8111+10   | 2024-06-16 22:01:22.811127+10 | waiting
 pulpcore.app.tasks.repository.add_and_remove |                               | 2024-06-17 13:01:25.590175+10 | 2024-06-17 13:01:25.5902+10   | waiting
 pulpcore.app.tasks.repository.add_and_remove |                               | 2024-06-17 13:01:25.757199+10 | 2024-06-17 13:01:25.757225+10 | waiting
SBR
Product(s)
Components
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.