Foreman tasks are running for several hours on Red Hat Satellite 6.x, without making any progress or reporting any errors

Solution Verified - Updated

Environment

  • Red Hat Satellite 6.15

Issue

  • Currently we have 6 tasks that are running for over 9 hours and are making no progress. They also don't display any error messages, and they are causing other scheduled tasks not to run.

Resolution

  1. Follow the steps in the Diagnostic Steps section of this KB solution.

  2. If you find any foreman tasks stuck in stopped:pending state, clean them up by running the following command on the Satellite server:

# foreman-rake foreman_tasks:cleanup TASK_SEARCH='result == pending' STATES='stopped' VERBOSE=true
  1. Restart Satellite services:
# satellite-maintain service restart
  1. Verify that Satellite services are up and running after the services restart:
# satellite-maintain service status -b

For more KB articles/solutions related to Red Hat Satellite 6.x Satellite Tasking System Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Satellite Tasking System Issues

Root Cause

Several foreman tasks were stuck in stopped:pending state, causing some foreman tasks to get stuck in running:pending state for several hours without displaying any error messages or warnings, and causing other scheduled tasks not to run at all.

Diagnostic Steps

From the sosreport of the Satellite server, check if there are any foreman tasks stuck in stopped:pending state:

$ grep ',stopped,pending,' sosreport-satellite-2024-11-19-pzjxgjg/sos_commands/foreman/foreman_tasks_tasks | wc -l
52
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.