Satellite Capsule pulp_streamer.service entered failed state

Solution Verified - Updated 13 Jun 2024

Environment

Red Hat Satellite 6.8+

Issue

After a new installation of Satellite Capsule 6.8.2 on a RHEL 7.9 server, pulp_streamer.service fails to start.

Output of satellite-maintain service status command reports failed state for pulp_streamer.service.

  # satellite-maintain service status
  / displaying pulp_streamer                                                      
  * pulp_streamer.service - The Pulp lazy content loading streamer
     Loaded: loaded (/usr/lib/systemd/system/pulp_streamer.service; enabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Tue 2021-01-12 22:05:46 +0530; 16min ago
    Process: 10867 ExecStart=/usr/bin/pulp_streamer --nodaemon --syslog --prefix=pulp_streamer --pidfile= --python /usr/share/pulp/wsgi/streamer.tac (code=exited, status=1/FAILURE)
   Main PID: 10867 (code=exited, status=1/FAILURE)

  Jan 12 22:05:46 rh-capsule pulp_streamer[11977]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3007, in <module>
  Jan 12 22:05:46 rh-capsule pulp_streamer[11977]: working_set.require(__requires__)
  Jan 12 22:05:46 rh-capsule pulp_streamer[11977]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
  Jan 12 22:05:46 rh-capsule pulp_streamer[11977]: needed = self.resolve(parse_requirements(requirements))
  Jan 12 22:05:46 rh-capsule pulp_streamer[11977]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
  Jan 12 22:05:46 rh-capsule pulp_streamer[11977]: raise DistributionNotFound(req)
  Jan 12 22:05:46 rh-capsule pulp_streamer[11977]: pkg_resources.DistributionNotFound: twisted
  Jan 12 22:05:46 rh-capsule systemd[1]: pulp_streamer.service: main process exited, code=exited, status=1/FAILURE
  Jan 12 22:05:46 rh-capsule systemd[1]: Unit pulp_streamer.service entered failed state.
  Jan 12 22:05:46 rh-capsule systemd[1]: pulp_streamer.service failed.
  [...]
  
  / All services displayed                                              [FAIL]    
  Some services are not running (pulp_streamer)
  --------------------------------------------------------------------------------
  Scenario [Status Services] failed.
  
  The following steps ended up in failing state:
  
    [service-status]

The service may also fail with:

Nov 04 16:28:25 capsule.example.com pulp_streamer[26491]: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 653, in _build_master
Nov 04 16:28:25 capsule.example.com pulp_streamer[26491]: return cls._build_from_requirements(__requires__)
Nov 04 16:28:25 capsule.example.com pulp_streamer[26491]: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 666, in _build_from_requirements
Nov 04 16:28:25 capsule.example.com pulp_streamer[26491]: dists = ws.resolve(reqs, Environment())
Nov 04 16:28:25 capsule.example.com pulp_streamer[26491]: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 839, in resolve
Nov 04 16:28:25 capsule.example.com pulp_streamer[26491]: raise DistributionNotFound(req, requirers)
Nov 04 16:28:25 capsule.example.com pulp_streamer[26491]: pkg_resources.DistributionNotFound: The 'urllib3<1.24,>=1.21.1' distribution was not found and is requir... requests
Nov 04 16:28:25 capsule.example.com systemd[1]: pulp_streamer.service: main process exited, code=exited, status=1/FAILURE
Nov 04 16:28:25 capsule.example.com systemd[1]: Unit pulp_streamer.service entered failed state.
Nov 04 16:28:25 capsule.example.com systemd[1]: pulp_streamer.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

Resolution

Check for the pkg_resources.DistributionNotFound in the error logs to find more about the python package with issue.
In this case issues were with python2-twisted and python2-urllib3 which were installed from upstream repositories.
Take a VM snapshot or proper backup of the server and remove the concerned upstream rpms with:
```
  # rpm -e --nodeps python2-urllib3
  # rpm -e --nodeps python2-twisted
```
Install python2-twisted and 'python-urllib3' rpms from the rhel-7-server-satellite-capsule-6.8-rpms repository.
```
  # yum install python2-twisted
  # yum install python-urllib3
```
Reboot Capsule server.

Check status of service. It should be active.

  # satellite-maintain service status --only pulp_streamer
  Running Status Services
  ================================================================================
  Get status of applicable services: 
  
  Displaying the following service(s):
  pulp_streamer
  / displaying pulp_streamer                                                      
  * pulp_streamer.service - The Pulp lazy content loading streamer
     Loaded: loaded (/usr/lib/systemd/system/pulp_streamer.service; enabled; vendor preset: disabled)
     Active: active (running) since Tue 2021-01-12 22:10:40 +0530; 1min ago

For more KB articles/solutions related to Red Hat Satellite 6.x Pulp 2.0 Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Pulp 2.0-related Issues

Diagnostic Steps

/var/log/messages contains a traceback for the pulp_streamer service entering failed state.

  Jan 12 22:05:39 rh-capsule pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
  Jan 12 22:05:46 rh-capsule systemd: Started The Pulp lazy content loading streamer.
  Jan 12 22:05:46 rh-capsule pulp_streamer: Traceback (most recent call last):
  Jan 12 22:05:46 rh-capsule pulp_streamer: File "/usr/bin/pulp_streamer", line 5, in <module>
  Jan 12 22:05:46 rh-capsule pulp_streamer: from pkg_resources import load_entry_point
  Jan 12 22:05:46 rh-capsule pulp_streamer: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3007, in <module>
  Jan 12 22:05:46 rh-capsule pulp_streamer: working_set.require(__requires__)
  Jan 12 22:05:46 rh-capsule pulp_streamer: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
  Jan 12 22:05:46 rh-capsule pulp_streamer: needed = self.resolve(parse_requirements(requirements))
  Jan 12 22:05:46 rh-capsule pulp_streamer: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
  Jan 12 22:05:46 rh-capsule pulp_streamer: raise DistributionNotFound(req)
  Jan 12 22:05:46 rh-capsule pulp_streamer: pkg_resources.DistributionNotFound: twisted
  Jan 12 22:05:46 rh-capsule systemd: pulp_streamer.service: main process exited, code=exited, status=1/FAILURE
  Jan 12 22:05:46 rh-capsule systemd: Unit pulp_streamer.service entered failed state.
  Jan 12 22:05:46 rh-capsule systemd: pulp_streamer.service failed.

SBR

SysMgmt

Product(s)

Red Hat Satellite

Components

Category

Troubleshoot

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.