Candlepin or HTTPD log files are not rotated and filling up the file system in Red Hat Satellite 6.

Environment

• Red Hat Satellite 6

Issue

• Parent filesystem /var/log is filled up with candlepin.log file.

    [root@test ]# ls -lrth /var/log/candlepin/*
  
     total 19316752
    -rw-r--r--. 1 tomcat tomcat        4081 Apr 11 13:51 audit.log
    -rw-r--r--. 1 tomcat tomcat       60916 Apr 11 13:38 audit.log.1.gz
    -rw-r--r--. 1 tomcat tomcat         203 Apr  8 16:37 audit.log.2.gz
    -rw-r--r--. 1 tomcat tomcat         412 Apr  8 16:34 audit.log.3.gz
    -rw-r--r--. 1 tomcat tomcat         279 Apr  8 16:21 audit.log.4.gz
    -rw-r--r--. 1 tomcat tomcat          20 Apr  8 16:13 audit.log.5.gz
    -rw-r--r--. 1 tomcat tomcat 10832789413 Apr 11 13:51 candlepin.log    << Check the size here.
    -rw-r--r--. 1 tomcat tomcat  8946950489 Apr 11 13:39 error.log
  

• Candlepin log files are not rotated as it keeps writing to the same file.

Resolution

• The file permission on /etc/cron.daily/logrotate fiile should be 0755 (executable).

  # chmod 0755 /etc/cron.daily/logrotate
  # ls -ltrh /etc/cron.daily/logrotate   
  -rwxr-xr-x. 1 root root 219 Jun 15  2018 /etc/cron.daily/logrotate
  

• Run the logrotate command manually and verify the /var/log filesystem size.

  # logrotate -v -f /etc/logrotate.d/candlepin 
  

• Remove the delaycompress option from /etc/logrotate.d/httpd file on Satellite.

• Uncomment the line compress from /etc/logrotate.conf

For more KB articles/solutions related to Red Hat Satellite 6.x Candlepin Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Candlepin Issues

Root Cause

• File permission of logrotate file was inappropriately set.

  # ls -ltrh /etc/cron.daily/logrotate 
  -rw-r-xr-x. 1 root root 219 Jun 15  2018 /etc/cron.daily/logrotate
  

• The delaycompress option is set on the configuration. This postpone compression of the previous log file to the next rotation cycle. This only has effect when used in combination with compress. It can be used when some program cannot be told to close its logfile and thus might continue writing to the previous log file for some time.

  $ cat etc/logrotate.d/httpd 
  /var/log/httpd/*log {
      missingok
      notifempty
      sharedscripts
      delaycompress             <============== HERE
      postrotate
          /bin/systemctl reload httpd.service > /dev/null 2>/dev/null || true
      endscript
  }
  

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.