General SSSD Debugging Procedures

Environment

• Red Hat Enterprise Linux 5.6 and above
• Red Hat Enterprise Linux 6
• Red Hat Enterprise Linux 7
• Red Hat Enterprise Linux 8
• Red Hat Enterprise Linux 9
• sssd

Issue

• How do enable debug log level on sssd ?
• When SSSD fails to function as expected, it can be helpful to enable debugging in order to identify the cause of the issue
• How to debug issue if sssd fails to start
• How to enable debugging if sssd fails to authenticate users
• Unable to start sssd service, how do I debug this issue ?
• sssd service stops randomly

Resolution

• When sssd daemon fails to start, debug output of following is very useful to get detailed information about the error:

  # sssd -i -d9
  

• Each domain defined in /etc/sssd/sssd.conf may contain its own debug level. The parameter used to set the verbosity of the log level is debug_level. Please see the level (0-9) explanations mentioned below:

    Level 	Description
    0 	Fatal failures. Anything that would prevent SSSD from starting up or causes it to cease running.
    1 	Critical failures. An error that doesn't kill the SSSD, but one that indicates that at least one major feature is not going to work properly.
    2 	Serious failures. An error announcing that a particular request or operation has failed.
    3 	Minor failures. These are the errors that would percolate down to cause the operation failure of 2.
    4 	Configuration settings.
    5 	Function data.
    6 	Trace messages for operation functions.
    7 	Trace messages for internal control functions.
    8 	Contents of function-internal variables that may be interesting.
    9 	Extremely low-level tracing information.
  

• NOTE: Prior to SSSD version 1.8, debug levels could be defined globally in the [sssd] section of /etc/sssd/sssd.conf. Now, with SSSD version 1.8 and higher, each domain and service must have its own debug level defined. Here is an example of how a debug_level parameter will look in /etc/sssd/sssd.conf:

  [sssd]
  debug_level = 9
  ...
  
  [domain/LDAP]
  debug_level = 9
  ...
  
  [nss]
  debug_level = 9
  ...
  
  [pam]
  debug_level = 9
  ...
  

• Once the debug_level parameter has been added to /etc/sssd/sssd.conf, the SSSD service must be restarted for the changes to take effect:

  # service sssd stop ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; service sssd start
  Or
  # systemctl stop sssd ; rm -rf /var/log/sssd/* /var/lib/sss/{db,mc}/* ; systemctl start sssd
  

• Debug output can be found in /var/log/sssd/*, but it may also be helpful to investigate /var/log/secure for authentication issues.

• Sometimes, the DEBUG messages are not enough in telling you what went wrong. To gather more information, it's best to start the back end under strace by adding the command option to the [domain] section:

  command = strace -ff -o /tmp/sssd_be_strace /usr/libexec/sssd/sssd_be --debug-level=10 --domain ipa.example.com
  

• The -ff options are significant for debugging the child processes sssd_be spawns. When the sssd_be process execs a subprocess, strace would also track the child process by creating one file per process and appending a numeric PID after the base filename /tmp/sssd_be_strace.

• Then restart SSSD and run the sequence of commands that triggered the bug.

• Please make sure SELinux should be set to Permissive, otherwise sssd_be might not be able to execute child programs through strace. After that, you should see several files under /tmp/ matching the base filename:

  $ ls /tmp/sssd_be_strace*
  /tmp/sssd_be_strace.27067  /tmp/sssd_be_strace.27071  /tmp/sssd_be_strace.27079
  

• If you're looking for a file from a specific subprocess, it's best to just grep the strace log file for the binary name of the subprocess:

  $ grep krb5_child /tmp/sssd_be_strace*
   /tmp/sssd_be_strace.27079:execve("/usr/libexec/sssd/krb5_child", ["/usr/libexec/sssd/krb5_child"], [/* 24 vars */]) = 0
   /tmp/sssd_be_strace.27079:write(2, "krb5_child started.\n", 20)   = 20
  

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.