RHEL-8.9 IdM update, web UI and CLI 401 Unauthorized with KDC S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC - user and group objects need SIDs
Environment
RHEL-8.9
ipa-server-4.9.12-11.module+el8.9.0+20824+f2605038.x86_64 or later
krb5-server-1.18.2-26.el8_9.x86_64
Issue
After updating to RHEL-8.9 with IPA packages from 4.9.12-9 to 4.9.12-11+( no errors ), a Kerberos kinit works correctly, but any ipa command line of WebUI access is denied, with an HTTP error 401:
/var/log/httpd/error_log
...ipa: INFO: 401 Unauthorized: Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty)
An ipa user-show command may fail like this, ending with a 401 Unauthorized error:
# ipa -d user-show
...
File "/usr/lib/python3.6/site-packages/ipalib/rpc.py", line 730, in single_request response.msg)
xmlrpc.client.ProtocolError: ... 401 Unauthorized>
The RHEL IdM KDC log was showing a trace similar to this example:
/var/log/krb5kdc.log
...(info): TGS_REQ ...: S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC: ..., KDC policy rejects request
Resolution
The error signature is with the KDC event log keyword S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC.
This indicates a RHEL IdM Kerberos user ticket did not have a PAC information associated with it.
It is necessary to enable the SID usage and trigger a SIDgen task to generate SIDs of the existing users and groups.
Note this task might be resource-intensive.
Run as the root user:
# ipa config-mod --enable-sid --add-sids
Once completed, verify the IdM admin user account entry has an ipaNTsecurityidentifier IPA LDAP attribute with a SID value that ends with -500, which is the SID reserved for the domain administrator, as root:
# ipa user-show admin --all | grep ipantsecurityidentifier
output example:
ipantsecurityidentifier: S-1-5-21-2633809701-976279387-419745629-500
Alternative option
If there is an issue with the admin user, the ipa commands will fail. However, it is possible to run this command as root:
# /usr/libexec/ipa/oddjob/org.freeipa.server.config-enable-sid --add-sids
Please note: Do not run "python3 /usr/libexec/ipa/oddjob/org.freeipa.server.config-enable-sid --add-sids" as doing so can break the SELinux fcontext and prevent command execution of the sidgen task.
If it fails with errors like
ERROR ('Tuple_to_LDAPMod(): expected a byte string in the list', None)
you should choose your IdM's domain NetBIOS name (15 max characters, NOT the same as the realm, more about limitations Content from learn.microsoft.com is not included.here), then run
# /usr/libexec/ipa/oddjob/org.freeipa.server.config-enable-sid --add-sids --netbios-name CHOSENNAME
References
RHEL-8.9 documentation references:
12.1. Privilege Attribute Certificate (PAC) use in IdM
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_idm_users_groups_hosts_and_access_control_rules/assembly_strengthening-kerberos-security-with-pac-information_managing-users-groups-hosts#con_privilege-attribute-certificate-pac-use-in-idm_assembly_strengthening-kerberos-security-with-pac-information
12.2. Enabling Security Identifiers (SIDs) in IdM
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_idm_users_groups_hosts_and_access_control_rules/assembly_strengthening-kerberos-security-with-pac-information_managing-users-groups-hosts#proc_enabling-security-identifiers-sids-in-idm_assembly_strengthening-kerberos-security-with-pac-information
Note the IdM KDC has been issuing Kerberos tickets with PAC information since RHEL-8.5
reference:
RHEL 8.5 release notes - 4.13. Identity Management
...
IdM KDC now issues Kerberos tickets with PAC information to increase security
Root Cause
Great article:
The dynamic Kerberos PAC ticket signature enforcement mechanism fixes cross-version incompatibility in IdM
https://access.redhat.com/articles/7046409
A RHEL IdM replica was installed before RHEL-8.5, does not have an AD trust, and did not have an ipa-sidgen task ran because it is part of configuring a trust with an Active Directory domain.
And over time with the various RHEL IdM updates, IPA LDAP user and group entries may not carry any Security Identifiers (SIDs) attributes.
With the RHEL-8.9 update, to increase security, Identity Management (IdM) now issues Kerberos tickets with Privilege Attribute Certificate (PAC) information by default in new deployments.
A PAC has rich information about a Kerberos principal, including its Security Identifier (SID), group memberships, and home directory information.
This change is applied to mitigate this security issue related to Kerberos delegation constrain bypass in S4U2Proxy : CVE-2020-17049, Bronze-bit attack
- ipa-kdb: Detect and block Bronze-Bit attacks
Resolves: RHEL-16532
Reference: https://access.redhat.com/security/cve/CVE-2020-17049
Any Kerberos account missing its SID attribute is now prevented from being used for delegation constrain operations in RHEL IdM.
This can be identified in the krb5kdc logs with error message that contains:
S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC
The solution is to make sure your users have SIDs associated.
The only ways to generate SIDs are either to run the ipa-adtrust-install command to add the Trust Controller role to an IdM server, or run a enable-sid task to add those SIDs IPA LDAP attributes.
Diagnostic Steps
/var/log/httpd/error_log
[Thu Jan 11 12:43:21.389345 2024] [wsgi:error] [pid 1234567:tid 139867429353216] [remote 10.10.10.10:35181] ipa: INFO: 401 Unauthorized: Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty)
An ipa user-show command may fail like this, ending with a 401 Unauthorized error:
# ipa -d user-show
...
Traceback (most recent call last):
File
"/usr/lib/python3.6/site-packages/ipaclient/remote_plugins/__init__.py",
line 120, in get_package
plugins = api._remote_plugins
AttributeError: 'API' object has no attribute '_remote_plugins'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ipalib/rpc.py", line 730, in single_request response.msg)
xmlrpc.client.ProtocolError: <ProtocolError for ipa2.idm.example.test/ipa/session/json: 401 Unauthorized>
ipa: INFO: Connection to https://ipa2.idm.example.test/ipa/session/json failed with <ProtocolError for ipa2.idm.example.test/ipa/session/json: 401 Unauthorized>
The RHEL IdM KDC log was showing a trace similar to this example:
/var/log/krb5kdc.log
Jan 11 17:41:35 ipa2.idm.example.test krb5kdc[1230](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 10.10.10.10: S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC: authtime 1704991295, etypes {rep=UNSUPPORTED:(0)} HTTP/ipa2.idm.example.test(a)idm.example.test for ldap/ipa2.idm.example.test(a)idm.example.test, KDC policy rejects request
Jan 11 17:41:35 ipa2.idm.example.test krb5kdc[1230](info): ...CONSTRAINED-DELEGATION s4u-client=<unknown>
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.