When upgrading to RHEL9, IDM users are not able to login anymore.
Environment
- Red Hat Entreprise Linux (RHEL) 9
- Identity Management (IDM)
Issue
The typical situation for this issue is :
- you have an old IPA version installed (for example RHEL-7) with a trust to Active Directory
Then you upgrade to RHEL-9.2, and your IDM users (other than admin) are not able to login anymore.
Resolution
You would have to create a SID for all your users and groups:
# kinit admin
# ipa config-mod --enable-sid --add-sids
Root Cause
In 2021 a number of attacks on Kerberos protocol, targetting specifically environments with cross-Active Directory/POSIX interoperability, were reported to Microsoft. Kerberos tickets without special extensions to store metadata about the Kerberos principal identities were prone for manipulation by malicious servers. To address these attacks, domain controllers must enforce use of these special extensions and legit servers must validate information provided by the extensions. The extension, MS-PAC, relies on additional information about Kerberos principal as stored in Active Directory. RHEL IdM and Samba AD DC for long implemented these extensions but only enforced their use for AD interoperability. In particular, RHEL IdM did not require to issue Kerberos tickets with MS-PAC structure to own users but required them to be present for all Active Directory users coming through a forest trust connection.
In order to address the issues, Microsoft added few more extensions to the MS-PAC structure and required their issuance and presence from conforming implementations. The original attack is known as CVE-2021-42287 and was reported by Andrew Bartlett of Samba Team. Other attacks were done against Active Directory, not Kerberos protocol, to help to issue this ticket.
RHEL IdM in RHEL 8.5.z or later releases moved to enforce issuance of Kerberos tickets with MS-PAC structure with these extensions. The presence of MS-PAC structure was still not required by the legit servers but RHEL IdM KDC did require information to issue MS-PAC structure to be present in the Kerberos principal in IdM database. This information includes Security Identifier (SID) detail, as designed in MS-PAC specification.
In 2022 another attack was devised that allowed malicious server to replace content of the MS-PAC structure extension of a user-issued Kerberos ticket at will. This attack is based on a pre-imaging of a PAC server checksum field. The fix to this attack from Microsoft side was an introduction of another, extended signature which is calculated over the whole MS-PAC structure rather than over the server checksum. Microsoft decided to force presence of the new signature starting in November 2023. Any Kerberos tickets without this signature will be rejected. This is known as CVE-2022-37967, reported by Tom Tervoort, Principal Security Specialist for Secura.
The new, extended, signature is a part of MS-PAC structure. Therefore, we have three requirements:
- extended KDC signature must be present in the MS-PAC structure;
- MS-PAC structure must be present, in order to contain extended KDC signature;
- SID of a Kerberos principal must be known, in order to allow KDC to issue a Kerberos ticket with MS-PAC structure.
With RHEL 8.5.z, RHEL IdM and MIT Kerberos issue MS-PAC structure with additional extensions as required to address CVE-2021-42287.
With RHEL 9.2, RHEL IdM and MIT Kerberos force presence of MS-PAC structure in the Kerberos tickets issued by RHEL IdM KDC. These tickets also contain the extended signature, as required to fix CVE-2022-37967 but the presence of new signature is not yet enforced.
Please also see our product documentation for additional details.
Diagnostic Steps
When the user is missing the SID (represented by the attribute ipaNTSecurityIdentifier), the user may get a failure when attempting to retrieve a TGT by running kinit:
[root@idm01 ]# KRB5_TRACE=/dev/stdout kinit -V admin
[2958484] 1688678719.152334: Matching admin@EXAMPLE.NET in collection with result: 0/Success
Using existing cache: 0:10108
Using principal: admin@EXAMPLE.NET
[2958484] 1688678719.152335: Getting initial credentials for admin@EXAMPLE.NET
-- snip --
[2958484] 1688678720.246239: Response was from primary KDC
[2958484] 1688678720.246240: Received error from KDC: -1765328324/Generic error (see e-text)
kinit: Generic error (see e-text) while getting initial credentials
And investigating /var/log/krb5kdc.log in the respective KDC, this can be observed:
Jul 05 14:50:39 idm01.example.net krb5kdc[2730307]: AS_REQ (4 etypes {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes128-cts-hmac-sha1-96(17)}) 172.20.90.120: HANDLE_AUTHDATA: admin@EXAMPLE.NET for krbtgt/EXAMPLE.NET@EXAMPLE.NET, No such file or directory
Jul 05 14:50:39 idm01.example.net krb5kdc[2730307]: closing down fd 11
So performing the steps in the Resolution section should add both the attribute ipaNTSecurityIdentifier along with the objectclass ipantuserattrs to the object, allowing it to perform the authentication.
In case the solution exits but the error persists, it's very likely that the sidgen task failed due to objects fielding gidNumber or uidNumber attributes out of the ranges defined in the IPA idranges, which in case would return an error similar to this in the directory server error logs (/var/log/dirsrv/slapd-INSTANCE-NAME/error_log)
[03/Oct/2023:13:02:43.529842808 -0400] - INFO - slapd_daemon - Listening on /run/slapd-EXAMPLE-NET.socket for LDAPI requests
[03/Oct/2023:13:02:43.845852460 -0400] - ERR - sidgen_task_thread - [file ipa_sidgen_task.c, line 194]: Sidgen task starts ...
[03/Oct/2023:13:02:44.033707984 -0400] - ERR - find_sid_for_ldap_entry - [file ipa_sidgen_common.c, line 521]: Cannot convert Posix ID [10011] into an unused SID.
[03/Oct/2023:13:02:44.034702871 -0400] - ERR - do_work - [file ipa_sidgen_task.c, line 154]: Cannot add SID to existing entry.
[03/Oct/2023:13:02:44.042168794 -0400] - ERR - sidgen_task_thread - [file ipa_sidgen_task.c, line 199]: Sidgen task finished [32].
This will cause the task to exit and not perform additional modifications as to avoid adding duplicate RID/SIDs in the environment.
If this is observed, you may refer to the documentation below to either modify the user/group objects, or add additional ID ranges that cover the stranded uids/gids, then retry the steps in the solution section.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.