DNA ranges in IPA

Updated

Intro

Distributed Numeric Assignment range, or DNA range, in IPA is a subset of an ipa-local ID range dedicated to a specific replica, which replica will use to create IDs. This mechanism ensures that all users and groups will get unique IDs from automatic assignment - even if two users are simultaneously created on two different replicas, before the replication can come through, each replica will provide a user ID from its own DNA range, and since DNA ranges are not allowed to intersect, both users will get unique IDs.

Once a user or a group is created, the replica will shrink its DNA range by one, until eventually depleting the range.

If a replica doesn't have a DNA range, or it would get smaller than a certain threshold, it would request other replicas to get a new DNA range; and if it fails, user or group creation will fail, too.

It's worth noticing that a replica that doesn't have an assigned DNA range is a completely normal situation - they all, except the first server, don't have it initially, the problems start if none of the replicas have a DNA range to work with. You don’t necessarily have to intervene into the distribution process, as a replica should receive a DNA range automatically when it first needs it.

Distribution design

DNA ranges
DNA ranges

DNA range distribution mechanism has to provide both reliability and balancing. In IPA the process looks like this:

  • First server will get DNA range as entire default ipa-local ID range (some IDs will be immediately spent on automatically created users and groups);
  • The next replica, when first tries to create a user, will request initial server to 'share' the DNA range, and initial server will serve half of its remaining DNA range to the new replica;
  • Once another replica is trying add a user, it will go asking all known servers in a loop, and the first server that is able to 'share' its DNA range will do so;
  • Process repeats automatically for all other replicas when required. Same happens if any of the replicas get close to running out of IDs in their already assigned DNA range (number of IDs left is smaller than set up dnaThreshold attribute).

Note here - a replica will not share a half of its assigned DNA range if the resulting range fails under the threshold setup in the DNA plugin - dnaThreshold attribute.

This design is pretty straightforward, but comes with several issues:

  • There's no way to guarantee which server will be requested to 'share' its DNA range first. This may lead to a balancing issue when one of the replicas will have big, half-IDrange DNA range, and all the others will have significantly smaller ones. This, however, should be balancing itself overtime, when replicas with smaller DNA ranges deplete them;
  • creating users on different replicas most necessarily means they will not get consecutive IDs, which can be important;
  • This mechanism requires all the servers to be able to talk to each other. It's not always the case, and some of problems with that are described in Disconnected topologies section;
  • If a replica is lost, its DNA range is lost, too - we don't keep track of what's happening with the replica's DNA plugin. This issue is covered in DNA range preservation section;
  • Since the plugin supports tracking only two ranges (current DNA range and dnaNextRange) it does not support working with multiple ipa-local ID ranges and switching between them automatically. This issue is covered in Moving to new a POSIX ID range section;
  • Manually setting up ranges can break the logic of automatic distribution - in such case, returning to automatic distribution can be quite a difficult task.

DNA range in Directory Server

DNA ranges are managed by Distributed Numeric Assignment Plugin for Directory server and are replicated LDAP entries located in cn=posix-ids,cn=dna,cn=ipa,cn=etc,$SUFFIX, that look like this:

dn: dnaHostname=$hostname+dnaPortNum=389,cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=local
objectClass: dnaSharedConfig
objectClass: top
dnaHostname: $hostname
dnaPortNum: 389
dnaSecurePortNum: 636
dnaRemainingValues: 249998
dnaRemoteBindMethod: SASL/GSSAPI
dnaRemoteConnProtocol: LDAP

As you can see, this is only a generic entry that advertises a replica's DNA plugin. The real state of current DNA range can be found in cn=Posix IDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config and looks something like this for a replica with present DNA range:

# Posix IDs, Distributed Numeric Assignment Plugin, plugins, config
dn: cn=Posix IDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config
cn: Posix IDs
dnaExcludeScope: cn=provisioning,dc=example,dc=local
dnaFilter: (|(objectClass=posixAccount)(objectClass=posixGroup)(objectClass=ipaIDobject))
dnaMagicRegen: -1
dnaMaxValue: 349999
dnaNextValue: 100002
dnaScope: dc=example,dc=local
dnaSharedCfgDN: cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=local
dnaThreshold: 500
dnaType: uidNumber
dnaType: gidNumber
objectClass: top
objectClass: extensibleObject

Note here - cn=config tree is not replicated. That means that in order to get info about current state of DNA plugin on a specific replica one has to contact it directly via LDAP.

Working with DNA ranges in IPA

You can list currently set DNA ranges with

ipa-replica-manage dnarange-show

If you need to set a DNA range manually, you can do it with

ipa-replica-manage dnarange-set replica.fqdn startID-endID

where startID and endID are integers of starting and ending POSIX IDs for the proposed DNA range. This proposed DNA range should meet several requirements:

  • be inside a defined ipa-local ID range;
  • not overlap with existing DNA ranges;
  • ideally - none of the IDs inside this new DNA range should be already in use. IPA will not check if ID is already in use during user/group creation, so this can lead to ID duplication, which can be a serious security risk and is not supported.

It's not really possible to clean up the current DNA range for a replica, you can only assign some new valid values - or deplete it naturally while creating users or groups.

Warning! If you manually assign new values to a replica that already had a DNA range, that 'old' range will not be reused automatically. Please refer to the DNA range preservation section if you want to preserve the IDs left in the ‘old’ DNA range.

DNANextRange

In order to speed up the process of getting a new DNA range if current is close to being depleted, DNA plugin uses the dnaNextRange attribute. Once a replica's DNA range becomes smaller than set dnaThreshold attribute, it will request the other replicas for a next DNA range, and will switch to it once current is completely depleted.

IPA administrators can also use the dnaNextRange attribute to force the next DNA range to be a specific one. The set up 'next' range will be used the next time any replica comes with a request for a DNA range - not necessarily the replica it is set on, but once the replica depletes its own DNA range, it will try to allocate its new DNA range from its own dnaNextRange attribute first.

By default, it's not set for any of the replicas. If you try to list it you will see No on-deck range set:

# ipa-replica-manage dnanextrange-show
ipa1.example.local: No on-deck range set
ipa2.example.local: No on-deck range set
ipa3.example.local: No on-deck range set

However, once replica is getting close to run out of current DNA range, it will get one:

# ipa-replica-manage dnanextrange-show
ipa1.example.local: No on-deck range set
ipa2.example.local: No on-deck range set
ipa3.example.local: 163000-225499

You can set it manually via

# ipa-replica-manage dnanextrange-set replica.fqdn startID-endID

and it must meet the same requirements as any other new DNA range - no overlaps with existing DNA ranges, inside ipa-local ID range, no used IDs inside proposed DNA range.

You can unset the dnaNextRange via

# ipa-replica-manage dnanextrange-set replica.fqdn 0-0

Challenges

As already noted, DNA range management comes with a set of challenges. The DNA plugin is provided by 389ds Directory Server, and it's not really aware of IPA internal ID ranges and complicated topologies.

Disconnected topologies

Disconnected topology
Disconnected topology

For any operation with DNA ranges, DNA plugin will try to contact all the IPA servers current replica is aware of. But what happens if direct LDAP connection is restricted by firewall? Let's say that ipa3.example.com from the illustration above is allowed to talk to ipa1, but not ipa2. The replication topology is matching the firewall rules, and every change from ipa2 will replicate to ipa1 and then ipa3, so the topology is working. However, for DNA plugin this means, that not only it will be unable to list ipa2's current state from ipa3:

[root@ipa3 ~]# ipa-replica-manage dnarange-show
ipa1.example.local: 100002-349999
ipa2.example.local: Connection failed: cannot connect to 'ldaps://ipa2.example.local:636': Transport endpoint is not connected
ipa3.example.local: No range set

It will also fail to request a DNA range from it, meaning in our example both of ipa2 and ipa3 can effectively request DNA ranges only from ipa1.

Depending on your topology, this can leave you with a group of servers in one location unable to get DNA ranges from within themselves, and unable to contact servers from different locations to resolve this, even though you still might have a lot of IDs to work with.

Our recommendation is to allow all IPA servers to talk to each other using ports listed. If that is impossible due to environment restrictions, this has to be monitored and managed manually.

DNA range preservation during migrations and decommissions

Once a replica is removed from the topology, if it had a DNA range set up, it can be lost if the automatic mechanism fails to reuse it. This means that some parts of the IDrange will never be used to create IDs, and may result in uneven distributions of IDs.

Automatic range preservation mechanism

If replica is being removed with ipa-replica-manage del <FQDN>, the automatic process for replica preservation will trigger. It is described Content from www.freeipa.org is not included.in this design document, and looks roughly like this:

  1. The replica removal script will take current replica's DNA range, if defined;
  2. Then it will try to assign it to dnaNextRange of one of other replicas, that doesn't have it assigned yet;
  3. If it fails for any reason, and if the ipa-replica-manage was run without --force flag, it will prompt the user on further actions.

If it fails to transfer the range automatically, the manual approach is preferred to ensure the correct range transition.

Note: if any other procedure of replica removal is used instead of running ipa-replica-manage del first, the automatic range preservation mechanism will not trigger at all, and you have to account for that manually.

Manual approach for range preservation

If the replica is being decommissioned, and no new replica is immediately replacing it, in order to preserve the DNA range manually:

  1. List current DNA ranges prior to replica removal via ipa-replica-manage dnarange-show, note the one on the to-be-removed replica;
  2. Remove replica from the topology, wait for the changes to replicate through the topology and make sure it's no longer listed in ipa server-find output;
  3. List current dnaNextRange attributes via ipa-replica-manage dnanextrange-show, then select one of the replicas with unset range and assign old DNA range from the removed replica via
# ipa-replica-manage dnanextrange-set replica.fqdn startID-endID
  1. If there are no replicas without dnaNextRange - this has to be documented and managed manually as needed.

If the replica is being replaced (e.g. in event of migration to newer RHEL version), the procedure is simpler:

  1. List current DNA ranges prior to replica removal via ipa-replica-manage dnarange-show, note the one on the to-be-removed replica;
  2. Remove replica from the topology, wait for the changes to replicate through the topology and make sure it's no longer listed in ipa server-find output;
  3. Add new replica to the topology, then re-assign the old DNA range to it via
# ipa-replica-manage dnarange-set newreplica.fqdn startID-endID

Moving to a new POSIX ID range

By design, the DNA plugin is limited to tracking only two ranges at a time - current DNA range and dnaNextRange. When a new ipa-local range is added for the IPA installation, DNA plugin will not automatically account for that - it doesn't have any means to learn about this new range. That means that it doesn't support automatically provisioning IDs from multiple IPA ID ranges or switching between them automatically - we will have to do it manually.

If we want to move to a new IDrange gradually, the best way is to assign the entirety of the newly added ID range as dnaNextRange for one of the replicas:

# ipa-replica-manage dnanextrange-set replica.fqdn startID-endID

In this scenario, as replicas will use up their current DNA ranges, they will receive new ones from the new ID range.

If we want to force move automatic ID assignment to the new range we can do it manually:

  1. Divide the entirety of new IDrange into non-intersecting subranges as per your requirements;
  2. Assign the sub-ranges to the replicas that have currently assigned DNA ranges via
# ipa-replica-manage dnarange-set replica.fqdn startID-endID

In this scenario, all the newly created users and groups will be from the new ID range, and the old ID range will not be used for auto ID assignment.

SBR
Product(s)
Category
Components
Tags
Article Type