Ceph - How to migrate a pool from using 'straw' to 'straw2'?

Environment

• Red Hat Ceph Storage

Issue

• Starting from RHCS hammer (0.94) onwards, Ceph Supports a new algorithm named straw2. Prior to RHCS luminous (3.x) the Crush buckets needed to be manually migrated in the CRUSH map from straw -> straw2, and re-injected back. There are newer commands that can be leveraged rather then manually updating the crush mapping.

Resolution

Step [1]:

• First validate the straw algorithm used for the crush bucket:
  - NOTE: in this example a crush bucket is named 'root' and still using 'straw'

# ceph osd getcrushmap -o crush.map; crushtool -d crush.map | grep straw; rm -f crush.map
44
tunable straw_calc_version 1
	alg straw2
	alg straw2
	alg straw2
	alg straw2
root straw {
	alg straw

Step [2]:

• Validate the versions of the active client connections:
  - NOTE: Some features of Ceph may require 'require_min_compat_client' to be set to a minimum Ceph version before the algorithm conversion can take place.
  - NOTE: A "features" hashing of "0x3ffddff8ffacffff" = nautilus. There is a known issue where this hashing will display incorrectly as luminous.

# ceph features
{
    "mon": {
        "group": {
            "features": "0x3ffddff8eeacfffb", 
            "release": "luminous",
            "num": 3
        }
    },
    "osd": {
        "group": {
            "features": "0x3ffddff8eeacfffb",
            "release": "luminous",
            "num": 9
        }
    },
    "client": {
        "group": {
            "features": "0x3ffddff8eeacfffb",
            "release": "luminous",
            "num": 7
        }
    }
}

Step [3] Take a Backup of the crushmap:

# ceph osd getcrushmap -o ./crushmap.bak

Step [4] Convert the running crushmap from straw -> straw2

# ceph osd crush set-all-straw-buckets-to-straw2

- Note: Changing a bucket type from straw to straw2 will result in a reasonably small amount of data movement, depending on how much the bucket item weights vary from each other. When the weights are all the same no data will move, and when item weights vary significantly there will be more movement.

- NOTE: As explained in Step [2] if the minimum required version criteria isn't met, you may be required set this to an appropriate value before proceeding as prompted by the error below:

# ceph osd crush set-all-straw-buckets-to-straw2
Error EINVAL: new crush map requires client version hammer but require_min_compat_client is firefly
# ceph osd set-require-min-compat-client luminous
set require_min_compat_client to luminous

Step [5] Apply optimal profile:

ceph osd crush tunables optimal

Step [6] validate the crush bucket has been updated:

# ceph osd getcrushmap -o crush.map; crushtool -d crush.map | grep straw; rm -f crush.map
44
tunable straw_calc_version 1
	alg straw2
	alg straw2
	alg straw2
	alg straw2
root straw {
	alg straw2

Root Cause

• A new algorithm named 'straw2' has been introduced in Ceph Hammer (0.94).

• This was not set to default in legacy environments, since older Ceph versions won't support this.

• Older legacy environment should update this to take advantages of the improved straw2. More information can be found upstream about it's development from the Ceph lead architect Content from www.spinics.net is not included.HERE

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.