An erasure coded pool with "4+2" chunks created on a ruleset with four OSD nodes, ends up moving the cluster status to HEALTH_WARN, why?

Solution Verified - Updated

Environment

  • Red Hat Ceph Storage 1.2.3

  • Red Hat Ceph Storage 1.3

Issue

  • An EC pool created with "4 + 2" chunks on a ruleset mapped to four OSD nodes, shifts the cluster from being HEALTH_OK to HEALTH_WARN.

  • A 'ceph -s' shows:

# ceph -s
cluster <cluster-id>
     health HEALTH_WARN
            600 pgs degraded
            600 pgs stuck degraded
            600 pgs stuck unclean
            600 pgs stuck undersized
            600 pgs undersized
     monmap e1: 3 mons at {mon1:6789/0,mon2:6789/0,mon3:6789/0}
            election epoch 6, quorum 0,1,2 mon1,mon2,mon3
     osdmap e361: 81 osds: 81 up, 81 in; 600 remapped pgs
      pgmap v1083: 864 pgs, 3 pools, 533 MB data, 181 objects
            5156 MB used, 61124 GB / 61129 GB avail
                 600 active+undersized+degraded+remapped
                 264 active+clean

Resolution

  • In case of an Erasure Coded pool, each object is cut to chunks and spread across the OSDs mapped to a placement group.

  • For example, an EC pool of 4+2 chunks on a RHCS cluster with a failure domain set to "host", a minimum of 6 hosts with OSDs will be needed.

  • Since the default "type" is set to "host", Ceph will ensure that each chunk will go to a different host. In this case, since there are only four hosts in the 'default' ruleset, the PGs won't reach a "active+clean" state at all.

  • An EC pool will have a profile tied to it and the default EC profile is named "default" which has a configuration of "2+1", ie.. 2 data objects and 1 coding object.

  • An EC pool once created and set to a profile, can't be changed to a different profile. The only way is to create a new pool with the default profile, and copy over the objects from the existing EC pool to the new one.

SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.