How can we get same Ceph osd name after repairing a failed disk ?

Solution Unverified - Updated

Environment

  • Inktank Ceph Enterprise 1.1
  • Inktank Ceph Enterprise 1.2

Issue

  • How can we get same Ceph osd name after repairing a failed disk ?
  • Ceph osd system had failed disk and after repairing the failed disk , not able to add the osd with same name as failed disk was having.
  • The failed disk name was /dev/sdh and osd name was osd.3 and we had executed following commands:
# ceph osd crush remove osd.3
# ceph auth del osd.3
# ceph osd rm osd.3
# ceph osd create
3
# ceph-deploy --overwrite-conf osd prepare osd02:/dev/sdh

Resolution

  • We should note that you can safely use osd.96 as the name of your new osd, this will not cause any problems within Ceph. If this is fine you can simply remove osd.3 from the crushmap by following the instructions here:

  • Content from ceph.com is not included.removing-osds-manual

  • Several of those commands will fail because there is no running osd.3 process but the end result will remove the osd from the crushmap and make itself available for use by the next disk.

  • The problem here originally happened due to a flaw in the procedure you followed. Since you are using ceph-deploy there is no need to run the ceph osd create command, ceph-deploy executes that internally.

  • If you do wish to reuse the osd.3 name, instead of the newly created osd.96, then go through the removal steps for both osd.3 and osd.96.

  • Content from ceph.com is not included.removing-osds-manual

  • After removing the osds successfully run the following command :

       # ceph-deploy --overwrite-conf osd prepare osd02:/dev/sdh --zap-disk

  • It will give you the name of osd.3 for that disk.

Root Cause

  • The problem here originally happened due to a flaw in the procedure which was followed. Since ceph-deploy was used here then there is no need to run the ceph osd create command, ceph-deploy executes that internally.
Product(s)
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.