How to edit a CRUSH map and upload it back to the Ceph cluster?

Solution Verified - Updated

Environment

  • Red Hat Ceph Storage 1.2.3
  • Red Hat Ceph Storage 1.3
  • Red Hat Ceph Storage 2.x
  • Red Hat Ceph Storage 3.x
  • Red Hat Ceph Storage 4.x

Issue

  • What are the steps to download, edit, and upload a CRUSH map to a Ceph cluster?

Resolution

  • Editing the CRUSH map is sometimes the only method to work around certain situations, for example, setting 'straw2' as the default algorithm.

  • The steps to fetch the CRUSH map from the cluster, and re-inject it, are as follows:

1.Get the current crush map from the Ceph cluster

# sudo ceph osd getcrushmap -o /tmp/crushmap.bin

2.Decode the binary crushmap file to a text file

# sudo crushtool -d /tmp/crushmap.bin -o /tmp/crushmap.txt

3.Edit the CRUSH map as required.
NOTE: THIS IS AN IMPORTANT STEP, AND REQUIRES THE ADMINISTRATOR UNDERSTAND THE CRUSH MAP FLOW AND HOW TO ORDER THE COMPONENTS.

4.Once the changes are per the requirement, recompile it back to the binary format

# crushtool -c /tmp/crushmap.txt -o /tmp/crushmap.new

5.Test the CRUSH map rules

# crushtool --test -i /tmp/crushmap.new --num-rep 3 --rule 1 --show-statistics

6.Re-inject the CRUSH map

# sudo ceph osd setcrushmap -i /tmp/crushmap.new
SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.