[Ceph] Recommended number of RGW multisite sync threads and endpoints

Environment

• Red Hat Ceph Storage
• RGW Multisite

Issue

What is the ideal number of RGW instances per site with sync thread enabled?

Resolution

For best possible RGW multisite sync performance it is recommended to only have two instances per site with rgw run sync thread set to true (which is the default setting). All other instances on the same zone should not have the sync thread enabled. Two instances is usually the best choice, as sync performance does not increase with further threads due to lock congestion and locking overhead. Having only one RGW instance with sync thread enabled per site would create a single point of failure.
Those two endpoints per zone should also be listed in the endpoints section of the respective zone and zonegroup configurations. Regular client traffic should not be directed to those endpoints, as best multisite performance is achieved if the sync endpoints only handle synchronization traffic. Hence they should be excluded in the load balancer or in the client configuration.

Root Cause

Having a high number of RGW instances with rgw enable sync thread enabled only adds locking overhead.
Using dedicated RGW endpoints for multisite synchronisation gives best possible performance.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.