Request to heat-api times out randomly with error "Unable to establish connection to"

Environment

• Red Hat Enterprise Linux OpenStack Platform 6.0 (RHOS)

Issue

• Connections to Heat APIs are timing out randomly with below error message. This usually happens when too many requests are sent to heat-api.

Unable to establish connection to http://heat-api-ip:8004/v1/9cfdcca495354d738ea9bd6220c413fd/stacks?

Resolution

• Increase the number of heat-api workers from the default value to be equal to the number of cores on the controller node. In /etc/heat/heat.conf, set

workers=x

Replace x with the number of cores on the system. Then restart heat-api service.

Root Cause

• By default, heat api spawns only a single thread for all heat-api calls which makes it overloaded when too many requests are spawned.

Diagnostic Steps

• Running heat stack-list would return the following:

Unable to establish connection to http://heat-api-ip:8004/v1/9cfdcca495354d738ea9bd6220c413fd/stacks?

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.