Attempt to scale out compute nodes in RHEL OSP 7 fails with "Maximum resources per stack exceeded"

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux OpenStack Platform 7

Issue

  • Attempting to add additional compute nodes to an existing RHEL OSP 7 environment fails with Heat reporting the error Maximum resources per stack exceeded.
"stack":{
"parent":null,
"disable_rollback":true,
"description":"Nova API,Keystone,Heat Engine and API,Glance,Neutron,Dedicated MySQL server,Dedicated RabbitMQ Server,Group of Nova Computes\n",
"links":[
{
"href":"http://192.168.24.1:8004/v1/f92abc8e8b4c4fe2a6a8d8e03ab0ce31/stacks/overcloud/64651112-aea9-4664-9070-a095a171fe92",
"rel":"self"
}
],
"stack_status_reason":"ResourceUnknownStatus: Resource failed - Unknown status FAILED due to \"RequestLimitExceeded: Request limit exceeded: Maximum resources per stack exceeded.\"",
"stack_name":"overcloud",
"stack_user_project_id":"2a96dacd134147db88f9c921ff1f9b11",
"stack_owner":"admin",
"creation_time":"2015-09-07T02:27:07Z",
"capabilities":[
],
"notification_topics":[
],
"updated_time":"2015-09-07T19:01:35Z",
"timeout_mins":240,
"stack_status":"UPDATE_FAILED",
"parameters":{
"NeutronPublicInterfaceRawDevice":"",
"KeystoneSigningKey":"******",
"NtpServer":"0.rhel.pool.ntp.org",
"NeutronDhcpAgentsPerNetwork":"3",
"NeutronFlatNetworks":"datacentre",
"NovaEnableRbdBackend":"False",
"controllerImage":"overcloud-full",
"GlancePassword":"******",
"NeutronTunnelIdRanges":"1:1000",
"CinderEnableRbdBackend":"False",
"SwiftReplicas":"3",
"NeutronPublicInterfaceTag":"",
"ControllerHostnameFormat":"%stackname%-controller-%index%",

Resolution

Modify the default max_resources_per_stack in heat.conf to an appropriate value. The number of resources could be up around 30 per compute node and a value of number of compute nodes * 30 is recommended.

Root Cause

Heat counts the number of resources in all nested stacks toward the max_resources_per_stack limit. The default limit is 1000. If this value is exceeded, deploying additional nodes will fail with the Maximum resources per stack exceeded error.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.