Enabling memcached in heat

Solution Verified - Updated

Environment

  • Red Hat OpenStack Platform 10

Issue

  • Larger deployments can benefit from having memcached enabled in heat

Resolution

  1. Configuring the [cache] section in heat.conf:

    1. This is the caching section based on nova.conf. If your nova.conf has no comment, it's probably because of this issue. Don't worry, you can skip to next step.

              [cache]

        # Prefix for building the configuration dictionary for the cache region. This
        # should not need to be changed unless there is another dogpile.cache region
        # with the same configuration name. (string value)
        config_prefix=cache.oslo

        # Default TTL, in seconds, for any cached item in the dogpile.cache region. This
        # applies to any cached method that doesn't have an explicit cache expiration
        # time defined for it. (integer value)
        expiration_time=600

        # Dogpile.cache backend module. It is recommended that Memcache or Redis
        # (dogpile.cache.redis) be used in production deployments. For eventlet-based or
        # highly threaded servers, Memcache with pooling (oslo_cache.memcache_pool) is
        # recommended. For low thread servers, dogpile.cache.memcached is recommended.
        # Test environments with a single instance of the server can use the
        # dogpile.cache.memory backend. (string value)
        #backend=dogpile.cache.null
        backend=oslo_cache.memcache_pool

        # Arguments supplied to the backend module. Specify this option once per
        # argument to be passed to the dogpile.cache backend. Example format:
        # "<argname>:<value>". (multi valued)
        #backend_argument =

        # Proxy classes to import that will affect the way the dogpile.cache backend
        # functions. See the dogpile.cache documentation on changing-backend-behavior.
        # (list value)
        #proxies =

        # Global toggle for caching. (boolean value)
        #enabled=false
        enabled=True

        # Extra debugging from the cache backend (cache keys, get/set/delete/etc calls).
        # This is only really useful if you need to see the specific cache-backend
        # get/set/delete calls with the keys/values.  Typically this should be left set
        # to false. (boolean value)
        #debug_cache_backend=false

        # Memcache servers in the format of "host:port". (dogpile.cache.memcache and
        # oslo_cache.memcache_pool backends only). (list value)
        #memcache_servers=localhost:11211
        memcache_servers=172.17.1.21:11211,172.17.1.11:11211,172.17.1.10:11211

        # Number of seconds memcached server is considered dead before it is tried
        # again. (dogpile.cache.memcache and oslo_cache.memcache_pool backends only).
        # (integer value)
        #memcache_dead_retry=300

        # Timeout in seconds for every call to a server. (dogpile.cache.memcache and
        # oslo_cache.memcache_pool backends only). (integer value)
        #memcache_socket_timeout=3

        # Max total number of open connections to every memcached server.
        # (oslo_cache.memcache_pool backend only). (integer value)
        #memcache_pool_maxsize=10

        # Number of seconds a connection to memcached is held unused in the pool before
        # it is closed. (oslo_cache.memcache_pool backend only). (integer value)
        #memcache_pool_unused_timeout=60

        # Number of seconds that an operation will wait to get a memcache client
        # connection. (integer value)
        #memcache_pool_connection_get_timeout=10

    2. With crudini:

      crudini --set /etc/heat/heat.conf cache config_prefix cache.oslo
crudini --set /etc/heat/heat.conf cache expiration_time 600
crudini --set /etc/heat/heat.conf cache backend dogpile.cache.memcached
crudini --set /etc/heat/heat.conf cache enabled True
crudini --set /etc/heat/heat.conf cache memcache_servers 127.0.0.1:11211

    3. If you have instack-undercloud-5.3.7-13.el7ost as described in this This content is not included.bugzilla, you can include this in your hiera_override.yaml to automatically configure caching during undercloud install or undercloud upgrade:

      heat::config::heat_config:
  cache/backend: 
    value: "dogpile.cache.memcached"
  cache/memcache_server: 
    value: "127.0.0.1:11211"
  cache/enabled:
    value: "True"

  2. Once caching has been enabled with oslo.cache, we can optionally disable various heat-based caching. If we want to fully benefit from the cache, we should let these enabled (default).

    1. This is the config section that has to be added in heat.conf

      [constraint_validation_cache]
caching=False
[service_extension_cache]
caching=False
[resource_finder_cache]
caching=False

    2. With crudini

      crudini --set /etc/heat/heat.conf constraint_validation_cache caching False
crudini --set /etc/heat/heat.conf service_extension_cache caching False
crudini --set /etc/heat/heat.conf resource_finder_cache caching False

    3. Again, with instack-undercloud-5.3.7-13.el7ost, this would work:

      heat::config::heat_config:
  constraint_validation_cache/caching: 
    value: "False"
  service_extension_cache/caching: 
    value: "False"
  resource_finder_cache/caching:
    value: "False"

  3. Followed by a restart of the heat services

    # systemctl restart openstack-heat-\*

** NOTE ** Enabling caching on heat is known to cause Content from bugs.launchpad.net is not included.issue in certain circumstances where you delete and re-create resources with the same name before the cache could be invalidated.

Root Cause

The cache config section is missing from heat.conf. We can take it from nova.conf because both components are leveraging from oslo.cache.

Diagnostic Steps

Heat Caching types

heat can benefit from 2 types of cache:

  1. Token caching: Each request sent to one of the heat APIs requires a token. Token caching can accelerate the response time of these request because it saves a call to keystone and validates directly in the RAM. This is a common option on pretty much all OpenStack APIs.

  2. There's also heat "object caching" (for the lack of an official term). This includes 3 types of cache which are described in the Content from github.com is not included.code.

    a) constraint_validation_cache: Toggle to enable/disable caching when Orchestration Engine validates property constraints of stack. During property validation with constraints Orchestration Engine caches requests to other OpenStack services
    b) service_extension_cache: Toggle to enable/disable caching when Orchestration Engine retrieves extensions from other OpenStack services.
    c) resource_finder_cache: Toggle to enable/disable caching when Orchestration Engine looks for other OpenStack service resources using name or id.

SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.