memcache_servers parameter format for IPv6 address/hostname
Environment
- Red Hat Openstack Platform 13
- Red Hat Openstack Platform 15
- Red Hat Openstack Platform 16
Issue
Memcache ia an open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
The Identity service authentication mechanism for services uses Memcached to cache tokens.
In order to setup a memcached based backend we need to pass the memcache_servers parameter to oslo.config. memcache_servers accepts different formats for IPv6 address/hostname but not all of them end in a correct configuration of the memcached backend.
Related articles:
Resolution
It is needed to ensure that address familly proto is given with IPv6 address/hostname with the format inet6:[IPv6 address | IPv6 hostname]:PORT
These are some examples of WORKING configuration for memcache_servers parameter:
memcache_servers=inet6:[fd00:fd00:fd00:2000::18]:11211,inet6:[fd00:fd00:fd00:2000::19]:11211,inet6:[fd00:fd00:fd00:2000::1c]:11211
memcache_servers=inet6:[controller-2.internalapi]:11211,inet6:[controller-1.internalapi]:11211,inet6:[controller-0.internalapi]:11211
These are some examples of NOT WORKING configuration for memcache_servers parameter:
memcache_servers=controller-2.internalapi:11211,controller-1.internalapi:11211,controller-0.internalapi:11211
memcache_servers=inet6:controller-2.internalapi:11211,inet6:controller-1.internalapi:11211,inet6:controller-0.internalapi:11211
memcache_servers=[controller-2.internalapi]:11211,[controller-1.internalapi]:11211,[controller-0.internalapi]:11211
Root Cause
python-memcached creates the client socket with given servers, it expects to get the address familly (the proto) given in the passed urls (memcache_servers parameter).
It could be an issue if an url at IPv6 format is given without the proto specified, then the socket will be initialized with a wrong address familly and the connection with the server will be not established. In this case the socket will be intialized with an IPv6 format and an AF_INET address familly, which correspond to an IPv4 familly address where normally it should use an AF_INET6 which is designed for IPv6 addresses.
Diagnostic Steps
Check memcache_servers parameter within keystone.conf.
It can be seen that memcached connections are not being established, this is an example:
[root@controller-1 keystone]# netstat -natp |grep 11211 |grep keystone:
tcp6 0 0 fd00:fd00:fd00:20:43420 fd00:fd00:fd00:20:11211 ESTABLISHED 242085/keystone-adm
tcp6 0 0 fd00:fd00:fd00:20:43408 fd00:fd00:fd00:20:11211 ESTABLISHED 242085/keystone-adm
tcp6 0 0 fd00:fd00:fd00:20:43224 fd00:fd00:fd00:20:11211 ESTABLISHED 242086/keystone-mai
tcp6 0 0 fd00:fd00:fd00:20:44010 fd00:fd00:fd00:20:11211 ESTABLISHED 242086/keystone-mai
tcp6 0 0 fd00:fd00:fd00:20:44746 fd00:fd00:fd00:20:11211 ESTABLISHED 242085/keystone-adm
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.