Enable debugging for OpenStack services in Red Hat OpenStack Platform 12 to 14

Solution Verified - Updated 1 Oct 2025

---

Environment

• Red Hat OpenStack Platform 12
• Red Hat OpenStack Platform 13
• Red Hat OpenStack Platform 14

Issue

• How to enable debugging for OpenStack services in Red Hat OpenStack Platform 12 to 14
• How to enable debug logging in RHOSP 12
• How to enable debug logging in RHOSP 13
• How to enable debug logging in RHOSP 14

Resolution

Note: For Red Hat OpenStack Platform versions from 7 to 11, please refer to Enable debugging for OpenStack services in Red Hat OpenStack Platform 7 to 11 .

Note: For Red Hat OpenStack Platform versions from 15 to 17, please refer Enable debugging for OpenStack services in Red Hat OpenStack Platform 15, 16 and 17.

Note: For Red Hat Openstack Services on OpenShift 18, please refer Enable debugging for OpenStack services in Red Hat Openstack Services on Openshift 18.

Note: Debug logging by default creates large amounts of data in /var/log on any node where it is enabled. Debug logging should only be enabled for short durations of time and only with sufficient amount of disk space. Should Red Hat support ask for debug logging to be switched on, it should be switched off as soon as troubleshooting is over and a support ticket was closed. If Red Hat support needs debug logging for a specific service on a specific node, they will explicitly request where and for which service this setting should be enabled.

Apply changes temporarily

This method of enabling debug logging is not persistent , a re-deployment of the overcloud will revert changes back to their defaults.

Nova

Enable debug logging

Controllers

Set debug to true in the nova.conf configuration file

crudini --set /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf DEFAULT debug true

For OSP 13 also set:

crudini --set /var/lib/config-data/puppet-generated/nova_placement/etc/nova/nova.conf DEFAULT debug true

Restart the Nova related containers
For OSP 13 :

docker ps | awk '/nova_/ {print $NF}' | xargs -I {} docker restart {}

Computes

For the compute nodes set:

crudini --set /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf DEFAULT debug true

Restart the Nova related containers

docker ps | awk '/nova_/ {print $NF}' | xargs -I {} docker restart {}

Disable debug logging

Controllers

Set debug to true in the nova.conf configuration file

crudini --set /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf DEFAULT debug false

For OSP 13 and later also set

crudini --set /var/lib/config-data/puppet-generated/nova_placement/etc/nova/nova.conf DEFAULT debug false

Restart the Nova related containers

docker ps | awk '/nova_/ {print $NF}' | xargs -I {} docker restart {}

Computes

For the compute nodes set:

crudini --set /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf DEFAULT debug false

Restart the Nova related containers

docker ps | awk '/nova_/ {print $NF}' | xargs -I {} docker restart {}

Cinder

Enable debug logging

OSP 12

Set debug to true in the /etc/cinder/cinder.conf configuration file

crudini --set /etc/cinder/cinder.conf DEFAULT debug true

Restart the Cinder related services

systemctl restart 'openstack-cinder-*'

OSP 13 to 14

Manipulate the settings in /var/lib/config-data/puppet-generated/cinder/etc/cinder/cinder.conf:

crudini --set /var/lib/config-data/puppet-generated/cinder/etc/cinder/cinder.conf DEFAULT debug true

Then, restart the cinder containers:

docker ps | awk '/cinder_/ {print $NF}' | xargs -I {} docker restart {}

Or for cinder volume in particular, one can restart the pacemaker resource:

pcs resource restart openstack-cinder-volume

Disable debug logging

OSP 12

Set debug to false in the /etc/cinder/cinder.conf configuration file

crudini --set /etc/cinder/cinder.conf DEFAULT debug false

Restart the Cinder related services

systemctl restart 'openstack-cinder-*'

OSP 13 to 14

Manipulate the settings in /var/lib/config-data/puppet-generated/cinder/etc/cinder/cinder.conf:

crudini --set /var/lib/config-data/puppet-generated/cinder/etc/cinder/cinder.conf DEFAULT debug false

Then, restart the cinder containers:

docker ps | awk '/cinder_/ {print $NF}' | xargs -I {} docker restart {}

Or for cinder volume in particular, one can restart the pacemaker resource:

pcs resource restart openstack-cinder-volume

Keystone

Enable debug logging

Set debug to true in the keystone.conf configuration file

crudini --set /var/lib/config-data/puppet-generated/keystone/etc/keystone/keystone.conf DEFAULT debug true

Restart the Keystone container

docker restart keystone

Disable debug logging

Set debug to false in the keystone.conf configuration file

crudini --set /var/lib/config-data/puppet-generated/keystone/etc/keystone/keystone.conf DEFAULT debug false

Restart the Keystone container

docker restart keystone

Neutron

Enable debug logging

OSP 12

Set debug to true in the /etc/neutron/neutron.conf configuration file

crudini --set /etc/neutron/neutron.conf DEFAULT debug true

Restart the Neutron related services.

systemctl restart 'neutron-*'

OSP 13 to 14

Set debug to true in the /etc/neutron/neutron.conf configuration file:

crudini --set  /var/lib/config-data/puppet-generated/neutron/etc/neutron/neutron.conf  DEFAULT debug true
crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/l3_agent.ini DEFAULT debug true
crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/metadata_agent.ini DEFAULT debug true
crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/dhcp_agent.ini DEFAULT debug true

Restart the Neutron related services:

docker ps | awk '/neutron_/ {print $NF}' | xargs -I {} docker restart {}

Disable debug logging

OSP 12

Set debug to false in the /etc/neutron/neutron.conf configuration file

crudini --set /etc/neutron/neutron.conf DEFAULT debug false

Restart the Neutron related services. This will cause a momentary loss in network connectivity to your instances

systemctl restart 'neutron-*'

OSP 13 to 14

Set debug to false in the /etc/neutron/neutron.conf configuration file:

crudini --set  /var/lib/config-data/puppet-generated/neutron/etc/neutron/neutron.conf  DEFAULT debug false
crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/l3_agent.ini DEFAULT debug false
crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/metadata_agent.ini DEFAULT debug false
crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/dhcp_agent.ini DEFAULT debug false

Restart the Neutron related services:

docker ps | awk '/neutron_/ {print $NF}' | xargs -I {} docker restart {}

Octavia

Enable debug logging

OSP 13 to 14

Set debug to true in the octavia.conf configuration file on the controller nodes:

crudini --set /var/lib/config-data/puppet-generated/octavia/etc/octavia/octavia.conf DEFAULT debug true

Restart the Octavia related containers

docker ps | awk '/octavia/ {print $NF}' | xargs docker restart

Disable debug logging

Set debug to false in the octavia.conf configuration file on the controller nodes:

crudini --set /var/lib/config-data/puppet-generated/octavia/etc/octavia/octavia.conf DEFAULT debug false

Restart the Octavia related containers

docker ps | awk '/octavia/ {print $NF}' | xargs docker restart

OVN

See https://access.redhat.com/solutions/4270652 for instructions about how to enable debug logging with OVN in Red Hat OpenStack Platform 13.

Glance

Enable debug logging

Set debug to true in the glance-api.conf configuration file

crudini --set /var/lib/config-data/puppet-generated/glance_api/etc/glance/glance-api.conf DEFAULT debug true

Restart the glance_api container

docker restart glance_api

Disable debug logging

Set debug to false in the glance-api.conf configuration file

crudini --set /var/lib/config-data/puppet-generated/glance_api/etc/glance/glance-api.conf DEFAULT debug false

Restart the glance_api container

docker restart glance_api

Heat

Enable debug logging

Set debug to true in the heat.conf configuration files

crudini --set /var/lib/config-data/puppet-generated/heat/etc/heat/heat.conf DEFAULT debug true
crudini --set /var/lib/config-data/puppet-generated/heat_api/etc/heat/heat.conf DEFAULT debug true
crudini --set /var/lib/config-data/puppet-generated/heat_api_cfn/etc/heat/heat.conf DEFAULT debug true

Restart the Heat related containers

docker ps | awk '/heat_/ {print $NF}' | xargs -I {} docker restart {}

Disable debug logging

Set debug to false in the heat.conf configuration files

crudini --set /var/lib/config-data/puppet-generated/heat/etc/heat/heat.conf DEFAULT debug false
crudini --set /var/lib/config-data/puppet-generated/heat_api/etc/heat/heat.conf DEFAULT debug false
crudini --set /var/lib/config-data/puppet-generated/heat_api_cfn/etc/heat/heat.conf DEFAULT debug false

Restart the Heat related containers

docker ps | awk '/heat_/ {print $NF}' | xargs -I {} docker restart {}

Horizon

Enable debug logging

Set debug to True in the /var/lib/config-data/puppet-generated/horizon/etc/openstack-dashboard/local_settings configuration file

DEBUG = True

Also change the handlers to DEBUG level

 'handlers': {
...
        'file': {
            'level': 'DEBUG', <===
        'horizon': {
             'handlers': ['console'], <===
             'level': 'DEBUG',
        'openstack_dashboard': {
             'handlers': ['file'], <===
            'level': 'DEBUG', <===
        'novaclient': {
            'handlers': ['file'], <===
             'level': 'DEBUG', <===
        'cinderclient': {
            'handlers': ['file'], <===
            'level': 'DEBUG', <===
        # repeat for each component that needs to be debugged

Restart the httpd related service

docker restart horizon

Disable debug logging

Set debug to True in the /var/lib/config-data/puppet-generated/horizon/etc/openstack-dashboard/local_settings configuration file

DEBUG = False

Revert the handlers back to INFO

 'handlers': {
...
        'file': {
            'level': 'INFO', <===
        'horizon': {
             'handlers': ['file'], <===
             'level': 'INFO',
        'openstack_dashboard': {
             'handlers': ['file'], <===
            'level': 'INFO', <===
        'novaclient': {
            'handlers': ['file'], <===
             'level': 'INFO', <===
        'cinderclient': {
            'handlers': ['file'], <===
            'level': 'INFO', <===
        # repeat for each component that needs to be debugged

Restart the httpd related service

docker restart horizon

Other services

For other services please refer to your specific Openstack Red Hat manual.

Root Cause

Note: This same method can be used for any other OpenStack service as well. However other core services will be found on the controller node.

Diagnostic Steps

Further details illustrated with the nova_compute service

• First Check for the nova_compute container on the compute node:

[root@compute-0 heat-admin]# docker ps |grep nova
8891d5b03157        192.168.24.1:8787/rhosp12/openstack-nova-compute-docker:12.0-20171129.1         "kolla_start"       6 days ago          Up 6 days (unhealthy)                       nova_migration_target
ec91402be9a8        192.168.24.1:8787/rhosp12/openstack-nova-compute-docker:12.0-20171129.1         "kolla_start"       6 days ago          Up 6 days (healthy)                         nova_compute
bd78adf6820c        192.168.24.1:8787/rhosp12/openstack-nova-libvirt-docker:12.0-20171129.1         "kolla_start"       6 days ago          Up 6 days                                   nova_libvirt
dbc1cbbe4e9e        192.168.24.1:8787/rhosp12/openstack-nova-libvirt-docker:12.0-20171129.1         "kolla_start"       6 days ago          Up 6 days                                   nova_virtlogd
[root@compute-0 heat-admin]# 

• Get into the container's bash shell:

[root@compute-0 heat-admin]# docker exec -it nova_compute /bin/bash
()[nova@compute-0 /]$  

• Notice debug in the container is set to False:

()[nova@compute-0 /]$ grep ^debug /etc/nova/nova.conf 
debug=False

• Exit the container and edit the puppet-generated nova.conf that will control the container's nova.conf.

()[nova@compute-0 /]$ exit
exit
[root@compute-0 heat-admin]# 
[root@compute-0 heat-admin]# grep ^debug /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf 
debug=True
[root@compute-0 heat-admin]# 

• restart the docker container to enable the config change. (This is equivelant to a service restart in previous versions of OpenStack)

[root@compute-0 heat-admin]# docker restart nova_compute
nova_compute
[root@compute-0 heat-admin]# 

• Finally check that debug logging is enabled within the container.

[root@compute-0 heat-admin]# docker exec -it nova_compute /bin/bash
()[nova@compute-0 /]$ grep ^debug /etc/nova/nova.conf                                                                 
debug=True

Note that logs are located within the container in Red Hat OpenStack 12 or /var/log/containers/$service/ not in /var/log/$service/ on the system.

[root@compute-0 heat-admin]# docker exec -it nova_compute /bin/bash

()[nova@compute-0 /]$ tail /var/log/nova/nova-compute.log                                                                                        
2017-12-12 16:00:11.085 1 INFO nova.compute.resource_tracker [req-a13294b9-7314-44b8-a5a7-2878b63d4f23 - - - - -] Final resource view: name=compute-0.localdomain phys_ram=6143MB used_ram=4096MB phys_disk=49GB used_disk=0GB total_vcpus=4 used_vcpus=0 pci_stats=[]
2017-12-12 16:00:59.912 1 INFO nova.compute.manager [req-a13294b9-7314-44b8-a5a7-2878b63d4f23 - - - - -] Running instance usage audit for host compute-0.localdomain from 2017-12-12 15:00:00 to 2017-12-12 16:00:00. 0 instances.
()[nova@compute-0 /]$ exit
exit

[root@compute-0 ~]$ ls /var/log/containers/nova/
nova-compute.log
[root@compute-0 heat-admin]# ll /var/log/nova/
total 0
[root@compute-0 heat-admin]# 

SBR

• Stack

Product(s)

• Red Hat OpenStack Platform

Components

• nova

Tags

• Debugging

---

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.