Enable debugging for OpenStack services in Red Hat Openstack Services on OpenShift 18 for troubleshooting

Solution Verified - Updated

Environment

  • Red Hat Openstack Services on OpenShift 18

For older versions of Red Hat OpenStack Platform, see these articles:

Issue

  • How to enable debug logging in RHOSO 18

Resolution

  1. Enabling or disabling debug logs on control plane OpenStack pods
  2. Enabling or disabling debug logs on EDPM nodes temporarily
    1. nova-compute
    2. neutron-ovn-metadata-agent
    3. neutron-sriov-agent

Enabling or disabling debug logs on control plane OpenStack pods

  1. Edit OpenStackControlPlane CRD, example:

    $ oc get -n openstack OpenStackControlPlane
NAME                      STATUS   MESSAGE
openstack-control-plane   True     Setup complete

$ oc edit -n openstack OpenStackControlPlane/openstack-control-plane

  2. Debug logs can be enabled by setting [DEFAULT] debug=True in OpenStackControlPlane.spec.<service_name>.template.customServiceConfig.
    Example for keystone service, find the keystone service definition, add or edit customServiceConfig under the template section and set debug=True under [DEFAULT] section:

    spec:
   :
  keystone:	
      :
    template:
        :
      customServiceConfig: |    <<<<<<< Add or edit this section
        [DEFAULT]               <<<<<<<
        debug=True              <<<<<<<

  3. Save the change.

  4. Verify new service pods are automatically created and use 'oc logs' to verify debug logs is enabled:

    $ oc get pods -n openstack -l service=keystone
NAME                        READY   STATUS    RESTARTS   AGE
keystone-78cc4f8b7b-dbl6g   0/1     Running   0          29s    <<<<New container starting up
keystone-7d9f57b786-6w5vv   1/1     Running   0          18m    <<<<Old containers will terminate
keystone-7d9f57b786-drpvn   1/1     Running   0          18m

$ oc logs keystone-78cc4f8b7b-dbl6g |grep debug |head -1
[Tue Jan 01 00:00:00.000000 2025] [wsgi:error] [pid 27:tid 58] 2025-01-01 00:00:00.000 27 DEBUG mod_wsgi [-] debug                          = True log_opt_values /usr/lib/python3.9/site-packages/oslo_config/cfg.py:2602\x1b[00m

Disable Control plane OpenStack pods debug logging

  1. Edit OpenStackControlPlane CRD, example:

    $ oc get -n openstack OpenStackControlPlane
NAME                      STATUS   MESSAGE
openstack-control-plane   True     Setup complete

$ oc edit -n openstack OpenStackControlPlane/openstack-control-plane

  2. Debug logs can be disabled by setting [DEFAULT] debug=False in OpenStackControlPlane.spec.<service_name>.template.customServiceConfig.
    Example for keystone service, find the keystone service definition, add or edit customServiceConfig under the template section and set debug=False under [DEFAULT] section:

    spec:
   :
  keystone:	
      :
    template:
        :
      customServiceConfig: |
        [DEFAULT]
        debug=False             <<<<<<< Modify this line

  3. Save the change.

  4. Verify new service pods are automatically created and use 'oc logs' to verify debug logs is disabled:

    $ oc get pods -n openstack -l service=keystone
NAME                        READY   STATUS    RESTARTS   AGE
keystone-78cc4f8b7b-dbl6g   0/1     Running   0          29s    <<<<New container starting up
keystone-7d9f57b786-6w5vv   1/1     Running   0          18m    <<<<Old containers will terminate
keystone-7d9f57b786-drpvn   1/1     Running   0          18m

$ oc logs keystone-78cc4f8b7b-dbl6g |grep DEBUG
$

Enabling or disabling debug logs on EDPM nodes temporarily

Note: Debug logs enabled by this procedure on EDPM nodes are not permanent. It is supposed to be reverted when a deployment runs by creating a OpenStackDataPlaneDeployment resource.

nova-compute

Note: nova-compute debug logging is enabled by default.

Run the following as the root user on the EDPM nodes:

# crudini --set /var/lib/openstack/config/nova/01-nova.conf DEFAULT debug true
# systemctl restart edpm_nova_compute.service

Disable nova-compute debug logging

Run the following as the root user on the EDPM nodes:

# crudini --set /var/lib/openstack/config/nova/01-nova.conf DEFAULT debug false
# systemctl restart edpm_nova_compute.service

neutron-ovn-metadata-agent

Note: neutron-ovn-metadata-agent debug logging is enabled by default.

Run the following as the root user on the EDPM nodes:

# crudini --set /var/lib/config-data/ansible-generated/neutron-ovn-metadata-agent/01-neutron-ovn-metadata-agent.conf DEFAULT debug true
# systemctl restart edpm_ovn_metadata_agent.service

Disable neutron-ovn-metadata-agent debug logging

Run the following as the root user on the EDPM nodes:

# crudini --set /var/lib/config-data/ansible-generated/neutron-ovn-metadata-agent/01-neutron-ovn-metadata-agent.conf DEFAULT debug false
# systemctl restart edpm_ovn_metadata_agent.service

neutron-sriov-agent

Run the following as the root user on the EDPM nodes:

# crudini --set /var/lib/config-data/ansible-generated/neutron-sriov-agent/01-neutron.conf DEFAULT debug true
# systemctl restart edpm_neutron_sriov_agent.service

Disable neutron-sriov-agent debug logging

Run the following as the root user on the EDPM nodes:

# crudini --set /var/lib/config-data/ansible-generated/neutron-sriov-agent/01-neutron.conf DEFAULT debug false
# systemctl restart edpm_neutron_sriov_agent.service
SBR
Product(s)
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.