How to use virsh cpu comparisson to determine the cpu compatibility
Environment
- Red Hat OpenStack Platform 10.0 (RHOSP)
- Red Hat OpenStack Platform 16.2
- Red Hat OpenStack Platform 17.1
Issue
- How to use virsh cpu-compare/hypervisor-cpu-compare and cpu-baseline/hypervisor-cpu-baseline to determine the cpu compatibility?
- Setting
cpu_mode=host-modelin nova config file on both compute nodes still instance live-migration is getting failed due toUnacceptable CPU infoerror.
Resolution
- Gather the
virsh capabilitiesorvirsh domcapabilitiesoutput from both compute nodes for which you want to check the compatibility. Visit libvirt Content from www.libvirt.org is not included.documentation for more details about commands.
node1 # virsh capabilities
<capabilities>
<host>
<uuid>411b330c-c7d4-11e4-1111-00000000001d</uuid>
<cpu>
<arch>x86_64</arch>
<model>Westmere</model>
<vendor>Intel</vendor>
<topology sockets='1' cores='6' threads='2'/>
<feature name='invtsc'/>
<feature name='rdtscp'/>
<feature name='pdpe1gb'/>
<feature name='dca'/>
<feature name='pcid'/>
<feature name='pdcm'/>
<feature name='xtpr'/>
<feature name='tm2'/>
<feature name='est'/>
<feature name='smx'/>
<feature name='vmx'/>
<feature name='ds_cpl'/>
<feature name='monitor'/>
<feature name='dtes64'/>
<feature name='pclmuldq'/>
<feature name='pbe'/>
<feature name='tm'/>
<feature name='ht'/>
<feature name='ss'/>
<feature name='acpi'/>
<feature name='ds'/>
<feature name='vme'/>
<pages unit='KiB' size='4'/>
<pages unit='KiB' size='2048'/>
<pages unit='KiB' size='1048576'/>
</cpu>
<power_management>
<suspend_mem/>
</power_management>
<migration_features>
<live/>
<uri_transports>
<uri_transport>tcp</uri_transport>
<uri_transport>rdma</uri_transport>
</uri_transports>
</migration_features>
<topology>
<cells num='2'>
<cell id='0'>
<memory unit='KiB'>50267000</memory>
<pages unit='KiB' size='4'>12566750</pages>
<pages unit='KiB' size='2048'>0</pages>
<pages unit='KiB' size='1048576'>0</pages>
<distances>
<sibling id='0' value='10'/>
<sibling id='1' value='21'/>
</distances>
<cpus num='12'>
<cpu id='0' socket_id='0' core_id='0' siblings='0,12'/>
<cpu id='2' socket_id='0' core_id='1' siblings='2,14'/>
<cpu id='4' socket_id='0' core_id='2' siblings='4,16'/>
<cpu id='6' socket_id='0' core_id='8' siblings='6,18'/>
<cpu id='8' socket_id='0' core_id='9' siblings='8,20'/>
<cpu id='10' socket_id='0' core_id='10' siblings='10,22'/>
<cpu id='12' socket_id='0' core_id='0' siblings='0,12'/>
<cpu id='14' socket_id='0' core_id='1' siblings='2,14'/>
<cpu id='16' socket_id='0' core_id='2' siblings='4,16'/>
<cpu id='18' socket_id='0' core_id='8' siblings='6,18'/>
<cpu id='20' socket_id='0' core_id='9' siblings='8,20'/>
<cpu id='22' socket_id='0' core_id='10' siblings='10,22'/>
</cpus>
</cell>
<cell id='1'>
<memory unit='KiB'>50331648</memory>
<pages unit='KiB' size='4'>12582912</pages>
<pages unit='KiB' size='2048'>0</pages>
<pages unit='KiB' size='1048576'>0</pages>
<distances>
<sibling id='0' value='21'/>
<sibling id='1' value='10'/>
</distances>
<cpus num='12'>
<cpu id='1' socket_id='1' core_id='0' siblings='1,13'/>
<cpu id='3' socket_id='1' core_id='1' siblings='3,15'/>
<cpu id='5' socket_id='1' core_id='2' siblings='5,17'/>
<cpu id='7' socket_id='1' core_id='8' siblings='7,19'/>
<cpu id='9' socket_id='1' core_id='9' siblings='9,21'/>
<cpu id='11' socket_id='1' core_id='10' siblings='11,23'/>
<cpu id='13' socket_id='1' core_id='0' siblings='1,13'/>
<cpu id='15' socket_id='1' core_id='1' siblings='3,15'/>
<cpu id='17' socket_id='1' core_id='2' siblings='5,17'/>
<cpu id='19' socket_id='1' core_id='8' siblings='7,19'/>
<cpu id='21' socket_id='1' core_id='9' siblings='9,21'/>
<cpu id='23' socket_id='1' core_id='10' siblings='11,23'/>
</cpus>
</cell>
</cells>
</topology>
<secmodel>
<model>selinux</model>
<doi>0</doi>
<baselabel type='kvm'>system_u:system_r:svirt_t:s0</baselabel>
<baselabel type='qemu'>system_u:system_r:svirt_tcg_t:s0</baselabel>
</secmodel>
<secmodel>
<model>dac</model>
<doi>0</doi>
<baselabel type='kvm'>+107:+107</baselabel>
<baselabel type='qemu'>+107:+107</baselabel>
</secmodel>
</host>
<guest>
<os_type>hvm</os_type>
<arch name='i686'>
<wordsize>32</wordsize>
<emulator>/usr/libexec/qemu-kvm</emulator>
<machine maxCpus='240'>pc-i440fx-rhel7.2.0</machine>
<machine canonical='pc-i440fx-rhel7.2.0' maxCpus='240'>pc</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.0.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.3.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.2.0</machine>
<machine canonical='pc-q35-rhel7.2.0' maxCpus='240'>q35</machine>
<machine maxCpus='240'>rhel6.4.0</machine>
<machine maxCpus='240'>rhel6.0.0</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.5.0</machine>
<machine maxCpus='240'>rhel6.6.0</machine>
<machine maxCpus='240'>rhel6.1.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.0.0</machine>
<machine maxCpus='240'>rhel6.2.0</machine>
<domain type='qemu'/>
<domain type='kvm'>
<emulator>/usr/libexec/qemu-kvm</emulator>
</domain>
</arch>
<features>
<cpuselection/>
<deviceboot/>
<disksnapshot default='on' toggle='no'/>
<acpi default='on' toggle='yes'/>
<apic default='on' toggle='no'/>
<pae/>
<nonpae/>
</features>
</guest>
<guest>
<os_type>hvm</os_type>
<arch name='x86_64'>
<wordsize>64</wordsize>
<emulator>/usr/libexec/qemu-kvm</emulator>
<machine maxCpus='240'>pc-i440fx-rhel7.2.0</machine>
<machine canonical='pc-i440fx-rhel7.2.0' maxCpus='240'>pc</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.0.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.3.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.2.0</machine>
<machine canonical='pc-q35-rhel7.2.0' maxCpus='240'>q35</machine>
<machine maxCpus='240'>rhel6.4.0</machine>
<machine maxCpus='240'>rhel6.0.0</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.5.0</machine>
<machine maxCpus='240'>rhel6.6.0</machine>
<machine maxCpus='240'>rhel6.1.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.0.0</machine>
<machine maxCpus='240'>rhel6.2.0</machine>
<domain type='qemu'/>
<domain type='kvm'>
<emulator>/usr/libexec/qemu-kvm</emulator>
</domain>
</arch>
<features>
<cpuselection/>
<deviceboot/>
<disksnapshot default='on' toggle='no'/>
<acpi default='on' toggle='yes'/>
<apic default='on' toggle='no'/>
</features>
</guest>
</capabilities>
node2# virsh capabilities
<capabilities>
<host>
<uuid>7ee6e5ca-3e9b-11e6-0000-00000000008f</uuid>
<cpu>
<arch>x86_64</arch>
<model>Broadwell</model>
<vendor>Intel</vendor>
<topology sockets='1' cores='10' threads='2'/>
<feature name='invtsc'/>
<feature name='abm'/>
<feature name='pdpe1gb'/>
<feature name='rdrand'/>
<feature name='f16c'/>
<feature name='osxsave'/>
<feature name='dca'/>
<feature name='pdcm'/>
<feature name='xtpr'/>
<feature name='tm2'/>
<feature name='est'/>
<feature name='smx'/>
<feature name='vmx'/>
<feature name='ds_cpl'/>
<feature name='monitor'/>
<feature name='dtes64'/>
<feature name='pbe'/>
<feature name='tm'/>
<feature name='ht'/>
<feature name='ss'/>
<feature name='acpi'/>
<feature name='ds'/>
<feature name='vme'/>
<pages unit='KiB' size='4'/>
<pages unit='KiB' size='2048'/>
<pages unit='KiB' size='1048576'/>
</cpu>
<power_management>
<suspend_mem/>
</power_management>
<migration_features>
<live/>
<uri_transports>
<uri_transport>tcp</uri_transport>
<uri_transport>rdma</uri_transport>
</uri_transports>
</migration_features>
<topology>
<cells num='2'>
<cell id='0'>
<memory unit='KiB'>133939292</memory>
<pages unit='KiB' size='4'>33484823</pages>
<pages unit='KiB' size='2048'>0</pages>
<pages unit='KiB' size='1048576'>0</pages>
<distances>
<sibling id='0' value='10'/>
<sibling id='1' value='21'/>
</distances>
<cpus num='20'>
<cpu id='0' socket_id='0' core_id='0' siblings='0,20'/>
<cpu id='1' socket_id='0' core_id='1' siblings='1,21'/>
<cpu id='2' socket_id='0' core_id='2' siblings='2,22'/>
<cpu id='3' socket_id='0' core_id='3' siblings='3,23'/>
<cpu id='4' socket_id='0' core_id='4' siblings='4,24'/>
<cpu id='5' socket_id='0' core_id='8' siblings='5,25'/>
<cpu id='6' socket_id='0' core_id='9' siblings='6,26'/>
<cpu id='7' socket_id='0' core_id='10' siblings='7,27'/>
<cpu id='8' socket_id='0' core_id='11' siblings='8,28'/>
<cpu id='9' socket_id='0' core_id='12' siblings='9,29'/>
<cpu id='20' socket_id='0' core_id='0' siblings='0,20'/>
<cpu id='21' socket_id='0' core_id='1' siblings='1,21'/>
<cpu id='22' socket_id='0' core_id='2' siblings='2,22'/>
<cpu id='23' socket_id='0' core_id='3' siblings='3,23'/>
<cpu id='24' socket_id='0' core_id='4' siblings='4,24'/>
<cpu id='25' socket_id='0' core_id='8' siblings='5,25'/>
<cpu id='26' socket_id='0' core_id='9' siblings='6,26'/>
<cpu id='27' socket_id='0' core_id='10' siblings='7,27'/>
<cpu id='28' socket_id='0' core_id='11' siblings='8,28'/>
<cpu id='29' socket_id='0' core_id='12' siblings='9,29'/>
</cpus>
</cell>
<cell id='1'>
<memory unit='KiB'>134217728</memory>
<pages unit='KiB' size='4'>33554432</pages>
<pages unit='KiB' size='2048'>0</pages>
<pages unit='KiB' size='1048576'>0</pages>
<distances>
<sibling id='0' value='21'/>
<sibling id='1' value='10'/>
</distances>
<cpus num='20'>
<cpu id='10' socket_id='1' core_id='0' siblings='10,30'/>
<cpu id='11' socket_id='1' core_id='1' siblings='11,31'/>
<cpu id='12' socket_id='1' core_id='2' siblings='12,32'/>
<cpu id='13' socket_id='1' core_id='3' siblings='13,33'/>
<cpu id='14' socket_id='1' core_id='4' siblings='14,34'/>
<cpu id='15' socket_id='1' core_id='8' siblings='15,35'/>
<cpu id='16' socket_id='1' core_id='9' siblings='16,36'/>
<cpu id='17' socket_id='1' core_id='10' siblings='17,37'/>
<cpu id='18' socket_id='1' core_id='11' siblings='18,38'/>
<cpu id='19' socket_id='1' core_id='12' siblings='19,39'/>
<cpu id='30' socket_id='1' core_id='0' siblings='10,30'/>
<cpu id='31' socket_id='1' core_id='1' siblings='11,31'/>
<cpu id='32' socket_id='1' core_id='2' siblings='12,32'/>
<cpu id='33' socket_id='1' core_id='3' siblings='13,33'/>
<cpu id='34' socket_id='1' core_id='4' siblings='14,34'/>
<cpu id='35' socket_id='1' core_id='8' siblings='15,35'/>
<cpu id='36' socket_id='1' core_id='9' siblings='16,36'/>
<cpu id='37' socket_id='1' core_id='10' siblings='17,37'/>
<cpu id='38' socket_id='1' core_id='11' siblings='18,38'/>
<cpu id='39' socket_id='1' core_id='12' siblings='19,39'/>
</cpus>
</cell>
</cells>
</topology>
<secmodel>
<model>selinux</model>
<doi>0</doi>
<baselabel type='kvm'>system_u:system_r:svirt_t:s0</baselabel>
<baselabel type='qemu'>system_u:system_r:svirt_tcg_t:s0</baselabel>
</secmodel>
<secmodel>
<model>dac</model>
<doi>0</doi>
<baselabel type='kvm'>+107:+107</baselabel>
<baselabel type='qemu'>+107:+107</baselabel>
</secmodel>
</host>
<guest>
<os_type>hvm</os_type>
<arch name='i686'>
<wordsize>32</wordsize>
<emulator>/usr/libexec/qemu-kvm</emulator>
<machine maxCpus='240'>pc-i440fx-rhel7.2.0</machine>
<machine canonical='pc-i440fx-rhel7.2.0' maxCpus='240'>pc</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.0.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.3.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.2.0</machine>
<machine canonical='pc-q35-rhel7.2.0' maxCpus='240'>q35</machine>
<machine maxCpus='240'>rhel6.4.0</machine>
<machine maxCpus='240'>rhel6.0.0</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.5.0</machine>
<machine maxCpus='240'>rhel6.6.0</machine>
<machine maxCpus='240'>rhel6.1.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.0.0</machine>
<machine maxCpus='240'>rhel6.2.0</machine>
<domain type='qemu'/>
<domain type='kvm'>
<emulator>/usr/libexec/qemu-kvm</emulator>
</domain>
</arch>
<features>
<cpuselection/>
<deviceboot/>
<disksnapshot default='on' toggle='no'/>
<acpi default='on' toggle='yes'/>
<apic default='on' toggle='no'/>
<pae/>
<nonpae/>
</features>
</guest>
<guest>
<os_type>hvm</os_type>
<arch name='x86_64'>
<wordsize>64</wordsize>
<emulator>/usr/libexec/qemu-kvm</emulator>
<machine maxCpus='240'>pc-i440fx-rhel7.2.0</machine>
<machine canonical='pc-i440fx-rhel7.2.0' maxCpus='240'>pc</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.0.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.3.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.2.0</machine>
<machine canonical='pc-q35-rhel7.2.0' maxCpus='240'>q35</machine>
<machine maxCpus='240'>rhel6.4.0</machine>
<machine maxCpus='240'>rhel6.0.0</machine>
<machine maxCpus='240'>pc-i440fx-rhel7.1.0</machine>
<machine maxCpus='240'>rhel6.5.0</machine>
<machine maxCpus='240'>rhel6.6.0</machine>
<machine maxCpus='240'>rhel6.1.0</machine>
<machine maxCpus='240'>pc-q35-rhel7.0.0</machine>
<machine maxCpus='240'>rhel6.2.0</machine>
<domain type='qemu'/>
<domain type='kvm'>
<emulator>/usr/libexec/qemu-kvm</emulator>
</domain>
</arch>
<features>
<cpuselection/>
<deviceboot/>
<disksnapshot default='on' toggle='no'/>
<acpi default='on' toggle='yes'/>
<apic default='on' toggle='no'/>
</features>
</guest>
</capabilities>
Note: In this case node1 contains the old CPU model and node2 is with new CPU model.
Using virsh cpu-compare/hypervisor-cpu-compare
- Create a new file
virsh-caps-workstation-cpu-only.xmlonnode2and add the following content ofnode1in the newly created file.
<cpu>
<arch>x86_64</arch>
<model>Westmere</model>
<vendor>Intel</vendor>
<topology sockets='1' cores='6' threads='2'/>
<feature name='invtsc'/>
<feature name='rdtscp'/>
<feature name='pdpe1gb'/>
<feature name='dca'/>
<feature name='pcid'/>
<feature name='pdcm'/>
<feature name='xtpr'/>
<feature name='tm2'/>
<feature name='est'/>
<feature name='smx'/>
<feature name='vmx'/>
<feature name='ds_cpl'/>
<feature name='monitor'/>
<feature name='dtes64'/>
<feature name='pclmuldq'/>
<feature name='pbe'/>
<feature name='tm'/>
<feature name='ht'/>
<feature name='ss'/>
<feature name='acpi'/>
<feature name='ds'/>
<feature name='vme'/>
<pages unit='KiB' size='4'/>
<pages unit='KiB' size='2048'/>
<pages unit='KiB' size='1048576'/>
</cpu>
- Issue the following command to compare the CPU of node1 with node2.
node2 # virsh cpu-compare virsh-caps-workstation-cpu-only.xml
or
node2 # virsh hypervisor-cpu-compare virsh-caps-workstation-cpu-only.xml
Host CPU is a superset of CPU described in virsh-caps-workstation-cpu-only.xml
Using virsh cpu-baseline/hypervisor-cpu-baseline
- Take the vcpu section from both outputs and put into a single file.
node2 # cat both-cpus.xml
<cpu>
<arch>x86_64</arch>
<model>Westmere</model>
<vendor>Intel</vendor>
<topology sockets='1' cores='6' threads='2'/>
<feature name='invtsc'/>
<feature name='rdtscp'/>
<feature name='pdpe1gb'/>
<feature name='dca'/>
<feature name='pcid'/>
<feature name='pdcm'/>
<feature name='xtpr'/>
<feature name='tm2'/>
<feature name='est'/>
<feature name='smx'/>
<feature name='vmx'/>
<feature name='ds_cpl'/>
<feature name='monitor'/>
<feature name='dtes64'/>
<feature name='pclmuldq'/>
<feature name='pbe'/>
<feature name='tm'/>
<feature name='ht'/>
<feature name='ss'/>
<feature name='acpi'/>
<feature name='ds'/>
<feature name='vme'/>
<pages unit='KiB' size='4'/>
<pages unit='KiB' size='2048'/>
<pages unit='KiB' size='1048576'/>
</cpu>
<cpu>
<arch>x86_64</arch>
<model>Broadwell</model>
<vendor>Intel</vendor>
<topology sockets='1' cores='10' threads='2'/>
<feature name='invtsc'/>
<feature name='abm'/>
<feature name='pdpe1gb'/>
<feature name='rdrand'/>
<feature name='f16c'/>
<feature name='osxsave'/>
<feature name='dca'/>
<feature name='pdcm'/>
<feature name='xtpr'/>
<feature name='tm2'/>
<feature name='est'/>
<feature name='smx'/>
<feature name='vmx'/>
<feature name='ds_cpl'/>
<feature name='monitor'/>
<feature name='dtes64'/>
<feature name='pbe'/>
<feature name='tm'/>
<feature name='ht'/>
<feature name='ss'/>
<feature name='acpi'/>
<feature name='ds'/>
<feature name='vme'/>
<pages unit='KiB' size='4'/>
<pages unit='KiB' size='2048'/>
<pages unit='KiB' size='1048576'/>
</cpu>
- Issue the following command to determine the common ground for migration. Here as our CPU model is different for both compute nodes hence it's showing that to support live-migration settings in
/etc/nova/nova.confof both compute nodes should becpu_mode=customandcpu_model=Westmere. Restart thenovaservice on compute nodes after making the change. Before trying the live-migrationStopandStartthe existing instance so that it can pick-up the new changes or try to spawn a new instance to verify that live-migration is working or not.
node2 # virsh cpu-baseline both-cpus.xml
or
node2 # virsh hypervisor-cpu-baseline both-cpus.xml
<cpu mode='custom' match='exact'>
<model fallback='allow'>Westmere</model>
<vendor>Intel</vendor>
<feature policy='require' name='vme'/>
<feature policy='require' name='ds'/>
<feature policy='require' name='acpi'/>
<feature policy='require' name='ss'/>
<feature policy='require' name='ht'/>
<feature policy='require' name='tm'/>
<feature policy='require' name='pbe'/>
<feature policy='require' name='pclmuldq'/>
<feature policy='require' name='dtes64'/>
<feature policy='require' name='monitor'/>
<feature policy='require' name='ds_cpl'/>
<feature policy='require' name='vmx'/>
<feature policy='require' name='smx'/>
<feature policy='require' name='est'/>
<feature policy='require' name='tm2'/>
<feature policy='require' name='xtpr'/>
<feature policy='require' name='pdcm'/>
<feature policy='require' name='pcid'/>
<feature policy='require' name='dca'/>
<feature policy='require' name='pdpe1gb'/>
<feature policy='require' name='rdtscp'/>
<feature policy='require' name='invtsc'/>
</cpu>
After making the changes to the nova.conf, you must restart the nova service on all affected compute nodes, and restart the VMs to ensure they are using the correct cpus. After the restarts, the VMs should migrate properly.
Root Cause
Nodes CPU do not have the same capabilities.
Diagnostic Steps
- Check nova configuration for cpu mode.
# RHOSP10
(compute)$ grep cpu_mode /etc/nova/nova.conf
cpu_mode=host-model
# RHOSP16/17
(compute)$ grep cpu_mode /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf
cpu_mode=host-model
- Live-migration is getting failed with following error message.
# RHOSP10
(compute)$ nova --debug live-migration 145d6082-0034-44b0-a944-632dedb36608 node1
# RHOSP16/17
(compute)$ openstack server migrate 145d6082-0034-44b0-a944-632dedb36608 --live-migration --host node1 --wait
# trace error:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 908, in main
OpenStackComputeShell().main(argv)
File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 835, in main
args.func(self.cs, args)
File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3396, in do_live_migration
args.disk_over_commit)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 351, in live_migrate
disk_over_commit)
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1122, in live_migrate
'disk_over_commit': disk_over_commit})
File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1238, in _action
return self.api.client.post(url, body=body)
File "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 176, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 92, in request
raise exceptions.from_response(resp, body, url, method)
BadRequest: Unacceptable CPU info: CPU doesn't have compatibility.
SBR
Product(s)
Components
Category
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.