Why the jobs on the automation controller are failing when using a newly created custom execution environment?

Solution Verified - Updated

Environment

  • Red Hat Ansible Automation Controller

Issue

  • How to Fix Job Template Failures on Ansible Automation Controller When Using Newly Created Custom Execution Environments?

  • Job Template Failures When Using New Custom Execution Environments in Automation Controller.

  • Unable to use Custom Execution Environment in Ansible Controller if it is built and saved on another node with a root user or a normal user, and then sent to the Ansible Controller host.

Resolution

  • Switch to the user where the newly created custom EE is present
  • Execute the following command: 
podman save -o custom_ee.tar localhost/my-custom-ee:latest

  • Copy the tar archive into /tmp on all Automation Controllers and Execution Nodes

  • Run the following command on each Automation Controller and Execution Node as the awx user : 

podman load -i /tmp/custom_ee.tar

To check further procedures of building a Custom Execution Environment, please refer to this KCS.

Root Cause

  • The custom execution environment was built using non awx user and was not present under awx user.

Diagnostic Steps

  • Output of the job template
Error: initializing source docker://localhost/my-custom-ee:latest: pinging container registry localhost: Get "https://localhost/v2/": tls: failed to verify certificate: x509: certificate is valid for https://<controller_url>, not localhost
  • Images present under awx user
  [root@...]# su - awx
  [awx@...]$ podman images
  REPOSITORY                                                            TAG         IMAGE ID      CREATED       SIZE
  registry.redhat.io/ansible-automation-platform-24/ee-supported-rhel8  latest      931ee203e2f9  2 months ago  1.72 GB
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.