Troubleshoot installation - Red Hat Ansible Automation Platform

Resolve preflight check failures

If aapctl preflight reports one or more failures, identify and address the specific failure before you install.

Procedure

Review the preflight output to identify which check failed.
Apply the fix for the reported failure:
- Kubeconfig not found: Verify that ~/.kube/config exists or set the KUBECONFIG environment variable. Use --kubeconfig /path/to/kubeconfig to specify an alternative file.
- Cluster unreachable: Run oc whoami to confirm cluster access. Check network connectivity, virtual private network (VPN), and proxy settings.
- OpenShift Container Platform version too low: Upgrade the cluster to the minimum supported version. See Automation orchestrator system requirements.
- No worker nodes found: Verify that nodes are labeled correctly:
```
$ oc get nodes --show-labels | grep node-role
```
Re-run the preflight check:
```
$ aapctl preflight ao
```

Resolve CatalogSource not ready

If the install command times out while waiting for the operator, the CatalogSource might not be ready.

Procedure

Verify that the CatalogSources exist and are healthy:
```
$ oc get catalogsource -A
```
Check that each CatalogSource shows READY in the STATUS column.
For disconnected environments, verify that the mirrored catalog is accessible from the cluster.

Resolve operator installation timeout

If aapctl install reports a timeout or appears to hang, investigate the Subscription and Operator Lifecycle Manager (OLM) state.

About this task

A common symptom is the error timed out waiting for InstallPlan or context deadline exceeded.

Procedure

Check the Subscription status:
```
$ oc get subscription -n automation-orchestrator
```
If the Subscription has no CURRENT CSV, OLM cannot resolve the operator package. Verify that the CatalogSource contains the operator:
```
$ oc get packagemanifests | grep automation-orchestrator
```

Check OLM logs for resolution errors:

$ oc logs -n openshift-operator-lifecycle-manager deploy/olm-operator --tail=50

Increase the timeout if the cluster is slow:
```
$ aapctl install ao --timeout 60m
```
Use --no-wait to apply resources without blocking, then monitor manually:
```
$ aapctl install ao --no-wait
$ aapctl status ao
```

Resolve operator deployment failure

If the ClusterServiceVersion (CSV) reports a failed phase with a reason such as InstallCheckFailed, investigate the CSV and operator pod.

Procedure

Examine the CSV for the failure reason:
```
$ oc get csv -n automation-orchestrator -o wide
```

Check the operator pod logs:

$ oc logs -n automation-orchestrator deploy/automation-orchestrator-operator-controller-manager --tail=50

Address the root cause based on the error:
- Image pull failure: Verify that the cluster can pull images from the operator catalog registry. Check ImagePullBackOff events on the operator pod.
- Insufficient resources: Check for resource quota or limit range violations in the namespace.
- Role-based access control (RBAC) errors: Verify that the operator ServiceAccount has the required permissions.

Resolve orphaned CSV blocking reinstall

If the install fails with a resolution error about an unreferenced ClusterServiceVersion (CSV), delete the orphaned CSV and reinstall.

About this task

The aapctl CLI detects orphaned CSVs and deletes them automatically. If automatic recovery fails, delete the CSV manually.

Procedure

Delete the orphaned CSV:

$ oc delete csv csv-name -n automation-orchestrator

Re-run the install command:
```
$ aapctl install ao
```

Resolve PostgreSQL connection failure

If the automation orchestrator operator reports a Degraded status condition and backend pods are in CrashLoopBackOff, the PostgreSQL connection might be misconfigured.

Procedure

Verify that the database Secrets exist in the correct namespace:
```
$ oc get secret -n automation-orchestrator | grep postgres
```
Verify that the Secret keys match the expected format (database, username, password). See Kubernetes Secrets for automation orchestrator.

Check that the PostgreSQL host is reachable from inside the cluster:

$ oc run pg-test --rm -it --image=registry.access.redhat.com/ubi9/ubi -- \
    bash -c "curl -v telnet://my-pg.example.com:5432"

Verify the sslMode setting matches your PostgreSQL server configuration. If the server does not present a trusted CA certificate, set sslMode to require instead of verify-ca.

Resolve CloudNativePG databases not ready

If aapctl status ao shows databases as "not applied," the CloudNativePG Cluster might not have any ready instances.

Procedure

Check that the CloudNativePG Cluster has at least one ready instance:

$ oc get clusters.postgresql.cnpg.io orchestrator-postgres -n automation-orchestrator -o jsonpath='{.status.readyInstances}'

If the ready instance count is 0, check the PostgreSQL pod status:
```
$ oc get pods -n automation-orchestrator -l cnpg.io/cluster=orchestrator-postgres
```
Verify that the CloudNativePG operator is running:
```
$ oc get pods -n cloudnative-pg
```
Check the CloudNativePG operator logs for errors:
```
$ oc logs -n cloudnative-pg deploy/cnpg-controller-manager --tail=50
```

Resolve install skipping existing resources

If re-running aapctl install does not update resources that already exist on the cluster, aapctl is skipping them by default.

Procedure

Re-run the install command with the --force flag to overwrite existing resources:

$ aapctl install ao --force

Alternatively, uninstall first and then install again:

$ aapctl uninstall ao
$ aapctl install ao

Interpret aapctl failure output

When an install command fails, aapctl automatically prints a resource status snapshot that shows which resources it deployed and which it did not.

Important:

aapctl is a Technology Preview feature. Technology Preview features provide early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. Technology Preview features are not fully supported under Red Hat production SLA and may not be functionally complete. Red Hat does not recommend using Technology Preview features in production. For more information on the support scope for Technology Preview features, see Technology Preview Features Support Scope.

The following example shows a typical failure output:

  [apply] Namespace "aap"
  [apply] OperatorGroup "ansible-automation-platform-operator"
  [apply] Subscription "ansible-automation-platform-operator"
  [wait]  waiting for AAP operator to be ready...

Resource status at time of failure:
  Namespace/aap
  OperatorGroup/ansible-automation-platform-operator  (aap)
  Subscription/ansible-automation-platform-operator   (aap)   no currentCSV

Error: timed out waiting for CRD to be registered: context deadline exceeded

This diagnostic runs only in live mode, not with --dry-run.

Use aapctl status after installation to recheck resource readiness at any time.

Check resource status after a failed installation

After an install command fails, check which resources aapctl deployed and identify the root cause of the failure.

Procedure

Check the overall resource status for your deployment target:
```
$ aapctl status ao
```

For additional detail, check individual resources:

$ oc get subscription -n automation-orchestrator -o yaml
$ oc get csv -n automation-orchestrator -o wide
$ oc get automationorchestrator -n automation-orchestrator -o yaml
$ oc get pods -n automation-orchestrator

Check the operator status conditions for specific error messages:
```
$ oc get automationorchestrator instance-name -n automation-orchestrator -o jsonpath='{.status.conditions}' | python3 -m json.tool
```
Replace instance-name with the name of your AutomationOrchestrator custom resource.