Troubleshoot credential errors

Credential API error responses follow the RFC 9457 Problem Details format. Each error includes a machine-readable code, a human-readable detail message, and a retryable flag.

Resolve a credential validation error (422)

Error code: CREDENTIAL_VALIDATION_ERROR

Symptom: A 422 Unprocessable Entity response when creating or updating a credential.

Possible causes:

Unknown fields in the input:

{
  "detail": "Unknown field(s): invalid_field"
}

The inputs object contains field names that do not match the credential type schema. Verify the field names by retrieving the credential type schema:

GET /api/v1/credential_types/{credential_type_id}

Compare the inputs.fields[].id values in the response with the field names in your request.

Missing required fields:

{
  "detail": "Missing required field(s): token"
}

The inputs object is missing one or more fields listed in the credential type's inputs.required array. Add the missing fields to your request.

Invalid choice value:

{
  "detail": "Invalid value 'certificate' for field 'auth_method'. Must be one of: basic, token"
}

The request includes a value that is not in the allowed list for that field. This error occurs when a credential type defines a choices array on a field. Check the credential type schema for the accepted values:

GET /api/v1/credential_types/{credential_type_id}

Check the inputs.fields[].choices values in the response for the affected field.

$encrypted$ sentinel used as input:

{
  "detail": "The '$encrypted$' sentinel value cannot be used as input for field(s): token"
}

The reserved $encrypted$ value was submitted as a field value when creating a credential. This reserved value is only valid in PATCH requests, where it signals that the existing encrypted value is preserved. Use the actual credential value instead.

Payload exceeds maximum size:

{
  "detail": "Inputs payload exceeds maximum size of 65536 bytes (70000 bytes)"
}

The serialized inputs object exceeds 64 KiB. Reduce the size of the credential input values.

Resolve a credential name conflict (409)

Error code: CREDENTIAL_NAME_CONFLICT

Symptom: A 409 Conflict response when creating or updating a credential.

{
  "detail": "Credential with name 'Production API Token' already exists"
}

Cause: A credential with the same name already exists. Credential names must be unique.

Solution: To resolve this error:

• Choose a different name for the credential.
• Delete the existing credential if it is no longer needed, then create the new credential.

Resolve a credential decryption error (500)

Error code: CREDENTIAL_DECRYPTION_ERROR

Symptom: A 500 Internal Server Error response when viewing or using a credential.

{
  "detail": "An error occurred while processing credential data"
}

The error detail is generic to protect encryption internals. Check the server logs for the full error.

Possible causes:

Incorrect or missing encryption key:

The credential encryption key is not configured, has changed since the credential was created, or the Kubernetes Secret contains an invalid value.

To diagnose the issue:

1. Check that the encryption key is mounted in the automation orchestrator pod:

   oc exec -n namespace deployment/cr-name-backend -c backend -- env | grep APP_SECRET_ENCRYPTION

   Copy code to clipboard
2. Verify the key is a valid 64-character hexadecimal string.
3. If the key was rotated or changed, credentials encrypted with the previous key cannot be decrypted. Create the affected credentials again with the current key.

Corrupted encrypted data:

The encrypted data in the database has been modified or corrupted. The AES-256-GCM authentication tag verification fails when the ciphertext has been tampered with.

To diagnose, check the server logs for messages containing "Credential decryption error" for the specific credential ID and field name. Create the credential again if the stored data is unrecoverable.

Resolve a credential not found error (404)

Error code: CREDENTIAL_NOT_FOUND

Symptom: A 404 Not Found response when accessing a credential by ID.

{
  "detail": "Credential not found"
}

Possible causes:

• The credential ID is incorrect or has a typo.
• The credential was deleted.
• The credential was created in a different project or organization.

Solution: List all credentials to verify the ID:

GET /api/v1/credentials?name=expected_name

Resolve a credential authorization error (403)

Error code: AUTHORIZATION_DENIED

Symptom: A 403 Forbidden response when creating, reading, updating, or deleting a credential.

{
  "type": "https://api.example.com/errors/forbidden",
  "title": "Authorization Denied",
  "detail": "Not authorized to perform create on credential",
  "code": "AUTHORIZATION_DENIED",
  "retryable": false
}

Cause: The authenticated user does not have the required role-based access control (RBAC) permission for the requested operation. Credential endpoints enforce permission checks for read, create, update, and delete actions. A user who lacks the appropriate role assignment for the target project receives this error.

Solution: Contact your platform administrator to verify your role assignments. The administrator can grant the required permission through the RBAC configuration.

For more information about role assignments and access control, see Assign a role and Troubleshoot authorization.

Diagnosing the denial: Authorization denial events are emitted as SECURITY_EVENT audit events with action authorization_denied. These events are forwarded to your configured OpenTelemetry (OTEL) backend. To investigate immediately, check the automation orchestrator pod logs for a structured log entry containing Authorization denied with the user_id, action, and denied_by fields:

oc logs pod-name | grep "Authorization denied"

Resolve a disabled credential error during workflow execution (422)

Error code: CREDENTIAL_DISABLED

Symptom: A workflow node fails with the following error during execution:

Credential 'Production API Token' is disabled. Re-enable it before running workflows.

Cause: The credential attached to the workflow node has been disabled. Disabled credentials cannot be used for workflow execution.

Solution:

1. Navigate to Configuration > Credentials in the automation orchestrator UI.
2. Find the disabled credential.
3. Use the toggle to re-enable it.
4. Re-run the workflow.

Alternatively, update the workflow node to use a different, enabled credential.

Resolve workflow credential resolution failures

The following errors occur during workflow execution when automation orchestrator attempts to resolve credentials for a node. These errors are nonretryable, meaning the workflow node fails immediately without automatic retry.

Credential resolution errors appear in the automation orchestrator UI in two places:

• Execution-level error banner: An error banner labeled "Execution failed" at the top of the execution details panel, displaying the error message.
• Activity-level inline error: An error message displayed below the failed activity in the execution activity table.

Credential not found:

Credential 'credential_id' not found

The credential referenced by the workflow node no longer exists. Edit the workflow node and attach a valid credential.

Credential belongs to a different project:

Credential 'credential_id' does not belong to workflow project

The credential was created in a different project than the workflow. Attach a credential that belongs to the same project as the workflow.

No stored secret data:

Credential 'name' has no stored secret data

The credential exists but has no associated encrypted data. This can occur if the storage backend experienced an error during credential creation. Delete the credential and create it again.

Credential type not found:

Credential type for credential 'name' not found

The credential type associated with the credential no longer exists. This condition indicates a data integrity issue. Contact your platform administrator.

Failed to decrypt credential:

Failed to decrypt credential 'name'

The credential values could not be decrypted. Check the encryption key configuration and verify that the key has not changed since the credential was created. If the key was rotated, create the affected credentials again with the current key.

Failed to resolve injector templates:

Failed to resolve injector templates for credential 'name'

The injector template resolution failed when substituting {{field_id}} placeholders with decrypted values. This can occur if the credential type schema was modified after the credential was created. Verify that the credential type schema matches the stored credential inputs.

User not authorized to use credential:

User user_id is not authorized to use credential credential_id

The user attempting to use the credential does not own it. Users can use only credentials that they created. Contact your platform administrator to create a shared credential or have the credential owner share access.

Database error during credential resolution:

Database error during credential resolution: exception_type

A database connectivity issue occurred while resolving credentials. Check the PostgreSQL database status and the automation orchestrator pod logs for connection errors.

Resolve key rotation failures

The key rotation command-line interface (CLI) tool (python -m syntara.credentials.cli rotate-keys) re-encrypts all stored credentials when rotating the encryption key. The tool exits with one of three codes:

Diagnosing a partial failure (exit code 1):

The tool logs a summary at completion:

Key rotation complete total=150 rotated=148 skipped=0 failed=2 last_processed_id=uuid
Some secrets failed to rotate — re-run after fixing issues failed=2

Individual failures are logged with the secret ID and error:

Failed to rotate secret secret_id=uuid error=error message

Common causes of individual rotation failures:

• Corrupted ciphertext: The encrypted data for a specific credential field was modified or corrupted in the database. The AES-256-GCM authentication tag check fails on decryption.
• Incorrect old key: The --old-key value does not match the key used to encrypt this credential. This can occur if credentials were created with different keys across restarts.
• Database write error: A transient database issue prevented the re-encrypted value from being committed.

Recovery steps:

1. Check the tool output for the secret_id values that failed.
2. Query the database to identify which credentials are associated with the failed secret IDs.
3. If the failure is due to corrupted data or a key mismatch, delete the affected credentials.
4. Create the credentials again after the rotation completes.
5. To verify which credentials were successfully rotated, run the tool in dry-run mode with the new key as the old key:

   python -m syntara.credentials.cli rotate-keys \
     --old-key new-key \
     --new-key new-key \
     --dry-run

   Copy code to clipboard

   Credentials that decrypt successfully with the new key were already rotated. Credentials that fail to decrypt still need attention.

6. After resolving all failures, update the Kubernetes Secret referenced by spec.secrets.secretEncryptionKeySecretRef with the new key value. The operator detects the change and triggers a rolling restart of the affected pods automatically.

Diagnosing a fatal error (exit code 2):

A fatal error means the tool could not connect to the database or encountered a configuration error before processing any credentials. Check the following:

• Database connectivity and credentials in the application configuration.
• That the --old-key and --new-key values are valid 64-character hexadecimal strings.