Understand audit event delivery

You can receive audit events from automation orchestrator through pod standard output and, when configured, an OpenTelemetry Protocol (OTLP) compatible observability backend.

Events are always written to pod standard output. When OTLP forwarding is enabled, events are also forwarded to your configured observability backend.

Event delivery mechanism

Automation orchestrator uses an outbox pattern (a write-ahead technique that helps ensure events are not lost by writing them to a database table before emitting them) to maximize delivery reliability. When a platform operation occurs, the application writes an audit record to a transactional outbox table. This write shares the same database transaction as the business data change, to help prevent audit records from being lost if a write fails mid-operation. A background worker then polls the outbox table and emits each event to the audit logger.

Event origins

Each audit event carries an audit.event_source field that identifies how it was generated:

  • business_event — emitted by application-level instrumentation. Reflects platform activity: user actions, agent execution, and settings changes. Use this value for compliance and security monitoring.
  • crud_event — generated by database triggers that automatically record all create, update, and delete operations on audited tables. Use this value when you need a complete audit trail.

For most compliance use cases, filtering to business_event removes background database noise and produces the most actionable signal.

Outbox storage and failure behavior

The outbox table is not a long-term storage layer. After a batch of events is confirmed as successfully exported to the OTLP collector, the background worker deletes those records from the outbox table. Under normal operation, the outbox table remains small.

If the OTLP backend is unreachable, the worker retries export for each affected record. After exceeding the maximum export attempts threshold (controlled by audit_outbox_max_dispatch_attempts, default: 5), the record is permanently deleted and the following CRITICAL-level log entry is emitted:

Audit event permanently failed OTEL export, deleting from outbox

A sustained OTLP outage results in audit event loss, not unbounded table growth. Monitor your pod logs for CRITICAL-level entries to detect when events are being dropped. There are no CLI or API controls for manually purging the outbox.

To proactively monitor outbox health before events are dropped, track the audit_outbox row count using your cluster's PostgreSQL monitoring tooling. A persistently large or growing row count indicates that OTLP export is failing.