Background execution reference

Use these custom resource (CR) fields and metrics to configure and monitor background execution.

AutomationOrchestrator CR fields

The following fields in the AutomationOrchestrator CR control the background worker deployment. All fields are under spec.backgroundWorker.

Table 1. AutomationOrchestrator CR fields for background worker configuration
Field Type Default Description
resources.requests.cpu string 250m CPU request for each background worker pod.
resources.requests.memory string 256Mi Memory request for each background worker pod.
resources.limits.memory string 1Gi Memory limit for each background worker pod.
hpa.minReplicas integer 1 Minimum number of background worker pods. The Horizontal Pod Autoscaler (HPA) manages replica count exclusively.
hpa.maxReplicas integer 5 Maximum number of background worker pods the HPA scales to.
hpa.targetCPUUtilization integer 80 Target CPU utilization percentage that triggers scaling.

The operator does not set CPU limits by default.

Note:

Do not set spec.backgroundWorker.replicas directly. The operator ignores this field because the HPA manages replica count. Setting it emits a ReplicasIgnored warning event.

Activity concurrency cap

The operator sets APP_BACKGROUND_WORKER_MAX_CONCURRENT_ACTIVITIES=10 on the background worker deployment. This value is not configurable through the CR.

The cap limits the number of Temporal activities that can run concurrently on a single background worker pod. When the cap is reached, additional activities queue in the Temporal task queue and are processed in FIFO order as worker slots become available. Activities are not rejected or failed. If sustained load consistently exceeds per-pod capacity, the HPA adds replicas to distribute the work.

Note:
This cap is independent of max_cached_workflows, which controls Temporal SDK workflow history caching. When diagnosing out-of-memory restarts on background worker pods, tune the pod memory limit in spec.backgroundWorker.resources.limits.memory, not the history cache.

Prometheus metrics

The following metric is relevant to background execution monitoring.

Table 2. Prometheus metrics for background execution
Metric Type Labels Description
orchestrator_temporal_queue_depth gauge component,task_queue Number of pending tasks in a Temporal task queue. Filter by the background worker task queue to monitor load.

Use this metric to monitor background worker capacity and inform scaling decisions. A sustained increase in queue depth can indicate that your workload requires additional background workers.

Built-in workflow activity registry

Background workers register only the activities that built-in workflows require. This is a smaller set than the full activity registry that user workflow workers use.