Understand metrics service architecture

Metrics service overview

Metrics service is a Django-based application component of Ansible Automation Platform 2.7. It collects anonymized usage and performance data from your automation controller. The service runs as containerized workloads managed by either systemd (containerized installer) or Kubernetes (operator). It maintains its own PostgreSQL database and read-only access to the controller database.

Default behavior in Ansible Automation Platform 2.7

Metrics service is enabled by default in Ansible Automation Platform 2.7 for both operator-based (OpenShift) and containerized installations that include automation controller. It is deployed automatically as a required component when you install automation controller.

• Containerized installations: [automationmetrics] inventory group is required when installing Ansible Automation Platform if automationcontroller is present in the inventory file, unless automationmetrics_skip_install=true is set in [all:vars]
• Operator installations: Enabled when metrics.disabled: false is set in the AnsibleAutomationPlatform custom resource. Operator deployments do not support automationmetrics_skip_install; they use metrics.disabled: true in the CR instead.

Key characteristics

• Django-based application: Built on Django framework with database migrations managed through /app/scripts/init.sh
• Containerized deployment: Runs as three containers (automation-metrics-web, automation-metrics-tasks, automation-metrics-scheduler) managed by systemd (containerized installer) or Kubernetes (operator)
• Optional deployment: Set automationmetrics_skip_install=true to opt out of metrics service installation when automation controller is present (containerized installations only)
• Automatic anonymization: Customer-defined object names are replaced with "Custom" before transmission. Only built-in object names and publicly available collection names are sent in their original form
• Auto-configured ingress: Establishes secure connection to Red Hat Data Ingress without manual setup
• Read-only access: Accesses automation controller database in read-only mode to prevent operational impact
• Isolated database: Maintains its own metrics_service database for metrics processing and storage
• Health monitoring: Provides health check endpoint (returns "status": "good" when healthy)
• HTTP proxy support: Supports HTTP/HTTPS proxy configuration for environments without direct internet access
• Startup validation: Validates required production environment variables at startup. Containers exit immediately with validation errors if required variables are missing or invalid
• Deployment flexibility: Supports both RHEL-based containerized deployments and OpenShift operator-based deployments with different configuration approaches
• Date filtering: Users can only use predefined date range options (Last 7 days, Last 30 days, Last 90 days, etc.)
• Currency: All cost values display in USD. Multi-currency support not available.

Capabilities unavailable without metrics service

• Automation dashboard has no backend data source and is non-functional
• No anonymized usage data is collected or transmitted
• ROI calculations, automation savings, and executive reporting are unavailable
• Dashboard collection tasks do not run

Relationship to automation dashboard

Metrics service serves as the unified backend for automation dashboard. Metrics service collects additional dashboard-specific metrics, including:

• Cost and pricing data for ROI calculations
• Automation savings metrics
• Executive reporting data

Dashboard collection is enabled by default when metrics service is installed. Dashboard collection runs on an hourly schedule..

Architecture components

Core components

Database architecture

Database schema architecture

Collection phase

1. Metrics service queries automation controller/awx database (read-only connection using ms_awx_readonly user)
2. Raw usage data extracted from the following tables: jobs, job host summaries, credentials, execution environments, controller versions, feature flags, and job events

What is NOT collected:

• Host inventory data
• Customer-defined template names, playbook names, or workflow names

Complete task schedule

Metrics service uses three task groups controlled by feature flags:

METRICS_COLLECTION group

Gated by METRICS_COLLECTION feature flag (default: true). When disabled, all tasks in this group skip execution.

ANONYMIZATION group

Gated by ANONYMIZED_DATA_COLLECTION feature flag (default: true). When disabled, anonymization and transmission skip execution.

SYSTEM group

Always enabled regardless of feature flag state. System maintenance tasks continue even when all collection is disabled.

Task dependencies

Tasks are scheduled in dependency order:

1. Hourly collection (XX:05, XX:10, XX:15, XX:20) - provides raw data
2. Daily snapshots (1:00 AM - 1:50 AM) - supplements hourly data
3. Daily rollup (2:00 AM) - aggregates hourly data (depends on hourly collectors completing)
4. Anonymization (3:00 AM) - anonymizes aggregated data (depends on rollup completing)
5. Cleanup (4:00 AM, 5:00 AM) - purges old data (runs after rollup/anonymization to avoid race conditions)

Anonymization phase

No personal data leaves the customer environment. Only publicly available collection names (for example from Galaxy) and built-in object names (such as built-in credential types and execution environments) are transmitted.

Anonymization method: Statistics and counts only

Metrics service transmits only aggregated statistics, counts, and built-in/public object names. Customer-defined object names are never transmitted. For example, if you have 3 custom credentials, metrics service sends the count "3" but does not send the credential names.

What data is sent:

• Job statistics: Aggregated job execution data grouped by job type, launch type, and version (success/failure counts, duration statistics, timestamps). Job template names are NOT included.
• Job host summary statistics: Aggregated job execution results per host (success/failure counts, execution performance metrics).
• Credential type statistics: Built-in credential type names (e.g., "Machine", "Source Control", "Vault") and count of customer-defined credentials (e.g., "3 custom credentials"). Customer-defined credential names are NOT sent.
• Execution environment statistics: Built-in EE names (e.g., "ee-minimal-rhel9", "ee-supported-rhel9") and count of customer-defined EEs (e.g., "5 custom EEs"). Customer-defined EE names are NOT sent.
• Collection statistics: Public collection names from Galaxy or Red Hat Certified Collections (e.g., "ansible.builtin", "ansible.posix") and count of customer-defined collections. Customer-defined collection names are NOT sent.
• Collection, module, and role usage statistics: Agregated counts of Ansible collection, module, and role usage extracted from job events. Data includes module execution counts, job success/failure rates, task retry and unreachable events, and collection-level host counts. Only publicly available (Certified, Community, and Validated) collection and module names are included. Custom-source entries and customer-defined role names are stripped during anonymization. Skipped tasks are excluded - only events from automation work that was actually performed are counted.
• Project SCM types: Source control management types for projects (e.g., Git, SVN).
• Organization counts: Total number of organizations configured.
• Version information: Controller version, Ansible core version, installed collection versions.
• Warnings and deprecations: Statistics on deprecation warnings encountered.
• Feature flag states: Boolean status of enabled/disabled feature flags.

What data is NOT sent:

• Job template names (not collected at all)
• Customer-defined credential names (only counts transmitted, e.g., "3 custom credentials")
• Customer-defined execution environment names (only counts transmitted, e.g., "5 custom EEs")
• Customer-defined collection names (only counts transmitted)
• Host inventory data (not queried by metrics service; handled separately by metrics-utility CLI for CCSP reports only)
• Playbook names (customer-defined content)
• Variable names and values (customer-defined content)
• Workflow template names (customer-defined content)

Transmission phase

1. Anonymized data packaged for transmission
2. Secure HTTPS connection established to Red Hat Data Ingress endpoint using Segment (a third-party data pipeline service)
3. Data transmitted through the Gateway /api/metrics/ endpoint (no direct customer-facing endpoints in 2.7 GA)
4. Transmission status logged in metrics_service database
5. HTTP/HTTPS proxy supported for environments without direct internet access

Message size handling

Metrics service splits anonymized data deterministically into separate messages. If any message exceeds 32 KB, the service splits it further. This ensures reliable transmission of large datasets without message size errors.

Deployment topology options

AIO Growth (All-in-One) topology

A single host runs all Ansible Automation Platform components, including metrics service. The [automationmetrics] inventory group points to the same host as [automationcontroller].

Characteristics:

• All components share the same host
• Simplest deployment for small to medium environments
• Metrics service, controller, and databases on one node
• Suitable for development, testing, or small production environments

To skip metrics service installation, set automationmetrics_skip_install=true in [all:vars] and omit the [automationmetrics] inventory group.

Enterprise / Multi-Node topology

Metrics service runs on a dedicated node. You define which node hosts metrics service by using the [automationmetrics] inventory group. In the growth topology (all-in-one), metrics service shares the single host with all other components. In the enterprise topology, metrics service requires its own dedicated node. Unlike controller, hub, and EDA, metrics service does not support multi-node or high-availability deployment.

Characteristics:

• Flexible topology: metrics service can run on any node
• Scaled independently from controller
• Metrics processing isolated from automation execution
• Suitable for large-scale production environments

To skip metrics service installation, set automationmetrics_skip_install=true in [all:vars] and omit the [automationmetrics] inventory group.

Deployment method comparison

Metrics service supports two deployment methods with different configuration complexity and automation levels:

Containerized installer deployment details

Metrics service is deployed through the automationmetrics Ansible role included in the Ansible Automation Platform 2.7 containerized installation program:

• Container runtime: Podman with host networking mode (rootless, user namespace)
• Service management: Three systemd user-scope units: automation-metrics-web.service, automation-metrics-tasks.service, automation-metrics-scheduler.service
• Data persistence: Host volumes mounted from {{ aap_volumes_dir }}/automationmetrics into container path /var/lib/ansible-automation-platform/metrics
• Secrets management: Four Podman secrets: automationmetrics_pg_password, automationmetrics_controller_read_pg_password, automationmetrics_secret_key, automationmetrics_resource_server
• Firewall integration: Ports 8087/tcp (HTTP) and 8450/tcp (HTTPS) opened in configured firewall zone (default: public)

Configuration requirements:

• Hosts must be added to [automationmetrics] inventory group (unless automationmetrics_skip_install=true is set)
• Database connection variables must be configured (automationmetrics_pg_host, automationmetrics_pg_password, etc.)
• Read-only controller database user must be provisioned manually or by installer
• Segment write key must be provided for data transmission

Operator-based deployment details

Metrics service is deployed through the automation-metrics-operator running on OpenShift. The AAP gateway operator manages the metrics operator lifecycle:

• Container runtime: Kubernetes pods managed by operator deployments
• Service management: Three Kubernetes deployments: metrics-api, metrics-tasks, metrics-scheduler
• Data persistence: Kubernetes persistent volumes managed by operator
• Secrets management: Kubernetes secrets automatically created and managed by operator (database credentials, OAuth tokens, Segment write key)
• Network integration: Kubernetes services and Envoy routing through platform gateway
• Database provisioning: Operator automatically creates and configures both metrics_service database and read-only controller database user

Configuration simplicity:

• Single CR field: spec.metrics.disabled: false in AnsibleAutomationPlatform CR
• Zero manual database configuration required
• Zero manual network configuration required
• Automatic OAuth integration with platform gateway
• Operator handles all credential rotation and secret management

Operator-managed resources:

• MetricsService CR: Created as <aap-name>-metrics in same namespace as AnsibleAutomationPlatform CR
• Database: PostgreSQL database for metrics service, automatically provisioned with credentials stored in Kubernetes secrets
• Read-only user: Automatically created in controller database with SELECT privileges
• Services: Kubernetes Service resources for metrics API, tasks, and scheduler pods
• Routes/Ingress: Traffic routing through platform gateway Envoy proxy (no direct external endpoints)

Network architecture

Required network connectivity

Firewall requirements

• Protocol: HTTPS (TLS 1.2 or higher); supported TLS versions: TLSv1.2, TLSv1.3
• Direction: Outbound from metrics service container to Red Hat Data Ingress
• Nginx ports: 8087/tcp (HTTP) and 8450/tcp (HTTPS) opened in firewall (default zone: public)
• Internal Gunicorn port: 8006 (localhost only, not opened in firewall)
• Proxy support: HTTP/HTTPS proxy configuration supported via standard environment variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY)

Naming conventions