How is Performance Co-Pilot (PCP) designed and structured?

Updated 3 Nov 2016

Performance Co-Pilot (PCP) is a tool which can be used to collect, view, and react to performance statistics.

PCP is built around the following individual components:

Metrics

A Performance Metric is any individual statistic able to be measured. There will usually be many metrics of interest. Some examples of metrics are: number of processes running, amount of free memory, errors on a network interface, CPU usage, amount of bytes written to a disk.

PMDA

A Performance Metric Domain Agent actually reads the performance statistic in question. There will usually be multiple PMDAs collecting multiple stats per PMDA. Some examples of PMDAs are: linux, nfsclient, apache, kvm, mysql, postfix.

PMCD

The Performance Metrics Collector Daemon is a process which runs and uses PMDAs to collect statistics. A pmcd instance allows other tools to connect to it for the purposes of reading statistics which pmcd can collect. There is usually one pmcd per system.

PMLogger

The Performance Metric Logger connects to a pmcd and gathers stats at pre-defined intervals. pmlogger then writes those stats to a PCP archive file to allow retrospective performance analysis. Each system can run its own pmlogger, providing local performance logging. Alternatively, a single system running multiple pmloggers can connect to the pmcd of multiple remote systems via network, providing centralised performance logging.

PMIE

The Performance Metric Inference Engine connects to one or more pmcd instances and can execute rules based on conditions at set intervals. For example, pmie could check the temperature of multiple systems every minute, and run a command to alert administrators if the temperature is above a certain value.

Tools

PCP provides commandline and graphical tools to allow real-time and retrospective performance analysis.

Commandline tools can dump raw metrics (pmval), or can display metrics in a structured fashion (pmrep), or can emulate existing well-known performance tools (pmiostat). Graphical tools can create graphs (pmchart).

The tools can usually connect to a live PMCD, either local or remote, or can replay statistics collected in a PCP log file.

The tools can replay at the original (usually small) collection interval, or can interpolate and display at a larger collection interval. If metrics were originally collected every second, a tool could display an hourly value, or every 10 minutes, or every minute, or every second. This allows both an overview of metrics, and "drilling down" to closer inspect time periods of interest in great detail.