Performance Co-Pilot (PCP) is a compact assessment and evaluation tool that is used to collect a wide array of server metrics and examines current and prior operational data. It is designed to be easily customizable utilizing modern APIs and other resources used to extend its functionality. Redhat has taken many of the best features of DStat and coupled them with additional capabilities that extended its usefulness, and have created their own updated version called Performance Co-Pilot.
PCP (or Performance Co-Pilot) now differentiates between the displayed metrics from the data collected via a PMAPI (or a performance metrics API). This feature lets programmers write unique functions to collect data once and then reuse that same operation across different requests. This allows for improved reporting by abstracting the returned data via calls using the PMAPI instead of the visualized sources directly.
PCP pulls much of the same information as top, sar, iotop, iostat, iftop, vmstat, netstat, and ifstat. Unfortunately, many of these tools do not work well together, but PCP solves that issue. Its plugin framework also allows for the collection of performance data from a single server. Additionally, PCP uses a client-server architecture that allows it to monitor a single host from multiple clients or multiple hosts from a single client.
Installation
Let’s begin by installing PCP.
Redhat/CentOS
[root@host ~]# yum install -y pcp pcp-doc pcp-gui pcp-system-tools
Debian/Ubuntu
[root@host ~]# apt-get install pcp pcp-do pcp-gui pcp-system-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libpcp-gui2 libpcp-import1 libpcp-mmv1 libpcp-pmda-perl libpcp-pmda3 libpcp-trace2 libpcp-web1 libpcp3 libpfm4 libqt5printsupport5 pcp-conf python3-pcp
Suggested packages:
libpcp-import-perl pcp-doc
The following NEW packages will be installed:
libpcp-gui2 libpcp-import1 libpcp-mmv1 libpcp-pmda-perl libpcp-pmda3 libpcp-trace2 libpcp-web1 libpcp3 libpfm4 libqt5printsupport5 pcp pcp-conf pcp-gui python3-pcp
0 upgraded, 14 newly installed, 0 to remove and 30 not upgraded.
Need to get 4,648 kB of archives.
After this operation, 18.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
…
…
…
Setting up python3-pcp (4.3.4-1build1) ...
Setting up pcp (4.3.4-1build1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/pmcd.service → /lib/systemd/system/pmcd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/pmlogger.service → /lib/systemd/system/pmlogger.service.
Created symlink /etc/systemd/system/multi-user.target.wants/pmie.service → /lib/systemd/system/pmie.service.
Created symlink /etc/systemd/system/multi-user.target.wants/pmproxy.service → /lib/systemd/system/pmproxy.service.
Processing triggers for libc-bin (2.30-0ubuntu2) ...
Processing triggers for systemd (242-7ubuntu3) ...
Processing triggers for man-db (2.8.7-3) ...
Processing triggers for desktop-file-utils (0.24-1ubuntu1) ...
Processing triggers for mime-support (3.63ubuntu1) ...
Processing triggers for gnome-menus (3.32.0-1ubuntu1) ...
[root@host ~]#
Git
[root@host ~]# git clone https://github.com/performancecopilot/pcp.git
cd pcp
[root@host ~]# ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var
[root@host ~]# make
[root@host ~]# make install
Enable The Service and Start The Daemons
[root@host ~]# systemctl enable pmcd
Created symlink from /etc/systemd/system/multi-user.target.wants/pmcd.service to /usr/lib/systemd/system/pmcd.service.
[root@host ~]#
[root@host ~]# systemctl enable pmlogger
Created symlink from /etc/systemd/system/multi-user.target.wants/pmlogger.service to /usr/lib/systemd/system/pmlogger.service.
[root@host ~]#
[root@host ~]# systemctl start pmcd
[root@host ~]# systemctl start pmlogger
Examples
System Info - Stats
[root@host ~]# pcp
Performance Co-Pilot configuration on host.bare-centos7.com:
platform: Linux host.test-centos7.com 3.10.0-957.27.2.el7.x86_64 #1 SMP Mon Jul 29 17:46:05 UTC 2019 x86_64
hardware: 2 cpus, 1 disk, 1 node, 1871MB RAM
timezone: EST+5
services: pmcd
pmcd: Version 4.3.2-1, 9 agents, 1 client
pmda: root pmcd proc pmproxy xfs linux mmv kvm jbd2
pmlogger: primary logger: /var/log/pcp/pmlogger/host.test-centos7.com/20191216.08.50
(Use Ctrl+C to exit)
PCP-Atop

(Use Ctrl+C to exit)
pmiostat
[root@host ~]# pmiostat -t 2sec
# Device rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await %util
sda 0.00 20.46 0.00 5.99 0.00 97.82 16.333 0.040 6.75 0.00 6.75 1.40
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
sdc 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
sda 0.00 12.96 0.00 5.98 0.00 67.81 11.333 0.034 5.75 0.00 5.75 1.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
sdc 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
sda 0.00 22.92 0.00 7.48 0.00 143.52 19.200 0.043 5.80 0.00 5.80 1.79
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
sdc 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
sda 0.00 18.44 0.00 5.98 0.00 89.70 15.000 0.034 5.75 0.00 5.75 1.79
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
sdc 0.00 0.00 0.00 0.00 0.00 0.00 0.000 0.000 0.00 0.00 0.00 0.00
[root@host ~]#
pmstat
[root@host ~]# pmstat
@ Mon Dec 16 09:36:05 2019
loadavg memory swap io system cpu
1 min swpd free buff cache pi po bi bo in cs us sy id
0.57 120320 1616m 1395m 3826m 0 0 0 281 1005 1536 2 1 98
0.52 120320 1612m 1395m 3826m 0 0 0 746 938 1362 2 0 98
0.48 120320 1556m 1395m 3859m 0 0 0 361 4757 7986 19 3 78
0.44 120320 1563m 1395m 3840m 0 0 0 343 4139 6424 16 3 81
0.57 120320 1541m 1395m 3853m 0 0 0 471 5071 8033 23 4 73
0.52 120320 1555m 1395m 3846m 0 0 0 494 4384 7030 12 4 84
[root@host ~]#
pcp dstat
The pcp dstat command is similar in nature to the older versions of dstat. It uses the options below:
pcp [ pcp options ] dstat [ -acdfghilmnpqrstvVy? ] [ -C cpus ] [ -D disks ] [ -I interrupts ] [ -N interfaces ] [ -o output-file ] [ -S swap-devices ] [ --bits ] [ --bw ] [ --color ] [ --float ] [ --integer ] [ --nocolor ] [ --noheaders ] [ --noupdate ] [ --list ] [ --pidfile pid-file ] [ -- plugin ] [ --all-plugins ] [ delay [ count ]]
The default output of the pcp dstat command looks like this.
[root@host ~]# pcp dstat
You did not select any stats, using -cdngy by default.
----total-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read writ| recv send| in out | int csw
3 1 95 1 0| 0 244k| 972 1623 | 0 0 |1519 2235
3 1 96 0 0| 0 0 | 414 264 | 0 0 |1074 1393
2 0 97 1 0| 0 136k| 189 189 | 0 0 |1103 1640
2 1 96 0 0| 0 0 |6928 4328 | 0 0 |1341 1758
4 1 94 1 0| 0 176k| 444 700 | 0 0 |1496 2065
2 1 97 0 0| 0 12k| 209 275 | 0 0 | 999 1360
3 1 96 1 0| 0 120k|2375 2423 | 0 0 |1386 1973
2 1 97 307 0| 0 8190B| 0 0 | 0 0 |1106 1532
3 1 94 2 0| 0 900k| 358 547 | 0 0 |1492 2110
2 1 96 0 0| 0 0 | 531 1695 | 0 0 |1267 1882
7 7 85 1 0| 0 296k| 190 178 | 0 0 |3634 5890
A full view of nearly all the metrics literally spans two widescreen monitors.

Performance Metrics Domain Agents (PMDAs)
The default installation of PCP provides the metrics for your cpu, file system, per-process, network, swap, memory, disk, interrupts, nfs/rpc and others. These metrics are processed using the platform PMDA - specifically the pmda_linux.so file in Linux. There are also corresponding pmda files for Mac and Windows.
PMDAs in the pcp package includes:
- apache - monitor apache web server stats
- cisco - monitor Cisco router stats
- dbping - query any database, extract response times
- elasticsearch - monitor an elasticsearch cluster
- kvm - monitor kernel-based virtual machine stats
- mailq - monitor the mail queue
- memcache - monitor memcache server stats
- mmv - export memory-mapped value stats from an application
- mounts - keep track of mounted file systems
- mysql - monitor MySQL relational databases
- oracle - monitor Oracle relational databases
- postgres - monitor PostGreSQL relational databases
- process - keep an eye on critical processes/daemons
- roomtemp - monitor room temp (needs suitable probe)
- rsyslog - monitor the reliable system log daemon
- sendmail - monitor sendmail statistics
- shping - ping critical system services, extract response times
- trace - for instrumenting arbitrary applications, see pmtrace(1)
- txmon - transaction and QOS monitoring
- statsd - StatsD protocol data collector
- sample - for testing
- simple - example src code if you want to write a new PMDA
- trivial - even easier src code for a new PMDA.
None of the PMDAs are setup by default. You select one of the many PMDAs needed and then run the install script found in the corresponding /var/lib/pcp/pmdas folders.
pmchart

Open View

Default SCSI Disk Traffic View

Default Live CPU and Load Average View

Start Collecting Info
To enable historical data collection, use the following commands:
[root@host ~]# systemctl enable pmlogger
[root@host ~]# systemctl startpmlogger
The configuration for pmlogger can be found in /etc/pcp/pmlogger. The default interval for collecting data is 1 minute, and archives are located in /var/log/pcp/pmlogger/hostname
.
Plugins
Plugins were a huge part of the original dstat, and in carrying on this vital work, Redhat has implemented this feature via the use of its PMAPI functionality. Now, all of the PCP dstat metrics that are displayed are from 'plugins' defined and outlined in config files. This provides an even greater advantage by making it easier to produce and build new plugins for PCP's dstat.
Exported Info
Previously, dstat only allowed the export of data to a CSV file. With dstat now being integrated into PCP, dstat now has the ability to export collected data into the following formats:
- pcp2elasticsearch
- pcp2graphite
- pcp2influxdb
- pcp2json
- pcp2spark
- pcp2xlsx
- pcp2xml
- pcp2zabbix
With the integration of dstat and PCP, the added benefit of utilizing PCP’s own 20-year-old archive format is the increased usability and functionality of the 'pmlogger' program which now runs as a service. When a PCP client tool uses the PMAPI (as this dstat implementation does), it also gains access to metrics stored in archives without having to implement the functionality itself. This means even the brand-new dstat implementation can display previously gathered values as if they were live, using PCP standard flags to specify the timeframe of metrics to display.
Graphical Output
PCP graphical charts work like a native client program that presents data that has been collected from localhost, a remote server, or from archived files. Multiple report types are presented, including the “Overview Report,” which provides a very nice console type view of the system. The graphics in the overview are very functional and easy to use for investigation.
Custom Widgets
You can implement the GUI feature in PCP by modifying the “enableCustomWidgetFeature” setting to true in the /src/app/config.js file and then rebuilding. Next, if you open the Dashboard dropdown, and then select “Custom Chart” or “Custom Table” from under the Custom section, this will generate a brand-new chart panel to your dash.
Commands
Here is a shortlist of some of the commands available for use with PCP. To see a full listing of all of the available options visit the Performance Co-Pilot home page.
[root@host ~]# man PCPIntro
[root@host ~]# man pcp
[root@host ~]# man pmrep
root@host ~]# man pmcd
root@host ~]# man pminfo
root@host ~]# man pmie
See It In Action Today!
Do you have a Liquid Web Dedicated server, Private Cloud or a Liquid Web Server Cluster that requires constant monitoring? Reach out to one of our experienced Linux System Administrators today to get more information.
Related Articles:
- ChatGPT Integration — How to Create a Plugin for ChatGPT
- Stable Diffusion AI Image Generator (SDXL) — Using the Web UI
- How to Install VMware Tools on Ubuntu: Step-by-Step Guide
- How to Install WordPress on Linux (AlmaLinux)
- What is CentOS? Everything You Need to Know
- Virtual Desktop Environment — Configuring Kasm Workspaces
About the Author: David Singer
I am a g33k, Linux blogger, developer, student, and former Tech Writer for Liquidweb.com. My passion for all things tech drives my hunt for all the coolz. I often need a vacation after I get back from vacation....
Our Sales and Support teams are available 24 hours by phone or e-mail to assist.
Latest Articles
How to Install WordPress on Linux (AlmaLinux)
Read ArticleWhat is CentOS? Everything You Need to Know
Read ArticleWhat is CentOS? Everything You Need to Know
Read ArticleRedis as Cache: How It Works and Why You Should Use It
Read ArticleRefer-a-Friend Program for Website Hosting: Get $100 for Each Friend!
Read Article