The Linux perf tool is a lightweight command-line utility for profiling and monitoring CPU performance on Linux systems. Although the tool is simple, it provides in-depth information that helps in analyzing CPUs.
The command contains many subcommands for collecting, tracing, and analyzing CPU event data.
This guide demonstrates the perf tool through examples.
- Access to the terminal.
- Access to root or a user with sudo privileges.
- A text editor, such as nano or Vi/Vim.
How to Install perf
The perf program does not come preinstalled on Linux systems. The installation differs based on the Linux distribution:
sudo apt install linux-tools-$(uname -r) linux-tools-generic
sudo yum install perf
sudo dnf install perf
Verify the installation with:
The output prints the version number, indicating the installation was successful.
Allow Regular Users to Use perf
perf command, by default, requires
sudo privileges. To allow regular users to use
perf, do the following:
1. Switch to the root user:
sudo su -
2. Enter the following command:
echo 0 > /proc/sys/kernel/perf_event_paranoid
The command permits regular users to use the perf utility in the current session.
3. Switch back to the regular user with:
To persist the changes, do the following:
1. Edit the sysctl config file:
sudo nano /etc/sysctl.conf
2. Add the following to the file:
kernel.perf_event_paranoid = 0
3. Save the changes and exit nano.
Linux perf Command Syntax
perf command syntax is:
perf <options> subcommand <options/arguments>
The perf tool works like git. It acts as an interface for various subcommands and different activities. Running the command without any options or arguments shows the list of available subcommands.
Linux perf Subcommands
Below is a table outlining commonly used
|Reads perf.data and shows annotated code.|
|Lists all measurable events.|
|Gathers performance statistics.|
|Records samples into perf.data.|
|Reads perf.data and displays the profile.|
|Reads perf.data and displays trace output.|
Subcommands have additional options. To display options for individual subcommands, run:
perf <subcommand> -h
The output displays a brief help window for the specific subcommand.
Linux perf Command Examples
To test the
perf command, we're using phoenixNAP's s0.d1 small BMC instance running Ubuntu 18.04.
Note: For other use cases of the small BMC instance, check out some of our guides:
When profiling a CPU with the
perf command, the typical workflow is to use:
perf list to find events.
perf stat to count the events.
perf record to write events to a file.
perf report to browse the recorded file.
perf script to dump events after processing.
The outputs differ based on the system and locally available resources.
1. List the Available Events
List all measurable events using perf with the
sudo perf list
The output lists all supported events, regardless of type. Without
sudo, the command shows a shorter list. When referencing events in other subcommands, use the
-e tag followed by the event name from the first column.
Add the filter parameter after the command to limit the list by event name (first column) or event type (second column). For example, display only hardware events with:
sudo perf list hardware
sudo perf list hw
The output shows the filtered result based on the provided parameter or parameters.
2. View CPU Real-Time System Profile
To view the CPU profile in real-time, use the
sudo perf top
The command displays sampled functions in real time, similar to the Linux top command. The output prints the following three columns in order from left to right:
1. CPU usage tied to a function expressed in percentages.
2. The library or program that is using the function.
3. The symbol and function name, where
[k] is kernel space and
[.] is user space.
perf top monitors all online CPUs. Additional options allow:
- Monitoring all CPUs (including idle) (
- Monitoring specific CPUs (
- Controlling the sampling frequency (
To display additional options during browsing mode, press h.
To exit the profiler and return to the terminal, press q.
3. View CPU Performance Statistics with perf
To display CPU performance statistics for all standard CPU-wide hardware and software events, run:
sudo perf stat -a sleep 5
The output shows a detailed report for the entire system collected over five seconds. Without
sleep 5, the system measures until termination with CTRL+C.
4. View CPU Performance for a Command
To check CPU performance statistics for a specific command, run:
sudo perf stat <command>
For example, check for the
ls command with:
sudo perf stat ls
The total time a command takes shows as time elapsed at the end of the output.
5. View CPU Performance for a Process
Attach CPU performance statistics to a specific running process by using the
-p tag and providing the process ID (PID):
sudo perf -p <PID> sleep 5
The output collects and displays performance statistics for the given process.
6. Count Event System Calls by Type
To count Linux kernel system calls by type, run:
sudo perf stat -e 'syscalls:sys_enter_*' -a sleep 5
After five seconds, the output displays all system-wide calls and their count.
7. Record CPU Cycles
CPU cycles are a hardware event. To record CPU cycles, use the
record subcommand and provide the event name with the
sudo perf record -e cycles sleep 10
The recording saves the data into a perf.data file. The output prints the file's size and how many data samples it contains.
8. View Performance Results
To view the performance results from the perf.data file, run:
sudo perf report
The command helps read the perf.data file, displaying all the collected events and statistics. To exit the viewer, press CTRL+C.
9. Modify Sample Output Format
To view the sample output in standard output format, run:
sudo perf report --stdio
Additional modifications include displaying the sample number for each event (
-n), and displaying specific columns (
--sort <column name>). For example:
sudo perf report -n --sort comm,symbol --stdio
The output adds a column for the sample number and the command and symbol information.
10. Display Trace Output
script subcommand to list all events from perf.data. For example:
sudo perf script
The output prints the perf.data details in time order. Use the
script subcommand as post-processing data.
11. Display Trace Header
To display all the events from perf.data with additional trace header information, run:
sudo perf script --header
The output shows the file's header information, such as when the trace started, how long it lasted, CPU info, and the command that fetched the data. The events list is after the header information.
12. Dump Raw Data
To dump raw data as hex from the perf.data file, use the
sudo perf script -D
The result is the raw event trace information in ASCII format. The option is helpful for event debugging.
13. Annotate Data
To annotate data and further disassemble, use the
sudo perf annotate --stdio -v
-v option provides a detailed output. The result shows the source code and disassembly of the events.
After going through the examples in this guide, you know the basics of using the Linux
perf command and some of the main subcommands. Use the man command to view the complete documentation of the performance analysis tool and the subcommands.