@brief Using command line perf and OpenCSD to collect and decode trace.
This HOWTO explains how to use the perf cmd line tools and the openCSD library to collect and extract program flow traces generated by the CoreSight IP blocks on a Linux system. The examples have been generated using an aarch64 Juno-r0 platform.
OpenCSD is developed independently of the linux kernel and perf, though the ABI between the library and the perf tool generally remains stable. This ABI is formed of a stream of decoded packets from the library to the tool.
The exception to this is when new ARM architecture results in significant changes to trace
elements and therefore to the decoder. This can then change the interface between the OpenCSD
library and client program i.e. perf, introducing new packet types. When this occurs, patches
for perf are submitted to the kernel at the same time as the decoder is updated.
The general advice is to always use the latest versions of OpenCSD and kernel/perf.
However the following points should be noted if this is not possible:
-
perf build enforces a minimum version of OpenCSD that it will compile against.
-
building perf from an older kernel against an OpenCSD version later than the minimum version will generally work, unless the version of OpenCSD has packet types that are not supported in the cs_etm_decoder handling code in perf. This may result in a build error such as enum types not supported.
-
The minimum version of kernel recommended is 5.16. However there is some limited support for older kernels at present in the form of backports at:-
https://gitlab.arm.com/linux-arm/linux-coresight-backports
See the auto-fdo documentation (@ref older_kernels) for more information on older kernels.
Compile the perf tool from the same kernel source code version you are using with:
make -C tools/perf
This will yield a perf executable that will support CoreSight trace collection.
Note: If traces are to be decompressed off target, there is no need to download and compile the openCSD library (on the target).
If you are instead planning to use perf to record and decode the trace on the target, compile the perf tool linking against the openCSD library, in the following way:
make -C tools/perf VF=1 CORESIGHT=1
Further information on the needed build environments and options are detailed later in the section Off Target Perf Tools Compilation.
Before launching a trace run a sink that will collect trace data needs to be identified. All CoreSight blocks identified by the framework are registed in sysFS:
linaro@linaro-nano:~$ ls /sys/bus/coresight/devices/
etm0 etm2 etm4 etm6 funnel0 funnel2 funnel4 stm0 tmc_etr0
etm1 etm3 etm5 etm7 funnel1 funnel3 replicator0 tmc_etf0
CoreSight blocks are listed in the device tree for a specific system and
discovered at boot time. Since tracers can be linked to more than one sink,
the sink that will recieve trace data needs to be identified and given as an
option on the perf command line. Once a sink has been identify trace collection
can start. An easy and yet interesting example is the uname command:
linaro@linaro-nano:~/kernel$ ./tools/perf/perf record -e cs_etm/@tmc_etr0/ --per-thread uname
This will generate a perf.data file where execution has been traced for both
user and kernel space. To narrow the field to either user or kernel space the
u and k options can be specified. For example the following will limit
traces to user space:
linaro@linaro-nano:~/kernel$ ./tools/perf/perf record -vvv -e cs_etm/@tmc_etr0/u --per-thread uname
Problems setting modules path maps, continuing anyway...
-----------------------------------------------------------
perf_event_attr:
type 8
size 112
{ sample_period, sample_freq } 1
sample_type IP|TID|IDENTIFIER
read_format ID
disabled 1
exclude_kernel 1
exclude_hv 1
enable_on_exec 1
sample_id_all 1
------------------------------------------------------------
sys_perf_event_open: pid 11375 cpu -1 group_fd -1 flags 0x8
------------------------------------------------------------
perf_event_attr:
type 1
size 112
config 0x9
{ sample_period, sample_freq } 1
sample_type IP|TID|IDENTIFIER
read_format ID
disabled 1
exclude_kernel 1
exclude_hv 1
mmap 1
comm 1
enable_on_exec 1
task 1
sample_id_all 1
mmap2 1
comm_exec 1
------------------------------------------------------------
sys_perf_event_open: pid 11375 cpu -1 group_fd -1 flags 0x8
mmap size 266240B
AUX area mmap length 131072
perf event ring buffer mmapped per thread
Synthesizing auxtrace information
Linux
auxtrace idx 0 old 0 head 0x11ea0 diff 0x11ea0
[ perf record: Woken up 1 times to write data ]
overlapping maps:
7f99daf000-7f99db0000 0 [vdso]
7f99d84000-7f99db3000 0 /lib/aarch64-linux-gnu/ld-2.21.so
7f99d84000-7f99daf000 0 /lib/aarch64-linux-gnu/ld-2.21.so
7f99db0000-7f99db3000 0 /lib/aarch64-linux-gnu/ld-2.21.so
failed to write feature 8
failed to write feature 9
failed to write feature 14
[ perf record: Captured and wrote 0.072 MB perf.data ]
linaro@linaro-nano:~/kernel$ ls -l ~/.debug/ perf.data
_-rw------- 1 linaro linaro 77888 Mar 2 20:41 perf.data
/home/linaro/.debug/:
total 16
drwxr-xr-x 2 linaro linaro 4096 Mar 2 20:40 [kernel.kallsyms]
drwxr-xr-x 2 linaro linaro 4096 Mar 2 20:40 [vdso]
drwxr-xr-x 3 linaro linaro 4096 Mar 2 20:40 bin
drwxr-xr-x 3 linaro linaro 4096 Mar 2 20:40 lib
The amount of traces generated by CoreSight tracers is staggering, event for the most simple trace scenario. Reducing trace generation to specific areas of interest is desirable to save trace buffer space and avoid getting lost in the trace data that isn't relevant. Supplementing the 'k' and 'u' options described above is the notion of address filters.
On CoreSight two types of address filter have been implemented - address range and start/stop filter:
Address range filters: With address range filters traces are generated if the instruction pointer falls within the specified range. Any work done by the CPU outside of that range will not be traced. Address range filters can be specified for both user and kernel space session:
perf record -e cs_etm/@tmc_etr0/k --filter 'filter 0xffffff8008562d0c/0x48' --per-thread uname
perf record -e cs_etm/@tmc_etr0/u --filter 'filter 0x72c/0x40@/opt/lib/libcstest.so.1.0' --per-thread ./main
When dealing with kernel space trace addresses are typically taken in the 'System.map' file. In user space addresses are relocatable and can be extracted from an objdump output:
$ aarch64-linux-gnu-objdump -d libcstest.so.1.0
...
...
000000000000072c <coresight_test1>: <------------ Beginning of traces
72c: d10083ff sub sp, sp, #0x20
730: b9000fe0 str w0, [sp,#12]
734: b9001fff str wzr, [sp,#28]
738: 14000007 b 754 <coresight_test1+0x28>
73c: b9400fe0 ldr w0, [sp,#12]
740: 11000800 add w0, w0, #0x2
744: b9000fe0 str w0, [sp,#12]
748: b9401fe0 ldr w0, [sp,#28]
74c: 11000400 add w0, w0, #0x1
750: b9001fe0 str w0, [sp,#28]
754: b9401fe0 ldr w0, [sp,#28]
758: 7100101f cmp w0, #0x4
75c: 54ffff0d b.le 73c <coresight_test1+0x10>
760: b9400fe0 ldr w0, [sp,#12]
764: 910083ff add sp, sp, #0x20
768: d65f03c0 ret
...
...
Following the address the amount of byte is specified and if tracing in user space, the full path to the binary (or library) being traced.
Start/Stop filters: With start/stop filters traces are generated when the instruction pointer is equal to the start address. Incidentally traces stop being generated when the insruction pointer is equal to the stop address. Anything that happens between there to events is traced:
perf record -e cs_etm/@tmc_etr0/k --filter 'start 0xffffff800856bc50,stop 0xffffff800856bcb0' --per-thread uname
perf record -vvv -e cs_etm/@tmc_etr0/u --filter 'start 0x72c@/opt/lib/libcstest.so.1.0, \
stop 0x40082c@/home/linaro/main' \
--per-thread ./main
Limitation on address filters: The only limitation on address filters is the amount of address comparator found on an implementation and the mutual exclusion between range and start stop filters. As such the following example would not work:
perf record -e cs_etm/@tmc_etr0/k --filter 'start 0xffffff800856bc50,stop 0xffffff800856bcb0, \ // start/stop
filter 0x72c/0x40@/opt/lib/libcstest.so.1.0' \ // address range
--per-thread uname
Additional options can be used during trace collection that add information to the captured trace.
- Timestamps: These packets are added to the trace streams to allow correlation of different sources where tools support this.
- Cycle Counts: These packets are added to get a count of cycles for blocks of executed instructions. Adding cycle counts will considerably increase the amount of generated trace.
The relationship between cycle counts and executed instructions differs according to the trace protocol.
For example, the ETMv4 protocol will emit counts for groups of instructions according to a minimum count threshold.
Presently this threshold is fixed at 256 cycles for
perf record.
Command line options in perf record to use these features are part of the options for the cs_etm event:
perf record -e cs_etm/timestamp,cycacc,@tmc_etr0/ --per-thread uname
At current version, perf record and perf script do not use this additional information.
System information for this perf pmu event can be found at:
/sys/devices/cs_etm
This contains internal format of the parameters described above:
root@linaro-developer:~# ls /sys/devices/cs_etm/format
contextid cycacc retstack sinkid timestamp
and names of registered sinks:
root@linaro-developer:~# ls /sys/devices/cs_etm/sinks
tmc_etf0 tmc_etr0 tpiu0
Note: The sinkid parameter is there to document the usage of a 32-bit internal parameter to
pass the sink name used in the cs_etm/@sink/ command to the kernel drivers. It can be used
directly as cs_etm/sinkid=<hash_value>/ but this is not recommended as the values used are
considered opaque and subject to changes.
The entire program flow will have been recorded in the perf.data file.
Information about libraries and executable is stored under $HOME/.debug:
linaro@linaro-nano:~/kernel$ tree ~/.debug
.debug
├── [kernel.kallsyms]
│ └── 0542921808098d591a7acba5a1163e8991897669
│ └── kallsyms
├── [vdso]
│ └── 551fbbe29579eb63be3178a04c16830b8d449769
│ └── vdso
├── bin
│ └── uname
│ └── ed95e81f97c4471fb2ccc21e356b780eb0c92676
│ └── elf
└── lib
└── aarch64-linux-gnu
├── ld-2.21.so
│ └── 94912dc5a1dc8c7ef2c4e4649d4b1639b6ebc8b7
│ └── elf
└── libc-2.21.so
└── 169a143e9c40cfd9d09695333e45fd67743cd2d6
└── elf
13 directories, 5 files
linaro@linaro-nano:~/kernel$
All this information needs to be collected in order to successfully decode traces off target:
linaro@linaro-nano:~/kernel$ tar czf results.trace.tgz perf.data ~/.debug
Note that file vmlinux should also be added to the bundle if kernel traces
have also been collected.
This results.trace.tgz can then be transferred to the host for decoding.
The openCSD library is not part of the perf tools. It is available on github and needs to be compiled before the perf tools. Checkout the required branch/tag version into a local directory.
linaro@t430:~/linaro/coresight$ git clone https://github.com./Linaro/OpenCSD.git my-opencsd
Cloning into 'OpenCSD'...
remote: Counting objects: 2063, done.
remote: Total 2063 (delta 0), reused 0 (delta 0), pack-reused 2063
Receiving objects: 100% (2063/2063), 2.51 MiB | 1.24 MiB/s, done.
Resolving deltas: 100% (1399/1399), done.
Checking connectivity... done.
linaro@t430:~/linaro/coresight$ ls my-opencsd
decoder LICENSE README.md HOWTO.md TODO
Once the source code has been acquired compilation of the openCSD library can
take place. By default this will build for the current host into the ./decoder/lib/builddir
directory.
linaro@t430:~/linaro/coresight$ cd my-opencsd
linaro@t430:~/linaro/coresight/my-opencsd$ ls ./decoder/build/linux
makefile makefile.dev rctdl_c_api_lib ref_trace_decode_lib
linaro@t430:~/linaro/coresight/my-opencsd$ make -C decoder/build/linux
...
...
linaro@t430:~/linaro/coresight/my-opencsd$ ls ./decoder/lib/builddir
libopencsd.a libopencsd_c_api.so libopencsd_c_api.so.1.5.5 libopencsd.so.1 libopencsd_c_api.a
libopencsd_c_api.so.1 libopencsd.so libopencsd.so.1.5.5
From there the header file and libraries need to be installed on the system, something that requires root privileges. The default installation path is /usr/include/opencsd for the header files and /usr/lib/ for the libraries:
linaro@t430:~/linaro/coresight/my-opencsd$ sudo make -C decoder/build/linux install
linaro@t430:~/linaro/coresight/my-opencsd$ ls -l /usr/include/opencsd
total 60
drwxr-xr-x 2 root root 4096 Feb 25 13:51 c_api
drwxr-xr-x 2 root root 4096 Feb 25 13:51 ete
drwxr-xr-x 2 root root 4096 Feb 25 13:51 etmv3
drwxr-xr-x 2 root root 4096 Feb 25 13:51 etmv4
-rw-r--r-- 1 root root 34668 Feb 25 13:51 ocsd_if_types.h
-rw-r--r-- 1 root root 2561 Feb 25 13:51 ocsd_if_version.h
drwxr-xr-x 2 root root 4096 Feb 25 13:51 ptm
drwxr-xr-x 2 root root 4096 Feb 25 13:51 stm
-rw-r--r-- 1 root root 10452 Feb 25 13:51 trc_gen_elem_types.h
-rw-r--r-- 1 root root 3972 Feb 25 13:51 trc_pkt_types.h
linaro@t430:~/linaro/coresight/my-opencsd$ ls -al /usr/lib/libopencsd*
-rw-r--r-- 1 root root 1709380 Feb 25 13:51 /usr/lib/libopencsd.a
-rw-r--r-- 1 root root 152314 Feb 25 13:51 /usr/lib/libopencsd_c_api.a
lrwxrwxrwx 1 root root 21 Feb 25 13:51 /usr/lib/libopencsd_c_api.so -> libopencsd_c_api.so.1
lrwxrwxrwx 1 root root 25 Feb 25 13:51 /usr/lib/libopencsd_c_api.so.1 -> libopencsd_c_api.so.1.5.5
-rw-r--r-- 1 root root 98816 Feb 25 13:51 /usr/lib/libopencsd_c_api.so.1.5.5
lrwxrwxrwx 1 root root 15 Feb 25 13:51 /usr/lib/libopencsd.so -> libopencsd.so.1
lrwxrwxrwx 1 root root 19 Feb 25 13:51 /usr/lib/libopencsd.so.1 -> libopencsd.so.1.5.5
-rw-r--r-- 1 root root 797680 Feb 25 13:51 /usr/lib/libopencsd.so.1.5.5
A "clean_install" target is also available so that openCSD installed files can be removed from a system. Going forward the goal is to have the openCSD library packaged as a Debian or RPM archive so that it can be installed from a distribution without having to be compiled.
Complete information on building and installing the library can be found in build_libs.md (@ref build_lib)
As mentioned above the openCSD library is not part of the perf tools' code base and needs to be installed on a system prior to compilation. Information about the status of the openCSD library on a system is given at compile time by the perf tools build script:
linaro@t430:~/linaro/linux-kernel$ make CORESIGHT=1 VF=1 -C tools/perf
Auto-detecting system features:
... dwarf: [ on ]
... dwarf_getlocations: [ on ]
... glibc: [ on ]
... gtk2: [ on ]
... libaudit: [ on ]
... libbfd: [ OFF ]
... libelf: [ on ]
... libnuma: [ OFF ]
... numa_num_possible_cpus: [ OFF ]
... libperl: [ on ]
... libpython: [ on ]
... libslang: [ on ]
... libcrypto: [ on ]
... libunwind: [ OFF ]
... libdw-dwarf-unwind: [ on ]
... zlib: [ on ]
... lzma: [ OFF ]
... get_cpuid: [ on ]
... bpf: [ on ]
... libopencsd: [ on ] <-------
If libopencsd is not installed, or the version is too old for correct compilation, then an error will be issued and compilation will fail.
At the end of the compilation a new perf binary is available in tools/perf/,
which we can verify is linked to libopencsd:
linaro@t430:~/linaro/linux-kernel$ ldd tools/perf/perf
linux-vdso.so.1 => (0x00007fff135db000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f15f9176000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f15f8f6e000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f15f8c64000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f15f8a60000)
libopencsd_c_api.so.1 => /usr/lib/libopencsd_c_api.so.1 (0x00007f5645cfb000) <-------
libelf.so.1 => /usr/lib/x86_64-linux-gnu/libelf.so.1 (0x00007f15f8635000)
libdw.so.1 => /usr/lib/x86_64-linux-gnu/libdw.so.1 (0x00007f15f83ec000)
libaudit.so.1 => /lib/x86_64-linux-gnu/libaudit.so.1 (0x00007f15f81c5000)
libslang.so.2 => /lib/x86_64-linux-gnu/libslang.so.2 (0x00007f15f7e38000)
libperl.so.5.22 => /usr/lib/x86_64-linux-gnu/libperl.so.5.22 (0x00007f15f7a5d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f15f7693000)
libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f15f7104000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f15f6eea000)
/lib64/ld-linux-x86-64.so.2 (0x0000559b88038000)
Additional debug output from the decoder can be compiled in by setting the
CSTRACE_RAW environment variable at build time. Setting this to packed gets trace frame
output as follows:-
Frame Data; Index 576; RAW_PACKED; d6 d6 d6 d6 d6 d6 d6 d6 fc fb d6 d6 d6 d6 e0 7f
Frame Data; Index 576; ID_DATA[0x14]; d7 d6 d7 d6 d7 d6 d7 d6 fd fb d7 d6 d7 d6 e0
Set to any other value will remove the RAW_PACKED lines.
When compiling the perf tools it is possible to reference another version of the openCSD library than the one installed on the system. This is useful when working with multiple development trees or having the desire to keep system libraries intact. Two environment variable are available to tell the perf tools build script where to get the header file and libraries, namely CSINCLUDES and CSLIBS:
linaro@t430:~/linaro/linux-kernel$ export CSINCLUDES=~/linaro/coresight/my-opencsd/decoder/include/
linaro@t430:~/linaro/linux-kernel$ export CSLIBS=~/linaro/coresight/my-opencsd/decoder/lib/builddir/
linaro@t430:~/linaro/linux-kernel$ make CORESIGHT=1 VF=1 -C tools/perf
This will have the effect of compiling and linking against the provided library. Since the system's openCSD library is in the loader's search patch the LD_LIBRARY_PATH environment variable needs to be set.
linaro@t430:~/linaro/linux-kernel$ export LD_LIBRARY_PATH=$CSLIBS
Trace bundles can be extracted anywhere and have no dependencies on where the perf tools and openCSD library have been compiled.
Take the results file from the target and extract into a working directory (e.g. the results.trace.tgz file from above)
linaro@t430:~/linaro/coresight$ mkdir trace-results
linaro@t430:~/linaro/coresight$ cd trace-results
linaro@t430:~/linaro/coresight/trace-results$ scp <path/to/target>/~/kernel/results.trace.tgz .
linaro@t430:~/linaro/coresight/trace-results$ tar xf results.trace.tgz
linaro@t430:~/linaro/coresight/trace-results$ ls -la
total 1312
drwxrwxr-x 3 linaro linaro 4096 Mar 3 10:26 .
drwxrwxr-x 5 linaro linaro 4096 Mar 3 10:13 ..
drwxr-xr-x 7 linaro linaro 4096 Feb 24 12:21 .debug
-rw------- 1 linaro linaro 78016 Feb 24 12:21 perf.data
-rw-rw-r-- 1 linaro linaro 1245881 Feb 24 12:25 results.trace.tgz
Perf is expecting files related to the trace capture (perf.data) to be located in the buildid directory.
By default this is under ~/.debug. Alternatively the default buildid directory can be changed
using the command:
perf config --system buildid.dir=/my/own/buildid/dir
This example will remove the current ~/.debug directory to be sure everything is clean.
linaro@t430:~/linaro/coresight/trace-results$ rm -rf ~/.debug
linaro@t430:~/linaro/coresight/trace-results$ cp -dpR .debug ~/
linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf report --stdio
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 0 of event 'cs_etm//u'
# Event count (approx.): 0
#
# Children Self Command Shared Object Symbol
# ........ ........ ....... ............. ......
#
# Samples: 0 of event 'dummy:u'
# Event count (approx.): 0
#
# Children Self Command Shared Object Symbol
# ........ ........ ....... ............. ......
#
# Samples: 115K of event 'instructions:u'
# Event count (approx.): 522009
#
# Children Self Command Shared Object Symbol
# ........ ........ ....... ................ ......................
#
4.13% 4.13% uname libc-2.21.so [.] 0x0000000000078758
3.81% 3.81% uname libc-2.21.so [.] 0x0000000000078e50
2.06% 2.06% uname libc-2.21.so [.] 0x00000000000fcaf4
1.65% 1.65% uname libc-2.21.so [.] 0x00000000000fcae4
1.59% 1.59% uname ld-2.21.so [.] 0x000000000000a7f4
1.50% 1.50% uname libc-2.21.so [.] 0x0000000000078e40
1.43% 1.43% uname libc-2.21.so [.] 0x00000000000fcac4
1.31% 1.31% uname libc-2.21.so [.] 0x000000000002f0c0
1.26% 1.26% uname ld-2.21.so [.] 0x0000000000016888
1.24% 1.24% uname libc-2.21.so [.] 0x0000000000078e7c
1.24% 1.24% uname libc-2.21.so [.] 0x00000000000fcab8
...
Additional data can be obtained, which contains a dump of the trace packets received using the command
mjl@ubuntu-vbox:./perf-opencsd-master/coresight/tools/perf/perf report --stdio --dump
resulting a large amount of data, trace looking like:-
0x618 [0x30]: PERF_RECORD_AUXTRACE size: 0x11ef0 offset: 0 ref: 0x4d881c1f13216016 idx: 0 tid: 15244 cpu: -1
. ... CoreSight ETM Trace data: size 73456 bytes
0: I_ASYNC : Alignment Synchronisation.
12: I_TRACE_INFO : Trace Info.
17: I_TRACE_ON : Trace On.
18: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000007F89F24D80; Ctxt: AArch64,EL0, NS;
28: I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
29: I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
30: I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
32: I_ATOM_F6 : Atom format 6.; EEEEN
33: I_ATOM_F1 : Atom format 1.; E
34: I_EXCEPT : Exception.; Data Fault; Ret Addr Follows;
36: I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000007F89F2832C;
45: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0xFFFFFFC000083400; Ctxt: AArch64,EL1, NS;
56: I_TRACE_ON : Trace On.
57: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000007F89F2832C; Ctxt: AArch64,EL0, NS;
68: I_ATOM_F3 : Atom format 3.; NEE
69: I_ATOM_F3 : Atom format 3.; NEN
70: I_ATOM_F3 : Atom format 3.; NNE
71: I_ATOM_F5 : Atom format 5.; ENENE
72: I_ATOM_F5 : Atom format 5.; NENEN
73: I_ATOM_F5 : Atom format 5.; ENENE
74: I_ATOM_F5 : Atom format 5.; NENEN
75: I_ATOM_F5 : Atom format 5.; ENENE
76: I_ATOM_F3 : Atom format 3.; NNE
77: I_ATOM_F3 : Atom format 3.; NNE
78: I_ATOM_F3 : Atom format 3.; NNE
80: I_ATOM_F3 : Atom format 3.; NNE
81: I_ATOM_F3 : Atom format 3.; ENN
82: I_EXCEPT : Exception.; Data Fault; Ret Addr Follows;
84: I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000007F89F283F0;
93: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0xFFFFFFC000083400; Ctxt: AArch64,EL1, NS;
104: I_TRACE_ON : Trace On.
105: I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000007F89F283F0; Ctxt: AArch64,EL0, NS;
116: I_ATOM_F5 : Atom format 5.; NNNNN
117: I_ATOM_F5 : Atom format 5.; NNNNN
Working with perf scripts needs more command line options but yields interesting results.
linaro@t430:~/linaro/coresight/trace-results$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-master/tools/perf/
linaro@t430:~/linaro/coresight/trace-results$ export SCRIPT_PATH=$EXEC_PATH/scripts/python/
linaro@t430:~/linaro/coresight/trace-results$ export XTOOL_PATH=/your/aarch64/toolchain/path/bin/
linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf --exec-path=${EXEC_PATH} script --script=python:${SCRIPT_PATH}/arm-cs-trace-disasm.py -- -d ${XTOOL_PATH}/aarch64-linux-gnu-objdump
7f89f24d80: 910003e0 mov x0, sp
7f89f24d84: 94000d53 bl 7f89f282d0 <free@plt+0x3790>
7f89f282d0: d11203ff sub sp, sp, #0x480
7f89f282d4: a9ba7bfd stp x29, x30, [sp,#-96]!
7f89f282d8: 910003fd mov x29, sp
7f89f282dc: a90363f7 stp x23, x24, [sp,#48]
7f89f282e0: 9101e3b7 add x23, x29, #0x78
7f89f282e4: a90573fb stp x27, x28, [sp,#80]
7f89f282e8: a90153f3 stp x19, x20, [sp,#16]
7f89f282ec: aa0003fb mov x27, x0
7f89f282f0: 910a82e1 add x1, x23, #0x2a0
7f89f282f4: a9025bf5 stp x21, x22, [sp,#32]
7f89f282f8: a9046bf9 stp x25, x26, [sp,#64]
7f89f282fc: 910102e0 add x0, x23, #0x40
7f89f28300: f800841f str xzr, [x0],#8
7f89f28304: eb01001f cmp x0, x1
7f89f28308: 54ffffc1 b.ne 7f89f28300 <free@plt+0x37c0>
7f89f28300: f800841f str xzr, [x0],#8
7f89f28304: eb01001f cmp x0, x1
7f89f28308: 54ffffc1 b.ne 7f89f28300 <free@plt+0x37c0>
7f89f28300: f800841f str xzr, [x0],#8
7f89f28304: eb01001f cmp x0, x1
7f89f28308: 54ffffc1 b.ne 7f89f28300 <free@plt+0x37c0>
When dealing with kernel space traces the vmlinux file has to be communicated explicitely to perf using the "--vmlinux" command line option:
linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf report --stdio --vmlinux=./vmlinux
...
...
linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf script --vmlinux=./vmlinux
When using scripts things get a little more convoluted. Using the same example an above but for traces but for kernel traces, the command line becomes:
linaro@t430:~/linaro/coresight/trace-results$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-master/tools/perf/
linaro@t430:~/linaro/coresight/trace-results$ export SCRIPT_PATH=$EXEC_PATH/scripts/python/
linaro@t430:~/linaro/coresight/trace-results$ export XTOOL_PATH=/your/aarch64/toolchain/path/bin/
linaro@t430:~/linaro/coresight/trace-results$ ../perf-opencsd-master/tools/perf/perf --exec-path=${EXEC_PATH} script \
--vmlinux=./vmlinux \
--script=python:${SCRIPT_PATH}/arm-cs-trace-disasm.py -- \
-d ${XTOOLS_PATH}/aarch64-linux-gnu-objdump \
-k ./vmlinux
...
...
The option "--vmlinux=./vmlinux" is interpreted by the "perf script" command the same way it is for "perf report". The option "-k ./vmlinux" is dependant on the script being executed and has no related to the "--vmlinux", though it is highly advised to keep them synchronized.
The decoder library comes with a number of bash scripts that ease the setting up of the
offline build and test environment for perf, and executing tests.
These scripts can be found in
decoder/tests/perf-test-scripts
There are three scripts provided:
perf-setup-env.bash: this sets up all the environment variables mentioned above.perf-test-report.bash: this runsperf report- using the environment setup byperf-setup-env.bashperf-test-script.bash: this runsperf script- using the environment setup byperf-setup-env.bash
Use as follows:-
-
Prior to building perf, edit
perf-setup-env.bashto conform to your environment. There are four lines at the top of the file that will require editing. -
Execute the script using the command:
source perf-setup-env.bashThis will set up a perf execute environment for using the perf report and script commands.
Alternatively use the command:
source perf-setup-env.base buildenvThis will add in the build environment variables mentioned in the sections on building above alongside the environment for using the used by the
perf-test...scripts to run the tests. -
Build perf as described above.
-
Follow the instructions for downloading the test capture, or create a capture from your target.
-
Copy the
perf-test...scripts into the capture data directory -> the one that containsperf.data. -
The scripts can now be run. No options are required for the default operation, but any command line options will be added to the perf report / perf script command line.
e.g.
./perf-test-report.bash --dump
will add the --dump option to the end of the command line and run
${PERF_EXEC_PATH}/perf report --stdio --dump
See autofdo.md (@ref AutoFDO) for details and scripts.
- Mike Leach
- Mathieu Poirier
We welcome help on this project. If you would like to add features or help improve the way things work, we want to hear from you.
Best regards, The Linaro CoreSight Team