perf/core improvements and fixes
Subsystems (5)
Arnaldo Carvalho de Melo
maintainerHi Ingo,
Please consider pulling, this is on top of my previous pull
request, perf-core-for-mingo-5.1-20190220.
- Arnaldo
Test results at the end of this message, as usual.
The following changes since commit b4409ae112caa6315f6ee678e953b9fc93e6919c:
perf tools: Make rm_rf() remove single file (2019-02-20 17:09:28 -0300)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-5.1-20190225
for you to fetch changes up to de667cce7f4f96b6e22da8fd9c065b961f355080:
perf script python: Add Python3 support to syscall-counts-by-pid.py (2019-02-25 17:17:13 -0300)
----------------------------------------------------------------
perf/core improvements and fixes:
perf annotate:
Wei Li:
- Fix getting source line failure.
perf script:
Andi Kleen:
- Handle missing fields with -F +...
perf data:
Jiri Olsa:
- Prep work to support per-cpu files in a directory.
Intel PT:
Adrian Hunter:
- Improve thread_stack__no_call_return()
- Hide x86 retpolines in thread stacks.
- exported SQL viewer refactorings, new 'top calls' report.
Alexander Shishkin:
- Copy parent's address filter offsets on clone.
- Fix address filters for vmas with non-zero offset. Applies to ARM's CoreSight as well.
python scripts:
Tony Jones:
- Python3 support for several 'perf script' python scripts.
----------------------------------------------------------------
Adrian Hunter (13):
perf thread-stack: Improve thread_stack__no_call_return()
perf thread-stack: Hide x86 retpolines
perf scripts python: exported-sql-viewer.py: Fix missing shebang
perf scripts python: exported-sql-viewer.py: Remove leftover debugging prints perf scripts python: exported-sql-viewer.py: Hide Call Graph option if no calls table perf scripts python: exported-sql-viewer.py: Move column headers
perf scripts python: exported-sql-viewer.py: Factor out ReportDialogBase perf scripts python: exported-sql-viewer.py: Factor out ReportVars perf scripts python: exported-sql-viewer.py: Move report name into ReportVars perf scripts python: exported-sql-viewer.py: Create new dialog data item classes perf scripts python: exported-sql-viewer.py: Remove SQLTableDialogDataItem perf scripts python: exported-sql-viewer.py: Remove no selection error perf scripts python: exported-sql-viewer.py: Add top calls report
Alexander Shishkin (2):
perf: Copy parent's address filter offsets on clone
perf, pt, coresight: Fix address filters for vmas with non-zero offset
Andi Kleen (2):
perf script: Handle missing fields with -F +.. perf tools: Add perf_exe() helper to find perf binary
Jiri Olsa (9):
perf data: Move size to struct perf_data_file perf data: Add global path holder
perf tools: Add depth checking to rm_rf
perf tools: Add pattern name checking to rm_rf perf tools: Add rm_rf_perf_data function
perf data: Make check_backup work over directories perf data: Fail check_backup in case of error perf data: Add perf_data__(create_dir|close_dir) functions perf data: Add perf_data__open_dir_data function
Tony Jones (10):
perf script python: Add Python3 support to netdev-times.py perf script python: Add Python3 support to failed-syscalls-by-pid.py perf script python: Add Python3 support to mem-phys-addr.py perf script python: Add Python3 support to net_dropmonitor.py perf script python: Add Python3 support to powerpc-hcalls.py perf script python: Add Python3 support to sctop.py
perf script python: Add Python3 support to stackcollapse.py perf script python: Add Python3 support to stat-cpi.py perf script python: Add Python3 support to syscall-counts.py perf script python: Add Python3 support to syscall-counts-by-pid.py
Wei Li (1):
perf annotate: Fix getting source line failure
arch/x86/events/intel/pt.c | 9 +- drivers/hwtracing/coresight/coresight-etm-perf.c | 7 +- include/linux/perf_event.h | 7 +- kernel/events/core.c | 90 ++-- tools/perf/builtin-annotate.c | 4 +- tools/perf/builtin-buildid-cache.c | 4 +- tools/perf/builtin-buildid-list.c | 8 +- tools/perf/builtin-c2c.c | 4 +- tools/perf/builtin-diff.c | 12 +- tools/perf/builtin-evlist.c | 4 +- tools/perf/builtin-inject.c | 10 +- tools/perf/builtin-kmem.c | 2 +- tools/perf/builtin-kvm.c | 8 +- tools/perf/builtin-lock.c | 8 +- tools/perf/builtin-mem.c | 8 +- tools/perf/builtin-record.c | 11 +- tools/perf/builtin-report.c | 6 +- tools/perf/builtin-sched.c | 16 +- tools/perf/builtin-script.c | 22 +- tools/perf/builtin-stat.c | 6 +- tools/perf/builtin-timechart.c | 8 +- tools/perf/builtin-trace.c | 8 +- tools/perf/scripts/python/exported-sql-viewer.py | 510 ++++++++++++++------- .../perf/scripts/python/failed-syscalls-by-pid.py | 21 +- tools/perf/scripts/python/mem-phys-addr.py | 24 +- tools/perf/scripts/python/net_dropmonitor.py | 10 +- tools/perf/scripts/python/netdev-times.py | 82 ++-- tools/perf/scripts/python/powerpc-hcalls.py | 18 +- tools/perf/scripts/python/sctop.py | 24 +- tools/perf/scripts/python/stackcollapse.py | 7 +- tools/perf/scripts/python/stat-cpi.py | 10 +- tools/perf/scripts/python/syscall-counts-by-pid.py | 22 +- tools/perf/scripts/python/syscall-counts.py | 18 +- tools/perf/util/annotate.c | 4 +- tools/perf/util/data-convert-bt.c | 4 +- tools/perf/util/data.c | 175 ++++++- tools/perf/util/data.h | 16 +- tools/perf/util/header.c | 12 +- tools/perf/util/thread-stack.c | 161 ++++++- tools/perf/util/util.c | 65 ++- tools/perf/util/util.h | 3 + 41 files changed, 1019 insertions(+), 429 deletions(-)
Test results:
The first ones are container based builds of tools/perf with and without libelf support. Where clang is available, it is also used to build perf with/without libelf, and building with LIBCLANGLLVM=1 (built-in clang) with gcc and clang when clang and its devel libraries are installed.
The objtool and samples/bpf/ builds are disabled now that I'm switching from using the sources in a local volume to fetching them from a http server to build it inside the container, to make it easier to build in a container cluster. Those will come back later.
Several are cross builds, the ones with -x-ARCH and the android one, and those may not have all the features built, due to lack of multi-arch devel packages, available and being used so far on just a few, like debian:experimental-x-{arm64,mipsel}.
The 'perf test' one will perform a variety of tests exercising tools/perf/util/, tools/lib/{bpf,traceevent,etc}, as well as run perf commands with a variety of command line event specifications to then intercept the sys_perf_event syscall to check that the perf_event_attr fields are set up as expected, among a variety of other unit tests.
Then there is the 'make -C tools/perf build-test' ones, that build tools/perf/ with a variety of feature sets, exercising the build with an incomplete set of features as well as with a complete one. It is planned to have it run on each of the containers mentioned above, using some container orchestration infrastructure. Get in contact if interested in helping having this in place.
$ export PERF_TARBALL=http://192.168.124.1/perf/perf-5.0.0-rc5.tar.xz $ dm 1 alpine:3.4 : Ok gcc (Alpine 5.3.0) 5.3.0 2 alpine:3.5 : Ok gcc (Alpine 6.2.1) 6.2.1 20160822 3 alpine:3.6 : Ok gcc (Alpine 6.3.0) 6.3.0 4 alpine:3.7 : Ok gcc (Alpine 6.4.0) 6.4.0 5 alpine:3.8 : Ok gcc (Alpine 6.4.0) 6.4.0 6 alpine:3.9 : Ok gcc (Alpine 8.2.0) 8.2.0 7 alpine:edge : Ok gcc (Alpine 8.2.0) 8.2.0 8 amazonlinux:1 : Ok gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28) 9 amazonlinux:2 : Ok gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5) 10 android-ndk:r12b-arm : Ok arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease) 11 android-ndk:r15c-arm : Ok arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease) 12 centos:5 : Ok gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-55) 13 centos:6 : Ok gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) 14 centos:7 : Ok gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36) 15 clearlinux:latest : Ok gcc (Clear Linux OS for Intel Architecture) 8.2.1 20180502 16 debian:7 : Ok gcc (Debian 4.7.2-5) 4.7.2 17 debian:8 : Ok gcc (Debian 4.9.2-10+deb8u2) 4.9.2 18 debian:9 : Ok gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 19 debian:experimental : Ok gcc (Debian 8.2.0-17) 8.2.1 20190204 20 debian:experimental-x-arm64 : Ok aarch64-linux-gnu-gcc (Debian 8.2.0-11) 8.2.0 21 debian:experimental-x-mips : Ok mips-linux-gnu-gcc (Debian 8.2.0-11) 8.2.0 22 debian:experimental-x-mips64 : Ok mips64-linux-gnuabi64-gcc (Debian 8.2.0-16) 8.2.0 23 fedora:20 : Ok gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7) 24 fedora:21 : Ok gcc (GCC) 4.9.2 20150212 (Red Hat 4.9.2-6) 25 fedora:22 : Ok gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6) 26 fedora:23 : Ok gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6) 27 fedora:24 : Ok gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1) 28 fedora:24-x-ARC-uClibc : Ok arc-linux-gcc (ARCompact ISA Linux uClibc toolchain 2017.09-rc2) 7.1.1 20170710 29 fedora:25 : Ok gcc (GCC) 6.4.1 20170727 (Red Hat 6.4.1-1) 30 fedora:26 : Ok gcc (GCC) 7.3.1 20180130 (Red Hat 7.3.1-2) 31 fedora:27 : Ok gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6) 32 fedora:28 : Ok gcc (GCC) 8.2.1 20181215 (Red Hat 8.2.1-6) 33 fedora:29 : Ok gcc (GCC) 8.2.1 20181215 (Red Hat 8.2.1-6) 34 fedora:30 : Ok gcc (GCC) 9.0.1 20190203 (Red Hat 9.0.1-0.3) 35 fedora:rawhide : Ok gcc (GCC) 9.0.0 20190119 (Red Hat 9.0.0-0.3) 36 gentoo-stage3-amd64:latest : Ok gcc (Gentoo 7.3.0-r3 p1.4) 7.3.0 37 mageia:5 : Ok gcc (GCC) 4.9.2 38 mageia:6 : Ok gcc (Mageia 5.5.0-1.mga6) 5.5.0 39 opensuse:13.2 : Ok gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064] 40 opensuse:15.0 : Ok gcc (SUSE Linux) 7.3.1 20180323 [gcc-7-branch revision 258812] 41 opensuse:15.1 : Ok gcc (SUSE Linux) 7.4.0 42 opensuse:42.1 : Ok gcc (SUSE Linux) 4.8.5 43 opensuse:42.2 : Ok gcc (SUSE Linux) 4.8.5 44 opensuse:42.3 : Ok gcc (SUSE Linux) 4.8.5 45 opensuse:tumbleweed : Ok gcc (SUSE Linux) 8.2.1 20190103 [gcc-8-branch revision 267549] 46 oraclelinux:6 : Ok gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23.0.1) 47 oraclelinux:7 : Ok gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36.0.1) 48 ubuntu:12.04.5 : Ok gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 49 ubuntu:14.04.4 : Ok gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4 50 ubuntu:14.04.4-x-linaro-arm64 : Ok aarch64-linux-gnu-gcc (Linaro GCC 5.5-2017.10) 5.5.0 51 ubuntu:16.04 : Ok gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609 52 ubuntu:16.04-x-arm : Ok arm-linux-gnueabihf-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 53 ubuntu:16.04-x-arm64 : Ok aarch64-linux-gnu-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 54 ubuntu:16.04-x-powerpc : Ok powerpc-linux-gnu-gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 55 ubuntu:16.04-x-powerpc64 : Ok powerpc64-linux-gnu-gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 56 ubuntu:16.04-x-powerpc64el : Ok powerpc64le-linux-gnu-gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 57 ubuntu:16.04-x-s390 : Ok s390x-linux-gnu-gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 58 ubuntu:17.10 : Ok gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0 59 ubuntu:18.04 : Ok gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 60 ubuntu:18.04-x-arm : Ok arm-linux-gnueabihf-gcc (Ubuntu/Linaro 7.3.0-27ubuntu1~18.04) 7.3.0 61 ubuntu:18.04-x-arm64 : Ok aarch64-linux-gnu-gcc (Ubuntu/Linaro 7.3.0-27ubuntu1~18.04) 7.3.0 62 ubuntu:18.04-x-m68k : Ok m68k-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 63 ubuntu:18.04-x-powerpc : Ok powerpc-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 64 ubuntu:18.04-x-powerpc64 : Ok powerpc64-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 65 ubuntu:18.04-x-powerpc64el : Ok powerpc64le-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 66 ubuntu:18.04-x-riscv64 : Ok riscv64-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 67 ubuntu:18.04-x-s390 : Ok s390x-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 68 ubuntu:18.04-x-sh4 : Ok sh4-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 69 ubuntu:18.04-x-sparc64 : Ok sparc64-linux-gnu-gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 70 ubuntu:18.10 : Ok gcc (Ubuntu 8.2.0-7ubuntu1) 8.2.0 71 ubuntu:19.04 : Ok gcc (Ubuntu 8.2.0-20ubuntu1) 8.2.0 72 ubuntu:19.04-x-alpha : Ok alpha-linux-gnu-gcc (Ubuntu 8.2.0-20ubuntu1) 8.2.0 73 ubuntu:19.04-x-arm64 : Ok aarch64-linux-gnu-gcc (Ubuntu/Linaro 8.2.0-20ubuntu1) 8.2.0 74 ubuntu:19.04-x-hppa : Ok hppa-linux-gnu-gcc (Ubuntu 8.2.0-20ubuntu1) 8.2.0 $
# uname -a
Linux quaco 5.0.0-rc7+ #20 SMP Mon Feb 25 16:16:50 -03 2019 x86_64 x86_64 x86_64 GNU/Linux
# git log --oneline -1
de667cce7f4f perf script python: Add Python3 support to syscall-counts-by-pid.py
# perf version --build-options
perf version 5.0.rc5.gde667c
dwarf: [ on ] # HAVE_DWARF_SUPPORT
dwarf_getlocations: [ on ] # HAVE_DWARF_GETLOCATIONS_SUPPORT
glibc: [ on ] # HAVE_GLIBC_SUPPORT
gtk2: [ on ] # HAVE_GTK2_SUPPORT
syscall_table: [ on ] # HAVE_SYSCALL_TABLE_SUPPORT
libbfd: [ on ] # HAVE_LIBBFD_SUPPORT
libelf: [ on ] # HAVE_LIBELF_SUPPORT
libnuma: [ on ] # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ on ] # HAVE_LIBNUMA_SUPPORT
libperl: [ on ] # HAVE_LIBPERL_SUPPORT
libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
libslang: [ on ] # HAVE_SLANG_SUPPORT
libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT
libunwind: [ on ] # HAVE_LIBUNWIND_SUPPORT
libdw-dwarf-unwind: [ on ] # HAVE_DWARF_SUPPORT
zlib: [ on ] # HAVE_ZLIB_SUPPORT
lzma: [ on ] # HAVE_LZMA_SUPPORT
get_cpuid: [ on ] # HAVE_AUXTRACE_SUPPORT
bpf: [ on ] # HAVE_LIBBPF_SUPPORT
# perf test
1: vmlinux symtab matches kallsyms : Ok
2: Detect openat syscall event : Ok
3: Detect openat syscall event on all cpus : Ok
4: Read samples using the mmap interface : Ok
5: Test data source output : Ok
6: Parse event definition strings : Ok
7: Simple expression parser : Ok
8: PERF_RECORD_* events & perf_sample fields : Ok
9: Parse perf pmu format : Ok
10: DSO data read : Ok
11: DSO data cache : Ok
12: DSO data reopen : Ok
13: Roundtrip evsel->name : Ok
14: Parse sched tracepoints fields : Ok
15: syscalls:sys_enter_openat event fields : Ok
16: Setup struct perf_event_attr : Ok
17: Match and link multiple hists : Ok
18: 'import perf' in python : Ok
19: Breakpoint overflow signal handler : Ok
20: Breakpoint overflow sampling : Ok
21: Breakpoint accounting : Ok
22: Watchpoint :
22.1: Read Only Watchpoint : Skip
22.2: Write Only Watchpoint : Ok
22.3: Read / Write Watchpoint : Ok
22.4: Modify Watchpoint : Ok
23: Number of exit events of a simple workload : Ok
24: Software clock events period values : Ok
25: Object code reading : $ make -C tools/perf build-test
make: Entering directory '/home/acme/git/perf/tools/perf'
- tarpkg: ./tests/perf-targz-src-pkg .
make_tags_O: make tags
make_help_O: make help
make_install_bin_O: make install-bin
make_install_prefix_slash_O: make install prefix=/tmp/krava/
make_no_libdw_dwarf_unwind_O: make NO_LIBDW_DWARF_UNWIND=1
make_no_libunwind_O: make NO_LIBUNWIND=1
make_cscope_O: make cscope
make_util_pmu_bison_o_O: make util/pmu-bison.o
make_no_libbionic_O: make NO_LIBBIONIC=1
make_install_prefix_O: make install prefix=/tmp/krava
make_pure_O: make
make_install_O: make install
make_clean_all_O: make clean all
make_no_gtk2_O: make NO_GTK2=1
make_doc_O: make doc
make_no_newt_O: make NO_NEWT=1
make_no_demangle_O: make NO_DEMANGLE=1
make_no_ui_O: make NO_NEWT=1 NO_SLANG=1 NO_GTK2=1
make_no_libnuma_O: make NO_LIBNUMA=1
make_no_libaudit_O: make NO_LIBAUDIT=1
make_perf_o_O: make perf.o
make_no_libperl_O: make NO_LIBPERL=1
make_no_auxtrace_O: make NO_AUXTRACE=1
make_no_libelf_O: make NO_LIBELF=1
make_no_libpython_O: make NO_LIBPYTHON=1
make_no_slang_O: make NO_SLANG=1
make_no_libbpf_O: make NO_LIBBPF=1
make_no_scripts_O: make NO_LIBPYTHON=1 NO_LIBPERL=1
make_with_babeltrace_O: make LIBBABELTRACE=1
make_minimal_O: make NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1 NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1 NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1 NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1 NO_LIBCRYPTO=1 NO_SDT=1 NO_JVMTI=1
make_with_clangllvm_O: make LIBCLANGLLVM=1
make_no_backtrace_O: make NO_BACKTRACE=1
make_static_O: make LDFLAGS=-static
make_util_map_o_O: make util/map.o
make_debug_O: make DEBUG=1
OK
make: Leaving directory '/home/acme/git/perf/tools/perf'
$Arnaldo Carvalho de Melo
maintainerPATCH 1/37 perf annotate: Fix getting source line failure
From: Wei Li <[email protected]>
The output of "perf annotate -l --stdio xxx" changed since commit 425859ff0de33 ("perf annotate: No need to calculate notes->start twice") removed notes->start assignment in symbol__calc_lines(). It will get failed in find_address_in_section() from symbol__tty_annotate() subroutine as the a2l->addr is wrong. So the annotate summary doesn't report the line number of source code correctly.
Before fix:
liwei@euler:~/main_code/hulk_work/hulk/tools/perf$ cat common_while_1.c
void hotspot_1(void)
{
volatile int i;for (i = 0; i < 0x10000000; i++); for (i = 0; i < 0x10000000; i++); for (i = 0; i < 0x10000000; i++);
}
int main(void)
{
hotspot_1();
return 0;
}
liwei@euler:~/main_code/hulk_work/hulk/tools/perf$ gcc common_while_1.c -g -o common_while_1liwei@euler:~/main_code/hulk_work/hulk/tools/perf$ sudo ./perf record ./common_while_1 [ perf record: Woken up 2 times to write data ] [ perf record: Captured and wrote 0.488 MB perf.data (12498 samples) ] liwei@euler:~/main_code/hulk_work/hulk/tools/perf$ sudo ./perf annotate -l -s hotspot_1 --stdio
Sorted summary for file /home/liwei/main_code/hulk_work/hulk/tools/perf/common_while_1
----------------------------------------------
19.30 common_while_1[32]
19.03 common_while_1[4e]
19.01 common_while_1[16]
5.04 common_while_1[13]
4.99 common_while_1[4b]
4.78 common_while_1[2c]
4.77 common_while_1[10]
4.66 common_while_1[2f]
4.59 common_while_1[51]
4.59 common_while_1[35]
4.52 common_while_1[19]
4.20 common_while_1[56]
0.51 common_while_1[48]
Percent | Source code & Disassembly of common_while_1 for cycles:ppp (12480 samples, percent: local period)
-----------------------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 00000000000005fa <hotspot_1>:
: hotspot_1():
: void hotspot_1(void)
: {
0.00 : 5fa: push %rbp
0.00 : 5fb: mov %rsp,%rbp
: volatile int i;
:
: for (i = 0; i < 0x10000000; i++);
0.00 : 5fe: movl $0x0,-0x4(%rbp)
0.00 : 605: jmp 610 <hotspot_1+0x16>
0.00 : 607: mov -0x4(%rbp),%eax
common_while_1[10] 4.77 : 60a: add $0x1,%eax
common_while_1[13] 5.04 : 60d: mov %eax,-0x4(%rbp)
common_while_1[16] 19.01 : 610: mov -0x4(%rbp),%eax
common_while_1[19] 4.52 : 613: cmp $0xfffffff,%eax
0.00 : 618: jle 607 <hotspot_1+0xd>
: for (i = 0; i < 0x10000000; i++);
...After fix:
liwei@euler:~/main_code/hulk_work/hulk/tools/perf$ sudo ./perf record ./common_while_1 [ perf record: Woken up 2 times to write data ] [ perf record: Captured and wrote 0.488 MB perf.data (12500 samples) ] liwei@euler:~/main_code/hulk_work/hulk/tools/perf$ sudo ./perf annotate -l -s hotspot_1 --stdio
Sorted summary for file /home/liwei/main_code/hulk_work/hulk/tools/perf/common_while_1
----------------------------------------------
33.34 common_while_1.c:5
33.34 common_while_1.c:6
33.32 common_while_1.c:7
Percent | Source code & Disassembly of common_while_1 for cycles:ppp (12482 samples, percent: local period)
-----------------------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 00000000000005fa <hotspot_1>:
: hotspot_1():
: void hotspot_1(void)
: {
0.00 : 5fa: push %rbp
0.00 : 5fb: mov %rsp,%rbp
: volatile int i;
:
: for (i = 0; i < 0x10000000; i++);
0.00 : 5fe: movl $0x0,-0x4(%rbp)
0.00 : 605: jmp 610 <hotspot_1+0x16>
0.00 : 607: mov -0x4(%rbp),%eax
common_while_1.c:5 4.70 : 60a: add $0x1,%eax
4.89 : 60d: mov %eax,-0x4(%rbp)
common_while_1.c:5 19.03 : 610: mov -0x4(%rbp),%eax
common_while_1.c:5 4.72 : 613: cmp $0xfffffff,%eax
0.00 : 618: jle 607 <hotspot_1+0xd>
: for (i = 0; i < 0x10000000; i++);
0.00 : 61a: movl $0x0,-0x4(%rbp)
0.00 : 621: jmp 62c <hotspot_1+0x32>
0.00 : 623: mov -0x4(%rbp),%eax
common_while_1.c:6 4.54 : 626: add $0x1,%eax
4.73 : 629: mov %eax,-0x4(%rbp)
common_while_1.c:6 19.54 : 62c: mov -0x4(%rbp),%eax
common_while_1.c:6 4.54 : 62f: cmp $0xfffffff,%eax
...--- tools/perf/util/annotate.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/annotate.c +2 -2
@@ -1891,6 +1891,7 @@ int symbol__annotate(struct symbol *sym, struct map *map,struct annotation_options *options, struct arch **parch) { + struct annotation *notes = symbol__annotation(sym); struct annotate_args args = { .privsize = privsize, .evsel = evsel,
@@ -1921,6 +1922,7 @@ int symbol__annotate(struct symbol *sym, struct map *map, args.ms.map = map; args.ms.sym = sym; + notes->start =map__rip_2objdump (map, sym->start);return symbol__disassemble (sym, &args); }
@@ -2796,8 +2798,6 @@ int symbol__annotate2(struct symbol *sym, struct map *map, struct perf_evsel *evsymbol__calc_percent (sym, evsel); - notes->start =map__rip_2objdump (map, sym->start); -annotation__set_offsets (notes, size);annotation__mark_jump_targets (notes, sym);annotation__compute_ipc (notes, size);
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 2/37 perf thread-stack: Improve thread_stack__no_call_return()
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Improve thread_stack__no_call_return() to better handle 'returns' that do not match the stack i.e. 'no call'. See code comments for details. The example below shows how retpolines are affected:
Example:
$ cat simple-retpoline.c
__attribute__((noinline)) int bar(void)
{
return -1;
} int foo(void)
{
return bar() + 1;
}
__attribute__((indirect_branch("thunk"))) int main()
{
int (*volatile fn)(void) = foo; fn();
return fn();
}
$ gcc -ggdb3 -Wall -Wextra -O2 -o simple-retpoline simple-retpoline.c
$ objdump -d simple-retpoline
<SNIP>
0000000000001040 <main>:
1040: 48 83 ec 18 sub $0x18,%rsp
1044: 48 8d 05 25 01 00 00 lea 0x125(%rip),%rax # 1170 <foo>
104b: 48 89 44 24 08 mov %rax,0x8(%rsp)
1050: 48 8b 44 24 08 mov 0x8(%rsp),%rax
1055: e8 1f 01 00 00 callq 1179 <__x86_indirect_thunk_rax>
105a: 48 8b 44 24 08 mov 0x8(%rsp),%rax
105f: 48 83 c4 18 add $0x18,%rsp
1063: e9 11 01 00 00 jmpq 1179 <__x86_indirect_thunk_rax>
<SNIP>
0000000000001160 <bar>:
1160: b8 ff ff ff ff mov $0xffffffff,%eax
1165: c3 retq
<SNIP>
0000000000001170 <foo>:
1170: e8 eb ff ff ff callq 1160 <bar>
1175: 83 c0 01 add $0x1,%eax
1178: c3 retq
0000000000001179 <__x86_indirect_thunk_rax>:
1179: e8 07 00 00 00 callq 1185 <__x86_indirect_thunk_rax+0xc>
117e: f3 90 pause
1180: 0f ae e8 lfence
1183: eb f9 jmp 117e <__x86_indirect_thunk_rax+0x5>
1185: 48 89 04 24 mov %rax,(%rsp)
1189: c3 retq
<SNIP>
$ perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0,017 MB simple-retpoline.perf.data ]
$ perf script -i simple-retpoline.perf.data --itrace=be -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls
2019-01-08 14:03:37.851655 Creating database...
2019-01-08 14:03:37.863256 Writing records...
2019-01-08 14:03:38.069750 Adding indexes
2019-01-08 14:03:38.078799 Done
$ ~/libexec/perf-core/scripts/python/exported-sql-viewer.py simple-retpoline.dbBefore:
main
-> __x86_indirect_thunk_rax
-> __x86_indirect_thunk_rax
-> __x86_indirect_thunk_rax
-> barAfter:
main
-> __x86_indirect_thunk_rax
-> __x86_indirect_thunk_rax
-> foo
-> barCommitter testing:
Chose "Reports", Then "Context-Sensitive Call Graph" and then go on expanding:
Before:
simple-retpolin
PID:PID
_start
_start
__libc_start_main
main
__x86_indirect_thunk_rax
__x86_indirect_thunk_rax
barAfter:
Remove the "simple.retpoline.db" file, run again the 'perf script' line to regenerate the .db file and run the exported-sql-viewer.py again to get the same all the way to 'main', then, from there, including 'main':
main
__x86_indirect_thunk_rax
__x86_indirect_thunk_rax
foo
bar--- tools/perf/util/thread-stack.c | 49 +++++++++++++++++++++++++++++++--- 1 file changed, 46 insertions(+), 3 deletions(-)
tools/perf/util/thread-stack.c +46 -3
@@ -638,14 +638,57 @@ static int thread_stack__no_call_return(struct thread *thread,else parent = root; -/* This 'return' had no 'call', so push and pop top of stack */ - cp =call_path__findnew (cpr, parent, fsym, ip, ks); +if (parent->sym == from_al->sym) { +/* +* At the bottom of the stack, assume the missing 'call' was +* before the trace started. So, pop the current symbol and push +* the 'to' symbol. +*/ +if (ts->cnt ==1 ) { + err =thread_stack__call_return (thread, ts, --ts->cnt, + tm, ref, false); +if (err) +return err; + } + +if (!ts->cnt) { + cp =call_path__findnew (cpr, root, tsym, addr, ks); + +return thread_stack__push_cp (ts, addr, tm, ref, cp, + true, false); + } + +/* +* Otherwise assume the 'return' is being used as a jump (e.g. +* retpoline) and just push the 'to' symbol. +*/ + cp =call_path__findnew (cpr, parent, tsym, addr, ks); + + err =thread_stack__push_cp (ts,0 , tm, ref, cp, true, false); +if (!err) + ts->stack[ts->cnt -1 ].non_call = true; + +return err; + } + +/* +* Assume 'parent' has not yet returned, so push 'to', and then push and +* pop 'from'. +*/ + + cp =call_path__findnew (cpr, parent, tsym, addr, ks); err =thread_stack__push_cp (ts, addr, tm, ref, cp, true, false);if (err)return err; -return thread_stack__pop_cp (thread, ts, addr, tm, ref, tsym); + cp =call_path__findnew (cpr, cp, fsym, ip, ks); + + err =thread_stack__push_cp (ts, ip, tm, ref, cp, true, false); +if (err) +return err; + +return thread_stack__call_return (thread, ts, --ts->cnt, tm, ref, false); } static int thread_stack__trace_begin(struct thread *thread,
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 3/37 perf thread-stack: Hide x86 retpolines
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
x86 retpoline functions pollute the call graph by showing up everywhere there is an indirect branch, but they do not really mean anything. Make changes so that the default retpoline functions will no longer appear in the call graph. Note this only affects the call graph, since all the original branches are left unchanged.
This does not handle function return thunks, nor is there any improvement for the handling of inline thunks or extern thunks.
Example:
$ cat simple-retpoline.c
__attribute__((noinline)) int bar(void)
{
return -1;
} int foo(void)
{
return bar() + 1;
}
__attribute__((indirect_branch("thunk"))) int main()
{
int (*volatile fn)(void) = foo; fn();
return fn();
}
$ gcc -ggdb3 -Wall -Wextra -O2 -o simple-retpoline simple-retpoline.c
$ objdump -d simple-retpoline
<SNIP>
0000000000001040 <main>:
1040: 48 83 ec 18 sub $0x18,%rsp
1044: 48 8d 05 25 01 00 00 lea 0x125(%rip),%rax # 1170 <foo>
104b: 48 89 44 24 08 mov %rax,0x8(%rsp)
1050: 48 8b 44 24 08 mov 0x8(%rsp),%rax
1055: e8 1f 01 00 00 callq 1179 <__x86_indirect_thunk_rax>
105a: 48 8b 44 24 08 mov 0x8(%rsp),%rax
105f: 48 83 c4 18 add $0x18,%rsp
1063: e9 11 01 00 00 jmpq 1179 <__x86_indirect_thunk_rax>
<SNIP>
0000000000001160 <bar>:
1160: b8 ff ff ff ff mov $0xffffffff,%eax
1165: c3 retq
<SNIP>
0000000000001170 <foo>:
1170: e8 eb ff ff ff callq 1160 <bar>
1175: 83 c0 01 add $0x1,%eax
1178: c3 retq
0000000000001179 <__x86_indirect_thunk_rax>:
1179: e8 07 00 00 00 callq 1185 <__x86_indirect_thunk_rax+0xc>
117e: f3 90 pause
1180: 0f ae e8 lfence
1183: eb f9 jmp 117e <__x86_indirect_thunk_rax+0x5>
1185: 48 89 04 24 mov %rax,(%rsp)
1189: c3 retq
<SNIP>
$ perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0,017 MB simple-retpoline.perf.data ]
$ perf script -i simple-retpoline.perf.data --itrace=be -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls
2019-01-08 14:03:37.851655 Creating database...
2019-01-08 14:03:37.863256 Writing records...
2019-01-08 14:03:38.069750 Adding indexes
2019-01-08 14:03:38.078799 Done
$ ~/libexec/perf-core/scripts/python/exported-sql-viewer.py simple-retpoline.dbBefore:
main
-> __x86_indirect_thunk_rax
-> __x86_indirect_thunk_rax
-> foo
-> barAfter:
main
-> foo
-> bar--- tools/perf/util/thread-stack.c | 112 ++++++++++++++++++++++++++++++++- 1 file changed, 109 insertions(+), 3 deletions(-)
tools/perf/util/thread-stack.c +109 -3
@@ -20,6 +20,7 @@#include "thread.h" #include "event.h" #include "machine.h" +#include "env.h" #include "util.h" #include "debug.h" #include "symbol.h"
@@ -29,6 +30,19 @@#define STACK_GROWTH 2048 +/* +* State of retpoline detection. +* +* RETPOLINE_NONE: no retpoline detection +* X86_RETPOLINE_POSSIBLE: x86 retpoline possible +* X86_RETPOLINE_DETECTED: x86 retpoline detected +*/ +enum retpoline_state_t { +RETPOLINE_NONE , +X86_RETPOLINE_POSSIBLE , +X86_RETPOLINE_DETECTED , +}; +/** * struct thread_stack_entry - thread stack entry. * @ret_addr: return address
@@ -64,6 +78,7 @@ struct thread_stack_entry {* @crp: call/return processor * @comm: current comm * @arr_sz: size of array if this is the first element of an array +* @rstate: used to detect retpolines */ struct thread_stack {struct thread_stack_entry *stack;
@@ -76,6 +91,7 @@ struct thread_stack {struct call_return_processor *crp;struct comm *comm;unsigned int arr_sz; +enum retpoline_state_t rstate; };/*
@@ -115,10 +131,16 @@ static int thread_stack__init(struct thread_stack *ts, struct thread *thread,if (err)return err; -if (thread->mg && thread->mg->machine) - ts->kernel_start =machine__kernel_start (thread->mg->machine); -else +if (thread->mg && thread->mg->machine) { +struct machine *machine = thread->mg->machine; +const char *arch =perf_env__arch (machine->env); + + ts->kernel_start =machine__kernel_start (machine); +if (!strcmp (arch,"x86" )) + ts->rstate =X86_RETPOLINE_POSSIBLE ; + }else { ts->kernel_start =1ULL <<63 ; + } ts->crp = crp;return 0 ;
@@ -733,6 +755,70 @@ static int thread_stack__trace_end(struct thread_stack *ts, false, true); } +static bool is_x86_retpoline (const char *name) +{ +const char *p =strstr (name,"__x86_indirect_thunk_" ); + +return p == name || !strcmp (name,"__indirect_thunk_start" ); +} + +/* +* x86 retpoline functions pollute the call graph. This function removes them. +* This does not handle function return thunks, nor is there any improvement +* for the handling of inline thunks or extern thunks. +*/ +static int thread_stack__x86_retpoline (struct thread_stack *ts, +struct perf_sample *sample, +struct addr_location *to_al) +{ +struct thread_stack_entry *tse = &ts->stack[ts->cnt -1 ]; +struct call_path_root *cpr = ts->crp->cpr; +struct symbol *sym = tse->cp->sym; +struct symbol *tsym = to_al->sym; +struct call_path *cp; + +if (sym &&is_x86_retpoline (sym->name)) { +/* +* This is a x86 retpoline fn. It pollutes the call graph by +* showing up everywhere there is an indirect branch, but does +* not itself mean anything. Here the top-of-stack is removed, +* by decrementing the stack count, and then further down, the +* resulting top-of-stack is replaced with the actual target. +* The result is that the retpoline functions will no longer +* appear in the call graph. Note this only affects the call +* graph, since all the original branches are left unchanged. +*/ + ts->cnt -=1 ; + sym = ts->stack[ts->cnt -2 ].cp->sym; +if (sym && sym == tsym && to_al->addr != tsym->start) { +/* +* Target is back to the middle of the symbol we came +* from so assume it is an indirect jmp and forget it +* altogether. +*/ + ts->cnt -=1 ; +return 0 ; + } + }else if (sym && sym == tsym) { +/* +* Target is back to the symbol we came from so assume it is an +* indirect jmp and forget it altogether. +*/ + ts->cnt -=1 ; +return 0 ; + } + + cp =call_path__findnew (cpr, ts->stack[ts->cnt -2 ].cp, tsym, + sample->addr, ts->kernel_start); +if (!cp) +return -ENOMEM ; + +/* Replace the top-of-stack with the actual target */ + ts->stack[ts->cnt -1 ].cp = cp; + +return 0 ; +} + int thread_stack__process(struct thread *thread, struct comm *comm, struct perf_sample *sample, struct addr_location *from_al,
@@ -740,6 +826,7 @@ int thread_stack__process(struct thread *thread, struct comm *comm, struct call_return_processor *crp) {struct thread_stack *ts =thread__stack (thread, sample->cpu); +enum retpoline_state_t rstate;int err =0 ;if (ts && !ts->crp) {
@@ -755,6 +842,10 @@ int thread_stack__process(struct thread *thread, struct comm *comm, ts->comm = comm; } + rstate = ts->rstate; +if (rstate ==X86_RETPOLINE_DETECTED ) + ts->rstate =X86_RETPOLINE_POSSIBLE ; +/* Flush stack on exec */ if (ts->comm != comm && thread->pid_ == thread->tid) { err =__thread_stack__flush (thread, ts);
@@ -791,10 +882,25 @@ int thread_stack__process(struct thread *thread, struct comm *comm, ts->kernel_start); err =thread_stack__push_cp (ts, ret_addr, sample->time, ref, cp, false, trace_end); + +/* +* A call to the same symbol but not the start of the symbol, +* may be the start of a x86 retpoline. +*/ +if (!err && rstate ==X86_RETPOLINE_POSSIBLE && to_al->sym && + from_al->sym == to_al->sym && + to_al->addr != to_al->sym->start) + ts->rstate =X86_RETPOLINE_DETECTED ; + }else if (sample->flags &PERF_IP_FLAG_RETURN ) {if (!sample->ip || !sample->addr)return 0 ; +/* x86 retpoline 'return' doesn't match the stack */ +if (rstate ==X86_RETPOLINE_DETECTED && ts->cnt >2 && + ts->stack[ts->cnt -1 ].ret_addr != sample->addr) +return thread_stack__x86_retpoline (ts, sample, to_al); + err =thread_stack__pop_cp (thread, ts, sample->addr, sample->time, ref, from_al->sym);if (err) {
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 4/37 perf scripts python: exported-sql-viewer.py: Fix missing shebang
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
exported-sql-viewer.py is a standalone python script and requires a shebang. Also only python2 is supported at present. Restore the shebang but use the more flexible 'env' form.
--- tools/perf/scripts/python/exported-sql-viewer.py | 1 + 1 file changed, 1 insertion(+)
tools/perf/scripts/python/exported-sql-viewer.py +1 -0
@@ -1,3 +1,4 @@ +#!/usr/bin/env python2 # SPDX-License-Identifier: GPL-2.0 # exported-sql-viewer.py: view data from sql database # Copyright (c) 2014-2018, Intel Corporation.
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 5/37 perf scripts python: exported-sql-viewer.py: Remove leftover debugging prints
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Remove leftover debugging prints.
--- tools/perf/scripts/python/exported-sql-viewer.py | 7 ------- 1 file changed, 7 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +0 -7
@@ -1560,7 +1560,6 @@ class SQLTableDialogDataItem(): return str(lower_id) def ConvertRelativeTime(self, val): - print "val ", val mult = 1 suffix = val[-2:] if suffix == "ms":
@@ -1582,29 +1581,23 @@ class SQLTableDialogDataItem(): return str(val) def ConvertTimeRange(self, vrange): - print "vrange ", vrange if vrange[0] == "": vrange[0] = str(self.first_time) if vrange[1] == "": vrange[1] = str(self.last_time) vrange[0] = self.ConvertRelativeTime(vrange[0]) vrange[1] = self.ConvertRelativeTime(vrange[1]) - print "vrange2 ", vrange if not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]): return False - print "ok1" beg_range = max(int(vrange[0]), self.first_time) end_range = min(int(vrange[1]), self.last_time) if beg_range > self.last_time or end_range < self.first_time: return False - print "ok2" vrange[0] = self.BinarySearchTime(0, self.last_id, beg_range, True) vrange[1] = self.BinarySearchTime(1, self.last_id + 1, end_range, False) - print "vrange3 ", vrange return True def AddTimeRange(self, value, ranges): - print "value ", value n = value.count("-") if n == 1: pass
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 6/37 perf scripts python: exported-sql-viewer.py: Hide Call Graph option if no calls table
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
The Call Graph depends on the calls table which is optional when exporting data, so hide the Call Graph option if there is no calls table.
--- tools/perf/scripts/python/exported-sql-viewer.py | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
tools/perf/scripts/python/exported-sql-viewer.py +12 -1
@@ -1786,6 +1786,16 @@ def GetEventList(db): events.append(query.value(0)) return events +# Is a table selectable + +def IsSelectable(db, table): + query = QSqlQuery(db) + try: + QueryExec(query, "SELECT * FROM " + table + " LIMIT 1") + except: + return False + return True + # SQL data preparation def SQLTableDataPrep(query, count):
@@ -2298,7 +2308,8 @@ class MainWindow(QMainWindow): edit_menu.addAction(CreateAction("&Enlarge Font", "Make text bigger", self.EnlargeFont, self, [QKeySequence("Ctrl++")])) reports_menu = menu.addMenu("&Reports") - reports_menu.addAction(CreateAction("Context-Sensitive Call &Graph", "Create a new window containing a context-sensitive call graph", self.NewCallGraph, self)) + if IsSelectable(glb.db, "calls"): + reports_menu.addAction(CreateAction("Context-Sensitive Call &Graph", "Create a new window containing a context-sensitive call graph", self.NewCallGraph, self)) self.EventMenu(GetEventList(glb.db), reports_menu)
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 7/37 perf scripts python: exported-sql-viewer.py: Move column headers
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Move column headers from SQLAutoTableModel into SQLTableModel so that they can be used for other models based on SQLTableModel.
--- .../scripts/python/exported-sql-viewer.py | 25 ++++++++++--------- 1 file changed, 13 insertions(+), 12 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +13 -12
@@ -1821,12 +1821,13 @@ class SQLTableModel(TableModel): progress = Signal(object) - def __init__(self, glb, sql, column_count, parent=None): + def __init__(self, glb, sql, column_headers, parent=None): super(SQLTableModel, self).__init__(parent) self.glb = glb self.more = True self.populated = 0 - self.fetcher = SQLFetcher(glb, sql, lambda x, y=column_count: SQLTableDataPrep(x, y), self.AddSample) + self.column_headers = column_headers + self.fetcher = SQLFetcher(glb, sql, lambda x, y=len(column_headers): SQLTableDataPrep(x, y), self.AddSample) self.fetcher.done.connect(self.Update) self.fetcher.Fetch(glb_chunk_sz)
@@ -1864,6 +1865,12 @@ class SQLTableModel(TableModel): def HasMoreRecords(self): return self.more + def columnCount(self, parent=None): + return len(self.column_headers) + + def columnHeader(self, column): + return self.column_headers[column] + # SQL automatic table data model class SQLAutoTableModel(SQLTableModel):
@@ -1873,12 +1880,12 @@ class SQLAutoTableModel(SQLTableModel): if table_name == "comm_threads_view": # For now, comm_threads_view has no id column sql = "SELECT * FROM " + table_name + " WHERE comm_id > $$last_id$$ ORDER BY comm_id LIMIT " + str(glb_chunk_sz) - self.column_headers = [] + column_headers = [] query = QSqlQuery(glb.db) if glb.dbref.is_sqlite3: QueryExec(query, "PRAGMA table_info(" + table_name + ")") while query.next(): - self.column_headers.append(query.value(1)) + column_headers.append(query.value(1)) if table_name == "sqlite_master": sql = "SELECT * FROM " + table_name else:
@@ -1891,14 +1898,8 @@ class SQLAutoTableModel(SQLTableModel): schema = "public" QueryExec(query, "SELECT column_name FROM information_schema.columns WHERE table_schema = '" + schema + "' and table_name = '" + select_table_name + "'") while query.next(): - self.column_headers.append(query.value(0)) - super(SQLAutoTableModel, self).__init__(glb, sql, len(self.column_headers), parent) - - def columnCount(self, parent=None): - return len(self.column_headers) - - def columnHeader(self, column): - return self.column_headers[column] + column_headers.append(query.value(0)) + super(SQLAutoTableModel, self).__init__(glb, sql, column_headers, parent) # Base class for custom ResizeColumnsToContents
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 8/37 perf scripts python: exported-sql-viewer.py: Factor out ReportDialogBase
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Factor out ReportDialogBase so it can be re-used.
--- .../scripts/python/exported-sql-viewer.py | 47 ++++++++++++------- 1 file changed, 30 insertions(+), 17 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +30 -17
@@ -1693,34 +1693,25 @@ class SQLTableDialogDataItem(): return False return True -# Selected branch report creation dialog +# Report Dialog Base -class SelectedBranchDialog(QDialog): +class ReportDialogBase(QDialog): - def __init__(self, glb, parent=None): - super(SelectedBranchDialog, self).__init__(parent) + def __init__(self, glb, title, items, partial, parent=None): + super(ReportDialogBase, self).__init__(parent) self.glb = glb self.name = "" self.where_clause = "" - self.setWindowTitle("Selected Branches") + self.setWindowTitle(title) self.setMinimumWidth(600) - items = ( - ("Report name:", "Enter a name to appear in the window title bar", "", "", "", ""), - ("Time ranges:", "Enter time ranges", "<timeranges>", "", "samples.id", ""), - ("CPUs:", "Enter CPUs or ranges e.g. 0,5-6", "<ranges>", "", "cpu", ""), - ("Commands:", "Only branches with these commands will be included", "comms", "comm", "comm_id", ""), - ("PIDs:", "Only branches with these process IDs will be included", "threads", "pid", "thread_id", ""), - ("TIDs:", "Only branches with these thread IDs will be included", "threads", "tid", "thread_id", ""), - ("DSOs:", "Only branches with these DSOs will be included", "dsos", "short_name", "samples.dso_id", "to_dso_id"), - ("Symbols:", "Only branches with these symbols will be included", "symbols", "name", "symbol_id", "to_symbol_id"), - ("Raw SQL clause: ", "Enter a raw SQL WHERE clause", "", "", "", ""), - ) self.data_items = [SQLTableDialogDataItem(glb, *x, parent=self) for x in items] + self.partial = partial + self.grid = QGridLayout() for row in xrange(len(self.data_items)):
@@ -1764,7 +1755,10 @@ class SelectedBranchDialog(QDialog): self.where_clause += " AND " self.where_clause += d.value if len(self.where_clause): - self.where_clause = " AND ( " + self.where_clause + " ) " + if self.partial: + self.where_clause = " AND ( " + self.where_clause + " ) " + else: + self.where_clause = " WHERE " + self.where_clause + " " else: self.ShowMessage("No selection") return
@@ -1776,6 +1770,25 @@ class SelectedBranchDialog(QDialog): def ClearMessage(self): self.status.setText("") +# Selected branch report creation dialog + +class SelectedBranchDialog(ReportDialogBase): + + def __init__(self, glb, parent=None): + title = "Selected Branches" + items = ( + ("Report name:", "Enter a name to appear in the window title bar", "", "", "", ""), + ("Time ranges:", "Enter time ranges", "<timeranges>", "", "samples.id", ""), + ("CPUs:", "Enter CPUs or ranges e.g. 0,5-6", "<ranges>", "", "cpu", ""), + ("Commands:", "Only branches with these commands will be included", "comms", "comm", "comm_id", ""), + ("PIDs:", "Only branches with these process IDs will be included", "threads", "pid", "thread_id", ""), + ("TIDs:", "Only branches with these thread IDs will be included", "threads", "tid", "thread_id", ""), + ("DSOs:", "Only branches with these DSOs will be included", "dsos", "short_name", "samples.dso_id", "to_dso_id"), + ("Symbols:", "Only branches with these symbols will be included", "symbols", "name", "symbol_id", "to_symbol_id"), + ("Raw SQL clause: ", "Enter a raw SQL WHERE clause", "", "", "", ""), + ) + super(SelectedBranchDialog, self).__init__(glb, title, items, True, parent) + # Event list def GetEventList(db):
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 9/37 perf scripts python: exported-sql-viewer.py: Factor out ReportVars
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Factor out ReportVars to provide a single container for information from report dialogs.
--- .../scripts/python/exported-sql-viewer.py | 37 ++++++++++++------- 1 file changed, 23 insertions(+), 14 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +23 -14
@@ -1398,18 +1398,26 @@ class BranchModel(TreeModel): def HasMoreRecords(self): return self.more +# Report Variables + +class ReportVars(): + + def __init__(self, where_clause = ""): + self.where_clause = where_clause + + def UniqueId(self): + return str(self.where_clause) + # Branch window class BranchWindow(QMdiSubWindow): - def __init__(self, glb, event_id, name, where_clause, parent=None): + def __init__(self, glb, event_id, name, report_vars, parent=None): super(BranchWindow, self).__init__(parent) - model_name = "Branch Events " + str(event_id) - if len(where_clause): - model_name = where_clause + " " + model_name + model_name = "Branch Events " + str(event_id) + " " + report_vars.UniqueId() - self.model = LookupCreateModel(model_name, lambda: BranchModel(glb, event_id, where_clause)) + self.model = LookupCreateModel(model_name, lambda: BranchModel(glb, event_id, report_vars.where_clause)) self.view = QTreeView() self.view.setUniformRowHeights(True)
@@ -1703,7 +1711,7 @@ class ReportDialogBase(QDialog): self.glb = glb self.name = "" - self.where_clause = "" + self.report_vars = ReportVars() self.setWindowTitle(title) self.setMinimumWidth(600)
@@ -1742,6 +1750,7 @@ class ReportDialogBase(QDialog): self.setLayout(self.vbox); def Ok(self): + vars = self.report_vars self.name = self.data_items[0].value if not self.name: self.ShowMessage("Report name is required")
@@ -1751,14 +1760,14 @@ class ReportDialogBase(QDialog): return for d in self.data_items[1:]: if len(d.value): - if len(self.where_clause): - self.where_clause += " AND " - self.where_clause += d.value - if len(self.where_clause): + if len(vars.where_clause): + vars.where_clause += " AND " + vars.where_clause += d.value + if len(vars.where_clause): if self.partial: - self.where_clause = " AND ( " + self.where_clause + " ) " + vars.where_clause = " AND ( " + vars.where_clause + " ) " else: - self.where_clause = " WHERE " + self.where_clause + " " + vars.where_clause = " WHERE " + vars.where_clause + " " else: self.ShowMessage("No selection") return
@@ -2383,13 +2392,13 @@ class MainWindow(QMainWindow): CallGraphWindow(self.glb, self) def NewBranchView(self, event_id): - BranchWindow(self.glb, event_id, "", "", self) + BranchWindow(self.glb, event_id, "", ReportVars(), self) def NewSelectedBranchView(self, event_id): dialog = SelectedBranchDialog(self.glb, self) ret = dialog.exec_() if ret: - BranchWindow(self.glb, event_id, dialog.name, dialog.where_clause, self) + BranchWindow(self.glb, event_id, dialog.name, dialog.report_vars, self) def NewTableView(self, table_name): TableWindow(self.glb, table_name, self)
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 10/37 perf scripts python: exported-sql-viewer.py: Move report name into ReportVars
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
The report name is a report variable so move it into into ReportVars.
--- tools/perf/scripts/python/exported-sql-viewer.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +8 -8
@@ -1402,7 +1402,8 @@ class BranchModel(TreeModel): class ReportVars(): - def __init__(self, where_clause = ""): + def __init__(self, name = "", where_clause = ""): + self.name = name self.where_clause = where_clause def UniqueId(self):
@@ -1412,7 +1413,7 @@ class ReportVars(): class BranchWindow(QMdiSubWindow): - def __init__(self, glb, event_id, name, report_vars, parent=None): + def __init__(self, glb, event_id, report_vars, parent=None): super(BranchWindow, self).__init__(parent) model_name = "Branch Events " + str(event_id) + " " + report_vars.UniqueId()
@@ -1435,7 +1436,7 @@ class BranchWindow(QMdiSubWindow): self.setWidget(self.vbox.Widget()) - AddSubWindow(glb.mainwindow.mdi_area, self, name + " Branch Events") + AddSubWindow(glb.mainwindow.mdi_area, self, report_vars.name + " Branch Events") def ResizeColumnToContents(self, column, n): # Using the view's resizeColumnToContents() here is extrememly slow
@@ -1710,7 +1711,6 @@ class ReportDialogBase(QDialog): self.glb = glb - self.name = "" self.report_vars = ReportVars() self.setWindowTitle(title)
@@ -1751,8 +1751,8 @@ class ReportDialogBase(QDialog): def Ok(self): vars = self.report_vars - self.name = self.data_items[0].value - if not self.name: + vars.name = self.data_items[0].value + if not vars.name: self.ShowMessage("Report name is required") return for d in self.data_items:
@@ -2392,13 +2392,13 @@ class MainWindow(QMainWindow): CallGraphWindow(self.glb, self) def NewBranchView(self, event_id): - BranchWindow(self.glb, event_id, "", ReportVars(), self) + BranchWindow(self.glb, event_id, ReportVars(), self) def NewSelectedBranchView(self, event_id): dialog = SelectedBranchDialog(self.glb, self) ret = dialog.exec_() if ret: - BranchWindow(self.glb, event_id, dialog.name, dialog.report_vars, self) + BranchWindow(self.glb, event_id, dialog.report_vars, self) def NewTableView(self, table_name): TableWindow(self.glb, table_name, self)
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 11/37 perf scripts python: exported-sql-viewer.py: Create new dialog data item classes
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Create new dialog data item classes to replace SQLTableDialogDataItem. This separates out different dialog data items and makes it easier to add new ones. SQLTableDialogDataItem is removed in a separate patch because it makes the diff more readable.
--- .../scripts/python/exported-sql-viewer.py | 285 +++++++++++++++++- 1 file changed, 272 insertions(+), 13 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +272 -13
@@ -1702,6 +1702,265 @@ class SQLTableDialogDataItem(): return False return True +# Line edit data item + +class LineEditDataItem(object): + + def __init__(self, glb, label, placeholder_text, parent, id = ""): + self.glb = glb + self.label = label + self.placeholder_text = placeholder_text + self.parent = parent + self.id = id + + self.value = "" + + self.widget = QLineEdit() + self.widget.editingFinished.connect(self.Validate) + self.widget.textChanged.connect(self.Invalidate) + self.red = False + self.error = "" + self.validated = True + + if placeholder_text: + self.widget.setPlaceholderText(placeholder_text) + + def TurnTextRed(self): + if not self.red: + palette = QPalette() + palette.setColor(QPalette.Text,Qt.red) + self.widget.setPalette(palette) + self.red = True + + def TurnTextNormal(self): + if self.red: + palette = QPalette() + self.widget.setPalette(palette) + self.red = False + + def InvalidValue(self, value): + self.value = "" + self.TurnTextRed() + self.error = self.label + " invalid value '" + value + "'" + self.parent.ShowMessage(self.error) + + def Invalidate(self): + self.validated = False + + def DoValidate(self, input_string): + self.value = input_string.strip() + + def Validate(self): + self.validated = True + self.error = "" + self.TurnTextNormal() + self.parent.ClearMessage() + input_string = self.widget.text() + if not len(input_string.strip()): + self.value = "" + return + self.DoValidate(input_string) + + def IsValid(self): + if not self.validated: + self.Validate() + if len(self.error): + self.parent.ShowMessage(self.error) + return False + return True + + def IsNumber(self, value): + try: + x = int(value) + except: + x = 0 + return str(x) == value + +# Non-negative integer ranges dialog data item + +class NonNegativeIntegerRangesDataItem(LineEditDataItem): + + def __init__(self, glb, label, placeholder_text, column_name, parent): + super(NonNegativeIntegerRangesDataItem, self).__init__(glb, label, placeholder_text, parent) + + self.column_name = column_name + + def DoValidate(self, input_string): + singles = [] + ranges = [] + for value in [x.strip() for x in input_string.split(",")]: + if "-" in value: + vrange = value.split("-") + if len(vrange) != 2 or not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]): + return self.InvalidValue(value) + ranges.append(vrange) + else: + if not self.IsNumber(value): + return self.InvalidValue(value) + singles.append(value) + ranges = [("(" + self.column_name + " >= " + r[0] + " AND " + self.column_name + " <= " + r[1] + ")") for r in ranges] + if len(singles): + ranges.append(self.column_name + " IN (" + ",".join(singles) + ")") + self.value = " OR ".join(ranges) + +# Dialog data item converted and validated using a SQL table + +class SQLTableDataItem(LineEditDataItem): + + def __init__(self, glb, label, placeholder_text, table_name, match_column, column_name1, column_name2, parent): + super(SQLTableDataItem, self).__init__(glb, label, placeholder_text, parent) + + self.table_name = table_name + self.match_column = match_column + self.column_name1 = column_name1 + self.column_name2 = column_name2 + + def ValueToIds(self, value): + ids = [] + query = QSqlQuery(self.glb.db) + stmt = "SELECT id FROM " + self.table_name + " WHERE " + self.match_column + " = '" + value + "'" + ret = query.exec_(stmt) + if ret: + while query.next(): + ids.append(str(query.value(0))) + return ids + + def DoValidate(self, input_string): + all_ids = [] + for value in [x.strip() for x in input_string.split(",")]: + ids = self.ValueToIds(value) + if len(ids): + all_ids.extend(ids) + else: + return self.InvalidValue(value) + self.value = self.column_name1 + " IN (" + ",".join(all_ids) + ")" + if self.column_name2: + self.value = "( " + self.value + " OR " + self.column_name2 + " IN (" + ",".join(all_ids) + ") )" + +# Sample time ranges dialog data item converted and validated using 'samples' SQL table + +class SampleTimeRangesDataItem(LineEditDataItem): + + def __init__(self, glb, label, placeholder_text, column_name, parent): + self.column_name = column_name + + self.last_id = 0 + self.first_time = 0 + self.last_time = 2 ** 64 + + query = QSqlQuery(glb.db) + QueryExec(query, "SELECT id, time FROM samples ORDER BY id DESC LIMIT 1") + if query.next(): + self.last_id = int(query.value(0)) + self.last_time = int(query.value(1)) + QueryExec(query, "SELECT time FROM samples WHERE time != 0 ORDER BY id LIMIT 1") + if query.next(): + self.first_time = int(query.value(0)) + if placeholder_text: + placeholder_text += ", between " + str(self.first_time) + " and " + str(self.last_time) + + super(SampleTimeRangesDataItem, self).__init__(glb, label, placeholder_text, parent) + + def IdBetween(self, query, lower_id, higher_id, order): + QueryExec(query, "SELECT id FROM samples WHERE id > " + str(lower_id) + " AND id < " + str(higher_id) + " ORDER BY id " + order + " LIMIT 1") + if query.next(): + return True, int(query.value(0)) + else: + return False, 0 + + def BinarySearchTime(self, lower_id, higher_id, target_time, get_floor): + query = QSqlQuery(self.glb.db) + while True: + next_id = int((lower_id + higher_id) / 2) + QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id)) + if not query.next(): + ok, dbid = self.IdBetween(query, lower_id, next_id, "DESC") + if not ok: + ok, dbid = self.IdBetween(query, next_id, higher_id, "") + if not ok: + return str(higher_id) + next_id = dbid + QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id)) + next_time = int(query.value(0)) + if get_floor: + if target_time > next_time: + lower_id = next_id + else: + higher_id = next_id + if higher_id <= lower_id + 1: + return str(higher_id) + else: + if target_time >= next_time: + lower_id = next_id + else: + higher_id = next_id + if higher_id <= lower_id + 1: + return str(lower_id) + + def ConvertRelativeTime(self, val): + mult = 1 + suffix = val[-2:] + if suffix == "ms": + mult = 1000000 + elif suffix == "us": + mult = 1000 + elif suffix == "ns": + mult = 1 + else: + return val + val = val[:-2].strip() + if not self.IsNumber(val): + return val + val = int(val) * mult + if val >= 0: + val += self.first_time + else: + val += self.last_time + return str(val) + + def ConvertTimeRange(self, vrange): + if vrange[0] == "": + vrange[0] = str(self.first_time) + if vrange[1] == "": + vrange[1] = str(self.last_time) + vrange[0] = self.ConvertRelativeTime(vrange[0]) + vrange[1] = self.ConvertRelativeTime(vrange[1]) + if not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]): + return False + beg_range = max(int(vrange[0]), self.first_time) + end_range = min(int(vrange[1]), self.last_time) + if beg_range > self.last_time or end_range < self.first_time: + return False + vrange[0] = self.BinarySearchTime(0, self.last_id, beg_range, True) + vrange[1] = self.BinarySearchTime(1, self.last_id + 1, end_range, False) + return True + + def AddTimeRange(self, value, ranges): + n = value.count("-") + if n == 1: + pass + elif n == 2: + if value.split("-")[1].strip() == "": + n = 1 + elif n == 3: + n = 2 + else: + return False + pos = findnth(value, "-", n) + vrange = [value[:pos].strip() ,value[pos+1:].strip()] + if self.ConvertTimeRange(vrange): + ranges.append(vrange) + return True + return False + + def DoValidate(self, input_string): + ranges = [] + for value in [x.strip() for x in input_string.split(",")]: + if not self.AddTimeRange(value, ranges): + return self.InvalidValue(value) + ranges = [("(" + self.column_name + " >= " + r[0] + " AND " + self.column_name + " <= " + r[1] + ")") for r in ranges] + self.value = " OR ".join(ranges) + # Report Dialog Base class ReportDialogBase(QDialog):
@@ -1716,7 +1975,7 @@ class ReportDialogBase(QDialog): self.setWindowTitle(title) self.setMinimumWidth(600) - self.data_items = [SQLTableDialogDataItem(glb, *x, parent=self) for x in items] + self.data_items = [x(glb, self) for x in items] self.partial = partial
@@ -1751,7 +2010,9 @@ class ReportDialogBase(QDialog): def Ok(self): vars = self.report_vars - vars.name = self.data_items[0].value + for d in self.data_items: + if d.id == "REPORTNAME": + vars.name = d.value if not vars.name: self.ShowMessage("Report name is required") return
@@ -1785,17 +2046,15 @@ class SelectedBranchDialog(ReportDialogBase): def __init__(self, glb, parent=None): title = "Selected Branches" - items = ( - ("Report name:", "Enter a name to appear in the window title bar", "", "", "", ""), - ("Time ranges:", "Enter time ranges", "<timeranges>", "", "samples.id", ""), - ("CPUs:", "Enter CPUs or ranges e.g. 0,5-6", "<ranges>", "", "cpu", ""), - ("Commands:", "Only branches with these commands will be included", "comms", "comm", "comm_id", ""), - ("PIDs:", "Only branches with these process IDs will be included", "threads", "pid", "thread_id", ""), - ("TIDs:", "Only branches with these thread IDs will be included", "threads", "tid", "thread_id", ""), - ("DSOs:", "Only branches with these DSOs will be included", "dsos", "short_name", "samples.dso_id", "to_dso_id"), - ("Symbols:", "Only branches with these symbols will be included", "symbols", "name", "symbol_id", "to_symbol_id"), - ("Raw SQL clause: ", "Enter a raw SQL WHERE clause", "", "", "", ""), - ) + items = (lambda g, p: LineEditDataItem(g, "Report name:", "Enter a name to appear in the window title bar", p, "REPORTNAME"), + lambda g, p: SampleTimeRangesDataItem(g, "Time ranges:", "Enter time ranges", "samples.id", p), + lambda g, p: NonNegativeIntegerRangesDataItem(g, "CPUs:", "Enter CPUs or ranges e.g. 0,5-6", "cpu", p), + lambda g, p: SQLTableDataItem(g, "Commands:", "Only branches with these commands will be included", "comms", "comm", "comm_id", "", p), + lambda g, p: SQLTableDataItem(g, "PIDs:", "Only branches with these process IDs will be included", "threads", "pid", "thread_id", "", p), + lambda g, p: SQLTableDataItem(g, "TIDs:", "Only branches with these thread IDs will be included", "threads", "tid", "thread_id", "", p), + lambda g, p: SQLTableDataItem(g, "DSOs:", "Only branches with these DSOs will be included", "dsos", "short_name", "samples.dso_id", "to_dso_id", p), + lambda g, p: SQLTableDataItem(g, "Symbols:", "Only branches with these symbols will be included", "symbols", "name", "symbol_id", "to_symbol_id", p), + lambda g, p: LineEditDataItem(g, "Raw SQL clause: ", "Enter a raw SQL WHERE clause", p)) super(SelectedBranchDialog, self).__init__(glb, title, items, True, parent) # Event list
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 12/37 perf scripts python: exported-sql-viewer.py: Remove SQLTableDialogDataItem
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Remove SQLTableDialogDataItem as it is no longer used.
--- .../scripts/python/exported-sql-viewer.py | 221 ------------------ 1 file changed, 221 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +0 -221
@@ -1481,227 +1481,6 @@ class BranchWindow(QMdiSubWindow): else: self.find_bar.NotFound() -# Dialog data item converted and validated using a SQL table - -class SQLTableDialogDataItem(): - - def __init__(self, glb, label, placeholder_text, table_name, match_column, column_name1, column_name2, parent): - self.glb = glb - self.label = label - self.placeholder_text = placeholder_text - self.table_name = table_name - self.match_column = match_column - self.column_name1 = column_name1 - self.column_name2 = column_name2 - self.parent = parent - - self.value = "" - - self.widget = QLineEdit() - self.widget.editingFinished.connect(self.Validate) - self.widget.textChanged.connect(self.Invalidate) - self.red = False - self.error = "" - self.validated = True - - self.last_id = 0 - self.first_time = 0 - self.last_time = 2 ** 64 - if self.table_name == "<timeranges>": - query = QSqlQuery(self.glb.db) - QueryExec(query, "SELECT id, time FROM samples ORDER BY id DESC LIMIT 1") - if query.next(): - self.last_id = int(query.value(0)) - self.last_time = int(query.value(1)) - QueryExec(query, "SELECT time FROM samples WHERE time != 0 ORDER BY id LIMIT 1") - if query.next(): - self.first_time = int(query.value(0)) - if placeholder_text: - placeholder_text += ", between " + str(self.first_time) + " and " + str(self.last_time) - - if placeholder_text: - self.widget.setPlaceholderText(placeholder_text) - - def ValueToIds(self, value): - ids = [] - query = QSqlQuery(self.glb.db) - stmt = "SELECT id FROM " + self.table_name + " WHERE " + self.match_column + " = '" + value + "'" - ret = query.exec_(stmt) - if ret: - while query.next(): - ids.append(str(query.value(0))) - return ids - - def IdBetween(self, query, lower_id, higher_id, order): - QueryExec(query, "SELECT id FROM samples WHERE id > " + str(lower_id) + " AND id < " + str(higher_id) + " ORDER BY id " + order + " LIMIT 1") - if query.next(): - return True, int(query.value(0)) - else: - return False, 0 - - def BinarySearchTime(self, lower_id, higher_id, target_time, get_floor): - query = QSqlQuery(self.glb.db) - while True: - next_id = int((lower_id + higher_id) / 2) - QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id)) - if not query.next(): - ok, dbid = self.IdBetween(query, lower_id, next_id, "DESC") - if not ok: - ok, dbid = self.IdBetween(query, next_id, higher_id, "") - if not ok: - return str(higher_id) - next_id = dbid - QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id)) - next_time = int(query.value(0)) - if get_floor: - if target_time > next_time: - lower_id = next_id - else: - higher_id = next_id - if higher_id <= lower_id + 1: - return str(higher_id) - else: - if target_time >= next_time: - lower_id = next_id - else: - higher_id = next_id - if higher_id <= lower_id + 1: - return str(lower_id) - - def ConvertRelativeTime(self, val): - mult = 1 - suffix = val[-2:] - if suffix == "ms": - mult = 1000000 - elif suffix == "us": - mult = 1000 - elif suffix == "ns": - mult = 1 - else: - return val - val = val[:-2].strip() - if not self.IsNumber(val): - return val - val = int(val) * mult - if val >= 0: - val += self.first_time - else: - val += self.last_time - return str(val) - - def ConvertTimeRange(self, vrange): - if vrange[0] == "": - vrange[0] = str(self.first_time) - if vrange[1] == "": - vrange[1] = str(self.last_time) - vrange[0] = self.ConvertRelativeTime(vrange[0]) - vrange[1] = self.ConvertRelativeTime(vrange[1]) - if not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]): - return False - beg_range = max(int(vrange[0]), self.first_time) - end_range = min(int(vrange[1]), self.last_time) - if beg_range > self.last_time or end_range < self.first_time: - return False - vrange[0] = self.BinarySearchTime(0, self.last_id, beg_range, True) - vrange[1] = self.BinarySearchTime(1, self.last_id + 1, end_range, False) - return True - - def AddTimeRange(self, value, ranges): - n = value.count("-") - if n == 1: - pass - elif n == 2: - if value.split("-")[1].strip() == "": - n = 1 - elif n == 3: - n = 2 - else: - return False - pos = findnth(value, "-", n) - vrange = [value[:pos].strip() ,value[pos+1:].strip()] - if self.ConvertTimeRange(vrange): - ranges.append(vrange) - return True - return False - - def InvalidValue(self, value): - self.value = "" - palette = QPalette() - palette.setColor(QPalette.Text,Qt.red) - self.widget.setPalette(palette) - self.red = True - self.error = self.label + " invalid value '" + value + "'" - self.parent.ShowMessage(self.error) - - def IsNumber(self, value): - try: - x = int(value) - except: - x = 0 - return str(x) == value - - def Invalidate(self): - self.validated = False - - def Validate(self): - input_string = self.widget.text() - self.validated = True - if self.red: - palette = QPalette() - self.widget.setPalette(palette) - self.red = False - if not len(input_string.strip()): - self.error = "" - self.value = "" - return - if self.table_name == "<timeranges>": - ranges = [] - for value in [x.strip() for x in input_string.split(",")]: - if not self.AddTimeRange(value, ranges): - return self.InvalidValue(value) - ranges = [("(" + self.column_name1 + " >= " + r[0] + " AND " + self.column_name1 + " <= " + r[1] + ")") for r in ranges] - self.value = " OR ".join(ranges) - elif self.table_name == "<ranges>": - singles = [] - ranges = [] - for value in [x.strip() for x in input_string.split(",")]: - if "-" in value: - vrange = value.split("-") - if len(vrange) != 2 or not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]): - return self.InvalidValue(value) - ranges.append(vrange) - else: - if not self.IsNumber(value): - return self.InvalidValue(value) - singles.append(value) - ranges = [("(" + self.column_name1 + " >= " + r[0] + " AND " + self.column_name1 + " <= " + r[1] + ")") for r in ranges] - if len(singles): - ranges.append(self.column_name1 + " IN (" + ",".join(singles) + ")") - self.value = " OR ".join(ranges) - elif self.table_name: - all_ids = [] - for value in [x.strip() for x in input_string.split(",")]: - ids = self.ValueToIds(value) - if len(ids): - all_ids.extend(ids) - else: - return self.InvalidValue(value) - self.value = self.column_name1 + " IN (" + ",".join(all_ids) + ")" - if self.column_name2: - self.value = "( " + self.value + " OR " + self.column_name2 + " IN (" + ",".join(all_ids) + ") )" - else: - self.value = input_string.strip() - self.error = "" - self.parent.ClearMessage() - - def IsValid(self): - if not self.validated: - self.Validate() - if len(self.error): - self.parent.ShowMessage(self.error) - return False - return True - # Line edit data item class LineEditDataItem(object):
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 13/37 perf scripts python: exported-sql-viewer.py: Remove no selection error
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
If no selection is made on the 'Selected branches' dialog, then the output is the same as the 'All branches' report. That is not really an error, and is not desirable for future reports, so remove it.
--- tools/perf/scripts/python/exported-sql-viewer.py | 3 --- 1 file changed, 3 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +0 -3
@@ -1808,9 +1808,6 @@ class ReportDialogBase(QDialog): vars.where_clause = " AND ( " + vars.where_clause + " ) " else: vars.where_clause = " WHERE " + vars.where_clause + " " - else: - self.ShowMessage("No selection") - return self.accept() def ShowMessage(self, msg):
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 14/37 perf scripts python: exported-sql-viewer.py: Add top calls report
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Add a new report to display top calls by elapsed time. It displays calls in descending order of time elapsed between when the function was called and when it returned.
--- .../scripts/python/exported-sql-viewer.py | 141 +++++++++++++++++- 1 file changed, 135 insertions(+), 6 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +135 -6
@@ -1402,12 +1402,13 @@ class BranchModel(TreeModel): class ReportVars(): - def __init__(self, name = "", where_clause = ""): + def __init__(self, name = "", where_clause = "", limit = ""): self.name = name self.where_clause = where_clause + self.limit = limit def UniqueId(self): - return str(self.where_clause) + return str(self.where_clause + ";" + self.limit) # Branch window
@@ -1485,16 +1486,16 @@ class BranchWindow(QMdiSubWindow): class LineEditDataItem(object): - def __init__(self, glb, label, placeholder_text, parent, id = ""): + def __init__(self, glb, label, placeholder_text, parent, id = "", default = ""): self.glb = glb self.label = label self.placeholder_text = placeholder_text self.parent = parent self.id = id - self.value = "" + self.value = default - self.widget = QLineEdit() + self.widget = QLineEdit(default) self.widget.editingFinished.connect(self.Validate) self.widget.textChanged.connect(self.Invalidate) self.red = False
@@ -1582,6 +1583,21 @@ class NonNegativeIntegerRangesDataItem(LineEditDataItem): ranges.append(self.column_name + " IN (" + ",".join(singles) + ")") self.value = " OR ".join(ranges) +# Positive integer dialog data item + +class PositiveIntegerDataItem(LineEditDataItem): + + def __init__(self, glb, label, placeholder_text, parent, id = "", default = ""): + super(PositiveIntegerDataItem, self).__init__(glb, label, placeholder_text, parent, id, default) + + def DoValidate(self, input_string): + if not self.IsNumber(input_string.strip()): + return self.InvalidValue(input_string) + value = int(input_string.strip()) + if value <= 0: + return self.InvalidValue(input_string) + self.value = str(value) + # Dialog data item converted and validated using a SQL table class SQLTableDataItem(LineEditDataItem):
@@ -1799,7 +1815,9 @@ class ReportDialogBase(QDialog): if not d.IsValid(): return for d in self.data_items[1:]: - if len(d.value): + if d.id == "LIMIT": + vars.limit = d.value + elif len(d.value): if len(vars.where_clause): vars.where_clause += " AND " vars.where_clause += d.value
@@ -2059,6 +2077,103 @@ def GetTableList(glb): tables.append("information_schema.columns") return tables +# Top Calls data model + +class TopCallsModel(SQLTableModel): + + def __init__(self, glb, report_vars, parent=None): + text = "" + if not glb.dbref.is_sqlite3: + text = "::text" + limit = "" + if len(report_vars.limit): + limit = " LIMIT " + report_vars.limit + sql = ("SELECT comm, pid, tid, name," + " CASE" + " WHEN (short_name = '[kernel.kallsyms]') THEN '[kernel]'" + text + + " ELSE short_name" + " END AS dso," + " call_time, return_time, (return_time - call_time) AS elapsed_time, branch_count, " + " CASE" + " WHEN (calls.flags = 1) THEN 'no call'" + text + + " WHEN (calls.flags = 2) THEN 'no return'" + text + + " WHEN (calls.flags = 3) THEN 'no call/return'" + text + + " ELSE ''" + text + + " END AS flags" + " FROM calls" + " INNER JOIN call_paths ON calls.call_path_id = call_paths.id" + " INNER JOIN symbols ON call_paths.symbol_id = symbols.id" + " INNER JOIN dsos ON symbols.dso_id = dsos.id" + " INNER JOIN comms ON calls.comm_id = comms.id" + " INNER JOIN threads ON calls.thread_id = threads.id" + + report_vars.where_clause + + " ORDER BY elapsed_time DESC" + + limit + ) + column_headers = ("Command", "PID", "TID", "Symbol", "Object", "Call Time", "Return Time", "Elapsed Time (ns)", "Branch Count", "Flags") + self.alignment = (Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignLeft) + super(TopCallsModel, self).__init__(glb, sql, column_headers, parent) + + def columnAlignment(self, column): + return self.alignment[column] + +# Top Calls report creation dialog + +class TopCallsDialog(ReportDialogBase): + + def __init__(self, glb, parent=None): + title = "Top Calls by Elapsed Time" + items = (lambda g, p: LineEditDataItem(g, "Report name:", "Enter a name to appear in the window title bar", p, "REPORTNAME"), + lambda g, p: SQLTableDataItem(g, "Commands:", "Only calls with these commands will be included", "comms", "comm", "comm_id", "", p), + lambda g, p: SQLTableDataItem(g, "PIDs:", "Only calls with these process IDs will be included", "threads", "pid", "thread_id", "", p), + lambda g, p: SQLTableDataItem(g, "TIDs:", "Only calls with these thread IDs will be included", "threads", "tid", "thread_id", "", p), + lambda g, p: SQLTableDataItem(g, "DSOs:", "Only calls with these DSOs will be included", "dsos", "short_name", "dso_id", "", p), + lambda g, p: SQLTableDataItem(g, "Symbols:", "Only calls with these symbols will be included", "symbols", "name", "symbol_id", "", p), + lambda g, p: LineEditDataItem(g, "Raw SQL clause: ", "Enter a raw SQL WHERE clause", p), + lambda g, p: PositiveIntegerDataItem(g, "Record limit:", "Limit selection to this number of records", p, "LIMIT", "100")) + super(TopCallsDialog, self).__init__(glb, title, items, False, parent) + +# Top Calls window + +class TopCallsWindow(QMdiSubWindow, ResizeColumnsToContentsBase): + + def __init__(self, glb, report_vars, parent=None): + super(TopCallsWindow, self).__init__(parent) + + self.data_model = LookupCreateModel("Top Calls " + report_vars.UniqueId(), lambda: TopCallsModel(glb, report_vars)) + self.model = self.data_model + + self.view = QTableView() + self.view.setModel(self.model) + self.view.setEditTriggers(QAbstractItemView.NoEditTriggers) + self.view.verticalHeader().setVisible(False) + + self.ResizeColumnsToContents() + + self.find_bar = FindBar(self, self, True) + + self.finder = ChildDataItemFinder(self.model) + + self.fetch_bar = FetchMoreRecordsBar(self.data_model, self) + + self.vbox = VBox(self.view, self.find_bar.Widget(), self.fetch_bar.Widget()) + + self.setWidget(self.vbox.Widget()) + + AddSubWindow(glb.mainwindow.mdi_area, self, report_vars.name) + + def Find(self, value, direction, pattern, context): + self.view.setFocus() + self.find_bar.Busy() + self.finder.Find(value, direction, pattern, context, self.FindDone) + + def FindDone(self, row): + self.find_bar.Idle() + if row >= 0: + self.view.setCurrentIndex(self.model.index(row, 0, QModelIndex())) + else: + self.find_bar.NotFound() + # Action Definition def CreateAction(label, tip, callback, parent=None, shortcut=None):
@@ -2162,6 +2277,7 @@ p.c2 { <p class=c2><a href=#callgraph>1.1 Context-Sensitive Call Graph</a></p> <p class=c2><a href=#allbranches>1.2 All branches</a></p> <p class=c2><a href=#selectedbranches>1.3 Selected branches</a></p> +<p class=c2><a href=#topcallsbyelapsedtime>1.4 Top calls by elapsed time</a></p> <p class=c1><a href=#tables>2. Tables</a></p> <h1 id=reports>1. Reports</h1> <h2 id=callgraph>1.1 Context-Sensitive Call Graph</h2>
@@ -2237,6 +2353,10 @@ ms, us or ns. Also, negative values are relative to the end of trace. Examples: -10ms- The last 10ms </pre> N.B. Due to the granularity of timestamps, there could be no branches in any given time range. +<h2 id=topcallsbyelapsedtime>1.4 Top calls by elapsed time</h2> +The Top calls by elapsed time report displays calls in descending order of time elapsed between when the function was called and when it returned. +The data is reduced by various selection criteria. A dialog box displays available criteria which are AND'ed together. +If not all data is fetched, a Fetch bar is provided. Ctrl-F displays a Find bar. <h1 id=tables>2. Tables</h1> The Tables menu shows all tables and views in the database. Most tables have an associated view which displays the information in a more friendly way. Not all data for large tables is fetched
@@ -2371,6 +2491,9 @@ class MainWindow(QMainWindow): self.EventMenu(GetEventList(glb.db), reports_menu) + if IsSelectable(glb.db, "calls"): + reports_menu.addAction(CreateAction("&Top calls by elapsed time", "Create a new window displaying top calls by elapsed time", self.NewTopCalls, self)) + self.TableMenu(GetTableList(glb), menu) self.window_menu = WindowMenu(self.mdi_area, menu)
@@ -2426,6 +2549,12 @@ class MainWindow(QMainWindow): def NewCallGraph(self): CallGraphWindow(self.glb, self) + def NewTopCalls(self): + dialog = TopCallsDialog(self.glb, self) + ret = dialog.exec_() + if ret: + TopCallsWindow(self.glb, dialog.report_vars, self) + def NewBranchView(self, event_id): BranchWindow(self.glb, event_id, ReportVars(), self)
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 15/37 perf: Copy parent's address filter offsets on clone
In reply to Arnaldo Carvalho de MeloFrom: Alexander Shishkin <[email protected]>
When a child event is allocated in the inherit_event() path, the VMA based filter offsets are not copied from the parent, even though the address space mapping of the new task remains the same, which leads to no trace for the new task until exec.
--- kernel/events/core.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
kernel/events/core.c +15 -0
@@ -1255,6 +1255,7 @@ static void put_ctx(struct perf_event_context *ctx)* perf_event_context::lock * perf_event::mmap_mutex * mmap_sem +* perf_addr_filters_head::lock * * cpu_hotplug_lock * pmus_lock
@@ -10312,6 +10313,20 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu, goto err_per_task; } +/* +* Clone the parent's vma offsets: they are valid until exec() +* even if the mm is not shared with the parent. +*/ +if (event->parent) { +struct perf_addr_filters_head *ifh =perf_event_addr_filters (event); + +raw_spin_lock_irq (&ifh->lock); +memcpy (event->addr_filters_offs, + event->parent->addr_filters_offs, + pmu->nr_addr_filters *sizeof (unsigned long )); +raw_spin_unlock_irq (&ifh->lock); + } +/* force hw sync on the address filters */ event->addr_filters_gen =1 ; }
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 16/37 perf, pt, coresight: Fix address filters for vmas with non-zero offset
In reply to Arnaldo Carvalho de MeloFrom: Alexander Shishkin <[email protected]>
Currently, the address range calculation for file-based filters works as long as the vma that maps the matching part of the object file starts from offset zero into the file (vm_pgoff==0). Otherwise, the resulting filter range would be off by vm_pgoff pages. Another related problem is that in case of a partially matching vma, that is, a vma that matches part of a filter region, the filter range size wouldn't be adjusted.
Fix the arithmetics around address filter range calculations, taking into account vma offset, so that the entire calculation is done before the filter configuration is passed to the PMU drivers instead of having those drivers do the final bit of arithmetics.
Based on the patch by Adrian Hunter <adrian.hunter.intel.com>.
--- arch/x86/events/intel/pt.c | 9 ++- .../hwtracing/coresight/coresight-etm-perf.c | 7 +- include/linux/perf_event.h | 7 +- kernel/events/core.c | 81 +++++++++++-------- 4 files changed, 62 insertions(+), 42 deletions(-)
arch/x86/events/intel/pt.c +5 -4
@@ -1223,7 +1223,8 @@ static int pt_event_addr_filters_validate(struct list_head *filters)static void pt_event_addr_filters_sync (struct perf_event *event) {struct perf_addr_filters_head *head =perf_event_addr_filters (event); -unsigned long msr_a, msr_b, *offs = event->addr_filters_offs; +unsigned long msr_a, msr_b; +struct perf_addr_filter_range *fr = event->addr_filter_ranges;struct pt_filters *filters = event->hw.addr_filters;struct perf_addr_filter *filter;int range =0 ;
@@ -1232,12 +1233,12 @@ static void pt_event_addr_filters_sync(struct perf_event *event)return ;list_for_each_entry (filter, &head->list, entry) { -if (filter->path.dentry && !offs[range]) { +if (filter->path.dentry && !fr[range].start) { msr_a = msr_b =0 ; }else {/* apply the offset */ - msr_a = filter->offset + offs[range]; - msr_b = filter->size + msr_a -1 ; + msr_a = fr[range].start; + msr_b = msr_a + fr[range].size -1 ; } filters->filter[range].msr_a = msr_a;
drivers/hwtracing/coresight/coresight-etm-perf.c +4 -3
@@ -433,15 +433,16 @@ static int etm_addr_filters_validate(struct list_head *filters)static void etm_addr_filters_sync (struct perf_event *event) {struct perf_addr_filters_head *head =perf_event_addr_filters (event); -unsigned long start, stop, *offs = event->addr_filters_offs; +unsigned long start, stop; +struct perf_addr_filter_range *fr = event->addr_filter_ranges;struct etm_filters *filters = event->hw.addr_filters;struct etm_filter *etm_filter;struct perf_addr_filter *filter;int i =0 ;list_for_each_entry (filter, &head->list, entry) { - start = filter->offset + offs[i]; - stop = start + filter->size; + start = fr[i].start; + stop = start + fr[i].size; etm_filter = &filters->etm_filter[i];switch (filter->action) {
include/linux/perf_event.h +6 -1
@@ -490,6 +490,11 @@ struct perf_addr_filters_head {unsigned int nr_file_filters; }; +struct perf_addr_filter_range { +unsigned long start; +unsigned long size; +}; +/** * enum perf_event_state - the states of an event: */
@@ -666,7 +671,7 @@ struct perf_event {/* address range filters */ struct perf_addr_filters_head addr_filters;/* vma address array for file-based filders */ -unsigned long *addr_filters_offs; +struct perf_addr_filter_range *addr_filter_ranges;unsigned long addr_filters_gen;void (*destroy)(struct perf_event *);
kernel/events/core.c +47 -34
163 lines not on this page · raw message
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 17/37 perf data: Move size to struct perf_data_file
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
We are about to add support for multiple files, so we need each file to keep its size.
--- tools/perf/builtin-record.c | 5 ++--- tools/perf/util/data.c | 2 +- tools/perf/util/data.h | 4 ++-- 3 files changed, 5 insertions(+), 6 deletions(-)
tools/perf/builtin-record.c +2 -3
@@ -660,10 +660,9 @@ static int process_sample_event(struct perf_tool *tool,static int process_buildids (struct record *rec) { -struct perf_data *data = &rec->data;struct perf_session *session = rec->session; -if (data->size ==0 ) +if (perf_data__size (&rec->data) ==0 )return 0 ;/*
@@ -851,7 +850,7 @@ record__finish_output(struct record *rec)return ; rec->session->header.data_size += rec->bytes_written; - data->size =lseek (perf_data__fd (data),0 ,SEEK_CUR ); + data->file.size =lseek (perf_data__fd (data),0 ,SEEK_CUR );if (!rec->no_buildid) {process_buildids (rec);
tools/perf/util/data.c +1 -1
@@ -82,7 +82,7 @@ static int open_file_read(struct perf_data *data) goto out_close; } - data->size = st.st_size; + data->file.size = st.st_size;return fd; out_close:
tools/perf/util/data.h +2 -2
@@ -12,13 +12,13 @@ enum perf_data_mode {struct perf_data_file {const char *path;int fd; +unsigned long size; };struct perf_data {struct perf_data_file file;bool is_pipe;bool force; -unsigned long size;enum perf_data_mode mode; };
@@ -44,7 +44,7 @@ static inline int perf_data__fd(struct perf_data *data)static inline unsigned long perf_data__size (struct perf_data *data) { -return data->size; +return data->file.size; }int perf_data__open (struct perf_data *data);
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 18/37 perf data: Add global path holder
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Add a 'path' member to 'struct perf_data'. It will keep the configured path for the data (const char *). The path in struct perf_data_file is now dynamically allocated (duped) from it.
This scheme is useful/used in following patches where struct perf_data::path holds the 'configure' directory path and struct perf_data_file::path holds the allocated path for specific files.
Also it actually makes the code little simpler.
--- tools/perf/builtin-annotate.c | 4 +-- tools/perf/builtin-buildid-cache.c | 4 +-- tools/perf/builtin-buildid-list.c | 8 +++--- tools/perf/builtin-c2c.c | 4 +-- tools/perf/builtin-diff.c | 12 ++++----- tools/perf/builtin-evlist.c | 4 +-- tools/perf/builtin-inject.c | 10 +++----- tools/perf/builtin-kmem.c | 2 +- tools/perf/builtin-kvm.c | 8 +++--- tools/perf/builtin-lock.c | 8 +++--- tools/perf/builtin-mem.c | 8 +++--- tools/perf/builtin-record.c | 6 ++--- tools/perf/builtin-report.c | 6 ++--- tools/perf/builtin-sched.c | 16 +++++------- tools/perf/builtin-script.c | 12 ++++----- tools/perf/builtin-stat.c | 6 ++--- tools/perf/builtin-timechart.c | 8 +++--- tools/perf/builtin-trace.c | 8 +++--- tools/perf/util/data-convert-bt.c | 4 +-- tools/perf/util/data.c | 39 +++++++++++++++++++++--------- tools/perf/util/data.h | 3 ++- 21 files changed, 87 insertions(+), 93 deletions(-)
tools/perf/builtin-annotate.c +2 -2
@@ -441,7 +441,7 @@ static int __cmd_annotate(struct perf_annotate *ann) }if (total_nr_samples ==0 ) { -ui__error ("The %s file has no samples!\n" , session->data->file.path); +ui__error ("The %s data has no samples!\n" , session->data->path); goto out; }
@@ -578,7 +578,7 @@ int cmd_annotate(int argc, const char **argv)if (quiet)perf_quiet_option (); - data.file.path = input_name; + data.path = input_name; annotate.session =perf_session__new (&data, false, &annotate.tool);if (annotate.session ==NULL )
tools/perf/builtin-buildid-cache.c +2 -2
@@ -416,8 +416,8 @@ int cmd_buildid_cache(int argc, const char **argv) nsi =nsinfo__new (ns_id);if (missing_filename) { - data.file.path = missing_filename; - data.force = force; + data.path = missing_filename; + data.force = force; session =perf_session__new (&data, false,NULL );if (session ==NULL )
tools/perf/builtin-buildid-list.c +3 -5
@@ -52,11 +52,9 @@ static int perf_session__list_build_ids(bool force, bool with_hits) {struct perf_session *session;struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = force, };symbol__elf_init ();
tools/perf/builtin-c2c.c +2 -2
@@ -2750,8 +2750,8 @@ static int perf_c2c__report(int argc, const char **argv)if (!input_name || !strlen (input_name)) input_name ="perf.data" ; - data.file.path = input_name; - data.force = symbol_conf.force; + data.path = input_name; + data.force = symbol_conf.force; err =setup_display (display);if (err)
tools/perf/builtin-diff.c +6 -6
@@ -708,7 +708,7 @@ static void data__fprintf(void)data__for_each_file (i, d)fprintf (stdout,"# [%d] %s %s\n" , - d->idx, d->data.file.path, + d->idx, d->data.path, !d->idx ?"(Baseline)" :"" );fprintf (stdout,"#\n" );
@@ -779,14 +779,14 @@ static int __cmd_diff(void)data__for_each_file (i, d) { d->session =perf_session__new (&d->data, false, &tool);if (!d->session) { -pr_err ("Failed to open %s\n" , d->data.file.path); +pr_err ("Failed to open %s\n" , d->data.path); ret =-1 ; goto out_delete; } ret =perf_session__process_events (d->session);if (ret) { -pr_err ("Failed to process %s\n" , d->data.file.path); +pr_err ("Failed to process %s\n" , d->data.path); goto out_delete; }
@@ -1289,9 +1289,9 @@ static int data_init(int argc, const char **argv)data__for_each_file (i, d) {struct perf_data *data = &d->data; - data->file.path = use_default ? defaults[i] : argv[i]; - data->mode =PERF_DATA_MODE_READ , - data->force = force, + data->path = use_default ? defaults[i] : argv[i]; + data->mode =PERF_DATA_MODE_READ , + data->force = force, d->idx = i; }
tools/perf/builtin-evlist.c +1 -3
@@ -23,9 +23,7 @@ static int __cmd_evlist(const char *file_name, struct perf_attr_details *detailsstruct perf_session *session;struct perf_evsel *pos;struct perf_data data = { - .file = { - .path = file_name, - }, + .path = file_name, .mode =PERF_DATA_MODE_READ , .force = details->force, };
tools/perf/builtin-inject.c +4 -6
@@ -770,10 +770,8 @@ int cmd_inject(int argc, const char **argv) .input_name = "-", .samples = LIST_HEAD_INIT(inject.samples), .output = { - .file = { - .path = "-", - }, - .mode = PERF_DATA_MODE_WRITE, + .path = "-", + .mode = PERF_DATA_MODE_WRITE, }, };struct perf_data data = {
@@ -786,7 +784,7 @@ int cmd_inject(int argc, const char **argv) "Inject build-ids into the output stream"),OPT_STRING ('i' ,"input" , &inject.input_name,"file" ,"input file name" ), -OPT_STRING ('o' ,"output" , &inject.output.file.path,"file" , +OPT_STRING ('o' ,"output" , &inject.output.path,"file" ,"output file name" ),OPT_BOOLEAN ('s' ,"sched-stat" , &inject.sched_stat, "Merge sched-stat and sched-switch for getting events "
@@ -834,7 +832,7 @@ int cmd_inject(int argc, const char **argv) inject.tool.ordered_events = inject.sched_stat; - data.file.path = inject.input_name; + data.path = inject.input_name; inject.session =perf_session__new (&data, true, &inject.tool);if (inject.session ==NULL )return -1 ;
tools/perf/builtin-kmem.c +1 -1
@@ -1949,7 +1949,7 @@ int cmd_kmem(int argc, const char **argv)return __cmd_record (argc, argv); } - data.file.path = input_name; + data.path = input_name; kmem_session = session =perf_session__new (&data, false, &perf_kmem);if (session ==NULL )
tools/perf/builtin-kvm.c +3 -5
@@ -1080,11 +1080,9 @@ static int read_events(struct perf_kvm_stat *kvm) .ordered_events = true, };struct perf_data file = { - .file = { - .path = kvm->file_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = kvm->force, + .path = kvm->file_name, + .mode =PERF_DATA_MODE_READ , + .force = kvm->force, }; kvm->tool = eops;
tools/perf/builtin-lock.c +3 -5
@@ -866,11 +866,9 @@ static int __cmd_report(bool display_info) .ordered_events = true, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = force, }; session =perf_session__new (&data, false, &eops);
tools/perf/builtin-mem.c +3 -5
@@ -239,11 +239,9 @@ static int process_sample_event(struct perf_tool *tool,static int report_raw_events (struct perf_mem *mem) {struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = mem->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = mem->force, };int ret; struct perf_session *session = perf_session__new(&data, false,
tools/perf/builtin-record.c +3 -3
@@ -918,7 +918,7 @@ record__switch_output(struct record *rec, bool at_exit)if (!quiet)fprintf (stderr,"[ perf record: Dump %s.%s ]\n" , - data->file.path, timestamp); + data->path, timestamp);/* Output tracking events */ if (!at_exit) {
@@ -1461,7 +1461,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)fprintf (stderr,"[ perf record: Captured and wrote %.3f MB %s%s%s ]\n" ,perf_data__size (data) /1024.0 /1024.0 , - data->file.path, postfix, samples); + data->path, postfix, samples); } out_delete_session:
@@ -1862,7 +1862,7 @@ static struct option __record_options[] = {OPT_STRING ('C' ,"cpu" , &record.opts.target.cpu_list,"cpu" ,"list of cpus to monitor" ),OPT_U64 ('c' ,"count" , &record.opts.user_interval,"event period to sample" ), -OPT_STRING ('o' ,"output" , &record.data.file.path,"file" , +OPT_STRING ('o' ,"output" , &record.data.path,"file" ,"output file name" ), OPT_BOOLEAN_SET('i', "no-inherit", &record.opts.no_inherit, &record.opts.no_inherit_set,
tools/perf/builtin-report.c +3 -3
@@ -899,7 +899,7 @@ static int __cmd_report(struct report *rep) rep->nr_entries +=evsel__hists (pos)->nr_entries;if (rep->nr_entries ==0 ) { -ui__error ("The %s file has no samples!\n" , data->file.path); +ui__error ("The %s data has no samples!\n" , data->path);return 0 ; }
@@ -1207,8 +1207,8 @@ int cmd_report(int argc, const char **argv) input_name ="perf.data" ; } - data.file.path = input_name; - data.force = symbol_conf.force; + data.path = input_name; + data.force = symbol_conf.force; repeat: session =perf_session__new (&data, false, &report.tool);
tools/perf/builtin-sched.c +6 -10
@@ -1785,11 +1785,9 @@ static int perf_sched__read_events(struct perf_sched *sched) };struct perf_session *session;struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = sched->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = sched->force, };int rc =-1 ;
@@ -2958,11 +2956,9 @@ static int perf_sched__timehist(struct perf_sched *sched) { "sched:sched_migrate_task", timehist_migrate_task_event, }, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = sched->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = sched->force, };struct perf_session *session;
tools/perf/builtin-script.c +5 -7
@@ -2951,10 +2951,8 @@ int find_scripts(char **scripts_array, char **scripts_path_array)DIR *scripts_dir, *lang_dir;struct perf_session *session;struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , + .path = input_name, + .mode =PERF_DATA_MODE_READ , };char *temp;int i =0 ;
@@ -3427,8 +3425,8 @@ int cmd_script(int argc, const char **argv) argc =parse_options_subcommand (argc, argv, options, script_subcommands, script_usage,PARSE_OPT_STOP_AT_NON_OPTION ); - data.file.path = input_name; - data.force = symbol_conf.force; + data.path = input_name; + data.force = symbol_conf.force;if (argc >1 && !strncmp (argv[0 ],"rec" ,strlen ("rec" ))) { rec_script_path =get_script_path (argv[1 ],RECORD_SUFFIX );
@@ -3654,7 +3652,7 @@ int cmd_script(int argc, const char **argv) goto out_delete; } - input =open (data.file.path,O_RDONLY );/* input_name */ + input =open (data.path,O_RDONLY );/* input_name */ if (input <0 ) { err = -errno;perror ("failed to open file" );
tools/perf/builtin-stat.c +3 -3
@@ -1322,7 +1322,7 @@ static int __cmd_record(int argc, const char **argv) PARSE_OPT_STOP_AT_NON_OPTION);if (output_name) - data->file.path = output_name; + data->path = output_name;if (stat_config.run_count !=1 || forever) {pr_err ("Cannot use -r option with perf stat record.\n" );
@@ -1523,8 +1523,8 @@ static int __cmd_report(int argc, const char **argv) input_name ="perf.data" ; } - perf_stat.data.file.path = input_name; - perf_stat.data.mode =PERF_DATA_MODE_READ ; + perf_stat.data.path = input_name; + perf_stat.data.mode =PERF_DATA_MODE_READ ; session =perf_session__new (&perf_stat.data, false, &perf_stat.tool);if (session ==NULL )
tools/perf/builtin-timechart.c +3 -5
@@ -1602,11 +1602,9 @@ static int __cmd_timechart(struct timechart *tchart, const char *output_name) { "syscalls:sys_exit_select", process_exit_poll }, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = tchart->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = tchart->force, }; struct perf_session *session = perf_session__new(&data, false,
tools/perf/builtin-trace.c +3 -5
@@ -3154,11 +3154,9 @@ static int trace__replay(struct trace *trace) { "probe:vfs_getname", trace__vfs_getname, }, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = trace->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = trace->force, };struct perf_session *session;struct perf_evsel *evsel;
tools/perf/util/data-convert-bt.c +2 -2
@@ -1578,7 +1578,7 @@ int bt_convert__perf2ctf(const char *input, const char *path, {struct perf_session *session;struct perf_data data = { - .file = { .path = input, .fd =-1 }, + .path = input, .mode =PERF_DATA_MODE_READ , .force = opts->force, };
@@ -1650,7 +1650,7 @@ int bt_convert__perf2ctf(const char *input, const char *path,fprintf (stderr,"[ perf data convert: Converted '%s' into CTF data '%s' ]\n" , - data.file.path, path); + data.path, path); fprintf(stderr, "[ perf data convert: Converted and wrote %.3f MB (%" PRIu64 " samples",
tools/perf/util/data.c +27 -12
@@ -19,11 +19,11 @@ static bool check_pipe(struct perf_data *data)int fd =perf_data__is_read (data) ?STDIN_FILENO :STDOUT_FILENO ; -if (!data->file.path) { +if (!data->path) {if (!fstat (fd, &st) &&S_ISFIFO (st.st_mode)) is_pipe = true; }else { -if (!strcmp (data->file.path,"-" )) +if (!strcmp (data->path,"-" )) is_pipe = true; }
@@ -37,13 +37,13 @@ static int check_backup(struct perf_data *data) {struct stat st; -if (!stat (data->file.path, &st) && st.st_size) { +if (!stat (data->path, &st) && st.st_size) {/* TODO check errors properly */ char oldname[PATH_MAX ];snprintf (oldname,sizeof (oldname),"%s.old" , - data->file.path); + data->path);unlink (oldname); -rename (data->file.path, oldname); +rename (data->path, oldname); }return 0 ;
@@ -115,8 +115,22 @@ static int open_file(struct perf_data *data) fd =perf_data__is_read (data) ?open_file_read (data) :open_file_write (data); +if (fd <0 ) { +free (data->file.path); +return -1 ; + } + data->file.fd = fd; -return fd <0 ?-1 :0 ; +return 0 ; +} + +static int open_file_dup (struct perf_data *data) +{ + data->file.path =strdup (data->path); +if (!data->file.path) +return -ENOMEM ; + +return open_file (data); }int perf_data__open (struct perf_data *data)
@@ -124,14 +138,15 @@ int perf_data__open(struct perf_data *data)if (check_pipe (data))return 0 ; -if (!data->file.path) - data->file.path ="perf.data" ; +if (!data->path) + data->path ="perf.data" ; -return open_file (data); +return open_file_dup (data); }void perf_data__close (struct perf_data *data) { +free (data->file.path);close (data->file.fd); }
@@ -159,15 +174,15 @@ int perf_data__switch(struct perf_data *data,if (perf_data__is_read (data))return -EINVAL ; -if (asprintf (&new_filepath,"%s.%s" , data->file.path, postfix) <0 ) +if (asprintf (&new_filepath,"%s.%s" , data->path, postfix) <0 )return -ENOMEM ;/* * Only fire a warning, don't return error, continue fill * original file. */ -if (rename (data->file.path, new_filepath)) -pr_warning ("Failed to rename %s to %s\n" , data->file.path, new_filepath); +if (rename (data->path, new_filepath)) +pr_warning ("Failed to rename %s to %s\n" , data->path, new_filepath);if (!at_exit) {close (data->file.fd);
tools/perf/util/data.h +2 -1
@@ -10,12 +10,13 @@ enum perf_data_mode { };struct perf_data_file { -const char *path; +char *path;int fd;unsigned long size; };struct perf_data { +const char *path;struct perf_data_file file;bool is_pipe;bool force;
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 19/37 perf tools: Add depth checking to rm_rf
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Adding depth argument to rm_rf (and renaming it to rm_rf_depth) to specify the depth we will go searching for files to remove.
It will be used to specify single depth for perf.data directory removal in following patch.
--- tools/perf/util/util.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/perf/util/util.c +12 -2
@@ -117,7 +117,12 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } -int rm_rf (const char *path) +/* +* The depth specify how deep the removal will go. +* 0 - will remove only files under the 'path' directory +* 1 .. x - will dive in x-level deep under the 'path' directory +*/ +static int rm_rf_depth (const char *path,int depth) {DIR *dir;int ret;
@@ -155,7 +160,7 @@ int rm_rf(const char *path) }if (S_ISDIR (statbuf.st_mode)) - ret =rm_rf (namebuf); + ret = depth ?rm_rf_depth (namebuf, depth -1 ) :0 ;else ret =unlink (namebuf); }
@@ -167,6 +172,11 @@ int rm_rf(const char *path)return rmdir (path); } +int rm_rf (const char *path) +{ +return rm_rf_depth (path,INT_MAX ); +} +/* A filter which removes dot files */ bool lsdir_no_dot_filter (const char *name__maybe_unused ,struct dirent *d) {
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 20/37 perf tools: Add pattern name checking to rm_rf
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Add pattern argument to rm_rf_depth() (and rename it to rm_rf_depth_pat()) to specify the name pattern files need to match inside the directory.
The function fails if we find different file to remove.
--- tools/perf/util/util.c | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)
tools/perf/util/util.c +33 -3
@@ -21,6 +21,7 @@#include <linux/time64.h> #include <unistd.h> #include "strlist.h" +#include "string2.h" /* * XXX We need to find a better place for these things...
@@ -117,12 +118,38 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } +static bool match_pat (char *file,const char **pat) +{ +int i =0 ; + +if (!pat) +return true; + +while (pat[i]) { +if (strglobmatch (file, pat[i])) +return true; + + i++; + } + +return false; +} +/* * The depth specify how deep the removal will go. * 0 - will remove only files under the 'path' directory * 1 .. x - will dive in x-level deep under the 'path' directory +* +* If specified the pat is array of string patterns ended with NULL, +* which are checked upon every file/directory found. Only matching +* ones are removed. +* +* The function returns: +* 0 on success +* -1 on removal failure with errno set +* -2 on pattern failure */ -static int rm_rf_depth (const char *path,int depth) +static int rm_rf_depth_pat (const char *path,int depth,const char **pat) {DIR *dir;int ret;
@@ -149,6 +176,9 @@ static int rm_rf_depth(const char *path, int depth)if (!strcmp (d->d_name,"." ) || !strcmp (d->d_name,".." ))continue ; +if (!match_pat (d->d_name, pat)) +return -2 ; +scnprintf (namebuf,sizeof (namebuf),"%s/%s" , path, d->d_name);
@@ -160,7 +190,7 @@ static int rm_rf_depth(const char *path, int depth) }if (S_ISDIR (statbuf.st_mode)) - ret = depth ?rm_rf_depth (namebuf, depth -1 ) :0 ; + ret = depth ?rm_rf_depth_pat (namebuf, depth -1 , pat) :0 ;else ret =unlink (namebuf); }
@@ -174,7 +204,7 @@ static int rm_rf_depth(const char *path, int depth)int rm_rf (const char *path) { -return rm_rf_depth (path,INT_MAX ); +return rm_rf_depth_pat (path,INT_MAX ,NULL ); }/* A filter which removes dot files */
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 21/37 perf tools: Add rm_rf_perf_data function
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
To remove perf.data including the directory, with checking on expected files and no other directories inside.
--- tools/perf/util/util.c | 11 +++++++++++ tools/perf/util/util.h | 1 + 2 files changed, 12 insertions(+)
tools/perf/util/util.c +11 -0
@@ -202,6 +202,17 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)return rmdir (path); } +int rm_rf_perf_data (const char *path) +{ +const char *pat[] = { +"header" , +"data.*" , +NULL , + }; + +return rm_rf_depth_pat (path,0 , pat); +} +int rm_rf (const char *path) {return rm_rf_depth_pat (path,INT_MAX ,NULL );
tools/perf/util/util.h +1 -0
@@ -31,6 +31,7 @@ struct strlist;int mkdir_p (char *path,mode_t mode);int rm_rf (const char *path); +int rm_rf_perf_data (const char *path);struct strlist *lsdir (const char *name,bool (*filter)(const char *,struct dirent *));bool lsdir_no_dot_filter (const char *name,struct dirent *d);int copyfile (const char *from,const char *to);
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 22/37 perf data: Make check_backup work over directories
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Change check_backup() to call rm_rf_perf_data() instead of unlink() to work over directory paths.
Also move the call earlier in the code, before we fork for file/dir, so it can backup also directory data.
--- tools/perf/util/data.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
tools/perf/util/data.c +7 -4
@@ -37,12 +37,15 @@ static int check_backup(struct perf_data *data) {struct stat st; +if (perf_data__is_read (data)) +return 0 ; +if (!stat (data->path, &st) && st.st_size) {/* TODO check errors properly */ char oldname[PATH_MAX ];snprintf (oldname,sizeof (oldname),"%s.old" , data->path); -unlink (oldname); +rm_rf_perf_data (oldname);rename (data->path, oldname); }
@@ -95,9 +98,6 @@ static int open_file_write(struct perf_data *data)int fd;char sbuf[STRERR_BUFSIZE ]; -if (check_backup (data)) -return -1 ; - fd =open (data->file.path,O_CREAT |O_RDWR |O_TRUNC |O_CLOEXEC ,S_IRUSR |S_IWUSR );
@@ -141,6 +141,9 @@ int perf_data__open(struct perf_data *data)if (!data->path) data->path ="perf.data" ; +if (check_backup (data)) +return -1 ; +return open_file_dup (data); }
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 23/37 perf data: Fail check_backup in case of error
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
And display the error message from removing the old data file:
$ perf record ls Can't remove old data: Permission denied (perf.data.old) Perf session creation failed.
$ perf record ls Can't remove old data: Unknown file found (perf.data.old) Perf session creation failed.
Not sure how to make fail the rename (after we successfully remove the destination file/dir) to show the message, anyway let's have it there.
--- tools/perf/util/data.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
tools/perf/util/data.c +18 -3
@@ -41,12 +41,27 @@ static int check_backup(struct perf_data *data)return 0 ;if (!stat (data->path, &st) && st.st_size) { -/* TODO check errors properly */ char oldname[PATH_MAX ]; +int ret; +snprintf (oldname,sizeof (oldname),"%s.old" , data->path); -rm_rf_perf_data (oldname); -rename (data->path, oldname); + + ret =rm_rf_perf_data (oldname); +if (ret) { +pr_err ("Can't remove old data: %s (%s)\n" , + ret ==-2 ? +"Unknown file found" :strerror (errno), + oldname); +return -1 ; + } + +if (rename (data->path, oldname)) { +pr_err ("Can't move data: %s (%s to %s)\n" , +strerror (errno), + data->path, oldname); +return -1 ; + } }return 0 ;
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 24/37 perf data: Add perf_data__(create_dir|close_dir) functions
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Add perf_data__create_dir() to create nr files inside 'struct perf_data' path directory:
int perf_data__create_dir(struct perf_data *data, int nr);
and function to close that data:
void perf_data__close_dir(struct perf_data *data);
--- tools/perf/util/data.c | 47 ++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data.h | 8 +++++++ 2 files changed, 55 insertions(+)
tools/perf/util/data.c +47 -0
@@ -7,11 +7,58 @@#include <fcntl.h> #include <unistd.h> #include <string.h> +#include <asm/bug.h> #include "data.h" #include "util.h" #include "debug.h" +static void close_dir (struct perf_data_file *files,int nr) +{ +while (--nr >=1 ) { +close (files[nr].fd); +free (files[nr].path); + } +free (files); +} + +void perf_data__close_dir (struct perf_data *data) +{ +close_dir (data->dir.files, data->dir.nr); +} + +int perf_data__create_dir (struct perf_data *data,int nr) +{ +struct perf_data_file *files =NULL ; +int i, ret =-1 ; + + files =zalloc (nr *sizeof (*files)); +if (!files) +return -ENOMEM ; + + data->dir.files = files; + data->dir.nr = nr; + +for (i =0 ; i < nr; i++) { +struct perf_data_file *file = &files[i]; + +if (asprintf (&file->path,"%s/data.%d" , data->path, i) <0 ) + goto out_err; + + ret =open (file->path,O_RDWR |O_CREAT |O_TRUNC ,S_IRUSR |S_IWUSR ); +if (ret <0 ) + goto out_err; + + file->fd = ret; + } + +return 0 ; + +out_err: +close_dir (files, i); +return ret; +} +static bool check_pipe (struct perf_data *data) {struct stat st;
tools/perf/util/data.h +8 -0
@@ -21,6 +21,11 @@ struct perf_data {bool is_pipe;bool force;enum perf_data_mode mode; + +struct { +struct perf_data_file *files; +int nr; + } dir; };static inline bool perf_data__is_read (struct perf_data *data)
@@ -64,4 +69,7 @@ ssize_t perf_data_file__write(struct perf_data_file *file,int perf_data__switch (struct perf_data *data,const char *postfix,size_t pos,bool at_exit); + +int perf_data__create_dir (struct perf_data *data,int nr); +void perf_data__close_dir (struct perf_data *data);#endif /* __PERF_DATA_H */
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 25/37 perf data: Add perf_data__open_dir_data function
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Add perf_data__open_dir_data to open files inside 'struct perf_data' path directory:
static int perf_data__open_dir(struct perf_data *data);
--- tools/perf/util/data.c | 59 ++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data.h | 1 + 2 files changed, 60 insertions(+)
tools/perf/util/data.c +59 -0
@@ -8,6 +8,8 @@#include <unistd.h> #include <string.h> #include <asm/bug.h> +#include <sys/types.h> +#include <dirent.h> #include "data.h" #include "util.h"
@@ -59,6 +61,63 @@ int perf_data__create_dir(struct perf_data *data, int nr)return ret; } +int perf_data__open_dir (struct perf_data *data) +{ +struct perf_data_file *files =NULL ; +struct dirent *dent; +int ret =-1 ; +DIR *dir; +int nr =0 ; + + dir =opendir (data->path); +if (!dir) +return -EINVAL ; + +while ((dent =readdir (dir)) !=NULL ) { +struct perf_data_file *file; +char path[PATH_MAX ]; +struct stat st; + +snprintf (path,sizeof (path),"%s/%s" , data->path, dent->d_name); +if (stat (path, &st)) +continue ; + +if (!S_ISREG (st.st_mode) ||strncmp (dent->d_name,"data" ,4 )) +continue ; + + ret = -ENOMEM ; + + file =realloc (files, (nr +1 ) *sizeof (*files)); +if (!file) + goto out_err; + + files = file; + file = &files[nr++]; + + file->path =strdup (path); +if (!file->path) + goto out_err; + + ret =open (file->path,O_RDONLY ); +if (ret <0 ) + goto out_err; + + file->fd = ret; + file->size = st.st_size; + } + +if (!files) +return -EINVAL ; + + data->dir.files = files; + data->dir.nr = nr; +return 0 ; + +out_err: +close_dir (files, nr); +return ret; +} +static bool check_pipe (struct perf_data *data) {struct stat st;
tools/perf/util/data.h +1 -0
@@ -71,5 +71,6 @@ int perf_data__switch(struct perf_data *data, size_t pos, bool at_exit);int perf_data__create_dir (struct perf_data *data,int nr); +int perf_data__open_dir (struct perf_data *data);void perf_data__close_dir (struct perf_data *data);#endif /* __PERF_DATA_H */
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 26/37 perf script: Handle missing fields with -F +..
In reply to Arnaldo Carvalho de MeloFrom: Andi Kleen <[email protected]>
When using -F + syntax to add a field the existing defaults are currently all marked user_set. This can cause errors when some field is missing in the perf.data
This patch tracks the actually user set fields separately, so that we don't error out in this case.
Before:
% perf record true
% perf script -F +metric
Samples for 'cycles:ppp' event do not have CPU attribute set. Cannot print 'cpu' field. %
After:
5 perf record true
% perf script -F +metric
perf 28936 278636.237688: 1 cycles:ppp: ffffffff8117da99 perf_event_exec+0x59 (/lib/modules/4.20.0-odilo/build/vmlinux)
...
%
--- tools/perf/builtin-script.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
tools/perf/builtin-script.c +7 -3
@@ -149,6 +149,7 @@ static struct {unsigned int print_ip_opts;u64 fields;u64 invalid_fields; +u64 user_set_fields; } output[OUTPUT_TYPE_MAX] = { [PERF_TYPE_HARDWARE] = {
@@ -345,7 +346,7 @@ static int perf_evsel__do_check_stype(struct perf_evsel *evsel,if (attr->sample_type & sample_type)return 0 ; -if (output[type].user_set) { +if (output[type].user_set_fields & field) {if (allow_user_set)return 0 ; evname =perf_evsel__name (evsel);
@@ -2632,10 +2633,13 @@ static int parse_output_fields(const struct option *opt __maybe_unused,pr_warning ("\'%s\' not valid for %s events. Ignoring.\n" , all_output_options[i].str,event_type (j)); }else { -if (change ==REMOVE ) +if (change ==REMOVE ) { output[j].fields &= ~all_output_options[i].field; -else + output[j].user_set_fields &= ~all_output_options[i].field; + }else { output[j].fields |= all_output_options[i].field; + output[j].user_set_fields |= all_output_options[i].field; + } output[j].user_set = true; output[j].wildcard_set = true; }
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 27/37 perf tools: Add perf_exe() helper to find perf binary
In reply to Arnaldo Carvalho de MeloFrom: Andi Kleen <[email protected]>
Also convert one existing user.
--- tools/perf/util/header.c | 12 +++--------- tools/perf/util/util.c | 10 ++++++++++ tools/perf/util/util.h | 2 ++ 3 files changed, 15 insertions(+), 9 deletions(-)
tools/perf/util/header.c +3 -9
@@ -527,17 +527,11 @@ static int write_event_desc(struct feat_fd *ff,static int write_cmdline (struct feat_fd *ff,struct perf_evlist *evlist__maybe_unused ) { -char buf[MAXPATHLEN ]; -u32 n; -int i, ret; +char pbuf[MAXPATHLEN ], *buf; +int i, ret, n;/* actual path to perf binary */ - ret =readlink ("/proc/self/exe" , buf,sizeof (buf) -1 ); -if (ret <=0 ) -return -1 ; - -/* readlink() does not add null termination */ - buf[ret] ='\0' ; + buf =perf_exe (pbuf,MAXPATHLEN );/* account for binary path */ n = perf_env.nr_cmdline +1 ;
tools/perf/util/util.c +10 -0
@@ -568,3 +568,13 @@ const char *perf_tip(const char *dirpath)return tip; } + +char *perf_exe (char *buf,int len) +{ +int n =readlink ("/proc/self/exe" , buf, len); +if (n >0 ) { + buf[n] =0 ; +return buf; + } +return strcpy (buf,"perf" ); +}
tools/perf/util/util.h +2 -0
@@ -77,6 +77,8 @@ extern bool perf_singlethreaded;void perf_set_singlethreaded (void );void perf_set_multithreaded (void ); +char *perf_exe (char *buf,int len); +#ifndef O_CLOEXEC #ifdef __sparc__#define O_CLOEXEC 0x400000
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 28/37 perf script python: Add Python3 support to netdev-times.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the netdev-times.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6.
--- tools/perf/scripts/python/netdev-times.py | 82 ++++++++++++----------- 1 file changed, 42 insertions(+), 40 deletions(-)
tools/perf/scripts/python/netdev-times.py +42 -40
@@ -8,6 +8,8 @@ # dev=: show only thing related to specified device # debug: work with debug mode. It shows buffer status. +from __future__ import print_function + import os import sys
@@ -17,6 +19,7 @@ sys.path.append(os.environ['PERF_EXEC_PATH'] + \ from perf_trace_context import * from Core import * from Util import * +from functools import cmp_to_key all_event_list = []; # insert all tracepoint event related with this script irq_dic = {}; # key is cpu and value is a list which stacks irqs
@@ -61,12 +64,12 @@ def diff_msec(src, dst): def print_transmit(hunk): if dev != 0 and hunk['dev'].find(dev) < 0: return - print "%7s %5d %6d.%06dsec %12.3fmsec %12.3fmsec" % \ + print("%7s %5d %6d.%06dsec %12.3fmsec %12.3fmsec" % (hunk['dev'], hunk['len'], nsecs_secs(hunk['queue_t']), nsecs_nsecs(hunk['queue_t'])/1000, diff_msec(hunk['queue_t'], hunk['xmit_t']), - diff_msec(hunk['xmit_t'], hunk['free_t'])) + diff_msec(hunk['xmit_t'], hunk['free_t']))) # Format for displaying rx packet processing PF_IRQ_ENTRY= " irq_entry(+%.3fmsec irq=%d:%s)"
@@ -98,55 +101,55 @@ def print_receive(hunk): if show_hunk == 0: return - print "%d.%06dsec cpu=%d" % \ - (nsecs_secs(base_t), nsecs_nsecs(base_t)/1000, cpu) + print("%d.%06dsec cpu=%d" % + (nsecs_secs(base_t), nsecs_nsecs(base_t)/1000, cpu)) for i in range(len(irq_list)): - print PF_IRQ_ENTRY % \ + print(PF_IRQ_ENTRY % (diff_msec(base_t, irq_list[i]['irq_ent_t']), - irq_list[i]['irq'], irq_list[i]['name']) - print PF_JOINT + irq_list[i]['irq'], irq_list[i]['name'])) + print(PF_JOINT) irq_event_list = irq_list[i]['event_list'] for j in range(len(irq_event_list)): irq_event = irq_event_list[j] if irq_event['event'] == 'netif_rx': - print PF_NET_RX % \ + print(PF_NET_RX % (diff_msec(base_t, irq_event['time']), - irq_event['skbaddr']) - print PF_JOINT - print PF_SOFT_ENTRY % \ - diff_msec(base_t, hunk['sirq_ent_t']) - print PF_JOINT + irq_event['skbaddr'])) + print(PF_JOINT) + print(PF_SOFT_ENTRY % + diff_msec(base_t, hunk['sirq_ent_t'])) + print(PF_JOINT) event_list = hunk['event_list'] for i in range(len(event_list)): event = event_list[i] if event['event_name'] == 'napi_poll': - print PF_NAPI_POLL % \ - (diff_msec(base_t, event['event_t']), event['dev']) + print(PF_NAPI_POLL % + (diff_msec(base_t, event['event_t']), event['dev'])) if i == len(event_list) - 1: - print "" + print("") else: - print PF_JOINT + print(PF_JOINT) else: - print PF_NET_RECV % \ + print(PF_NET_RECV % (diff_msec(base_t, event['event_t']), event['skbaddr'], - event['len']) + event['len'])) if 'comm' in event.keys(): - print PF_WJOINT - print PF_CPY_DGRAM % \ + print(PF_WJOINT) + print(PF_CPY_DGRAM % (diff_msec(base_t, event['comm_t']), - event['pid'], event['comm']) + event['pid'], event['comm'])) elif 'handle' in event.keys(): - print PF_WJOINT + print(PF_WJOINT) if event['handle'] == "kfree_skb": - print PF_KFREE_SKB % \ + print(PF_KFREE_SKB % (diff_msec(base_t, event['comm_t']), - event['location']) + event['location'])) elif event['handle'] == "consume_skb": - print PF_CONS_SKB % \ + print(PF_CONS_SKB % diff_msec(base_t, - event['comm_t']) - print PF_JOINT + event['comm_t'])) + print(PF_JOINT) def trace_begin(): global show_tx
@@ -172,8 +175,7 @@ def trace_begin(): def trace_end(): # order all events in time - all_event_list.sort(lambda a,b :cmp(a[EINFO_IDX_TIME], - b[EINFO_IDX_TIME])) + all_event_list.sort(key=cmp_to_key(lambda a,b :a[EINFO_IDX_TIME] < b[EINFO_IDX_TIME])) # process all events for i in range(len(all_event_list)): event_info = all_event_list[i]
@@ -210,19 +212,19 @@ def trace_end(): print_receive(receive_hunk_list[i]) # display transmit hunks if show_tx: - print " dev len Qdisc " \ - " netdevice free" + print(" dev len Qdisc " + " netdevice free") for i in range(len(tx_free_list)): print_transmit(tx_free_list[i]) if debug: - print "debug buffer status" - print "----------------------------" - print "xmit Qdisc:remain:%d overflow:%d" % \ - (len(tx_queue_list), of_count_tx_queue_list) - print "xmit netdevice:remain:%d overflow:%d" % \ - (len(tx_xmit_list), of_count_tx_xmit_list) - print "receive:remain:%d overflow:%d" % \ - (len(rx_skb_list), of_count_rx_skb_list) + print("debug buffer status") + print("----------------------------") + print("xmit Qdisc:remain:%d overflow:%d" % + (len(tx_queue_list), of_count_tx_queue_list)) + print("xmit netdevice:remain:%d overflow:%d" % + (len(tx_xmit_list), of_count_tx_xmit_list)) + print("receive:remain:%d overflow:%d" % + (len(rx_skb_list), of_count_rx_skb_list)) # called from perf, when it finds a correspoinding event def irq__softirq_entry(name, context, cpu, sec, nsec, pid, comm, callchain, vec):
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 29/37 perf script python: Add Python3 support to failed-syscalls-by-pid.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the failed-syscalls-by-pid.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- .../scripts/python/failed-syscalls-by-pid.py | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)
tools/perf/scripts/python/failed-syscalls-by-pid.py +11 -10
@@ -5,6 +5,8 @@ # Displays system-wide failed system call totals, broken down by pid. # If a [comm] arg is specified, only syscalls called by [comm] are displayed. +from __future__ import print_function + import os import sys
@@ -32,7 +34,7 @@ if len(sys.argv) > 1: syscalls = autodict() def trace_begin(): - print "Press control+C to stop and show the summary" + print("Press control+C to stop and show the summary") def trace_end(): print_error_totals()
@@ -57,22 +59,21 @@ def syscalls__sys_exit(event_name, context, common_cpu, def print_error_totals(): if for_comm is not None: - print "\nsyscall errors for %s:\n\n" % (for_comm), + print("\nsyscall errors for %s:\n" % (for_comm)) else: - print "\nsyscall errors:\n\n", + print("\nsyscall errors:\n") - print "%-30s %10s\n" % ("comm [pid]", "count"), - print "%-30s %10s\n" % ("------------------------------", \ - "----------"), + print("%-30s %10s" % ("comm [pid]", "count")) + print("%-30s %10s" % ("------------------------------", "----------")) comm_keys = syscalls.keys() for comm in comm_keys: pid_keys = syscalls[comm].keys() for pid in pid_keys: - print "\n%s [%d]\n" % (comm, pid), + print("\n%s [%d]" % (comm, pid)) id_keys = syscalls[comm][pid].keys() for id in id_keys: - print " syscall: %-16s\n" % syscall_name(id), + print(" syscall: %-16s" % syscall_name(id)) ret_keys = syscalls[comm][pid][id].keys() - for ret, val in sorted(syscalls[comm][pid][id].iteritems(), key = lambda(k, v): (v, k), reverse = True): - print " err = %-20s %10d\n" % (strerror(ret), val), + for ret, val in sorted(syscalls[comm][pid][id].items(), key = lambda kv: (kv[1], kv[0]), reverse = True): + print(" err = %-20s %10d" % (strerror(ret), val))
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 30/37 perf script python: Add Python3 support to mem-phys-addr.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the mem-phys-addr.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- tools/perf/scripts/python/mem-phys-addr.py | 24 +++++++++++++--------- 1 file changed, 14 insertions(+), 10 deletions(-)
tools/perf/scripts/python/mem-phys-addr.py +14 -10
@@ -4,6 +4,8 @@ # Copyright (c) 2018, Intel Corporation. from __future__ import division +from __future__ import print_function + import os import sys import struct
@@ -31,21 +33,23 @@ def parse_iomem(): for i, j in enumerate(f): m = re.split('-|:',j,2) if m[2].strip() == 'System RAM': - system_ram.append(long(m[0], 16)) - system_ram.append(long(m[1], 16)) + system_ram.append(int(m[0], 16)) + system_ram.append(int(m[1], 16)) if m[2].strip() == 'Persistent Memory': - pmem.append(long(m[0], 16)) - pmem.append(long(m[1], 16)) + pmem.append(int(m[0], 16)) + pmem.append(int(m[1], 16)) def print_memory_type(): - print "Event: %s" % (event_name) - print "%-40s %10s %10s\n" % ("Memory type", "count", "percentage"), - print "%-40s %10s %10s\n" % ("----------------------------------------", \ + print("Event: %s" % (event_name)) + print("%-40s %10s %10s\n" % ("Memory type", "count", "percentage"), end='') + print("%-40s %10s %10s\n" % ("----------------------------------------", "-----------", "-----------"), + end=''); total = sum(load_mem_type_cnt.values()) for mem_type, count in sorted(load_mem_type_cnt.most_common(), \ - key = lambda(k, v): (v, k), reverse = True): - print "%-40s %10d %10.1f%%\n" % (mem_type, count, 100 * count / total), + key = lambda kv: (kv[1], kv[0]), reverse = True): + print("%-40s %10d %10.1f%%\n" % (mem_type, count, 100 * count / total), + end='') def trace_begin(): parse_iomem()
@@ -80,7 +84,7 @@ def find_memory_type(phys_addr): f.seek(0, 0) for j in f: m = re.split('-|:',j,2) - if long(m[0], 16) <= phys_addr <= long(m[1], 16): + if int(m[0], 16) <= phys_addr <= int(m[1], 16): return m[2] return "N/A"
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 31/37 perf script python: Add Python3 support to net_dropmonitor.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the net_dropmonitor.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- tools/perf/scripts/python/net_dropmonitor.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
tools/perf/scripts/python/net_dropmonitor.py +6 -4
@@ -1,6 +1,8 @@ # Monitor the system for dropped packets and proudce a report of drop locations and counts # SPDX-License-Identifier: GPL-2.0 +from __future__ import print_function + import os import sys
@@ -50,19 +52,19 @@ def get_sym(sloc): return (None, 0) def print_drop_table(): - print "%25s %25s %25s" % ("LOCATION", "OFFSET", "COUNT") + print("%25s %25s %25s" % ("LOCATION", "OFFSET", "COUNT")) for i in drop_log.keys(): (sym, off) = get_sym(i) if sym == None: sym = i - print "%25s %25s %25s" % (sym, off, drop_log[i]) + print("%25s %25s %25s" % (sym, off, drop_log[i])) def trace_begin(): - print "Starting trace (Ctrl-C to dump results)" + print("Starting trace (Ctrl-C to dump results)") def trace_end(): - print "Gathering kallsyms data" + print("Gathering kallsyms data") get_kallsyms_table() print_drop_table()
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 32/37 perf script python: Add Python3 support to powerpc-hcalls.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the powerpc-hcalls.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- tools/perf/scripts/python/powerpc-hcalls.py | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
tools/perf/scripts/python/powerpc-hcalls.py +10 -8
@@ -4,6 +4,8 @@ # # Hypervisor call statisics +from __future__ import print_function + import os import sys
@@ -149,7 +151,7 @@ hcall_table = { } def hcall_table_lookup(opcode): - if (hcall_table.has_key(opcode)): + if (opcode in hcall_table): return hcall_table[opcode] else: return opcode
@@ -157,8 +159,8 @@ def hcall_table_lookup(opcode): print_ptrn = '%-28s%10s%10s%10s%10s' def trace_end(): - print print_ptrn % ('hcall', 'count', 'min(ns)', 'max(ns)', 'avg(ns)') - print '-' * 68 + print(print_ptrn % ('hcall', 'count', 'min(ns)', 'max(ns)', 'avg(ns)')) + print('-' * 68) for opcode in output: h_name = hcall_table_lookup(opcode) time = output[opcode]['time']
@@ -166,14 +168,14 @@ def trace_end(): min_t = output[opcode]['min'] max_t = output[opcode]['max'] - print print_ptrn % (h_name, cnt, min_t, max_t, time/cnt) + print(print_ptrn % (h_name, cnt, min_t, max_t, time//cnt)) def powerpc__hcall_exit(name, context, cpu, sec, nsec, pid, comm, callchain, opcode, retval): - if (d_enter.has_key(cpu) and d_enter[cpu].has_key(opcode)): + if (cpu in d_enter and opcode in d_enter[cpu]): diff = nsecs(sec, nsec) - d_enter[cpu][opcode] - if (output.has_key(opcode)): + if (opcode in output): output[opcode]['time'] += diff output[opcode]['cnt'] += 1 if (output[opcode]['min'] > diff):
@@ -190,11 +192,11 @@ def powerpc__hcall_exit(name, context, cpu, sec, nsec, pid, comm, callchain, del d_enter[cpu][opcode] # else: -# print "Can't find matching hcall_enter event. Ignoring sample" +# print("Can't find matching hcall_enter event. Ignoring sample") def powerpc__hcall_entry(event_name, context, cpu, sec, nsec, pid, comm, callchain, opcode): - if (d_enter.has_key(cpu)): + if (cpu in d_enter): d_enter[cpu][opcode] = nsecs(sec, nsec) else: d_enter[cpu] = {opcode: nsecs(sec, nsec)}
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 33/37 perf script python: Add Python3 support to sctop.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the sctop.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- tools/perf/scripts/python/sctop.py | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
tools/perf/scripts/python/sctop.py +16 -8
@@ -8,7 +8,14 @@ # will be refreshed every [interval] seconds. The default interval is # 3 seconds. -import os, sys, thread, time +from __future__ import print_function + +import os, sys, time + +try: + import thread +except ImportError: + import _thread as thread sys.path.append(os.environ['PERF_EXEC_PATH'] + \ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
@@ -62,18 +69,19 @@ def print_syscall_totals(interval): while 1: clear_term() if for_comm is not None: - print "\nsyscall events for %s:\n\n" % (for_comm), + print("\nsyscall events for %s:\n" % (for_comm)) else: - print "\nsyscall events:\n\n", + print("\nsyscall events:\n") - print "%-40s %10s\n" % ("event", "count"), - print "%-40s %10s\n" % ("----------------------------------------", \ - "----------"), + print("%-40s %10s" % ("event", "count")) + print("%-40s %10s" % + ("----------------------------------------", + "----------")) - for id, val in sorted(syscalls.iteritems(), key = lambda(k, v): (v, k), \ + for id, val in sorted(syscalls.items(), key = lambda kv: (kv[1], kv[0]), \ reverse = True): try: - print "%-40s %10d\n" % (syscall_name(id), val), + print("%-40s %10d" % (syscall_name(id), val)) except TypeError: pass syscalls.clear()
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 34/37 perf script python: Add Python3 support to stackcollapse.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the stackcollapse.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- tools/perf/scripts/python/stackcollapse.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
tools/perf/scripts/python/stackcollapse.py +4 -3
@@ -19,6 +19,8 @@ # Written by Paolo Bonzini <[email protected]> # Based on Brendan Gregg's stackcollapse-perf.pl script. +from __future__ import print_function + import os import sys from collections import defaultdict
@@ -120,7 +122,6 @@ def process_event(param_dict): lines[stack_string] = lines[stack_string] + 1 def trace_end(): - list = lines.keys() - list.sort() + list = sorted(lines) for stack in list: - print "%s %d" % (stack, lines[stack]) + print("%s %d" % (stack, lines[stack]))
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 35/37 perf script python: Add Python3 support to stat-cpi.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the stat-cpi.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- tools/perf/scripts/python/stat-cpi.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
tools/perf/scripts/python/stat-cpi.py +6 -4
@@ -1,5 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +from __future__ import print_function + data = {} times = [] threads = []
@@ -19,8 +21,8 @@ def store_key(time, cpu, thread): threads.append(thread) def store(time, event, cpu, thread, val, ena, run): - #print "event %s cpu %d, thread %d, time %d, val %d, ena %d, run %d" % \ - # (event, cpu, thread, time, val, ena, run) + #print("event %s cpu %d, thread %d, time %d, val %d, ena %d, run %d" % + # (event, cpu, thread, time, val, ena, run)) store_key(time, cpu, thread) key = get_key(time, event, cpu, thread)
@@ -58,7 +60,7 @@ def stat__interval(time): if ins != 0: cpi = cyc/float(ins) - print "%15f: cpu %d, thread %d -> cpi %f (%d/%d)" % (time/(float(1000000000)), cpu, thread, cpi, cyc, ins) + print("%15f: cpu %d, thread %d -> cpi %f (%d/%d)" % (time/(float(1000000000)), cpu, thread, cpi, cyc, ins)) def trace_end(): pass
@@ -74,4 +76,4 @@ def trace_end(): # if ins != 0: # cpi = cyc/float(ins) # -# print "time %.9f, cpu %d, thread %d -> cpi %f" % (time/(float(1000000000)), cpu, thread, cpi) +# print("time %.9f, cpu %d, thread %d -> cpi %f" % (time/(float(1000000000)), cpu, thread, cpi))
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 36/37 perf script python: Add Python3 support to syscall-counts.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the syscall-counts.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- tools/perf/scripts/python/syscall-counts.py | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
tools/perf/scripts/python/syscall-counts.py +10 -8
@@ -5,6 +5,8 @@ # Displays system-wide system call totals, broken down by syscall. # If a [comm] arg is specified, only syscalls called by [comm] are displayed. +from __future__ import print_function + import os import sys
@@ -28,7 +30,7 @@ if len(sys.argv) > 1: syscalls = autodict() def trace_begin(): - print "Press control+C to stop and show the summary" + print("Press control+C to stop and show the summary") def trace_end(): print_syscall_totals()
@@ -51,14 +53,14 @@ def syscalls__sys_enter(event_name, context, common_cpu, def print_syscall_totals(): if for_comm is not None: - print "\nsyscall events for %s:\n\n" % (for_comm), + print("\nsyscall events for %s:\n" % (for_comm)) else: - print "\nsyscall events:\n\n", + print("\nsyscall events:\n") - print "%-40s %10s\n" % ("event", "count"), - print "%-40s %10s\n" % ("----------------------------------------", \ - "-----------"), + print("%-40s %10s" % ("event", "count")) + print("%-40s %10s" % ("----------------------------------------", + "-----------")) - for id, val in sorted(syscalls.iteritems(), key = lambda(k, v): (v, k), \ + for id, val in sorted(syscalls.items(), key = lambda kv: (kv[1], kv[0]), \ reverse = True): - print "%-40s %10d\n" % (syscall_name(id), val), + print("%-40s %10d" % (syscall_name(id), val))
Signature · 1 line
2.20.1
Arnaldo Carvalho de Melo
maintainerPATCH 37/37 perf script python: Add Python3 support to syscall-counts-by-pid.py
In reply to Arnaldo Carvalho de MeloFrom: Tony Jones <[email protected]>
Support both Python2 and Python3 in the syscall-counts-by-pid.py script
There may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from __future__' implies the minimum supported Python2 version is now v2.6
--- .../scripts/python/syscall-counts-by-pid.py | 22 ++++++++++--------- 1 file changed, 12 insertions(+), 10 deletions(-)
tools/perf/scripts/python/syscall-counts-by-pid.py +12 -10
@@ -5,6 +5,8 @@ # Displays system-wide system call totals, broken down by syscall. # If a [comm] arg is specified, only syscalls called by [comm] are displayed. +from __future__ import print_function + import os, sys sys.path.append(os.environ['PERF_EXEC_PATH'] + \
@@ -31,7 +33,7 @@ if len(sys.argv) > 1: syscalls = autodict() def trace_begin(): - print "Press control+C to stop and show the summary" + print("Press control+C to stop and show the summary") def trace_end(): print_syscall_totals()
@@ -55,20 +57,20 @@ def syscalls__sys_enter(event_name, context, common_cpu, def print_syscall_totals(): if for_comm is not None: - print "\nsyscall events for %s:\n\n" % (for_comm), + print("\nsyscall events for %s:\n" % (for_comm)) else: - print "\nsyscall events by comm/pid:\n\n", + print("\nsyscall events by comm/pid:\n") - print "%-40s %10s\n" % ("comm [pid]/syscalls", "count"), - print "%-40s %10s\n" % ("----------------------------------------", \ - "----------"), + print("%-40s %10s" % ("comm [pid]/syscalls", "count")) + print("%-40s %10s" % ("----------------------------------------", + "----------")) comm_keys = syscalls.keys() for comm in comm_keys: pid_keys = syscalls[comm].keys() for pid in pid_keys: - print "\n%s [%d]\n" % (comm, pid), + print("\n%s [%d]" % (comm, pid)) id_keys = syscalls[comm][pid].keys() - for id, val in sorted(syscalls[comm][pid].iteritems(), \ - key = lambda(k, v): (v, k), reverse = True): - print " %-38s %10d\n" % (syscall_name(id), val), + for id, val in sorted(syscalls[comm][pid].items(), \ + key = lambda kv: (kv[1], kv[0]), reverse = True): + print(" %-38s %10d" % (syscall_name(id), val))
Signature · 1 line
2.20.1
Re: [PATCH 34/37] perf script python: Add Python3 support to stackcollapse.py
In reply to Arnaldo Carvalho de MeloQuoted 35 lines · Arnaldo Carvalho de Melo
From: Tony Jones <[email protected]>
Support both Python2 and Python3 in the
stackcollapse.pyscriptThere may be differences in the ordering of output lines due to differences in dictionary ordering etc. However the format within lines should be unchanged.
The use of 'from
__future__' implies the minimum supported Python2 version is now v2.6Signed-off-by Tony Jones <[email protected]>Cc Paolo Bonzini <[email protected]> <[email protected]>Signed-off-by Seeteena Thoufeek <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/scripts/python/stackcollapse.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)tools/perf/scripts/python/stackcollapse.py +2
-0@@ -19,6 +19,8 @@ # Written by Paolo Bonzini <[email protected]> # Based on Brendan Gregg's stackcollapse-perf.pl script. +from __future__ import print_function + import os import sys from collections import defaultdict
tools/perf/scripts/python/stackcollapse.py def process_event(param_dict): · line 127
Quoted 9 lines
tools/perf/scripts/python/stackcollapse.py +2
-3@@ -120,7 +122,6 @@ def process_event(param_dict): lines[stack_string] = lines[stack_string] + 1 def trace_end(): - list = lines.keys() - list.sort() + list = sorted(lines) for stack in list: - print "%s %d" % (stack, lines[stack]) + print("%s %d" % (stack, lines[stack]))
Quoted 151 lines · Arnaldo Carvalho de Melo
Hi Ingo,
Please consider pulling, this is on top of my previous pull
request, perf-core-for-mingo-5.1-20190220.- Arnaldo
Test results at the end of this message, as usual.
The following changes since commit
b4409ae112caa6315f6ee678e953b9fc93e6919c:perf tools: Make
rm_rf()remove single file (2019-02-20 17:09:28 -0300)are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.gittags/perf-core-for-mingo-5.1-20190225for you to fetch changes up to
de667cce7f4f96b6e22da8fd9c065b961f355080:perf script python: Add Python3 support to
syscall-counts-by-pid.py(2019-02-25 17:17:13 -0300)----------------------------------------------------------------
perf/core improvements and fixes:perf annotate:
Wei Li:
- Fix getting source line failure.
perf script:
Andi Kleen:
- Handle missing fields with -F +...
perf data:
Jiri Olsa:
- Prep work to support per-cpu files in a directory.
Intel PT:
Adrian Hunter:
- Improve
thread_stack__no_call_return()- Hide x86 retpolines in thread stacks.
- exported SQL viewer refactorings, new 'top calls' report.
Alexander Shishkin:
- Copy parent's address filter offsets on clone.
- Fix address filters for vmas with non-zero offset. Applies to ARM's CoreSight as well.
python scripts:
Tony Jones:
- Python3 support for several 'perf script' python scripts.
Signed-off-by Arnaldo Carvalho de Melo <[email protected]>----------------------------------------------------------------
Adrian Hunter (13):
perf thread-stack: Improvethread_stack__no_call_return()
perf thread-stack: Hide x86 retpolines
perf scripts python:exported-sql-viewer.py: Fix missing shebang
perf scripts python:exported-sql-viewer.py: Remove leftover debugging prints perf scripts python:exported-sql-viewer.py: Hide Call Graph option if no calls table perf scripts python:exported-sql-viewer.py: Move column headers
perf scripts python:exported-sql-viewer.py: Factor out ReportDialogBase perf scripts python:exported-sql-viewer.py: Factor out ReportVars perf scripts python:exported-sql-viewer.py: Move report name into ReportVars perf scripts python:exported-sql-viewer.py: Create new dialog data item classes perf scripts python:exported-sql-viewer.py: Remove SQLTableDialogDataItem perf scripts python:exported-sql-viewer.py: Remove no selection error perf scripts python:exported-sql-viewer.py: Add top calls reportAlexander Shishkin (2):
perf: Copy parent's address filter offsets on clone
perf, pt, coresight: Fix address filters for vmas with non-zero offsetAndi Kleen (2):
perf script: Handle missing fields with -F +.. perf tools: Addperf_exe()helper to find perf binaryJiri Olsa (9):
perf data: Move size to structperf_data_fileperf data: Add global path holder
perf tools: Add depth checking torm_rf
perf tools: Add pattern name checking torm_rfperf tools: Addrm_rf_perf_datafunction
perf data: Makecheck_backupwork over directories perf data: Failcheck_backupin case of error perf data: Addperf_data__(create_dir|close_dir)functions perf data: Addperf_data__open_dir_datafunctionTony Jones (10):
perf script python: Add Python3 support tonetdev-times.pyperf script python: Add Python3 support tofailed-syscalls-by-pid.pyperf script python: Add Python3 support tomem-phys-addr.pyperf script python: Add Python3 support tonet_dropmonitor.pyperf script python: Add Python3 support topowerpc-hcalls.pyperf script python: Add Python3 support tosctop.py
perf script python: Add Python3 support tostackcollapse.pyperf script python: Add Python3 support tostat-cpi.pyperf script python: Add Python3 support tosyscall-counts.pyperf script python: Add Python3 support tosyscall-counts-by-pid.pyWei Li (1):
perf annotate: Fix getting source line failurearch/x86/events/intel/pt.c | 9 +- drivers/hwtracing/coresight/coresight-etm-perf.c | 7 +- include/linux/perf_event.h | 7 +- kernel/events/core.c | 90 ++-- tools/perf/builtin-annotate.c | 4 +- tools/perf/builtin-buildid-cache.c | 4 +- tools/perf/builtin-buildid-list.c | 8 +- tools/perf/builtin-c2c.c | 4 +- tools/perf/builtin-diff.c | 12 +- tools/perf/builtin-evlist.c | 4 +- tools/perf/builtin-inject.c | 10 +- tools/perf/builtin-kmem.c | 2 +- tools/perf/builtin-kvm.c | 8 +- tools/perf/builtin-lock.c | 8 +- tools/perf/builtin-mem.c | 8 +- tools/perf/builtin-record.c | 11 +- tools/perf/builtin-report.c | 6 +- tools/perf/builtin-sched.c | 16 +- tools/perf/builtin-script.c | 22 +- tools/perf/builtin-stat.c | 6 +- tools/perf/builtin-timechart.c | 8 +- tools/perf/builtin-trace.c | 8 +- tools/perf/scripts/python/exported-sql-viewer.py | 510 ++++++++++++++------- .../perf/scripts/python/failed-syscalls-by-pid.py | 21 +- tools/perf/scripts/python/mem-phys-addr.py | 24 +- tools/perf/scripts/python/net_dropmonitor.py | 10 +- tools/perf/scripts/python/netdev-times.py | 82 ++-- tools/perf/scripts/python/powerpc-hcalls.py | 18 +- tools/perf/scripts/python/sctop.py | 24 +- tools/perf/scripts/python/stackcollapse.py | 7 +- tools/perf/scripts/python/stat-cpi.py | 10 +- tools/perf/scripts/python/syscall-counts-by-pid.py | 22 +- tools/perf/scripts/python/syscall-counts.py | 18 +- tools/perf/util/annotate.c | 4 +-
Quoted 8 lines
tools/perf/util/data-convert-bt.c | 4 +- tools/perf/util/data.c | 175 ++++++- tools/perf/util/data.h | 16 +- tools/perf/util/header.c | 12 +- tools/perf/util/thread-stack.c | 161 ++++++- tools/perf/util/util.c | 65 ++- tools/perf/util/util.h | 3 + 41 files changed, 1019 insertions(+), 429 deletions(-)
Pulled, thanks a lot Arnaldo!
Ingo