linux: build host perf (branch yem/linux-host-perf)
cover linux: build host perf (branch yem/linux-host-perf)
Hello All!
This short series make it possible to build a host perf.
When run on the target, perf may generate raw dumps. Those dumps are to be parsed by perf itself to gfenerate proper reports; this can (and should probably) be done off-line ona fast machine, rather than on the target.
First, we apply a number of fixes to the perf linux tool:
- really do not build the documentation
- honour the number of parallel jobs
- really disable unavailable features
- conditioanlly add support for gz and xz
Then, we build perf for the target in an empty directory, rather than in-place, so that we do not mix the target .o files with the host .o files.
Finally, we do build the host perf, alsao in its own directory.
Regards,
Yann E. MORIN.
The following changes since commit 0b7b84310c3781c346eea4c114c8cb4e97bbd8ea:
olimex_imx233_olinuxino_defconfig: genimage support (2016-03-11 13:14:54 +0100)
are available in the git repository at:
git://git.busybox.net/~ymorin/git/buildroot yem/linux-host-perf
for you to fetch changes up to b5f50fa3c517ddd37dbf339c011f6bf0621f55ac:
linux/perf: build the host perf tool (2016-03-11 18:11:58 +0100)
----------------------------------------------------------------
Yann E. MORIN (6):
linux/perf: really do not build the documentation linux/perf: honour the number of parallel jobs linux/perf: forcefully disable the features with missing dependencies linux/perf: conditionally enable support for gz/xz compression linux/perf: build in a sub-directory
linux/perf: build the host perf tool
linux/Config.tools.in | 4 +++ linux/linux-tool-perf.mk | 93 ++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 87 insertions(+), 10 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/6 linux/perf: really do not build the documentation
The perf buildsystem, inside the kernel, is not really amenable to be easily used...
Regarding the documentation, it will forcefully try to detect asciidoc and, with the latest versions, xmlto, completely disregarding what the user may provide.
We currently pass ASCIIDOC= (the empty string) on the make command line, as an attempt to disable building the documentation, but that has no effect whatsoever on perf: that variable is not passed down to the sub-sub-make (yes, a two-level depth) that is responsible for building the documentation.
We really do not want to build any of the documentation (the user can refer to the documentation on his own development machine), so we use a little dirty trick: we provide a GNUmakefile beside the existing Makefile for the documentation; GNUmakefile always takes precedence over a Makefile when both are present. We only provide a catch-all-no-recipe rule in that GNUmakefile, so it really does nothing useful, except avoid building the documentation.
--- linux/linux-tool-perf.mk | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
linux/linux-tool-perf.mk +14 -2
@@ -24,8 +24,7 @@ PERF_MAKE_FLAGS = \ NO_LIBPYTHON=1 \ DESTDIR=$(TARGET_DIR) \ prefix=/usr \ - WERROR=0 \ - ASCIIDOC= + WERROR=0 # We need to pass an argument to ld for setting the endianness when # building it for MIPS architecture, otherwise the default one will
@@ -68,6 +67,19 @@ else PERF_MAKE_FLAGS += NO_LIBELF=1 NO_DWARF=1 endif +# We really do not want to build the perf documentation, because it +# has stringent requirement on the documentation generation tools, +# like xmlto and asciidoc), which may be lagging behind on some +# distributions. +# We name it 'GNUmakefile' so that GNU make will use it instead of +# the existing 'Makefile'. +define PERF_DISABLE_DOCUMENTATION + if [ -f $(@D)/tools/perf/Documentation/Makefile ]; then \ + printf "%%:\n\t@:\n" >$(@D)/tools/perf/Documentation/GNUmakefile; \ + fi +endef +LINUX_POST_PATCH_HOOKS += PERF_DISABLE_DOCUMENTATION + # O must be redefined here to overwrite the one used by Buildroot for # out of tree build. We build perf in $(@D)/tools/perf/ and not just # $(@D) so that it isn't built in the root directory of the kernel
Signature · 1 line
1.9.1
PATCH 2/6 linux/perf: honour the number of parallel jobs
In reply to Yann E. MORINperf does not honour the -j flags we pass to make; it yet again tries to reinvent the wheel and by default uses the number of CPUs as the number of parallel jobs.
Fortunately, in their inifinite wisdom, the insane developpers of the perf buildsystem were kind enough to provide us with a variable we can set to specify the number of parallel jobs.
--- linux/linux-tool-perf.mk | 1 + 1 file changed, 1 insertion(+)
linux/linux-tool-perf.mk +1 -0
@@ -16,6 +16,7 @@ endif PERF_MAKE_FLAGS = \ $(LINUX_MAKE_FLAGS) \ + JOBS=$(PARALLEL_JOBS) \ ARCH=$(PERF_ARCH) \ NO_LIBAUDIT=1 \ NO_NEWT=1 \
Signature · 1 line
1.9.1
PATCH 3/6 linux/perf: forcefully disable the features with missing dependencies
In reply to Yann E. MORINForcefully disable the features that have optional dependencies that are not enabled in Buildroot.
Disable support for bionic since, well, we're not Android.
Slightly re-order the variables to have semantically-related variables together, with features last.
--- linux/linux-tool-perf.mk | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
linux/linux-tool-perf.mk +10 -3
@@ -18,14 +18,15 @@ PERF_MAKE_FLAGS = \ $(LINUX_MAKE_FLAGS) \ JOBS=$(PARALLEL_JOBS) \ ARCH=$(PERF_ARCH) \ + DESTDIR=$(TARGET_DIR) \ + prefix=/usr \ + WERROR=0 \ NO_LIBAUDIT=1 \ NO_NEWT=1 \ NO_GTK2=1 \ NO_LIBPERL=1 \ NO_LIBPYTHON=1 \ - DESTDIR=$(TARGET_DIR) \ - prefix=/usr \ - WERROR=0 + NO_LIBBIONIC=1 # We need to pass an argument to ld for setting the endianness when # building it for MIPS architecture, otherwise the default one will
@@ -52,14 +53,20 @@ endif ifeq ($(BR2_PACKAGE_SLANG),y) PERF_DEPENDENCIES += slang +else +PERF_MAKE_FLAGS += NO_SLANG=1 endif ifeq ($(BR2_PACKAGE_LIBUNWIND),y) PERF_DEPENDENCIES += libunwind +else +PERF_MAKE_FLAGS += NO_LIBUNWIND=1 endif ifeq ($(BR2_PACKAGE_NUMACTL),y) PERF_DEPENDENCIES += numactl +else +PERF_MAKE_FLAGS += NO_LIBNUMA=1 endif ifeq ($(BR2_PACKAGE_ELFUTILS),y)
Signature · 1 line
1.9.1
PATCH 4/6 linux/perf: conditionally enable support for gz/xz compression
In reply to Yann E. MORIN--- linux/linux-tool-perf.mk | 13 +++++++++++++ 1 file changed, 13 insertions(+)
linux/linux-tool-perf.mk +13 -0
@@ -75,6 +75,19 @@ else PERF_MAKE_FLAGS += NO_LIBELF=1 NO_DWARF=1 endif +ifeq ($(BR2_PACKAGE_ZLIB),y) +PERF_DEPENDENCIES += zlib +else +PERF_MAKE_FLAGS += NO_ZLIB=1 +endif + +# lzma is provided by xz +ifeq ($(BR2_PACKAGE_XZ),y) +PERF_DEPENDENCIES += xz +else +PERF_MAKE_FLAGS += NO_LZMA=1 +endif + # We really do not want to build the perf documentation, because it # has stringent requirement on the documentation generation tools, # like xmlto and asciidoc), which may be lagging behind on some
Signature · 1 line
1.9.1
PATCH 5/6 linux/perf: build in a sub-directory
In reply to Yann E. MORINCurrently, we only build perf for the target.
However, analysing some of the results gathered by perf (aka trace) may require running perf (aka trace) on the host, so we'll have to build perf for the host too.
To avoid mixing target and host binaries, we need to build them in separate directories.
This commit only moves the target build, in preparation of the host build to come later.
--- linux/linux-tool-perf.mk | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
linux/linux-tool-perf.mk +3 -2
@@ -120,15 +120,16 @@ define PERF_BUILD_CMDS fi \ fi \ fi + $(Q)mkdir -p $(@D)/tools/perf/br-target $(TARGET_MAKE_ENV) $(MAKE1) $(PERF_MAKE_FLAGS) \ - -C $(@D)/tools/perf O=$(@D)/tools/perf/ + -C $(@D)/tools/perf O=$(@D)/tools/perf/br-target endef # After installation, we remove the Perl and Python scripts from the # target. define PERF_INSTALL_TARGET_CMDS $(TARGET_MAKE_ENV) $(MAKE1) $(PERF_MAKE_FLAGS) \ - -C $(@D)/tools/perf O=$(@D)/tools/perf/ install + -C $(@D)/tools/perf O=$(@D)/tools/perf/br-target install $(RM) -rf $(TARGET_DIR)/usr/libexec/perf-core/scripts/ $(RM) -rf $(TARGET_DIR)/usr/libexec/perf-core/tests/ endef
Signature · 1 line
1.9.1
PATCH 6/6 linux/perf: build the host perf tool
In reply to Yann E. MORINCurrently, we only build the target variant of the perf tool. However, perf on the target may generate a bunch of data files that may have to be analysed on the host.
There is no host-variant of the linux-tools infrastructure, so we just build the host perf at the same time we build the target one.
--- linux/Config.tools.in | 4 ++++ linux/linux-tool-perf.mk | 51 ++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 49 insertions(+), 6 deletions(-)
linux/Config.tools.in +4 -0
@@ -26,4 +26,8 @@ config BR2_LINUX_KERNEL_TOOL_PERF https://perf.wiki.kernel.org/ +config BR2_LINUX_KERNEL_TOOL_HOST_PERF + bool "host-perf" + depends on BR2_LINUX_KERNEL_TOOL_PERF + endmenu
linux/linux-tool-perf.mk +45 -6
91 lines not on this page · raw message
Signature · 1 line
1.9.1
PATCH 6/6 linux/perf: build the host perf tool
Yann,
Quoted 9 lines · Yann E. MORIN
Currently, we only build the target variant of the perf tool. However, perf on the target may generate a bunch of data files that may have to be analysed on the host.
There is no host-variant of the linux-tools infrastructure, so we just build the host perf at the same time we build the target one.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Thomas De Schampheleire <[email protected]>
First of all, I have to say I hate this target package that builds and installs host stuff. But, since I don't have a better solution to offer, and it's anyway not the only package in this case (qt/qt5 are also target packages, and they install stuff in host), I'm fine with the general approach.
I have some comments about the details, though.
Quoted 10 lines
--- linux/Config.tools.in | 4 ++++ linux/linux-tool-perf.mk | 51 ++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 49 insertions(+), 6 deletions(-)linux/Config.tools.in +0
-0
linux/Config.tools.in config BR2_LINUX_KERNEL_TOOL_PERF · line 31
Quoted 6 lines
linux/Config.tools.in +3
-0@@ -26,4 +26,8 @@ config BR2_LINUX_KERNEL_TOOL_PERF https://perf.wiki.kernel.org/ +config BR2_LINUX_KERNEL_TOOL_HOST_PERF + bool "host-perf" + depends on BR2_LINUX_KERNEL_TOOL_PERF
Indentation should just tab.
linux/Config.tools.in config BR2_LINUX_KERNEL_TOOL_PERF · line 33
Quoted 2 lines
linux/Config.tools.in +2
-0+# For the host, we inherit the same flags as for the target, +# overriding just those that are needed.
I don't really understand this, it seems really weird to me, and I would find it much more future proof to have separate target and host flags. Otherwise, any addition of a flag to the target variant can break in subtle way the host variant.
linux/Config.tools.in config BR2_LINUX_KERNEL_TOOL_PERF
Quoted 3 lines
linux/Config.tools.in +3
-0+HOST_PERF_MAKE_FLAGS = \ + $(PERF_MAKE_FLAGS) \ + CFLAGS="$(HOST_CFLAGS) -I$(HOST_DIR)/usr/include/elfutils" \
We don't have this -I for the target variant. Why do we need it for the host variant?
linux/Config.tools.in
Quoted 9 lines
linux/Config.tools.in +9
-0+ LDFLAGS="$(HOST_LDFLAGS)" \ + LD="$(HOSTLD)" \ + ARCH=$(PERF_ARCH) \ + CROSS_COMPILE= \ + DESTDIR=$(HOST_DIR) + +# No host-packages in Buildroot, but we don't let it depend on the +# libraries provided by the host system (they may be there or not, +# depending on the user's system, but we won't play that game).
This sentence is not really clear IMO. What about:
# For the following optional dependencies of perf, Buildroot doesn't
# provide any host package, and we don't want to rely on those # libraries being provided by the host system.
Quoted 10 lines
linux/Config.tools.in +6
-0+HOST_PERF_MAKE_FLAGS += \ + NO_SLANG=1 \ + NO_LIBUNWIND=1 \ + NO_LIBNUMA=1 \ + NO_LIBELF=1 NO_DWARF=1 + # The call to backtrace() function fails for ARC, because for some # reason the unwinder from libgcc returns early. Thus the usage of # backtrace() should be disabled in perf explicitly: at build time
linux/Config.tools.in endif · line 72
Quoted 4 lines
linux/Config.tools.in +1
-0@@ -49,6 +69,7 @@ endif # instead of the complete backtrace. ifeq ($(BR2_arc),y) PERF_MAKE_FLAGS += NO_BACKTRACE=1 +HOST_PERF_MAKE_FLAGS += NO_BACKTRACE=1
Why ?
linux/Config.tools.in PERF_MAKE_FLAGS += NO_LIBELF=1 NO_DWARF=1 · line 100
Quoted 9 lines
linux/Config.tools.in +1
-1endif ifeq ($(BR2_PACKAGE_SLANG),y)@@ -76,14 +97,14 @@ PERF_MAKE_FLAGS += NO_LIBELF=1 NO_DWARF=1 endif ifeq ($(BR2_PACKAGE_ZLIB),y) -PERF_DEPENDENCIES += zlib +PERF_DEPENDENCIES += zlib host-zlib
This is the part I'm really not a big fan of. You enable a target package, and it affects the configuration of a host package. Not nice.
Is there an actual connection between zlib/xz support in the target perf and the need to have it in host-perf? Like is zlib/xz support used to compress the data output by the target perf, which needs to be uncompressed by the host perf ?
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH 6/6 linux/perf: build the host perf tool
Thomas, All,
On 2016-03-19 14:55 +0100, Thomas Petazzoni spake thusly:
Quoted 5 lines · Yann E. MORIN
Quoted 4 lines
Currently, we only build the target variant of the perf tool. However, perf on the target may generate a bunch of data files that may have to be analysed on the host.
Quoted 8 lines
Quoted 5 lines
There is no host-variant of the linux-tools infrastructure, so we just build the host perf at the same time we build the target one.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Thomas De Schampheleire <[email protected]>First of all, I have to say I hate this target package that builds and installs host stuff.
Yeah, that bothered me too, enough that I had to explain it in the commit log.
[--SNIP--]
Quoted 6 lines
Quoted 6 lines
linux/Config.tools.in +0
-0@@ -26,4 +26,8 @@ config BR2_LINUX_KERNEL_TOOL_PERF
Quoted 6 lines
linux/Config.tools.in config BR2_LINUX_KERNEL_TOOL_PERF · line 31
Quoted 5 lines
linux/Config.tools.in +3
-0https://perf.wiki.kernel.org/ +config BR2_LINUX_KERNEL_TOOL_HOST_PERF + bool "host-perf" + depends on BR2_LINUX_KERNEL_TOOL_PERFIndentation should just tab.
Damned... :-)
Quoted 7 lines
linux/Config.tools.in config BR2_LINUX_KERNEL_TOOL_PERF · line 33
Quoted 2 lines
linux/Config.tools.in +2
-0+# For the host, we inherit the same flags as for the target, +# overriding just those that are needed.I don't really understand this, it seems really weird to me, and I would find it much more future proof to have separate target and host flags. Otherwise, any addition of a flag to the target variant can break in subtle way the host variant.
Well, I'm not a perf expert, but my reasoning is that, if the target perf stores info about, say, function unwinding, the host perf has to be able to parse it. Conversely, if the target perf does not support tracing function unwinding, there is no point in having that support in the host perf.
So, we just use the same set of options, except for the compiler suite and the destination directory.
Quoted 6 lines
linux/Config.tools.in config BR2_LINUX_KERNEL_TOOL_PERF
Quoted 3 lines
linux/Config.tools.in +3
-0+HOST_PERF_MAKE_FLAGS = \ + $(PERF_MAKE_FLAGS) \ + CFLAGS="$(HOST_CFLAGS) -I$(HOST_DIR)/usr/include/elfutils" \We don't have this -I for the target variant. Why do we need it for the host variant?
I can retry without it, but IIRC it did not work. I'll double-check.
Quoted 12 lines
linux/Config.tools.in
Quoted 9 lines
linux/Config.tools.in +9
-0+ LDFLAGS="$(HOST_LDFLAGS)" \ + LD="$(HOSTLD)" \ + ARCH=$(PERF_ARCH) \ + CROSS_COMPILE= \ + DESTDIR=$(HOST_DIR) + +# No host-packages in Buildroot, but we don't let it depend on the +# libraries provided by the host system (they may be there or not, +# depending on the user's system, but we won't play that game).This sentence is not really clear IMO. What about:
Quoted 3 lines
# For the following optional dependencies of perf, Buildroot doesn't
# provide any host package, and we don't want to rely on those # libraries being provided by the host system.
Yeah, better. Thanks!
Quoted 8 lines
Quoted 8 lines
linux/Config.tools.in +6
-0+HOST_PERF_MAKE_FLAGS += \ + NO_SLANG=1 \ + NO_LIBUNWIND=1 \ + NO_LIBNUMA=1 \ + NO_LIBELF=1 NO_DWARF=1 + # The call to backtrace() function fails for ARC, because for some # reason the unwinder from libgcc returns early. Thus the usage of
Quoted 8 lines
linux/Config.tools.in endif · line 72
Quoted 6 lines
#
backtrace()should be disabled in perf explicitly: at build timelinux/Config.tools.in +1
-0@@ -49,6 +69,7 @@ endif # instead of the complete backtrace. ifeq ($(BR2_arc),y) PERF_MAKE_FLAGS += NO_BACKTRACE=1 +HOST_PERF_MAKE_FLAGS += NO_BACKTRACE=1Why ?
Ditto the explanation about inheriting the target options, above.
Quoted 6 lines
Quoted 6 lines
linux/Config.tools.in +0
-0endif ifeq ($(BR2_PACKAGE_SLANG),y)@@ -76,14 +97,14 @@ PERF_MAKE_FLAGS += NO_LIBELF=1 NO_DWARF=1 endif
Quoted 6 lines
linux/Config.tools.in PERF_MAKE_FLAGS += NO_LIBELF=1 NO_DWARF=1 · line 100
Quoted 3 lines
linux/Config.tools.in +1
-1ifeq ($(BR2_PACKAGE_ZLIB),y) -PERF_DEPENDENCIES += zlib +PERF_DEPENDENCIES += zlib host-zlibThis is the part I'm really not a big fan of. You enable a target package, and it affects the configuration of a host package. Not nice.
Ditto, to match what the target perf supports.
However, sicne there is no host variant, there's not much we can do to clean this mess. And I doubt we want to add a host-linux package just for host-perf.
Quoted 4 lines
Is there an actual connection between zlib/xz support in the target perf and the need to have it in host-perf? Like is zlib/xz support used to compress the data output by the target perf, which needs to be uncompressed by the host perf ?
TBH, I did not try. I was secretly hoping that Thomas DS would test-run it... (Niark!)
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 6/6 linux/perf: build the host perf tool
In reply to Thomas PetazzoniQuoted 8 lines · Thomas Petazzoni
Quoted 7 lines · Yann E. MORIN
Quoted 7 lines
Currently, we only build the target variant of the perf tool. However, perf on the target may generate a bunch of data files that may have to be analysed on the host.
There is no host-variant of the linux-tools infrastructure, so we just build the host perf at the same time we build the target one.
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Signed-off-by "Yann E. MORIN"<[email protected]>Cc Thomas De Schampheleire<[email protected]>First of all, I have to say I hate this target package that builds and installs host stuff. But, since I don't have a better solution to offer, and it's anyway not the only package in this case (qt/qt5 are also target packages, and they install stuff in host), I'm fine with the general approach.
I think it's OK to build and install host tools that are bound tightly with
the target package. E.g. qmake only makes sense when you have qt for the target. perf is in the same boat.
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
cover linux: build host perf (branch yem/linux-host-perf)
In reply to Yann E. MORINHello,
Quoted 4 lines · Yann E. MORIN
linux/perf: really do not build the documentation linux/perf: honour the number of parallel jobs linux/perf: forcefully disable the features with missing dependencies linux/perf: conditionally enable support for gz/xz compression
I've applied those four first patches.
Quoted 1 line
linux/perf: build in a sub-directory
This one looks OK to me, but it doesn't make much sense applying it without applying the last patch, on which I have comments.
Quoted 1 line
linux/perf: build the host perf tool
I'll reply on this one.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com