Skip to contentSkip to conversation

Fixes for perf build and feature detection

PATCH Applied David Carrillo-Cisneros 5 participants 21 messages Feb 2, 2017 – Feb 10, 2017 perf events

lore
Share & tools
lore
#0 lore raw

cover Fixes for perf build and feature detection

Fixes for minor problems with Makefiles and libs that I found while building perf.

David Carrillo-Cisneros (4):
perf tools: pass PYTHON config to feature detection tools lib traceevent: Robustify do_generate_dynamic_list_file tools lib feature: Do not redefine compiler configuration tools include: Fix include path for uapi/asm-generic/mman.h

 tools/build/feature/Makefile    |  8 ++++----
 tools/include/uapi/linux/mman.h |  2 +-
 tools/lib/traceevent/Makefile   | 13 +++++++++----
 tools/perf/Makefile.config      | 23 ++++++++++-------------
 4 files changed, 24 insertions(+), 22 deletions(-)
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#1 lore raw

PATCH 1/4 perf tools: pass PYTHON config to feature detection

Python's CC and link Makefile variables were not passed to feature detection, causing feature detection to use system's Python rather than PYTHON_CONFIG's one. This created a mismatch between the detected Python support and the one actually used by perf when PYTHON_CONFIG is specified.

Fix it by moving Python's variable initialization to before feature detection and pass FLAGS_PYTHON_EMBED to Python's feature detection's build target.

---
 tools/build/feature/Makefile |  2 +-
 tools/perf/Makefile.config   | 23 ++++++++++-------------
 2 files changed, 11 insertions(+), 14 deletions(-)
tools/build/feature/Makefile +1 -1
index b564a2eea039..3dd2c600d250 100644
@@ -171,7 +171,7 @@ $(OUTPUT)test-libperl.bin:
 	$(BUILD) $(FLAGS_PERL_EMBED)
 
 $(OUTPUT)test-libpython.bin:
-	$(BUILD)
+	$(BUILD) $(FLAGS_PYTHON_EMBED)
 
 $(OUTPUT)test-libpython-version.bin:
 	$(BUILD)
tools/perf/Makefile.config +10 -13
index 03cf947755b9..41f5c41c5660 100644
@@ -166,10 +166,18 @@ PYTHON2_CONFIG := \
 override PYTHON_CONFIG := \
   $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON2_CONFIG))
 
+grep-libs  = $(filter -l%,$(1))
+strip-libs  = $(filter-out -l%,$(1))
+
 PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
 
-PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
-PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
+ifdef PYTHON_CONFIG
+  PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
+  PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
+  PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
+  PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
+  FLAGS_PYTHON_EMBED := $(PYTHON_EMBED_CCOPTS) $(PYTHON_EMBED_LDOPTS)
+endif
 
 FEATURE_CHECK_CFLAGS-libpython := $(PYTHON_EMBED_CCOPTS)
 FEATURE_CHECK_LDFLAGS-libpython := $(PYTHON_EMBED_LDOPTS)
@@ -542,8 +550,6 @@ ifndef NO_GTK2
   endif
 endif
 
-grep-libs  = $(filter -l%,$(1))
-strip-libs = $(filter-out -l%,$(1))
 
 ifdef NO_LIBPERL
   CFLAGS += -DNO_LIBPERL
@@ -591,18 +597,9 @@ else
       $(call disable-python,No 'python-config' tool was found: disables Python support - please install python-devel/python-dev)
     else
 
-      PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
-
-      PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
-      PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
-      PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
-      PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
-      FLAGS_PYTHON_EMBED := $(PYTHON_EMBED_CCOPTS) $(PYTHON_EMBED_LDOPTS)
-
       ifneq ($(feature-libpython), 1)
         $(call disable-python,No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev)
       else
-
         ifneq ($(feature-libpython-version), 1)
           $(warning Python 3 is not yet supported; please set)
           $(warning PYTHON and/or PYTHON_CONFIG appropriately.)
Signature · 1 line
2.11.0.483.g087da7b7c-goog
maintainer Tested-by Thomas Ilsche on v1 1/4
#2 lore raw

Re: [PATCH 1/4] perf tools: pass PYTHON config to feature detection

Em Wed, Feb 01, 2017 at 10:38:01PM -0800, David Carrillo-Cisneros escreveu:

Quoted 8 lines

Python's CC and link Makefile variables were not passed to feature detection, causing feature detection to use system's Python rather than PYTHON_CONFIG's one. This created a mismatch between the detected Python support and the one actually used by perf when PYTHON_CONFIG is specified.

Fix it by moving Python's variable initialization to before feature detection and pass FLAGS_PYTHON_EMBED to Python's feature detection's build target.

Hi Namhyung, Thomas, since this was last touched in the cset below, can you please check this if possible and provide Acked-by tags?

Thanks,

commit 56c7d79e49776084b852e451bda5f59dc3bcf894

Author: Namhyung Kim <[email protected]>
Date:   Tue Jul 29 15:57:19 2014 +0900

perf tools: Fix make PYTHON override

Thomas reported that make PYTHON=python2 is not work on some systems. I can reproduce it on my ArchLinux box too.

This is because it's overridden by config/feature-checks/Makefile regardless of PYTHON setting. I guess it's a bug slipped into during the feature checking change.

Actually, we don't need to check python-config in the feature-checks. We can just pass appropriate FEATURE_CHECK_*FLAGS.

Quoted 74 lines
---
 tools/build/feature/Makefile |  2 +-
 tools/perf/Makefile.config   | 23 ++++++++++-------------
 2 files changed, 11 insertions(+), 14 deletions(-)
tools/build/feature/Makefile +1 -1
index b564a2eea039..3dd2c600d250 100644
@@ -171,7 +171,7 @@ $(OUTPUT)test-libperl.bin:
 	$(BUILD) $(FLAGS_PERL_EMBED)
 
 $(OUTPUT)test-libpython.bin:
-	$(BUILD)
+	$(BUILD) $(FLAGS_PYTHON_EMBED)
 
 $(OUTPUT)test-libpython-version.bin:
 	$(BUILD)
tools/perf/Makefile.config +10 -13
index 03cf947755b9..41f5c41c5660 100644
@@ -166,10 +166,18 @@ PYTHON2_CONFIG := \
 override PYTHON_CONFIG := \
   $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON2_CONFIG))
 
+grep-libs  = $(filter -l%,$(1))
+strip-libs  = $(filter-out -l%,$(1))
+
 PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
 
-PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
-PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
+ifdef PYTHON_CONFIG
+  PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
+  PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
+  PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
+  PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
+  FLAGS_PYTHON_EMBED := $(PYTHON_EMBED_CCOPTS) $(PYTHON_EMBED_LDOPTS)
+endif
 
 FEATURE_CHECK_CFLAGS-libpython := $(PYTHON_EMBED_CCOPTS)
 FEATURE_CHECK_LDFLAGS-libpython := $(PYTHON_EMBED_LDOPTS)
@@ -542,8 +550,6 @@ ifndef NO_GTK2
   endif
 endif
 
-grep-libs  = $(filter -l%,$(1))
-strip-libs = $(filter-out -l%,$(1))
 
 ifdef NO_LIBPERL
   CFLAGS += -DNO_LIBPERL
@@ -591,18 +597,9 @@ else
       $(call disable-python,No 'python-config' tool was found: disables Python support - please install python-devel/python-dev)
     else
 
-      PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
-
-      PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
-      PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
-      PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
-      PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
-      FLAGS_PYTHON_EMBED := $(PYTHON_EMBED_CCOPTS) $(PYTHON_EMBED_LDOPTS)
-
       ifneq ($(feature-libpython), 1)
         $(call disable-python,No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev)
       else
-
         ifneq ($(feature-libpython-version), 1)
           $(warning Python 3 is not yet supported; please set)
           $(warning PYTHON and/or PYTHON_CONFIG appropriately.)
Signature · 1 line
2.11.0.483.g087da7b7c-goog
maintainer
#3 lore raw

Re: [PATCH 1/4] perf tools: pass PYTHON config to feature detection

Quoted 78 lines · Arnaldo Carvalho de Melo

Em Wed, Feb 01, 2017 at 10:38:01PM -0800, David Carrillo-Cisneros escreveu:

Quoted 8 lines

Python's CC and link Makefile variables were not passed to feature detection, causing feature detection to use system's Python rather than PYTHON_CONFIG's one. This created a mismatch between the detected Python support and the one actually used by perf when PYTHON_CONFIG is specified.

Fix it by moving Python's variable initialization to before feature detection and pass FLAGS_PYTHON_EMBED to Python's feature detection's build target.

Hi Namhyung, Thomas, since this was last touched in the cset below, can you please check this if possible and provide Acked-by tags?

Thanks,

commit 56c7d79e49776084b852e451bda5f59dc3bcf894

Author: Namhyung Kim <[email protected]>
Date:   Tue Jul 29 15:57:19 2014 +0900

perf tools: Fix make PYTHON override

Thomas reported that make PYTHON=python2 is not work on some systems. I can reproduce it on my ArchLinux box too.

This is because it's overridden by config/feature-checks/Makefile regardless of PYTHON setting. I guess it's a bug slipped into during the feature checking change.

Actually, we don't need to check python-config in the feature-checks. We can just pass appropriate FEATURE_CHECK_*FLAGS.

Quoted 42 lines
---
 tools/build/feature/Makefile |  2 +-
 tools/perf/Makefile.config   | 23 ++++++++++-------------
 2 files changed, 11 insertions(+), 14 deletions(-)
tools/build/feature/Makefile +1 -1
index b564a2eea039..3dd2c600d250 100644
@@ -171,7 +171,7 @@ $(OUTPUT)test-libperl.bin:
 	$(BUILD) $(FLAGS_PERL_EMBED)
 
 $(OUTPUT)test-libpython.bin:
-	$(BUILD)
+	$(BUILD) $(FLAGS_PYTHON_EMBED)
 
 $(OUTPUT)test-libpython-version.bin:
 	$(BUILD)
tools/perf/Makefile.config +10 -2
index 03cf947755b9..41f5c41c5660 100644
@@ -166,10 +166,18 @@ PYTHON2_CONFIG := \
 override PYTHON_CONFIG := \
   $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON2_CONFIG))
 
+grep-libs  = $(filter -l%,$(1))
+strip-libs  = $(filter-out -l%,$(1))
+
 PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
 
-PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
-PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
+ifdef PYTHON_CONFIG
+  PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
+  PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
+  PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
+  PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
+  FLAGS_PYTHON_EMBED := $(PYTHON_EMBED_CCOPTS) $(PYTHON_EMBED_LDOPTS)
+endif
 
Quoted 2 lines

tools/perf/Makefile.config PYTHON2_CONFIG := \ · line 183

Quoted 2 lines
tools/perf/Makefile.config +0 -0
 FEATURE_CHECK_CFLAGS-libpython := $(PYTHON_EMBED_CCOPTS)
 FEATURE_CHECK_LDFLAGS-libpython := $(PYTHON_EMBED_LDOPTS)

Hmm.. I guess these feature check flags should be passed to the above Makefile but it seems not.

Thanks,
Namhyung

Quoted 30 lines
Quoted 30 lines
tools/perf/Makefile.config +0 -11
@@ -542,8 +550,6 @@ ifndef NO_GTK2
   endif
 endif
 
-grep-libs  = $(filter -l%,$(1))
-strip-libs = $(filter-out -l%,$(1))
 
 ifdef NO_LIBPERL
   CFLAGS += -DNO_LIBPERL
@@ -591,18 +597,9 @@ else
       $(call disable-python,No 'python-config' tool was found: disables Python support - please install python-devel/python-dev)
     else
 
-      PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
-
-      PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
-      PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
-      PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
-      PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
-      FLAGS_PYTHON_EMBED := $(PYTHON_EMBED_CCOPTS) $(PYTHON_EMBED_LDOPTS)
-
       ifneq ($(feature-libpython), 1)
         $(call disable-python,No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev)
       else
-
         ifneq ($(feature-libpython-version), 1)
           $(warning Python 3 is not yet supported; please set)
           $(warning PYTHON and/or PYTHON_CONFIG appropriately.)
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#4 lore raw

Re: [PATCH 1/4] perf tools: pass PYTHON config to feature detection

In reply to David Carrillo-Cisneros

Em Wed, Feb 01, 2017 at 10:38:01PM -0800, David Carrillo-Cisneros escreveu:

Quoted 8 lines

Python's CC and link Makefile variables were not passed to feature detection, causing feature detection to use system's Python rather than PYTHON_CONFIG's one. This created a mismatch between the detected Python support and the one actually used by perf when PYTHON_CONFIG is specified.

Fix it by moving Python's variable initialization to before feature detection and pass FLAGS_PYTHON_EMBED to Python's feature detection's build target.

So, all builds on Ubuntu with the devel files for perf end up with:

ubuntu:12.04.5, 14.04.4, 16.04, 16.10 (cross builds, native ones)

GEN /tmp/build/perf/libtraceevent-dynamic-list
/bin/sh: 1: [: U: unexpected operator
Either missing one of [ /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so] or bad version of nm

Details:

perfbuilder@341e71da5019:/git/linux$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="16.10 (Yakkety Yak)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.10"
VERSION_ID="16.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="http://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=yakkety
UBUNTU_CODENAME=yakkety
perfbuilder@341e71da5019:/git/linux$ ls -la /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 13576 Feb  7 19:38 /tmp/build/perf/plugin_cfg80211.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 20256 Feb  7 19:38 /tmp/build/perf/plugin_function.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 13680 Feb  7 19:38 /tmp/build/perf/plugin_hrtimer.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 13760 Feb  7 19:38 /tmp/build/perf/plugin_jbd2.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 13976 Feb  7 19:38 /tmp/build/perf/plugin_kmem.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 28680 Feb  7 19:38 /tmp/build/perf/plugin_kvm.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 14248 Feb  7 19:38 /tmp/build/perf/plugin_mac80211.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 18800 Feb  7 19:38 /tmp/build/perf/plugin_sched_switch.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 20136 Feb  7 19:38 /tmp/build/perf/plugin_scsi.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 14504 Feb  7 19:38 /tmp/build/perf/plugin_xen.so
perfbuilder@341e71da5019:/git/linux$
perfbuilder@341e71da5019:/git/linux$ file=/tmp/build/perf/plugin_jbd2.so ; nm -u -D $file | awk 'NF>1 {print $$file}' | sort -u 
                 U pevent_register_print_function
                 U pevent_unregister_print_function
                 U trace_seq_printf
perfbuilder@341e71da5019:/git/linux$

Now running with "make V=1 ...".

make -C /git/linux/tools/lib/traceevent/ plugin_dir= O=/tmp/build/perf/ /tmp/build/perf/libtraceevent-dynamic-list
symbol_type=`nm -u -D /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so | awk 'NF>1 {print $1}' | sort -u`; if [ "$symbol_type" == "U" ];then (echo '{'; nm -u -D /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so | awk 'NF>1 {print "\t"$2";"}' | sort -u; echo '};'; ) > /tmp/build/perf/libtraceevent-dynamic-list; else (echo Either missing one of [ /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so] or bad version of nm) 1>&2; fi /bin/sh: 1: [: U: unexpected operator
Either missing one of [ /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so] or bad version of nm

The output below looks like the same as on fedora25, where this seems to work:

perfbuilder@341e71da5019:/git/linux$ nm -u -D  /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so

/tmp/build/perf/plugin_jbd2.so:
U pevent_register_print_function U pevent_unregister_print_function U trace_seq_printf

/tmp/build/perf/plugin_hrtimer.so:
U pevent_print_func_field U pevent_print_num_field U pevent_register_event_handler U pevent_unregister_event_handler U trace_seq_printf

/tmp/build/perf/plugin_kmem.so:
U __stack_chk_fail
U pevent_find_field
U pevent_find_function
U pevent_find_function_address U pevent_read_number_field U pevent_register_event_handler U pevent_unregister_event_handler U trace_seq_printf

/tmp/build/perf/plugin_kvm.so:
U __stack_chk_fail
U pevent_get_field_raw
U pevent_get_field_val
U pevent_print_num_field
U pevent_register_event_handler U pevent_register_print_function U pevent_unregister_event_handler U pevent_unregister_print_function U sprintf
U trace_seq_printf

/tmp/build/perf/plugin_mac80211.so:
U __stack_chk_fail
U pevent_find_field
U pevent_print_num_field U pevent_read_number_field U pevent_register_event_handler U pevent_unregister_event_handler U strncmp
U trace_seq_printf
U trace_seq_putc

/tmp/build/perf/plugin_sched_switch.so:
U __stack_chk_fail
U pevent_find_any_field
U pevent_get_field_val
U pevent_register_comm
U pevent_register_event_handler U pevent_unregister_event_handler U trace_seq_printf
U trace_seq_putc
U trace_seq_puts
U trace_seq_terminate

/tmp/build/perf/plugin_function.so:
U __stack_chk_fail
U free
U memset
U pevent_find_function
U pevent_get_field_val
U pevent_register_event_handler U pevent_unregister_event_handler U realloc
U strcmp
U strdup
U trace_seq_printf
U trace_seq_putc
U traceevent_plugin_add_options U traceevent_plugin_remove_options U warning

/tmp/build/perf/plugin_xen.so:
U pevent_register_print_function U pevent_unregister_print_function U trace_seq_printf

/tmp/build/perf/plugin_scsi.so:
U pevent_register_print_function U pevent_unregister_print_function U trace_seq_printf
U trace_seq_putc

/tmp/build/perf/plugin_cfg80211.so:
U pevent_register_print_function U pevent_unregister_print_function
perfbuilder@341e71da5019:/git/linux$

I've run out of time this afternoon to look at this, will continue later or tomorrow, unless, preferrably, that you find this problem first :-)

- Arnaldo

#5 lore raw

Re: [PATCH 1/4] perf tools: pass PYTHON config to feature detection

Quoted 46 lines · Arnaldo Carvalho de Melo

Em Wed, Feb 01, 2017 at 10:38:01PM -0800, David Carrillo-Cisneros escreveu:

Quoted 8 lines

Python's CC and link Makefile variables were not passed to feature detection, causing feature detection to use system's Python rather than PYTHON_CONFIG's one. This created a mismatch between the detected Python support and the one actually used by perf when PYTHON_CONFIG is specified.

Fix it by moving Python's variable initialization to before feature detection and pass FLAGS_PYTHON_EMBED to Python's feature detection's build target.

So, all builds on Ubuntu with the devel files for perf end up with:

ubuntu:12.04.5, 14.04.4, 16.04, 16.10 (cross builds, native ones)

GEN /tmp/build/perf/libtraceevent-dynamic-list
/bin/sh: 1: [: U: unexpected operator
Either missing one of [ /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so] or bad version of nm

Details:

perfbuilder@341e71da5019:/git/linux$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.10 (Yakkety Yak)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.10"
VERSION_ID="16.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="http://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=yakkety
UBUNTU_CODENAME=yakkety
perfbuilder@341e71da5019:/git/linux$ ls -la /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so
(quoted diff) +0 -10
-rwxr-xr-x. 1 perfbuilder perfbuilder 13576 Feb  7 19:38 /tmp/build/perf/plugin_cfg80211.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 20256 Feb  7 19:38 /tmp/build/perf/plugin_function.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 13680 Feb  7 19:38 /tmp/build/perf/plugin_hrtimer.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 13760 Feb  7 19:38 /tmp/build/perf/plugin_jbd2.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 13976 Feb  7 19:38 /tmp/build/perf/plugin_kmem.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 28680 Feb  7 19:38 /tmp/build/perf/plugin_kvm.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 14248 Feb  7 19:38 /tmp/build/perf/plugin_mac80211.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 18800 Feb  7 19:38 /tmp/build/perf/plugin_sched_switch.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 20136 Feb  7 19:38 /tmp/build/perf/plugin_scsi.so
-rwxr-xr-x. 1 perfbuilder perfbuilder 14504 Feb  7 19:38 /tmp/build/perf/plugin_xen.so

perfbuilder@341e71da5019:/git/linux$

Quoted 7 lines
perfbuilder@341e71da5019:/git/linux$ file=/tmp/build/perf/plugin_jbd2.so ; nm -u -D $file | awk 'NF>1 {print $$file}' | sort -u
                 U pevent_register_print_function
                 U pevent_unregister_print_function
                 U trace_seq_printf
perfbuilder@341e71da5019:/git/linux$

Thanks for testing it.

I found that some *.so files in Ubuntu have the "w" value in nm's otput symbol type. From man nm:

"W"
"w" The symbol is a weak symbol that has not been
specifically tagged as a weak object symbol. When a
weak defined symbol is linked with a normal defined
symbol, the normal defined symbol is used with
no error. When a weak undefined symbol is linked and
the symbol is not defined, the value of the
symbol is determined in a system-specific manner
without error. On some systems, uppercase
indicates that a default value has been specified.

I'll send an updated patch that expects and includes undefined weak symbols.

Thanks,
David

#6 lore raw

PATCH v2 1/1 tools lib traceevent: Robustify do_generate_dynamic_list_file

v2: Accept "W" and "w" symbol options.

The dynamic-list-file used to export dynamic symbols introduced in

commit e3d09ec8126f ("tools lib traceevent: Export dynamic symbols used by traceevent plugins")

is generated without any sort of error checking.

I experienced problems due to an old version of nm (v 0.158) that outputs in a format distinct from the assumed by the script.

Robustify the built of dynamic symbol list by enforcing that the second column of $(NM) -u <files> is either "U" (Undefined), "W" or "w" (undefined weak), which are the possible outputs from non-ancient $(NM) versions. Print an error if format is unexpected.

---
 tools/lib/traceevent/Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
tools/lib/traceevent/Makefile +10 -4
index 2616c66e10c1..a7046ec01a5b 100644
@@ -257,10 +257,16 @@ define do_install_plugins
 endef
 
 define do_generate_dynamic_list_file
-	(echo '{';							\
-	$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;	\
-	echo '};';							\
-	) > $2
+	symbol_type=`$(NM) -u -D $1 | awk 'NF>1 {print $$1}' | \
+	xargs echo "U W w" | tr ' ' '\n' | sort -u | xargs echo`;\
+	if [ "$$symbol_type" == "U W w" ];then				\
+		(echo '{';						\
+		$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;\
+		echo '};';						\
+		) > $2;							\
+	else								\
+		(echo Either missing one of [$1] or bad version of $(NM)) 1>&2;\
+	fi
 endef
 
 install_lib: all_cmd install_plugins
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#7 lore raw

Re: [PATCH v2] tools lib traceevent: Robustify do_generate_dynamic_list_file

Em Tue, Feb 07, 2017 at 09:28:40PM -0800, David Carrillo-Cisneros escreveu:

Quoted 9 lines

v2: Accept "W" and "w" symbol options.

The dynamic-list-file used to export dynamic symbols introduced in

commit e3d09ec8126f ("tools lib traceevent: Export dynamic symbols used by traceevent plugins")

is generated without any sort of error checking.

Quoted 7 lines

I experienced problems due to an old version of nm (v 0.158) that outputs in a format distinct from the assumed by the script.

Robustify the built of dynamic symbol list by enforcing that the second column of $(NM) -u <files> is either "U" (Undefined), "W" or "w" (undefined weak), which are the possible outputs from non-ancient $(NM) versions. Print an error if format is unexpected.

The problem persists, now with a slightly different message, including your latest effort at weak symbols, that looked unrelated at first:

GEN /tmp/build/perf/pmu-events/pmu-events.c
/bin/sh: 1: [: U W w: unexpected operator
Either missing one of [ /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so] or bad version of nm
CC /tmp/build/perf/ui/gtk/hists.o

CC /tmp/build/perf/util/symbol.o
/usr/bin/ld: cannot open linker script file /tmp/build/perf/libtraceevent-dynamic-list: No such file or directory
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
cp: cannot stat '/tmp/build/perf/python_ext_build/lib/perf.so': No such file or directory Makefile.perf:381: recipe for target '/tmp/build/perf/python/perf.so' failed
make[2]: *** [/tmp/build/perf/python/perf.so] Error 1
make[2]: *** Waiting for unfinished jobs....
CC /tmp/build/perf/arch/common.o

perfbuilder@341e71da5019:/git/linux$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="16.10 (Yakkety Yak)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.10"
VERSION_ID="16.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="http://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=yakkety
UBUNTU_CODENAME=yakkety
perfbuilder@341e71da5019:/git/linux$ apt-cache show binutils
Package: binutils
Priority: optional
Section: devel
Installed-Size: 13828
Maintainer: Ubuntu Core developers <[email protected]>
Original-Maintainer: Matthias Klose <[email protected]>
Architecture: amd64
Version: 2.27-8ubuntu2
Replaces: binutils-gold (<< 2.20.51.20100415), binutils-mingw-w64-i686 (<< 2.23.52.20130612-1+3), binutils-mingw-w64-x86-64 (<< 2.23.52.20130612-1+3)
Provides: binutils-gold, elf-binutils
Depends: libc6 (>= 2.14), zlib1g (>= 1:1.2.0)
Suggests: binutils-doc (>= 2.27-8ubuntu2)
Conflicts: binutils-gold (<< 2.20.51.20100415), binutils-mingw-w64-i686 (<< 2.23.52.20130612-1+3), binutils-mingw-w64-x86-64 (<< 2.23.52.20130612-1+3), binutils-multiarch (<< 2.27-8), elf-binutils, gas, modutils (<< 2.4.19-1)
Breaks: binutils-mingw-w64-i686 (<< 2.23.52.20130612-1+3), binutils-mingw-w64-x86-64 (<< 2.23.52.20130612-1+3), hardening-wrapper (<< 2.8+nmu3)
Filename: pool/main/b/binutils/binutils_2.27-8ubuntu2_amd64.deb
Size: 2416388
MD5sum: c90cc5eef9d5bf00d1de0f1c8cf98a06
SHA1: 9b7a3cb2b140f0d6903888724efe93427382fdb0
SHA256: b7bbbadfebc615e74bf2d332a581f059a453038685548919d6db230b9732febd
Description: GNU assembler, linker and binary utilities
Description-md5: fde49b4cfeaad346a6e094f973da28d7
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Build-Essential: yes
Origin: Ubuntu
Supported: 9m
Task: ubuntu-desktop, ubuntu-usb, kubuntu-desktop, kubuntu-full, edubuntu-desktop, edubuntu-usb, xubuntu-core, xubuntu-desktop, mythbuntu-frontend, mythbuntu-desktop, mythbuntu-backend-slave, mythbuntu-backend-master, mythbuntu-backend-master, ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntu-gnome-desktop, ubuntu-touch, ubuntukylin-desktop, ubuntu-mate-core, ubuntu-mate-desktop

perfbuilder@341e71da5019:/git/linux$

Quoted 32 lines
---
 tools/lib/traceevent/Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
tools/lib/traceevent/Makefile +10 -4
index 2616c66e10c1..a7046ec01a5b 100644
@@ -257,10 +257,16 @@ define do_install_plugins
 endef
 
 define do_generate_dynamic_list_file
-	(echo '{';							\
-	$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;	\
-	echo '};';							\
-	) > $2
+	symbol_type=`$(NM) -u -D $1 | awk 'NF>1 {print $$1}' | \
+	xargs echo "U W w" | tr ' ' '\n' | sort -u | xargs echo`;\
+	if [ "$$symbol_type" == "U W w" ];then				\
+		(echo '{';						\
+		$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;\
+		echo '};';						\
+		) > $2;							\
+	else								\
+		(echo Either missing one of [$1] or bad version of $(NM)) 1>&2;\
+	fi
 endef
 
 install_lib: all_cmd install_plugins
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#8 lore raw

Re: [PATCH v2] tools lib traceevent: Robustify do_generate_dynamic_list_file

Em Wed, Feb 08, 2017 at 09:44:50AM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 18 lines

Em Tue, Feb 07, 2017 at 09:28:40PM -0800, David Carrillo-Cisneros escreveu:

Quoted 9 lines

v2: Accept "W" and "w" symbol options.

The dynamic-list-file used to export dynamic symbols introduced in

commit e3d09ec8126f ("tools lib traceevent: Export dynamic symbols used by traceevent plugins")

is generated without any sort of error checking.

Quoted 7 lines

I experienced problems due to an old version of nm (v 0.158) that outputs in a format distinct from the assumed by the script.

Robustify the built of dynamic symbol list by enforcing that the second column of $(NM) -u <files> is either "U" (Undefined), "W" or "w" (undefined weak), which are the possible outputs from non-ancient $(NM) versions. Print an error if format is unexpected.

Quoted 7 lines

The problem persists, now with a slightly different message, including your latest effort at weak symbols, that looked unrelated at first:

GEN /tmp/build/perf/pmu-events/pmu-events.c
/bin/sh: 1: [: U W w: unexpected operator
Either missing one of [ /tmp/build/perf/plugin_jbd2.so /tmp/build/perf/plugin_hrtimer.so /tmp/build/perf/plugin_kmem.so /tmp/build/perf/plugin_kvm.so /tmp/build/perf/plugin_mac80211.so /tmp/build/perf/plugin_sched_switch.so /tmp/build/perf/plugin_function.so /tmp/build/perf/plugin_xen.so /tmp/build/perf/plugin_scsi.so /tmp/build/perf/plugin_cfg80211.so] or bad version of nm
CC /tmp/build/perf/ui/gtk/hists.o

So, after I applied this patch:

---------------------

tools/lib/traceevent/Makefile +1 -1
index a7046ec01a5b..47076b15eebe 100644
@@ -259,7 +259,7 @@ endef
 define do_generate_dynamic_list_file
 	symbol_type=`$(NM) -u -D $1 | awk 'NF>1 {print $$1}' | \
 	xargs echo "U W w" | tr ' ' '\n' | sort -u | xargs echo`;\
-	if [ "$$symbol_type" == "U W w" ];then				\
+	if [ "$$symbol_type" = "U W w" ];then				\
 		(echo '{';						\
 		$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;\
 		echo '};';						\

---------------------

It works.

Look at the file produced by that:

In Ubuntu 16.10:

perfbuilder@341e71da5019:/git/linux$ cat /tmp/build/perf/libtraceevent-dynamic-list
{
	__stack_chk_fail;
	free;
	memset;
	pevent_find_any_field;
	pevent_find_field;
	pevent_find_function;
	pevent_find_function_address;
	pevent_get_field_raw;
	pevent_get_field_val;
	pevent_print_func_field;
	pevent_print_num_field;
	pevent_read_number_field;
	pevent_register_comm;
	pevent_register_event_handler;
	pevent_register_print_function;
	pevent_unregister_event_handler;
	pevent_unregister_print_function;
	realloc;
	sprintf;
	strcmp;
	strdup;
	strncmp;
	trace_seq_printf;
	trace_seq_putc;
	trace_seq_puts;
	trace_seq_terminate;
	traceevent_plugin_add_options;
	traceevent_plugin_remove_options;
	warning;
};
perfbuilder@341e71da5019:/git/linux$

And in fedora 25, with or without this patch, i.e. with = or ==:

[acme@jouet linux]$ cat /tmp/build/perf/libtraceevent-dynamic-list

{
	free;
	memset;
	pevent_find_any_field;
	pevent_find_field;
	pevent_find_function;
	pevent_find_function_address;
	pevent_get_field_raw;
	pevent_get_field_val;
	pevent_print_func_field;
	pevent_print_num_field;
	pevent_read_number_field;
	pevent_register_comm;
	pevent_register_event_handler;
	pevent_register_print_function;
	pevent_unregister_event_handler;
	pevent_unregister_print_function;
	realloc;
	sprintf;
	strcmp;
	strdup;
	strncmp;
	trace_seq_printf;
	trace_seq_putc;
	trace_seq_puts;
	trace_seq_terminate;
	traceevent_plugin_add_options;
	traceevent_plugin_remove_options;
	warning;
};

[acme@jouet linux]$

Then try to find the '==' operator on 'man test'. Albeit command line tests, outside the Makefile shows it works, it is not documented there, just "STRING = STRING". Some subtlety:

Fedora 25:

[acme@jouet linux]$ symbol_type="U W w"
[acme@jouet linux]$ [ "$symbol_type" == "U W w" ] && echo matches matches
[acme@jouet linux]$ [ "$symbol_type" == "U W w A" ] && echo matches [acme@jouet linux]$ [ "$symbol_type" = "U W w" ] && echo matches matches
[acme@jouet linux]$ [ "$symbol_type" = "U W w A" ] && echo matches [acme@jouet linux]$

[acme@jouet linux]$ rpm -qf /bin/\[
coreutils-8.25-15.fc25.x86_64
[acme@jouet linux]$ rpm -q bash
bash-4.3.43-4.fc25.x86_64

Ubuntu 16.10:

perfbuilder@341e71da5019:/git/linux$ echo $symbol_type
U W w
perfbuilder@341e71da5019:/git/linux$ [ "$symbol_type" == "U W w" ] && echo matches
matches
perfbuilder@341e71da5019:/git/linux$ [ "$symbol_type" == "U W w AA" ] && echo matches
perfbuilder@341e71da5019:/git/linux$ [ "$symbol_type" = "U W w" ] && echo matches
matches
perfbuilder@341e71da5019:/git/linux$ [ "$symbol_type" = "U W w AA" ] && echo matches
perfbuilder@341e71da5019:/git/linux$ 
perfbuilder@341e71da5019:/git/linux$ ls -la /bin/\[
ls: cannot access '/bin/[': No such file or directory
perfbuilder@341e71da5019:/git/linux$ ls -la /bin/[
ls: cannot access '/bin/[': No such file or directory
perfbuilder@341e71da5019:/git/linux$
perfbuilder@341e71da5019:/git/linux$ dpkg -l | grep coreutils
ii  coreutils                          8.25-2ubuntu2                  amd64        GNU core utilities
perfbuilder@341e71da5019:/git/linux$ dpkg -l | grep bash
ii  bash                               4.3-15ubuntu1                  amd64        GNU Bourne Again SHell

bash has some shell opts to control some aspects of compatibility with string comparisions that could have some default different from ubuntu and fedora, I couldn't find one that looked related, sigh.

Anyway, the difference in the order of the symbol types is related to locales, which are fixed in the Makefiles to avoid those problems, so in both Fedora and Ubuntu that symbol_type evaluates to "U W w" and the file is generated as intended.

So I'm just replacing == with = and running a new set of container based distro tests.

- Arnaldo

#9 lore raw

Re: [PATCH v2] tools lib traceevent: Robustify do_generate_dynamic_list_file

Quoted 3 lines

bash has some shell opts to control some aspects of compatibility with string comparisions that could have some default different from ubuntu and fedora, I couldn't find one that looked related, sigh.

The joys of batch scripting...

Quoted 7 lines

Anyway, the difference in the order of the symbol types is related to locales, which are fixed in the Makefiles to avoid those problems, so in both Fedora and Ubuntu that symbol_type evaluates to "U W w" and the file is generated as intended.

So I'm just replacing == with = and running a new set of container based distro tests.

"=" works fine in my build. Thanks for looking into this. Next time I will pay attention to bash portability.

Thanks,
David

#10 lore raw

[tip:perf/core] tools lib traceevent: Robustify do_generate_dynamic_list_file

In reply to David Carrillo-Cisneros
tools lib traceevent: Robustify do_generate_dynamic_list_file
Commit-ID:  6467753d613993f30965d4e0717f7304eb933ef7
Gitweb:     http://git.kernel.org/tip/6467753d613993f30965d4e0717f7304eb933ef7
Author:     David Carrillo-Cisneros <[email protected]>
AuthorDate: Tue, 7 Feb 2017 21:28:40 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 8 Feb 2017 10:27:45 -0300

tools lib traceevent: Robustify do_generate_dynamic_list_file

The dynamic-list-file used to export dynamic symbols introduced in

commit e3d09ec8126f ("tools lib traceevent: Export dynamic symbols used by traceevent plugins")

is generated without any sort of error checking.

I experienced problems due to an old version of nm (v 0.158) that outputs in a format distinct from the assumed by the script.

Robustify the built of dynamic symbol list by enforcing that the second column of $(NM) -u <files> is either "U" (Undefined), "W" or "w" (undefined weak), which are the possible outputs from non-ancient $(NM) versions. Print an error if format is unexpected.

v2: Accept "W" and "w" symbol options.

Cc Alexander Shishkin <[email protected]>
Cc He Kuang <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Michal Marek <[email protected]>
Cc Paul Turner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Uwe Kleine-König <[email protected]>
Cc Wang Nan <[email protected]>
[ Use STRING1 = STRING1 instead of == to make this work on Ubuntu systems ]
---
 tools/lib/traceevent/Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
tools/lib/traceevent/Makefile +10 -4
index 2616c66..47076b1 100644
@@ -257,10 +257,16 @@ define do_install_plugins
 endef
 
 define do_generate_dynamic_list_file
-	(echo '{';							\
-	$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;	\
-	echo '};';							\
-	) > $2
+	symbol_type=`$(NM) -u -D $1 | awk 'NF>1 {print $$1}' | \
+	xargs echo "U W w" | tr ' ' '\n' | sort -u | xargs echo`;\
+	if [ "$$symbol_type" = "U W w" ];then				\
+		(echo '{';						\
+		$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;\
+		echo '};';						\
+		) > $2;							\
+	else								\
+		(echo Either missing one of [$1] or bad version of $(NM)) 1>&2;\
+	fi
 endef
 
 install_lib: all_cmd install_plugins
#11 lore raw

PATCH 2/4 tools lib traceevent: Robustify do_generate_dynamic_list_file

In reply to David Carrillo-Cisneros

The dynamic-list-file used to export dynamic symbols introduced in

commit e3d09ec8126f ("tools lib traceevent: Export dynamic symbols used by traceevent plugins")

is generated without any sort of error checking.

I experienced problems due to an old version of nm (v 0.158) that outputs in a format distinct from the assumed by the script.

Robustify this by enforcing that the second column in the symbol output is an "U" (Undefined) as it should be since we are calling $(NM) -u ... and print an error message otherwise.

---
 tools/lib/traceevent/Makefile | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
tools/lib/traceevent/Makefile +9 -4
index 2616c66e10c1..1b6e146429d0 100644
@@ -257,10 +257,15 @@ define do_install_plugins
 endef
 
 define do_generate_dynamic_list_file
-	(echo '{';							\
-	$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;	\
-	echo '};';							\
-	) > $2
+	symbol_type=`$(NM) -u -D $1 | awk 'NF>1 {print $$1}' | sort -u`;\
+	if [ "$$symbol_type" == "U" ];then				\
+		(echo '{';						\
+		$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;\
+		echo '};';						\
+		) > $2;							\
+	else								\
+		(echo Either missing one of [$1] or bad version of $(NM)) 1>&2;\
+	fi
 endef
 
 install_lib: all_cmd install_plugins
Signature · 1 line
2.11.0.483.g087da7b7c-goog
maintainer Applied
#12 lore raw

Re: [PATCH 2/4] tools lib traceevent: Robustify do_generate_dynamic_list_file

Em Wed, Feb 01, 2017 at 10:38:02PM -0800, David Carrillo-Cisneros escreveu:

Quoted 5 lines

The dynamic-list-file used to export dynamic symbols introduced in

commit e3d09ec8126f ("tools lib traceevent: Export dynamic symbols used by traceevent plugins")

Quoted 8 lines

is generated without any sort of error checking.

I experienced problems due to an old version of nm (v 0.158) that outputs in a format distinct from the assumed by the script.

Robustify this by enforcing that the second column in the symbol output is an "U" (Undefined) as it should be since we are calling $(NM) -u ... and print an error message otherwise.

Thanks, applied.

- Arnaldo

Quoted 31 lines
---
 tools/lib/traceevent/Makefile | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
tools/lib/traceevent/Makefile +9 -4
index 2616c66e10c1..1b6e146429d0 100644
@@ -257,10 +257,15 @@ define do_install_plugins
 endef
 
 define do_generate_dynamic_list_file
-	(echo '{';							\
-	$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;	\
-	echo '};';							\
-	) > $2
+	symbol_type=`$(NM) -u -D $1 | awk 'NF>1 {print $$1}' | sort -u`;\
+	if [ "$$symbol_type" == "U" ];then				\
+		(echo '{';						\
+		$(NM) -u -D $1 | awk 'NF>1 {print "\t"$$2";"}' | sort -u;\
+		echo '};';						\
+		) > $2;							\
+	else								\
+		(echo Either missing one of [$1] or bad version of $(NM)) 1>&2;\
+	fi
 endef
 
 install_lib: all_cmd install_plugins
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#13 lore raw

PATCH 3/4 tools lib feature: Do not redefine compiler configuration

In reply to David Carrillo-Cisneros

Feature detection redefines CC, CCX and PKG_CONFIG, making the output of feature detection inconsistent with the actual features available during compilation when the above variables are used.

Fix it by using conditional assignment.

---
 tools/build/feature/Makefile | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/build/feature/Makefile +3 -3
index 3dd2c600d250..d39e76f3a063 100644
@@ -52,9 +52,9 @@ FILES=                                          \
 
 FILES := $(addprefix $(OUTPUT),$(FILES))
 
-CC := $(CROSS_COMPILE)gcc -MD
-CXX := $(CROSS_COMPILE)g++ -MD
-PKG_CONFIG := $(CROSS_COMPILE)pkg-config
+CC ?= $(CROSS_COMPILE)gcc -MD
+CXX ?= $(CROSS_COMPILE)g++ -MD
+PKG_CONFIG ?= $(CROSS_COMPILE)pkg-config
 LLVM_CONFIG ?= llvm-config
 
 all: $(FILES)
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#14 lore raw

Re: [PATCH 3/4] tools lib feature: Do not redefine compiler configuration

Em Wed, Feb 01, 2017 at 10:38:03PM -0800, David Carrillo-Cisneros escreveu:

Quoted 5 lines

Feature detection redefines CC, CCX and PKG_CONFIG, making the output of feature detection inconsistent with the actual features available during compilation when the above variables are used.

Fix it by using conditional assignment.

This one is tricky, the real meaning of that line got lost somewhere, its original intent, in commit 8b6eb56a9570 ("tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically") was to just add that "-MD" to whatever definition CC had, be it the default one or a value set by the user in the make command line.

At some point someone added that CROSS_COMPILE there, which it should have gotten from the default CC definition

It was here: a8a5cd8b472c ("perf: tools: Fix cross building")

perf: tools: Fix cross building

Currently the feature-checks Makefile does not inherit $(CC), and calls cc rather than $(CROSS_COMPILE)gcc. Thus the feature checks invoke the native toolchain rather than the cross toolchain, and can identify features as available when they are not. This can break the build

----------------------------------

Mark, do you recall why can't we make tools/perf/Makefile.perf pass the definitions of CC, AR and PKG_CONFIG to tools/perf/config/Makefile (now tools/build/feature/Makefile) so that the original intent (adding -MD to whatever was in CC) can be kept?

Jiri?

- Arnaldo

Quoted 24 lines
---
 tools/build/feature/Makefile | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/build/feature/Makefile +3 -3
index 3dd2c600d250..d39e76f3a063 100644
@@ -52,9 +52,9 @@ FILES=                                          \
 
 FILES := $(addprefix $(OUTPUT),$(FILES))
 
-CC := $(CROSS_COMPILE)gcc -MD
-CXX := $(CROSS_COMPILE)g++ -MD
-PKG_CONFIG := $(CROSS_COMPILE)pkg-config
+CC ?= $(CROSS_COMPILE)gcc -MD
+CXX ?= $(CROSS_COMPILE)g++ -MD
+PKG_CONFIG ?= $(CROSS_COMPILE)pkg-config
 LLVM_CONFIG ?= llvm-config
 
 all: $(FILES)
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#15 lore raw

Re: [PATCH 3/4] tools lib feature: Do not redefine compiler configuration

Quoted 26 lines · Arnaldo Carvalho de Melo

Em Wed, Feb 01, 2017 at 10:38:03PM -0800, David Carrillo-Cisneros escreveu:

Quoted 5 lines

Feature detection redefines CC, CCX and PKG_CONFIG, making the output of feature detection inconsistent with the actual features available during compilation when the above variables are used.

Fix it by using conditional assignment.

This one is tricky, the real meaning of that line got lost somewhere, its original intent, in commit 8b6eb56a9570 ("tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically") was to just add that "-MD" to whatever definition CC had, be it the default one or a value set by the user in the make command line.

At some point someone added that CROSS_COMPILE there, which it should have gotten from the default CC definition

It was here: a8a5cd8b472c ("perf: tools: Fix cross building")

perf: tools: Fix cross building

Currently the feature-checks Makefile does not inherit $(CC), and calls cc rather than $(CROSS_COMPILE)gcc. Thus the feature checks invoke the native toolchain rather than the cross toolchain, and can identify features as available when they are not. This can break the build

Quoted 8 lines

----------------------------------

Mark, do you recall why can't we make tools/perf/Makefile.perf pass the definitions of CC, AR and PKG_CONFIG to tools/perf/config/Makefile (now tools/build/feature/Makefile) so that the original intent (adding -MD to whatever was in CC) can be kept?

Jiri?

I guess we could remove -MD from the gcc like and add it to the CFLAGS if we want to keep the CC/CXX lines alone..

jirka

#16 lore raw

PATCH 4/4 tools include: Fix include path for uapi/asm-generic/mman.h

In reply to David Carrillo-Cisneros

commit f3539c12d819 ("tools include: Add uapi mman.h for each architecture")

copied include/uapi/linux/mman.h into tools/include/uapi/linux/mman.h but did not update the include path for uapi/asm-generic/mman.h. Fix it.

---
 tools/include/uapi/linux/mman.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/include/uapi/linux/mman.h +1 -1
index 81d8edf11789..74a19ae15cd1 100644
@@ -1,7 +1,7 @@
 #ifndef _UAPI_LINUX_MMAN_H
 #define _UAPI_LINUX_MMAN_H
 
-#include <uapi/asm/mman.h>
+#include <uapi/asm-generic/mman.h>
 
 #define MREMAP_MAYMOVE	1
 #define MREMAP_FIXED	2
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#17 lore raw

Re: [PATCH 4/4] tools include: Fix include path for uapi/asm-generic/mman.h

Quoted 26 lines · David Carrillo-Cisneros

commit f3539c12d819 ("tools include: Add uapi mman.h for each architecture")

copied include/uapi/linux/mman.h into tools/include/uapi/linux/mman.h but did not update the include path for uapi/asm-generic/mman.h. Fix it.

---
 tools/include/uapi/linux/mman.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/include/uapi/linux/mman.h +1 -1
index 81d8edf11789..74a19ae15cd1 100644
@@ -1,7 +1,7 @@
 #ifndef _UAPI_LINUX_MMAN_H
 #define _UAPI_LINUX_MMAN_H
 
-#include <uapi/asm/mman.h>
+#include <uapi/asm-generic/mman.h>
 
 #define MREMAP_MAYMOVE	1
 #define MREMAP_FIXED	2
Signature · 1 line
2.11.0.483.g087da7b7c-goog

Arnaldo,
could you please squash in the check-headers.sh update as well..

thanks,
jirka

---

tools/perf/check-headers.sh +1 -1
index c747bfd7f14d..87187b0b3bc0 100755
@@ -56,4 +56,4 @@ done
 check arch/x86/lib/memcpy_64.S        -B -I "^EXPORT_SYMBOL" -I "^#include <asm/export.h>"
 check arch/x86/lib/memset_64.S        -B -I "^EXPORT_SYMBOL" -I "^#include <asm/export.h>"
 check include/uapi/asm-generic/mman.h -B -I "^#include <\(uapi/\)*asm-generic/mman-common.h>"
-check include/uapi/linux/mman.h       -B -I "^#include <\(uapi/\)*asm/mman.h>"
+check include/uapi/linux/mman.h       -B -I "^#include <\(uapi/\)*asm\(-generic\)*/mman.h>"
#18 lore raw

Re: [PATCH 4/4] tools include: Fix include path for uapi/asm-generic/mman.h

In reply to David Carrillo-Cisneros

Em Wed, Feb 01, 2017 at 10:38:04PM -0800, David Carrillo-Cisneros escreveu:

Quoted 4 lines

commit f3539c12d819 ("tools include: Add uapi mman.h for each architecture")

copied include/uapi/linux/mman.h into tools/include/uapi/linux/mman.h but did not update the include path for uapi/asm-generic/mman.h. Fix it.

What problem is this fixing?

I.e. one is supposed to include <linux/mman.h>, get the tools/include/uapi/linux/mman.h and it in turn will include the per-arch mman.h, say tools/arch/sparc/include/uapi/asm/mman.h, that in turn will include uapi/asm-generic/mman-common.h after defining its diverging bits.

Some, like the ones for the Alpha and MIPS architectures, will not even include mman-common.h, so diverging they are.

Lots of arches, like s390, will just include uapi/asm-generic/mman.h from its mman.h file (tools/arch/s390/include/uapi/asm/mman.h), just like you'd do for _all_ arches in your patch below.

tools/perf/Makefile.config is the one doing:

INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/

and:

[acme@jouet linux]$ ls tools/arch/*/include/uapi/asm/mman.h

tools/arch/alpha/include/uapi/asm/mman.h    tools/arch/ia64/include/uapi/asm/mman.h        tools/arch/s390/include/uapi/asm/mman.h
tools/arch/arc/include/uapi/asm/mman.h      tools/arch/m32r/include/uapi/asm/mman.h        tools/arch/score/include/uapi/asm/mman.h
tools/arch/arm64/include/uapi/asm/mman.h    tools/arch/microblaze/include/uapi/asm/mman.h  tools/arch/sh/include/uapi/asm/mman.h
tools/arch/arm/include/uapi/asm/mman.h      tools/arch/mips/include/uapi/asm/mman.h        tools/arch/sparc/include/uapi/asm/mman.h
tools/arch/frv/include/uapi/asm/mman.h      tools/arch/mn10300/include/uapi/asm/mman.h     tools/arch/tile/include/uapi/asm/mman.h
tools/arch/h8300/include/uapi/asm/mman.h    tools/arch/parisc/include/uapi/asm/mman.h      tools/arch/x86/include/uapi/asm/mman.h
tools/arch/hexagon/include/uapi/asm/mman.h  tools/arch/powerpc/include/uapi/asm/mman.h     tools/arch/xtensa/include/uapi/asm/mman.h

[acme@jouet linux]$

Am I missing something?

- Arnaldo

Quoted 20 lines
---
 tools/include/uapi/linux/mman.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/include/uapi/linux/mman.h +1 -1
index 81d8edf11789..74a19ae15cd1 100644
@@ -1,7 +1,7 @@
 #ifndef _UAPI_LINUX_MMAN_H
 #define _UAPI_LINUX_MMAN_H
 
-#include <uapi/asm/mman.h>
+#include <uapi/asm-generic/mman.h>
 
 #define MREMAP_MAYMOVE	1
 #define MREMAP_FIXED	2
Signature · 1 line
2.11.0.483.g087da7b7c-goog
#19 lore raw

Re: [PATCH 4/4] tools include: Fix include path for uapi/asm-generic/mman.h

Quoted 29 lines

What problem is this fixing?

I.e. one is supposed to include <linux/mman.h>, get the tools/include/uapi/linux/mman.h and it in turn will include the per-arch mman.h, say tools/arch/sparc/include/uapi/asm/mman.h, that in turn will include uapi/asm-generic/mman-common.h after defining its diverging bits.

Some, like the ones for the Alpha and MIPS architectures, will not even include mman-common.h, so diverging they are.

Lots of arches, like s390, will just include uapi/asm-generic/mman.h from its mman.h file (tools/arch/s390/include/uapi/asm/mman.h), just like you'd do for _all_ arches in your patch below.

tools/perf/Makefile.config is the one doing:

INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/

and:

[acme@jouet linux]$ ls tools/arch/*/include/uapi/asm/mman.h

tools/arch/alpha/include/uapi/asm/mman.h    tools/arch/ia64/include/uapi/asm/mman.h        tools/arch/s390/include/uapi/asm/mman.h
tools/arch/arc/include/uapi/asm/mman.h      tools/arch/m32r/include/uapi/asm/mman.h        tools/arch/score/include/uapi/asm/mman.h
tools/arch/arm64/include/uapi/asm/mman.h    tools/arch/microblaze/include/uapi/asm/mman.h  tools/arch/sh/include/uapi/asm/mman.h
tools/arch/arm/include/uapi/asm/mman.h      tools/arch/mips/include/uapi/asm/mman.h        tools/arch/sparc/include/uapi/asm/mman.h
tools/arch/frv/include/uapi/asm/mman.h      tools/arch/mn10300/include/uapi/asm/mman.h     tools/arch/tile/include/uapi/asm/mman.h
tools/arch/h8300/include/uapi/asm/mman.h    tools/arch/parisc/include/uapi/asm/mman.h      tools/arch/x86/include/uapi/asm/mman.h
tools/arch/hexagon/include/uapi/asm/mman.h  tools/arch/powerpc/include/uapi/asm/mman.h     tools/arch/xtensa/include/uapi/asm/mman.h
Quoted 4 lines

[acme@jouet linux]$

Am I missing something?

The original error is:

In file included from util/event.c:2:0:
...tools/include/uapi/linux/mman.h:4:27: fatal error: uapi/asm/mman.h:
No such file or directory

 #include <uapi/asm/mman.h>
                           ^

compilation terminated.

After reading your explanation, seems like the problem is caused by util/event.c directly including uapi/linux/mman.h :

Line 2: #include <uapi/linux/mman.h> /* To get things like
MAP_HUGETLB even on older libc headers */

introduced in commit fbef103fad ("perf tools: Do hugetlb handling in more systems").

Doing:

tools/include/uapi/linux/mman.h +1 -1
@@ -1,7 +1,7 @@
 #ifndef _UAPI_LINUX_MMAN_H
 #define _UAPI_LINUX_MMAN_H
 
-#include <uapi/asm-generic/mman.h>
+#include <uapi/asm/mman.h>
 

fixes by build. I don't know if it breaks something with HugeTLB, though.

Thanks,
David

Acked-by on the v1 series
#20 lore raw
In reply to David Carrillo-Cisneros
Quoted 8 lines · David Carrillo-Cisneros

Fixes for minor problems with Makefiles and libs that I found while building perf.

David Carrillo-Cisneros (4):
perf tools: pass PYTHON config to feature detection tools lib traceevent: Robustify do_generate_dynamic_list_file tools lib feature: Do not redefine compiler configuration tools include: Fix include path for uapi/asm-generic/mman.h

thanks,
jirka