Skip to contentSkip to conversation

perf: rewire generic library stuff, p1

PATCH Borislav Petkov 3 participants 7 messages Jul 1, 2010 10 messages not in the archive perf events

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, Jul 01, 2010 at 12:10:54PM -0400

Hi Arnaldo,

Quoted 36 lines

Em Thu, Jul 01, 2010 at 12:47:37PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 9 lines

Em Thu, Jul 01, 2010 at 03:49:19PM +0200, Borislav Petkov escreveu:

Quoted 6 lines

From: Borislav Petkov <[email protected]>

Carve out generic library stuff into tools/lib/lk/ and rewire it with perf. Add a top-level Makefile which selects between targets depending on the tool we want to build. Also, add a Makefile.lib for common facilities used by all the Makefiles.

Testing this now, thanks,

Needs the patch below for "make O=~/build/perf tools" to continue working. Needs some more polishing so as not to break the non O= case.

After lunch will investigate why my usual way to build the tools isn't working anymore, some missing -I probably:

[acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/perf install
make: Entering directory `/home/acme/git/linux-2.6-tip/tools/perf'
CC /home/acme/git/build/perf/builtin-record.o
builtin-record.c:24:23: error: lk/cpumap.h: No such file or directory
cc1: warnings being treated as errors
builtin-record.c: In function ‘__cmd_record’:
builtin-record.c:622: error: implicit declaration of function ‘read_cpu_map’ builtin-record.c:632: error: ‘cpumap’ undeclared (first use in this function) builtin-record.c:632: error: (Each undeclared identifier is reported only once builtin-record.c:632: error: for each function it appears in.)
make: *** [/home/acme/git/build/perf/builtin-record.o] Error 1
make: Leaving directory `/home/acme/git/linux-2.6-tip/tools/perf' [acme@emilia linux-2.6-tip]$

tools/lib/Makefile +0 -0
index 48f8e21..814673e 100644

tools/lib/Makefile LIB_H = · line 13

Quoted 9 lines
tools/lib/Makefile +3 -0
@@ -5,6 +5,9 @@ LIB_H =
 LIB_OBJS =
 
 LIB_H += lk/cpumap.h
+OUTPUT := $(OUTPUT)/lib/
+
+$(shell sh -c 'mkdir -p $(OUTPUT)/lk' 2> /dev/null)
 
 LIB_OBJS += $(OUTPUT)lk/bitmap.o
 LIB_OBJS += $(OUTPUT)lk/cpumap.o

I'll fix that up, thanks for testing.

Signature · 8 lines
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#5 lore raw

Em Thu, Jul 01, 2010 at 06:17:38PM +0200, Borislav Petkov escreveu:

Quoted 27 lines

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, Jul 01, 2010 at 12:10:54PM -0400

Hi Arnaldo,

Quoted 22 lines

Em Thu, Jul 01, 2010 at 12:47:37PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 9 lines

Em Thu, Jul 01, 2010 at 03:49:19PM +0200, Borislav Petkov escreveu:

Quoted 6 lines

From: Borislav Petkov <[email protected]>

Carve out generic library stuff into tools/lib/lk/ and rewire it with perf. Add a top-level Makefile which selects between targets depending on the tool we want to build. Also, add a Makefile.lib for common facilities used by all the Makefiles.

Testing this now, thanks,

Needs the patch below for "make O=~/build/perf tools" to continue working. Needs some more polishing so as not to break the non O= case.

After lunch will investigate why my usual way to build the tools isn't working anymore, some missing -I probably:

[acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/perf install
make: Entering directory `/home/acme/git/linux-2.6-tip/tools/perf'
CC /home/acme/git/build/perf/builtin-record.o
builtin-record.c:24:23: error: lk/cpumap.h: No such file or directory
cc1: warnings being treated as errors

The patch below fixes it.

But we have one other problem:

[acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/
make: Entering directory /home/acme/git/linux-2.6-tip/tools' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: `lklib.a' is up to date.
make[1]: Leaving directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/perf'
GEN perf-archive
make[1]: Leaving directory `/home/acme/git/linux-2.6-tip/tools/perf'
make: Leaving directory `/home/acme/git/linux-2.6-tip/tools' [acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/perf/
make: Entering directory `/home/acme/git/linux-2.6-tip/tools/perf'
* new build flags or prefix
CC /home/acme/git/build/perf/perf.o
CC /home/acme/git/build/perf/builtin-annotate.o
CC /home/acme/git/build/perf/builtin-bench.o

The logic that detects "new build flags or prefix" gets confused when we alternate between tools/ and tools/perf/, checking that.

- Arnaldo

tools/perf/Makefile +1 -1
index 200fc13..72ccd17 100644
@@ -268,7 +268,7 @@ endif
 # Those must not be GNU-specific; they are shared with perl/ which may
 # be built by a different compiler. (Note that this is an artifact now
 # but it still might be nice to keep that distinction.)
-BASIC_CFLAGS += -Iutil/include -Iarch/$(ARCH)/include
+BASIC_CFLAGS += -Iutil/include -Iarch/$(ARCH)/include -I../lib/
 BASIC_LDFLAGS =
 
 # Guard against environment variables
#6 lore raw

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, Jul 01, 2010 at 01:14:49PM -0400

Quoted 32 lines
Quoted 11 lines
Quoted 11 lines

Needs the patch below for "make O=~/build/perf tools" to continue working. Needs some more polishing so as not to break the non O= case.

After lunch will investigate why my usual way to build the tools isn't working anymore, some missing -I probably:

[acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/perf install
make: Entering directory `/home/acme/git/linux-2.6-tip/tools/perf'
CC /home/acme/git/build/perf/builtin-record.o
builtin-record.c:24:23: error: lk/cpumap.h: No such file or directory
cc1: warnings being treated as errors

The patch below fixes it.

But we have one other problem:

[acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/
make: Entering directory /home/acme/git/linux-2.6-tip/tools' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: `lklib.a' is up to date.
make[1]: Leaving directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/perf'
GEN perf-archive
make[1]: Leaving directory `/home/acme/git/linux-2.6-tip/tools/perf'
make: Leaving directory `/home/acme/git/linux-2.6-tip/tools' [acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/perf/
make: Entering directory `/home/acme/git/linux-2.6-tip/tools/perf'
* new build flags or prefix
CC /home/acme/git/build/perf/perf.o
CC /home/acme/git/build/perf/builtin-annotate.o
CC /home/acme/git/build/perf/builtin-bench.o

Quoted 2 lines

The logic that detects "new build flags or prefix" gets confused when we alternate between tools/ and tools/perf/, checking that.

This might need an up-to-date check for the lklib.a archive first and not do "new build flags or prefix" if it is. Yeah, this is a different library and the logic doesn't know about it.

tools/perf/Makefile endif · line 272

Quoted 13 lines

- Arnaldo

tools/perf/Makefile +1 -1
index 200fc13..72ccd17 100644
@@ -268,7 +268,7 @@ endif
 # Those must not be GNU-specific; they are shared with perl/ which may
 # be built by a different compiler. (Note that this is an artifact now
 # but it still might be nice to keep that distinction.)
-BASIC_CFLAGS += -Iutil/include -Iarch/$(ARCH)/include
+BASIC_CFLAGS += -Iutil/include -Iarch/$(ARCH)/include -I../lib/
 BASIC_LDFLAGS =

Right, the whole scheme with the include paths is kinda shaky. I have the -I$(CURDIR)/lib BASIC_CFLAGS in the top Makefile for the case where we're in tools/ but this doesn't help with the -C switch as you show above.

Maybe we should add more logic later to make the include path guessing more robust no matter from which directory you execute make instead of polluting BASIC_CFLAGS with duplicated path entries ... Hmmm...

Signature · 8 lines
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#7 lore raw

Em Thu, Jul 01, 2010 at 10:14:18PM +0200, Borislav Petkov escreveu:

Quoted 19 lines

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, Jul 01, 2010 at 01:14:49PM -0400

Quoted 17 lines

But we have one other problem:

[acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/
make: Entering directory /home/acme/git/linux-2.6-tip/tools' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: `lklib.a' is up to date.
make[1]: Leaving directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/perf'
GEN perf-archive
make[1]: Leaving directory `/home/acme/git/linux-2.6-tip/tools/perf'
make: Leaving directory `/home/acme/git/linux-2.6-tip/tools' [acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/perf/
make: Entering directory `/home/acme/git/linux-2.6-tip/tools/perf'
* new build flags or prefix
CC /home/acme/git/build/perf/perf.o
CC /home/acme/git/build/perf/builtin-annotate.o

Quoted 6 lines
Quoted 2 lines

The logic that detects "new build flags or prefix" gets confused when we alternate between tools/ and tools/perf/, checking that.

This might need an up-to-date check for the lklib.a archive first and not do "new build flags or prefix" if it is. Yeah, this is a different library and the logic doesn't know about it.

I got sidetracked, but this is what is in $(O)/PERF-CFLAGS for make -C tools/

[acme@emilia linux-2.6-tip]$ cat ../build/perf/PERF-CFLAGS
-ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Wformat-y2k -Wshadow -Winit-self -Wpacked -Wredundant-decls -Wstack-protector -Wstrict-aliasing=3 -Wswitch-default -Wswitch-enum -Wno-system-headers -Wundef -Wvolatile-register-var -Wwrite-strings -Wbad-function-cast -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wstrict-prototypes -Wdeclaration-after-statement -fstack-protector-all -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/home/acme/git/linux-2.6-tip/tools/lib -I/home/acme/git/linux-2.6-tip/tools/perf -I/home/acme/git/linux-2.6-tip/tools/perf/util/include -Iutil/include -Iarch/x86/include -I../lib/ -I/home/acme/git/build/perf/ -DLIBELF_NO_MMAP -I/usr/include/elfutils -DDWARF_SUPPORT -I/usr/include/slang -DSHA1_HEADER='<openssl/sha.h>' : /home/acme/bin:libexec/perf-core:share/perf-core/templates:/home/acme [acme@emilia linux-2.6-tip]$

And this is its contents if we use "make -C tools/perf/"

-ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -D_FORTIFY_SOURCE=2 -fstack-protector-all -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Iutil/include -Iarch/x86/include -I../lib/ -I/home/acme/git/build/perf/ -DLIBELF_NO_MMAP -I/usr/include/elfutils -DDWARF_SUPPORT -I/usr/include/slang -DSHA1_HEADER='<openssl/sha.h>' : /home/acme/bin:libexec/perf-core:share/perf-core/templates:/home/acme

Its just a matter of making both match :-)

- Arnaldo

#8 lore raw

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, Jul 01, 2010 at 05:13:31PM -0400

Quoted 34 lines

Em Thu, Jul 01, 2010 at 10:14:18PM +0200, Borislav Petkov escreveu:

Quoted 19 lines

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, Jul 01, 2010 at 01:14:49PM -0400

Quoted 17 lines

But we have one other problem:

[acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/
make: Entering directory /home/acme/git/linux-2.6-tip/tools' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: `lklib.a' is up to date.
make[1]: Leaving directory /home/acme/git/linux-2.6-tip/tools/lib' make[1]: Entering directory /home/acme/git/linux-2.6-tip/tools/perf'
GEN perf-archive
make[1]: Leaving directory `/home/acme/git/linux-2.6-tip/tools/perf'
make: Leaving directory `/home/acme/git/linux-2.6-tip/tools' [acme@emilia linux-2.6-tip]$ make O=~/git/build/perf -C tools/perf/
make: Entering directory `/home/acme/git/linux-2.6-tip/tools/perf'
* new build flags or prefix
CC /home/acme/git/build/perf/perf.o
CC /home/acme/git/build/perf/builtin-annotate.o

Quoted 6 lines
Quoted 2 lines

The logic that detects "new build flags or prefix" gets confused when we alternate between tools/ and tools/perf/, checking that.

This might need an up-to-date check for the lklib.a archive first and not do "new build flags or prefix" if it is. Yeah, this is a different library and the logic doesn't know about it.

I got sidetracked, but this is what is in $(O)/PERF-CFLAGS for make -C tools/

[acme@emilia linux-2.6-tip]$ cat ../build/perf/PERF-CFLAGS
-ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Wformat-y2k -Wshadow -Winit-self -Wpacked -Wredundant-decls -Wstack-protector -Wstrict-aliasing=3 -Wswitch-default -Wswitch-enum -Wno-system-headers -Wundef -Wvolatile-register-var -Wwrite-strings -Wbad-function-cast -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wstrict-prototypes -Wdeclaration-after-statement -fstack-protector-all -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/home/acme/git/linux-2.6-tip/tools/lib -I/home/acme/git/linux-2.6-tip/tools/perf -I/home/acme/git/linux-2.6-tip/tools/perf/util/include -Iutil/include -Iarch/x86/include -I../lib/ -I/home/acme/git/build/perf/ -DLIBELF_NO_MMAP -I/usr/include/elfutils -DDWARF_SUPPORT -I/usr/include/slang -DSHA1_HEADER='<openssl/sha.h>' : /home/acme/bin:libexec/perf-core:share/perf-core/templates:/home/acme [acme@emilia linux-2.6-tip]$

Quoted 5 lines

And this is its contents if we use "make -C tools/perf/"

-ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -D_FORTIFY_SOURCE=2 -fstack-protector-all -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Iutil/include -Iarch/x86/include -I../lib/ -I/home/acme/git/build/perf/ -DLIBELF_NO_MMAP -I/usr/include/elfutils -DDWARF_SUPPORT -I/usr/include/slang -DSHA1_HEADER='<openssl/sha.h>' : /home/acme/bin:libexec/perf-core:share/perf-core/templates:/home/acme

Its just a matter of making both match :-)

Oh yeah, I see the difference. The second one is missing all the -Wxxx settings and the BASIS_CFLAGS with all the include paths which get computed in the top Makefile and exported to the Makefiles in the subdirectories. Hmm, maybe we should reroute the "make -C tools/perf/" invocation to go up one dir and execute the top-Makefile after all...

I'll look into it soon, I'm almost falling asleep now :)

Good night.

Signature · 8 lines
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): 1277992805.1917.142.camel@laptop

A message that is not in the archive (sent off-list or not yet received): 20100701141352.GA17823@aftab

A message that is not in the archive (sent off-list or not yet received): 20100701161133.GE10616@nowhere

A message that is not in the archive (sent off-list or not yet received): 1278000765.1917.220.camel@laptop

A message that is not in the archive (sent off-list or not yet received): 20100701161448.GF10616@nowhere

#15 lore raw

Re: [PATCH 03/21] perf_events: Add a helper to search for an event in a context

From: Frederic Weisbecker <[email protected]>
Date: Thu, Jul 01, 2010 at 12:14:50PM -0400

Hi Frederic,

Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Frederic Weisbecker
Quoted 6 lines

I suspect we need another syscall that can list all the persistent events with a unique id and the attrs that follow.

So you get a unique id for all of them and you can create an fd on top of this id by using a PERF_FLAG_REQUEST_PERSISTENT and this id put in attr.config.

Quoted 5 lines
Quoted 1 line

Isn't that what filesystems were invented for?

The problem is when you create a persistent event, you lose the fd. So you need to retrieve it somehow.

actually the idea is to decouple those from the fd alltogether and provide specific file_operations in debugfs and such, as Peter suggested. Which sounds much more sane to me especially since, at least in the MCE case, all the entities that register into that event need to see the same samples (and read the same buffers etc).

And let's try not to read too much into those persistent events - it may just as well be that we need them only for MCEs and nothing else :)

Signature · 8 lines
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#16 lore raw

Re: [PATCH 03/21] perf_events: Add a helper to search for an event in a context

Quoted 17 lines · Borislav Petkov

From: Frederic Weisbecker <[email protected]>
Date: Thu, Jul 01, 2010 at 12:14:50PM -0400

Hi Frederic,

Quoted 12 lines · Peter Zijlstra
Quoted 9 lines · Frederic Weisbecker
Quoted 6 lines

I suspect we need another syscall that can list all the persistent events with a unique id and the attrs that follow.

So you get a unique id for all of them and you can create an fd on top of this id by using a PERF_FLAG_REQUEST_PERSISTENT and this id put in attr.config.

Isn't that what filesystems were invented for?

Quoted 8 lines
Quoted 2 lines

The problem is when you create a persistent event, you lose the fd. So you need to retrieve it somehow.

actually the idea is to decouple those from the fd alltogether and provide specific file_operations in debugfs and such, as Peter suggested. Which sounds much more sane to me especially since, at least in the MCE case, all the entities that register into that event need to see the same samples (and read the same buffers etc).

Sure the idea of putting that in fs is better.

Note the idea of a new syscall was quite close: it would have listed unique ids of the persistent events, but not fds, you'd have yet to create those fds on top of the ids.

Quoted 3 lines

And let's try not to read too much into those persistent events - it may just as well be that we need them only for MCEs and nothing else :)

Not really. It would be useful for boot tracing, amongst various other things like flight recorder tracing, etc...