perf: add support for arch-dependent symbolic event names to "perf stat"
RFC PATCH 1/1 perf: add support for arch-dependent symbolic event names to "perf stat"
For your review, this patch adds support for arch-dependent symbolic event names to the "perf stat" tool, and could be expanded to other "perf *" commands fairly easily, I suspect.
To support arch-dependent event names without adding arch-dependent code to perf, I added a callout mechanism whereby perf will look for the environment variable: PERF_ARCH_DEP_LIB, and if it exists, it will try to open it as a shared object. If that succeeds, it looks for the symbol "parse_arch_dep_event". If that exists, that function will be called by parse_events() before all of the other event parsing functions in parse-events.c. It is passed the same arguments as the other parse_*_event functions, namely the event string and a pointer to an event attribute structure.
As the code existed, "perf stat" would print out the count results, but for raw events (which is how arch-dependent events are supported in perf_events), it would just print out a raw code. This is not acceptable, especially when a symbolic name was placed on the command line. So I changed the code to save away the event name that was passed on the command line, rather than doing a reverse translation to an event string based on the event type and config fields of the attr structure. In this way, there's no need for a reverse translation function in the arch-dependent library; only a event string->attr struct function is needed.
I could well be missing something, but I don't understand why reverse translation is ever needed in perf, as long as the tool keeps track of the original event strings.
Thanks for your consideration,
- Corey
Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
tools/perf/Makefile +1 -1
@@ -199,7 +199,7 @@ ifndef PERF_DEBUG endif CFLAGS = -ggdb3 -Wall -Wextra -std=gnu99 -Werror $(CFLAGS_OPTIMIZE) -D_FORTIFY_SOURCE=2 $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) -EXTLIBS = -lpthread -lrt -lelf -lm +EXTLIBS = -lpthread -lrt -lelf -lm -ldl ALL_CFLAGS = $(CFLAGS) ALL_LDFLAGS = $(LDFLAGS) STRIP ?= strip
tools/perf/util/parse-events.c +67 -4
131 lines not on this page · raw message
Quoted 25 lines · Corey Ashford
For your review, this patch adds support for arch-dependent symbolic event names to the "perf stat" tool, and could be expanded to other "perf *" commands fairly easily, I suspect.
To support arch-dependent event names without adding arch-dependent code to perf, I added a callout mechanism whereby perf will look for the environment variable:
PERF_ARCH_DEP_LIB, and if it exists, it will try to open it as a shared object. If that succeeds, it looks for the symbol "parse_arch_dep_event". If that exists, that function will be called byparse_events()before all of the other event parsing functions inparse-events.c. It is passed the same arguments as the other parse_*_event functions, namely the event string and a pointer to an event attribute structure.As the code existed, "perf stat" would print out the count results, but for raw events (which is how arch-dependent events are supported in
perf_events), it would just print out a raw code. This is not acceptable, especially when a symbolic name was placed on the command line. So I changed the code to save away the event name that was passed on the command line, rather than doing a reverse translation to an event string based on the event type and config fields of the attr structure. In this way, there's no need for a reverse translation function in the arch-dependent library; only a eventstring->attrstruct function is needed.
Quoted 3 lines
I could well be missing something, but I don't understand why reverse translation is ever needed in perf, as long as the tool keeps track of the original event strings.
A couple of follow-up comments on this patch:
This functionality was designed to provide a generalized interface to an external event name -> attr struct library, such as libpfm4. libpfm4 has an interface that nearly exactly matches parse_*_event() profiles, so it's quite easy to write a small wrapper function to call libpfm4's function.
Ingo Molnar discussed adding some visibility to the arch-dependent event names through some other interface, such as through /sys/devices/pmus perhaps, but that discussion is a long way (as far as I know) from having something usable today. So you could think of this external library approach to be a stop-gap until something better is developed. When/if that new event naming mechanism becomes available, we can easily remove this external library support from perf.
Signature · 10 lines
Regards, - Corey Corey Ashford Software Engineer IBM Linux Technology Center, Linux Toolchain Beaverton, OR 503-578-3507 [email protected]
Quoted 37 lines · Corey Ashford
Quoted 25 lines · Corey Ashford
For your review, this patch adds support for arch-dependent symbolic event names to the "perf stat" tool, and could be expanded to other "perf *" commands fairly easily, I suspect.
To support arch-dependent event names without adding arch-dependent code to perf, I added a callout mechanism whereby perf will look for the environment variable:
PERF_ARCH_DEP_LIB, and if it exists, it will try to open it as a shared object. If that succeeds, it looks for the symbol "parse_arch_dep_event". If that exists, that function will be called byparse_events()before all of the other event parsing functions inparse-events.c. It is passed the same arguments as the other parse_*_event functions, namely the event string and a pointer to an event attribute structure.As the code existed, "perf stat" would print out the count results, but for raw events (which is how arch-dependent events are supported in
perf_events), it would just print out a raw code. This is not acceptable, especially when a symbolic name was placed on the command line. So I changed the code to save away the event name that was passed on the command line, rather than doing a reverse translation to an event string based on the event type and config fields of the attr structure. In this way, there's no need for a reverse translation function in the arch-dependent library; only a eventstring->attrstruct function is needed.Quoted 3 lines
I could well be missing something, but I don't understand why reverse translation is ever needed in perf, as long as the tool keeps track of the original event strings.
A couple of follow-up comments on this patch:
This functionality was designed to provide a generalized interface to an external event name -> attr struct library, such as libpfm4. libpfm4 has an interface that nearly exactly matches parse_*_event() profiles, so it's quite easy to write a small wrapper function to call libpfm4's function.
Quoted 7 lines
Ingo Molnar discussed adding some visibility to the arch-dependent event names through some other interface, such as through
/sys/devices/pmusperhaps, but that discussion is a long way (as far as I know) from having something usable today. So you could think of this external library approach to be a stop-gap until something better is developed. When/if that new event naming mechanism becomes available, we can easily remove this external library support from perf.
I'm quite much against stop-gap measures like this - they tend to become tomorrow's impossible-to-remove quirk.
If you want extensible events you can already do it by providing an ftrace tracepoint event via TRACE_EVENT. They are easy to add and ad-hoc, and are supported throughout by perf.
That could be librarized further by providing an /eventfs or /proc/events interface to enumerate them.
Or if you want to extend the perf events namespace ABI you can send patches for that as well. (It's not a big issue if a particular event is currently only supported on Power for example - as long as you make a good effort naming and structuring it in a reasonably generic way.)
Thanks,
Ingo
On 3/11/2010 4:46 AM, Ingo Molnar wrote:
[snip]
Quoted 3 lines
If you want extensible events you can already do it by providing an ftrace tracepoint event via
TRACE_EVENT. They are easy to add and ad-hoc, and are supported throughout by perf.
Is TRACE_EVENT an appropriate way to add hardware-specific counter events? I will look into this. Thanks for the pointer.
Quoted 3 lines
That could be librarized further by providing an /eventfs or
/proc/eventsinterface to enumerate them.
We can enumerate events this way, but there are other aspects to events than just their names (see below).
Quoted 5 lines
Or if you want to extend the perf events namespace ABI you can send patches for that as well. (It's not a big issue if a particular event is currently only supported on Power for example - as long as you make a good effort naming and structuring it in a reasonably generic way.)
I'm not sure how that would work. The issue I am trying to solve here is that Power arch chips have a large number of very hardware-specific events that are not generalizable. Many of these events not only have names, but other user-configurable bits as well that select or narrow the scope of which exact events are recorded. This issue is dealt with nicely in libpfm4, as it has mechanisms for parsing event names and attributes (aka modifiers or unit masks), and then produces a usable config field for the perf_events_attr struct.
Should I take it from the above that you are completely against the idea of using an external library for hardware-specific event and attribute naming?
Signature · 3 lines
Regards, - Corey
Quoted 32 lines · Corey Ashford
On 3/11/2010 4:46 AM, Ingo Molnar wrote:
[snip]Quoted 3 lines
If you want extensible events you can already do it by providing an ftrace tracepoint event via
TRACE_EVENT. They are easy to add and ad-hoc, and are supported throughout by perf.Is
TRACE_EVENTan appropriate way to add hardware-specific counter events? I will look into this. Thanks for the pointer.Quoted 3 lines
That could be librarized further by providing an /eventfs or
/proc/eventsinterface to enumerate them.We can enumerate events this way, but there are other aspects to events than just their names (see below).
Quoted 5 lines
Or if you want to extend the perf events namespace ABI you can send patches for that as well. (It's not a big issue if a particular event is currently only supported on Power for example - as long as you make a good effort naming and structuring it in a reasonably generic way.)
I'm not sure how that would work. The issue I am trying to solve here is that Power arch chips have a large number of very hardware-specific events that are not generalizable. Many of these events not only have names, but other user-configurable bits as well that select or narrow the scope of which exact events are recorded. This issue is dealt with nicely in libpfm4, as it has mechanisms for parsing event names and attributes (aka modifiers or unit masks), and then produces a usable config field for the
perf_events_attrstruct.
Quoted 3 lines
Should I take it from the above that you are completely against the idea of using an external library for hardware-specific event and attribute naming?
Could you give a few relevant examples of events in question, and the kind of configurability/attributes they have on Power?
Thanks,
Ingo
Quoted 2 lines · Ingo Molnar
* Corey Ashford<[email protected]> wrote:
[snip]
Quoted 10 lines
Quoted 10 lines
I'm not sure how that would work. The issue I am trying to solve here is that Power arch chips have a large number of very hardware-specific events that are not generalizable. Many of these events not only have names, but other user-configurable bits as well that select or narrow the scope of which exact events are recorded. This issue is dealt with nicely in libpfm4, as it has mechanisms for parsing event names and attributes (aka modifiers or unit masks), and then produces a usable config field for the
perf_events_attrstruct.
Quoted 6 lines
Quoted 3 lines
Should I take it from the above that you are completely against the idea of using an external library for hardware-specific event and attribute naming?
Could you give a few relevant examples of events in question, and the kind of configurability/attributes they have on Power?
Here are a few examples for the Power A2 processor. I've distorted the names because PMU architecture isn't publicly released yet.
PM_DE_PMC_9:hrd_mask=0xff:hrd=0x22:pma_mask=0x3fff:pma=0x1b2d:culling_mode=3 PM_EX_0x03:lane=2:vlane=1
PM_OWE_ENG_MAC_FULL:usu=3
Note that the attribute fields shown above are fitted into the config field of the perf_event_attr struct.
Quoted 4 lines
Thanks,
Ingo
Regards,
- Corey
Quoted 24 lines · Corey Ashford
Quoted 2 lines · Ingo Molnar
* Corey Ashford<[email protected]> wrote:
[snip]
Quoted 10 lines
Quoted 10 lines
I'm not sure how that would work. The issue I am trying to solve here is that Power arch chips have a large number of very hardware-specific events that are not generalizable. Many of these events not only have names, but other user-configurable bits as well that select or narrow the scope of which exact events are recorded. This issue is dealt with nicely in libpfm4, as it has mechanisms for parsing event names and attributes (aka modifiers or unit masks), and then produces a usable config field for the
perf_events_attrstruct.Quoted 7 lines
Quoted 3 lines
Should I take it from the above that you are completely against the idea of using an external library for hardware-specific event and attribute naming?
Could you give a few relevant examples of events in question, and the kind of
configurability/attributes they have on Power?
Quoted 7 lines
Here are a few examples for the Power A2 processor. I've distorted the names because PMU architecture isn't publicly released yet.
PM_DE_PMC_9:hrd_mask=0xff:hrd=0x22:pma_mask=0x3fff:pma=0x1b2d:culling_mode=3
PM_EX_0x03:lane=2:vlane=1
PM_OWE_ENG_MAC_FULL:usu=3
Just a follow-up note to this...
I learned that the much of the high-level architecture of the new chip that IBM is working on has been publicly released recently, so I have "undistorted" the event names below:
PM_DC_PMC_9:lpid_mask=0xff:lpid=0x22:pid_mask=0x3fff:pid=0x1b2d:marking_mode=3 PM_REGX_0x03:lane=2:vlane=1
PM_XML_ENG_MAC_FULL:sus=3
DC = Decompression/Compression acceleratorPMC_9 = Peformance monitoring event 9
REGX = Regular eXpression accelerator
XML = XML parsing accelerator
pid = process id to matchpid_mask = process id match mask
lpid = logical partition idlpid_mask = logical partition id mask
sus = source unit select
lane, vlane = signal routing fieldsmarking_mode = used to determine which accelerator work units to mark for performance monitoring
- Corey
Quoted 44 lines · Corey Ashford
Quoted 24 lines · Corey Ashford
Quoted 2 lines · Ingo Molnar
* Corey Ashford<[email protected]> wrote:
[snip]
Quoted 10 lines
Quoted 10 lines
I'm not sure how that would work. The issue I am trying to solve here is that Power arch chips have a large number of very hardware-specific events that are not generalizable. Many of these events not only have names, but other user-configurable bits as well that select or narrow the scope of which exact events are recorded. This issue is dealt with nicely in libpfm4, as it has mechanisms for parsing event names and attributes (aka modifiers or unit masks), and then produces a usable config field for the
perf_events_attrstruct.Quoted 7 lines
Quoted 3 lines
Should I take it from the above that you are completely against the idea of using an external library for hardware-specific event and attribute naming?
Could you give a few relevant examples of events in question, and the kind of
configurability/attributes they have on Power?Quoted 7 lines
Here are a few examples for the Power A2 processor. I've distorted the names because PMU architecture isn't publicly released yet.
PM_DE_PMC_9:hrd_mask=0xff:hrd=0x22:pma_mask=0x3fff:pma=0x1b2d:culling_mode=3
PM_EX_0x03:lane=2:vlane=1
PM_OWE_ENG_MAC_FULL:usu=3Just a follow-up note to this...
I learned that the much of the high-level architecture of the new chip that IBM is working on has been publicly released recently, so I have "undistorted" the event names below:
PM_DC_PMC_9:lpid_mask=0xff:lpid=0x22:pid_mask=0x3fff:pid=0x1b2d:marking_mode=3 PM_REGX_0x03:lane=2:vlane=1
PM_XML_ENG_MAC_FULL:sus=3
Quoted 12 lines
DC = Decompression/Compression accelerator
PMC_9= Peformance monitoring event 9
REGX = Regular eXpression accelerator
XML = XML parsing accelerator
pid = process id to matchpid_mask= process id match mask
lpid = logical partition idlpid_mask= logical partition id mask
sus = source unit select
lane, vlane = signal routing fieldsmarking_mode= used to determine which accelerator work units to mark for performance monitoring
Are these special-purpose instructions for compression/regex/xml-parsing speedups?
I think it would be rather useful to merge the hw (and sw) perf events with the ftrace/tracepoints symbolic events space. That would be a one-stop-shop for both perf and other tools to figure out the events we offer, their characteristics, format, relationship to other events, etc.
Ingo
Quoted 60 lines · Ingo Molnar
Quoted 44 lines · Corey Ashford
Quoted 24 lines · Corey Ashford
Quoted 2 lines · Ingo Molnar
* Corey Ashford<[email protected]> wrote:
[snip]
Quoted 10 lines
Quoted 10 lines
I'm not sure how that would work. The issue I am trying to solve here is that Power arch chips have a large number of very hardware-specific events that are not generalizable. Many of these events not only have names, but other user-configurable bits as well that select or narrow the scope of which exact events are recorded. This issue is dealt with nicely in libpfm4, as it has mechanisms for parsing event names and attributes (aka modifiers or unit masks), and then produces a usable config field for the
perf_events_attrstruct.Quoted 7 lines
Quoted 3 lines
Should I take it from the above that you are completely against the idea of using an external library for hardware-specific event and attribute naming?
Could you give a few relevant examples of events in question, and the kind of
configurability/attributes they have on Power?Quoted 7 lines
Here are a few examples for the Power A2 processor. I've distorted the names because PMU architecture isn't publicly released yet.
PM_DE_PMC_9:hrd_mask=0xff:hrd=0x22:pma_mask=0x3fff:pma=0x1b2d:culling_mode=3
PM_EX_0x03:lane=2:vlane=1
PM_OWE_ENG_MAC_FULL:usu=3Just a follow-up note to this...
I learned that much of the high-level architecture of the new chip that IBM is working on has been publicly released recently, so I have "undistorted" the event names below:
PM_DC_PMC_9:lpid_mask=0xff:lpid=0x22:pid_mask=0x3fff:pid=0x1b2d:marking_mode=3 PM_REGX_0x03:lane=2:vlane=1
PM_XML_ENG_MAC_FULL:sus=3Quoted 12 lines
DC = Decompression/Compression accelerator
PMC_9= Peformance monitoring event 9
REGX = Regular eXpression accelerator
XML = XML parsing accelerator
pid = process id to matchpid_mask= process id match mask
lpid = logical partition idlpid_mask= logical partition id mask
sus = source unit select
lane, vlane = signal routing fieldsmarking_mode= used to determine which accelerator work units to mark for performance monitoring
Quoted 2 lines
Are these special-purpose instructions for compression/regex/xml-parsing speedups?
No, these events are for nest (aka uncore) accelerators for compression/regex/xml-parsing. These accelerators operate independently of the CPU threads and are given work units via request blocks which are then queued up by the accelerator.
Quoted 7 lines
I think it would be rather useful to merge the hw (and sw) perf events with the ftrace/tracepoints symbolic events space. That would be a one-stop-shop for both perf and other tools to figure out the events we offer, their characteristics, format, relationship to other events, etc.
Ingo
Ok, I will look into this. Thank you for your advice.
- Corey
Quoted 8 lines · Ingo Molnar
Quoted 4 lines · Corey Ashford
Quoted 3 lines · Corey Ashford
For your review, this patch adds support for arch-dependent symbolic event names to the "perf stat" tool, and could be expanded to other "perf *" commands fairly easily, I suspect.
Quoted 6 lines
I'm quite much against stop-gap measures like this - they tend to become tomorrow's impossible-to-remove quirk.
If you want extensible events you can already do it by providing an ftrace tracepoint event via
TRACE_EVENT. They are easy to add and ad-hoc, and are supported throughout by perf.
If I've understood correctly what Corey is doing, I think you're missing the point. The idea, I thought, was to provide a way to be able to use symbolic names for raw hardware events rather than just numbers. I don't see how ftrace tracepoint events are relevant to that.
Now as to whether an external .so is the best way to provide the processor-specific mapping of names to raw events, I'm not sure. If the kernel can provide that mapping via procfs, sysfs or eventfs, that would be an alternative, but it does mean the kernel has those tables in unswappable memory (and potentially the tables for all the processors that the kernel supports), which seems unnecessary. Or they can just be added to the perf source code.
Paul.
Quoted 12 lines · Paul Mackerras
Quoted 11 lines · Ingo Molnar
Quoted 4 lines · Corey Ashford
Quoted 3 lines · Corey Ashford
For your review, this patch adds support for arch-dependent symbolic event names to the "perf stat" tool, and could be expanded to other "perf *" commands fairly easily, I suspect.
I'm quite much against stop-gap measures like this - they tend to become tomorrow's impossible-to-remove quirk.
Quoted 8 lines
Quoted 3 lines
If you want extensible events you can already do it by providing an ftrace tracepoint event via
TRACE_EVENT. They are easy to add and ad-hoc, and are supported throughout by perf.If I've understood correctly what Corey is doing, I think you're missing the point. The idea, I thought, was to provide a way to be able to use symbolic names for raw hardware events rather than just numbers.
Yes, that's what I meant.
Quoted 10 lines
I don't see how ftrace tracepoint events are relevant to that.
Now as to whether an external .so is the best way to provide the processor-specific mapping of names to raw events, I'm not sure. If the kernel can provide that mapping via procfs, sysfs or eventfs, that would be an alternative, but it does mean the kernel has those tables in unswappable memory (and potentially the tables for all the processors that the kernel supports), which seems unnecessary. Or they can just be added to the perf source code.
In addition to the names and attributes, we'd also need text-based descriptions of the events and attributes.
I'm not opposed to the idea of placing them in sysfs (or other pseudo fs), but it's also not clear to me how to represent the event data in a clean, extensible, and space/performance efficient way. That said, I do like the idea of being able to navigate events by looking through a directory structure which is possibly organized by the physical topology of the system and its PMUs.
- Corey
Quoted 12 lines · Paul Mackerras
Quoted 11 lines · Ingo Molnar
Quoted 4 lines · Corey Ashford
Quoted 3 lines · Corey Ashford
For your review, this patch adds support for arch-dependent symbolic event names to the "perf stat" tool, and could be expanded to other "perf *" commands fairly easily, I suspect.
I'm quite much against stop-gap measures like this - they tend to become tomorrow's impossible-to-remove quirk.
Quoted 8 lines
Quoted 3 lines
If you want extensible events you can already do it by providing an ftrace tracepoint event via
TRACE_EVENT. They are easy to add and ad-hoc, and are supported throughout by perf.If I've understood correctly what Corey is doing, I think you're missing the point. The idea, I thought, was to provide a way to be able to use symbolic names for raw hardware events rather than just numbers. I don't see how ftrace tracepoint events are relevant to that.
tracepoints are relevant because they are the currently best way of how we assign symbolic names to various kernel-internal events. For ad-hoc usecases like this:
http://dri.freedesktop.org/wiki/IntelPerformanceTuning
I'd much rather see that facility used (and, to the extent needed, extended) to provide support for rare arch events that we dont want to enumerate in a generic way.
Or, if the events are important enough to be hardcoded into the perf ABI itself, they should be generalized in a meaningful way - even if you dont expect them to show up on other CPUs.
Ingo
(I posted this yesterday, but I think LKML rejected it because there was more quoted text than my text. So I am reposting with no quoted text).
A couple of follow-up comments on this patch:
This functionality was designed to provide a generalized interface to an external event name -> attr struct library, such as libpfm4. libpfm4 has an interface that nearly exactly matches parse_*_event() profiles, so it's quite easy to write a small wrapper function to call libpfm4's function.
Ingo Molnar discussed adding some visibility to the arch-dependent event names through some other interface, such as through /sys/devices/pmus perhaps, but that discussion is a long way (as far as I know) from having something usable today. So you could think of this external library approach to be a stop-gap until something better is developed. When/if that new event naming mechanism becomes available, we can easily remove this external library support from perf.
Signature · 9 lines
Regards, - Corey Corey Ashford Software Engineer IBM Linux Technology Center, Linux Toolchain Beaverton, OR [email protected]