perf: implement AMD IBS (v2)

This patch introduces a method to specify the type of a raw sample. This can be used to setup hardware events other than generic performance counters by passing special config data to the pmu. The config data can be interpreted different from generic events and thus can be used for other purposes.

The raw_type attribute is an extension of the ABI. It reuses the unused bp_type space for this. Generic performance counters can be setup by setting the raw_type attribute to null. Thus special raw events must have a type other than null.

Raw types can be defined as needed for cpu models or architectures. To keep backward compatibility all architectures must return an error for an event with a raw_type other than null that is not supported.

E.g., raw_type can be used to setup IBS on an AMD cpu. IBS is not common to pmu features from other vendors or architectures. The pmu must be setup with a special config value. Sample data is returned in a certain format back to the userland. An IBS event is created by setting a raw event and encoding the IBS type in raw_type. The pmu handles this raw event then and passes raw sample data back.

Raw type could be architecure specific, e.g. for x86:

enum perf_raw_type {

        PERF_RAW_PERFCTR                        = 0,
        PERF_RAW_IBS_FETCH                      = 1,
        PERF_RAW_IBS_OP                         = 2,

PERF_RAW_MAX,
};

Null is the architecture's default, meaning for x86 a perfctr.

Maybe the raw type definition could also be part of the ABI with one definition for all architectures.

To use raw events with perf, the raw event syntax could be suffixed by the type (as for breakpoints):

   -e rNNN[:TYPE]

Example:

perf record -e r186A:1 # ... meaning IBS fetch, cycle count 100000
perf record -e r0:1 -c 100000 # ... the same

Or with named types:

perf record -e r186A:IBS_FETCH ...
perf record -e r0:IBS_FETCH -c 100000 ...

[...]

Have you looked at Lin Ming's patch series? I think it offers another way to support IBS and other arch-specific and off-chip PMUs in a more general way, though it's not quite fully-baked yet.

Whether IBS fits that category is an open question - ideally it should have similar high level usage as the PEBS/LBR bits on Intel CPUs. (at least as far as tooling goes)

Quoted 2 lines

perf record -e r186A:IBS_FETCH ...
perf record -e r0:IBS_FETCH -c 100000 ...

Should this raw value have been 186A0 instead of 186A?

Where is the named type translation coming from? Is this something that needs to be hard-coded into perf?

Have you looked at Lin Ming's patch series? I think it offers another way to support IBS and other arch-specific and off-chip PMUs in a more general way, though it's not quite fully-baked yet.

This patch introduces a method to specify the type of a raw sample. This can be used to setup hardware events other than generic performance counters by passing special config data to the pmu. The config data can be interpreted different from generic events and thus can be used for other purposes.

The raw_type attribute is an extension of the ABI. It reuses the unused bp_type space for this. Generic performance counters can be setup by setting the raw_type attribute to null. Thus special raw events must have a type other than null.

Raw types can be defined as needed for cpu models or architectures. To keep backward compatibility all architectures must return an error for an event with a raw_type other than null that is not supported.

E.g., raw_type can be used to setup IBS on an AMD cpu. IBS is not common to pmu features from other vendors or architectures. The pmu must be setup with a special config value. Sample data is returned in a certain format back to the userland. An IBS event is created by setting a raw event and encoding the IBS type in raw_type. The pmu handles this raw event then and passes raw sample data back.

Raw type could be architecure specific, e.g. for x86:

enum perf_raw_type {
       PERF_RAW_PERFCTR                        = 0, PERF_RAW_IBS_FETCH                      = 1, PERF_RAW_IBS_OP                         = 2,

       PERF_RAW_MAX,
};

Null is the architecture's default, meaning for x86 a perfctr.

Maybe the raw type definition could also be part of the ABI with one definition for all architectures.

To use raw events with perf, the raw event syntax could be suffixed by the type (as for breakpoints):

  -e rNNN[:TYPE]

Example:

 perf record -e r186A:1          # ... meaning IBS fetch, cycle count 100000 perf record -e r0:1 -c 100000   # ... the same

Or with named types:

 perf record -e r186A:IBS_FETCH ...
 perf record -e r0:IBS_FETCH -c 100000 ...

This solution has a number of advantages: A raw event type may be specified without to encode the type in the config value. The attr flags are not 'polluted'. We can follow the already existing breakpoint concept in syntax and encoding.

Signed-off-by Robert Richter <[email protected]>

---
 arch/arm/kernel/perf_event.c             |    5 +++++ arch/powerpc/kernel/perf_event.c         |    2 ++ arch/powerpc/kernel/perf_event_fsl_emb.c |    2 ++ arch/sh/kernel/perf_event.c              |    2 ++ arch/x86/kernel/cpu/perf_event.c         |    5 ++++- arch/x86/kernel/cpu/perf_event_amd.c     |    3 +++ arch/x86/kernel/cpu/perf_event_intel.c   |    3 +++ arch/x86/kernel/cpu/perf_event_p4.c      |    5 +++++ include/linux/perf_event.h               |    5 ++++-

 9 files changed, 30 insertions(+), 2 deletions(-)

arch/arm/kernel/perf_event.c +5 -0

index 9e70f20..73d680c 100644
@@ -388,6 +388,11 @@ __hw_perf_event_init(struct perf_event *event)
        } else if (PERF_TYPE_HW_CACHE == event->attr.type) {
                mapping = armpmu_map_cache_event(event->attr.config);
        } else if (PERF_TYPE_RAW == event->attr.type) {
+               if (event->attr.raw_type) {
+                       pr_debug("invalid raw type %x\n",
+                                event->attr.raw_type);
+                       return -EINVAL;
+               }
                mapping = armpmu->raw_event(event->attr.config);
        } else {
                pr_debug("event type %x not supported\n", event->attr.type);

arch/powerpc/kernel/perf_event.c +0 -0

index 43b83c3..c8fb3cf 100644
@@ -1036,6 +1036,8 @@ const struct pmu *hw_perf_event_init(struct perf_event *event)
                        return ERR_PTR(err);
                break;

       case PERF_TYPE_RAW:
+               if (event->attr.raw_type)
+                       return ERR_PTR(-EINVAL);
               ev = event->attr.config;
               break;
       default:

arch/powerpc/kernel/perf_event_fsl_emb.c +0 -0

index 369872f..7547e96 100644
@@ -452,6 +452,8 @@ const struct pmu *hw_perf_event_init(struct perf_event *event)
                break;
 

       case PERF_TYPE_RAW:
+               if (event->attr.raw_type)
+                       return ERR_PTR(-EINVAL);
               ev = event->attr.config;
               break;

arch/sh/kernel/perf_event.c +0 -0

index 81b6de4..482cf48 100644
@@ -142,6 +142,8 @@ static int __hw_perf_event_init(struct perf_event *event)
 
        switch (attr->type) {

       case PERF_TYPE_RAW:
+               if (attr->raw_type)
+                       return -EINVAL;
               config = attr->config & sh_pmu->raw_event_mask; break;
       case PERF_TYPE_HW_CACHE:

arch/x86/kernel/cpu/perf_event.c +4 -1

index fd4db0d..3539b53 100644
@@ -449,8 +449,11 @@ static int x86_setup_perfctr(struct perf_event *event)
                        return -EOPNOTSUPP;
        }
 
-       if (attr->type == PERF_TYPE_RAW)
+       if (attr->type == PERF_TYPE_RAW) {
+               if (attr->raw_type)
+                       return -EINVAL;
                return 0;
+       }
 
        if (attr->type == PERF_TYPE_HW_CACHE)
                return set_ext_hw_attr(hwc, attr);

arch/x86/kernel/cpu/perf_event_amd.c +3 -0

index 611df11..87e5ae4 100644
@@ -121,6 +121,9 @@ static int amd_pmu_hw_config(struct perf_event *event)
        if (event->attr.type != PERF_TYPE_RAW)
                return 0;
 
+       if (event->attr.raw_type)
+               return -EINVAL;
+
        event->hw.config |= event->attr.config & AMD64_RAW_EVENT_MASK;
 
        return 0;

arch/x86/kernel/cpu/perf_event_intel.c +3 -0

index fdbc652..d15faf5 100644
@@ -770,6 +770,9 @@ static int intel_pmu_hw_config(struct perf_event *event)
        if (event->attr.type != PERF_TYPE_RAW)
                return 0;
 
+       if (event->attr.raw_type)
+               return -EINVAL;
+
        if (!(event->attr.config & ARCH_PERFMON_EVENTSEL_ANY))
                return 0;
 

arch/x86/kernel/cpu/perf_event_p4.c +5 -0

index 87e1803..1001892 100644
@@ -437,6 +437,11 @@ static int p4_hw_config(struct perf_event *event)
                event->hw.config = p4_set_ht_bit(event->hw.config);
 
        if (event->attr.type == PERF_TYPE_RAW) {
+               /* only raw perfctr config supported */
+               if (event->attr.raw_type) {
+                       rc = -EINVAL;
+                       goto out;
+               }
 
                /* user data may have out-of-bound event index */
                evnt = p4_config_unpack_event(event->attr.config);

include/linux/perf_event.h +0 -0

index fe50347..f9d2d5e 100644
@@ -222,7 +222,10 @@ struct perf_event_attr {

               __u32           wakeup_watermark; /* bytes before wakeup   */ };

-       __u32                   bp_type;
+       union {
+               __u32           bp_type;
+               __u32           raw_type;
+       };
       __u64                   bp_addr;
       __u64                   bp_len;
 };
--
1.7.1

I still don't understand why you need all of this to encode IBS. I still believe that with attr.config there is plenty of bits to choose from. I do understand the need for PERF_SAMPLE_RAW. I think there is no other way.

You simply need to pick an encoding to mark the config as IBS. You need two bits for this: 00 regular counters, 01 IBS Fetch, 10 IBS op. Regular counters use 43 bits, IBS fetch uses 58, IBS op uses 52. So you could use bits 62-63 for instance. You don't need to encode the sampling period in attr.config for either IBS. You can use attr.sample_period, so you free up 16 bits.

I understand that IBS may evolve and thus may use more bits. But you still have at least 16 bits of margin.

Users and tools would rely on an library to provide the event encoding. No need to come up with some raw hex number on the cmdline.

Quoted 12 lines · Stephane Eranian

I still don't understand why you need all of this to encode IBS. I still believe that with attr.config there is plenty of bits to choose from. I do understand the need for PERF_SAMPLE_RAW. I think there is no other way.

You simply need to pick an encoding to mark the config as IBS. You need two bits for this: 00 regular counters, 01 IBS Fetch, 10 IBS op. Regular counters use 43 bits, IBS fetch uses 58, IBS op uses 52. So you could use bits 62-63 for instance. You don't need to encode the sampling period in attr.config for either IBS. You can use attr.sample_period, so you free up 16 bits.

Quoted 5 lines

I understand that IBS may evolve and thus may use more bits. But you still have at least 16 bits of margin.

Users and tools would rely on an library to provide the event encoding. No need to come up with some raw hex number on the cmdline.

No need for any of that afaict, how about:

For Instruction-Fetch:

 0:15 sample-period (r/w)
 16:31 cnt           (r/w)
 32:47 latency       (r/w)
   48 enable        (r/w)
   49 valid         (r/w)
 50:56               (ro)
   57 randomized    (r/w)

For Instruction-Execution:

 0:15 sample-period (r/w)
   17 enable        (r/w)
   18 valid         (r/w)

So if we add perf_event_attr::latency (can also be used for PEBS-load-latency, can Sparc/PowerPC/ARM/SH use this too?), we can

encode these IBS things as:

 0x87 Instruction Fetch Stall -- Ins-Fetch
 0xC0 Retired Instructions    -- Ins-Exec

When we set perf_event_attr::precise > 0

The Ins-Exec will have to re-construct the actual event->count by adding sample-period on each interrupt, as it seems we lack an actual counter in hardware.

Furthermore, these counters will have to deal with sample-period > 2^16 by 'ignoring' interrupts until we get ->period_left down to 0.

The extra data could possibly be exposed through attaching non-sampling group events and using SAMPLE_READ, like L1-misses, although reconstructing the count from just one bit seems 'interesting'.

The IbsFetchLinAd/IbsOpRip would go straight into PERF_SAMPLE_IP by replacing pt_regs->ip I guess.

IbsDcLinAd goes into PERF_SAMPLE_ADDR

Quoted 3 lines

For Instruction-Fetch:

32:47 latency (r/w)

Your are mixing output and input parameters.

The only input parameters you have are:
- sample-period, enable, random
The rest is output only.

Quoted 5 lines

encode these IBS things as:

0x87 Instruction Fetch Stall -- Ins-Fetch
0xC0 Retired Instructions -- Ins-Exec

I think those events do not map to the behavior of IBS. We have add that discussion before.

Quoted 4 lines

The Ins-Exec will have to re-construct the actual event->count by adding sample-period on each interrupt, as it seems we lack an actual counter in hardware.

For what? counting mode?

Quoted 3 lines

Furthermore, these counters will have to deal with sample-period > 2^16 by 'ignoring' interrupts until we get ->period_left down to 0.

Well, it's not 2^16, it's 2^20 but bottom 4 bits must be zero. What about simply failing perf_event_open() is sample_period does not fit the constraint?

Quoted 9 lines

The extra data could possibly be exposed through attaching non-sampling group events and using SAMPLE_READ, like L1-misses, although reconstructing the count from just one bit seems 'interesting'.

The IbsFetchLinAd/IbsOpRip would go straight into PERF_SAMPLE_IP by replacing pt_regs->ip I guess.

IbsDcLinAd goes into PERF_SAMPLE_ADDR

What about the rest, the TLB, alignment, data sources?

Quoted 9 lines · Stephane Eranian

Quoted 3 lines

For Instruction-Fetch:

 32:47 latency       (r/w)

Your are mixing output and input parameters.

The only input parameters you have are:
- sample-period, enable, random
The rest is output only.

Ah, my bad, I thought it was a r/w field.

Quoted 5 lines

Quoted 5 lines

encode these IBS things as:

 0x87 Instruction Fetch Stall -- Ins-Fetch
 0xC0 Retired Instructions    -- Ins-Exec

Quoted 2 lines

I think those events do not map to the behavior of IBS. We have add that discussion before.

Hrm,. so there are no regular events that count the same thing as the IBS things? That really sucks.

So yeah, you might as well expose it as a whole separate PMU using Lin's stuff.

Quoted 6 lines

Quoted 5 lines

The Ins-Exec will have to re-construct the actual event->count by adding sample-period on each interrupt, as it seems we lack an actual counter in hardware.

For what? counting mode?

Yeah, events are supposed to count.

Quoted 6 lines

Quoted 3 lines

Furthermore, these counters will have to deal with sample-period > 2^16 by 'ignoring' interrupts until we get ->period_left down to 0.

Well, it's not 2^16, it's 2^20 but bottom 4 bits must be zero. What about simply failing perf_event_open() is sample_period does not fit the constraint?

Why, its simple enough to ignore a few interrupts, we do the same for all other implementations.

Quoted 7 lines

Quoted 7 lines

The extra data could possibly be exposed through attaching non-sampling group events and using SAMPLE_READ, like L1-misses, although reconstructing the count from just one bit seems 'interesting'.

The IbsFetchLinAd/IbsOpRip would go straight into PERF_SAMPLE_IP by replacing pt_regs->ip I guess.

Quoted 3 lines

Quoted 2 lines

IbsDcLinAd goes into PERF_SAMPLE_ADDR

What about the rest, the TLB, alignment, data sources?

Dunno, reconstruct sensible counters? Surely the software that uses IBS does something useful with that data? What does libpfm do with the IBS data?

What's wrong with creating pseudo-events for IBS? We'd have to pick two unused event codes. That would have the advantage of making it explicit you're using IBS. I think we could still use the precise_ip field if people are only interested in the IP. They would use PERF_SAMPLE_RAW if they need more.

Quoted 8 lines

Quoted 5 lines

Quoted 4 lines

The Ins-Exec will have to re-construct the actual event->count by adding sample-period on each interrupt, as it seems we lack an actual counter in hardware.

For what? counting mode?

Yeah, events are supposed to count.

IBS is a sampling only feature. I suspect it would be okay to return 0 here or do as you said, count the number of IBS interrupts and multiply by the sampling period.

Quoted 3 lines

Dunno, reconstruct sensible counters? Surely the software that uses IBS does something useful with that data? What does libpfm do with the IBS data?

Quoted 3 lines

Quoted 3 lines

Dunno, reconstruct sensible counters? Surely the software that uses IBS does something useful with that data? What does libpfm do with the IBS data?

Here are some use cases described:

http://developer.amd.com/assets/AMD_IBS_paper_EN.pdf
http://developer.amd.com/cpu/CodeAnalyst/assets/ISPASS2010_IBS_CA_abstract.pdf

So yeah, you might as well expose it as a whole separate PMU using Lin's stuff.

Quoted 2 lines · Peter Zijlstra

So yeah, you might as well expose it as a whole separate PMU using Lin's stuff.

This seems to be the most promising alternative to raw_type. But it's also much more bloated requiring sysfs and additional file descriptors.

Not really, fds are really cheap on Linux, and in any case it's only needed when the event is created - can be closed afterwards.

Quoted 2 lines · Ingo Molnar

Not really, fds are really cheap on Linux, and in any case it's only needed when the event is created - can be closed afterwards.

I was wondering if we could get the perf event fd directly with the open("/sys/...") command? This would be really easy to set up.

Quoted 12 lines · Stephane Eranian

I still don't understand why you need all of this to encode IBS. I still believe that with attr.config there is plenty of bits to choose from. I do understand the need for PERF_SAMPLE_RAW. I think there is no other way.

You simply need to pick an encoding to mark the config as IBS. You need two bits for this: 00 regular counters, 01 IBS Fetch, 10 IBS op. Regular counters use 43 bits, IBS fetch uses 58, IBS op uses 52. So you could use bits 62-63 for instance. You don't need to encode the sampling period in attr.config for either IBS. You can use attr.sample_period, so you free up 16 bits.

Quoted 2 lines

I understand that IBS may evolve and thus may use more bits. But you still have at least 16 bits of margin.

Quoted 2 lines

Users and tools would rely on an library to provide the event encoding. No need to come up with some raw hex number on the cmdline.

For Instruction-Fetch:

0:15 sample-period (r/w)
57 randomized (r/w)

For Instruction-Execution:

0:15 sample-period (r/w)

0x87 Instruction Fetch Stall -- Ins-Fetch
0xC0 Retired Instructions -- Ins-Exec

Furthermore, these counters will have to deal with sample-period > 2^16 by 'ignoring' interrupts until we get ->period_left down to 0.

The extra data could possibly be exposed through attaching non-sampling group events and using SAMPLE_READ, like L1-misses, although reconstructing the count from just one bit seems 'interesting'.

The IbsFetchLinAd/IbsOpRip would go straight into PERF_SAMPLE_IP by replacing pt_regs->ip I guess.

IbsDcLinAd goes into PERF_SAMPLE_ADDR

Quoted 13 lines · Peter Zijlstra

Quoted 12 lines · Stephane Eranian

I still don't understand why you need all of this to encode IBS. I still believe that with attr.config there is plenty of bits to choose from. I do understand the need for PERF_SAMPLE_RAW. I think there is no other way.

You simply need to pick an encoding to mark the config as IBS. You need two bits for this: 00 regular counters, 01 IBS Fetch, 10 IBS op. Regular counters use 43 bits, IBS fetch uses 58, IBS op uses 52. So you could use bits 62-63 for instance. You don't need to encode the sampling period in attr.config for either IBS. You can use attr.sample_period, so you free up 16 bits.

Quoted 2 lines

Quoted 2 lines

I understand that IBS may evolve and thus may use more bits. But you still have at least 16 bits of margin.

We have some bits in the msrs that are reserved now, for perfctr and also for ibs. We could start reusing it to mark a special sample type and so on. So far, ok. Somewhen in the future there is a hw extensions and these bits are not reserved anymore, what now?

In case the register layout changes, you start workaround this shifting and masking bits back and forth. This will end in a mess. We don't know what will change and when, but *reserved* means it could be subject to change. So, actually you have to assume arbitrary 64 bit config values for perfctrs and ibs. Thus, you cannot use reserved bits for encoding, doing this would be a hack.

The approach is also not extendable for new pmu features. The question will rise again then how to encode it.

So encoding this information in some other attribute like raw_type or any other field is a much cleaner and a more stable solution, easier to program and to maintain. And there is enough space in the attribute structure.

Quoted 2 lines

Quoted 2 lines

Users and tools would rely on an library to provide the event encoding. No need to come up with some raw hex number on the cmdline.

We can pack everything in tools and libraries and hide direct access to the user. Now, one reads the IBS spec and wants to start using it. He will have to learn (by reading the source or documentation) now, how to pass parameters using the tool or library to set certain bits in the registers. This adds an additional i/f layer what needs to be defined, documented, implemented and maintained. Why not keep things simple and let the user pass a register value to the pmu? Only the config has to be validated in the kernel and that's it. The hw spec is then the reference for using the interface. We have already raw config values for counters and there was a reason for it.

Quoted 11 lines

For Instruction-Fetch:

  0:15 sample-period (r/w)
    57 randomized    (r/w)

For Instruction-Execution:

  0:15 sample-period (r/w)

  0x87 Instruction Fetch Stall -- Ins-Fetch
  0xC0 Retired Instructions    -- Ins-Exec

I agree, where possible we should reuse attributes or existing events for configuration. But since the pmu behavior differs, there is the risk of mixing different things together. "precise" would be different on Intel and AMD. This will be hidden to the user leading to wrong assumptions and results. And, what about new options that don't have equivalent attribute flags?

But it's not the question how to pass the config, the question is how to mark the event configuration as a certain IBS event.

Quoted 2 lines

Furthermore, these counters will have to deal with sample-period > 2^16 by 'ignoring' interrupts until we get ->period_left down to 0.

No question here, ibs counters should be extended to 64 bit counters, but this is not addressed with this patch set, I have kept this for later work.

Quoted 7 lines

The extra data could possibly be exposed through attaching non-sampling group events and using SAMPLE_READ, like L1-misses, although reconstructing the count from just one bit seems 'interesting'.

The IbsFetchLinAd/IbsOpRip would go straight into PERF_SAMPLE_IP by replacing pt_regs->ip I guess.

Quoted 1 line

IbsDcLinAd goes into PERF_SAMPLE_ADDR

It's the same for sampling data as with config data. We can spread it all over its data structures, but a user will then recollect all of it again needing to know where the information can be found. And, it will never be the same kind of information if the pmus are different.

Quoted 3 lines

But it's not the question how to pass the config, the question is how to mark the event configuration as a certain IBS event.

Yes, we agree on that.

Quoted 5 lines

We have some bits in the msrs that are reserved now, for perfctr and also for ibs. We could start reusing it to mark a special sample type and so on. So far, ok. Somewhen in the future there is a hw extensions and these bits are not reserved anymore, what now?

You will always have the sampling period bits. Those will always be handled by attr->sample_period. And they correspond to the lower bits which are also used to encode the event select on regular counters. So you could identify IBS with special event selects in the lower 8 bits for instance.

IBS is a very model specific feature, as such the method you use to encode it may change.

In my new proposal, I am not using reserved bits, I am using bits which are stored elsewhere in the attr structure.

There is no guarantee your scheme will help there either. Nobody knows the interface to those new features. It may be that you will need more than 64 bits.

What I am missing with raw_type is how it plays out with attr->type which already has RAW? Seems like you have a second level of raw.

Well, I don't think you want to define an attribute for each and every bit of info returned by IBS. I think you need to return the IBSDATA regs as is and let the user pick and choose. Of course, the information can change, but IBS is model specific, it returns something richer than just a count.