Update new perf PEBS tables patchkit
Subsystems (2)
This patchkit revamps the PEBS tables on most Intel CPUs, both simplifying and fixing a couple of problems on Haswell:
- All PEBS events are supported
- The address and data source is now always correctly reported. It also fixes one regression added earlier.
I added the fix from Stephane to correctly initialize the data source default for other events. No open issues in this version, should be good to merge.
-Andi
PATCH 1/4 perf, x86: Revamp PEBS event selection
From: Andi Kleen <[email protected]>
The basic idea is that it does not make sense to list all PEBS events individually. The list is very long, sometimes outdated and the hardware doesn't need it. If an event does not support PEBS it will just not count, there is no security issue.
We need to only list events that something special, like supporting load or store addresses.
This vastly simplifies the PEBS event selection. It also speeds up the scheduling because the scheduler doesn't have to walk as many constraints.
Bugs fixed:
- We do not allow setting forbidden flags with PEBS anymore (SDM 18.9.4), except for the special cycle event.
This is done using a new constraint macro that also
matches on the event flags.
- Correct DataLA and load/store/na flags reporting on Haswell [Requires a followon patch]
- We did not allow all PEBS events on Haswell:
We were missing some valid subevents in d1-d2 (MEM_LOAD_UOPS_RETIRED.*, MEM_LOAD_UOPS_RETIRED_L3_HIT_RETIRED.*)
This includes the changes proposed by Stephane earlier and obsoletes his patchkit (except for some changes on pre Sandy Bridge/Silvermont CPUs)
I only did Sandy Bridge and Silvermont and later so far, mostly because these are the parts I could directly confirm the hardware behavior with hardware architects. Also I do not believe the older CPUs have any
missing events in their PEBS list, so there's no pressing
need to change them.
I did not implement the flag proposed by Peter to allow setting forbidden flags. If really needed this could be implemented on to of this patch.
Cc: [email protected]
v2: Fix broken store events on SNB/IVB (Stephane Eranian)
v3: More fixes. Rename some arguments (Stephane Eranian)
v4: List most Haswell events individually again to report memory operation type correctly.
Add new flags to describe load/store/na for datala. Update description.
--- arch/x86/include/asm/perf_event.h | 8 +++ arch/x86/kernel/cpu/perf_event.h | 48 ++++++++++++-- arch/x86/kernel/cpu/perf_event_intel_ds.c | 107 ++++++++++-------------------- 3 files changed, 85 insertions(+), 78 deletions(-)
arch/x86/include/asm/perf_event.h +8 -0
@@ -51,6 +51,14 @@ ARCH_PERFMON_EVENTSEL_EDGE | \ ARCH_PERFMON_EVENTSEL_INV | \ ARCH_PERFMON_EVENTSEL_CMASK) +#define X86_ALL_EVENT_FLAGS \ + (ARCH_PERFMON_EVENTSEL_EDGE | \ + ARCH_PERFMON_EVENTSEL_INV | \ + ARCH_PERFMON_EVENTSEL_CMASK | \ + ARCH_PERFMON_EVENTSEL_ANY | \ + ARCH_PERFMON_EVENTSEL_PIN_CONTROL | \ + HSW_IN_TX | \ + HSW_IN_TX_CHECKPOINTED)#define AMD64_RAW_EVENT_MASK \ (X86_RAW_EVENT_MASK | \ AMD64_EVENTSEL_EVENT)
arch/x86/kernel/cpu/perf_event.h +42 -6
68 lines not on this page · raw message
arch/x86/kernel/cpu/perf_event_intel_ds.c +35 -72
145 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH 2/4 perf, x86: Don't mark DataLA addresses as store
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Haswell supports reporting the data address for a range of PEBS events, including:
UOPS_RETIRED.ALL
MEM_UOPS_RETIRED.STLB_MISS_LOADS
MEM_UOPS_RETIRED.STLB_MISS_STORES
MEM_UOPS_RETIRED.LOCK_LOADS
MEM_UOPS_RETIRED.SPLIT_LOADS
MEM_UOPS_RETIRED.SPLIT_STORES
MEM_UOPS_RETIRED.ALL_LOADS
MEM_UOPS_RETIRED.ALL_STORES
MEM_LOAD_UOPS_RETIRED.L1_HIT
MEM_LOAD_UOPS_RETIRED.L2_HIT
MEM_LOAD_UOPS_RETIRED.L3_HIT
MEM_LOAD_UOPS_RETIRED.L1_MISS
MEM_LOAD_UOPS_RETIRED.L2_MISS
MEM_LOAD_UOPS_RETIRED.L3_MISS
MEM_LOAD_UOPS_RETIRED.HIT_LFB
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_MISS
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HIT
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HITM
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_NONE
MEM_LOAD_UOPS_L3_MISS_RETIRED.LOCAL_DRAM
This facility was already enabled earlier with the original Haswell perf changes.
However these addresses were always reports as stores by perf, which is wrong, as they could be loads or NA too.
This patch uses the load/store/na flags added earlier to report the correct operation based on the event type. For some events this is NA.
v2: Supports load/stores/na again instead of marking everything NA
--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_ds.c +10 -3
@@ -108,13 +108,19 @@ static u64 precise_store_data(u64 status)return val; } -static u64 precise_store_data_hsw (struct perf_event *event,u64 status) +static u64 precise_store_data_hsw (struct perf_event *event,u64 status, +unsigned flags) {union perf_mem_data_src dse;u64 cfg = event->hw.config &INTEL_ARCH_EVENT_MASK ; dse.val =0 ; - dse.mem_op =PERF_MEM_OP_STORE ; +if (flags &PERF_X86_EVENT_PEBS_LD_HSW ) + dse.mem_op =PERF_MEM_OP_LOAD ; +else if (flags &PERF_X86_EVENT_PEBS_ST_HSW ) + dse.mem_op =PERF_MEM_OP_STORE ; +else + dse.mem_op =PERF_MEM_OP_NA ; dse.mem_lvl =PERF_MEM_LVL_NA ;/*
@@ -868,7 +874,8 @@ static void __intel_pmu_pebs_event(struct perf_event *event,PERF_X86_EVENT_PEBS_LD_HSW | PERF_X86_EVENT_PEBS_NA_HSW)) data.data_src.val = -precise_store_data_hsw (event, pebs->dse); +precise_store_data_hsw (event, pebs->dse, + event->hw.flags); else data.data_src.val = precise_store_data(pebs->dse); }
Signature · 1 line
1.9.3
Re: [PATCH 2/4] perf, x86: Don't mark DataLA addresses as store
Quoted 47 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Haswell supports reporting the data address for a range of PEBS events, including:
UOPS_RETIRED.ALL
MEM_UOPS_RETIRED.STLB_MISS_LOADS
MEM_UOPS_RETIRED.STLB_MISS_STORES
MEM_UOPS_RETIRED.LOCK_LOADS
MEM_UOPS_RETIRED.SPLIT_LOADS
MEM_UOPS_RETIRED.SPLIT_STORES
MEM_UOPS_RETIRED.ALL_LOADS
MEM_UOPS_RETIRED.ALL_STORES
MEM_LOAD_UOPS_RETIRED.L1_HIT
MEM_LOAD_UOPS_RETIRED.L2_HIT
MEM_LOAD_UOPS_RETIRED.L3_HIT
MEM_LOAD_UOPS_RETIRED.L1_MISS
MEM_LOAD_UOPS_RETIRED.L2_MISS
MEM_LOAD_UOPS_RETIRED.L3_MISS
MEM_LOAD_UOPS_RETIRED.HIT_LFB
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_MISS
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HIT
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_HITM
MEM_LOAD_UOPS_L3_HIT_RETIRED.XSNP_NONE
MEM_LOAD_UOPS_L3_MISS_RETIRED.LOCAL_DRAMThis facility was already enabled earlier with the original Haswell perf changes.
However these addresses were always reports as stores by perf, which is wrong, as they could be loads or NA too.
This patch uses the load/store/na flags added earlier to report the correct operation based on the event type. For some events this is NA.
v2: Supports load/stores/na again instead of marking everything NA
Signed-off-by Andi Kleen <[email protected]>--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)arch/x86/kernel/cpu/perf_event_intel_ds.c +0
-0@@ -108,13 +108,19 @@ static u64 precise_store_data(u64 status)return val;
arch/x86/kernel/cpu/perf_event_intel_ds.c static u64 precise_store_data(u64 status) · line 117
Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +2
-1} -static u64 precise_store_data_hsw (struct perf_event *event,u64 status) +static u64 precise_store_data_hsw (struct perf_event *event,u64 status, +unsigned flags) {union perf_mem_data_src dse;u64 cfg = event->hw.config &INTEL_ARCH_EVENT_MASK ; dse.val =0 ;
No. This is not valid. You are not initializing all the field later on. Needs to be:
dse.val = PERF_MEM_NA
(which is defined in my patch).
arch/x86/kernel/cpu/perf_event_intel_ds.c static u64 precise_store_data(u64 status) · line 125
Quoted 9 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +6
-1- dse.mem_op =PERF_MEM_OP_STORE ; +if (flags &PERF_X86_EVENT_PEBS_LD_HSW ) + dse.mem_op =PERF_MEM_OP_LOAD ; +else if (flags &PERF_X86_EVENT_PEBS_ST_HSW ) + dse.mem_op =PERF_MEM_OP_STORE ; +else + dse.mem_op =PERF_MEM_OP_NA ; dse.mem_lvl =PERF_MEM_LVL_NA ;
But you have LOCK, SNOOP TLB, ....
So it is better to initialize globally dse.val once, and then patch based on what you can gather.
I will repost the whole series with my fixes and cleanups.
Quoted 14 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +2
-2/* @@ -868,7 +874,8 @@ static void __intel_pmu_pebs_event(struct perf_event *event,PERF_X86_EVENT_PEBS_LD_HSW | PERF_X86_EVENT_PEBS_NA_HSW)) data.data_src.val = -precise_store_data_hsw (event, pebs->dse); +precise_store_data_hsw (event, pebs->dse, + event->hw.flags); else data.data_src.val = precise_store_data(pebs->dse); } --1.9.3
PATCH 3/4 perf, x86: Fix haswell mem hierarchy flags reporting
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
This fixes a bug introduced with
commit 722e76e60f2775c21b087ff12c5e678cf0ebcaaf
Author: Stephane Eranian <[email protected]> Date: Thu May 15 17:56:44 2014 +0200
fix Haswell precise store data source encoding
When returning early we need to return the complete value of the memory hierarchy, not just the mem_lvl. Otherwise any load/store/na flags set early get lost.
--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_ds.c +4 -3
@@ -121,7 +121,6 @@ static u64 precise_store_data_hsw(struct perf_event *event, u64 status, dse.mem_op =PERF_MEM_OP_STORE ; else dse.mem_op = PERF_MEM_OP_NA; - dse.mem_lvl =PERF_MEM_LVL_NA ;/* * L1 info only valid for following events:
@@ -131,8 +130,10 @@ static u64 precise_store_data_hsw(struct perf_event *event, u64 status,* MEM_UOPS_RETIRED.SPLIT_STORES * MEM_UOPS_RETIRED.ALL_STORES */ -if (cfg !=0x12d0 && cfg !=0x22d0 && cfg !=0x42d0 && cfg !=0x82d0 ) -return dse.mem_lvl; +if (cfg !=0x12d0 && cfg !=0x22d0 && cfg !=0x42d0 && cfg !=0x82d0 ) { + dse.mem_lvl =PERF_MEM_LVL_NA ; +return dse.val; + }if (status &1 ) dse.mem_lvl =PERF_MEM_LVL_L1 |PERF_MEM_LVL_HIT ;
Signature · 1 line
1.9.3
PATCH 4/4 perf: Fix the default data source
In reply to Andi KleenFrom: Stephane Eranian <[email protected]>
- the default of 0 for the data source in perf_sample_data_init() was wrong. 0 is not a valid value. So defined PERF_MEM_NA (not available)
--- include/linux/perf_event.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
include/linux/perf_event.h +8 -1
@@ -604,6 +604,13 @@ struct perf_sample_data {u64 txn; }; +/* default value for data source */ +#define PERF_MEM_NA (PERF_MEM_S(OP, NA) |\ + PERF_MEM_S(LVL, NA) |\ + PERF_MEM_S(SNOOP, NA) |\ + PERF_MEM_S(LOCK, NA) |\ + PERF_MEM_S(TLB, NA)) +static inline void perf_sample_data_init (struct perf_sample_data *data,u64 addr,u64 period) {
@@ -616,7 +623,7 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, data->regs_user.regs =NULL ; data->stack_user_size =0 ; data->weight =0 ; - data->data_src.val =0 ; + data->data_src.val =PERF_MEM_NA ; data->txn =0 ; }
Signature · 1 line
1.9.3