perf, amd: Enable AMD family 15h northbridge counters
Subsystems (2)
v4 cover perf, amd: Enable AMD family 15h northbridge counters
The following patchset enables 4 additional performance counters in AMD family 15h processors that count northbridge events -- such as number of DRAM accesses.
This patchset is based on previous work done by Robert Richter <[email protected]> :
https://lkml.org/lkml/2012/6/19/324
The main differences are:
* The northbridge counters are indexed contiguously right above the core performance counters.
* MSR address offset calculations are moved to architecture specific files.
* Interrups are set up to be delivered only to a single core.
V4:
* Moved interrupt core select set up back to event constraints function, sicne during ->hw_config time we do not yet know on which CPU the the event will run on.
* Tested on and made minor revisions to make sure that the patchset is compatible with upcoming AMD Family 16h processors, and will support core and NB counters without any further patches.
V3:
Addressed the following feedback/comments from Robert's review
* https://lkml.org/lkml/2012/11/16/484
* https://lkml.org/lkml/2012/11/26/162
V2:
Separate out Robert's patches, and add properly ordered certificate of origins.
Jacob Shin (4):
perf, amd: Use proper naming scheme for AMD bit field definitions perf, x86: Move MSR address offset calculation to architecture
specific files
perf, x86: Allow for architecture specific RDPMC indexes perf, amd: Enable northbridge performance counters on AMD family 15h
Robert Richter (2):
perf, amd: Rework northbridge event constraints handler perf, amd: Generalize northbridge constraints code for family 15h
arch/x86/include/asm/cpufeature.h | 2 + arch/x86/include/asm/msr-index.h | 2 + arch/x86/include/asm/perf_event.h | 13 +- arch/x86/kernel/cpu/perf_event.c | 2 +- arch/x86/kernel/cpu/perf_event.h | 25 ++- arch/x86/kernel/cpu/perf_event_amd.c | 318 ++++++++++++++++++++++++++-------- 6 files changed, 268 insertions(+), 94 deletions(-)
Signature · 1 line
1.7.9.5
PATCH v4 1/6 perf, amd: Rework northbridge event constraints handler
From: Robert Richter <[email protected]>
Code simplification. No functional changes.
--- arch/x86/kernel/cpu/perf_event_amd.c | 68 +++++++++++++--------------------- 1 file changed, 26 insertions(+), 42 deletions(-)
arch/x86/kernel/cpu/perf_event_amd.c +26 -42
@@ -256,9 +256,8 @@ amd_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event) {struct hw_perf_event *hwc = &event->hw;struct amd_nb *nb = cpuc->amd_nb; -struct perf_event *old =NULL ; -int max = x86_pmu.num_counters; -int i, j, k =-1 ; +struct perf_event *old; +int idx, new =-1 ;/* * if not NB event or no NB, then no constraints
@@ -276,48 +275,33 @@ amd_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)* because of successive calls to x86_schedule_events() from * hw_perf_group_sched_in() without hw_perf_enable() */ -for (i =0 ; i < max; i++) { -/* -* keep track of first free slot -*/ -if (k ==-1 && !nb->owners[i]) - k = i; +for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +if (new ==-1 || hwc->idx == idx) +/* assign free slot, prefer hwc->idx */ + old =cmpxchg (nb->owners + idx,NULL , event); +else if (nb->owners[idx] == event) +/* event already present */ + old = event; +else +continue ; + +if (old && old != event) +continue ; + +/* reassign to this slot */ +if (new !=-1 ) +cmpxchg (nb->owners + new, event,NULL ); + new = idx;/* already present, reuse */ -if (nb->owners[i] == event) - goto done; - } -/* -* not present, so grab a new slot -* starting either at: -*/ -if (hwc->idx !=-1 ) { -/* previous assignment */ - i = hwc->idx; - }else if (k !=-1 ) { -/* start from free slot found */ - i = k; - }else { -/* -* event not found, no slot found in -* first pass, try again from the -* beginning -*/ - i =0 ; - } - j = i; -do { - old =cmpxchg (nb->owners+i,NULL , event); -if (!old) +if (old == event)break ; -if (++i == max) - i =0 ; - }while (i != j); -done: -if (!old) -return &nb->event_constraints[i]; - -return &emptyconstraint; + } + +if (new ==-1 ) +return &emptyconstraint; + +return &nb->event_constraints[new]; }static struct amd_nb *amd_alloc_nb (int cpu)
Signature · 1 line
1.7.9.5
PATCH v4 2/6 perf, amd: Generalize northbridge constraints code for family 15h
In reply to Jacob ShinFrom: Robert Richter <[email protected]>
Generalize northbridge constraints code for family 10h so that later we can reuse the same code path with other AMD processor families that have the same northbridge event constraints.
--- arch/x86/kernel/cpu/perf_event_amd.c | 43 ++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 18 deletions(-)
arch/x86/kernel/cpu/perf_event_amd.c +25 -18
@@ -188,20 +188,13 @@ static inline int amd_has_nb(struct cpu_hw_events *cpuc)return nb && nb->nb_id !=-1 ; } -static void amd_put_event_constraints (struct cpu_hw_events *cpuc, -struct perf_event *event) +static void __amd_put_nb_event_constraints (struct cpu_hw_events *cpuc, +struct perf_event *event) { -struct hw_perf_event *hwc = &event->hw;struct amd_nb *nb = cpuc->amd_nb;int i;/* -* only care about NB events -*/ -if (!(amd_has_nb (cpuc) &&amd_is_nb_event (hwc))) -return ; - -/* * need to scan whole list because event may not have * been assigned during scheduling *
@@ -247,12 +240,13 @@ static void amd_put_event_constraints(struct cpu_hw_events *cpuc,* * Given that resources are allocated (cmpxchg), they must be * eventually freed for others to use. This is accomplished by -* calling amd_put_event_constraints(). +* calling __amd_put_nb_event_constraints() * * Non NB events are not impacted by this restriction. */ static struct event_constraint * -amd_get_event_constraints (struct cpu_hw_events *cpuc,struct perf_event *event) +__amd_get_nb_event_constraints (struct cpu_hw_events *cpuc,struct perf_event *event, +struct event_constraint *c) {struct hw_perf_event *hwc = &event->hw;struct amd_nb *nb = cpuc->amd_nb;
@@ -260,12 +254,6 @@ amd_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)int idx, new =-1 ;/* -* if not NB event or no NB, then no constraints -*/ -if (!(amd_has_nb (cpuc) &&amd_is_nb_event (hwc))) -return &unconstrained; - -/* * detect if already present, if so reuse * * cannot merge with actual allocation
@@ -275,7 +263,7 @@ amd_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)* because of successive calls to x86_schedule_events() from * hw_perf_group_sched_in() without hw_perf_enable() */ -for (idx =0 ; idx < x86_pmu.num_counters; idx++) { +for_each_set_bit (idx, c->idxmsk,X86_PMC_IDX_MAX ) {if (new ==-1 || hwc->idx == idx)/* assign free slot, prefer hwc->idx */ old =cmpxchg (nb->owners + idx,NULL , event);
@@ -391,6 +379,25 @@ static void amd_pmu_cpu_dead(int cpu) } } +static struct event_constraint * +amd_get_event_constraints (struct cpu_hw_events *cpuc,struct perf_event *event) +{ +/* +* if not NB event or no NB, then no constraints +*/ +if (!(amd_has_nb (cpuc) &&amd_is_nb_event (&event->hw))) +return &unconstrained; + +return __amd_get_nb_event_constraints (cpuc, event, &unconstrained); +} + +static void amd_put_event_constraints (struct cpu_hw_events *cpuc, +struct perf_event *event) +{ +if (amd_has_nb (cpuc) &&amd_is_nb_event (&event->hw)) +__amd_put_nb_event_constraints (cpuc, event); +} +PMU_FORMAT_ATTR (event,"config:0-7,32-35" );PMU_FORMAT_ATTR (umask,"config:8-15" );PMU_FORMAT_ATTR (edge,"config:18" );
Signature · 1 line
1.7.9.5
PATCH v4 3/6 perf, amd: Use proper naming scheme for AMD bit field definitions
In reply to Jacob ShinUpdate these AMD bit field names to be consistent with naming convention followed by the rest of the file.
--- arch/x86/include/asm/perf_event.h | 4 ++-- arch/x86/kernel/cpu/perf_event_amd.c | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/include/asm/perf_event.h +2 -2
@@ -29,8 +29,8 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL -#define AMD_PERFMON_EVENTSEL_GUESTONLY (1ULL << 40) -#define AMD_PERFMON_EVENTSEL_HOSTONLY (1ULL << 41) +#define AMD64_EVENTSEL_GUESTONLY (1ULL << 40) +#define AMD64_EVENTSEL_HOSTONLY (1ULL << 41)#define AMD64_EVENTSEL_EVENT \ (ARCH_PERFMON_EVENTSEL_EVENT | (0x0FULL << 32))
arch/x86/kernel/cpu/perf_event_amd.c +4 -4
@@ -156,9 +156,9 @@ static int amd_pmu_hw_config(struct perf_event *event) event->hw.config &= ~(ARCH_PERFMON_EVENTSEL_USR |ARCH_PERFMON_EVENTSEL_OS ); else if (event->attr.exclude_host) - event->hw.config |= AMD_PERFMON_EVENTSEL_GUESTONLY; + event->hw.config |= AMD64_EVENTSEL_GUESTONLY; else if (event->attr.exclude_guest) - event->hw.config |= AMD_PERFMON_EVENTSEL_HOSTONLY; + event->hw.config |= AMD64_EVENTSEL_HOSTONLY;if (event->attr.type !=PERF_TYPE_RAW )return 0 ;
@@ -336,7 +336,7 @@ static void amd_pmu_cpu_starting(int cpu)struct amd_nb *nb;int i, nb_id; - cpuc->perf_ctr_virt_mask =AMD_PERFMON_EVENTSEL_HOSTONLY ; + cpuc->perf_ctr_virt_mask =AMD64_EVENTSEL_HOSTONLY ;if (boot_cpu_data.x86_max_cores <2 )return ;
@@ -669,7 +669,7 @@ void amd_pmu_disable_virt(void)* SVM is disabled the Guest-only bits still gets set and the counter * will not count anything. */ - cpuc->perf_ctr_virt_mask =AMD_PERFMON_EVENTSEL_HOSTONLY ; + cpuc->perf_ctr_virt_mask =AMD64_EVENTSEL_HOSTONLY ;/* Reload all events */ x86_pmu_disable_all ();
Signature · 1 line
1.7.9.5
PATCH v4 4/6 perf, x86: Move MSR address offset calculation to architecture specific files
In reply to Jacob ShinMove counter index to MSR address offset calculation to architecture specific files. This prepares the way for perf_event_amd to enable counter addresses that are not contiguous -- for example AMD Family 15h processors have 6 core performance counters starting at 0xc0010200 and 4 northbridge performance counters starting at 0xc0010240.
--- arch/x86/kernel/cpu/perf_event.h | 21 ++++------------- arch/x86/kernel/cpu/perf_event_amd.c | 42 ++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 16 deletions(-)
arch/x86/kernel/cpu/perf_event.h +5 -16
@@ -325,6 +325,7 @@ struct x86_pmu {int (*schedule_events)(struct cpu_hw_events *cpuc,int n,int *assign);unsigned eventsel;unsigned perfctr; +int (*addr_offset)(int index,int eventsel);u64 (*event_map)(int);int max_events;int num_counters;
@@ -446,28 +447,16 @@ extern u64 __read_mostly hw_cache_extra_regsu64 x86_perf_event_update (struct perf_event *event); -static inline int x86_pmu_addr_offset (int index) -{ -int offset; - -/* offset = X86_FEATURE_PERFCTR_CORE ? index << 1 : index */ -alternative_io (ASM_NOP2 , -"shll $1, %%eax" , -X86_FEATURE_PERFCTR_CORE , -"=a" (offset), -"a" (index)); - -return offset; -} -static inline unsigned int x86_pmu_config_addr (int index) { -return x86_pmu.eventsel +x86_pmu_addr_offset (index); +return x86_pmu.eventsel + + (x86_pmu.addr_offset ? x86_pmu.addr_offset (index,1 ) : index); }static inline unsigned int x86_pmu_event_addr (int index) { -return x86_pmu.perfctr +x86_pmu_addr_offset (index); +return x86_pmu.perfctr + + (x86_pmu.addr_offset ? x86_pmu.addr_offset (index,0 ) : index); }int x86_setup_perfctr (struct perf_event *event);
arch/x86/kernel/cpu/perf_event_amd.c +42 -0
54 lines not on this page · raw message
Signature · 1 line
1.7.9.5
PATCH v4 5/6 perf, x86: Allow for architecture specific RDPMC indexes
In reply to Jacob ShinSimilar to config_base and event_base, allow architecture specific RDPMC ECX values.
--- arch/x86/kernel/cpu/perf_event.c | 2 +- arch/x86/kernel/cpu/perf_event.h | 6 ++++++ arch/x86/kernel/cpu/perf_event_amd.c | 6 ++++++ 3 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/perf_event.c +1 -1
@@ -835,7 +835,7 @@ static inline void x86_assign_hw_event(struct perf_event *event, }else { hwc->config_base =x86_pmu_config_addr (hwc->idx); hwc->event_base =x86_pmu_event_addr (hwc->idx); - hwc->event_base_rdpmc = hwc->idx; + hwc->event_base_rdpmc =x86_pmu_rdpmc_index (hwc->idx); } }
arch/x86/kernel/cpu/perf_event.h +6 -0
@@ -326,6 +326,7 @@ struct x86_pmu {unsigned eventsel;unsigned perfctr;int (*addr_offset)(int index,int eventsel); +int (*rdpmc_index)(int index);u64 (*event_map)(int);int max_events;int num_counters;
@@ -459,6 +460,11 @@ static inline unsigned int x86_pmu_event_addr(int index) (x86_pmu.addr_offset ? x86_pmu.addr_offset (index,0 ) : index); } +static inline int x86_pmu_rdpmc_index (int index) +{ +return x86_pmu.rdpmc_index ? x86_pmu.rdpmc_index (index) : index; +} +int x86_setup_perfctr (struct perf_event *event);int x86_pmu_hw_config (struct perf_event *event);
arch/x86/kernel/cpu/perf_event_amd.c +6 -0
@@ -173,6 +173,11 @@ static inline int amd_pmu_addr_offset(int index, int eventsel)return offset; } +static inline int amd_pmu_rdpmc_index (int index) +{ +return index; +} +static int amd_pmu_hw_config (struct perf_event *event) {int ret;
@@ -620,6 +625,7 @@ static __initconst const struct x86_pmu amd_pmu = { .eventsel = MSR_K7_EVNTSEL0, .perfctr = MSR_K7_PERFCTR0, .addr_offset = amd_pmu_addr_offset, + .rdpmc_index = amd_pmu_rdpmc_index, .event_map = amd_pmu_event_map, .max_events = ARRAY_SIZE(amd_perfmon_event_map), .num_counters = AMD64_NUM_COUNTERS,
Signature · 1 line
1.7.9.5
PATCH v4 6/6 perf, amd: Enable northbridge performance counters on AMD family 15h
In reply to Jacob ShinOn AMD family 15h processors, there are 4 new performance counters (in addition to 6 core performance counters) that can be used for counting northbridge events (i.e. DRAM accesses). Their bit fields are almost identical to the core performance counters. However, unlike the core performance counters, these MSRs are shared between multiple cores (that share the same northbridge). We will reuse the same code path as existing family 10h northbridge event constraints handler logic to enforce this sharing.
--- arch/x86/include/asm/cpufeature.h | 2 + arch/x86/include/asm/msr-index.h | 2 + arch/x86/include/asm/perf_event.h | 9 ++ arch/x86/kernel/cpu/perf_event_amd.c | 167 ++++++++++++++++++++++++++++++---- 4 files changed, 160 insertions(+), 20 deletions(-)
arch/x86/include/asm/cpufeature.h +2 -0
@@ -167,6 +167,7 @@#define X86_FEATURE_TBM (6*32+21)/* trailing bit manipulations */ #define X86_FEATURE_TOPOEXT (6*32+22)/* topology extensions CPUID leafs */ #define X86_FEATURE_PERFCTR_CORE (6*32+23)/* core performance counter extensions */ +#define X86_FEATURE_PERFCTR_NB (6*32+24)/* NB performance counter extensions */ /* * Auxiliary flags: Linux defined - For features scattered in various
@@ -308,6 +309,7 @@ extern const char * const x86_power_flags[32];#define cpu_has_hypervisor boot_cpu_has(X86_FEATURE_HYPERVISOR)#define cpu_has_pclmulqdq boot_cpu_has(X86_FEATURE_PCLMULQDQ)#define cpu_has_perfctr_core boot_cpu_has(X86_FEATURE_PERFCTR_CORE) +#define cpu_has_perfctr_nb boot_cpu_has(X86_FEATURE_PERFCTR_NB)#define cpu_has_cx8 boot_cpu_has(X86_FEATURE_CX8)#define cpu_has_cx16 boot_cpu_has(X86_FEATURE_CX16)#define cpu_has_eager_fpu boot_cpu_has(X86_FEATURE_EAGER_FPU)
arch/x86/include/asm/msr-index.h +2 -0
@@ -157,6 +157,8 @@/* Fam 15h MSRs */ #define MSR_F15H_PERF_CTL 0xc0010200#define MSR_F15H_PERF_CTR 0xc0010201 +#define MSR_F15H_NB_PERF_CTL 0xc0010240 +#define MSR_F15H_NB_PERF_CTR 0xc0010241/* Fam 10h MSRs */ #define MSR_FAM10H_MMIO_CONF_BASE 0xc0010058
arch/x86/include/asm/perf_event.h +9 -0
@@ -29,9 +29,14 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL +#define AMD64_EVENTSEL_INT_CORE_ENABLE (1ULL << 36)#define AMD64_EVENTSEL_GUESTONLY (1ULL << 40)#define AMD64_EVENTSEL_HOSTONLY (1ULL << 41) +#define AMD64_EVENTSEL_INT_CORE_SEL_SHIFT 37 +#define AMD64_EVENTSEL_INT_CORE_SEL_MASK \ + (0xFULL << AMD64_EVENTSEL_INT_CORE_SEL_SHIFT) +#define AMD64_EVENTSEL_EVENT \ (ARCH_PERFMON_EVENTSEL_EVENT | (0x0FULL << 32))#define INTEL_ARCH_EVENT_MASK \
@@ -46,8 +51,12 @@#define AMD64_RAW_EVENT_MASK \ (X86_RAW_EVENT_MASK | \ AMD64_EVENTSEL_EVENT) +#define AMD64_RAW_EVENT_MASK_NB \ + (AMD64_EVENTSEL_EVENT | \ + ARCH_PERFMON_EVENTSEL_UMASK)#define AMD64_NUM_COUNTERS 4#define AMD64_NUM_COUNTERS_CORE 6 +#define AMD64_NUM_COUNTERS_NB 4#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_SEL 0x3c#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_UMASK (0x00 << 8)
arch/x86/kernel/cpu/perf_event_amd.c +147 -20
279 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Quoted 47 lines · Jacob Shin
The following patchset enables 4 additional performance counters in AMD family 15h processors that count northbridge events -- such as number of DRAM accesses.
This patchset is based on previous work done by Robert Richter <[email protected]> :
https://lkml.org/lkml/2012/6/19/324
The main differences are:
* The northbridge counters are indexed contiguously right above the core performance counters.
* MSR address offset calculations are moved to architecture specific files.
* Interrups are set up to be delivered only to a single core.
V4:
* Moved interrupt core select set up back to event constraints function, sicne during ->hw_config time we do not yet know on which CPU the the event will run on.
* Tested on and made minor revisions to make sure that the patchset is compatible with upcoming AMD Family 16h processors, and will support core and NB counters without any further patches.V3:
Addressed the following feedback/comments from Robert's review
* https://lkml.org/lkml/2012/11/16/484
* https://lkml.org/lkml/2012/11/26/162V2:
Separate out Robert's patches, and add properly ordered certificate of origins.Jacob Shin (4):
perf, amd: Use proper naming scheme for AMD bit field definitions perf, x86: Move MSR address offset calculation to architecture
specific files
perf, x86: Allow for architecture specific RDPMC indexes perf, amd: Enable northbridge performance counters on AMD family 15hRobert Richter (2):
perf, amd: Rework northbridge event constraints handler perf, amd: Generalize northbridge constraints code for family 15h
Quoted 9 lines
arch/x86/include/asm/cpufeature.h | 2 + arch/x86/include/asm/msr-index.h | 2 + arch/x86/include/asm/perf_event.h | 13 +- arch/x86/kernel/cpu/perf_event.c | 2 +- arch/x86/kernel/cpu/perf_event.h | 25 ++- arch/x86/kernel/cpu/perf_event_amd.c | 318 ++++++++++++++++++++++++++-------- 6 files changed, 268 insertions(+), 94 deletions(-)
Ping .. ? any comments/feedback ? If things look okay, could you please commit to tip perf/core ?
Thank you,
-Jacob
Quoted 56 lines · Jacob Shin
Quoted 55 lines · Jacob Shin
The following patchset enables 4 additional performance counters in AMD family 15h processors that count northbridge events -- such as number of DRAM accesses.
This patchset is based on previous work done by Robert Richter <[email protected]> :
https://lkml.org/lkml/2012/6/19/324
The main differences are:
* The northbridge counters are indexed contiguously right above the core performance counters.
* MSR address offset calculations are moved to architecture specific files.
* Interrups are set up to be delivered only to a single core.
V4:
* Moved interrupt core select set up back to event constraints function, sicne during ->hw_config time we do not yet know on which CPU the the event will run on.
* Tested on and made minor revisions to make sure that the patchset is compatible with upcoming AMD Family 16h processors, and will support core and NB counters without any further patches.V3:
Addressed the following feedback/comments from Robert's review
* https://lkml.org/lkml/2012/11/16/484
* https://lkml.org/lkml/2012/11/26/162V2:
Separate out Robert's patches, and add properly ordered certificate of origins.Jacob Shin (4):
perf, amd: Use proper naming scheme for AMD bit field definitions perf, x86: Move MSR address offset calculation to architecture
specific files
perf, x86: Allow for architecture specific RDPMC indexes perf, amd: Enable northbridge performance counters on AMD family 15hRobert Richter (2):
perf, amd: Rework northbridge event constraints handler perf, amd: Generalize northbridge constraints code for family 15harch/x86/include/asm/cpufeature.h | 2 + arch/x86/include/asm/msr-index.h | 2 + arch/x86/include/asm/perf_event.h | 13 +- arch/x86/kernel/cpu/perf_event.c | 2 +- arch/x86/kernel/cpu/perf_event.h | 25 ++- arch/x86/kernel/cpu/perf_event_amd.c | 318 ++++++++++++++++++++++++++-------- 6 files changed, 268 insertions(+), 94 deletions(-)
Quoted 4 lines
Quoted 1 line
Ping .. ? any comments/feedback ? If things look okay, could you please commit to tip perf/core ?
Ping #2 .. ?
Thanks!
-Jacob