perf: Add Icelake support (kernel only, except Topdown)
Subsystems (3)
v5 cover perf: Add Icelake support (kernel only, except Topdown)
From: Kan Liang <[email protected]>
To make the patch series easier to be reviewed and integrated, it is condensed into a smaller set of patches, which only include essential features of Icelake for kernel except new hardware Metrics counters for Topdown events. The user space patch and Topdown support will be submitted separately once the patch series is integrated. (Please let me know if the Topdown/user space patches are still expected to be included in this series)
The patch series intends to add Icelake support (except Topdown) for Linux perf.
PATCH 1: Patch to fix wrong PEBS_REGS for large PEBS PATCH 2-12: Kernel patches to support Icelake.
- 2-6: Support adaptive PEBS feature
- 7-8: Enable core support with some new features, e.g. 8 generic counters, new event constraints, a new fixed counter.
- 9-12: Enable cstate, rapl, msr and uncore support on Icelake
Changes since V4:
- Separate the changes. Topdown support and user space patches will be submitted separately.
- A patch to fix wrong PEBS_REGS for large PEBS
- Rename has_xmm_regs to pebs_no_xmm_regs
- Rename PEBS_REGS to PEBS_GPRS_REGS
Add PEBS_XMM_REGS for XMM registers
Refine the extra check in x86_pmu_hw_config()
Changes since V3:
- Keep the old names for GPRs. Rename PERF_REG_X86_MAX to PERF_REG_X86_XMM_MAX
- Remove unnecessary REG_RESERVED
- Add REG_NOSUPPORT for 32bit
Changes since V2:
- Make the setup_pebs_sample_data() a function pointer argument
- Use cpuc->pebs_record_size unconditionally
- Add comments for EVENT_CONSTRAINT_RANGE
- Correct the Author of "perf/x86: Support constraint ranges"
Changes since V1:
- Avoid the interface changes for perf_reg_value() and perf_output_sample_regs().
- Remove the extra_regs in struct perf_sample_data.
- Add struct x86_perf_regs
- Add has_xmm_regs to indicate the specific platform which support XMM registers collection.
- Add check in x86_pmu_hw_config() to reject invalid config of regs_user and regs_intr.
- Rename intel_hsw_weight and intel_hsw_transaction
- Add missed inline for intel_get_tsx_transaction()
- Add new patch to extract code of event update in short period
- Code rebase on top of c634dc6bdede
- Rename @d to pebs_data_cfg
- Make pebs_update_adaptive_cfg readable
- Clear pebs_data_cfg and pebs_record_size for first PEBS in add
- Don't clear ICL_EVENTSEL_ADAPTIVE. Rely on MSR_PEBS_CFG settings
- Change PEBS record parsing order (bug fix)
- Support struct x86_perf_regs
- make get_pebs_status generic
- specific intel_pmu_drain_pebs_icl()
- Use cpuc->pebs_record_size to replace format_size
- Use 'size' to replace 'range_end' for constraint ranges
- Add x86_pmu.has_xmm_regs = true;
- Add more explanation in change log of REMOVE transaction
- Make perf_regs.h consistent between kernel and user space
Andi Kleen (2):
perf/x86/intel: Extract memory code PEBS parser for reuse perf/x86/lbr: Avoid reading the LBRs when adaptive PEBS handles them
Kan Liang (9):
perf/x86: Fix wrong PEBS_REGS
perf/x86: Support outputting XMM registers perf/x86/intel/ds: Extract code of event update in short period perf/x86/intel: Support adaptive PEBSv4 perf/x86/intel: Add Icelake support perf/x86/intel/cstate: Add Icelake support perf/x86/intel/rapl: Add Icelake support perf/x86/msr: Add Icelake support perf/x86/intel/uncore: Add Intel Icelake uncore support
Peter Zijlstra (1):
perf/x86: Support constraint ranges
arch/x86/events/core.c | 15 + arch/x86/events/intel/core.c | 117 +++++- arch/x86/events/intel/cstate.c | 2 + arch/x86/events/intel/ds.c | 501 ++++++++++++++++++++++---- arch/x86/events/intel/lbr.c | 35 +- arch/x86/events/intel/rapl.c | 2 + arch/x86/events/intel/uncore.c | 6 + arch/x86/events/intel/uncore.h | 1 + arch/x86/events/intel/uncore_snb.c | 91 +++++ arch/x86/events/msr.c | 1 + arch/x86/events/perf_event.h | 113 ++++-- arch/x86/include/asm/intel_ds.h | 2 +- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/perf_event.h | 49 ++- arch/x86/include/uapi/asm/perf_regs.h | 23 +- arch/x86/kernel/perf_regs.c | 27 +- 16 files changed, 882 insertions(+), 104 deletions(-)
Signature · 1 line
2.17.1
PATCH v5 1/12 perf/x86: Fix wrong PEBS_REGS
From: Kan Liang <[email protected]>
PEBS_REGS used as mask for the supported registers for large PEBS. However, the mask cannot filter the sample_regs_user/sample_regs_intr correctly.
(1ULL << PERF_REG_X86_*) should be used to replace PERF_REG_X86_*, which is only the index.
Rename PEBS_REGS to PEBS_GPRS_REGS. Because the mask is only for general purpose registers.
Fixes 2fe1bc1f501d ("perf/x86: Enable free running PEBS for REGS_USER/INTR")
Signed-off-by Kan Liang <[email protected]>
---
New patch for V5
arch/x86/events/intel/core.c | 2 +- arch/x86/events/perf_event.h | 38 ++++++++++++++++++------------------ 2 files changed, 20 insertions(+), 20 deletions(-)
arch/x86/events/intel/core.c +1 -1
@@ -3131,7 +3131,7 @@ static unsigned long intel_pmu_large_pebs_flags(struct perf_event *event) flags &= ~PERF_SAMPLE_TIME ;if (!event->attr.exclude_kernel) flags &= ~PERF_SAMPLE_REGS_USER ; -if (event->attr.sample_regs_user & ~PEBS_REGS ) +if (event->attr.sample_regs_user & ~PEBS_GPRS_REGS ) flags &= ~(PERF_SAMPLE_REGS_USER |PERF_SAMPLE_REGS_INTR );return flags; }
arch/x86/events/perf_event.h +19 -19
@@ -96,25 +96,25 @@ struct amd_nb { PERF_SAMPLE_REGS_INTR | PERF_SAMPLE_REGS_USER | \ PERF_SAMPLE_PERIOD) -#define PEBS_REGS \ - (PERF_REG_X86_AX | \ - PERF_REG_X86_BX | \ - PERF_REG_X86_CX | \ - PERF_REG_X86_DX | \ - PERF_REG_X86_DI | \ - PERF_REG_X86_SI | \ - PERF_REG_X86_SP | \ - PERF_REG_X86_BP | \ - PERF_REG_X86_IP | \ - PERF_REG_X86_FLAGS | \ - PERF_REG_X86_R8 | \ - PERF_REG_X86_R9 | \ - PERF_REG_X86_R10 | \ - PERF_REG_X86_R11 | \ - PERF_REG_X86_R12 | \ - PERF_REG_X86_R13 | \ - PERF_REG_X86_R14 | \ - PERF_REG_X86_R15) +#define PEBS_GPRS_REGS \ + ((1ULL << PERF_REG_X86_AX) | \ + (1ULL << PERF_REG_X86_BX) | \ + (1ULL << PERF_REG_X86_CX) | \ + (1ULL << PERF_REG_X86_DX) | \ + (1ULL << PERF_REG_X86_DI) | \ + (1ULL << PERF_REG_X86_SI) | \ + (1ULL << PERF_REG_X86_SP) | \ + (1ULL << PERF_REG_X86_BP) | \ + (1ULL << PERF_REG_X86_IP) | \ + (1ULL << PERF_REG_X86_FLAGS) | \ + (1ULL << PERF_REG_X86_R8) | \ + (1ULL << PERF_REG_X86_R9) | \ + (1ULL << PERF_REG_X86_R10) | \ + (1ULL << PERF_REG_X86_R11) | \ + (1ULL << PERF_REG_X86_R12) | \ + (1ULL << PERF_REG_X86_R13) | \ + (1ULL << PERF_REG_X86_R14) | \ + (1ULL << PERF_REG_X86_R15))/* * Per register state.
Signature · 1 line
2.17.1
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86: Fix incorrect PEBS_REGS
perf/x86: Fix incorrect PEBS_REGS
Commit-ID: 9d5dcc93a6ddfc78124f006ccd3637ce070ef2fc Gitweb: https://git.kernel.org/tip/9d5dcc93a6ddfc78124f006ccd3637ce070ef2fc Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:44:58 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:13:58 +0200
perf/x86: Fix incorrect PEBS_REGS
PEBS_REGS used as mask for the supported registers for large PEBS. However, the mask cannot filter the sample_regs_user/sample_regs_intr correctly.
(1ULL << PERF_REG_X86_*) should be used to replace PERF_REG_X86_*, which is only the index.
Rename PEBS_REGS to PEBS_GP_REGS, because the mask is only for general purpose registers.
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Fixes 2fe1bc1f501d ("perf/x86: Enable free running PEBS for REGS_USER/INTR")
[ Renamed it to PEBS_GP_REGS - as 'GPRS' is used elsewhere ;-) ]
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/core.c | 2 +- arch/x86/events/perf_event.h | 38 +++++++++++++++++++------------------- 2 files changed, 20 insertions(+), 20 deletions(-)
arch/x86/events/intel/core.c +1 -1
@@ -3131,7 +3131,7 @@ static unsigned long intel_pmu_large_pebs_flags(struct perf_event *event) flags &= ~PERF_SAMPLE_TIME ;if (!event->attr.exclude_kernel) flags &= ~PERF_SAMPLE_REGS_USER ; -if (event->attr.sample_regs_user & ~PEBS_REGS ) +if (event->attr.sample_regs_user & ~PEBS_GP_REGS ) flags &= ~(PERF_SAMPLE_REGS_USER |PERF_SAMPLE_REGS_INTR );return flags; }
arch/x86/events/perf_event.h +19 -19
@@ -96,25 +96,25 @@ struct amd_nb { PERF_SAMPLE_REGS_INTR | PERF_SAMPLE_REGS_USER | \ PERF_SAMPLE_PERIOD) -#define PEBS_REGS \ - (PERF_REG_X86_AX | \ - PERF_REG_X86_BX | \ - PERF_REG_X86_CX | \ - PERF_REG_X86_DX | \ - PERF_REG_X86_DI | \ - PERF_REG_X86_SI | \ - PERF_REG_X86_SP | \ - PERF_REG_X86_BP | \ - PERF_REG_X86_IP | \ - PERF_REG_X86_FLAGS | \ - PERF_REG_X86_R8 | \ - PERF_REG_X86_R9 | \ - PERF_REG_X86_R10 | \ - PERF_REG_X86_R11 | \ - PERF_REG_X86_R12 | \ - PERF_REG_X86_R13 | \ - PERF_REG_X86_R14 | \ - PERF_REG_X86_R15) +#define PEBS_GP_REGS \ + ((1ULL << PERF_REG_X86_AX) | \ + (1ULL << PERF_REG_X86_BX) | \ + (1ULL << PERF_REG_X86_CX) | \ + (1ULL << PERF_REG_X86_DX) | \ + (1ULL << PERF_REG_X86_DI) | \ + (1ULL << PERF_REG_X86_SI) | \ + (1ULL << PERF_REG_X86_SP) | \ + (1ULL << PERF_REG_X86_BP) | \ + (1ULL << PERF_REG_X86_IP) | \ + (1ULL << PERF_REG_X86_FLAGS) | \ + (1ULL << PERF_REG_X86_R8) | \ + (1ULL << PERF_REG_X86_R9) | \ + (1ULL << PERF_REG_X86_R10) | \ + (1ULL << PERF_REG_X86_R11) | \ + (1ULL << PERF_REG_X86_R12) | \ + (1ULL << PERF_REG_X86_R13) | \ + (1ULL << PERF_REG_X86_R14) | \ + (1ULL << PERF_REG_X86_R15))/* * Per register state.
PATCH v5 2/12 perf/x86: Support outputting XMM registers
In reply to [email protected]From: Kan Liang <[email protected]>
Starting from Icelake, XMM registers can be collected in PEBS record. But current code only output the pt_regs.
Add a new struct x86_perf_regs for both pt_regs and xmm_regs. The xmm_regs will be used later to keep a pointer to PEBS record which has XMM information.
XMM registers are 128 bit. To simplify the code, they are handled like two different registers, which means setting two bits in the register bitmap. This also allows only sampling the lower 64bit bits in XMM.
The index of XMM registers starts from 32. There are 16 XMM registers. So all reserved space for regs are used. Remove REG_RESERVED.
Add PERF_REG_X86_XMM_MAX, which stands for the max number of all x86 regs including both GPRs and XMM.
Add REG_NOSUPPORT for 32bit to exclude unsupported registers.
Previous platforms can not collect XMM information in PEBS record. Adding pebs_no_xmm_regs to indicate the unsupported platforms.
The common code still validate the supported registers. However, it cannot check model specific registers, e.g. XMM. Add extra check in x86_pmu_hw_config() to reject invalid config of regs_user and regs_intr. The regs_user never supports XMM collection.
The regs_intr only supports XMM collection when sampling PEBS event on icelake and later platforms.
Originally-by Andi Kleen <[email protected]>
Suggested-by Peter Zijlstra (Intel) <[email protected]>
Signed-off-by Kan Liang <[email protected]>
---
Changes since V4:
- Rename has_xmm_regs to pebs_no_xmm_regs
- Rename PEBS_REGS to PEBS_GPRS_REGS
Add PEBS_XMM_REGS for XMM registers
Refine the extra check in x86_pmu_hw_config()
arch/x86/events/core.c | 15 +++++++++++++++ arch/x86/events/intel/ds.c | 4 +++- arch/x86/events/perf_event.h | 21 ++++++++++++++++++++- arch/x86/include/asm/perf_event.h | 5 +++++ arch/x86/include/uapi/asm/perf_regs.h | 23 ++++++++++++++++++++++- arch/x86/kernel/perf_regs.c | 27 ++++++++++++++++++++------- 6 files changed, 85 insertions(+), 10 deletions(-)
arch/x86/events/core.c +15 -0
@@ -560,6 +560,21 @@ int x86_pmu_hw_config(struct perf_event *event)return -EINVAL ; } +/* sample_regs_user never support XMM registers */ +if (unlikely (event->attr.sample_regs_user &PEBS_XMM_REGS )) +return -EINVAL ; +/* +* Besides the general purpose registers, XMM registers may +* be collected in PEBS on some platforms, e.g. Icelake +*/ +if (unlikely (event->attr.sample_regs_intr &PEBS_XMM_REGS )) { +if (!is_sampling_event (event) || + !event->attr.precise_ip || + x86_pmu.pebs_no_xmm_regs) +return -EINVAL ; + + } +return x86_setup_perfctr (event); }
arch/x86/events/intel/ds.c +3 -1
@@ -1628,8 +1628,10 @@ void __init intel_ds_init(void) x86_pmu.bts =boot_cpu_has (X86_FEATURE_BTS ); x86_pmu.pebs =boot_cpu_has (X86_FEATURE_PEBS ); x86_pmu.pebs_buffer_size =PEBS_BUFFER_SIZE ; -if (x86_pmu.version <=4 ) +if (x86_pmu.version <=4 ) { x86_pmu.pebs_no_isolation =1 ; + x86_pmu.pebs_no_xmm_regs =1 ; + }if (x86_pmu.pebs) {char pebs_type = x86_pmu.intel_cap.pebs_trap ?'+' :'-' ;int format = x86_pmu.intel_cap.pebs_format;
arch/x86/events/perf_event.h +20 -1
@@ -116,6 +116,24 @@ struct amd_nb { (1ULL << PERF_REG_X86_R14) | \ (1ULL << PERF_REG_X86_R15)) +#define PEBS_XMM_REGS \ + ((1ULL << PERF_REG_X86_XMM0) | \ + (1ULL << PERF_REG_X86_XMM1) | \ + (1ULL << PERF_REG_X86_XMM2) | \ + (1ULL << PERF_REG_X86_XMM3) | \ + (1ULL << PERF_REG_X86_XMM4) | \ + (1ULL << PERF_REG_X86_XMM5) | \ + (1ULL << PERF_REG_X86_XMM6) | \ + (1ULL << PERF_REG_X86_XMM7) | \ + (1ULL << PERF_REG_X86_XMM8) | \ + (1ULL << PERF_REG_X86_XMM9) | \ + (1ULL << PERF_REG_X86_XMM10) | \ + (1ULL << PERF_REG_X86_XMM11) | \ + (1ULL << PERF_REG_X86_XMM12) | \ + (1ULL << PERF_REG_X86_XMM13) | \ + (1ULL << PERF_REG_X86_XMM14) | \ + (1ULL << PERF_REG_X86_XMM15)) +/* * Per register state. */
@@ -613,7 +631,8 @@ struct x86_pmu { pebs_broken :1, pebs_prec_dist :1, pebs_no_tlb :1, - pebs_no_isolation :1; + pebs_no_isolation :1, + pebs_no_xmm_regs :1;int pebs_record_size;int pebs_buffer_size;void (*drain_pebs)(struct pt_regs *regs);
arch/x86/include/asm/perf_event.h +5 -0
@@ -248,6 +248,11 @@ extern void perf_events_lapic_init(void);#define PERF_EFLAGS_VM (1UL << 5)struct pt_regs ; +struct x86_perf_regs { +struct pt_regs regs; +u64 *xmm_regs; +}; +extern unsigned long perf_instruction_pointer (struct pt_regs *regs);extern unsigned long perf_misc_flags (struct pt_regs *regs);#define perf_misc_flags (regs) perf_misc_flags(regs)
arch/x86/include/uapi/asm/perf_regs.h +22 -1
@@ -27,8 +27,29 @@ enum perf_event_x86_regs {PERF_REG_X86_R13 ,PERF_REG_X86_R14 ,PERF_REG_X86_R15 , - +/* These are the limits for the GPRs. */ PERF_REG_X86_32_MAX =PERF_REG_X86_GS +1 ,PERF_REG_X86_64_MAX =PERF_REG_X86_R15 +1 , + +/* These all need two bits set because they are 128bit */ +PERF_REG_X86_XMM0 =32 , +PERF_REG_X86_XMM1 =34 , +PERF_REG_X86_XMM2 =36 , +PERF_REG_X86_XMM3 =38 , +PERF_REG_X86_XMM4 =40 , +PERF_REG_X86_XMM5 =42 , +PERF_REG_X86_XMM6 =44 , +PERF_REG_X86_XMM7 =46 , +PERF_REG_X86_XMM8 =48 , +PERF_REG_X86_XMM9 =50 , +PERF_REG_X86_XMM10 =52 , +PERF_REG_X86_XMM11 =54 , +PERF_REG_X86_XMM12 =56 , +PERF_REG_X86_XMM13 =58 , +PERF_REG_X86_XMM14 =60 , +PERF_REG_X86_XMM15 =62 , + +/* These include both GPRs and XMMX registers */ + PERF_REG_X86_XMM_MAX = PERF_REG_X86_XMM15 + 2, };#endif /* _ASM_X86_PERF_REGS_H */
arch/x86/kernel/perf_regs.c +20 -7
@@ -59,18 +59,34 @@ static unsigned int pt_regs_offset[PERF_REG_X86_MAX] = {u64 perf_reg_value (struct pt_regs *regs,int idx) { +struct x86_perf_regs *perf_regs; + +if (idx >=PERF_REG_X86_XMM0 && idx <PERF_REG_X86_XMM_MAX ) { + perf_regs = container_of(regs, struct x86_perf_regs, regs); +if (!perf_regs->xmm_regs) +return 0 ; +return perf_regs->xmm_regs[idx -PERF_REG_X86_XMM0 ]; + } +if (WARN_ON_ONCE (idx >=ARRAY_SIZE (pt_regs_offset)))return 0 ;return regs_get_register (regs, pt_regs_offset[idx]); } -#define REG_RESERVED (~((1ULL << PERF_REG_X86_MAX) - 1ULL)) -#ifdef CONFIG_X86_32 +#define REG_NOSUPPORT ((1ULL << PERF_REG_X86_R8) | \ + (1ULL << PERF_REG_X86_R9) | \ + (1ULL << PERF_REG_X86_R10) | \ + (1ULL << PERF_REG_X86_R11) | \ + (1ULL << PERF_REG_X86_R12) | \ + (1ULL << PERF_REG_X86_R13) | \ + (1ULL << PERF_REG_X86_R14) | \ + (1ULL << PERF_REG_X86_R15)) +int perf_reg_validate (u64 mask) { -if (!mask || mask ®_RESERVED ) +if (!mask || (mask ®_NOSUPPORT ))return -EINVAL ;return 0 ;
@@ -96,10 +112,7 @@ void perf_get_regs_user(struct perf_regs *regs_user,int perf_reg_validate (u64 mask) { -if (!mask || mask ®_RESERVED ) -return -EINVAL ; - -if (mask ®_NOSUPPORT ) +if (!mask || (mask ®_NOSUPPORT ))return -EINVAL ;return 0 ;
Signature · 1 line
2.17.1
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86: Support outputting XMM registers
perf/x86: Support outputting XMM registers
Commit-ID: 878068ea270ea82767ff1d26c91583263c81fba0 Gitweb: https://git.kernel.org/tip/878068ea270ea82767ff1d26c91583263c81fba0 Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:44:59 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:19:36 +0200
perf/x86: Support outputting XMM registers
Starting from Icelake, XMM registers can be collected in PEBS record. But current code only output the pt_regs.
Add a new struct x86_perf_regs for both pt_regs and xmm_regs. The xmm_regs will be used later to keep a pointer to PEBS record which has XMM information.
XMM registers are 128 bit. To simplify the code, they are handled like two different registers, which means setting two bits in the register bitmap. This also allows only sampling the lower 64bit bits in XMM.
The index of XMM registers starts from 32. There are 16 XMM registers. So all reserved space for regs are used. Remove REG_RESERVED.
Add PERF_REG_X86_XMM_MAX, which stands for the max number of all x86 regs including both GPRs and XMM.
Add REG_NOSUPPORT for 32bit to exclude unsupported registers.
Previous platforms can not collect XMM information in PEBS record. Adding pebs_no_xmm_regs to indicate the unsupported platforms.
The common code still validates the supported registers. However, it cannot check model specific registers, e.g. XMM. Add extra check in x86_pmu_hw_config() to reject invalid config of regs_user and regs_intr. The regs_user never supports XMM collection.
The regs_intr only supports XMM collection when sampling PEBS event on icelake and later platforms.
Originally-by Andi Kleen <[email protected]>
Suggested-by Peter Zijlstra (Intel) <[email protected]>
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/core.c | 15 +++++++++++++++ arch/x86/events/intel/ds.c | 4 +++- arch/x86/events/perf_event.h | 21 ++++++++++++++++++++- arch/x86/include/asm/perf_event.h | 5 +++++ arch/x86/include/uapi/asm/perf_regs.h | 23 ++++++++++++++++++++++- arch/x86/kernel/perf_regs.c | 27 ++++++++++++++++++++------- 6 files changed, 85 insertions(+), 10 deletions(-)
arch/x86/events/core.c +15 -0
@@ -560,6 +560,21 @@ int x86_pmu_hw_config(struct perf_event *event)return -EINVAL ; } +/* sample_regs_user never support XMM registers */ +if (unlikely (event->attr.sample_regs_user &PEBS_XMM_REGS )) +return -EINVAL ; +/* +* Besides the general purpose registers, XMM registers may +* be collected in PEBS on some platforms, e.g. Icelake +*/ +if (unlikely (event->attr.sample_regs_intr &PEBS_XMM_REGS )) { +if (x86_pmu.pebs_no_xmm_regs) +return -EINVAL ; + +if (!event->attr.precise_ip) +return -EINVAL ; + } +return x86_setup_perfctr (event); }
arch/x86/events/intel/ds.c +3 -1
@@ -1628,8 +1628,10 @@ void __init intel_ds_init(void) x86_pmu.bts =boot_cpu_has (X86_FEATURE_BTS ); x86_pmu.pebs =boot_cpu_has (X86_FEATURE_PEBS ); x86_pmu.pebs_buffer_size =PEBS_BUFFER_SIZE ; -if (x86_pmu.version <=4 ) +if (x86_pmu.version <=4 ) { x86_pmu.pebs_no_isolation =1 ; + x86_pmu.pebs_no_xmm_regs =1 ; + }if (x86_pmu.pebs) {char pebs_type = x86_pmu.intel_cap.pebs_trap ?'+' :'-' ;int format = x86_pmu.intel_cap.pebs_format;
arch/x86/events/perf_event.h +20 -1
@@ -115,6 +115,24 @@ struct amd_nb { (1ULL << PERF_REG_X86_R14) | \ (1ULL << PERF_REG_X86_R15)) +#define PEBS_XMM_REGS \ + ((1ULL << PERF_REG_X86_XMM0) | \ + (1ULL << PERF_REG_X86_XMM1) | \ + (1ULL << PERF_REG_X86_XMM2) | \ + (1ULL << PERF_REG_X86_XMM3) | \ + (1ULL << PERF_REG_X86_XMM4) | \ + (1ULL << PERF_REG_X86_XMM5) | \ + (1ULL << PERF_REG_X86_XMM6) | \ + (1ULL << PERF_REG_X86_XMM7) | \ + (1ULL << PERF_REG_X86_XMM8) | \ + (1ULL << PERF_REG_X86_XMM9) | \ + (1ULL << PERF_REG_X86_XMM10) | \ + (1ULL << PERF_REG_X86_XMM11) | \ + (1ULL << PERF_REG_X86_XMM12) | \ + (1ULL << PERF_REG_X86_XMM13) | \ + (1ULL << PERF_REG_X86_XMM14) | \ + (1ULL << PERF_REG_X86_XMM15)) +/* * Per register state. */
@@ -612,7 +630,8 @@ struct x86_pmu { pebs_broken :1, pebs_prec_dist :1, pebs_no_tlb :1, - pebs_no_isolation :1; + pebs_no_isolation :1, + pebs_no_xmm_regs :1;int pebs_record_size;int pebs_buffer_size;void (*drain_pebs)(struct pt_regs *regs);
arch/x86/include/asm/perf_event.h +5 -0
@@ -248,6 +248,11 @@ extern void perf_events_lapic_init(void);#define PERF_EFLAGS_VM (1UL << 5)struct pt_regs ; +struct x86_perf_regs { +struct pt_regs regs; +u64 *xmm_regs; +}; +extern unsigned long perf_instruction_pointer (struct pt_regs *regs);extern unsigned long perf_misc_flags (struct pt_regs *regs);#define perf_misc_flags (regs) perf_misc_flags(regs)
arch/x86/include/uapi/asm/perf_regs.h +22 -1
@@ -27,8 +27,29 @@ enum perf_event_x86_regs {PERF_REG_X86_R13 ,PERF_REG_X86_R14 ,PERF_REG_X86_R15 , - +/* These are the limits for the GPRs. */ PERF_REG_X86_32_MAX =PERF_REG_X86_GS +1 ,PERF_REG_X86_64_MAX =PERF_REG_X86_R15 +1 , + +/* These all need two bits set because they are 128bit */ +PERF_REG_X86_XMM0 =32 , +PERF_REG_X86_XMM1 =34 , +PERF_REG_X86_XMM2 =36 , +PERF_REG_X86_XMM3 =38 , +PERF_REG_X86_XMM4 =40 , +PERF_REG_X86_XMM5 =42 , +PERF_REG_X86_XMM6 =44 , +PERF_REG_X86_XMM7 =46 , +PERF_REG_X86_XMM8 =48 , +PERF_REG_X86_XMM9 =50 , +PERF_REG_X86_XMM10 =52 , +PERF_REG_X86_XMM11 =54 , +PERF_REG_X86_XMM12 =56 , +PERF_REG_X86_XMM13 =58 , +PERF_REG_X86_XMM14 =60 , +PERF_REG_X86_XMM15 =62 , + +/* These include both GPRs and XMMX registers */ + PERF_REG_X86_XMM_MAX = PERF_REG_X86_XMM15 + 2, };#endif /* _ASM_X86_PERF_REGS_H */
arch/x86/kernel/perf_regs.c +20 -7
@@ -59,18 +59,34 @@ static unsigned int pt_regs_offset[PERF_REG_X86_MAX] = {u64 perf_reg_value (struct pt_regs *regs,int idx) { +struct x86_perf_regs *perf_regs; + +if (idx >=PERF_REG_X86_XMM0 && idx <PERF_REG_X86_XMM_MAX ) { + perf_regs = container_of(regs, struct x86_perf_regs, regs); +if (!perf_regs->xmm_regs) +return 0 ; +return perf_regs->xmm_regs[idx -PERF_REG_X86_XMM0 ]; + } +if (WARN_ON_ONCE (idx >=ARRAY_SIZE (pt_regs_offset)))return 0 ;return regs_get_register (regs, pt_regs_offset[idx]); } -#define REG_RESERVED (~((1ULL << PERF_REG_X86_MAX) - 1ULL)) -#ifdef CONFIG_X86_32 +#define REG_NOSUPPORT ((1ULL << PERF_REG_X86_R8) | \ + (1ULL << PERF_REG_X86_R9) | \ + (1ULL << PERF_REG_X86_R10) | \ + (1ULL << PERF_REG_X86_R11) | \ + (1ULL << PERF_REG_X86_R12) | \ + (1ULL << PERF_REG_X86_R13) | \ + (1ULL << PERF_REG_X86_R14) | \ + (1ULL << PERF_REG_X86_R15)) +int perf_reg_validate (u64 mask) { -if (!mask || mask ®_RESERVED ) +if (!mask || (mask ®_NOSUPPORT ))return -EINVAL ;return 0 ;
@@ -96,10 +112,7 @@ void perf_get_regs_user(struct perf_regs *regs_user,int perf_reg_validate (u64 mask) { -if (!mask || mask ®_RESERVED ) -return -EINVAL ; - -if (mask ®_NOSUPPORT ) +if (!mask || (mask ®_NOSUPPORT ))return -EINVAL ;return 0 ;
PATCH v5 3/12 perf/x86/intel: Extract memory code PEBS parser for reuse
In reply to [email protected]From: Andi Kleen <[email protected]>
Extract some code related to memory profiling from the PEBS record parser into separate functions. It can be reused by the upcoming adaptive PEBS parser. No functional changes.
Rename intel_hsw_weight to intel_get_tsx_weight, and intel_hsw_transaction to intel_get_tsx_transaction. Because the input is not the hsw pebs format anymore.
Signed-off-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/intel/ds.c | 63 ++++++++++++++++++++------------------ 1 file changed, 34 insertions(+), 29 deletions(-)
arch/x86/events/intel/ds.c +34 -29
@@ -1125,34 +1125,50 @@ static int intel_pmu_pebs_fixup_ip(struct pt_regs *regs)return 0 ; } -static inline u64 intel_hsw_weight (struct pebs_record_skl *pebs) +static inline u64 intel_get_tsx_weight (u64 tsx_tuning) { -if (pebs->tsx_tuning) { -union hsw_tsx_tuning tsx = { .value = pebs->tsx_tuning }; +if (tsx_tuning) { +union hsw_tsx_tuning tsx = { .value = tsx_tuning };return tsx.cycles_last_block; }return 0 ; } -static inline u64 intel_hsw_transaction (struct pebs_record_skl *pebs) +static inline u64 intel_get_tsx_transaction (u64 tsx_tuning,u64 ax) { -u64 txn = (pebs->tsx_tuning &PEBS_HSW_TSX_FLAGS ) >>32 ; +u64 txn = (tsx_tuning &PEBS_HSW_TSX_FLAGS ) >>32 ;/* For RTM XABORTs also log the abort code from AX */ -if ((txn &PERF_TXN_TRANSACTION ) && (pebs->ax &1 )) - txn |= ((pebs->ax >>24 ) &0xff ) <<PERF_TXN_ABORT_SHIFT ; +if ((txn &PERF_TXN_TRANSACTION ) && (ax &1 )) + txn |= ((ax >>24 ) &0xff ) <<PERF_TXN_ABORT_SHIFT ;return txn; } +#define PERF_X86_EVENT_PEBS_HSW_PREC \ + (PERF_X86_EVENT_PEBS_ST_HSW | \ + PERF_X86_EVENT_PEBS_LD_HSW | \ + PERF_X86_EVENT_PEBS_NA_HSW) + +static u64 get_data_src (struct perf_event *event,u64 aux) +{ +u64 val =PERF_MEM_NA ; +int fl = event->hw.flags; +bool fst = fl & (PERF_X86_EVENT_PEBS_ST |PERF_X86_EVENT_PEBS_HSW_PREC ); + +if (fl &PERF_X86_EVENT_PEBS_LDLAT ) + val =load_latency_data (aux); +else if (fst && (fl &PERF_X86_EVENT_PEBS_HSW_PREC )) + val =precise_datala_hsw (event, aux); +else if (fst) + val =precise_store_data (aux); +return val; +} +static void setup_pebs_sample_data (struct perf_event *event,struct pt_regs *iregs,void *__pebs,struct perf_sample_data *data,struct pt_regs *regs) { -#define PERF_X86_EVENT_PEBS_HSW_PREC \ - (PERF_X86_EVENT_PEBS_ST_HSW | \ - PERF_X86_EVENT_PEBS_LD_HSW | \ - PERF_X86_EVENT_PEBS_NA_HSW)/* * We cast to the biggest pebs_record but are careful not to * unconditionally access the 'extra' entries.
@@ -1160,17 +1176,13 @@ static void setup_pebs_sample_data(struct perf_event *event,struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct pebs_record_skl *pebs = __pebs;u64 sample_type; -int fll, fst, dsrc; -int fl = event->hw.flags; +int fll;if (pebs ==NULL )return ; sample_type = event->attr.sample_type; - dsrc = sample_type &PERF_SAMPLE_DATA_SRC ; - - fll = fl &PERF_X86_EVENT_PEBS_LDLAT ; - fst = fl & (PERF_X86_EVENT_PEBS_ST |PERF_X86_EVENT_PEBS_HSW_PREC ); + fll = event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ;perf_sample_data_init (data,0 , event->hw.last_period);
@@ -1185,16 +1197,8 @@ static void setup_pebs_sample_data(struct perf_event *event,/* * data.data_src encodes the data source */ -if (dsrc) { -u64 val =PERF_MEM_NA ; -if (fll) - val =load_latency_data (pebs->dse); -else if (fst && (fl &PERF_X86_EVENT_PEBS_HSW_PREC )) - val =precise_datala_hsw (event, pebs->dse); -else if (fst) - val =precise_store_data (pebs->dse); - data->data_src.val = val; - } +if (sample_type &PERF_SAMPLE_DATA_SRC ) + data->data_src.val =get_data_src (event, pebs->dse);/* * We must however always use iregs for the unwinder to stay sane; the
@@ -1281,10 +1285,11 @@ static void setup_pebs_sample_data(struct perf_event *event,if (x86_pmu.intel_cap.pebs_format >=2 ) {/* Only set the TSX weight when no memory weight. */ if ((sample_type &PERF_SAMPLE_WEIGHT ) && !fll) - data->weight =intel_hsw_weight (pebs); + data->weight =intel_get_tsx_weight (pebs->tsx_tuning);if (sample_type &PERF_SAMPLE_TRANSACTION ) - data->txn =intel_hsw_transaction (pebs); + data->txn =intel_get_tsx_transaction (pebs->tsx_tuning, + pebs->ax); }/*
Signature · 1 line
2.17.1
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/intel: Extract memory code PEBS parser for reuse
perf/x86/intel: Extract memory code PEBS parser for reuse
Commit-ID: 48f38aa4cc5a48bc0fe85c5c4b1ab171fbb539b6 Gitweb: https://git.kernel.org/tip/48f38aa4cc5a48bc0fe85c5c4b1ab171fbb539b6 Author: Andi Kleen <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:00 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:19:39 +0200
perf/x86/intel: Extract memory code PEBS parser for reuse
Extract some code related to memory profiling from the PEBS record parser into separate functions. It can be reused by the upcoming adaptive PEBS parser. No functional changes.
Rename intel_hsw_weight to intel_get_tsx_weight, and intel_hsw_transaction to intel_get_tsx_transaction. Because the input is not the hsw pebs format anymore.
Signed-off-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/ds.c | 63 +++++++++++++++++++++++++--------------------- 1 file changed, 34 insertions(+), 29 deletions(-)
arch/x86/events/intel/ds.c +34 -29
@@ -1125,34 +1125,50 @@ static int intel_pmu_pebs_fixup_ip(struct pt_regs *regs)return 0 ; } -static inline u64 intel_hsw_weight (struct pebs_record_skl *pebs) +static inline u64 intel_get_tsx_weight (u64 tsx_tuning) { -if (pebs->tsx_tuning) { -union hsw_tsx_tuning tsx = { .value = pebs->tsx_tuning }; +if (tsx_tuning) { +union hsw_tsx_tuning tsx = { .value = tsx_tuning };return tsx.cycles_last_block; }return 0 ; } -static inline u64 intel_hsw_transaction (struct pebs_record_skl *pebs) +static inline u64 intel_get_tsx_transaction (u64 tsx_tuning,u64 ax) { -u64 txn = (pebs->tsx_tuning &PEBS_HSW_TSX_FLAGS ) >>32 ; +u64 txn = (tsx_tuning &PEBS_HSW_TSX_FLAGS ) >>32 ;/* For RTM XABORTs also log the abort code from AX */ -if ((txn &PERF_TXN_TRANSACTION ) && (pebs->ax &1 )) - txn |= ((pebs->ax >>24 ) &0xff ) <<PERF_TXN_ABORT_SHIFT ; +if ((txn &PERF_TXN_TRANSACTION ) && (ax &1 )) + txn |= ((ax >>24 ) &0xff ) <<PERF_TXN_ABORT_SHIFT ;return txn; } +#define PERF_X86_EVENT_PEBS_HSW_PREC \ + (PERF_X86_EVENT_PEBS_ST_HSW | \ + PERF_X86_EVENT_PEBS_LD_HSW | \ + PERF_X86_EVENT_PEBS_NA_HSW) + +static u64 get_data_src (struct perf_event *event,u64 aux) +{ +u64 val =PERF_MEM_NA ; +int fl = event->hw.flags; +bool fst = fl & (PERF_X86_EVENT_PEBS_ST |PERF_X86_EVENT_PEBS_HSW_PREC ); + +if (fl &PERF_X86_EVENT_PEBS_LDLAT ) + val =load_latency_data (aux); +else if (fst && (fl &PERF_X86_EVENT_PEBS_HSW_PREC )) + val =precise_datala_hsw (event, aux); +else if (fst) + val =precise_store_data (aux); +return val; +} +static void setup_pebs_sample_data (struct perf_event *event,struct pt_regs *iregs,void *__pebs,struct perf_sample_data *data,struct pt_regs *regs) { -#define PERF_X86_EVENT_PEBS_HSW_PREC \ - (PERF_X86_EVENT_PEBS_ST_HSW | \ - PERF_X86_EVENT_PEBS_LD_HSW | \ - PERF_X86_EVENT_PEBS_NA_HSW)/* * We cast to the biggest pebs_record but are careful not to * unconditionally access the 'extra' entries.
@@ -1160,17 +1176,13 @@ static void setup_pebs_sample_data(struct perf_event *event,struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct pebs_record_skl *pebs = __pebs;u64 sample_type; -int fll, fst, dsrc; -int fl = event->hw.flags; +int fll;if (pebs ==NULL )return ; sample_type = event->attr.sample_type; - dsrc = sample_type &PERF_SAMPLE_DATA_SRC ; - - fll = fl &PERF_X86_EVENT_PEBS_LDLAT ; - fst = fl & (PERF_X86_EVENT_PEBS_ST |PERF_X86_EVENT_PEBS_HSW_PREC ); + fll = event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ;perf_sample_data_init (data,0 , event->hw.last_period);
@@ -1185,16 +1197,8 @@ static void setup_pebs_sample_data(struct perf_event *event,/* * data.data_src encodes the data source */ -if (dsrc) { -u64 val =PERF_MEM_NA ; -if (fll) - val =load_latency_data (pebs->dse); -else if (fst && (fl &PERF_X86_EVENT_PEBS_HSW_PREC )) - val =precise_datala_hsw (event, pebs->dse); -else if (fst) - val =precise_store_data (pebs->dse); - data->data_src.val = val; - } +if (sample_type &PERF_SAMPLE_DATA_SRC ) + data->data_src.val =get_data_src (event, pebs->dse);/* * We must however always use iregs for the unwinder to stay sane; the
@@ -1281,10 +1285,11 @@ static void setup_pebs_sample_data(struct perf_event *event,if (x86_pmu.intel_cap.pebs_format >=2 ) {/* Only set the TSX weight when no memory weight. */ if ((sample_type &PERF_SAMPLE_WEIGHT ) && !fll) - data->weight =intel_hsw_weight (pebs); + data->weight =intel_get_tsx_weight (pebs->tsx_tuning);if (sample_type &PERF_SAMPLE_TRANSACTION ) - data->txn =intel_hsw_transaction (pebs); + data->txn =intel_get_tsx_transaction (pebs->tsx_tuning, + pebs->ax); }/*
PATCH v5 4/12 perf/x86/intel/ds: Extract code of event update in short period
In reply to [email protected]From: Kan Liang <[email protected]>
The drain_pebs() could be called twice in a short period for auto-reload event in pmu::read(). The intel_pmu_save_and_restart_reload() should be called to update the event->count.
This case should also be handled on Icelake. Extract the codes for reuse later.
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/intel/ds.c | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-)
arch/x86/events/intel/ds.c +21 -13
@@ -1491,6 +1491,26 @@ static void intel_pmu_drain_pebs_core(struct pt_regs *iregs)__intel_pmu_pebs_event (event, iregs, at, top,0 , n); } +static void intel_pmu_pebs_event_update_no_drain (struct cpu_hw_events *cpuc, +int size) +{ +struct perf_event *event; +int bit; + +/* +* The drain_pebs() could be called twice in a short period +* for auto-reload event in pmu::read(). There are no +* overflows have happened in between. +* It needs to call intel_pmu_save_and_restart_reload() to +* update the event->count for this case. +*/ +for_each_set_bit (bit, (unsigned long *)&cpuc->pebs_enabled, size) { + event = cpuc->events[bit]; +if (event->hw.flags &PERF_X86_EVENT_AUTO_RELOAD ) +intel_pmu_save_and_restart_reload (event,0 ); + } +} +static void intel_pmu_drain_pebs_nhm (struct pt_regs *iregs) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);
@@ -1518,19 +1538,7 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs) }if (unlikely (base >= top)) { -/* -* The drain_pebs() could be called twice in a short period -* for auto-reload event in pmu::read(). There are no -* overflows have happened in between. -* It needs to call intel_pmu_save_and_restart_reload() to -* update the event->count for this case. -*/ -for_each_set_bit (bit, (unsigned long *)&cpuc->pebs_enabled, - size) { - event = cpuc->events[bit]; -if (event->hw.flags &PERF_X86_EVENT_AUTO_RELOAD ) -intel_pmu_save_and_restart_reload (event,0 ); - } +intel_pmu_pebs_event_update_no_drain (cpuc, size);return ; }
Signature · 1 line
2.17.1
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86/intel/ds: Extract code of event update in short period
perf/x86/intel/ds: Extract code of event update in short period
Commit-ID: 477f00f9617009a9a3a9271885231573b728ca4f Gitweb: https://git.kernel.org/tip/477f00f9617009a9a3a9271885231573b728ca4f Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:01 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:19:39 +0200
perf/x86/intel/ds: Extract code of event update in short period
The drain_pebs() could be called twice in a short period for auto-reload event in pmu::read(). The intel_pmu_save_and_restart_reload() should be called to update the event->count.
This case should also be handled on Icelake. Extract the code for later reuse.
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/ds.c | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-)
arch/x86/events/intel/ds.c +20 -13
@@ -1491,6 +1491,25 @@ static void intel_pmu_drain_pebs_core(struct pt_regs *iregs)__intel_pmu_pebs_event (event, iregs, at, top,0 , n); } +static void intel_pmu_pebs_event_update_no_drain (struct cpu_hw_events *cpuc,int size) +{ +struct perf_event *event; +int bit; + +/* +* The drain_pebs() could be called twice in a short period +* for auto-reload event in pmu::read(). There are no +* overflows have happened in between. +* It needs to call intel_pmu_save_and_restart_reload() to +* update the event->count for this case. +*/ +for_each_set_bit (bit, (unsigned long *)&cpuc->pebs_enabled, size) { + event = cpuc->events[bit]; +if (event->hw.flags &PERF_X86_EVENT_AUTO_RELOAD ) +intel_pmu_save_and_restart_reload (event,0 ); + } +} +static void intel_pmu_drain_pebs_nhm (struct pt_regs *iregs) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);
@@ -1518,19 +1537,7 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs) }if (unlikely (base >= top)) { -/* -* The drain_pebs() could be called twice in a short period -* for auto-reload event in pmu::read(). There are no -* overflows have happened in between. -* It needs to call intel_pmu_save_and_restart_reload() to -* update the event->count for this case. -*/ -for_each_set_bit (bit, (unsigned long *)&cpuc->pebs_enabled, - size) { - event = cpuc->events[bit]; -if (event->hw.flags &PERF_X86_EVENT_AUTO_RELOAD ) -intel_pmu_save_and_restart_reload (event,0 ); - } +intel_pmu_pebs_event_update_no_drain (cpuc, size);return ; }
PATCH v5 5/12 perf/x86/intel: Support adaptive PEBSv4
In reply to [email protected]From: Kan Liang <[email protected]>
Adaptive PEBS is a new way to report PEBS sampling information. Instead of a fixed size record for all PEBS events it allows to configure the PEBS record to only include the information needed. Events can then opt in to use such an extended record, or stay with a basic record which only contains the IP.
The major new feature is to support LBRs in PEBS record. Besides normal LBR, this allows (much faster) large PEBS, while still supporting callstacks through callstack LBR. So essentially a lot of profiling can now be done without frequent interrupts, dropping the overhead significantly.
The main requirement still is to use a period, and not use frequency mode, because frequency mode requires reevaluating the frequency on each overflow.
The floating point state (XMM) is also supported, which allows efficient profiling of FP function arguments.
Introduce specific drain function to handle variable length records. Use a new callback to parse the new record format, and also handle the STATUS field now being at a different offset.
Add code to set up the configuration register. Since there is only a single register, all events either get the full super set of all events, or only the basic record.
Originally-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
---
Changes since V4:
- set pebs_no_xmm_regs to true if only basic record supported
arch/x86/events/intel/core.c | 2 + arch/x86/events/intel/ds.c | 374 ++++++++++++++++++++++++++++-- arch/x86/events/intel/lbr.c | 22 ++ arch/x86/events/perf_event.h | 9 + arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/perf_event.h | 42 ++++ 6 files changed, 430 insertions(+), 20 deletions(-)
arch/x86/events/intel/core.c +2 -0
@@ -3507,6 +3507,8 @@ static struct intel_excl_cntrs *allocate_excl_cntrs(int cpu)int intel_cpuc_prepare (struct cpu_hw_events *cpuc,int cpu) { + cpuc->pebs_record_size = x86_pmu.pebs_record_size; +if (x86_pmu.extra_regs || x86_pmu.lbr_sel_map) { cpuc->shared_regs =allocate_shared_regs (cpu);if (!cpuc->shared_regs)
arch/x86/events/intel/ds.c +354 -20
500 lines not on this page · raw message
arch/x86/events/intel/lbr.c +22 -0
@@ -1080,6 +1080,28 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) } } +void intel_pmu_store_pebs_lbrs (struct pebs_lbr *lbr) +{ +struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +int i; + + cpuc->lbr_stack.nr = x86_pmu.lbr_nr; +for (i =0 ; i < x86_pmu.lbr_nr; i++) { +u64 info = lbr->lbr[i].info; +struct perf_branch_entry *e = &cpuc->lbr_entries[i]; + + e->from = lbr->lbr[i].from; + e->to = lbr->lbr[i].to; + e->mispred = !!(info &LBR_INFO_MISPRED ); + e->predicted = !(info &LBR_INFO_MISPRED ); + e->in_tx = !!(info &LBR_INFO_IN_TX ); + e->abort = !!(info &LBR_INFO_ABORT ); + e->cycles = info &LBR_INFO_CYCLES ; + e->reserved =0 ; + } +intel_pmu_lbr_filter (cpuc); +} +/* * Map interface branch filters onto LBR filters */
arch/x86/events/perf_event.h +9 -0
@@ -225,6 +225,11 @@ struct cpu_hw_events {int n_pebs;int n_large_pebs; +/* Current super set of events hardware configuration */ +u64 pebs_data_cfg; +u64 active_pebs_data_cfg; +int pebs_record_size; +/* * Intel LBR bits */
@@ -491,6 +496,7 @@ union perf_capabilities {* values > 32bit. */ u64 full_width_write:1; + u64 pebs_baseline:1; };u64 capabilities; };
@@ -640,6 +646,7 @@ struct x86_pmu {void (*pebs_aliases)(struct perf_event *event);int max_pebs_events;unsigned long large_pebs_flags; +u64 force_gpr_event;/* * Intel LBR
@@ -978,6 +985,8 @@ void intel_pmu_pebs_sched_task(struct perf_event_context *ctx, bool sched_in);void intel_pmu_auto_reload_read (struct perf_event *event); +void intel_pmu_store_pebs_lbrs (struct pebs_lbr *lbr); +void intel_ds_init (void );void intel_pmu_lbr_sched_task (struct perf_event_context *ctx,bool sched_in);
arch/x86/include/asm/msr-index.h +1 -0
@@ -116,6 +116,7 @@#define LBR_INFO_CYCLES 0xffff#define MSR_IA32_PEBS_ENABLE 0x000003f1 +#define MSR_PEBS_DATA_CFG 0x000003f2#define MSR_IA32_DS_AREA 0x00000600#define MSR_IA32_PERF_CAPABILITIES 0x00000345#define MSR_PEBS_LD_LAT_THRESHOLD 0x000003f6
arch/x86/include/asm/perf_event.h +42 -0
60 lines not on this page · raw message
Signature · 1 line
2.17.1
Peter Zijlstra
maintainerRe: [PATCH V5 05/12] perf/x86/intel: Support adaptive PEBSv4
Quoted 13 lines · On Tue, Apr 02, 2019 at 12:45:02PM -0700, [email protected] wrote:
arch/x86/events/intel/ds.c +13
-0+static u64 pebs_update_adaptive_cfg (struct perf_event *event) +{ +struct perf_event_attr *attr = &event->attr; +u64 sample_type = attr->sample_type; +u64 pebs_data_cfg =0 ; +bool gprs, tsx_weight; + +if ((sample_type & ~(PERF_SAMPLE_IP |PERF_SAMPLE_TIME )) || + attr->precise_ip <2 ) { + +if (sample_type &PERF_PEBS_MEMINFO_TYPE ) + pebs_data_cfg |=PEBS_DATACFG_MEMINFO ; +
arch/x86/events/intel/ds.c static inline void pebs_update_threshold(struct cpu_hw_events *cpuc) · line 961
Quoted 10 lines
arch/x86/events/intel/ds.c +10
-0+/* +* Cases we need the registers: +* + user requested registers +* + precise_ip < 2 for the non event IP +* + For RTM TSX weight we need GPRs too for the abort +* code. But we don't want to force GPRs for all other +* weights. So add it only collectfor the RTM abort event. +*/ + gprs = (sample_type &PERF_SAMPLE_REGS_INTR ) && + (attr->sample_regs_intr &0xffffffff );
did that want to be:
(attr->sample_regs_intr & PEBS_GPRS_REGS);
?
arch/x86/events/intel/ds.c static inline void pebs_update_threshold(struct cpu_hw_events *cpuc) · line 963
Quoted 2 lines
arch/x86/events/intel/ds.c +2
-0+ tsx_weight = (sample_type &PERF_SAMPLE_WEIGHT ) && + ((attr->config &0xffff ) == x86_pmu.force_gpr_event);
Why does that force_gpr_event exist? and why isn't it called rtm_abort_event?
Also did that want to be:
(attr->config & INTEL_ARCH_EVENT_MASK)
?
arch/x86/events/intel/ds.c static inline void pebs_update_threshold(struct cpu_hw_events *cpuc) · line 968
Quoted 5 lines
arch/x86/events/intel/ds.c +5
-0+if (gprs || (attr->precise_ip <2 ) || tsx_weight) + pebs_data_cfg |=PEBS_DATACFG_GPRS ; + +if ((sample_type &PERF_SAMPLE_REGS_INTR ) && + (attr->sample_regs_intr >>32 ))
Surely, you meant:
(attr->sample_regs_intr & PEBS_XMM_REGS)
?
Quoted 13 lines
arch/x86/events/intel/ds.c +13
-0+ pebs_data_cfg |=PEBS_DATACFG_XMMS ; + +if (sample_type &PERF_SAMPLE_BRANCH_STACK ) { +/* +* For now always log all LBRs. Could configure this +* later. +*/ + pebs_data_cfg |=PEBS_DATACFG_LBRS | + ((x86_pmu.lbr_nr-1 ) <<PEBS_DATACFG_LBR_SHIFT ); + } + } +return pebs_data_cfg; +}
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86/intel: Support adaptive PEBS v4
In reply to [email protected]perf/x86/intel: Support adaptive PEBS v4
Commit-ID: c22497f5838c237e3094a4dfb99d1c5de6353239 Gitweb: https://git.kernel.org/tip/c22497f5838c237e3094a4dfb99d1c5de6353239 Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:02 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:25:47 +0200
perf/x86/intel: Support adaptive PEBS v4
Adaptive PEBS is a new way to report PEBS sampling information. Instead of a fixed size record for all PEBS events it allows to configure the PEBS record to only include the information needed. Events can then opt in to use such an extended record, or stay with a basic record which only contains the IP.
The major new feature is to support LBRs in PEBS record. Besides normal LBR, this allows (much faster) large PEBS, while still supporting callstacks through callstack LBR. So essentially a lot of profiling can now be done without frequent interrupts, dropping the overhead significantly.
The main requirement still is to use a period, and not use frequency mode, because frequency mode requires reevaluating the frequency on each overflow.
The floating point state (XMM) is also supported, which allows efficient profiling of FP function arguments.
Introduce specific drain function to handle variable length records. Use a new callback to parse the new record format, and also handle the STATUS field now being at a different offset.
Add code to set up the configuration register. Since there is only a single register, all events either get the full super set of all events, or only the basic record.
Originally-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
[ Renamed GPRS => GP. ]
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/core.c | 7 + arch/x86/events/intel/ds.c | 380 +++++++++++++++++++++++++++++++++++--- arch/x86/events/intel/lbr.c | 22 +++ arch/x86/events/perf_event.h | 11 +- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/perf_event.h | 43 +++++ 6 files changed, 438 insertions(+), 26 deletions(-)
arch/x86/events/intel/core.c +7 -0
@@ -2145,6 +2145,11 @@ static void intel_pmu_enable_fixed(struct perf_event *event) bits <<= (idx *4 ); mask =0xfULL << (idx *4 ); +if (x86_pmu.intel_cap.pebs_baseline && event->attr.precise_ip) { + bits |=ICL_FIXED_0_ADAPTIVE << (idx *4 ); + mask |=ICL_FIXED_0_ADAPTIVE << (idx *4 ); + } +rdmsrl (hwc->config_base, ctrl_val); ctrl_val &= ~mask; ctrl_val |= bits;
@@ -3510,6 +3515,8 @@ static struct intel_excl_cntrs *allocate_excl_cntrs(int cpu)int intel_cpuc_prepare (struct cpu_hw_events *cpuc,int cpu) { + cpuc->pebs_record_size = x86_pmu.pebs_record_size; +if (x86_pmu.extra_regs || x86_pmu.lbr_sel_map) { cpuc->shared_regs =allocate_shared_regs (cpu);if (!cpuc->shared_regs)
arch/x86/events/intel/ds.c +355 -25
522 lines not on this page · raw message
arch/x86/events/intel/lbr.c +22 -0
@@ -1080,6 +1080,28 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) } } +void intel_pmu_store_pebs_lbrs (struct pebs_lbr *lbr) +{ +struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); +int i; + + cpuc->lbr_stack.nr = x86_pmu.lbr_nr; +for (i =0 ; i < x86_pmu.lbr_nr; i++) { +u64 info = lbr->lbr[i].info; +struct perf_branch_entry *e = &cpuc->lbr_entries[i]; + + e->from = lbr->lbr[i].from; + e->to = lbr->lbr[i].to; + e->mispred = !!(info &LBR_INFO_MISPRED ); + e->predicted = !(info &LBR_INFO_MISPRED ); + e->in_tx = !!(info &LBR_INFO_IN_TX ); + e->abort = !!(info &LBR_INFO_ABORT ); + e->cycles = info &LBR_INFO_CYCLES ; + e->reserved =0 ; + } +intel_pmu_lbr_filter (cpuc); +} +/* * Map interface branch filters onto LBR filters */
arch/x86/events/perf_event.h +10 -1
@@ -224,6 +224,11 @@ struct cpu_hw_events {int n_pebs;int n_large_pebs; +/* Current super set of events hardware configuration */ +u64 pebs_data_cfg; +u64 active_pebs_data_cfg; +int pebs_record_size; +/* * Intel LBR bits */
@@ -490,6 +495,7 @@ union perf_capabilities {* values > 32bit. */ u64 full_width_write:1; + u64 pebs_baseline:1; };u64 capabilities; };
@@ -634,11 +640,12 @@ struct x86_pmu { pebs_no_xmm_regs :1 ;int pebs_record_size;int pebs_buffer_size; +int max_pebs_events;void (*drain_pebs)(struct pt_regs *regs);struct event_constraint *pebs_constraints;void (*pebs_aliases)(struct perf_event *event); -int max_pebs_events;unsigned long large_pebs_flags; +u64 rtm_abort_event;/* * Intel LBR
@@ -978,6 +985,8 @@ void intel_pmu_pebs_sched_task(struct perf_event_context *ctx, bool sched_in);void intel_pmu_auto_reload_read (struct perf_event *event); +void intel_pmu_store_pebs_lbrs (struct pebs_lbr *lbr); +void intel_ds_init (void );void intel_pmu_lbr_sched_task (struct perf_event_context *ctx,bool sched_in);
arch/x86/include/asm/msr-index.h +1 -0
@@ -116,6 +116,7 @@#define LBR_INFO_CYCLES 0xffff#define MSR_IA32_PEBS_ENABLE 0x000003f1 +#define MSR_PEBS_DATA_CFG 0x000003f2#define MSR_IA32_DS_AREA 0x00000600#define MSR_IA32_PERF_CAPABILITIES 0x00000345#define MSR_PEBS_LD_LAT_THRESHOLD 0x000003f6
arch/x86/include/asm/perf_event.h +43 -0
61 lines not on this page · raw message
PATCH v5 6/12 perf/x86/lbr: Avoid reading the LBRs when adaptive PEBS handles them
In reply to [email protected]From: Andi Kleen <[email protected]>
With adaptive PEBS the CPU can directly supply the LBR information, so we don't need to read it again. But the LBRs still need to be enabled. Add a special count to the cpuc that distinguishes these two cases, and avoid reading the LBRs unnecessarily when PEBS is active.
Signed-off-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/intel/lbr.c | 13 ++++++++++++- arch/x86/events/perf_event.h | 1 + 2 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/events/intel/lbr.c +12 -1
@@ -488,6 +488,8 @@ void intel_pmu_lbr_add(struct perf_event *event)* be 'new'. Conversely, a new event can get installed through the * context switch path for the first time. */ +if (x86_pmu.intel_cap.pebs_baseline && event->attr.precise_ip >0 ) + cpuc->lbr_pebs_users++;perf_sched_cb_inc (event->ctx->pmu);if (!cpuc->lbr_users++ && !event->total_time_running)intel_pmu_lbr_reset ();
@@ -507,8 +509,11 @@ void intel_pmu_lbr_del(struct perf_event *event) task_ctx->lbr_callstack_users--; } +if (x86_pmu.intel_cap.pebs_baseline && event->attr.precise_ip >0 ) + cpuc->lbr_pebs_users--; cpuc->lbr_users--;WARN_ON_ONCE (cpuc->lbr_users <0 ); +WARN_ON_ONCE (cpuc->lbr_pebs_users <0 );perf_sched_cb_dec (event->ctx->pmu); }
@@ -658,7 +663,13 @@ void intel_pmu_lbr_read(void) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); -if (!cpuc->lbr_users) +/* +* Don't read when all LBRs users are using adaptive PEBS. +* +* This could be smarter and actually check the event, +* but this simple approach seems to work for now. +*/ +if (!cpuc->lbr_users || cpuc->lbr_users == cpuc->lbr_pebs_users)return ;if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_32 )
arch/x86/events/perf_event.h +1 -0
@@ -234,6 +234,7 @@ struct cpu_hw_events {* Intel LBR bits */ int lbr_users; +int lbr_pebs_users;struct perf_branch_stack lbr_stack;struct perf_branch_entry lbr_entries[MAX_LBR_ENTRIES ];struct er_account *lbr_sel;
Signature · 1 line
2.17.1
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/lbr: Avoid reading the LBRs when adaptive PEBS handles them
perf/x86/lbr: Avoid reading the LBRs when adaptive PEBS handles them
Commit-ID: d3617b98b04583df222f34992e65712862a77bf1 Gitweb: https://git.kernel.org/tip/d3617b98b04583df222f34992e65712862a77bf1 Author: Andi Kleen <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:03 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:26:17 +0200
perf/x86/lbr: Avoid reading the LBRs when adaptive PEBS handles them
With adaptive PEBS the CPU can directly supply the LBR information, so we don't need to read it again. But the LBRs still need to be enabled. Add a special count to the cpuc that distinguishes these two cases, and avoid reading the LBRs unnecessarily when PEBS is active.
Signed-off-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/lbr.c | 13 ++++++++++++- arch/x86/events/perf_event.h | 1 + 2 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/events/intel/lbr.c +12 -1
@@ -488,6 +488,8 @@ void intel_pmu_lbr_add(struct perf_event *event)* be 'new'. Conversely, a new event can get installed through the * context switch path for the first time. */ +if (x86_pmu.intel_cap.pebs_baseline && event->attr.precise_ip >0 ) + cpuc->lbr_pebs_users++;perf_sched_cb_inc (event->ctx->pmu);if (!cpuc->lbr_users++ && !event->total_time_running)intel_pmu_lbr_reset ();
@@ -507,8 +509,11 @@ void intel_pmu_lbr_del(struct perf_event *event) task_ctx->lbr_callstack_users--; } +if (x86_pmu.intel_cap.pebs_baseline && event->attr.precise_ip >0 ) + cpuc->lbr_pebs_users--; cpuc->lbr_users--;WARN_ON_ONCE (cpuc->lbr_users <0 ); +WARN_ON_ONCE (cpuc->lbr_pebs_users <0 );perf_sched_cb_dec (event->ctx->pmu); }
@@ -658,7 +663,13 @@ void intel_pmu_lbr_read(void) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); -if (!cpuc->lbr_users) +/* +* Don't read when all LBRs users are using adaptive PEBS. +* +* This could be smarter and actually check the event, +* but this simple approach seems to work for now. +*/ +if (!cpuc->lbr_users || cpuc->lbr_users == cpuc->lbr_pebs_users)return ;if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_32 )
arch/x86/events/perf_event.h +1 -0
@@ -233,6 +233,7 @@ struct cpu_hw_events {* Intel LBR bits */ int lbr_users; +int lbr_pebs_users;struct perf_branch_stack lbr_stack;struct perf_branch_entry lbr_entries[MAX_LBR_ENTRIES ];struct er_account *lbr_sel;
PATCH v5 7/12 perf/x86: Support constraint ranges
In reply to [email protected]From: Peter Zijlstra <[email protected]>
Icelake extended the general counters to 8, even when SMT is enabled. However only a (large) subset of the events can be used on all 8 counters.
The events that can or cannot be used on all counters are organized in ranges.
A lot of scheduler constraints are required to handle all this.
To avoid blowing up the tables add event code ranges to the constraint tables, and a new inline function to match them.
Originally-by Andi Kleen <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/intel/core.c | 2 +- arch/x86/events/intel/ds.c | 2 +- arch/x86/events/perf_event.h | 42 ++++++++++++++++++++++++++++++------ 3 files changed, 38 insertions(+), 8 deletions(-)
arch/x86/events/intel/core.c +1 -1
@@ -2688,7 +2688,7 @@ x86_get_event_constraints(struct cpu_hw_events *cpuc, int idx,if (x86_pmu.event_constraints) {for_each_event_constraint (c, x86_pmu.event_constraints) { -if ((event->hw.config & c->cmask) == c->code) { +if (constraint_match (c, event->hw.config)) { event->hw.flags |= c->flags;return c; }
arch/x86/events/intel/ds.c +1 -1
@@ -858,7 +858,7 @@ struct event_constraint *intel_pebs_constraints(struct perf_event *event)if (x86_pmu.pebs_constraints) {for_each_event_constraint (c, x86_pmu.pebs_constraints) { -if ((event->hw.config & c->cmask) == c->code) { +if (constraint_match (c, event->hw.config)) { event->hw.flags |= c->flags;return c; }
arch/x86/events/perf_event.h +36 -6
89 lines not on this page · raw message
Signature · 1 line
2.17.1
tip-bot for Peter Zijlstra
merge bot[tip:perf/core] perf/x86: Support constraint ranges
perf/x86: Support constraint ranges
Commit-ID: 63b79f6ebc464afb730bc45762c820795e276da1 Gitweb: https://git.kernel.org/tip/63b79f6ebc464afb730bc45762c820795e276da1 Author: Peter Zijlstra <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:04 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:26:17 +0200
perf/x86: Support constraint ranges
Icelake extended the general counters to 8, even when SMT is enabled. However only a (large) subset of the events can be used on all 8 counters.
The events that can or cannot be used on all counters are organized in ranges.
A lot of scheduler constraints are required to handle all this.
To avoid blowing up the tables add event code ranges to the constraint tables, and a new inline function to match them.
Originally-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/core.c | 2 +- arch/x86/events/intel/ds.c | 2 +- arch/x86/events/perf_event.h | 43 +++++++++++++++++++++++++++++++++++++------ 3 files changed, 39 insertions(+), 8 deletions(-)
arch/x86/events/intel/core.c +1 -1
@@ -2693,7 +2693,7 @@ x86_get_event_constraints(struct cpu_hw_events *cpuc, int idx,if (x86_pmu.event_constraints) {for_each_event_constraint (c, x86_pmu.event_constraints) { -if ((event->hw.config & c->cmask) == c->code) { +if (constraint_match (c, event->hw.config)) { event->hw.flags |= c->flags;return c; }
arch/x86/events/intel/ds.c +1 -1
@@ -858,7 +858,7 @@ struct event_constraint *intel_pebs_constraints(struct perf_event *event)if (x86_pmu.pebs_constraints) {for_each_event_constraint (c, x86_pmu.pebs_constraints) { -if ((event->hw.config & c->cmask) == c->code) { +if (constraint_match (c, event->hw.config)) { event->hw.flags |= c->flags;return c; }
arch/x86/events/perf_event.h +37 -6
86 lines not on this page · raw message
PATCH v5 8/12 perf/x86/intel: Add Icelake support
In reply to [email protected]From: Kan Liang <[email protected]>
Add Icelake core PMU perf code, including constraint tables and the main enable code.
Icelake expanded the generic counters to always 8 even with HT on, but a range of events cannot be scheduled on the extra 4 counters. Add new constraint ranges to describe this to the scheduler. The number of constraints that need to be checked is larger now than with earlier CPUs.
At some point we may need a new data structure to look them up more efficiently than with linear search. So far it still seems to be acceptable however.
Icelake added a new fixed counter SLOTS. Full support for it is added later in the patch series.
The cache events table is identical to Skylake.
Compare to PEBS instruction event on generic counter, fixed counter 0 has less skid. Force instruction:ppp always in fixed counter 0.
Originally-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
---
Changes since V4:
- remove has_xmm_regs
arch/x86/events/intel/core.c | 111 ++++++++++++++++++++++++++++++ arch/x86/events/intel/ds.c | 26 ++++++- arch/x86/events/perf_event.h | 2 + arch/x86/include/asm/intel_ds.h | 2 +- arch/x86/include/asm/perf_event.h | 2 +- 5 files changed, 139 insertions(+), 4 deletions(-)
arch/x86/events/intel/core.c +111 -0
@@ -239,6 +239,35 @@ static struct extra_reg intel_skl_extra_regs[] __read_mostly = {EVENT_EXTRA_END }; +static struct event_constraint intel_icl_event_constraints[] = { +FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ),/* INST_RETIRED.ANY */ +INTEL_UEVENT_CONSTRAINT (0x1c0 ,0 ),/* INST_RETIRED.PREC_DIST */ +FIXED_EVENT_CONSTRAINT (0x003c ,1 ),/* CPU_CLK_UNHALTED.CORE */ +FIXED_EVENT_CONSTRAINT (0x0300 ,2 ),/* CPU_CLK_UNHALTED.REF */ +FIXED_EVENT_CONSTRAINT (0x0400 ,3 ),/* SLOTS */ +INTEL_EVENT_CONSTRAINT_RANGE (0x03 ,0x0a ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0x1f ,0x28 ,0xf ), +INTEL_EVENT_CONSTRAINT (0x32 ,0xf ),/* SW_PREFETCH_ACCESS.* */ +INTEL_EVENT_CONSTRAINT_RANGE (0x48 ,0x54 ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0x60 ,0x8b ,0xf ), +INTEL_UEVENT_CONSTRAINT (0x04a3 ,0xff ),/* CYCLE_ACTIVITY.STALLS_TOTAL */ +INTEL_UEVENT_CONSTRAINT (0x10a3 ,0xff ),/* CYCLE_ACTIVITY.STALLS_MEM_ANY */ +INTEL_EVENT_CONSTRAINT (0xa3 ,0xf ),/* CYCLE_ACTIVITY.* */ +INTEL_EVENT_CONSTRAINT_RANGE (0xa8 ,0xb0 ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0xb7 ,0xbd ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0xd0 ,0xe6 ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0xf0 ,0xf4 ,0xf ), +EVENT_CONSTRAINT_END +}; + +static struct extra_reg intel_icl_extra_regs[]__read_mostly = { +INTEL_UEVENT_EXTRA_REG (0x01b7 ,MSR_OFFCORE_RSP_0 ,0x3fffff9fffull ,RSP_0 ), +INTEL_UEVENT_EXTRA_REG (0x01bb ,MSR_OFFCORE_RSP_1 ,0x3fffff9fffull ,RSP_1 ), +INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG (0x01cd ), +INTEL_UEVENT_EXTRA_REG (0x01c6 ,MSR_PEBS_FRONTEND ,0x7fff17 ,FE ), +EVENT_EXTRA_END +}; +EVENT_ATTR_STR (mem-loads, mem_ld_nhm,"event=0x0b,umask=0x10,ldlat=3" );EVENT_ATTR_STR (mem-loads, mem_ld_snb,"event=0xcd,umask=0x1,ldlat=3" );EVENT_ATTR_STR (mem-stores, mem_st_snb,"event=0xcd,umask=0x2" );
@@ -3366,6 +3395,9 @@ static struct event_constraint counter0_constraint =static struct event_constraint counter2_constraint =EVENT_CONSTRAINT (0 ,0x4 ,0 ); +static struct event_constraint fixed_counter0_constraint = +FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ); +static struct event_constraint *hsw_get_event_constraints (struct cpu_hw_events *cpuc,int idx,struct perf_event *event)
@@ -3384,6 +3416,21 @@ hsw_get_event_constraints(struct cpu_hw_events *cpuc, int idx,return c; } +static struct event_constraint * +icl_get_event_constraints (struct cpu_hw_events *cpuc,int idx, +struct perf_event *event) +{ +/* +* Fixed counter 0 has less skid. +* Force instruction:ppp in Fixed counter 0 +*/ +if ((event->attr.precise_ip ==3 ) && + ((event->hw.config &X86_RAW_EVENT_MASK ) ==0x00c0 )) +return &fixed_counter0_constraint; + +return hsw_get_event_constraints (cpuc, idx, event); +} +static struct event_constraint *glp_get_event_constraints (struct cpu_hw_events *cpuc,int idx,struct perf_event *event)
@@ -4110,6 +4157,42 @@ static struct attribute *hsw_tsx_events_attrs[] = { NULL }; +EVENT_ATTR_STR (tx-capacity-read, tx_capacity_read,"event=0x54,umask=0x80" ); +EVENT_ATTR_STR (tx-capacity-write, tx_capacity_write,"event=0x54,umask=0x2" ); +EVENT_ATTR_STR (el-capacity-read, el_capacity_read,"event=0x54,umask=0x80" ); +EVENT_ATTR_STR (el-capacity-write, el_capacity_write,"event=0x54,umask=0x2" ); + +static struct attribute *icl_events_attrs[] = { +EVENT_PTR (mem_ld_hsw), +EVENT_PTR (mem_st_hsw), +NULL , +}; + +static struct attribute *icl_tsx_events_attrs[] = { +EVENT_PTR (tx_start), +EVENT_PTR (tx_abort), +EVENT_PTR (tx_commit), +EVENT_PTR (tx_capacity_read), +EVENT_PTR (tx_capacity_write), +EVENT_PTR (tx_conflict), +EVENT_PTR (el_start), +EVENT_PTR (el_abort), +EVENT_PTR (el_commit), +EVENT_PTR (el_capacity_read), +EVENT_PTR (el_capacity_write), +EVENT_PTR (el_conflict), +EVENT_PTR (cycles_t), +EVENT_PTR (cycles_ct), +NULL , +}; + +static __init struct attribute **get_icl_events_attrs (void ) +{ +return boot_cpu_has (X86_FEATURE_RTM ) ? +merge_attr (icl_events_attrs, icl_tsx_events_attrs) : + icl_events_attrs; +} +static ssize_t freeze_on_smi_show (struct device *cdev,struct device_attribute *attr,char *buf)
@@ -4697,6 +4780,34 @@ __init int intel_pmu_init(void) name ="skylake" ;break ; +case INTEL_FAM6_ICELAKE_MOBILE : + x86_pmu.late_ack = true; +memcpy (hw_cache_event_ids, skl_hw_cache_event_ids,sizeof (hw_cache_event_ids)); +memcpy (hw_cache_extra_regs, skl_hw_cache_extra_regs,sizeof (hw_cache_extra_regs)); + hw_cache_event_ids[C (ITLB )][C (OP_READ )][C (RESULT_ACCESS )] =-1 ; +intel_pmu_lbr_init_skl (); + + x86_pmu.event_constraints = intel_icl_event_constraints; + x86_pmu.pebs_constraints = intel_icl_pebs_event_constraints; + x86_pmu.extra_regs = intel_icl_extra_regs; + x86_pmu.pebs_aliases =NULL ; + x86_pmu.pebs_prec_dist = true; + x86_pmu.flags |=PMU_FL_HAS_RSP_1 ; + x86_pmu.flags |=PMU_FL_NO_HT_SHARING ; + + x86_pmu.hw_config = hsw_hw_config; + x86_pmu.get_event_constraints = icl_get_event_constraints; + extra_attr =boot_cpu_has (X86_FEATURE_RTM ) ? + hsw_format_attr : nhm_format_attr; + extra_attr =merge_attr (extra_attr, skl_format_attr); + x86_pmu.cpu_events =get_icl_events_attrs (); + x86_pmu.force_gpr_event =0x2ca ; + x86_pmu.lbr_pt_coexist = true; +intel_pmu_pebs_data_source_skl (false); +pr_cont ("Icelake events, " ); + name ="icelake" ; +break ; +default :switch (x86_pmu.version) {case 1 :
arch/x86/events/intel/ds.c +24 -2
@@ -849,6 +849,26 @@ struct event_constraint intel_skl_pebs_event_constraints[] = { EVENT_CONSTRAINT_END }; +struct event_constraint intel_icl_pebs_event_constraints[] = { +INTEL_FLAGS_UEVENT_CONSTRAINT (0x1c0 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ +INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x400000000ULL ),/* SLOTS */ + +INTEL_PLD_CONSTRAINT (0x1cd ,0xff ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x1d0 ,0xf ),/* MEM_INST_RETIRED.LOAD */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x2d0 ,0xf ),/* MEM_INST_RETIRED.STORE */ + +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD_RANGE (0xd1 ,0xd4 ,0xf ),/* MEM_LOAD_*_RETIRED.* */ + +INTEL_FLAGS_EVENT_CONSTRAINT (0xd0 ,0xf ),/* MEM_INST_RETIRED.* */ + +/* +* Everything else is handled by PMU_FL_PEBS_ALL, because we +* need the full constraints from the main table. +*/ + +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *c;
@@ -1056,7 +1076,8 @@ void intel_pmu_pebs_enable(struct perf_event *event) cpuc->pebs_enabled |=1ULL << hwc->idx; -if (event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) +if ((event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) && + (x86_pmu.version <5 )) cpuc->pebs_enabled |=1ULL << (hwc->idx +32 );else if (event->hw.flags &PERF_X86_EVENT_PEBS_ST ) cpuc->pebs_enabled |=1ULL <<63 ;
@@ -1108,7 +1129,8 @@ void intel_pmu_pebs_disable(struct perf_event *event) cpuc->pebs_enabled &= ~(1ULL << hwc->idx); -if (event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) +if ((event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) && + (x86_pmu.version <5 )) cpuc->pebs_enabled &= ~(1ULL << (hwc->idx +32 ));else if (event->hw.flags &PERF_X86_EVENT_PEBS_ST ) cpuc->pebs_enabled &= ~(1ULL <<63 );
arch/x86/events/perf_event.h +2 -0
@@ -998,6 +998,8 @@ extern struct event_constraint intel_bdw_pebs_event_constraints[];extern struct event_constraint intel_skl_pebs_event_constraints[]; +extern struct event_constraint intel_icl_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_add (struct perf_event *event);
arch/x86/include/asm/intel_ds.h +1 -1
@@ -8,7 +8,7 @@/* The maximal number of PEBS events: */ #define MAX_PEBS_EVENTS 8 -#define MAX_FIXED_PEBS_EVENTS 3 +#define MAX_FIXED_PEBS_EVENTS 4/* * A debug store configuration.
arch/x86/include/asm/perf_event.h +1 -1
@@ -7,7 +7,7 @@*/ #define INTEL_PMC_MAX_GENERIC 32 -#define INTEL_PMC_MAX_FIXED 3 +#define INTEL_PMC_MAX_FIXED 4#define INTEL_PMC_IDX_FIXED 32#define X86_PMC_IDX_MAX 64
Signature · 1 line
2.17.1
Peter Zijlstra
maintainerRe: [PATCH V5 08/12] perf/x86/intel: Add Icelake support
arch/x86/events/intel/core.c hsw_get_event_constraints(struct cpu_hw_events *cpuc, int idx, · line 3429
Quoted 11 lines · On Tue, Apr 02, 2019 at 12:45:05PM -0700, [email protected] wrote:
arch/x86/events/intel/core.c +11
-0+static struct event_constraint * +icl_get_event_constraints (struct cpu_hw_events *cpuc,int idx, +struct perf_event *event) +{ +/* +* Fixed counter 0 has less skid. +* Force instruction:ppp in Fixed counter 0 +*/ +if ((event->attr.precise_ip ==3 ) && + ((event->hw.config &X86_RAW_EVENT_MASK ) ==0x00c0 )) +return &fixed_counter0_constraint;
Does that want to be:
event->hw.config == X86_CONFIG(.event=0xc0)
?
That is, are there really bits we want to mask in there?
Quoted 3 lines
arch/x86/events/intel/core.c +3
-0+ +return hsw_get_event_constraints (cpuc, idx, event); +}
Re: [PATCH V5 08/12] perf/x86/intel: Add Icelake support
Quoted 15 lines · Peter Zijlstra
arch/x86/events/intel/core.c hsw_get_event_constraints(struct cpu_hw_events *cpuc, int idx, · line 3429
Quoted 11 lines · On Tue, Apr 02, 2019 at 12:45:05PM -0700, [email protected] wrote:
arch/x86/events/intel/core.c +11
-0+static struct event_constraint * +icl_get_event_constraints (struct cpu_hw_events *cpuc,int idx, +struct perf_event *event) +{ +/* +* Fixed counter 0 has less skid. +* Force instruction:ppp in Fixed counter 0 +*/ +if ((event->attr.precise_ip ==3 ) && + ((event->hw.config &X86_RAW_EVENT_MASK ) ==0x00c0 )) +return &fixed_counter0_constraint;Does that want to be:
Quoted 5 lines
event->hw.config == X86_CONFIG(.event=0xc0)?
That is, are there really bits we want to mask in there?
For instruction event, right, we don't need mask it. I will change it.
Thanks,
Kan
Quoted 4 lines
Quoted 4 lines
arch/x86/events/intel/core.c +3
-0+ +return hsw_get_event_constraints (cpuc, idx, event); +}
Re: [PATCH V5 08/12] perf/x86/intel: Add Icelake support
Quoted 18 lines · Kan
Quoted 15 lines · Peter Zijlstra
Review
Quoted 11 lines · On Tue, Apr 02, 2019 at 12:45:05PM -0700, [email protected] wrote:
(quoted diff) +11
-0+static struct event_constraint * +icl_get_event_constraints(struct cpu_hw_events *cpuc, int idx, + struct perf_event *event) +{ + /* + * Fixed counter 0 has less skid. + * Force instruction:ppp in Fixed counter 0 + */ + if ((event->attr.precise_ip == 3) && + ((event->hw.config & X86_RAW_EVENT_MASK) == 0x00c0)) + return &fixed_counter0_constraint;Does that want to be:
Quoted 9 lines
Quoted 5 lines
event->hw.config==X86_CONFIG(.event=0xc0)?
That is, are there really bits we want to mask in there?
For instruction event, right, we don't need mask it. I will change it.
Actually, we have to mask some bits here, e.g. ARCH_PERFMON_EVENTSEL_INT, ARCH_PERFMON_EVENTSEL_USR and ARCH_PERFMON_EVENTSEL_OS. Those bits will be set in hw_config().
Also, other filds, e.g the INV, ANY, E, or CMASK fields are not allowed for reduced Skid PEBS.
arch/x86/events/intel/core.c +4 -1
@@ -3463,6 +3463,9 @@ hsw_get_event_constraints(struct cpu_hw_events *cpuc, int idx,return c; } +#define EVENT_CONFIG (config) \ + (config & (X86_ALL_EVENT_FLAGS | INTEL_ARCH_EVENT_MASK)) +static struct event_constraint *icl_get_event_constraints (struct cpu_hw_events *cpuc,int idx,struct perf_event *event)
@@ -3472,7 +3475,7 @@ icl_get_event_constraints(struct cpu_hw_events *cpuc, int idx,* Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip ==3 ) && - (event->hw.config == X86_CONFIG(.event=0xc0))) + (EVENT_CONFIG(event->hw.config) == X86_CONFIG(.event=0xc0)))return &fixed_counter0_constraint;return hsw_get_event_constraints (cpuc, idx, event);
Thanks,
Kan
Peter Zijlstra
maintainerRe: [PATCH V5 08/12] perf/x86/intel: Add Icelake support
Quoted 9 lines · Kan
Quoted 5 lines
Quoted 1 line
That is, are there really bits we want to mask in there?
For instruction event, right, we don't need mask it. I will change it.
Actually, we have to mask some bits here, e.g.
ARCH_PERFMON_EVENTSEL_INT,ARCH_PERFMON_EVENTSEL_USRandARCH_PERFMON_EVENTSEL_OS. Those bits will be set inhw_config().
Ah, bah, You're right. I misread and though we were comparing against the user provided raw config.
Quoted 3 lines
Also, other filds, e.g the INV, ANY, E, or CMASK fields are not allowed for reduced Skid PEBS.
Sure, those are actually forced 0 with the existing thing.
I'll go fold smething like back in. Thanks!
Quoted 29 lines
arch/x86/events/intel/core.c +3
-0@@ -3463,6 +3463,9 @@ hsw_get_event_constraints(struct cpu_hw_events *cpuc,int idx, return c; } +#define EVENT_CONFIG (config) \ + (config & (X86_ALL_EVENT_FLAGS | INTEL_ARCH_EVENT_MASK)) +static struct event_constraint * icl_get_event_constraints(struct cpu_hw_events *cpuc, int idx,arch/x86/events/intel/core.c +1
-1struct perf_event *event)@@ -3472,7 +3475,7 @@ icl_get_event_constraints(struct cpu_hw_events *cpuc,int idx, * Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip ==3 ) && - (event->hw.config == X86_CONFIG(.event=0xc0))) + (EVENT_CONFIG(event->hw.config) == X86_CONFIG(.event=0xc0)))return &fixed_counter0_constraint;return
hsw_get_event_constraints(cpuc, idx, event);Thanks,
Kan
Peter Zijlstra
maintainerRe: [PATCH V5 08/12] perf/x86/intel: Add Icelake support
Quoted 13 lines · Peter Zijlstra
Sure, those are actually forced 0 with the existing thing.
I'll go fold smething like back in. Thanks!
Quoted 9 lines
(quoted diff) +1
-1@@ -3472,7 +3475,7 @@ icl_get_event_constraints(struct cpu_hw_events *cpuc, int idx, * Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip == 3) && - (event->hw.config == X86_CONFIG(.event=0xc0))) + (EVENT_CONFIG(event->hw.config) == X86_CONFIG(.event=0xc0))) return &fixed_counter0_constraint;
Quoted 1 line
Quoted 1 line
return
hsw_get_event_constraints(cpuc, idx, event);
Staring at that I came up with the below; how does that look?
Note: FIXED_EVENT_CONSTRAINT(0x00c0, 0) has FIXED_EVENT_FLAGS for ->cmask and includes the correct event code.
---
arch/x86/events/intel/core.c +1 -1
@@ -3433,7 +3433,7 @@ icl_get_event_constraints(struct cpu_hw_* Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip ==3 ) && - (event->hw.config == X86_CONFIG(.event=0xc0))) +constraint_match (&fixed_counter0_constraint, event->hw.config))return &fixed_counter0_constraint;return hsw_get_event_constraints (cpuc, idx, event);
Re: [PATCH V5 08/12] perf/x86/intel: Add Icelake support
Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Peter Zijlstra
Sure, those are actually forced 0 with the existing thing.
I'll go fold smething like back in. Thanks!
Quoted 9 lines
arch/x86/events/intel/core.c +1
-1@@ -3472,7 +3475,7 @@ icl_get_event_constraints(struct cpu_hw_events *cpuc,int idx, * Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip ==3 ) && - (event->hw.config == X86_CONFIG(.event=0xc0))) + (EVENT_CONFIG(event->hw.config) == X86_CONFIG(.event=0xc0)))return &fixed_counter0_constraint;
Quoted 4 lines
Quoted 1 line
Quoted 1 line
return hsw_get_event_constraints(cpuc, idx, event);Staring at that I came up with the below; how does that look?
Yes, it looks better.
Thanks,
Kan
Quoted 16 lines
Note:
FIXED_EVENT_CONSTRAINT(0x00c0, 0)hasFIXED_EVENT_FLAGSfor ->cmask and includes the correct event code.---
arch/x86/events/intel/core.c +1
-1@@ -3433,7 +3433,7 @@ icl_get_event_constraints(struct cpu_hw_* Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip ==3 ) && - (event->hw.config == X86_CONFIG(.event=0xc0))) +constraint_match (&fixed_counter0_constraint, event->hw.config))return &fixed_counter0_constraint;return hsw_get_event_constraints (cpuc, idx, event);
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86/intel: Add Icelake support
In reply to [email protected]perf/x86/intel: Add Icelake support
Commit-ID: 6017608936c1825ff5d7325270484042f597edff Gitweb: https://git.kernel.org/tip/6017608936c1825ff5d7325270484042f597edff Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:05 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:26:18 +0200
perf/x86/intel: Add Icelake support
Add Icelake core PMU perf code, including constraint tables and the main enable code.
Icelake expanded the generic counters to always 8 even with HT on, but a range of events cannot be scheduled on the extra 4 counters. Add new constraint ranges to describe this to the scheduler. The number of constraints that need to be checked is larger now than with earlier CPUs.
At some point we may need a new data structure to look them up more efficiently than with linear search. So far it still seems to be acceptable however.
Icelake added a new fixed counter SLOTS. Full support for it is added later in the patch series.
The cache events table is identical to Skylake.
Compare to PEBS instruction event on generic counter, fixed counter 0 has less skid. Force instruction:ppp always in fixed counter 0.
Originally-by Andi Kleen <[email protected]>
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/core.c | 111 ++++++++++++++++++++++++++++++++++++++ arch/x86/events/intel/ds.c | 25 ++++++++- arch/x86/events/perf_event.h | 2 + arch/x86/include/asm/intel_ds.h | 2 +- arch/x86/include/asm/perf_event.h | 2 +- 5 files changed, 138 insertions(+), 4 deletions(-)
arch/x86/events/intel/core.c +111 -0
@@ -239,6 +239,35 @@ static struct extra_reg intel_skl_extra_regs[] __read_mostly = {EVENT_EXTRA_END }; +static struct event_constraint intel_icl_event_constraints[] = { +FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ),/* INST_RETIRED.ANY */ +INTEL_UEVENT_CONSTRAINT (0x1c0 ,0 ),/* INST_RETIRED.PREC_DIST */ +FIXED_EVENT_CONSTRAINT (0x003c ,1 ),/* CPU_CLK_UNHALTED.CORE */ +FIXED_EVENT_CONSTRAINT (0x0300 ,2 ),/* CPU_CLK_UNHALTED.REF */ +FIXED_EVENT_CONSTRAINT (0x0400 ,3 ),/* SLOTS */ +INTEL_EVENT_CONSTRAINT_RANGE (0x03 ,0x0a ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0x1f ,0x28 ,0xf ), +INTEL_EVENT_CONSTRAINT (0x32 ,0xf ),/* SW_PREFETCH_ACCESS.* */ +INTEL_EVENT_CONSTRAINT_RANGE (0x48 ,0x54 ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0x60 ,0x8b ,0xf ), +INTEL_UEVENT_CONSTRAINT (0x04a3 ,0xff ),/* CYCLE_ACTIVITY.STALLS_TOTAL */ +INTEL_UEVENT_CONSTRAINT (0x10a3 ,0xff ),/* CYCLE_ACTIVITY.STALLS_MEM_ANY */ +INTEL_EVENT_CONSTRAINT (0xa3 ,0xf ),/* CYCLE_ACTIVITY.* */ +INTEL_EVENT_CONSTRAINT_RANGE (0xa8 ,0xb0 ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0xb7 ,0xbd ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0xd0 ,0xe6 ,0xf ), +INTEL_EVENT_CONSTRAINT_RANGE (0xf0 ,0xf4 ,0xf ), +EVENT_CONSTRAINT_END +}; + +static struct extra_reg intel_icl_extra_regs[]__read_mostly = { +INTEL_UEVENT_EXTRA_REG (0x01b7 ,MSR_OFFCORE_RSP_0 ,0x3fffff9fffull ,RSP_0 ), +INTEL_UEVENT_EXTRA_REG (0x01bb ,MSR_OFFCORE_RSP_1 ,0x3fffff9fffull ,RSP_1 ), +INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG (0x01cd ), +INTEL_UEVENT_EXTRA_REG (0x01c6 ,MSR_PEBS_FRONTEND ,0x7fff17 ,FE ), +EVENT_EXTRA_END +}; +EVENT_ATTR_STR (mem-loads, mem_ld_nhm,"event=0x0b,umask=0x10,ldlat=3" );EVENT_ATTR_STR (mem-loads, mem_ld_snb,"event=0xcd,umask=0x1,ldlat=3" );EVENT_ATTR_STR (mem-stores, mem_st_snb,"event=0xcd,umask=0x2" );
@@ -3374,6 +3403,9 @@ static struct event_constraint counter0_constraint =static struct event_constraint counter2_constraint =EVENT_CONSTRAINT (0 ,0x4 ,0 ); +static struct event_constraint fixed0_constraint = +FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ); +static struct event_constraint *hsw_get_event_constraints (struct cpu_hw_events *cpuc,int idx,struct perf_event *event)
@@ -3392,6 +3424,21 @@ hsw_get_event_constraints(struct cpu_hw_events *cpuc, int idx,return c; } +static struct event_constraint * +icl_get_event_constraints (struct cpu_hw_events *cpuc,int idx, +struct perf_event *event) +{ +/* +* Fixed counter 0 has less skid. +* Force instruction:ppp in Fixed counter 0 +*/ +if ((event->attr.precise_ip ==3 ) && +constraint_match (&fixed0_constraint, event->hw.config)) +return &fixed0_constraint; + +return hsw_get_event_constraints (cpuc, idx, event); +} +static struct event_constraint *glp_get_event_constraints (struct cpu_hw_events *cpuc,int idx,struct perf_event *event)
@@ -4124,6 +4171,42 @@ static struct attribute *hsw_tsx_events_attrs[] = { NULL }; +EVENT_ATTR_STR (tx-capacity-read, tx_capacity_read,"event=0x54,umask=0x80" ); +EVENT_ATTR_STR (tx-capacity-write, tx_capacity_write,"event=0x54,umask=0x2" ); +EVENT_ATTR_STR (el-capacity-read, el_capacity_read,"event=0x54,umask=0x80" ); +EVENT_ATTR_STR (el-capacity-write, el_capacity_write,"event=0x54,umask=0x2" ); + +static struct attribute *icl_events_attrs[] = { +EVENT_PTR (mem_ld_hsw), +EVENT_PTR (mem_st_hsw), +NULL , +}; + +static struct attribute *icl_tsx_events_attrs[] = { +EVENT_PTR (tx_start), +EVENT_PTR (tx_abort), +EVENT_PTR (tx_commit), +EVENT_PTR (tx_capacity_read), +EVENT_PTR (tx_capacity_write), +EVENT_PTR (tx_conflict), +EVENT_PTR (el_start), +EVENT_PTR (el_abort), +EVENT_PTR (el_commit), +EVENT_PTR (el_capacity_read), +EVENT_PTR (el_capacity_write), +EVENT_PTR (el_conflict), +EVENT_PTR (cycles_t), +EVENT_PTR (cycles_ct), +NULL , +}; + +static __init struct attribute **get_icl_events_attrs (void ) +{ +return boot_cpu_has (X86_FEATURE_RTM ) ? +merge_attr (icl_events_attrs, icl_tsx_events_attrs) : + icl_events_attrs; +} +static ssize_t freeze_on_smi_show (struct device *cdev,struct device_attribute *attr,char *buf)
@@ -4757,6 +4840,34 @@ __init int intel_pmu_init(void) name ="skylake" ;break ; +case INTEL_FAM6_ICELAKE_MOBILE : + x86_pmu.late_ack = true; +memcpy (hw_cache_event_ids, skl_hw_cache_event_ids,sizeof (hw_cache_event_ids)); +memcpy (hw_cache_extra_regs, skl_hw_cache_extra_regs,sizeof (hw_cache_extra_regs)); + hw_cache_event_ids[C (ITLB )][C (OP_READ )][C (RESULT_ACCESS )] =-1 ; +intel_pmu_lbr_init_skl (); + + x86_pmu.event_constraints = intel_icl_event_constraints; + x86_pmu.pebs_constraints = intel_icl_pebs_event_constraints; + x86_pmu.extra_regs = intel_icl_extra_regs; + x86_pmu.pebs_aliases =NULL ; + x86_pmu.pebs_prec_dist = true; + x86_pmu.flags |=PMU_FL_HAS_RSP_1 ; + x86_pmu.flags |=PMU_FL_NO_HT_SHARING ; + + x86_pmu.hw_config = hsw_hw_config; + x86_pmu.get_event_constraints = icl_get_event_constraints; + extra_attr =boot_cpu_has (X86_FEATURE_RTM ) ? + hsw_format_attr : nhm_format_attr; + extra_attr =merge_attr (extra_attr, skl_format_attr); + x86_pmu.cpu_events =get_icl_events_attrs (); + x86_pmu.rtm_abort_event = X86_CONFIG(.event=0xca, .umask=0x02); + x86_pmu.lbr_pt_coexist = true; +intel_pmu_pebs_data_source_skl (false); +pr_cont ("Icelake events, " ); + name ="icelake" ; +break ; +default :switch (x86_pmu.version) {case 1 :
arch/x86/events/intel/ds.c +23 -2
@@ -849,6 +849,26 @@ struct event_constraint intel_skl_pebs_event_constraints[] = { EVENT_CONSTRAINT_END }; +struct event_constraint intel_icl_pebs_event_constraints[] = { +INTEL_FLAGS_UEVENT_CONSTRAINT (0x1c0 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ +INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x400000000ULL ),/* SLOTS */ + +INTEL_PLD_CONSTRAINT (0x1cd ,0xff ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x1d0 ,0xf ),/* MEM_INST_RETIRED.LOAD */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x2d0 ,0xf ),/* MEM_INST_RETIRED.STORE */ + +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD_RANGE (0xd1 ,0xd4 ,0xf ),/* MEM_LOAD_*_RETIRED.* */ + +INTEL_FLAGS_EVENT_CONSTRAINT (0xd0 ,0xf ),/* MEM_INST_RETIRED.* */ + +/* +* Everything else is handled by PMU_FL_PEBS_ALL, because we +* need the full constraints from the main table. +*/ + +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *c;
@@ -1053,7 +1073,7 @@ void intel_pmu_pebs_enable(struct perf_event *event) cpuc->pebs_enabled |=1ULL << hwc->idx; -if (event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) +if ((event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) && (x86_pmu.version <5 )) cpuc->pebs_enabled |=1ULL << (hwc->idx +32 );else if (event->hw.flags &PERF_X86_EVENT_PEBS_ST ) cpuc->pebs_enabled |=1ULL <<63 ;
@@ -1105,7 +1125,8 @@ void intel_pmu_pebs_disable(struct perf_event *event) cpuc->pebs_enabled &= ~(1ULL << hwc->idx); -if (event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) +if ((event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) && + (x86_pmu.version <5 )) cpuc->pebs_enabled &= ~(1ULL << (hwc->idx +32 ));else if (event->hw.flags &PERF_X86_EVENT_PEBS_ST ) cpuc->pebs_enabled &= ~(1ULL <<63 );
arch/x86/events/perf_event.h +2 -0
@@ -999,6 +999,8 @@ extern struct event_constraint intel_bdw_pebs_event_constraints[];extern struct event_constraint intel_skl_pebs_event_constraints[]; +extern struct event_constraint intel_icl_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_add (struct perf_event *event);
arch/x86/include/asm/intel_ds.h +1 -1
@@ -8,7 +8,7 @@/* The maximal number of PEBS events: */ #define MAX_PEBS_EVENTS 8 -#define MAX_FIXED_PEBS_EVENTS 3 +#define MAX_FIXED_PEBS_EVENTS 4/* * A debug store configuration.
arch/x86/include/asm/perf_event.h +1 -1
@@ -7,7 +7,7 @@*/ #define INTEL_PMC_MAX_GENERIC 32 -#define INTEL_PMC_MAX_FIXED 3 +#define INTEL_PMC_MAX_FIXED 4#define INTEL_PMC_IDX_FIXED 32#define X86_PMC_IDX_MAX 64
PATCH v5 9/12 perf/x86/intel/cstate: Add Icelake support
In reply to [email protected]From: Kan Liang <[email protected]>
Icelake uses the same C-state residency events as Sandy Bridge.
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/intel/cstate.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/events/intel/cstate.c +2 -0
@@ -578,6 +578,8 @@ static const struct x86_cpu_id intel_cstates_match[] __initconst = {X86_CSTATES_MODEL (INTEL_FAM6_ATOM_GOLDMONT_X , glm_cstates),X86_CSTATES_MODEL (INTEL_FAM6_ATOM_GOLDMONT_PLUS , glm_cstates), + +X86_CSTATES_MODEL (INTEL_FAM6_ICELAKE_MOBILE , snb_cstates), { }, };MODULE_DEVICE_TABLE (x86cpu, intel_cstates_match);
Signature · 1 line
2.17.1
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86/intel/cstate: Add Icelake support
perf/x86/intel/cstate: Add Icelake support
Commit-ID: f08c47d1f86c6dc666c7e659d94bf6d4492aa9d7 Gitweb: https://git.kernel.org/tip/f08c47d1f86c6dc666c7e659d94bf6d4492aa9d7 Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:06 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:26:18 +0200
perf/x86/intel/cstate: Add Icelake support
Icelake uses the same C-state residency events as Sandy Bridge.
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/cstate.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/events/intel/cstate.c +2 -0
@@ -578,6 +578,8 @@ static const struct x86_cpu_id intel_cstates_match[] __initconst = {X86_CSTATES_MODEL (INTEL_FAM6_ATOM_GOLDMONT_X , glm_cstates),X86_CSTATES_MODEL (INTEL_FAM6_ATOM_GOLDMONT_PLUS , glm_cstates), + +X86_CSTATES_MODEL (INTEL_FAM6_ICELAKE_MOBILE , snb_cstates), { }, };MODULE_DEVICE_TABLE (x86cpu, intel_cstates_match);
PATCH v5 10/12 perf/x86/intel/rapl: Add Icelake support
In reply to [email protected]From: Kan Liang <[email protected]>
Icelake support the same RAPL counters as Skylake.
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/intel/rapl.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/events/intel/rapl.c +2 -0
@@ -775,6 +775,8 @@ static const struct x86_cpu_id rapl_cpu_match[] __initconst = {X86_RAPL_MODEL_MATCH (INTEL_FAM6_ATOM_GOLDMONT_X , hsw_rapl_init),X86_RAPL_MODEL_MATCH (INTEL_FAM6_ATOM_GOLDMONT_PLUS , hsw_rapl_init), + + X86_RAPL_MODEL_MATCH(INTEL_FAM6_ICELAKE_MOBILE, skl_rapl_init), {}, };
Signature · 1 line
2.17.1
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86/intel/rapl: Add Icelake support
perf/x86/intel/rapl: Add Icelake support
Commit-ID: b3377c3acb9e54cf86efcfe25f2e792bca599ed4 Gitweb: https://git.kernel.org/tip/b3377c3acb9e54cf86efcfe25f2e792bca599ed4 Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:07 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:26:18 +0200
perf/x86/intel/rapl: Add Icelake support
Icelake support the same RAPL counters as Skylake.
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/rapl.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/events/intel/rapl.c +2 -0
@@ -775,6 +775,8 @@ static const struct x86_cpu_id rapl_cpu_match[] __initconst = {X86_RAPL_MODEL_MATCH (INTEL_FAM6_ATOM_GOLDMONT_X , hsw_rapl_init),X86_RAPL_MODEL_MATCH (INTEL_FAM6_ATOM_GOLDMONT_PLUS , hsw_rapl_init), + + X86_RAPL_MODEL_MATCH(INTEL_FAM6_ICELAKE_MOBILE, skl_rapl_init), {}, };
PATCH v5 11/12 perf/x86/msr: Add Icelake support
In reply to [email protected]From: Kan Liang <[email protected]>
Icelake is the same as the existing Skylake parts.
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/msr.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/events/msr.c +1 -0
@@ -89,6 +89,7 @@ static bool test_intel(int idx)case INTEL_FAM6_SKYLAKE_X :case INTEL_FAM6_KABYLAKE_MOBILE :case INTEL_FAM6_KABYLAKE_DESKTOP : +case INTEL_FAM6_ICELAKE_MOBILE :if (idx ==PERF_MSR_SMI || idx ==PERF_MSR_PPERF )return true;break ;
Signature · 1 line
2.17.1
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86/msr: Add Icelake support
perf/x86/msr: Add Icelake support
Commit-ID: cf50d79a8cfe5adae37fec026220b009559bbeed Gitweb: https://git.kernel.org/tip/cf50d79a8cfe5adae37fec026220b009559bbeed Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:08 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:26:19 +0200
perf/x86/msr: Add Icelake support
Icelake is the same as the existing Skylake parts.
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/msr.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/events/msr.c +1 -0
@@ -89,6 +89,7 @@ static bool test_intel(int idx)case INTEL_FAM6_SKYLAKE_X :case INTEL_FAM6_KABYLAKE_MOBILE :case INTEL_FAM6_KABYLAKE_DESKTOP : +case INTEL_FAM6_ICELAKE_MOBILE :if (idx ==PERF_MSR_SMI || idx ==PERF_MSR_PPERF )return true;break ;
PATCH v5 12/12 perf/x86/intel/uncore: Add Intel Icelake uncore support
In reply to [email protected]From: Kan Liang <[email protected]>
Add Intel Icelake uncore support,
- The init code is based on Skylake
- Add new pci id for IMC
- New MSR address for CBOX
- Get CBOX# from CNL_UNC_CBO_CONFIG MSR directly
- Create a new PMU for fixed clocktick counter
Signed-off-by Kan Liang <[email protected]>
---
No changes since V4.
arch/x86/events/intel/uncore.c | 6 ++ arch/x86/events/intel/uncore.h | 1 + arch/x86/events/intel/uncore_snb.c | 91 ++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+)
arch/x86/events/intel/uncore.c +6 -0
@@ -1367,6 +1367,11 @@ static const struct intel_uncore_init_fun skx_uncore_init __initconst = { .pci_init = skx_uncore_pci_init, }; +static const struct intel_uncore_init_fun icl_uncore_init__initconst = { + .cpu_init = icl_uncore_cpu_init, + .pci_init = skl_uncore_pci_init, +}; +static const struct x86_cpu_id intel_uncore_match[]__initconst = {X86_UNCORE_MODEL_MATCH (INTEL_FAM6_NEHALEM_EP , nhm_uncore_init),X86_UNCORE_MODEL_MATCH (INTEL_FAM6_NEHALEM , nhm_uncore_init),
@@ -1393,6 +1398,7 @@ static const struct x86_cpu_id intel_uncore_match[] __initconst = {X86_UNCORE_MODEL_MATCH (INTEL_FAM6_SKYLAKE_X , skx_uncore_init),X86_UNCORE_MODEL_MATCH (INTEL_FAM6_KABYLAKE_MOBILE , skl_uncore_init),X86_UNCORE_MODEL_MATCH (INTEL_FAM6_KABYLAKE_DESKTOP , skl_uncore_init), + X86_UNCORE_MODEL_MATCH(INTEL_FAM6_ICELAKE_MOBILE, icl_uncore_init), {}, };
arch/x86/events/intel/uncore.h +1 -0
@@ -512,6 +512,7 @@ int skl_uncore_pci_init(void);void snb_uncore_cpu_init (void );void nhm_uncore_cpu_init (void );void skl_uncore_cpu_init (void ); +void icl_uncore_cpu_init (void );int snb_pci2phy_map_init (int devid);/* uncore_snbep.c */
arch/x86/events/intel/uncore_snb.c +91 -0
127 lines not on this page · raw message
Signature · 1 line
2.17.1
tip-bot for Kan Liang
merge bot[tip:perf/core] perf/x86/intel/uncore: Add Intel Icelake uncore support
perf/x86/intel/uncore: Add Intel Icelake uncore support
Commit-ID: 6e394376ee89233508fa21d006546357f8efee31 Gitweb: https://git.kernel.org/tip/6e394376ee89233508fa21d006546357f8efee31 Author: Kan Liang <[email protected]> AuthorDate: Tue, 2 Apr 2019 12:45:09 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 16 Apr 2019 12:26:19 +0200
perf/x86/intel/uncore: Add Intel Icelake uncore support
Add Intel Icelake uncore support:
- The init code is based on Skylake
- Add new PCI id for IMC
- New MSR address for CBOX
- Get CBOX# from CNL_UNC_CBO_CONFIG MSR directly
- Create a new PMU for fixed clocktick counter
Signed-off-by Kan Liang <[email protected]>
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/events/intel/uncore.c | 6 +++ arch/x86/events/intel/uncore.h | 1 + arch/x86/events/intel/uncore_snb.c | 91 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+)
arch/x86/events/intel/uncore.c +6 -0
@@ -1367,6 +1367,11 @@ static const struct intel_uncore_init_fun skx_uncore_init __initconst = { .pci_init = skx_uncore_pci_init, }; +static const struct intel_uncore_init_fun icl_uncore_init__initconst = { + .cpu_init = icl_uncore_cpu_init, + .pci_init = skl_uncore_pci_init, +}; +static const struct x86_cpu_id intel_uncore_match[]__initconst = {X86_UNCORE_MODEL_MATCH (INTEL_FAM6_NEHALEM_EP , nhm_uncore_init),X86_UNCORE_MODEL_MATCH (INTEL_FAM6_NEHALEM , nhm_uncore_init),
@@ -1393,6 +1398,7 @@ static const struct x86_cpu_id intel_uncore_match[] __initconst = {X86_UNCORE_MODEL_MATCH (INTEL_FAM6_SKYLAKE_X , skx_uncore_init),X86_UNCORE_MODEL_MATCH (INTEL_FAM6_KABYLAKE_MOBILE , skl_uncore_init),X86_UNCORE_MODEL_MATCH (INTEL_FAM6_KABYLAKE_DESKTOP , skl_uncore_init), + X86_UNCORE_MODEL_MATCH(INTEL_FAM6_ICELAKE_MOBILE, icl_uncore_init), {}, };
arch/x86/events/intel/uncore.h +1 -0
@@ -512,6 +512,7 @@ int skl_uncore_pci_init(void);void snb_uncore_cpu_init (void );void nhm_uncore_cpu_init (void );void skl_uncore_cpu_init (void ); +void icl_uncore_cpu_init (void );int snb_pci2phy_map_init (int devid);/* uncore_snbep.c */
arch/x86/events/intel/uncore_snb.c +91 -0
127 lines not on this page · raw message
Peter Zijlstra
maintainerI currently have something like the below on top, is that correct?
If so, I'll fold it back in.
arch/x86/events/core.c +4 -3
@@ -563,16 +563,17 @@ int x86_pmu_hw_config(struct perf_event/* sample_regs_user never support XMM registers */ if (unlikely (event->attr.sample_regs_user &PEBS_XMM_REGS ))return -EINVAL ; +/* * Besides the general purpose registers, XMM registers may * be collected in PEBS on some platforms, e.g. Icelake */ if (unlikely (event->attr.sample_regs_intr &PEBS_XMM_REGS )) { -if (!is_sampling_event (event) || - !event->attr.precise_ip || - x86_pmu.pebs_no_xmm_regs) +if (x86_pmu.pebs_no_xmm_regs)return -EINVAL ; +if (!event->attr.precise_ip) +return -EINVAL ; }return x86_setup_perfctr (event);
arch/x86/events/intel/core.c +2 -2
@@ -3428,7 +3428,7 @@ icl_get_event_constraints(struct cpu_hw_* Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip ==3 ) && - ((event->hw.config &X86_RAW_EVENT_MASK ) ==0x00c0 )) + (event->hw.config == X86_CONFIG(.event=0xc0)))return &fixed_counter0_constraint;return hsw_get_event_constraints (cpuc, idx, event);
@@ -4810,7 +4810,7 @@ __init int intel_pmu_init(void) hsw_format_attr : nhm_format_attr; extra_attr =merge_attr (extra_attr, skl_format_attr); x86_pmu.cpu_events =get_icl_events_attrs (); - x86_pmu.force_gpr_event =0x2ca ; + x86_pmu.rtm_abort_event = X86_CONFIG(.event=0xca, .umask=0x02); x86_pmu.lbr_pt_coexist = true;intel_pmu_pebs_data_source_skl (false);pr_cont ("Icelake events, " );
arch/x86/events/intel/ds.c +46 -52
172 lines not on this page · raw message
arch/x86/events/perf_event.h +4 -4
@@ -303,8 +303,8 @@ struct cpu_hw_events {__EVENT_CONSTRAINT (c, n, m,HWEIGHT (n),0 ,0 )/* -* Only works for Intel events, which has 'small' event codes. -* Need to fix the rang compare for 'big' event codes, e.g AMD64_EVENTSEL_EVENT +* The constraint_match() function only works for 'simple' event codes +* and not for extended (AMD64_EVENTSEL_EVENT) events codes. */#define EVENT_CONSTRAINT_RANGE (c, e, n, m) \ __EVENT_CONSTRAINT_RANGE(c, e, n, m, HWEIGHT(n), 0, 0)
@@ -672,12 +672,12 @@ struct x86_pmu { pebs_no_xmm_regs :1 ;int pebs_record_size;int pebs_buffer_size; +int max_pebs_events;void (*drain_pebs)(struct pt_regs *regs);struct event_constraint *pebs_constraints;void (*pebs_aliases)(struct perf_event *event); -int max_pebs_events;unsigned long large_pebs_flags; -u64 force_gpr_event; +u64 rtm_abort_event;/* * Intel LBR
Quoted 2 lines · Peter Zijlstra
I currently have something like the below on top, is that correct?
Yes, it's correct.
Quoted 2 lines
If so, I'll fold it back in.
Thanks. It's really appreciated.
Kan
Quoted 259 lines
arch/x86/events/core.c +4
-3@@ -563,16 +563,17 @@ int x86_pmu_hw_config(struct perf_event/* sample_regs_user never support XMM registers */ if (unlikely (event->attr.sample_regs_user &PEBS_XMM_REGS ))return -EINVAL ; +/* * Besides the general purpose registers, XMM registers may * be collected in PEBS on some platforms, e.g. Icelake */ if (unlikely (event->attr.sample_regs_intr &PEBS_XMM_REGS )) { -if (!is_sampling_event (event) || - !event->attr.precise_ip || - x86_pmu.pebs_no_xmm_regs) +if (x86_pmu.pebs_no_xmm_regs)return -EINVAL ; +if (!event->attr.precise_ip) +return -EINVAL ; }return x86_setup_perfctr (event);arch/x86/events/intel/core.c +2
-2@@ -3428,7 +3428,7 @@ icl_get_event_constraints(struct cpu_hw_* Force instruction:ppp in Fixed counter 0 */ if ((event->attr.precise_ip ==3 ) && - ((event->hw.config &X86_RAW_EVENT_MASK ) ==0x00c0 )) + (event->hw.config == X86_CONFIG(.event=0xc0)))return &fixed_counter0_constraint;return hsw_get_event_constraints (cpuc, idx, event);@@ -4810,7 +4810,7 @@ __init int intel_pmu_init(void) hsw_format_attr : nhm_format_attr; extra_attr =merge_attr (extra_attr, skl_format_attr); x86_pmu.cpu_events =get_icl_events_attrs (); - x86_pmu.force_gpr_event =0x2ca ; + x86_pmu.rtm_abort_event = X86_CONFIG(.event=0xca, .umask=0x02); x86_pmu.lbr_pt_coexist = true;intel_pmu_pebs_data_source_skl (false);pr_cont ("Icelake events, " );arch/x86/events/intel/ds.c +46
-52@@ -853,13 +853,13 @@ struct event_constraint intel_icl_pebs_eINTEL_FLAGS_UEVENT_CONSTRAINT (0x1c0 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x400000000ULL ),/* SLOTS */ -INTEL_PLD_CONSTRAINT (0x1cd ,0xff ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ -INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x1d0 ,0xf ),/* MEM_INST_RETIRED.LOAD */ -INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x2d0 ,0xf ),/* MEM_INST_RETIRED.STORE */ +INTEL_PLD_CONSTRAINT (0x1cd ,0xff ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x1d0 ,0xf ),/* MEM_INST_RETIRED.LOAD */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x2d0 ,0xf ),/* MEM_INST_RETIRED.STORE */ INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD_RANGE (0xd1 ,0xd4 ,0xf ),/* MEM_LOAD_*_RETIRED.* */ -INTEL_FLAGS_EVENT_CONSTRAINT (0xd0 ,0xf ),/* MEM_INST_RETIRED.* */ +INTEL_FLAGS_EVENT_CONSTRAINT (0xd0 ,0xf ),/* MEM_INST_RETIRED.* */ /* * Everything else is handled by PMU_FL_PEBS_ALL, because we @@ -963,40 +963,42 @@ static u64 pebs_update_adaptive_cfg(struu64 pebs_data_cfg =0 ;bool gprs, tsx_weight; -if ((sample_type & ~(PERF_SAMPLE_IP |PERF_SAMPLE_TIME )) || - attr->precise_ip <2 ) { +if (!(sample_type & ~(PERF_SAMPLE_IP |PERF_SAMPLE_TIME )) && + attr->precise_ip >1 ) +return pebs_data_cfs; -if (sample_type &PERF_PEBS_MEMINFO_TYPE ) - pebs_data_cfg |=PEBS_DATACFG_MEMINFO ; +if (sample_type &PERF_PEBS_MEMINFO_TYPE ) + pebs_data_cfg |=PEBS_DATACFG_MEMINFO ; +/* +* We need GPRs when: +* + user requested them +* + precise_ip < 2 for the non event IP +* + For RTM TSX weight we need GPRs for the abort code. +*/ + gprs = (sample_type &PERF_SAMPLE_REGS_INTR ) && + (attr->sample_regs_intr &PEBS_GPRS_REGS ); + + tsx_weight = (sample_type &PERF_SAMPLE_WEIGHT ) && + ((attr->config &INTEL_ARCH_EVENT_MASK ) == + x86_pmu.rtm_abort_event); + +if (gprs || (attr->precise_ip <2 ) || tsx_weight) + pebs_data_cfg |=PEBS_DATACFG_GPRS ; + +if ((sample_type &PERF_SAMPLE_REGS_INTR ) && + (attr->sample_regs_intr &PERF_XMM_REGS )) + pebs_data_cfg |=PEBS_DATACFG_XMMS ; + +if (sample_type &PERF_SAMPLE_BRANCH_STACK ) {/* -* Cases we need the registers: -* + user requested registers -* + precise_ip < 2 for the non event IP -* + For RTM TSX weight we need GPRs too for the abort -* code. But we don't want to force GPRs for all other -* weights. So add it only collectfor the RTM abort event. +* For now always log all LBRs. Could configure this +* later. */ - gprs = (sample_type &PERF_SAMPLE_REGS_INTR ) && - (attr->sample_regs_intr &0xffffffff ); - tsx_weight = (sample_type &PERF_SAMPLE_WEIGHT ) && - ((attr->config &0xffff ) == x86_pmu.force_gpr_event); -if (gprs || (attr->precise_ip <2 ) || tsx_weight) - pebs_data_cfg |=PEBS_DATACFG_GPRS ; - -if ((sample_type &PERF_SAMPLE_REGS_INTR ) && - (attr->sample_regs_intr >>32 )) - pebs_data_cfg |=PEBS_DATACFG_XMMS ; - -if (sample_type &PERF_SAMPLE_BRANCH_STACK ) { -/* -* For now always log all LBRs. Could configure this -* later. -*/ - pebs_data_cfg |=PEBS_DATACFG_LBRS | - ((x86_pmu.lbr_nr-1 ) <<PEBS_DATACFG_LBR_SHIFT ); - } + pebs_data_cfg |=PEBS_DATACFG_LBRS | + ((x86_pmu.lbr_nr-1 ) <<PEBS_DATACFG_LBR_SHIFT ); } +return pebs_data_cfg; }@@ -1022,13 +1024,8 @@ pebs_update_state(bool needed_cb, struct }/* -* The PEBS record doesn't shrink on the del. Because to get -* an accurate config needs to go through all the existing pebs events. -* It's not necessary. -* There is no harmful for a bigger PEBS record, except little -* performance impacts. -* Also, for most cases, the same pebs config is applied for all -* pebs events. +* The PEBS record doesn't shrink on pmu::del(). Doing so would require +* iterating all remaining PEBS events to reconstruct the config. */ if (x86_pmu.intel_cap.pebs_baseline && add) {u64 pebs_data_cfg;@@ -1076,8 +1073,7 @@ void intel_pmu_pebs_enable(struct perf_e cpuc->pebs_enabled |=1ULL << hwc->idx; -if ((event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) && - (x86_pmu.version <5 )) +if ((event->hw.flags &PERF_X86_EVENT_PEBS_LDLAT ) && (x86_pmu.version <5 )) cpuc->pebs_enabled |=1ULL << (hwc->idx +32 );else if (event->hw.flags &PERF_X86_EVENT_PEBS_ST ) cpuc->pebs_enabled |=1ULL <<63 ;@@ -1766,8 +1762,7 @@ static void intel_pmu_drain_pebs_core(st setup_pebs_fixed_sample_data); } -static void intel_pmu_pebs_event_update_no_drain (struct cpu_hw_events *cpuc, -int size) +static void intel_pmu_pebs_event_update_no_drain (struct cpu_hw_events *cpuc,int size) {struct perf_event *event;int bit;@@ -1826,8 +1821,7 @@ static void intel_pmu_drain_pebs_nhm(str/* PEBS v3 has more accurate status bits */ if (x86_pmu.intel_cap.pebs_format >=3 ) { -for_each_set_bit (bit, (unsigned long *)&pebs_status, - size) +for_each_set_bit (bit, (unsigned long *)&pebs_status, size) counts[bit]++;continue ;@@ -1866,8 +1860,7 @@ static void intel_pmu_drain_pebs_nhm(str* If collision happened, the record will be dropped. */ if (p->status != (1ULL << bit)) { -for_each_set_bit (i, (unsigned long *)&pebs_status, - x86_pmu.max_pebs_events) +for_each_set_bit (i, (unsigned long *)&pebs_status, size) error[i]++;continue ; }@@ -1875,7 +1868,7 @@ static void intel_pmu_drain_pebs_nhm(str counts[bit]++; } -for (bit =0 ; bit < size; bit++) { +for_each_set_bit (bit, (unsigned long *)&mask, size) {if ((counts[bit] ==0 ) && (error[bit] ==0 ))continue ;@@ -1939,7 +1932,7 @@ static void intel_pmu_drain_pebs_icl(str counts[bit]++; } -for (bit =0 ; bit < size; bit++) { +for_each_set_bit (bit, (unsigned long *)mask, size) {if (counts[bit] ==0 )continue ;@@ -1980,6 +1973,9 @@ void __init intel_ds_init(void)char *pebs_qual ="" ;int format = x86_pmu.intel_cap.pebs_format; +if (format <4 ) + x86_pmu.intel_cap.pebs_baseline =0 ; +switch (format) {case 0 :pr_cont ("PEBS fmt0%c, " , pebs_type);@@ -2042,8 +2038,6 @@ void __init intel_ds_init(void)pr_cont ("no PEBS fmt%d%c, " , format, pebs_type); x86_pmu.pebs =0 ; } -if (format !=4 ) - x86_pmu.intel_cap.pebs_baseline =0 ; } }arch/x86/events/perf_event.h +4
-4@@ -303,8 +303,8 @@ struct cpu_hw_events {__EVENT_CONSTRAINT (c, n, m,HWEIGHT (n),0 ,0 )/* -* Only works for Intel events, which has 'small' event codes. -* Need to fix the rang compare for 'big' event codes, e.g AMD64_EVENTSEL_EVENT +* The constraint_match() function only works for 'simple' event codes +* and not for extended (AMD64_EVENTSEL_EVENT) events codes. */#define EVENT_CONSTRAINT_RANGE (c, e, n, m) \ __EVENT_CONSTRAINT_RANGE(c, e, n, m, HWEIGHT(n), 0, 0)@@ -672,12 +672,12 @@ struct x86_pmu { pebs_no_xmm_regs :1 ;int pebs_record_size;int pebs_buffer_size; +int max_pebs_events;void (*drain_pebs)(struct pt_regs *regs);struct event_constraint *pebs_constraints;void (*pebs_aliases)(struct perf_event *event); -int max_pebs_events;unsigned long large_pebs_flags; -u64 force_gpr_event; +u64 rtm_abort_event;/* * Intel LBR
Quoted 9 lines
arch/x86/events/intel/ds.c static u64 pebs_update_adaptive_cfg(stru · line 967
Quoted 9 lines
arch/x86/events/intel/ds.c +3
-2@@ -963,40 +963,42 @@ static u64 pebs_update_adaptive_cfg(stru u64 pebs_data_cfg = 0; bool gprs, tsx_weight; - if ((sample_type & ~(PERF_SAMPLE_IP|PERF_SAMPLE_TIME)) || - attr->precise_ip < 2) { + if (!(sample_type & ~(PERF_SAMPLE_IP|PERF_SAMPLE_TIME)) && + attr->precise_ip > 1) + return pebs_data_cfs;
Sorry, there are two typos. I didn't find in the previous email. Should be pebs_data_cfg.
Quoted 14 lines
Quoted 14 lines
arch/x86/events/intel/ds.c +12
-2- if (sample_type & PERF_PEBS_MEMINFO_TYPE) - pebs_data_cfg |= PEBS_DATACFG_MEMINFO; + if (sample_type & PERF_PEBS_MEMINFO_TYPE) + pebs_data_cfg |= PEBS_DATACFG_MEMINFO; + /* + * We need GPRs when: + * + user requested them + * + precise_ip < 2 for the non event IP + * + For RTM TSX weight we need GPRs for the abort code. + */ + gprs = (sample_type & PERF_SAMPLE_REGS_INTR) && + (attr->sample_regs_intr & PEBS_GPRS_REGS); + + tsx_weight = (sample_type & PERF_SAMPLE_WEIGHT) &&
Quoted 8 lines
arch/x86/events/intel/ds.c static u64 pebs_update_adaptive_cfg(stru · line 990
Quoted 8 lines
arch/x86/events/intel/ds.c +8
-0+ ((attr->config & INTEL_ARCH_EVENT_MASK) == + x86_pmu.rtm_abort_event); + + if (gprs || (attr->precise_ip < 2) || tsx_weight) + pebs_data_cfg |= PEBS_DATACFG_GPRS; + + if ((sample_type & PERF_SAMPLE_REGS_INTR) && + (attr->sample_regs_intr & PERF_XMM_REGS))
Should be PEBS_XMM_REGS.
Thanks,
Kan
Peter Zijlstra
maintainerQuoted 29 lines · Kan
Quoted 9 lines
arch/x86/events/intel/ds.c static u64 pebs_update_adaptive_cfg(stru · line 967
Quoted 9 lines
arch/x86/events/intel/ds.c +3
-2@@ -963,40 +963,42 @@ static u64 pebs_update_adaptive_cfg(stru u64 pebs_data_cfg = 0; bool gprs, tsx_weight; - if ((sample_type & ~(PERF_SAMPLE_IP|PERF_SAMPLE_TIME)) || - attr->precise_ip < 2) { + if (!(sample_type & ~(PERF_SAMPLE_IP|PERF_SAMPLE_TIME)) && + attr->precise_ip > 1) + return pebs_data_cfs;Sorry, there are two typos. I didn't find in the previous email. Should be
pebs_data_cfg.Quoted 16 lines
Quoted 16 lines
arch/x86/events/intel/ds.c +14
-2- if (sample_type & PERF_PEBS_MEMINFO_TYPE) - pebs_data_cfg |= PEBS_DATACFG_MEMINFO; + if (sample_type & PERF_PEBS_MEMINFO_TYPE) + pebs_data_cfg |= PEBS_DATACFG_MEMINFO; + /* + * We need GPRs when: + * + user requested them + * + precise_ip < 2 for the non event IP + * + For RTM TSX weight we need GPRs for the abort code. + */ + gprs = (sample_type & PERF_SAMPLE_REGS_INTR) && + (attr->sample_regs_intr & PEBS_GPRS_REGS); + + tsx_weight = (sample_type & PERF_SAMPLE_WEIGHT) && + ((attr->config & INTEL_ARCH_EVENT_MASK) == + x86_pmu.rtm_abort_event);
Quoted 9 lines
Quoted 6 lines
arch/x86/events/intel/ds.c static u64 pebs_update_adaptive_cfg(stru · line 990
Quoted 6 lines
arch/x86/events/intel/ds.c +6
-0+ + if (gprs || (attr->precise_ip < 2) || tsx_weight) + pebs_data_cfg |= PEBS_DATACFG_GPRS; + + if ((sample_type & PERF_SAMPLE_REGS_INTR) && + (attr->sample_regs_intr & PERF_XMM_REGS))Should be
PEBS_XMM_REGS.
Ha!, clearly I compile-tested it... oh wait :-)
Quoted 8 lines · Kan
arch/x86/events/intel/ds.c +1
-1@@ -1875,7 +1868,7 @@ static void intel_pmu_drain_pebs_nhm(str counts[bit]++; } - for (bit = 0; bit < size; bit++) { + for_each_set_bit(bit, (unsigned long *)&mask, size) { if ((counts[bit] == 0) && (error[bit] == 0)) continue;
arch/x86/events/intel/ds.c static void intel_pmu_drain_pebs_icl(str · line 1936
Quoted 6 lines
arch/x86/events/intel/ds.c +1
-1@@ -1939,7 +1932,7 @@ static void intel_pmu_drain_pebs_icl(str counts[bit]++; } - for (bit = 0; bit < size; bit++) { + for_each_set_bit(bit, (unsigned long *)mask, size) { if (counts[bit] == 0) continue;
I have finished the tests for the changes. There is one more regression found by another typo on ICL.
Should be "for_each_set_bit(bit, (unsigned long *)&mask, size) {"
Thanks,
Kan