perf mem/c2c: Add support for AMD
Subsystems (2)
v3 cover perf mem/c2c: Add support for AMD
Perf mem and c2c tools are wrappers around perf record with mem load/ store events. IBS tagged load/store sample provides most of the information needed for these tools. Enable support for these tools on AMD Zen processors based on IBS Op pmu.
There are some limitations though: Only load/store micro-ops provide mem/c2c information. Whereas, IBS does not have a way to choose a particular type of micro-op to tag. This results in many non-LS micro-ops being tagged which appear as N/A in the perf report. IBS, being an uncore pmu from kernel point of view[1], does not support per process monitoring. Thus, perf mem/c2c on AMD are currently supported in per-cpu mode only.
Example:
$ sudo ./perf mem record -- -c 10000 ^C[ perf record: Woken up 227 times to write data ] [ perf record: Captured and wrote 58.760 MB perf.data (836978 samples) ]
$ sudo ./perf mem report -F mem,sample,snoop Samples: 836K of event 'ibs_op//', Event count (approx.): 8418762 Memory access Samples Snoop N/A 700620 N/A L1 hit 126675 N/A L2 hit 424 N/A L3 hit 664 HitM L3 hit 10 N/A Local RAM hit 2 N/A Remote RAM (1 hop) hit 8558 N/A Remote Cache (1 hop) hit 3 N/A Remote Cache (1 hop) hit 2 HitM Remote Cache (2 hops) hit 10 HitM Remote Cache (2 hops) hit 6 N/A Uncached hit 4 N/A
Prepared on queue/perf/core (cce6a2d7e0e49).
v2: https://lore.kernel.org/all/[email protected]v2->v3:
- Use sample_flags instead of __PERF_SAMPLE_*_EARLY varients
- Make PERF_SAMPLE_WEIGHT independent of PERF_SAMPLE_DATA_SRC
- Add a patch to reverse sync PERF_MEM_SNOOPX_PEER from tools to kernel uapi header
- Add Acked-by: Jiri Olsa for tool side unchanged patches
Also, a recent patch[2] to test perf mem fails on AMD because of aforementioned limitations.
[1]: https://lore.kernel.org/lkml/[email protected]
[2]: https://lore.kernel.org/lkml/20220924133408.1125903-1-leo.yan%40linaro.org
Ravi Bangoria (15):
perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO} perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions perf/x86/amd: Support PERF_SAMPLE_DATA_SRC perf/x86/amd: Support PERF_SAMPLE_{WEIGHT|WEIGHT_STRUCT} perf/x86/amd: Support PERF_SAMPLE_ADDR
perf/x86/amd: Support PERF_SAMPLE_PHY_ADDR
perf/uapi: Define PERF_MEM_SNOOPX_PEER in kernel header file perf tool: Sync include/uapi/linux/perf_event.h header perf tool: Sync arch/x86/include/asm/amd-ibs.h header perf mem: Add support for printing PERF_MEM_LVLNUM_{EXTN_MEM|IO} perf mem/c2c: Set PERF_SAMPLE_WEIGHT for LOAD_STORE events perf mem/c2c: Add load store event mappings for AMD perf mem/c2c: Avoid printing empty lines for unsupported events perf mem: Use more generic term for LFB
perf script: Add missing fields in usage hint
arch/x86/events/amd/ibs.c | 345 ++++++++++++++++++++++- arch/x86/include/asm/amd-ibs.h | 16 ++ include/uapi/linux/perf_event.h | 6 +- kernel/events/core.c | 3 +- tools/arch/x86/include/asm/amd-ibs.h | 16 ++ tools/include/uapi/linux/perf_event.h | 4 +- tools/perf/Documentation/perf-c2c.txt | 14 +- tools/perf/Documentation/perf-mem.txt | 3 +- tools/perf/Documentation/perf-record.txt | 1 + tools/perf/arch/x86/util/mem-events.c | 31 +- tools/perf/builtin-c2c.c | 1 + tools/perf/builtin-mem.c | 1 + tools/perf/builtin-script.c | 7 +- tools/perf/util/mem-events.c | 17 +- 14 files changed, 438 insertions(+), 27 deletions(-)
Signature · 1 line
2.31.1
PATCH v3 1/15 perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
PERF_MEM_LVLNUM_EXTN_MEM which can be used to indicate accesses to extension memory like CXL etc. PERF_MEM_LVL_IO can be used for IO accesses but it can not distinguish between local and remote IO. Introduce new field PERF_MEM_LVLNUM_IO which can be clubbed with PERF_MEM_REMOTE_REMOTE to indicate Remote IO accesses.
--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/uapi/linux/perf_event.h +3 -1
@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ +#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ +#define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */ #define PERF_MEM_LVLNUM_RAM 0x0d/* RAM */
Signature · 1 line
2.31.1
[tip: perf/core] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: ee3e88dfec23153d0675b5d00522297b9adf657c Gitweb: https://git.kernel.org/tip/ee3e88dfec23153d0675b5d00522297b9adf657c Author: Ravi Bangoria <[email protected]> AuthorDate: Wed, 28 Sep 2022 15:27:51 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 29 Sep 2022 12:20:54 +02:00
perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
PERF_MEM_LVLNUM_EXTN_MEM which can be used to indicate accesses to extension memory like CXL etc. PERF_MEM_LVL_IO can be used for IO accesses but it can not distinguish between local and remote IO. Introduce new field PERF_MEM_LVLNUM_IO which can be clubbed with PERF_MEM_REMOTE_REMOTE to indicate Remote IO accesses.
--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/uapi/linux/perf_event.h +3 -1
@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ +#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ +#define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */ #define PERF_MEM_LVLNUM_RAM 0x0d/* RAM */
Re: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
In reply to Ravi BangoriaQuoted 16 lines · Ravi Bangoria
PERF_MEM_LVLNUM_EXTN_MEMwhich can be used to indicate accesses to extension memory like CXL etc.PERF_MEM_LVL_IOcan be used for IO accesses but it can not distinguish between local and remote IO. Introduce new fieldPERF_MEM_LVLNUM_IOwhich can be clubbed withPERF_MEM_REMOTE_REMOTEto indicate Remote IO accesses.Signed-off-by Ravi Bangoria <[email protected]>--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)include/uapi/linux/perf_event.h +0
-0
include/uapi/linux/perf_event.h union perf_mem_data_src { · line 1340
Quoted 6 lines
include/uapi/linux/perf_event.h +2
-1@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ +#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */
Hi Ravi,
Here we are adding entry explicitly for accesses to Extension memory
like CXL. In future if we want to extend it for cache or other accesses , we again need to add new entries.
Can we rather add single entry say PERF_MEM_LVLNUM_EXTN and further can use reserved bits to specify memory/cache?
Thanks,
Kajol Jain
Quoted 4 lines
include/uapi/linux/perf_event.h +1
-0+#define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */ #define PERF_MEM_LVLNUM_RAM 0x0d/* RAM */
Re: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
Quoted 24 lines · kajoljain
Quoted 21 lines · Ravi Bangoria
PERF_MEM_LVLNUM_EXTN_MEMwhich can be used to indicate accesses to extension memory like CXL etc.PERF_MEM_LVL_IOcan be used for IO accesses but it can not distinguish between local and remote IO. Introduce new fieldPERF_MEM_LVLNUM_IOwhich can be clubbed withPERF_MEM_REMOTE_REMOTEto indicate Remote IO accesses.Signed-off-by Ravi Bangoria <[email protected]>--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)include/uapi/linux/perf_event.h +1
-1@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */
Quoted 8 lines
include/uapi/linux/perf_event.h union perf_mem_data_src { · line 1340
Quoted 1 line
include/uapi/linux/perf_event.h +1
-0+#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ Hi Ravi,
Here we are adding entry explicitly for accesses to Extension memory
like CXL. In future if we want to extend it for cache or other accesses , we again need to add new entries.
Can we rather add single entry sayPERF_MEM_LVLNUM_EXTNand further can use reserved bits to specify memory/cache?
Is everybody okay with this:
#define PERF_MEM_LVLNUM_EXTN 0x09 /* CXL */
And a 3 bit variable to define what type of cxl that would be:
#define PERF_MEM_EXTN_CXL_ANY 0x1
#define PERF_MEM_EXTN_CXL_MEM 0x2
#define PERF_MEM_EXTN_CXL_CACHE 0x2
#define PERF_MEM_EXTN_CXL_IO 0x3
Peter, Shall I send this as addon patch series or are you okay reverting current patches?
Thanks,
Ravi
Re: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
Quoted 34 lines · Ravi Bangoria
Quoted 24 lines · kajoljain
Quoted 21 lines · Ravi Bangoria
PERF_MEM_LVLNUM_EXTN_MEMwhich can be used to indicate accesses to extension memory like CXL etc.PERF_MEM_LVL_IOcan be used for IO accesses but it can not distinguish between local and remote IO. Introduce new fieldPERF_MEM_LVLNUM_IOwhich can be clubbed withPERF_MEM_REMOTE_REMOTEto indicate Remote IO accesses.Signed-off-by Ravi Bangoria <[email protected]>--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)include/uapi/linux/perf_event.h +1
-1@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ Quoted 8 lines
include/uapi/linux/perf_event.h union perf_mem_data_src { · line 1340
Quoted 1 line
include/uapi/linux/perf_event.h +1
-0+#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ Hi Ravi,
Here we are adding entry explicitly for accesses to Extension memory
like CXL. In future if we want to extend it for cache or other accesses , we again need to add new entries.
Can we rather add single entry sayPERF_MEM_LVLNUM_EXTNand further can use reserved bits to specify memory/cache?
Quoted 3 lines
Is everybody okay with this:
#define
PERF_MEM_LVLNUM_EXTN0x09/* CXL */
I think a generic name, PERF_MEM_LVLNUM_EXTN, only make sense, when it wants to include all the types of the Extension memory, e.g., CXL, PMEM, HBM, etc. Then we can set this bit and the corresponding CXL bits to understand the real source. Is it the case here?
But if it's only for the CXL, I think it's better to use a dedicated name, PERF_MEM_LVLNUM_CXL. (as we did for PMEM, PERF_MEM_LVLNUM_PMEM). If so, I don't think we need the PERF_MEM_EXTN_CXL_ANY.
Thanks,
Kan
Quoted 13 lines
And a 3 bit variable to define what type of cxl that would be:
#define
PERF_MEM_EXTN_CXL_ANY0x1
#definePERF_MEM_EXTN_CXL_MEM0x2
#definePERF_MEM_EXTN_CXL_CACHE0x2
#definePERF_MEM_EXTN_CXL_IO0x3Peter, Shall I send this as addon patch series or are you okay reverting current patches?
Thanks,
Ravi
Re: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
Quoted 41 lines · Kan
Quoted 34 lines · Ravi Bangoria
Quoted 24 lines · kajoljain
Quoted 21 lines · Ravi Bangoria
PERF_MEM_LVLNUM_EXTN_MEMwhich can be used to indicate accesses to extension memory like CXL etc.PERF_MEM_LVL_IOcan be used for IO accesses but it can not distinguish between local and remote IO. Introduce new fieldPERF_MEM_LVLNUM_IOwhich can be clubbed withPERF_MEM_REMOTE_REMOTEto indicate Remote IO accesses.Signed-off-by Ravi Bangoria <[email protected]>--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)include/uapi/linux/perf_event.h +1
-1@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ Quoted 8 lines
include/uapi/linux/perf_event.h union perf_mem_data_src { · line 1340
Quoted 1 line
include/uapi/linux/perf_event.h +1
-0+#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ Hi Ravi,
Here we are adding entry explicitly for accesses to Extension memory
like CXL. In future if we want to extend it for cache or other accesses , we again need to add new entries.
Can we rather add single entry sayPERF_MEM_LVLNUM_EXTNand further can use reserved bits to specify memory/cache?Quoted 3 lines
Is everybody okay with this:
#define
PERF_MEM_LVLNUM_EXTN0x09/* CXL */
Quoted 8 lines
I think a generic name,
PERF_MEM_LVLNUM_EXTN, only make sense, when it wants to include all the types of the Extension memory, e.g., CXL, PMEM, HBM, etc. Then we can set this bit and the corresponding CXL bits to understand the real source. Is it the case here?But if it's only for the CXL, I think it's better to use a dedicated name,
PERF_MEM_LVLNUM_CXL. (as we did for PMEM,PERF_MEM_LVLNUM_PMEM). If so, I don't think we need thePERF_MEM_EXTN_CXL_ANY.
Ok. For now, I think below is good enough? Later we can introduce new variable to provide type of cxl device.
From 5deb2055e2b5b0a61403f2d5f4e5a784b14a65e3 Mon Sep 17 00:00:00 2001
From: Ravi Bangoria <[email protected]>
Date: Sat, 1 Oct 2022 11:37:05 +0530
Subject: [PATCH] perf/mem: Rename PERF_MEM_LVLNUM_EXTN_MEM to
PERF_MEM_LVLNUM_CXL
PERF_MEM_LVLNUM_EXTN_MEM was introduced to cover CXL devices but it's bit ambiguous name and also not generic enough to cover cxl.cache and cxl.io devices. Rename it to PERF_MEM_LVLNUM_CXL to be more specific.
--- arch/x86/events/amd/ibs.c | 2 +- include/uapi/linux/perf_event.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/events/amd/ibs.c +1 -1
@@ -801,7 +801,7 @@ static void perf_ibs_get_mem_lvl(union ibs_op_data2 *op_data2,/* Extension Memory */ if (ibs_caps &IBS_CAPS_ZEN4 && ibs_data_src ==IBS_DATA_SRC_EXT_EXT_MEM ) { - data_src->mem_lvl_num =PERF_MEM_LVLNUM_EXTN_MEM ; + data_src->mem_lvl_num =PERF_MEM_LVLNUM_CXL ;if (op_data2->rmt_node) { data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ;/* IBS doesn't provide Remote socket detail */
include/uapi/linux/perf_event.h +1 -1
@@ -1337,7 +1337,7 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ /* 5-0x8 available */ -#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ +#define PERF_MEM_LVLNUM_CXL 0x09/* CXL */ #define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */
Signature · 1 line
2.31.1
Liang, Kan
Reviewed-byRe: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
Quoted 61 lines · Ravi Bangoria
Quoted 41 lines · Kan
Quoted 34 lines · Ravi Bangoria
Quoted 24 lines · kajoljain
Quoted 21 lines · Ravi Bangoria
PERF_MEM_LVLNUM_EXTN_MEMwhich can be used to indicate accesses to extension memory like CXL etc.PERF_MEM_LVL_IOcan be used for IO accesses but it can not distinguish between local and remote IO. Introduce new fieldPERF_MEM_LVLNUM_IOwhich can be clubbed withPERF_MEM_REMOTE_REMOTEto indicate Remote IO accesses.Signed-off-by Ravi Bangoria <[email protected]>--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)include/uapi/linux/perf_event.h +1
-1@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ Quoted 8 lines
include/uapi/linux/perf_event.h union perf_mem_data_src { · line 1340
Quoted 1 line
include/uapi/linux/perf_event.h +1
-0+#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ Hi Ravi,
Here we are adding entry explicitly for accesses to Extension memory
like CXL. In future if we want to extend it for cache or other accesses , we again need to add new entries.
Can we rather add single entry sayPERF_MEM_LVLNUM_EXTNand further can use reserved bits to specify memory/cache?Quoted 3 lines
Is everybody okay with this:
#define
PERF_MEM_LVLNUM_EXTN0x09/* CXL */Quoted 8 lines
I think a generic name,
PERF_MEM_LVLNUM_EXTN, only make sense, when it wants to include all the types of the Extension memory, e.g., CXL, PMEM, HBM, etc. Then we can set this bit and the corresponding CXL bits to understand the real source. Is it the case here?But if it's only for the CXL, I think it's better to use a dedicated name,
PERF_MEM_LVLNUM_CXL. (as we did for PMEM,PERF_MEM_LVLNUM_PMEM). If so, I don't think we need thePERF_MEM_EXTN_CXL_ANY.Ok. For now, I think below is good enough? Later we can introduce new variable to provide type of cxl device.
From
5deb2055e2b5b0a61403f2d5f4e5a784b14a65e3Mon Sep 17 00:00:00 2001
From: Ravi Bangoria <[email protected]>
Date: Sat, 1 Oct 2022 11:37:05 +0530
Subject: [PATCH] perf/mem: RenamePERF_MEM_LVLNUM_EXTN_MEMto
PERF_MEM_LVLNUM_CXL
Quoted 3 lines
PERF_MEM_LVLNUM_EXTN_MEMwas introduced to cover CXL devices but it's bit ambiguous name and also not generic enough to cover cxl.cache and cxl.io devices. Rename it toPERF_MEM_LVLNUM_CXLto be more specific.
Looks good to me.
Thanks,
Kan
Quoted 33 lines
Signed-off-by Ravi Bangoria <[email protected]>--- arch/x86/events/amd/ibs.c | 2 +- include/uapi/linux/perf_event.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/events/amd/ibs.c +1
-1@@ -801,7 +801,7 @@ static void perf_ibs_get_mem_lvl(union ibs_op_data2 *op_data2,/* Extension Memory */ if (ibs_caps &IBS_CAPS_ZEN4 && ibs_data_src ==IBS_DATA_SRC_EXT_EXT_MEM ) { - data_src->mem_lvl_num =PERF_MEM_LVLNUM_EXTN_MEM ; + data_src->mem_lvl_num =PERF_MEM_LVLNUM_CXL ;if (op_data2->rmt_node) { data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ;/* IBS doesn't provide Remote socket detail */ include/uapi/linux/perf_event.h +1
-1@@ -1337,7 +1337,7 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ /* 5-0x8 available */ -#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ +#define PERF_MEM_LVLNUM_CXL 0x09/* CXL */ #define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */
Re: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
Quoted 64 lines · Kan
Quoted 61 lines · Ravi Bangoria
Quoted 41 lines · Kan
Quoted 34 lines · Ravi Bangoria
Quoted 24 lines · kajoljain
Quoted 21 lines · Ravi Bangoria
PERF_MEM_LVLNUM_EXTN_MEMwhich can be used to indicate accesses to extension memory like CXL etc.PERF_MEM_LVL_IOcan be used for IO accesses but it can not distinguish between local and remote IO. Introduce new fieldPERF_MEM_LVLNUM_IOwhich can be clubbed withPERF_MEM_REMOTE_REMOTEto indicate Remote IO accesses.Signed-off-by Ravi Bangoria <[email protected]>--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)include/uapi/linux/perf_event.h +1
-1@@ -1336,7 +1336,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ Quoted 8 lines
include/uapi/linux/perf_event.h union perf_mem_data_src { · line 1340
Quoted 1 line
include/uapi/linux/perf_event.h +1
-0+#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ Hi Ravi,
Here we are adding entry explicitly for accesses to Extension memory
like CXL. In future if we want to extend it for cache or other accesses , we again need to add new entries.
Can we rather add single entry sayPERF_MEM_LVLNUM_EXTNand further can use reserved bits to specify memory/cache?Quoted 3 lines
Is everybody okay with this:
#define
PERF_MEM_LVLNUM_EXTN0x09/* CXL */Quoted 8 lines
I think a generic name,
PERF_MEM_LVLNUM_EXTN, only make sense, when it wants to include all the types of the Extension memory, e.g., CXL, PMEM, HBM, etc. Then we can set this bit and the corresponding CXL bits to understand the real source. Is it the case here?But if it's only for the CXL, I think it's better to use a dedicated name,
PERF_MEM_LVLNUM_CXL. (as we did for PMEM,PERF_MEM_LVLNUM_PMEM). If so, I don't think we need thePERF_MEM_EXTN_CXL_ANY.Ok. For now, I think below is good enough? Later we can introduce new variable to provide type of cxl device.
From
5deb2055e2b5b0a61403f2d5f4e5a784b14a65e3Mon Sep 17 00:00:00 2001
From: Ravi Bangoria <[email protected]>
Date: Sat, 1 Oct 2022 11:37:05 +0530
Subject: [PATCH] perf/mem: RenamePERF_MEM_LVLNUM_EXTN_MEMto
PERF_MEM_LVLNUM_CXL
Quoted 7 lines
Quoted 3 lines
PERF_MEM_LVLNUM_EXTN_MEMwas introduced to cover CXL devices but it's bit ambiguous name and also not generic enough to cover cxl.cache and cxl.io devices. Rename it toPERF_MEM_LVLNUM_CXLto be more specific.Looks good to me.
Reviewed-by Kan Liang <[email protected]>
Thanks Kan.
Peter, can you please include this patch along with the series?
Arnaldo, I'll respin tool side of patches with this change.
Thanks,
Ravi
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
Em Thu, Oct 06, 2022 at 05:08:55PM +0530, Ravi Bangoria escreveu:
Quoted 7 lines · Kan
Quoted 5 lines
Quoted 3 lines
PERF_MEM_LVLNUM_EXTN_MEMwas introduced to cover CXL devices but it's bit ambiguous name and also not generic enough to cover cxl.cache and cxl.io devices. Rename it toPERF_MEM_LVLNUM_CXLto be more specific.Looks good to me.
Quoted 7 lines
Quoted 1 line
Reviewed-by Kan Liang <[email protected]>Thanks Kan.
Peter, can you please include this patch along with the series?
Arnaldo, I'll respin tool side of patches with this change.
Its already upstream, so please go on from there, ok?
I'm now processing perf patches after getting lost in pahole land for a bit :-)
- Arnaldo
Re: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
Quoted 1 line
Its already upstream, so please go on from there, ok?
Right. Only PERF_MEM_LVLNUM_EXTN_MEM -> PERF_MEM_LVLNUM_CXL kernel side patch is pending. Tool side already uses PERF_MEM_LVLNUM_CXL macro.
Peter, let me know if you want me to resend.
Thanks,
Ravi
Peter Zijlstra
maintainerRe: [PATCH v3 01/15] perf/mem: Introduce PERF_MEM_LVLNUM_{EXTN_MEM|IO}
In reply to Ravi BangoriaQuoted 11 lines · Ravi Bangoria
From
5deb2055e2b5b0a61403f2d5f4e5a784b14a65e3Mon Sep 17 00:00:00 2001
From: Ravi Bangoria <[email protected]>
Date: Sat, 1 Oct 2022 11:37:05 +0530
Subject: [PATCH] perf/mem: RenamePERF_MEM_LVLNUM_EXTN_MEMto
PERF_MEM_LVLNUM_CXL
PERF_MEM_LVLNUM_EXTN_MEMwas introduced to cover CXL devices but it's bit ambiguous name and also not generic enough to cover cxl.cache and cxl.io devices. Rename it toPERF_MEM_LVLNUM_CXLto be more specific.Signed-off-by Ravi Bangoria <[email protected]>
Thanks!
Quoted 33 lines
--- arch/x86/events/amd/ibs.c | 2 +- include/uapi/linux/perf_event.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/events/amd/ibs.c +1
-1@@ -801,7 +801,7 @@ static void perf_ibs_get_mem_lvl(union ibs_op_data2 *op_data2,/* Extension Memory */ if (ibs_caps &IBS_CAPS_ZEN4 && ibs_data_src ==IBS_DATA_SRC_EXT_EXT_MEM ) { - data_src->mem_lvl_num =PERF_MEM_LVLNUM_EXTN_MEM ; + data_src->mem_lvl_num =PERF_MEM_LVLNUM_CXL ;if (op_data2->rmt_node) { data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ;/* IBS doesn't provide Remote socket detail */ include/uapi/linux/perf_event.h +1
-1@@ -1337,7 +1337,7 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ /* 5-0x8 available */ -#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ +#define PERF_MEM_LVLNUM_CXL 0x09/* CXL */ #define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */ Signature · 1 line
2.31.1
[tip: perf/urgent] perf/mem: Rename PERF_MEM_LVLNUM_EXTN_MEM to PERF_MEM_LVLNUM_CXL
In reply to Ravi BangoriaThe following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: cb6c18b5a41622c7a439508f7421f8766a91cb87 Gitweb: https://git.kernel.org/tip/cb6c18b5a41622c7a439508f7421f8766a91cb87 Author: Ravi Bangoria <[email protected]> AuthorDate: Sat, 01 Oct 2022 11:37:05 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 27 Oct 2022 10:27:32 +02:00
perf/mem: Rename PERF_MEM_LVLNUM_EXTN_MEM to PERF_MEM_LVLNUM_CXL
PERF_MEM_LVLNUM_EXTN_MEM was introduced to cover CXL devices but it's bit ambiguous name and also not generic enough to cover cxl.cache and cxl.io devices. Rename it to PERF_MEM_LVLNUM_CXL to be more specific.
--- arch/x86/events/amd/ibs.c | 2 +- include/uapi/linux/perf_event.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/events/amd/ibs.c +1 -1
@@ -801,7 +801,7 @@ static void perf_ibs_get_mem_lvl(union ibs_op_data2 *op_data2,/* Extension Memory */ if (ibs_caps &IBS_CAPS_ZEN4 && ibs_data_src ==IBS_DATA_SRC_EXT_EXT_MEM ) { - data_src->mem_lvl_num =PERF_MEM_LVLNUM_EXTN_MEM ; + data_src->mem_lvl_num =PERF_MEM_LVLNUM_CXL ;if (op_data2->rmt_node) { data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ;/* IBS doesn't provide Remote socket detail */
include/uapi/linux/perf_event.h +1 -1
@@ -1337,7 +1337,7 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ /* 5-0x8 available */ -#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ +#define PERF_MEM_LVLNUM_CXL 0x09/* CXL */ #define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */
PATCH v3 2/15 perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions
In reply to Ravi BangoriaIBS_OP_DATA2 DataSrc provides detail about location of the data being accessed from by load ops. Define macros for legacy and extended DataSrc values.
--- arch/x86/include/asm/amd-ibs.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/include/asm/amd-ibs.h +16 -0
@@ -6,6 +6,22 @@#include <asm/msr-index.h> +/* IBS_OP_DATA2 DataSrc */ +#define IBS_DATA_SRC_LOC_CACHE 2 +#define IBS_DATA_SRC_DRAM 3 +#define IBS_DATA_SRC_REM_CACHE 4 +#define IBS_DATA_SRC_IO 7 + +/* IBS_OP_DATA2 DataSrc Extension */ +#define IBS_DATA_SRC_EXT_LOC_CACHE 1 +#define IBS_DATA_SRC_EXT_NEAR_CCX_CACHE 2 +#define IBS_DATA_SRC_EXT_DRAM 3 +#define IBS_DATA_SRC_EXT_FAR_CCX_CACHE 5 +#define IBS_DATA_SRC_EXT_PMEM 6 +#define IBS_DATA_SRC_EXT_IO 7 +#define IBS_DATA_SRC_EXT_EXT_MEM 8 +#define IBS_DATA_SRC_EXT_PEER_AGENT_MEM 12 +/* * IBS Hardware MSRs */
Signature · 1 line
2.31.1
Namhyung Kim
maintainerRe: [PATCH v3 02/15] perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions
Hi Ravi,
Quoted 19 lines · Ravi Bangoria
IBS_OP_DATA2DataSrc provides detail about location of the data being accessed from by load ops. Define macros for legacy and extended DataSrc values.Signed-off-by Ravi Bangoria <[email protected]>--- arch/x86/include/asm/amd-ibs.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)arch/x86/include/asm/amd-ibs.h +1
-0@@ -6,6 +6,22 @@#include <asm/msr-index.h> +/* IBS_OP_DATA2 DataSrc */
arch/x86/include/asm/amd-ibs.h line 19
Quoted 10 lines
arch/x86/include/asm/amd-ibs.h +10
-0+#define IBS_DATA_SRC_LOC_CACHE 2 +#define IBS_DATA_SRC_DRAM 3 +#define IBS_DATA_SRC_REM_CACHE 4 +#define IBS_DATA_SRC_IO 7 + +/* IBS_OP_DATA2 DataSrc Extension */ +#define IBS_DATA_SRC_EXT_LOC_CACHE 1 +#define IBS_DATA_SRC_EXT_NEAR_CCX_CACHE 2 +#define IBS_DATA_SRC_EXT_DRAM 3 +#define IBS_DATA_SRC_EXT_FAR_CCX_CACHE 5
Is 4 undefined intentionally?
Thanks,
Namhyung
Quoted 11 lines
arch/x86/include/asm/amd-ibs.h +5
-1+#define IBS_DATA_SRC_EXT_PMEM 6 +#define IBS_DATA_SRC_EXT_IO 7 +#define IBS_DATA_SRC_EXT_EXT_MEM 8 +#define IBS_DATA_SRC_EXT_PEER_AGENT_MEM 12 +/* * IBS Hardware MSRs */ --2.31.1
Re: [PATCH v3 02/15] perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions
Quoted 21 lines · Namhyung Kim
Hi Ravi,
Quoted 18 lines · Ravi Bangoria
IBS_OP_DATA2DataSrc provides detail about location of the data being accessed from by load ops. Define macros for legacy and extended DataSrc values.Signed-off-by Ravi Bangoria <[email protected]>--- arch/x86/include/asm/amd-ibs.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)arch/x86/include/asm/amd-ibs.h +0
-0@@ -6,6 +6,22 @@#include <asm/msr-index.h>
Quoted 13 lines
arch/x86/include/asm/amd-ibs.h line 19
Quoted 11 lines
arch/x86/include/asm/amd-ibs.h +11
-0+/* IBS_OP_DATA2 DataSrc */ +#define IBS_DATA_SRC_LOC_CACHE 2 +#define IBS_DATA_SRC_DRAM 3 +#define IBS_DATA_SRC_REM_CACHE 4 +#define IBS_DATA_SRC_IO 7 + +/* IBS_OP_DATA2 DataSrc Extension */ +#define IBS_DATA_SRC_EXT_LOC_CACHE 1 +#define IBS_DATA_SRC_EXT_NEAR_CCX_CACHE 2 +#define IBS_DATA_SRC_EXT_DRAM 3 +#define IBS_DATA_SRC_EXT_FAR_CCX_CACHE 5Is 4 undefined intentionally?
Yes, Here is the snippet from PPR (Processor Programming Reference) doc:
Values | Description
---------------------------------------------------------------------
0h | No valid status.
1h | Local L3 or other L1/L2 in CCX.
2h | Another CCX cache in the same NUMA node.
3h | DRAM.
4h | Reserved.
5h | Another CCX cache in a different NUMA node.
6h | DRAM address map with "long latency" bit set.
7h | MMIO/Config/PCI/APIC.
8h | Extension Memory (S-Link, GenZ, etc - identified by the CS
| target and/or address map at DF's choice).
9h-Bh | Reserved.
Ch | Peer Agent Memory.
Dh-1Fh | Reserved.Thanks,
Ravi
Namhyung Kim
maintainerRe: [PATCH v3 02/15] perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions
Quoted 41 lines · Ravi Bangoria
Quoted 21 lines · Namhyung Kim
Hi Ravi,
Quoted 18 lines · Ravi Bangoria
IBS_OP_DATA2DataSrc provides detail about location of the data being accessed from by load ops. Define macros for legacy and extended DataSrc values.Signed-off-by Ravi Bangoria <[email protected]>--- arch/x86/include/asm/amd-ibs.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)arch/x86/include/asm/amd-ibs.h +0
-0@@ -6,6 +6,22 @@#include <asm/msr-index.h> Quoted 13 lines
arch/x86/include/asm/amd-ibs.h line 19
Quoted 11 lines
arch/x86/include/asm/amd-ibs.h +11
-0+/* IBS_OP_DATA2 DataSrc */ +#define IBS_DATA_SRC_LOC_CACHE 2 +#define IBS_DATA_SRC_DRAM 3 +#define IBS_DATA_SRC_REM_CACHE 4 +#define IBS_DATA_SRC_IO 7 + +/* IBS_OP_DATA2 DataSrc Extension */ +#define IBS_DATA_SRC_EXT_LOC_CACHE 1 +#define IBS_DATA_SRC_EXT_NEAR_CCX_CACHE 2 +#define IBS_DATA_SRC_EXT_DRAM 3 +#define IBS_DATA_SRC_EXT_FAR_CCX_CACHE 5Is 4 undefined intentionally?
Yes, Here is the snippet from PPR (Processor Programming Reference) doc:
Values | Description
---------------------------------------------------------------------
Quoted 13 lines
0h | No valid status.
1h | Local L3 or other L1/L2 in CCX.
2h | Another CCX cache in the same NUMA node.
3h | DRAM.
4h | Reserved.
5h | Another CCX cache in a different NUMA node.
6h | DRAM address map with "long latency" bit set.
7h | MMIO/Config/PCI/APIC.
8h | Extension Memory (S-Link, GenZ, etc - identified by the CS
| target and/or address map at DF's choice).
9h-Bh | Reserved.
Ch | Peer Agent Memory.
Dh-1Fh | Reserved.Thanks for sharing it. It's a bit confusing since it was available before.
Anyway, is the PPR for Zen4 publicly available now?
Thanks,
Namhyung
Re: [PATCH v3 02/15] perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions
Quoted 32 lines
Quoted 18 lines
Quoted 5 lines
Quoted 5 lines
arch/x86/include/asm/amd-ibs.h +5
-0+/* IBS_OP_DATA2 DataSrc */ +#define IBS_DATA_SRC_LOC_CACHE 2 +#define IBS_DATA_SRC_DRAM 3 +#define IBS_DATA_SRC_REM_CACHE 4 +#define IBS_DATA_SRC_IO 7Quoted 8 lines
arch/x86/include/asm/amd-ibs.h line 19
Quoted 6 lines
arch/x86/include/asm/amd-ibs.h +6
-0+ +/* IBS_OP_DATA2 DataSrc Extension */ +#define IBS_DATA_SRC_EXT_LOC_CACHE 1 +#define IBS_DATA_SRC_EXT_NEAR_CCX_CACHE 2 +#define IBS_DATA_SRC_EXT_DRAM 3 +#define IBS_DATA_SRC_EXT_FAR_CCX_CACHE 5Is 4 undefined intentionally?
Yes, Here is the snippet from PPR (Processor Programming Reference) doc:
Values | Description
---------------------------------------------------------------------Quoted 13 lines
0h | No valid status. 1h | Local L3 or other L1/L2 in CCX. 2h | Another CCX cache in the same NUMA node. 3h | DRAM. 4h | Reserved. 5h | Another CCX cache in a different NUMA node. 6h | DRAM address map with "long latency" bit set. 7h | MMIO/Config/PCI/APIC. 8h | Extension Memory (S-Link, GenZ, etc - identified by the CS | target and/or address map at DF's choice). 9h-Bh | Reserved. Ch | Peer Agent Memory. Dh-1Fh | Reserved.
Quoted 1 line
Thanks for sharing it. It's a bit confusing since it was available before.
Right, these bit definitions have changed in Zen4.
Quoted 2 lines
Anyway, is the PPR for Zen4 publicly available now?
Sadly, no. But it's in progress.
Thanks,
Ravi
[tip: perf/core] perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions
In reply to Ravi BangoriaThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 610c238041fbc682936d34132362a54a802600fe Gitweb: https://git.kernel.org/tip/610c238041fbc682936d34132362a54a802600fe Author: Ravi Bangoria <[email protected]> AuthorDate: Wed, 28 Sep 2022 15:27:52 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 29 Sep 2022 12:20:54 +02:00
perf/x86/amd: Add IBS OP_DATA2 DataSrc bit definitions
IBS_OP_DATA2 DataSrc provides detail about location of the data being accessed from by load ops. Define macros for legacy and extended DataSrc values.
--- arch/x86/include/asm/amd-ibs.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/include/asm/amd-ibs.h +16 -0
@@ -6,6 +6,22 @@#include <asm/msr-index.h> +/* IBS_OP_DATA2 DataSrc */ +#define IBS_DATA_SRC_LOC_CACHE 2 +#define IBS_DATA_SRC_DRAM 3 +#define IBS_DATA_SRC_REM_CACHE 4 +#define IBS_DATA_SRC_IO 7 + +/* IBS_OP_DATA2 DataSrc Extension */ +#define IBS_DATA_SRC_EXT_LOC_CACHE 1 +#define IBS_DATA_SRC_EXT_NEAR_CCX_CACHE 2 +#define IBS_DATA_SRC_EXT_DRAM 3 +#define IBS_DATA_SRC_EXT_FAR_CCX_CACHE 5 +#define IBS_DATA_SRC_EXT_PMEM 6 +#define IBS_DATA_SRC_EXT_IO 7 +#define IBS_DATA_SRC_EXT_EXT_MEM 8 +#define IBS_DATA_SRC_EXT_PEER_AGENT_MEM 12 +/* * IBS Hardware MSRs */
PATCH v3 3/15 perf/x86/amd: Support PERF_SAMPLE_DATA_SRC
In reply to Ravi Bangoriastruct perf_mem_data_src is used to pass arch specific memory access details into generic form. These details gets consumed by tools like perf mem and c2c. IBS tagged load/store sample provides most of the information needed for these tools. Add a logic to convert IBS specific raw data into perf_mem_data_src.
--- arch/x86/events/amd/ibs.c | 318 +++++++++++++++++++++++++++++++++++++- 1 file changed, 312 insertions(+), 6 deletions(-)
arch/x86/events/amd/ibs.c +312 -6
@@ -678,6 +678,312 @@ static struct perf_ibs perf_ibs_op = { .get_count = get_ibs_op_count, }; +static void perf_ibs_get_mem_op (union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; + + data_src->mem_op =PERF_MEM_OP_NA ; + +if (op_data3->ld_op) + data_src->mem_op =PERF_MEM_OP_LOAD ; +else if (op_data3->st_op) + data_src->mem_op =PERF_MEM_OP_STORE ; +} + +/* +* Processors having CPUID_Fn8000001B_EAX[11] aka IBS_CAPS_ZEN4 has +* more fine granular DataSrc encodings. Others have coarse. +*/ +static u8 perf_ibs_data_src (union ibs_op_data2 *op_data2) +{ +if (ibs_caps &IBS_CAPS_ZEN4 ) +return (op_data2->data_src_hi <<3 ) | op_data2->data_src_lo; + +return op_data2->data_src_lo; +} + +static void perf_ibs_get_mem_lvl (union ibs_op_data2 *op_data2, +union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; +u8 ibs_data_src =perf_ibs_data_src (op_data2); + + data_src->mem_lvl =0 ; + +/* +* DcMiss, L2Miss, DataSrc, DcMissLat etc. are all invalid for Uncached +* memory accesses. So, check DcUcMemAcc bit early. +*/ +if (op_data3->dc_uc_mem_acc && ibs_data_src !=IBS_DATA_SRC_EXT_IO ) { + data_src->mem_lvl =PERF_MEM_LVL_UNC |PERF_MEM_LVL_HIT ; +return ; + } + +/* L1 Hit */ +if (op_data3->dc_miss ==0 ) { + data_src->mem_lvl =PERF_MEM_LVL_L1 |PERF_MEM_LVL_HIT ; +return ; + } + +/* L2 Hit */ +if (op_data3->l2_miss ==0 ) { +/* Erratum #1293 */ +if (boot_cpu_data.x86 !=0x19 || boot_cpu_data.x86_model >0xF || + !(op_data3->sw_pf || op_data3->dc_miss_no_mab_alloc)) { + data_src->mem_lvl =PERF_MEM_LVL_L2 |PERF_MEM_LVL_HIT ; +return ; + } + } + +/* +* OP_DATA2 is valid only for load ops. Skip all checks which +* uses OP_DATA2[DataSrc]. +*/ +if (data_src->mem_op !=PERF_MEM_OP_LOAD ) + goto check_mab; + +/* L3 Hit */ +if (ibs_caps &IBS_CAPS_ZEN4 ) { +if (ibs_data_src ==IBS_DATA_SRC_EXT_LOC_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_L3 |PERF_MEM_LVL_HIT ; +return ; + } + }else { +if (ibs_data_src ==IBS_DATA_SRC_LOC_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_L3 |PERF_MEM_LVL_REM_CCE1 | +PERF_MEM_LVL_HIT ; +return ; + } + } + +/* A peer cache in a near CCX */ +if (ibs_caps &IBS_CAPS_ZEN4 && + ibs_data_src ==IBS_DATA_SRC_EXT_NEAR_CCX_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_REM_CCE1 |PERF_MEM_LVL_HIT ; +return ; + } + +/* A peer cache in a far CCX */ +if (ibs_caps &IBS_CAPS_ZEN4 ) { +if (ibs_data_src ==IBS_DATA_SRC_EXT_FAR_CCX_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_REM_CCE2 |PERF_MEM_LVL_HIT ; +return ; + } + }else { +if (ibs_data_src ==IBS_DATA_SRC_REM_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_REM_CCE2 |PERF_MEM_LVL_HIT ; +return ; + } + } + +/* DRAM */ +if (ibs_data_src ==IBS_DATA_SRC_EXT_DRAM ) { +if (op_data2->rmt_node ==0 ) + data_src->mem_lvl =PERF_MEM_LVL_LOC_RAM |PERF_MEM_LVL_HIT ; +else + data_src->mem_lvl =PERF_MEM_LVL_REM_RAM1 |PERF_MEM_LVL_HIT ; +return ; + } + +/* PMEM */ +if (ibs_caps &IBS_CAPS_ZEN4 && ibs_data_src ==IBS_DATA_SRC_EXT_PMEM ) { + data_src->mem_lvl_num =PERF_MEM_LVLNUM_PMEM ; +if (op_data2->rmt_node) { + data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ; +/* IBS doesn't provide Remote socket detail */ + data_src->mem_hops =PERF_MEM_HOPS_1 ; + } +return ; + } + +/* Extension Memory */ +if (ibs_caps &IBS_CAPS_ZEN4 && + ibs_data_src ==IBS_DATA_SRC_EXT_EXT_MEM ) { + data_src->mem_lvl_num =PERF_MEM_LVLNUM_EXTN_MEM ; +if (op_data2->rmt_node) { + data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ; +/* IBS doesn't provide Remote socket detail */ + data_src->mem_hops =PERF_MEM_HOPS_1 ; + } +return ; + } + +/* IO */ +if (ibs_data_src ==IBS_DATA_SRC_EXT_IO ) { + data_src->mem_lvl =PERF_MEM_LVL_IO ; + data_src->mem_lvl_num =PERF_MEM_LVLNUM_IO ; +if (op_data2->rmt_node) { + data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ; +/* IBS doesn't provide Remote socket detail */ + data_src->mem_hops =PERF_MEM_HOPS_1 ; + } +return ; + } + +check_mab: +/* +* MAB (Miss Address Buffer) Hit. MAB keeps track of outstanding +* DC misses. However, such data may come from any level in mem +* hierarchy. IBS provides detail about both MAB as well as actual +* DataSrc simultaneously. Prioritize DataSrc over MAB, i.e. set +* MAB only when IBS fails to provide DataSrc. +*/ +if (op_data3->dc_miss_no_mab_alloc) { + data_src->mem_lvl =PERF_MEM_LVL_LFB |PERF_MEM_LVL_HIT ; +return ; + } + + data_src->mem_lvl =PERF_MEM_LVL_NA ; +} + +static bool perf_ibs_cache_hit_st_valid (void ) +{ +/* 0: Uninitialized, 1: Valid, -1: Invalid */ +static int cache_hit_st_valid; + +if (unlikely (!cache_hit_st_valid)) { +if (boot_cpu_data.x86 ==0x19 && + (boot_cpu_data.x86_model <=0xF || + (boot_cpu_data.x86_model >=0x20 && + boot_cpu_data.x86_model <=0x5F ))) { + cache_hit_st_valid =-1 ; + }else { + cache_hit_st_valid =1 ; + } + } + +return cache_hit_st_valid ==1 ; +} + +static void perf_ibs_get_mem_snoop (union ibs_op_data2 *op_data2, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; +u8 ibs_data_src; + + data_src->mem_snoop =PERF_MEM_SNOOP_NA ; + +if (!perf_ibs_cache_hit_st_valid () || + data_src->mem_op !=PERF_MEM_OP_LOAD || + data_src->mem_lvl &PERF_MEM_LVL_L1 || + data_src->mem_lvl &PERF_MEM_LVL_L2 || + op_data2->cache_hit_st) +return ; + + ibs_data_src =perf_ibs_data_src (op_data2); + +if (ibs_caps &IBS_CAPS_ZEN4 ) { +if (ibs_data_src ==IBS_DATA_SRC_EXT_LOC_CACHE || + ibs_data_src ==IBS_DATA_SRC_EXT_NEAR_CCX_CACHE || + ibs_data_src ==IBS_DATA_SRC_EXT_FAR_CCX_CACHE ) + data_src->mem_snoop =PERF_MEM_SNOOP_HITM ; + }else if (ibs_data_src ==IBS_DATA_SRC_LOC_CACHE ) { + data_src->mem_snoop =PERF_MEM_SNOOP_HITM ; + } +} + +static void perf_ibs_get_tlb_lvl (union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; + + data_src->mem_dtlb =PERF_MEM_TLB_NA ; + +if (!op_data3->dc_lin_addr_valid) +return ; + +if (!op_data3->dc_l1tlb_miss) { + data_src->mem_dtlb =PERF_MEM_TLB_L1 |PERF_MEM_TLB_HIT ; +return ; + } + +if (!op_data3->dc_l2tlb_miss) { + data_src->mem_dtlb =PERF_MEM_TLB_L2 |PERF_MEM_TLB_HIT ; +return ; + } + + data_src->mem_dtlb =PERF_MEM_TLB_L2 |PERF_MEM_TLB_MISS ; +} + +static void perf_ibs_get_mem_lock (union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; + + data_src->mem_lock =PERF_MEM_LOCK_NA ; + +if (op_data3->dc_locked_op) + data_src->mem_lock =PERF_MEM_LOCK_LOCKED ; +} + +#define ibs_op_msr_idx (msr) (msr - MSR_AMD64_IBSOPCTL) + +static void perf_ibs_get_data_src (struct perf_ibs_data *ibs_data, +struct perf_sample_data *data, +union ibs_op_data2 *op_data2, +union ibs_op_data3 *op_data3) +{ +perf_ibs_get_mem_lvl (op_data2, op_data3, data); +perf_ibs_get_mem_snoop (op_data2, data); +perf_ibs_get_tlb_lvl (op_data3, data); +perf_ibs_get_mem_lock (op_data3, data); +} + +static __u64 perf_ibs_get_op_data2 (struct perf_ibs_data *ibs_data, +union ibs_op_data3 *op_data3) +{ +__u64 val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA2 )]; + +/* Erratum #1293 */ +if (boot_cpu_data.x86 ==0x19 && boot_cpu_data.x86_model <=0xF && + (op_data3->sw_pf || op_data3->dc_miss_no_mab_alloc)) { +/* +* OP_DATA2 has only two fields on Zen3: DataSrc and RmtNode. +* DataSrc=0 is 'No valid status' and RmtNode is invalid when +* DataSrc=0. +*/ + val =0 ; + } +return val; +} + +static void perf_ibs_parse_ld_st_data (__u64 sample_type, +struct perf_ibs_data *ibs_data, +struct perf_sample_data *data) +{ +union ibs_op_data3 op_data3; +union ibs_op_data2 op_data2; + + data->data_src.val =PERF_MEM_NA ; + op_data3.val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA3 )]; + +perf_ibs_get_mem_op (&op_data3, data); +if (data->data_src.mem_op !=PERF_MEM_OP_LOAD && + data->data_src.mem_op !=PERF_MEM_OP_STORE ) +return ; + + op_data2.val =perf_ibs_get_op_data2 (ibs_data, &op_data3); + +if (sample_type &PERF_SAMPLE_DATA_SRC ) { +perf_ibs_get_data_src (ibs_data, data, &op_data2, &op_data3); + data->sample_flags |=PERF_SAMPLE_DATA_SRC ; + } +} + +static int perf_ibs_get_offset_max (struct perf_ibs *perf_ibs,u64 sample_type, +int check_rip) +{ +if (sample_type &PERF_SAMPLE_RAW || + (perf_ibs == &perf_ibs_op && + sample_type &PERF_SAMPLE_DATA_SRC )) +return perf_ibs->offset_max; +else if (check_rip) +return 3 ; +return 1 ; +} +static int perf_ibs_handle_irq (struct perf_ibs *perf_ibs,struct pt_regs *iregs) {struct cpu_perf_ibs *pcpu =this_cpu_ptr (perf_ibs->pcpu);
@@ -725,12 +1031,9 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) size =1 ; offset =1 ; check_rip = (perf_ibs == &perf_ibs_op && (ibs_caps &IBS_CAPS_RIPINVALIDCHK )); -if (event->attr.sample_type &PERF_SAMPLE_RAW ) - offset_max = perf_ibs->offset_max; -else if (check_rip) - offset_max =3 ; -else - offset_max =1 ; + + offset_max =perf_ibs_get_offset_max (perf_ibs, event->attr.sample_type, check_rip); + do { rdmsrl(msr + offset, *buf++); size++;
@@ -784,6 +1087,9 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) data.sample_flags |=PERF_SAMPLE_RAW ; } +if (perf_ibs == &perf_ibs_op) +perf_ibs_parse_ld_st_data (event->attr.sample_type, &ibs_data, &data); +/* * rip recorded by IbsOpRip will not be consistent with rsp and rbp * recorded as part of interrupt regs. Thus we need to use rip from
Signature · 1 line
2.31.1
[tip: perf/core] perf/x86/amd: Support PERF_SAMPLE_DATA_SRC
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 7c10dd0a88b1cc6ae4637fffb494c5e080027eb6 Gitweb: https://git.kernel.org/tip/7c10dd0a88b1cc6ae4637fffb494c5e080027eb6 Author: Ravi Bangoria <[email protected]> AuthorDate: Wed, 28 Sep 2022 15:27:53 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 29 Sep 2022 12:20:55 +02:00
perf/x86/amd: Support PERF_SAMPLE_DATA_SRC
struct perf_mem_data_src is used to pass arch specific memory access details into generic form. These details gets consumed by tools like perf mem and c2c. IBS tagged load/store sample provides most of the information needed for these tools. Add a logic to convert IBS specific raw data into perf_mem_data_src.
--- arch/x86/events/amd/ibs.c | 318 ++++++++++++++++++++++++++++++++++++- 1 file changed, 312 insertions(+), 6 deletions(-)
arch/x86/events/amd/ibs.c +312 -6
@@ -678,6 +678,312 @@ static struct perf_ibs perf_ibs_op = { .get_count = get_ibs_op_count, }; +static void perf_ibs_get_mem_op (union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; + + data_src->mem_op =PERF_MEM_OP_NA ; + +if (op_data3->ld_op) + data_src->mem_op =PERF_MEM_OP_LOAD ; +else if (op_data3->st_op) + data_src->mem_op =PERF_MEM_OP_STORE ; +} + +/* +* Processors having CPUID_Fn8000001B_EAX[11] aka IBS_CAPS_ZEN4 has +* more fine granular DataSrc encodings. Others have coarse. +*/ +static u8 perf_ibs_data_src (union ibs_op_data2 *op_data2) +{ +if (ibs_caps &IBS_CAPS_ZEN4 ) +return (op_data2->data_src_hi <<3 ) | op_data2->data_src_lo; + +return op_data2->data_src_lo; +} + +static void perf_ibs_get_mem_lvl (union ibs_op_data2 *op_data2, +union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; +u8 ibs_data_src =perf_ibs_data_src (op_data2); + + data_src->mem_lvl =0 ; + +/* +* DcMiss, L2Miss, DataSrc, DcMissLat etc. are all invalid for Uncached +* memory accesses. So, check DcUcMemAcc bit early. +*/ +if (op_data3->dc_uc_mem_acc && ibs_data_src !=IBS_DATA_SRC_EXT_IO ) { + data_src->mem_lvl =PERF_MEM_LVL_UNC |PERF_MEM_LVL_HIT ; +return ; + } + +/* L1 Hit */ +if (op_data3->dc_miss ==0 ) { + data_src->mem_lvl =PERF_MEM_LVL_L1 |PERF_MEM_LVL_HIT ; +return ; + } + +/* L2 Hit */ +if (op_data3->l2_miss ==0 ) { +/* Erratum #1293 */ +if (boot_cpu_data.x86 !=0x19 || boot_cpu_data.x86_model >0xF || + !(op_data3->sw_pf || op_data3->dc_miss_no_mab_alloc)) { + data_src->mem_lvl =PERF_MEM_LVL_L2 |PERF_MEM_LVL_HIT ; +return ; + } + } + +/* +* OP_DATA2 is valid only for load ops. Skip all checks which +* uses OP_DATA2[DataSrc]. +*/ +if (data_src->mem_op !=PERF_MEM_OP_LOAD ) + goto check_mab; + +/* L3 Hit */ +if (ibs_caps &IBS_CAPS_ZEN4 ) { +if (ibs_data_src ==IBS_DATA_SRC_EXT_LOC_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_L3 |PERF_MEM_LVL_HIT ; +return ; + } + }else { +if (ibs_data_src ==IBS_DATA_SRC_LOC_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_L3 |PERF_MEM_LVL_REM_CCE1 | +PERF_MEM_LVL_HIT ; +return ; + } + } + +/* A peer cache in a near CCX */ +if (ibs_caps &IBS_CAPS_ZEN4 && + ibs_data_src ==IBS_DATA_SRC_EXT_NEAR_CCX_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_REM_CCE1 |PERF_MEM_LVL_HIT ; +return ; + } + +/* A peer cache in a far CCX */ +if (ibs_caps &IBS_CAPS_ZEN4 ) { +if (ibs_data_src ==IBS_DATA_SRC_EXT_FAR_CCX_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_REM_CCE2 |PERF_MEM_LVL_HIT ; +return ; + } + }else { +if (ibs_data_src ==IBS_DATA_SRC_REM_CACHE ) { + data_src->mem_lvl =PERF_MEM_LVL_REM_CCE2 |PERF_MEM_LVL_HIT ; +return ; + } + } + +/* DRAM */ +if (ibs_data_src ==IBS_DATA_SRC_EXT_DRAM ) { +if (op_data2->rmt_node ==0 ) + data_src->mem_lvl =PERF_MEM_LVL_LOC_RAM |PERF_MEM_LVL_HIT ; +else + data_src->mem_lvl =PERF_MEM_LVL_REM_RAM1 |PERF_MEM_LVL_HIT ; +return ; + } + +/* PMEM */ +if (ibs_caps &IBS_CAPS_ZEN4 && ibs_data_src ==IBS_DATA_SRC_EXT_PMEM ) { + data_src->mem_lvl_num =PERF_MEM_LVLNUM_PMEM ; +if (op_data2->rmt_node) { + data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ; +/* IBS doesn't provide Remote socket detail */ + data_src->mem_hops =PERF_MEM_HOPS_1 ; + } +return ; + } + +/* Extension Memory */ +if (ibs_caps &IBS_CAPS_ZEN4 && + ibs_data_src ==IBS_DATA_SRC_EXT_EXT_MEM ) { + data_src->mem_lvl_num =PERF_MEM_LVLNUM_EXTN_MEM ; +if (op_data2->rmt_node) { + data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ; +/* IBS doesn't provide Remote socket detail */ + data_src->mem_hops =PERF_MEM_HOPS_1 ; + } +return ; + } + +/* IO */ +if (ibs_data_src ==IBS_DATA_SRC_EXT_IO ) { + data_src->mem_lvl =PERF_MEM_LVL_IO ; + data_src->mem_lvl_num =PERF_MEM_LVLNUM_IO ; +if (op_data2->rmt_node) { + data_src->mem_remote =PERF_MEM_REMOTE_REMOTE ; +/* IBS doesn't provide Remote socket detail */ + data_src->mem_hops =PERF_MEM_HOPS_1 ; + } +return ; + } + +check_mab: +/* +* MAB (Miss Address Buffer) Hit. MAB keeps track of outstanding +* DC misses. However, such data may come from any level in mem +* hierarchy. IBS provides detail about both MAB as well as actual +* DataSrc simultaneously. Prioritize DataSrc over MAB, i.e. set +* MAB only when IBS fails to provide DataSrc. +*/ +if (op_data3->dc_miss_no_mab_alloc) { + data_src->mem_lvl =PERF_MEM_LVL_LFB |PERF_MEM_LVL_HIT ; +return ; + } + + data_src->mem_lvl =PERF_MEM_LVL_NA ; +} + +static bool perf_ibs_cache_hit_st_valid (void ) +{ +/* 0: Uninitialized, 1: Valid, -1: Invalid */ +static int cache_hit_st_valid; + +if (unlikely (!cache_hit_st_valid)) { +if (boot_cpu_data.x86 ==0x19 && + (boot_cpu_data.x86_model <=0xF || + (boot_cpu_data.x86_model >=0x20 && + boot_cpu_data.x86_model <=0x5F ))) { + cache_hit_st_valid =-1 ; + }else { + cache_hit_st_valid =1 ; + } + } + +return cache_hit_st_valid ==1 ; +} + +static void perf_ibs_get_mem_snoop (union ibs_op_data2 *op_data2, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; +u8 ibs_data_src; + + data_src->mem_snoop =PERF_MEM_SNOOP_NA ; + +if (!perf_ibs_cache_hit_st_valid () || + data_src->mem_op !=PERF_MEM_OP_LOAD || + data_src->mem_lvl &PERF_MEM_LVL_L1 || + data_src->mem_lvl &PERF_MEM_LVL_L2 || + op_data2->cache_hit_st) +return ; + + ibs_data_src =perf_ibs_data_src (op_data2); + +if (ibs_caps &IBS_CAPS_ZEN4 ) { +if (ibs_data_src ==IBS_DATA_SRC_EXT_LOC_CACHE || + ibs_data_src ==IBS_DATA_SRC_EXT_NEAR_CCX_CACHE || + ibs_data_src ==IBS_DATA_SRC_EXT_FAR_CCX_CACHE ) + data_src->mem_snoop =PERF_MEM_SNOOP_HITM ; + }else if (ibs_data_src ==IBS_DATA_SRC_LOC_CACHE ) { + data_src->mem_snoop =PERF_MEM_SNOOP_HITM ; + } +} + +static void perf_ibs_get_tlb_lvl (union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; + + data_src->mem_dtlb =PERF_MEM_TLB_NA ; + +if (!op_data3->dc_lin_addr_valid) +return ; + +if (!op_data3->dc_l1tlb_miss) { + data_src->mem_dtlb =PERF_MEM_TLB_L1 |PERF_MEM_TLB_HIT ; +return ; + } + +if (!op_data3->dc_l2tlb_miss) { + data_src->mem_dtlb =PERF_MEM_TLB_L2 |PERF_MEM_TLB_HIT ; +return ; + } + + data_src->mem_dtlb =PERF_MEM_TLB_L2 |PERF_MEM_TLB_MISS ; +} + +static void perf_ibs_get_mem_lock (union ibs_op_data3 *op_data3, +struct perf_sample_data *data) +{ +union perf_mem_data_src *data_src = &data->data_src; + + data_src->mem_lock =PERF_MEM_LOCK_NA ; + +if (op_data3->dc_locked_op) + data_src->mem_lock =PERF_MEM_LOCK_LOCKED ; +} + +#define ibs_op_msr_idx (msr) (msr - MSR_AMD64_IBSOPCTL) + +static void perf_ibs_get_data_src (struct perf_ibs_data *ibs_data, +struct perf_sample_data *data, +union ibs_op_data2 *op_data2, +union ibs_op_data3 *op_data3) +{ +perf_ibs_get_mem_lvl (op_data2, op_data3, data); +perf_ibs_get_mem_snoop (op_data2, data); +perf_ibs_get_tlb_lvl (op_data3, data); +perf_ibs_get_mem_lock (op_data3, data); +} + +static __u64 perf_ibs_get_op_data2 (struct perf_ibs_data *ibs_data, +union ibs_op_data3 *op_data3) +{ +__u64 val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA2 )]; + +/* Erratum #1293 */ +if (boot_cpu_data.x86 ==0x19 && boot_cpu_data.x86_model <=0xF && + (op_data3->sw_pf || op_data3->dc_miss_no_mab_alloc)) { +/* +* OP_DATA2 has only two fields on Zen3: DataSrc and RmtNode. +* DataSrc=0 is 'No valid status' and RmtNode is invalid when +* DataSrc=0. +*/ + val =0 ; + } +return val; +} + +static void perf_ibs_parse_ld_st_data (__u64 sample_type, +struct perf_ibs_data *ibs_data, +struct perf_sample_data *data) +{ +union ibs_op_data3 op_data3; +union ibs_op_data2 op_data2; + + data->data_src.val =PERF_MEM_NA ; + op_data3.val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA3 )]; + +perf_ibs_get_mem_op (&op_data3, data); +if (data->data_src.mem_op !=PERF_MEM_OP_LOAD && + data->data_src.mem_op !=PERF_MEM_OP_STORE ) +return ; + + op_data2.val =perf_ibs_get_op_data2 (ibs_data, &op_data3); + +if (sample_type &PERF_SAMPLE_DATA_SRC ) { +perf_ibs_get_data_src (ibs_data, data, &op_data2, &op_data3); + data->sample_flags |=PERF_SAMPLE_DATA_SRC ; + } +} + +static int perf_ibs_get_offset_max (struct perf_ibs *perf_ibs,u64 sample_type, +int check_rip) +{ +if (sample_type &PERF_SAMPLE_RAW || + (perf_ibs == &perf_ibs_op && + sample_type &PERF_SAMPLE_DATA_SRC )) +return perf_ibs->offset_max; +else if (check_rip) +return 3 ; +return 1 ; +} +static int perf_ibs_handle_irq (struct perf_ibs *perf_ibs,struct pt_regs *iregs) {struct cpu_perf_ibs *pcpu =this_cpu_ptr (perf_ibs->pcpu);
@@ -725,12 +1031,9 @@ fail: size =1 ; offset =1 ; check_rip = (perf_ibs == &perf_ibs_op && (ibs_caps &IBS_CAPS_RIPINVALIDCHK )); -if (event->attr.sample_type &PERF_SAMPLE_RAW ) - offset_max = perf_ibs->offset_max; -else if (check_rip) - offset_max =3 ; -else - offset_max =1 ; + + offset_max =perf_ibs_get_offset_max (perf_ibs, event->attr.sample_type, check_rip); + do { rdmsrl(msr + offset, *buf++); size++;
@@ -784,6 +1087,9 @@ fail: data.sample_flags |=PERF_SAMPLE_RAW ; } +if (perf_ibs == &perf_ibs_op) +perf_ibs_parse_ld_st_data (event->attr.sample_type, &ibs_data, &data); +/* * rip recorded by IbsOpRip will not be consistent with rsp and rbp * recorded as part of interrupt regs. Thus we need to use rip from
PATCH v3 4/15 perf/x86/amd: Support PERF_SAMPLE_{WEIGHT|WEIGHT_STRUCT}
In reply to Ravi BangoriaIbsDcMissLat indicates the number of clock cycles from when a miss is detected in the data cache to when the data was delivered to the core. Similarly, IbsTagToRetCtr provides number of cycles from when the op was tagged to when the op was retired. Consider these fields for sample->weight.
---
Note:
While opening a new event, perf tool starts with a set of attributes and goes on reverting some attributes in a predefined order until it succeeds or run out or all attempts. Here, 1st attempt includes both WEIGHT_STRUCT and exclude_guest which always fails because IBS does not support guest filtering. The problem however is, perf reverts WEIGHT_STRUCT but keeps trying with exclude_guest. Thus, although, this patch enables WEIGHT_STRUCT support from kernel, using it from the perf tool needs more changes(not included in this series).
arch/x86/events/amd/ibs.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
arch/x86/events/amd/ibs.c +16 -1
@@ -955,6 +955,7 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, {union ibs_op_data3 op_data3;union ibs_op_data2 op_data2; +union ibs_op_data op_data; data->data_src.val =PERF_MEM_NA ; op_data3.val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA3 )];
@@ -970,6 +971,19 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type,perf_ibs_get_data_src (ibs_data, data, &op_data2, &op_data3); data->sample_flags |=PERF_SAMPLE_DATA_SRC ; } + +if (sample_type &PERF_SAMPLE_WEIGHT_TYPE && op_data3.dc_miss && + data->data_src.mem_op ==PERF_MEM_OP_LOAD ) { + op_data.val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA )]; + +if (sample_type &PERF_SAMPLE_WEIGHT_STRUCT ) { + data->weight.var1_dw = op_data3.dc_miss_lat; + data->weight.var2_w = op_data.tag_to_ret_ctr; + }else if (sample_type &PERF_SAMPLE_WEIGHT ) { + data->weight.full = op_data3.dc_miss_lat; + } + data->sample_flags |=PERF_SAMPLE_WEIGHT_TYPE ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,
@@ -977,7 +991,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type, {if (sample_type &PERF_SAMPLE_RAW || (perf_ibs == &perf_ibs_op && - sample_type &PERF_SAMPLE_DATA_SRC )) + (sample_type &PERF_SAMPLE_DATA_SRC || + sample_type &PERF_SAMPLE_WEIGHT_TYPE )))return perf_ibs->offset_max;else if (check_rip)return 3 ;
Signature · 1 line
2.31.1
Namhyung Kim
maintainerRe: [PATCH v3 04/15] perf/x86/amd: Support PERF_SAMPLE_{WEIGHT|WEIGHT_STRUCT}
Quoted 8 lines · Ravi Bangoria
IbsDcMissLat indicates the number of clock cycles from when a miss is detected in the data cache to when the data was delivered to the core. Similarly, IbsTagToRetCtr provides number of cycles from when the op was tagged to when the op was retired. Consider these fields for
sample->weight.Signed-off-by Ravi Bangoria <[email protected]>
Quoted 10 lines
---
Note:
While opening a new event, perf tool starts with a set of attributes and goes on reverting some attributes in a predefined order until it succeeds or run out or all attempts. Here, 1st attempt includes bothWEIGHT_STRUCTandexclude_guestwhich always fails because IBS does not support guest filtering. The problem however is, perf revertsWEIGHT_STRUCTbut keeps trying withexclude_guest. Thus, although, this patch enablesWEIGHT_STRUCTsupport from kernel, using it from the perf tool needs more changes(not included in this series).
Yeah, it'd be nice if kernel could expose more pmu capabilities like no-exclude then tools can skip setting it for them.
Thanks,
Namhyung
[tip: perf/core] perf/x86/amd: Support PERF_SAMPLE_{WEIGHT|WEIGHT_STRUCT}
In reply to Ravi BangoriaThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 6b2ae4952ef8ac23b467bc10776404092b581143 Gitweb: https://git.kernel.org/tip/6b2ae4952ef8ac23b467bc10776404092b581143 Author: Ravi Bangoria <[email protected]> AuthorDate: Wed, 28 Sep 2022 15:27:54 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 29 Sep 2022 12:20:55 +02:00
perf/x86/amd: Support PERF_SAMPLE_{WEIGHT|WEIGHT_STRUCT}
IbsDcMissLat indicates the number of clock cycles from when a miss is detected in the data cache to when the data was delivered to the core. Similarly, IbsTagToRetCtr provides number of cycles from when the op was tagged to when the op was retired. Consider these fields for sample->weight.
--- arch/x86/events/amd/ibs.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
arch/x86/events/amd/ibs.c +16 -1
@@ -955,6 +955,7 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, {union ibs_op_data3 op_data3;union ibs_op_data2 op_data2; +union ibs_op_data op_data; data->data_src.val =PERF_MEM_NA ; op_data3.val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA3 )];
@@ -970,6 +971,19 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type,perf_ibs_get_data_src (ibs_data, data, &op_data2, &op_data3); data->sample_flags |=PERF_SAMPLE_DATA_SRC ; } + +if (sample_type &PERF_SAMPLE_WEIGHT_TYPE && op_data3.dc_miss && + data->data_src.mem_op ==PERF_MEM_OP_LOAD ) { + op_data.val = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSOPDATA )]; + +if (sample_type &PERF_SAMPLE_WEIGHT_STRUCT ) { + data->weight.var1_dw = op_data3.dc_miss_lat; + data->weight.var2_w = op_data.tag_to_ret_ctr; + }else if (sample_type &PERF_SAMPLE_WEIGHT ) { + data->weight.full = op_data3.dc_miss_lat; + } + data->sample_flags |=PERF_SAMPLE_WEIGHT_TYPE ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,
@@ -977,7 +991,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type, {if (sample_type &PERF_SAMPLE_RAW || (perf_ibs == &perf_ibs_op && - sample_type &PERF_SAMPLE_DATA_SRC )) + (sample_type &PERF_SAMPLE_DATA_SRC || + sample_type &PERF_SAMPLE_WEIGHT_TYPE )))return perf_ibs->offset_max;else if (check_rip)return 3 ;
PATCH v3 5/15 perf/x86/amd: Support PERF_SAMPLE_ADDR
In reply to Ravi BangoriaIBS_DC_LINADDR provides the linear data address for the tagged load/ store operation. Populate perf sample address using it.
--- arch/x86/events/amd/ibs.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/x86/events/amd/ibs.c +7 -1
@@ -984,6 +984,11 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, } data->sample_flags |=PERF_SAMPLE_WEIGHT_TYPE ; } + +if (sample_type &PERF_SAMPLE_ADDR && op_data3.dc_lin_addr_valid) { + data->addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCLINAD )]; + data->sample_flags |=PERF_SAMPLE_ADDR ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,
@@ -992,7 +997,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,if (sample_type &PERF_SAMPLE_RAW || (perf_ibs == &perf_ibs_op && (sample_type &PERF_SAMPLE_DATA_SRC || - sample_type &PERF_SAMPLE_WEIGHT_TYPE ))) + sample_type &PERF_SAMPLE_WEIGHT_TYPE || + sample_type &PERF_SAMPLE_ADDR )))return perf_ibs->offset_max;else if (check_rip)return 3 ;
Signature · 1 line
2.31.1
[tip: perf/core] perf/x86/amd: Support PERF_SAMPLE_ADDR
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: cb2bb85f7ed8740ab5fc06bbec386faa39ba44ef Gitweb: https://git.kernel.org/tip/cb2bb85f7ed8740ab5fc06bbec386faa39ba44ef Author: Ravi Bangoria <[email protected]> AuthorDate: Wed, 28 Sep 2022 15:27:55 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 29 Sep 2022 12:20:55 +02:00
perf/x86/amd: Support PERF_SAMPLE_ADDR
IBS_DC_LINADDR provides the linear data address for the tagged load/ store operation. Populate perf sample address using it.
--- arch/x86/events/amd/ibs.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/x86/events/amd/ibs.c +7 -1
@@ -984,6 +984,11 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, } data->sample_flags |=PERF_SAMPLE_WEIGHT_TYPE ; } + +if (sample_type &PERF_SAMPLE_ADDR && op_data3.dc_lin_addr_valid) { + data->addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCLINAD )]; + data->sample_flags |=PERF_SAMPLE_ADDR ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,
@@ -992,7 +997,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,if (sample_type &PERF_SAMPLE_RAW || (perf_ibs == &perf_ibs_op && (sample_type &PERF_SAMPLE_DATA_SRC || - sample_type &PERF_SAMPLE_WEIGHT_TYPE ))) + sample_type &PERF_SAMPLE_WEIGHT_TYPE || + sample_type &PERF_SAMPLE_ADDR )))return perf_ibs->offset_max;else if (check_rip)return 3 ;
PATCH v3 6/15 perf/x86/amd: Support PERF_SAMPLE_PHY_ADDR
In reply to Ravi BangoriaIBS_DC_PHYSADDR provides the physical data address for the tagged load/ store operation. Populate perf sample physical address using it.
--- arch/x86/events/amd/ibs.c | 8 +++++++- kernel/events/core.c | 3 ++- 2 files changed, 9 insertions(+), 2 deletions(-)
arch/x86/events/amd/ibs.c +7 -1
@@ -989,6 +989,11 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, data->addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCLINAD )]; data->sample_flags |=PERF_SAMPLE_ADDR ; } + +if (sample_type &PERF_SAMPLE_PHYS_ADDR && op_data3.dc_phy_addr_valid) { + data->phys_addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCPHYSAD )]; + data->sample_flags |=PERF_SAMPLE_PHYS_ADDR ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,
@@ -998,7 +1003,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type, (perf_ibs == &perf_ibs_op && (sample_type &PERF_SAMPLE_DATA_SRC || sample_type &PERF_SAMPLE_WEIGHT_TYPE || - sample_type & PERF_SAMPLE_ADDR))) + sample_type &PERF_SAMPLE_ADDR || + sample_type & PERF_SAMPLE_PHYS_ADDR))) return perf_ibs->offset_max; else if (check_rip) return 3;
kernel/events/core.c +2 -1
@@ -7435,7 +7435,8 @@ void perf_prepare_sample(struct perf_event_header *header, header->size += size; } -if (sample_type &PERF_SAMPLE_PHYS_ADDR ) +if (sample_type &PERF_SAMPLE_PHYS_ADDR && + filtered_sample_type &PERF_SAMPLE_PHYS_ADDR ) data->phys_addr =perf_virt_to_phys (data->addr);#ifdef CONFIG_CGROUP_PERF
Signature · 1 line
2.31.1
Namhyung Kim
maintainerRe: [PATCH v3 06/15] perf/x86/amd: Support PERF_SAMPLE_PHY_ADDR
Quoted 41 lines · Ravi Bangoria
IBS_DC_PHYSADDRprovides the physical data address for the tagged load/ store operation. Populate perf sample physical address using it.Signed-off-by Ravi Bangoria <[email protected]>--- arch/x86/events/amd/ibs.c | 8 +++++++- kernel/events/core.c | 3 ++- 2 files changed, 9 insertions(+), 2 deletions(-)arch/x86/events/amd/ibs.c +7
-1@@ -989,6 +989,11 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, data->addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCLINAD )]; data->sample_flags |=PERF_SAMPLE_ADDR ; } + +if (sample_type &PERF_SAMPLE_PHYS_ADDR && op_data3.dc_phy_addr_valid) { + data->phys_addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCPHYSAD )]; + data->sample_flags |=PERF_SAMPLE_PHYS_ADDR ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,@@ -998,7 +1003,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type, (perf_ibs == &perf_ibs_op && (sample_type &PERF_SAMPLE_DATA_SRC || sample_type &PERF_SAMPLE_WEIGHT_TYPE || - sample_type & PERF_SAMPLE_ADDR))) + sample_type &PERF_SAMPLE_ADDR || + sample_type & PERF_SAMPLE_PHYS_ADDR))) return perf_ibs->offset_max; else if (check_rip) return 3;kernel/events/core.c +0
-0
kernel/events/core.c void perf_prepare_sample(struct perf_event_header *header, · line 7439
Quoted 6 lines
kernel/events/core.c +2
-1@@ -7435,7 +7435,8 @@ void perf_prepare_sample(struct perf_event_header *header, header->size += size; } -if (sample_type &PERF_SAMPLE_PHYS_ADDR ) +if (sample_type &PERF_SAMPLE_PHYS_ADDR && + filtered_sample_type &PERF_SAMPLE_PHYS_ADDR )
It'd be enough to check the filtered_sample_type only.
Thanks,
Namhyung
Quoted 6 lines
kernel/events/core.c +0
-1data->phys_addr =perf_virt_to_phys (data->addr);#ifdef CONFIG_CGROUP_PERF --2.31.1
Re: [PATCH v3 06/15] perf/x86/amd: Support PERF_SAMPLE_PHY_ADDR
Quoted 8 lines
Quoted 8 lines
kernel/events/core.c +0
-0@@ -7435,7 +7435,8 @@ void perf_prepare_sample(struct perf_event_header *header, header->size += size; }
Quoted 5 lines
kernel/events/core.c void perf_prepare_sample(struct perf_event_header *header, · line 7439
Quoted 3 lines
kernel/events/core.c +2
-1-if (sample_type &PERF_SAMPLE_PHYS_ADDR ) +if (sample_type &PERF_SAMPLE_PHYS_ADDR && + filtered_sample_type &PERF_SAMPLE_PHYS_ADDR )It'd be enough to check the
filtered_sample_typeonly.
+1. Will fix it.
Thanks,
Ravi
[tip: perf/core] perf/x86/amd: Support PERF_SAMPLE_PHY_ADDR
In reply to Ravi BangoriaThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 5b26af6d2b7854639ddf893366bbca7e74fa7c54 Gitweb: https://git.kernel.org/tip/5b26af6d2b7854639ddf893366bbca7e74fa7c54 Author: Ravi Bangoria <[email protected]> AuthorDate: Wed, 28 Sep 2022 15:27:56 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 29 Sep 2022 12:20:56 +02:00
perf/x86/amd: Support PERF_SAMPLE_PHY_ADDR
IBS_DC_PHYSADDR provides the physical data address for the tagged load/ store operation. Populate perf sample physical address using it.
--- arch/x86/events/amd/ibs.c | 8 +++++++- kernel/events/core.c | 3 ++- 2 files changed, 9 insertions(+), 2 deletions(-)
arch/x86/events/amd/ibs.c +7 -1
@@ -989,6 +989,11 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, data->addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCLINAD )]; data->sample_flags |=PERF_SAMPLE_ADDR ; } + +if (sample_type &PERF_SAMPLE_PHYS_ADDR && op_data3.dc_phy_addr_valid) { + data->phys_addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCPHYSAD )]; + data->sample_flags |=PERF_SAMPLE_PHYS_ADDR ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,
@@ -998,7 +1003,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type, (perf_ibs == &perf_ibs_op && (sample_type &PERF_SAMPLE_DATA_SRC || sample_type &PERF_SAMPLE_WEIGHT_TYPE || - sample_type & PERF_SAMPLE_ADDR))) + sample_type &PERF_SAMPLE_ADDR || + sample_type & PERF_SAMPLE_PHYS_ADDR))) return perf_ibs->offset_max; else if (check_rip) return 3;
kernel/events/core.c +2 -1
@@ -7435,7 +7435,8 @@ void perf_prepare_sample(struct perf_event_header *header, header->size += size; } -if (sample_type &PERF_SAMPLE_PHYS_ADDR ) +if (sample_type &PERF_SAMPLE_PHYS_ADDR && + filtered_sample_type &PERF_SAMPLE_PHYS_ADDR ) data->phys_addr =perf_virt_to_phys (data->addr);#ifdef CONFIG_CGROUP_PERF
Re: [PATCH v3 06/15] perf/x86/amd: Support PERF_SAMPLE_PHY_ADDR
In reply to Ravi Bangoriahi,
there's typo in the subject 'PHYS' ;-)
jirka
Quoted 52 lines · Ravi Bangoria
IBS_DC_PHYSADDRprovides the physical data address for the tagged load/ store operation. Populate perf sample physical address using it.Signed-off-by Ravi Bangoria <[email protected]>--- arch/x86/events/amd/ibs.c | 8 +++++++- kernel/events/core.c | 3 ++- 2 files changed, 9 insertions(+), 2 deletions(-)arch/x86/events/amd/ibs.c +7
-1@@ -989,6 +989,11 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, data->addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCLINAD )]; data->sample_flags |=PERF_SAMPLE_ADDR ; } + +if (sample_type &PERF_SAMPLE_PHYS_ADDR && op_data3.dc_phy_addr_valid) { + data->phys_addr = ibs_data->regs[ibs_op_msr_idx (MSR_AMD64_IBSDCPHYSAD )]; + data->sample_flags |=PERF_SAMPLE_PHYS_ADDR ; + } } static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type,@@ -998,7 +1003,8 @@ static int perf_ibs_get_offset_max(struct perf_ibs *perf_ibs, u64 sample_type, (perf_ibs == &perf_ibs_op && (sample_type &PERF_SAMPLE_DATA_SRC || sample_type &PERF_SAMPLE_WEIGHT_TYPE || - sample_type & PERF_SAMPLE_ADDR))) + sample_type &PERF_SAMPLE_ADDR || + sample_type & PERF_SAMPLE_PHYS_ADDR))) return perf_ibs->offset_max; else if (check_rip) return 3;kernel/events/core.c +2
-1@@ -7435,7 +7435,8 @@ void perf_prepare_sample(struct perf_event_header *header, header->size += size; } -if (sample_type &PERF_SAMPLE_PHYS_ADDR ) +if (sample_type &PERF_SAMPLE_PHYS_ADDR && + filtered_sample_type &PERF_SAMPLE_PHYS_ADDR ) data->phys_addr =perf_virt_to_phys (data->addr);#ifdef CONFIG_CGROUP_PERF Signature · 1 line
2.31.1
PATCH v3 7/15 perf/uapi: Define PERF_MEM_SNOOPX_PEER in kernel header file
In reply to Ravi BangoriaPERF_MEM_SNOOPX_PEER is defined only in tools uapi header. Although it's used only by perf tool, not defining it in kernel header can create problems in future.
--- include/uapi/linux/perf_event.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/uapi/linux/perf_event.h +1 -1
@@ -1356,7 +1356,7 @@ union perf_mem_data_src {#define PERF_MEM_SNOOP_SHIFT 19#define PERF_MEM_SNOOPX_FWD 0x01/* forward */ -/* 1 free */ +#define PERF_MEM_SNOOPX_PEER 0x02/* xfer from peer */ #define PERF_MEM_SNOOPX_SHIFT 38/* locked instruction */
Signature · 1 line
2.31.1
[tip: perf/core] perf/uapi: Define PERF_MEM_SNOOPX_PEER in kernel header file
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: cfef80bad4cf79cdc964a53c98254dfa462be83f Gitweb: https://git.kernel.org/tip/cfef80bad4cf79cdc964a53c98254dfa462be83f Author: Ravi Bangoria <[email protected]> AuthorDate: Wed, 28 Sep 2022 15:27:57 +05:30 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 29 Sep 2022 12:20:56 +02:00
perf/uapi: Define PERF_MEM_SNOOPX_PEER in kernel header file
PERF_MEM_SNOOPX_PEER is defined only in tools uapi header. Although it's used only by perf tool, not defining it in kernel header can create problems in future.
--- include/uapi/linux/perf_event.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/uapi/linux/perf_event.h +1 -1
@@ -1356,7 +1356,7 @@ union perf_mem_data_src {#define PERF_MEM_SNOOP_SHIFT 19#define PERF_MEM_SNOOPX_FWD 0x01/* forward */ -/* 1 free */ +#define PERF_MEM_SNOOPX_PEER 0x02/* xfer from peer */ #define PERF_MEM_SNOOPX_SHIFT 38/* locked instruction */
PATCH v3 8/15 perf tool: Sync include/uapi/linux/perf_event.h header
In reply to Ravi BangoriaTwo new fields for mem_lvl_num has been introduced: PERF_MEM_LVLNUM_IO and PERF_MEM_LVLNUM_EXTN_MEM which are required to support perf mem/c2c on AMD platform.
--- tools/include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
tools/include/uapi/linux/perf_event.h +3 -1
@@ -1295,7 +1295,9 @@ union perf_mem_data_src {#define PERF_MEM_LVLNUM_L2 0x02/* L2 */ #define PERF_MEM_LVLNUM_L3 0x03/* L3 */ #define PERF_MEM_LVLNUM_L4 0x04/* L4 */ -/* 5-0xa available */ +/* 5-0x8 available */ +#define PERF_MEM_LVLNUM_EXTN_MEM 0x09/* Extension memory */ +#define PERF_MEM_LVLNUM_IO 0x0a/* I/O */ #define PERF_MEM_LVLNUM_ANY_CACHE 0x0b/* Any cache */ #define PERF_MEM_LVLNUM_LFB 0x0c/* LFB */ #define PERF_MEM_LVLNUM_RAM 0x0d/* RAM */
Signature · 1 line
2.31.1
PATCH v3 9/15 perf tool: Sync arch/x86/include/asm/amd-ibs.h header
In reply to Ravi BangoriaAlthough new details added into this header is currently used by kernel only, tools copy needs to be in sync with kernel file.
--- tools/arch/x86/include/asm/amd-ibs.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
tools/arch/x86/include/asm/amd-ibs.h +16 -0
@@ -6,6 +6,22 @@#include "msr-index.h" +/* IBS_OP_DATA2 DataSrc */ +#define IBS_DATA_SRC_LOC_CACHE 2 +#define IBS_DATA_SRC_DRAM 3 +#define IBS_DATA_SRC_REM_CACHE 4 +#define IBS_DATA_SRC_IO 7 + +/* IBS_OP_DATA2 DataSrc Extension */ +#define IBS_DATA_SRC_EXT_LOC_CACHE 1 +#define IBS_DATA_SRC_EXT_NEAR_CCX_CACHE 2 +#define IBS_DATA_SRC_EXT_DRAM 3 +#define IBS_DATA_SRC_EXT_FAR_CCX_CACHE 5 +#define IBS_DATA_SRC_EXT_PMEM 6 +#define IBS_DATA_SRC_EXT_IO 7 +#define IBS_DATA_SRC_EXT_EXT_MEM 8 +#define IBS_DATA_SRC_EXT_PEER_AGENT_MEM 12 +/* * IBS Hardware MSRs */
Signature · 1 line
2.31.1
PATCH v3 10/15 perf mem: Add support for printing PERF_MEM_LVLNUM_{EXTN_MEM|IO}
In reply to Ravi BangoriaAdd support for printing these new fields in perf mem report.
--- tools/perf/util/mem-events.c | 2 ++ 1 file changed, 2 insertions(+)
tools/perf/util/mem-events.c +2 -0
@@ -294,6 +294,8 @@ static const char * const mem_lvl[] = { };static const char *const mem_lvlnum[] = { + [PERF_MEM_LVLNUM_EXTN_MEM ] ="Ext Mem" , + [PERF_MEM_LVLNUM_IO ] ="I/O" , [PERF_MEM_LVLNUM_ANY_CACHE ] ="Any cache" , [PERF_MEM_LVLNUM_LFB ] ="LFB" , [PERF_MEM_LVLNUM_RAM ] ="RAM" ,
Signature · 1 line
2.31.1
PATCH v3 11/15 perf mem/c2c: Set PERF_SAMPLE_WEIGHT for LOAD_STORE events
In reply to Ravi BangoriaCurrently perf sets PERF_SAMPLE_WEIGHT flag only for mem load events. Set it for combined load-store event as well which will enable recording of load latency by default on arch that does not support independent mem load event.
Also document missing -W in perf-record man page.
--- tools/perf/Documentation/perf-record.txt | 1 + tools/perf/builtin-c2c.c | 1 + tools/perf/builtin-mem.c | 1 + 3 files changed, 3 insertions(+)
tools/perf/Documentation/perf-record.txt +1 -0
@@ -407,6 +407,7 @@ is enabled for all the sampling events. The sampled branch type is the same for The various filters must be specified as a comma separated list: --branch-filter any_ret,u,k Note that this feature may not be available on all processors. +-W:: --weight:: Enable weightened sampling. An additional weight is recorded per sample and can be displayed with the weight and local_weight sort keys. This currently works for TSX
tools/perf/builtin-c2c.c +1 -0
@@ -3284,6 +3284,7 @@ static int perf_c2c__record(int argc, const char **argv)*/ if (e->tag) { e->record = true; + rec_argv[i++] ="-W" ; }else { e =perf_mem_events__ptr (PERF_MEM_EVENTS__LOAD ); e->record = true;
tools/perf/builtin-mem.c +1 -0
@@ -122,6 +122,7 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem) (mem->operation & MEM_OPERATION_LOAD) && (mem->operation & MEM_OPERATION_STORE)) { e->record = true; + rec_argv[i++] = "-W"; } else {if (mem->operation &MEM_OPERATION_LOAD ) { e =perf_mem_events__ptr (PERF_MEM_EVENTS__LOAD );
Signature · 1 line
2.31.1
PATCH v3 12/15 perf mem/c2c: Add load store event mappings for AMD
In reply to Ravi BangoriaPerf mem and c2c tools are wrappers around perf record with mem load/ store events. IBS tagged load/store sample provides most of the information needed for these tools. Wire in ibs_op// event as mem-ldst event for AMD.
There are some limitations though: Only load/store micro-ops provide mem/c2c information. Whereas, IBS does not have a way to choose a particular type of micro-op to tag. This results in many non-LS micro-ops being tagged which appear as N/A in the perf report. IBS, being an uncore pmu from kernel point of view[1], does not support per process monitoring. Thus, perf mem/c2c on AMD are currently supported in per-cpu mode only.
Example:
$ sudo ./perf mem record -- -c 10000 ^C[ perf record: Woken up 227 times to write data ] [ perf record: Captured and wrote 58.760 MB perf.data (836978 samples) ]
$ sudo ./perf mem report -F mem,sample,snoop Samples: 836K of event 'ibs_op//', Event count (approx.): 8418762 Memory access Samples Snoop N/A 700620 N/A L1 hit 126675 N/A L2 hit 424 N/A L3 hit 664 HitM L3 hit 10 N/A Local RAM hit 2 N/A Remote RAM (1 hop) hit 8558 N/A Remote Cache (1 hop) hit 3 N/A Remote Cache (1 hop) hit 2 HitM Remote Cache (2 hops) hit 10 HitM Remote Cache (2 hops) hit 6 N/A Uncached hit 4 N/A
[1]: https://lore.kernel.org/lkml/[email protected]
--- tools/perf/Documentation/perf-c2c.txt | 14 ++++++++---- tools/perf/Documentation/perf-mem.txt | 3 ++- tools/perf/arch/x86/util/mem-events.c | 31 +++++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 7 deletions(-)
tools/perf/Documentation/perf-c2c.txt +10 -4
@@ -19,9 +19,10 @@ C2C stands for Cache To Cache. The perf c2c tool provides means for Shared Data C2C/HITM analysis. It allows you to track down the cacheline contentions. -On x86, the tool is based on load latency and precise store facility events +On Intel, the tool is based on load latency and precise store facility events provided by Intel CPUs. On PowerPC, the tool uses random instruction sampling -with thresholding feature. +with thresholding feature. On AMD, the tool uses IBS op pmu (due to hardware +limitations, perf c2c is not supported on Zen3 cpus). These events provide: - memory address of the access
@@ -49,7 +50,8 @@ RECORD OPTIONS -l:: --ldlat:: - Configure mem-loads latency. (x86 only) + Configure mem-loads latency. Supported on Intel and Arm64 processors + only. Ignored on other archs. -k:: --all-kernel::
@@ -135,11 +137,15 @@ Following perf record options are configured by default: -W,-d,--phys-data,--sample-cpu Unless specified otherwise with '-e' option, following events are monitored by -default on x86: +default on Intel: cpu/mem-loads,ldlat=30/P cpu/mem-stores/P +following on AMD: + + ibs_op// + and following on PowerPC: cpu/mem-loads/
tools/perf/Documentation/perf-mem.txt +2 -1
@@ -85,7 +85,8 @@ RECORD OPTIONS Be more verbose (show counter open errors, etc) --ldlat <n>:: - Specify desired latency for loads event. (x86 only) + Specify desired latency for loads event. Supported on Intel and Arm64 + processors only. Ignored on other archs. In addition, for report all perf report options are valid, and for record all perf record options.
tools/perf/arch/x86/util/mem-events.c +29 -2
@@ -1,7 +1,9 @@// SPDX-License-Identifier: GPL-2.0 #include "util/pmu.h" +#include "util/env.h" #include "map_symbol.h" #include "mem-events.h" +#include "linux/string.h" static char mem_loads_name[100 ];static bool mem_loads_name__init;
@@ -12,18 +14,43 @@ static char mem_stores_name[100];#define E (t, n, s) { .tag = t, .name = n, .sysfs_name = s } -static struct perf_mem_event perf_mem_events[PERF_MEM_EVENTS__MAX ] = { +static struct perf_mem_event perf_mem_events_intel[PERF_MEM_EVENTS__MAX ] = {E ("ldlat-loads" ,"%s/mem-loads,ldlat=%u/P" ,"%s/events/mem-loads" ),E ("ldlat-stores" ,"%s/mem-stores/P" ,"%s/events/mem-stores" ),E (NULL ,NULL ,NULL ), }; +static struct perf_mem_event perf_mem_events_amd[PERF_MEM_EVENTS__MAX ] = { +E (NULL ,NULL ,NULL ), +E (NULL ,NULL ,NULL ), +E ("mem-ldst" ,"ibs_op//" ,"ibs_op" ), +}; + +static int perf_mem_is_amd_cpu (void ) +{ +struct perf_env env = { .total_mem =0 , }; + +perf_env__cpuid (&env); +if (env.cpuid &&strstarts (env.cpuid,"AuthenticAMD" )) +return 1 ; +return -1 ; +} +struct perf_mem_event *perf_mem_events__ptr (int i) { +/* 0: Uninitialized, 1: Yes, -1: No */ +static int is_amd; +if (i >=PERF_MEM_EVENTS__MAX )return NULL ; -return &perf_mem_events[i]; +if (!is_amd) + is_amd =perf_mem_is_amd_cpu (); + +if (is_amd ==1 ) +return &perf_mem_events_amd[i]; + +return &perf_mem_events_intel[i]; }bool is_mem_loads_aux_event (struct evsel *leader)
Signature · 1 line
2.31.1
PATCH v3 13/15 perf mem/c2c: Avoid printing empty lines for unsupported events
In reply to Ravi BangoriaPerf mem and c2c can be used with 3 different events: load, store and combined load-store. Some architectures might support only partial set of events in which case, perf prints empty line for unsupported events. Avoid that.
Ex, AMD Zen cpus supports only combined load-store event and does not support individual load and store event.
Before patch:
$ ./perf mem record -e list
mem-ldst : available
After patch:
$ ./perf mem record -e list mem-ldst : available
--- tools/perf/util/mem-events.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
tools/perf/util/mem-events.c +6 -5
@@ -156,11 +156,12 @@ void perf_mem_events__list(void)for (j =0 ; j <PERF_MEM_EVENTS__MAX ; j++) {struct perf_mem_event *e =perf_mem_events__ptr (j); -fprintf (stderr,"%-13s%-*s%s\n" , - e->tag ?:"" , - verbose >0 ?25 :0 , - verbose >0 ?perf_mem_events__name (j,NULL ) :"" , - e->supported ?": available" :"" ); +fprintf (stderr,"%-*s%-*s%s" , + e->tag ?13 :0 , + e->tag ? :"" , + e->tag && verbose >0 ?25 :0 , + e->tag && verbose >0 ?perf_mem_events__name (j,NULL ) :"" , + e->supported ?": available\n" :"" ); } }
Signature · 1 line
2.31.1
PATCH v3 14/15 perf mem: Use more generic term for LFB
In reply to Ravi BangoriaA hw component to track outstanding L1 Data Cache misses is called LFB (Line Fill Buffer) on Intel and Arm. However similar component exists on other arch with different names, for ex, it's called MAB (Miss Address Buffer) on AMD. Use 'LFB/MAB' instead of just 'LFB'.
--- tools/perf/util/mem-events.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/mem-events.c +2 -2
@@ -282,7 +282,7 @@ static const char * const mem_lvl[] = {"HIT" ,"MISS" ,"L1" , -"LFB" , +"LFB/MAB" ,"L2" ,"L3" , "Local RAM",
@@ -298,7 +298,7 @@ static const char * const mem_lvlnum[] = { [PERF_MEM_LVLNUM_EXTN_MEM] = "Ext Mem", [PERF_MEM_LVLNUM_IO] = "I/O", [PERF_MEM_LVLNUM_ANY_CACHE] = "Any cache", - [PERF_MEM_LVLNUM_LFB] = "LFB", + [PERF_MEM_LVLNUM_LFB] = "LFB/MAB", [PERF_MEM_LVLNUM_RAM] = "RAM", [PERF_MEM_LVLNUM_PMEM] = "PMEM", [PERF_MEM_LVLNUM_NA] = "N/A",
Signature · 1 line
2.31.1
PATCH v3 15/15 perf script: Add missing fields in usage hint
In reply to Ravi BangoriaFew fields are missing in the usage message printed when wrong field option is passed. Add them in the list.
--- tools/perf/builtin-script.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
tools/perf/builtin-script.c +4 -3
@@ -3844,9 +3844,10 @@ int cmd_script(int argc, const char **argv) "Valid types: hw,sw,trace,raw,synth. " "Fields: comm,tid,pid,time,cpu,event,trace,ip,sym,dso," "addr,symoff,srcline,period,iregs,uregs,brstack," - "brstacksym,flags,bpf-output,brstackinsn,brstackinsnlen,brstackoff," - "callindent,insn,insnlen,synth,phys_addr,metric,misc,ipc,tod," - "data_page_size,code_page_size,ins_lat", + "brstacksym,flags,data_src,weight,bpf-output,brstackinsn," + "brstackinsnlen,brstackoff,callindent,insn,insnlen,synth," + "phys_addr,metric,misc,srccode,ipc,tod,data_page_size," + "code_page_size,ins_lat", parse_output_fields),OPT_BOOLEAN ('a' ,"all-cpus" , &system_wide, "system-wide collection from all CPUs"),
Signature · 1 line
2.31.1