Add DPA->HPA translation to dram & general_media events
[email protected]
maintainerv5 cover Add DPA->HPA translation to dram & general_media events
From: Alison Schofield <[email protected]>
Changes in v5:
- Add Shiyang Ruan's DPA mask fixup patch[1] (Dan)
The DPA flags fixup is a required precursor patch to this set so I updated Ruan's patch based on the last review messages and added it here. That patch can be merged ahead of this set.
- Link to v4: https://lore.kernel.org/[email protected]/
[1] https://lore.kernel.org/[email protected]/
Begin Cover Letter:
Add HPA translations to CXL events: cxl_dram and cxl_general_media
Patch 1:
The DPA mask fix up patch must precede or be part of this set. It fixes the issue where upper 32 bits of DPA are truncated.
Patches 2 & 3:
Before adding the new support, do some housekeeping and move related helpers to the region driver because there is no looking up region related info without CONFIG_CXL_REGION.
Patch 4:
The new functionality is introduced - cxl_dram & cxl_general_media events. Lookup and log the DPA->HPA translation along with the region name and region uuid.
Alison Schofield (4):
cxl/trace: Correct DPA field masks for general_media & dram events cxl/region: Move cxl_dpa_to_region() work to the region driver cxl/region: Move cxl_trace_hpa() work to the region driver cxl/core: Add region info to cxl_general_media and cxl_dram events
drivers/cxl/core/core.h | 14 ++++ drivers/cxl/core/mbox.c | 36 ++++++++-- drivers/cxl/core/memdev.c | 44 ------------- drivers/cxl/core/region.c | 135 ++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/trace.c | 91 ------------------------- drivers/cxl/core/trace.h | 50 ++++++++++---- include/linux/cxl-event.h | 10 +++ 7 files changed, 226 insertions(+), 154 deletions(-)
base-commit: 4cece764965020c22cff7665b18a012006359095
Signature · 1 line
2.37.3
[email protected]
maintainerPATCH v5 1/4 cxl/trace: Correct DPA field masks for general_media & dram events
From: Alison Schofield <[email protected]>
The length of Physical Address in General Media and DRAM event records is 64-bit, so the field mask for extracting the DPA should be 64-bit also, otherwise the trace event reports DPA's with the upper 32 bits of a DPA address masked off. If userspace was doing DPA-to-HPA translations this could lead to incorrect page retirement decisions, but there is no known consumer (like rasdaemon) of this event today.
Use GENMASK_ULL() for CXL_DPA_MASK to get all the DPA address bits.
Tidy up CXL_DPA_FLAGS_MASK by using GENMASK() to only mask the exact flag bits.
These bits are defined as part of the event record physical address descriptions of General Media and DRAM events in CXL Spec 3.1 Section 8.2.9.2 Events.
--- drivers/cxl/core/trace.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/cxl/core/trace.h +2 -2
@@ -253,8 +253,8 @@ TRACE_EVENT(cxl_generic_event,* DRAM Event Record * CXL rev 3.0 section 8.2.9.2.1.2; Table 8-44 */ -#define CXL_DPA_FLAGS_MASK 0x3F -#define CXL_DPA_MASK (~CXL_DPA_FLAGS_MASK) +#define CXL_DPA_FLAGS_MASK GENMASK(1, 0) +#define CXL_DPA_MASK GENMASK_ULL(63, 6)#define CXL_DPA_VOLATILE BIT(0)#define CXL_DPA_NOT_REPAIRABLE BIT(1)
Signature · 1 line
2.37.3
Re: [PATCH v5 1/4] cxl/trace: Correct DPA field masks for general_media & dram events
Quoted 15 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
The length of Physical Address in General Media and DRAM event records is 64-bit, so the field mask for extracting the DPA should be 64-bit also, otherwise the trace event reports DPA's with the upper 32 bits of a DPA address masked off. If userspace was doing DPA-to-HPA translations this could lead to incorrect page retirement decisions, but there is no known consumer (like rasdaemon) of this event today.
Use
GENMASK_ULL()forCXL_DPA_MASKto get all the DPA address bits.Tidy up
CXL_DPA_FLAGS_MASKby usingGENMASK()to only mask the exact flag bits.
Quoted 7 lines
These bits are defined as part of the event record physical address descriptions of General Media and DRAM events in CXL Spec 3.1 Section 8.2.9.2 Events.
Co-developed-by Shiyang Ruan <[email protected]>Signed-off-by Shiyang Ruan <[email protected]>Signed-off-by Alison Schofield <[email protected]>
Re: [PATCH v5 1/4] cxl/trace: Correct DPA field masks for general_media & dram events
In reply to [email protected]Quoted 9 lines · On Mon, 29 Apr 2024 17:34:21 -0700 [email protected] wrote:
From: Alison Schofield <[email protected]>
The length of Physical Address in General Media and DRAM event records is 64-bit, so the field mask for extracting the DPA should be 64-bit also, otherwise the trace event reports DPA's with the upper 32 bits of a DPA address masked off. If userspace was doing DPA-to-HPA translations this could lead to incorrect page retirement decisions, but there is no known consumer (like rasdaemon) of this event today.
https://github.com/mchehab/rasdaemon/blob/master/ras-cxl-handler.c#L205
Yes there is, though there may well not have been back in at v1.
Quoted 6 lines
Use
GENMASK_ULL()forCXL_DPA_MASKto get all the DPA address bits.Tidy up
CXL_DPA_FLAGS_MASKby usingGENMASK()to only mask the exact flag bits.
Quoted 7 lines
These bits are defined as part of the event record physical address descriptions of General Media and DRAM events in CXL Spec 3.1 Section 8.2.9.2 Events.
Co-developed-by Shiyang Ruan <[email protected]>Signed-off-by Shiyang Ruan <[email protected]>Signed-off-by Alison Schofield <[email protected]>
Fixes tag would probably be appropriate I think as this should be backported.
Quoted 19 lines
--- drivers/cxl/core/trace.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/cxl/core/trace.h +2
-2@@ -253,8 +253,8 @@ TRACE_EVENT(cxl_generic_event,* DRAM Event Record * CXL rev 3.0 section 8.2.9.2.1.2; Table 8-44 */ -#define CXL_DPA_FLAGS_MASK 0x3F -#define CXL_DPA_MASK (~CXL_DPA_FLAGS_MASK) +#define CXL_DPA_FLAGS_MASK GENMASK(1, 0) +#define CXL_DPA_MASK GENMASK_ULL(63, 6)#define CXL_DPA_VOLATILE BIT(0)#define CXL_DPA_NOT_REPAIRABLE BIT(1)
[email protected]
maintainerPATCH v5 2/4 cxl/region: Move cxl_dpa_to_region() work to the region driver
In reply to [email protected]From: Alison Schofield <[email protected]>
This helper belongs in the region driver as it is only useful with CONFIG_CXL_REGION. Add a stub in core.h for when the region driver is not built.
--- drivers/cxl/core/core.h | 7 +++++++ drivers/cxl/core/memdev.c | 44 --------------------------------------- drivers/cxl/core/region.c | 44 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 51 insertions(+), 44 deletions(-)
drivers/cxl/core/core.h +7 -0
@@ -27,7 +27,14 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled);int cxl_region_init (void );void cxl_region_exit (void );int cxl_get_poison_by_endpoint (struct cxl_port *port); +struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa); +#else +static inline +struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa) +{ +return NULL ; +}static inline int cxl_get_poison_by_endpoint (struct cxl_port *port) {return 0 ;
drivers/cxl/core/memdev.c +0 -44
50 lines not on this page · raw message
drivers/cxl/core/region.c +44 -0
50 lines not on this page · raw message
Signature · 1 line
2.37.3
[email protected]
maintainerPATCH v5 3/4 cxl/region: Move cxl_trace_hpa() work to the region driver
In reply to [email protected]From: Alison Schofield <[email protected]>
This work belongs in the region driver as it is only useful with CONFIG_CXL_REGION. Add a stub in core.h for when the region driver is not built.
--- drivers/cxl/core/core.h | 7 +++ drivers/cxl/core/region.c | 91 +++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/trace.c | 91 --------------------------------------- drivers/cxl/core/trace.h | 2 - 4 files changed, 98 insertions(+), 93 deletions(-)
drivers/cxl/core/core.h +7 -0
@@ -28,8 +28,15 @@ int cxl_region_init(void);void cxl_region_exit (void );int cxl_get_poison_by_endpoint (struct cxl_port *port);struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa); +u64 cxl_trace_hpa (struct cxl_region *cxlr,const struct cxl_memdev *cxlmd, +u64 dpa);#else +static inline u64 +cxl_trace_hpa (struct cxl_region *cxlr,const struct cxl_memdev *cxlmd,u64 dpa) +{ +return ULLONG_MAX ; +}static inline struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa) {
drivers/cxl/core/region.c +91 -0
97 lines not on this page · raw message
drivers/cxl/core/trace.c +0 -91
94 lines not on this page · raw message
drivers/cxl/core/trace.h +0 -2
@@ -642,8 +642,6 @@ TRACE_EVENT(cxl_memory_module,#define cxl_poison_overflow (flags, time) \ (flags & CXL_POISON_FLAG_OVERFLOW ? le64_to_cpu(time) : 0) -u64 cxl_trace_hpa (struct cxl_region *cxlr,struct cxl_memdev *memdev,u64 dpa); - TRACE_EVENT(cxl_poison, TP_PROTO(struct cxl_memdev *cxlmd, struct cxl_region *cxlr,
Signature · 1 line
2.37.3
Re: [PATCH v5 3/4] cxl/region: Move cxl_trace_hpa() work to the region driver
Quoted 9 lines · On Mon, 29 Apr 2024 17:34:23 -0700 [email protected] wrote:
From: Alison Schofield <[email protected]>
This work belongs in the region driver as it is only useful with
CONFIG_CXL_REGION. Add a stub incore.hfor when the region driver is not built.Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Ira Weiny <[email protected]>
No blank lines in a tag block. Linux next bot will complain and it can mess up some parsing tools.
Quoted 242 lines
Signed-off-by Alison Schofield <[email protected]>--- drivers/cxl/core/core.h | 7 +++ drivers/cxl/core/region.c | 91 +++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/trace.c | 91 --------------------------------------- drivers/cxl/core/trace.h | 2 - 4 files changed, 98 insertions(+), 93 deletions(-)drivers/cxl/core/core.h +7
-0@@ -28,8 +28,15 @@ int cxl_region_init(void);void cxl_region_exit (void );int cxl_get_poison_by_endpoint (struct cxl_port *port);struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa); +u64 cxl_trace_hpa (struct cxl_region *cxlr,const struct cxl_memdev *cxlmd, +u64 dpa);#else +static inline u64 +cxl_trace_hpa (struct cxl_region *cxlr,const struct cxl_memdev *cxlmd,u64 dpa) +{ +return ULLONG_MAX ; +}static inline struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa) {drivers/cxl/core/region.c +91
-0@@ -2723,6 +2723,97 @@ struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa)return ctx.cxlr; } +static bool cxl_is_hpa_in_range (u64 hpa,struct cxl_region *cxlr,int pos) +{ +struct cxl_region_params *p = &cxlr->params; +int gran = p->interleave_granularity; +int ways = p->interleave_ways; +u64 offset; + +/* Is the hpa within this region at all */ +if (hpa < p->res->start || hpa > p->res->end) { +dev_dbg (&cxlr->dev, +"Addr trans fail: hpa 0x%llx not in region\n" , hpa); +return false; + } + +/* Is the hpa in an expected chunk for its pos(-ition) */ + offset = hpa - p->res->start; + offset =do_div (offset, gran * ways); +if ((offset >= pos * gran) && (offset < (pos +1 ) * gran)) +return true; + +dev_dbg (&cxlr->dev, +"Addr trans fail: hpa 0x%llx not in expected chunk\n" , hpa); + +return false; +} + +static u64 cxl_dpa_to_hpa (u64 dpa,struct cxl_region *cxlr, +struct cxl_endpoint_decoder *cxled) +{ +u64 dpa_offset, hpa_offset, bits_upper, mask_upper, hpa; +struct cxl_region_params *p = &cxlr->params; +int pos = cxled->pos; +u16 eig =0 ; +u8 eiw =0 ; + +ways_to_eiw (p->interleave_ways, &eiw); +granularity_to_eig (p->interleave_granularity, &eig); + +/* +* The device position in the region interleave set was removed +* from the offset at HPA->DPA translation. To reconstruct the +* HPA, place the 'pos' in the offset. +* +* The placement of 'pos' in the HPA is determined by interleave +* ways and granularity and is defined in the CXL Spec 3.0 Section +* 8.2.4.19.13 Implementation Note: Device Decode Logic +*/ + +/* Remove the dpa base */ + dpa_offset = dpa -cxl_dpa_resource_start (cxled); + + mask_upper =GENMASK_ULL (51 , eig +8 ); + +if (eiw <8 ) { + hpa_offset = (dpa_offset & mask_upper) << eiw; + hpa_offset |= pos << (eig +8 ); + }else { + bits_upper = (dpa_offset & mask_upper) >> (eig +8 ); + bits_upper = bits_upper *3 ; + hpa_offset = ((bits_upper << (eiw -8 )) + pos) << (eig +8 ); + } + +/* The lower bits remain unchanged */ + hpa_offset |= dpa_offset &GENMASK_ULL (eig +7 ,0 ); + +/* Apply the hpa_offset to the region base address */ + hpa = hpa_offset + p->res->start; + +if (!cxl_is_hpa_in_range (hpa, cxlr, cxled->pos)) +return ULLONG_MAX ; + +return hpa; +} + +u64 cxl_trace_hpa (struct cxl_region *cxlr,const struct cxl_memdev *cxlmd, +u64 dpa) +{ +struct cxl_region_params *p = &cxlr->params; +struct cxl_endpoint_decoder *cxled =NULL ; + +for (int i =0 ; i < p->nr_targets; i++) { + cxled = p->targets[i]; +if (cxlmd ==cxled_to_memdev (cxled)) +break ; + } +if (!cxled || cxlmd !=cxled_to_memdev (cxled)) +return ULLONG_MAX ; + +return cxl_dpa_to_hpa (dpa, cxlr, cxled); +} +static struct lock_class_key cxl_pmem_region_key;static struct cxl_pmem_region *cxl_pmem_region_alloc (struct cxl_region *cxlr)drivers/cxl/core/trace.c +0
-91@@ -6,94 +6,3 @@#define CREATE_TRACE_POINTS #include "trace.h" - -static bool cxl_is_hpa_in_range (u64 hpa,struct cxl_region *cxlr,int pos) -{ -struct cxl_region_params *p = &cxlr->params; -int gran = p->interleave_granularity; -int ways = p->interleave_ways; -u64 offset; - -/* Is the hpa within this region at all */ -if (hpa < p->res->start || hpa > p->res->end) { -dev_dbg (&cxlr->dev, -"Addr trans fail: hpa 0x%llx not in region\n" , hpa); -return false; - } - -/* Is the hpa in an expected chunk for its pos(-ition) */ - offset = hpa - p->res->start; - offset =do_div (offset, gran * ways); -if ((offset >= pos * gran) && (offset < (pos +1 ) * gran)) -return true; - -dev_dbg (&cxlr->dev, -"Addr trans fail: hpa 0x%llx not in expected chunk\n" , hpa); - -return false; -} - -static u64 cxl_dpa_to_hpa (u64 dpa,struct cxl_region *cxlr, -struct cxl_endpoint_decoder *cxled) -{ -u64 dpa_offset, hpa_offset, bits_upper, mask_upper, hpa; -struct cxl_region_params *p = &cxlr->params; -int pos = cxled->pos; -u16 eig =0 ; -u8 eiw =0 ; - -ways_to_eiw (p->interleave_ways, &eiw); -granularity_to_eig (p->interleave_granularity, &eig); - -/* -* The device position in the region interleave set was removed -* from the offset at HPA->DPA translation. To reconstruct the -* HPA, place the 'pos' in the offset. -* -* The placement of 'pos' in the HPA is determined by interleave -* ways and granularity and is defined in the CXL Spec 3.0 Section -* 8.2.4.19.13 Implementation Note: Device Decode Logic -*/ - -/* Remove the dpa base */ - dpa_offset = dpa -cxl_dpa_resource_start (cxled); - - mask_upper =GENMASK_ULL (51 , eig +8 ); - -if (eiw <8 ) { - hpa_offset = (dpa_offset & mask_upper) << eiw; - hpa_offset |= pos << (eig +8 ); - }else { - bits_upper = (dpa_offset & mask_upper) >> (eig +8 ); - bits_upper = bits_upper *3 ; - hpa_offset = ((bits_upper << (eiw -8 )) + pos) << (eig +8 ); - } - -/* The lower bits remain unchanged */ - hpa_offset |= dpa_offset &GENMASK_ULL (eig +7 ,0 ); - -/* Apply the hpa_offset to the region base address */ - hpa = hpa_offset + p->res->start; - -if (!cxl_is_hpa_in_range (hpa, cxlr, cxled->pos)) -return ULLONG_MAX ; - -return hpa; -} - -u64 cxl_trace_hpa (struct cxl_region *cxlr,struct cxl_memdev *cxlmd, -u64 dpa) -{ -struct cxl_region_params *p = &cxlr->params; -struct cxl_endpoint_decoder *cxled =NULL ; - -for (int i =0 ; i < p->nr_targets; i++) { - cxled = p->targets[i]; -if (cxlmd ==cxled_to_memdev (cxled)) -break ; - } -if (!cxled || cxlmd !=cxled_to_memdev (cxled)) -return ULLONG_MAX ; - -return cxl_dpa_to_hpa (dpa, cxlr, cxled); -}drivers/cxl/core/trace.h +0
-2@@ -642,8 +642,6 @@ TRACE_EVENT(cxl_memory_module,#define cxl_poison_overflow (flags, time) \ (flags & CXL_POISON_FLAG_OVERFLOW ? le64_to_cpu(time) : 0) -u64 cxl_trace_hpa (struct cxl_region *cxlr,struct cxl_memdev *memdev,u64 dpa); - TRACE_EVENT(cxl_poison, TP_PROTO(struct cxl_memdev *cxlmd, struct cxl_region *cxlr,
[email protected]
maintainerPATCH v5 4/4 cxl/core: Add region info to cxl_general_media and cxl_dram events
In reply to [email protected]From: Alison Schofield <[email protected]>
User space may need to know which region, if any, maps the DPAs (device physical addresses) reported in a cxl_general_media or cxl_dram event. Since the mapping can change, the kernel provides this information at the time the event occurs. This informs user space that at event <timestamp> this <region> mapped this <DPA> to this <HPA>.
Add the same region info that is included in the cxl_poison trace
event: the DPA->HPA translation, region name, and region uuid.
The new fields are inserted in the trace event and no existing fields are modified. If the DPA is not mapped, user will see:
hpa=ULLONG_MAX, region="", and uuid=0
This work must be protected by dpa_rwsem & region_rwsem since it is looking up region mappings.
--- drivers/cxl/core/mbox.c | 36 ++++++++++++++++++++++++++------ drivers/cxl/core/trace.h | 44 +++++++++++++++++++++++++++++++-------- include/linux/cxl-event.h | 10 +++++++++ 3 files changed, 75 insertions(+), 15 deletions(-)
drivers/cxl/core/mbox.c +30 -6
@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,enum cxl_event_type event_type, const uuid_t *uuid, union cxl_event *evt) { - if (event_type == CXL_CPER_EVENT_GEN_MEDIA) - trace_cxl_general_media(cxlmd, type, &evt->gen_media); - else if (event_type == CXL_CPER_EVENT_DRAM) - trace_cxl_dram(cxlmd, type, &evt->dram); - else if (event_type == CXL_CPER_EVENT_MEM_MODULE) + if (event_type == CXL_CPER_EVENT_MEM_MODULE) { trace_cxl_memory_module(cxlmd, type, &evt->mem_module); - else +return ; + } +if (event_type ==CXL_CPER_EVENT_GENERIC ) {trace_cxl_generic_event (cxlmd, type, uuid, &evt->generic); +return ; + } + +if (trace_cxl_general_media_enabled () ||trace_cxl_dram_enabled ()) { +u64 dpa, hpa =ULLONG_MAX ; +struct cxl_region *cxlr; + +/* +* These trace points are annotated with HPA and region +* translations. Take topology mutation locks and lookup +* { HPA, REGION } from { DPA, MEMDEV } in the event record. +*/ +guard (rwsem_read)(&cxl_region_rwsem); +guard (rwsem_read)(&cxl_dpa_rwsem); + + dpa =le64_to_cpu (evt->common.phys_addr) &CXL_DPA_MASK ; + cxlr =cxl_dpa_to_region (cxlmd, dpa); +if (cxlr) + hpa =cxl_trace_hpa (cxlr, cxlmd, dpa); + +if (event_type ==CXL_CPER_EVENT_GEN_MEDIA ) +trace_cxl_general_media (cxlmd, type, cxlr, hpa, + &evt->gen_media); +else if (event_type ==CXL_CPER_EVENT_DRAM ) +trace_cxl_dram (cxlmd, type, cxlr, hpa, &evt->dram); + } }EXPORT_SYMBOL_NS_GPL (cxl_event_trace_record,CXL );
drivers/cxl/core/trace.h +35 -9
110 lines not on this page · raw message
include/linux/cxl-event.h +10 -0
@@ -91,11 +91,21 @@ struct cxl_event_mem_module {u8 reserved[0x3d ]; } __packed; +/* +* General Media or DRAM Event Common Fields +* - provides common access to phys_addr +*/ +struct cxl_event_common { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +} __packed; +union cxl_event {struct cxl_event_generic generic;struct cxl_event_gen_media gen_media;struct cxl_event_dram dram;struct cxl_event_mem_module mem_module; +struct cxl_event_common common; } __packed;/*
Signature · 1 line
2.37.3
Re: [PATCH v5 4/4] cxl/core: Add region info to cxl_general_media and cxl_dram events
Quoted 2 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
[snip]
Quoted 6 lines
include/linux/cxl-event.h +0
-0@@ -91,11 +91,21 @@ struct cxl_event_mem_module {u8 reserved[0x3d ];
include/linux/cxl-event.h struct cxl_event_mem_module { · line 101
Quoted 10 lines
include/linux/cxl-event.h +8
-0} __packed; +/* +* General Media or DRAM Event Common Fields +* - provides common access to phys_addr +*/ +struct cxl_event_common { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +} __packed;
Was this left over from a previous version?
I don't see it used. Nor is it a defined event in the spec AFAIK.
The rest looks good.
Ira
Quoted 13 lines
include/linux/cxl-event.h +2
-0+union cxl_event {struct cxl_event_generic generic;struct cxl_event_gen_media gen_media;struct cxl_event_dram dram;struct cxl_event_mem_module mem_module; +struct cxl_event_common common; } __packed;/* Signature · 1 line
2.37.3
Alison Schofield
maintainerRe: [PATCH v5 4/4] cxl/core: Add region info to cxl_general_media and cxl_dram events
Quoted 25 lines · Ira Weiny
Quoted 2 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
[snip]
Quoted 6 lines
include/linux/cxl-event.h +0
-0@@ -91,11 +91,21 @@ struct cxl_event_mem_module {u8 reserved[0x3d ];include/linux/cxl-event.h struct cxl_event_mem_module { · line 101
Quoted 10 lines
include/linux/cxl-event.h +8
-0} __packed; +/* +* General Media or DRAM Event Common Fields +* - provides common access to phys_addr +*/ +struct cxl_event_common { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +} __packed;Was this left over from a previous version?
Quoted 2 lines
I don't see it used. Nor is it a defined event in the spec AFAIK.
I introduced it in v4. It is for the 'convenience' of getting at that common phys_addr field. You snipped the usage away. I'll paste it back here:
drivers/cxl/core/mbox.c +0 -0
@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,
enum cxl_event_type event_type,
const uuid_t *uuid, union cxl_event *evt)
{
my snip...
+ if (trace_cxl_general_media_enabled() || trace_cxl_dram_enabled()) {
+ u64 dpa, hpa = ULLONG_MAX;
+ struct cxl_region *cxlr;
+
+ /*
+ * These trace points are annotated with HPA and region
+ * translations. Take topology mutation locks and lookup
+ * { HPA, REGION } from { DPA, MEMDEV } in the event record.
+ */
+ guard(rwsem_read)(&cxl_region_rwsem);
+ guard(rwsem_read)(&cxl_dpa_rwsem);
+
+ dpa = le64_to_cpu(evt->common.phys_addr) & CXL_DPA_MASK;
Used above. OK?
+ cxlr = cxl_dpa_to_region(cxlmd, dpa);
+ if (cxlr)
+ hpa = cxl_trace_hpa(cxlr, cxlmd, dpa);
+
+ if (event_type == CXL_CPER_EVENT_GEN_MEDIA)
+ trace_cxl_general_media(cxlmd, type, cxlr, hpa,
+ &evt->gen_media);
+ else if (event_type == CXL_CPER_EVENT_DRAM)
+ trace_cxl_dram(cxlmd, type, cxlr, hpa, &evt->dram);
+ }
} EXPORT_SYMBOL_NS_GPL(cxl_event_trace_record, CXL);
Quoted 19 lines
The rest looks good.
Ira
Quoted 13 lines
include/linux/cxl-event.h +2
-0+union cxl_event {struct cxl_event_generic generic;struct cxl_event_gen_media gen_media;struct cxl_event_dram dram;struct cxl_event_mem_module mem_module; +struct cxl_event_common common; } __packed;/* Signature · 1 line
2.37.3
Re: [PATCH v5 4/4] cxl/core: Add region info to cxl_general_media and cxl_dram events
Quoted 56 lines · Alison Schofield
Quoted 25 lines · Ira Weiny
Quoted 2 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
[snip]
Quoted 6 lines
include/linux/cxl-event.h +0
-0@@ -91,11 +91,21 @@ struct cxl_event_mem_module {u8 reserved[0x3d ];include/linux/cxl-event.h struct cxl_event_mem_module { · line 101
Quoted 10 lines
include/linux/cxl-event.h +8
-0} __packed; +/* +* General Media or DRAM Event Common Fields +* - provides common access to phys_addr +*/ +struct cxl_event_common { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +} __packed;Was this left over from a previous version?
Quoted 2 lines
I don't see it used. Nor is it a defined event in the spec AFAIK.
I introduced it in v4. It is for the 'convenience' of getting at that common
phys_addrfield. You snipped the usage away. I'll paste it back here:drivers/cxl/core/mbox.c +0
-0@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,enum cxl_event_type event_type, const uuid_t *uuid, union cxl_event *evt) {my snip...
drivers/cxl/core/mbox.c +13
-0+if (trace_cxl_general_media_enabled () ||trace_cxl_dram_enabled ()) { +u64 dpa, hpa =ULLONG_MAX ; +struct cxl_region *cxlr; + +/* +* These trace points are annotated with HPA and region +* translations. Take topology mutation locks and lookup +* { HPA, REGION } from { DPA, MEMDEV } in the event record. +*/ +guard (rwsem_read)(&cxl_region_rwsem); +guard (rwsem_read)(&cxl_dpa_rwsem); + + dpa =le64_to_cpu (evt->common.phys_addr) &CXL_DPA_MASK ;
Quoted 1 line
Used above. OK?
Ah sneaky. It was not clear to me from your comment why this new 'event' existed.
Ira
Quoted 21 lines
drivers/cxl/core/mbox.c +10
-0+ cxlr =cxl_dpa_to_region (cxlmd, dpa); +if (cxlr) + hpa =cxl_trace_hpa (cxlr, cxlmd, dpa); + +if (event_type ==CXL_CPER_EVENT_GEN_MEDIA ) +trace_cxl_general_media (cxlmd, type, cxlr, hpa, + &evt->gen_media); +else if (event_type ==CXL_CPER_EVENT_DRAM ) +trace_cxl_dram (cxlmd, type, cxlr, hpa, &evt->dram); + } }EXPORT_SYMBOL_NS_GPL (cxl_event_trace_record,CXL );Quoted 8 lines
The rest looks good.
Ira
Ira Weiny
Reviewed-byRe: [PATCH v5 4/4] cxl/core: Add region info to cxl_general_media and cxl_dram events
Quoted 57 lines · Ira Weiny
Quoted 56 lines · Alison Schofield
Quoted 25 lines · Ira Weiny
Quoted 2 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
[snip]
Quoted 6 lines
include/linux/cxl-event.h +0
-0@@ -91,11 +91,21 @@ struct cxl_event_mem_module {u8 reserved[0x3d ];include/linux/cxl-event.h struct cxl_event_mem_module { · line 101
Quoted 10 lines
include/linux/cxl-event.h +8
-0} __packed; +/* +* General Media or DRAM Event Common Fields +* - provides common access to phys_addr +*/ +struct cxl_event_common { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +} __packed;Was this left over from a previous version?
Quoted 2 lines
I don't see it used. Nor is it a defined event in the spec AFAIK.
I introduced it in v4. It is for the 'convenience' of getting at that common
phys_addrfield. You snipped the usage away. I'll paste it back here:drivers/cxl/core/mbox.c +0
-0@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,enum cxl_event_type event_type, const uuid_t *uuid, union cxl_event *evt) {my snip...
drivers/cxl/core/mbox.c +13
-0+if (trace_cxl_general_media_enabled () ||trace_cxl_dram_enabled ()) { +u64 dpa, hpa =ULLONG_MAX ; +struct cxl_region *cxlr; + +/* +* These trace points are annotated with HPA and region +* translations. Take topology mutation locks and lookup +* { HPA, REGION } from { DPA, MEMDEV } in the event record. +*/ +guard (rwsem_read)(&cxl_region_rwsem); +guard (rwsem_read)(&cxl_dpa_rwsem); + + dpa =le64_to_cpu (evt->common.phys_addr) &CXL_DPA_MASK ;
Quoted 4 lines
Quoted 1 line
Used above. OK?
Ah sneaky. It was not clear to me from your comment why this new 'event' existed.
Almost forgot.
Re: [PATCH v5 4/4] cxl/core: Add region info to cxl_general_media and cxl_dram events
In reply to [email protected]Quoted 16 lines · On Mon, 29 Apr 2024 17:34:24 -0700 [email protected] wrote:
From: Alison Schofield <[email protected]>
User space may need to know which region, if any, maps the DPAs (device physical addresses) reported in a
cxl_general_mediaorcxl_dramevent. Since the mapping can change, the kernel provides this information at the time the event occurs. This informs user space that at event <timestamp> this <region> mapped this <DPA> to this <HPA>.Add the same region info that is included in the
cxl_poisontrace
event: theDPA->HPAtranslation, region name, and region uuid.The new fields are inserted in the trace event and no existing fields are modified. If the DPA is not mapped, user will see:
hpa=ULLONG_MAX, region="", and uuid=0
Quoted 5 lines
This work must be protected by
dpa_rwsem®ion_rwsemsince it is looking up region mappings.Signed-off-by Alison Schofield <[email protected]>Reviewed-by Dan Williams <[email protected]>
+CC Shiju: Not sure if this one is on your radar for rasdaemon updates.
LGTM
Quoted 203 lines
--- drivers/cxl/core/mbox.c | 36 ++++++++++++++++++++++++++------ drivers/cxl/core/trace.h | 44 +++++++++++++++++++++++++++++++-------- include/linux/cxl-event.h | 10 +++++++++ 3 files changed, 75 insertions(+), 15 deletions(-)drivers/cxl/core/mbox.c +30
-6@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,enum cxl_event_type event_type, const uuid_t *uuid, union cxl_event *evt) { - if (event_type == CXL_CPER_EVENT_GEN_MEDIA) - trace_cxl_general_media(cxlmd, type, &evt->gen_media); - else if (event_type == CXL_CPER_EVENT_DRAM) - trace_cxl_dram(cxlmd, type, &evt->dram); - else if (event_type == CXL_CPER_EVENT_MEM_MODULE) + if (event_type == CXL_CPER_EVENT_MEM_MODULE) { trace_cxl_memory_module(cxlmd, type, &evt->mem_module); - else +return ; + } +if (event_type ==CXL_CPER_EVENT_GENERIC ) {trace_cxl_generic_event (cxlmd, type, uuid, &evt->generic); +return ; + } + +if (trace_cxl_general_media_enabled () ||trace_cxl_dram_enabled ()) { +u64 dpa, hpa =ULLONG_MAX ; +struct cxl_region *cxlr; + +/* +* These trace points are annotated with HPA and region +* translations. Take topology mutation locks and lookup +* { HPA, REGION } from { DPA, MEMDEV } in the event record. +*/ +guard (rwsem_read)(&cxl_region_rwsem); +guard (rwsem_read)(&cxl_dpa_rwsem); + + dpa =le64_to_cpu (evt->common.phys_addr) &CXL_DPA_MASK ; + cxlr =cxl_dpa_to_region (cxlmd, dpa); +if (cxlr) + hpa =cxl_trace_hpa (cxlr, cxlmd, dpa); + +if (event_type ==CXL_CPER_EVENT_GEN_MEDIA ) +trace_cxl_general_media (cxlmd, type, cxlr, hpa, + &evt->gen_media); +else if (event_type ==CXL_CPER_EVENT_DRAM ) +trace_cxl_dram (cxlmd, type, cxlr, hpa, &evt->dram); + } }EXPORT_SYMBOL_NS_GPL (cxl_event_trace_record,CXL );drivers/cxl/core/trace.h +35
-9@@ -316,9 +316,9 @@ TRACE_EVENT(cxl_generic_event, TRACE_EVENT(cxl_general_media, TP_PROTO(const struct cxl_memdev *cxlmd, enum cxl_event_log_type log, - struct cxl_event_gen_media *rec), + struct cxl_region *cxlr, u64 hpa, struct cxl_event_gen_media *rec), - TP_ARGS(cxlmd, log, rec), + TP_ARGS(cxlmd, log, cxlr, hpa, rec), TP_STRUCT__entry( CXL_EVT_TP_entry@@ -330,10 +330,13 @@ TRACE_EVENT(cxl_general_media, __field(u8, channel) __field(u32, device)__array (u8, comp_id,CXL_EVENT_GEN_MED_COMP_ID_SIZE ) -__field (u16, validity_flags) /* Following are out of order to pack trace record */ + __field(u64, hpa) + __field_struct(uuid_t, region_uuid) + __field(u16, validity_flags) __field(u8, rank) __field(u8, dpa_flags) +__string (region_name, cxlr ?dev_name (&cxlr->dev) :"" ) ), TP_fast_assign(@@ -354,18 +357,28 @@ TRACE_EVENT(cxl_general_media,memcpy (__entry->comp_id, &rec->component_id,CXL_EVENT_GEN_MED_COMP_ID_SIZE ); __entry->validity_flags =get_unaligned_le16 (&rec->validity_flags); + __entry->hpa = hpa; +if (cxlr) { +__assign_str (region_name,dev_name (&cxlr->dev)); +uuid_copy (&__entry->region_uuid, &cxlr->params.uuid); + }else { +__assign_str (region_name,"" ); +uuid_copy (&__entry->region_uuid, &uuid_null); + } ),CXL_EVT_TP_printk ("dpa=%llx dpa_flags='%s' " \"descriptor='%s' type='%s' transaction_type='%s' channel=%u rank=%u " \ -"device=%x comp_id=%s validity_flags='%s'" , +"device=%x comp_id=%s validity_flags='%s' " \ +"hpa=%llx region=%s region_uuid=%pUb" , __entry->dpa,show_dpa_flags (__entry->dpa_flags),show_event_desc_flags (__entry->descriptor),show_mem_event_type (__entry->type),show_trans_type (__entry->transaction_type), __entry->channel, __entry->rank, __entry->device,__print_hex (__entry->comp_id,CXL_EVENT_GEN_MED_COMP_ID_SIZE ), -show_valid_flags (__entry->validity_flags) +show_valid_flags (__entry->validity_flags), + __entry->hpa,__get_str (region_name), &__entry->region_uuid ) );@@ -400,9 +413,9 @@ TRACE_EVENT(cxl_general_media, TRACE_EVENT(cxl_dram, TP_PROTO(const struct cxl_memdev *cxlmd, enum cxl_event_log_type log, - struct cxl_event_dram *rec), + struct cxl_region *cxlr, u64 hpa, struct cxl_event_dram *rec), - TP_ARGS(cxlmd, log, rec), + TP_ARGS(cxlmd, log, cxlr, hpa, rec), TP_STRUCT__entry( CXL_EVT_TP_entry@@ -417,10 +430,13 @@ TRACE_EVENT(cxl_dram, __field(u32, nibble_mask) __field(u32, row) __array(u8, cor_mask, CXL_EVENT_DER_CORRECTION_MASK_SIZE) + __field(u64, hpa) + __field_struct(uuid_t, region_uuid) __field(u8, rank) /* Out of order to pack trace record */ __field(u8, bank_group) /* Out of order to pack trace record */ __field(u8, bank) /* Out of order to pack trace record */ __field(u8, dpa_flags) /* Out of order to pack trace record */ +__string (region_name, cxlr ?dev_name (&cxlr->dev) :"" ) ), TP_fast_assign(@@ -444,12 +460,21 @@ TRACE_EVENT(cxl_dram, __entry->column =get_unaligned_le16 (rec->column);memcpy (__entry->cor_mask, &rec->correction_mask,CXL_EVENT_DER_CORRECTION_MASK_SIZE ); + __entry->hpa = hpa; +if (cxlr) { +__assign_str (region_name,dev_name (&cxlr->dev)); +uuid_copy (&__entry->region_uuid, &cxlr->params.uuid); + }else { +__assign_str (region_name,"" ); +uuid_copy (&__entry->region_uuid, &uuid_null); + } ), CXL_EVT_TP_printk("dpa=%llx dpa_flags='%s' descriptor='%s' type='%s' " \ "transaction_type='%s' channel=%u rank=%u nibble_mask=%x " \ "bank_group=%u bank=%u row=%u column=%u cor_mask=%s " \ - "validity_flags='%s'", + "validity_flags='%s' " \ + "hpa=%llx region=%s region_uuid=%pUb", __entry->dpa, show_dpa_flags(__entry->dpa_flags), show_event_desc_flags(__entry->descriptor), show_mem_event_type(__entry->type),@@ -458,7 +483,8 @@ TRACE_EVENT(cxl_dram, __entry->bank_group, __entry->bank, __entry->row, __entry->column,__print_hex (__entry->cor_mask,CXL_EVENT_DER_CORRECTION_MASK_SIZE ), -show_dram_valid_flags (__entry->validity_flags) +show_dram_valid_flags (__entry->validity_flags), + __entry->hpa,__get_str (region_name), &__entry->region_uuid ) );include/linux/cxl-event.h +10
-0@@ -91,11 +91,21 @@ struct cxl_event_mem_module {u8 reserved[0x3d ]; } __packed; +/* +* General Media or DRAM Event Common Fields +* - provides common access to phys_addr +*/ +struct cxl_event_common { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +} __packed; +union cxl_event {struct cxl_event_generic generic;struct cxl_event_gen_media gen_media;struct cxl_event_dram dram;struct cxl_event_mem_module mem_module; +struct cxl_event_common common; } __packed;/*