Add DPA->HPA translation to dram & general_media events
[email protected]
maintainerv4 cover Add DPA->HPA translation to dram & general_media events
From: Alison Schofield <[email protected]>
Changes in v4:
- Pick up Dan's style suggestions I missed in v3 (Dan)
- Lookup hpa earlier (within trace_enabled()) and pass to the event handler (Dan)
- Add and use a new member in cxl_event union to access the common phys_addr field. Link to v3: https://lore.kernel.org/[email protected]/
Changes in v3:
- Harvest the cxlr before calling trace handler (Dan)
- Replace macros w inline fast assigns (made easier w available cxlr)
- Drop prior review (Jonathan) on Patch 3 due to churn.
- Remove Patch 4 which was making cxl_poison use the (now gone) macros. (A uuid_t change was tucked in Patch 4 and can be posted as a separate tidy-up patch.)
Link to v2: https://lore.kernel.org/[email protected]/
An update to the cxl_events unit test is in review here:
https://lore.kernel.org/[email protected]/
Begin Cover Letter:
Add HPA translations to CXL events: cxl_dram and cxl_general_media
Patches 1 & 2:
Before adding the new support, do some housekeeping and move related helpers to the region driver because there is no looking up region related info without CONFIG_CXL_REGION.
Patch 3:
The new functionality is introduced - cxl_dram & cxl_general_media events. Lookup and log the DPA->HPA translation along with the region name and region uuid.
Alison Schofield (3):
cxl/region: Move cxl_dpa_to_region() work to the region driver cxl/region: Move cxl_trace_hpa() work to the region driver cxl/core: Add region info to cxl_general_media and cxl_dram events
drivers/cxl/core/core.h | 14 ++++ drivers/cxl/core/mbox.c | 36 ++++++++-- drivers/cxl/core/memdev.c | 44 ------------- drivers/cxl/core/region.c | 135 ++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/trace.c | 91 ------------------------- drivers/cxl/core/trace.h | 46 +++++++++---- include/linux/cxl-event.h | 10 +++ 7 files changed, 224 insertions(+), 152 deletions(-)
base-commit: 4cece764965020c22cff7665b18a012006359095
Signature · 1 line
2.37.3
[email protected]
maintainerPATCH v4 1/3 cxl/region: Move cxl_dpa_to_region() work to the region driver
From: Alison Schofield <[email protected]>
This helper belongs in the region driver as it is only useful with CONFIG_CXL_REGION. Add a stub in core.h for when the region driver is not built.
--- drivers/cxl/core/core.h | 7 +++++++ drivers/cxl/core/memdev.c | 44 --------------------------------------- drivers/cxl/core/region.c | 44 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 51 insertions(+), 44 deletions(-)
drivers/cxl/core/core.h +7 -0
@@ -27,7 +27,14 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled);int cxl_region_init (void );void cxl_region_exit (void );int cxl_get_poison_by_endpoint (struct cxl_port *port); +struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa); +#else +static inline +struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa) +{ +return NULL ; +}static inline int cxl_get_poison_by_endpoint (struct cxl_port *port) {return 0 ;
drivers/cxl/core/memdev.c +0 -44
50 lines not on this page · raw message
drivers/cxl/core/region.c +44 -0
50 lines not on this page · raw message
Signature · 1 line
2.37.3
[email protected]
maintainerPATCH v4 2/3 cxl/region: Move cxl_trace_hpa() work to the region driver
In reply to [email protected]From: Alison Schofield <[email protected]>
This work belongs in the region driver as it is only useful with CONFIG_CXL_REGION. Add a stub in core.h for when the region driver is not built.
--- drivers/cxl/core/core.h | 7 +++ drivers/cxl/core/region.c | 91 +++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/trace.c | 91 --------------------------------------- drivers/cxl/core/trace.h | 2 - 4 files changed, 98 insertions(+), 93 deletions(-)
drivers/cxl/core/core.h +7 -0
@@ -28,8 +28,15 @@ int cxl_region_init(void);void cxl_region_exit (void );int cxl_get_poison_by_endpoint (struct cxl_port *port);struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa); +u64 cxl_trace_hpa (struct cxl_region *cxlr,const struct cxl_memdev *cxlmd, +u64 dpa);#else +static inline u64 +cxl_trace_hpa (struct cxl_region *cxlr,const struct cxl_memdev *cxlmd,u64 dpa) +{ +return ULLONG_MAX ; +}static inline struct cxl_region *cxl_dpa_to_region (const struct cxl_memdev *cxlmd,u64 dpa) {
drivers/cxl/core/region.c +91 -0
97 lines not on this page · raw message
drivers/cxl/core/trace.c +0 -91
94 lines not on this page · raw message
drivers/cxl/core/trace.h +0 -2
@@ -642,8 +642,6 @@ TRACE_EVENT(cxl_memory_module,#define cxl_poison_overflow (flags, time) \ (flags & CXL_POISON_FLAG_OVERFLOW ? le64_to_cpu(time) : 0) -u64 cxl_trace_hpa (struct cxl_region *cxlr,struct cxl_memdev *memdev,u64 dpa); - TRACE_EVENT(cxl_poison, TP_PROTO(struct cxl_memdev *cxlmd, struct cxl_region *cxlr,
Signature · 1 line
2.37.3
[email protected]
maintainerPATCH v4 3/3 cxl/core: Add region info to cxl_general_media and cxl_dram events
In reply to [email protected]From: Alison Schofield <[email protected]>
User space may need to know which region, if any, maps the DPAs (device physical addresses) reported in a cxl_general_media or cxl_dram event. Since the mapping can change, the kernel provides this information at the time the event occurs. This informs user space that at event <timestamp> this <region> mapped this <DPA> to this <HPA>.
Add the same region info that is included in the cxl_poison trace
event: the DPA->HPA translation, region name, and region uuid.
The new fields are inserted in the trace event and no existing fields are modified. If the DPA is not mapped, user will see:
hpa=ULLONG_MAX, region="", and uuid=0
This work must be protected by dpa_rwsem & region_rwsem since it is looking up region mappings.
--- drivers/cxl/core/mbox.c | 36 ++++++++++++++++++++++++++------ drivers/cxl/core/trace.h | 44 +++++++++++++++++++++++++++++++-------- include/linux/cxl-event.h | 10 +++++++++ 3 files changed, 75 insertions(+), 15 deletions(-)
drivers/cxl/core/mbox.c +30 -6
@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,enum cxl_event_type event_type, const uuid_t *uuid, union cxl_event *evt) { - if (event_type == CXL_CPER_EVENT_GEN_MEDIA) - trace_cxl_general_media(cxlmd, type, &evt->gen_media); - else if (event_type == CXL_CPER_EVENT_DRAM) - trace_cxl_dram(cxlmd, type, &evt->dram); - else if (event_type == CXL_CPER_EVENT_MEM_MODULE) + if (event_type == CXL_CPER_EVENT_MEM_MODULE) { trace_cxl_memory_module(cxlmd, type, &evt->mem_module); - else +return ; + } +if (event_type ==CXL_CPER_EVENT_GENERIC ) {trace_cxl_generic_event (cxlmd, type, uuid, &evt->generic); +return ; + } + +if (trace_cxl_general_media_enabled () ||trace_cxl_dram_enabled ()) { +u64 dpa, hpa =ULLONG_MAX ; +struct cxl_region *cxlr; + +/* +* These trace points are annotated with HPA and region +* translations. Take topology mutation locks and lookup +* { HPA, REGION } from { DPA, MEMDEV } in the event record. +*/ +guard (rwsem_read)(&cxl_region_rwsem); +guard (rwsem_read)(&cxl_dpa_rwsem); + + dpa =le64_to_cpu (evt->common.phys_addr) &CXL_DPA_MASK ; + cxlr =cxl_dpa_to_region (cxlmd, dpa); +if (cxlr) + hpa =cxl_trace_hpa (cxlr, cxlmd, dpa); + +if (event_type ==CXL_CPER_EVENT_GEN_MEDIA ) +trace_cxl_general_media (cxlmd, type, cxlr, hpa, + &evt->gen_media); +else if (event_type ==CXL_CPER_EVENT_DRAM ) +trace_cxl_dram (cxlmd, type, cxlr, hpa, &evt->dram); + } }EXPORT_SYMBOL_NS_GPL (cxl_event_trace_record,CXL );
drivers/cxl/core/trace.h +35 -9
110 lines not on this page · raw message
include/linux/cxl-event.h +10 -0
@@ -91,11 +91,21 @@ struct cxl_event_mem_module {u8 reserved[0x3d ]; } __packed; +/* +* General Media or DRAM Event Common Fields +* - provides common access to phys_addr +*/ +struct cxl_event_common { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +} __packed; +union cxl_event {struct cxl_event_generic generic;struct cxl_event_gen_media gen_media;struct cxl_event_dram dram;struct cxl_event_mem_module mem_module; +struct cxl_event_common common; } __packed;/*
Signature · 1 line
2.37.3
Re: [PATCH v4 3/3] cxl/core: Add region info to cxl_general_media and cxl_dram events
Quoted 37 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
User space may need to know which region, if any, maps the DPAs (device physical addresses) reported in a
cxl_general_mediaorcxl_dramevent. Since the mapping can change, the kernel provides this information at the time the event occurs. This informs user space that at event <timestamp> this <region> mapped this <DPA> to this <HPA>.Add the same region info that is included in the
cxl_poisontrace
event: theDPA->HPAtranslation, region name, and region uuid.The new fields are inserted in the trace event and no existing fields are modified. If the DPA is not mapped, user will see:
hpa=ULLONG_MAX, region="", and uuid=0This work must be protected by
dpa_rwsem®ion_rwsemsince it is looking up region mappings.Signed-off-by Alison Schofield <[email protected]>--- drivers/cxl/core/mbox.c | 36 ++++++++++++++++++++++++++------ drivers/cxl/core/trace.h | 44 +++++++++++++++++++++++++++++++-------- include/linux/cxl-event.h | 10 +++++++++ 3 files changed, 75 insertions(+), 15 deletions(-)drivers/cxl/core/mbox.c +0
-3@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,enum cxl_event_type event_type, const uuid_t *uuid, union cxl_event *evt) { - if (event_type == CXL_CPER_EVENT_GEN_MEDIA) - trace_cxl_general_media(cxlmd, type, &evt->gen_media); - else if (event_type == CXL_CPER_EVENT_DRAM)
drivers/cxl/core/mbox.c void cxl_event_trace_record(const struct cxl_memdev *cxlmd, · line 851
Quoted 10 lines
drivers/cxl/core/mbox.c +5
-3-trace_cxl_dram (cxlmd, type, &evt->dram); - else if (event_type == CXL_CPER_EVENT_MEM_MODULE) +if (event_type ==CXL_CPER_EVENT_MEM_MODULE ) {trace_cxl_memory_module (cxlmd, type, &evt->mem_module); - else +return ; + } +if (event_type ==CXL_CPER_EVENT_GENERIC ) {trace_cxl_generic_event (cxlmd, type, uuid, &evt->generic); +return ;
Minor, but I think you could shave a couple more lines by keeping the else-if tree going and skip the early "return" calls with a final:
else if (trace_cxl_general_media_enabled() || trace_cxl_dram_enabled())
Quoted 5 lines
drivers/cxl/core/mbox.c +5
-0+ } + +if (trace_cxl_general_media_enabled () ||trace_cxl_dram_enabled ()) { +u64 dpa, hpa =ULLONG_MAX ; +struct cxl_region *cxlr;
drivers/cxl/core/mbox.c void cxl_event_trace_record(const struct cxl_memdev *cxlmd, · line 866
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+ +/* +* These trace points are annotated with HPA and region +* translations. Take topology mutation locks and lookup +* { HPA, REGION } from { DPA, MEMDEV } in the event record. +*/ +guard (rwsem_read)(&cxl_region_rwsem); +guard (rwsem_read)(&cxl_dpa_rwsem); + + dpa =le64_to_cpu (evt->common.phys_addr) &CXL_DPA_MASK ;
A merge issue here for Dave to handle here since CXL_DPA_MASK is known broken. Do you want to wait until that bikeshedding on CXL_DPA_MASK replacement name completes, or just take the simple:
-#define CXL_DPA_FLAGS_MASK 0x3F
+#define CXL_DPA_FLAGS_MASK 0x3FULL
...for now?
Otherwise,
Alison Schofield
maintainerRe: [PATCH v4 3/3] cxl/core: Add region info to cxl_general_media and cxl_dram events
Quoted 52 lines · Dan Williams
Quoted 37 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
User space may need to know which region, if any, maps the DPAs (device physical addresses) reported in a
cxl_general_mediaorcxl_dramevent. Since the mapping can change, the kernel provides this information at the time the event occurs. This informs user space that at event <timestamp> this <region> mapped this <DPA> to this <HPA>.Add the same region info that is included in the
cxl_poisontrace
event: theDPA->HPAtranslation, region name, and region uuid.The new fields are inserted in the trace event and no existing fields are modified. If the DPA is not mapped, user will see:
hpa=ULLONG_MAX, region="", and uuid=0This work must be protected by
dpa_rwsem®ion_rwsemsince it is looking up region mappings.Signed-off-by Alison Schofield <[email protected]>--- drivers/cxl/core/mbox.c | 36 ++++++++++++++++++++++++++------ drivers/cxl/core/trace.h | 44 +++++++++++++++++++++++++++++++-------- include/linux/cxl-event.h | 10 +++++++++ 3 files changed, 75 insertions(+), 15 deletions(-)drivers/cxl/core/mbox.c +0
-3@@ -842,14 +842,38 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,enum cxl_event_type event_type, const uuid_t *uuid, union cxl_event *evt) { - if (event_type == CXL_CPER_EVENT_GEN_MEDIA) - trace_cxl_general_media(cxlmd, type, &evt->gen_media); - else if (event_type == CXL_CPER_EVENT_DRAM)drivers/cxl/core/mbox.c void cxl_event_trace_record(const struct cxl_memdev *cxlmd, · line 851
Quoted 10 lines
drivers/cxl/core/mbox.c +5
-3-trace_cxl_dram (cxlmd, type, &evt->dram); - else if (event_type == CXL_CPER_EVENT_MEM_MODULE) +if (event_type ==CXL_CPER_EVENT_MEM_MODULE ) {trace_cxl_memory_module (cxlmd, type, &evt->mem_module); - else +return ; + } +if (event_type ==CXL_CPER_EVENT_GENERIC ) {trace_cxl_generic_event (cxlmd, type, uuid, &evt->generic); +return ;Minor, but I think you could shave a couple more lines by keeping the else-if tree going and skip the early "return" calls with a final:
Quoted 2 lines
else if (trace_cxl_general_media_enabled() || trace_cxl_dram_enabled())
I think I didn't do that because this last 'if' breaks the pattern since it's not based on event_type. We're doing something different in this chunk of code.
A switch would be well suited, but it uses even more LOC - like this:
+
+
+ switch (event_type) {
+ case CXL_CPER_EVENT_MEM_MODULE:
+ trace_cxl_memory_module(cxlmd, type, &evt->mem_module);
+ break;
+
+ case CXL_CPER_EVENT_GENERIC:
+ trace_cxl_generic_event(cxlmd, type, uuid, &evt->generic);
+ break;
+
+ case CXL_CPER_EVENT_GEN_MEDIA:
+ case CXL_CPER_EVENT_DRAM:
+ u64 dpa, hpa = ULLONG_MAX;
+ struct cxl_region *cxlr;
+
+ if (!trace_cxl_general_media_enabled() &&
+ !trace_cxl_dram_enabled())
+ break;
+ /*
+ * These trace points are annotated with HPA and region
+ * translations. Take topology mutation locks and lookup
+ * { HPA, REGION } from { DPA, MEMDEV } in the event record.
+ */
+ guard(rwsem_read)(&cxl_region_rwsem);
+ guard(rwsem_read)(&cxl_dpa_rwsem);
+
+ dpa = le64_to_cpu(evt->common.phys_addr) & CXL_DPA_MASK;
+ cxlr = cxl_dpa_to_region(cxlmd, dpa);
+ if (cxlr)
+ hpa = cxl_trace_hpa(cxlr, cxlmd, dpa);
+
+ if (event_type == CXL_CPER_EVENT_GEN_MEDIA)
+ trace_cxl_general_media(cxlmd, type, cxlr, hpa,
+ &evt->gen_media);
+ else if (event_type == CXL_CPER_EVENT_DRAM)
+ trace_cxl_dram(cxlmd, type, cxlr, hpa, &evt->dram);
+ break;
+ default:
+ break;
+ }
+
Maybe if I just move the comment before the changed chunk of code, the break in flow will be easier to follow.
Quoted 1 line
drivers/cxl/core/mbox.c void cxl_event_trace_record(const struct cxl_memdev *cxlmd, · line 852
Quoted 1 line
drivers/cxl/core/mbox.c +1
-0+ }
/* Move the comment below to here */
Quoted 19 lines
Quoted 4 lines
drivers/cxl/core/mbox.c +4
-0+ +if (trace_cxl_general_media_enabled () ||trace_cxl_dram_enabled ()) { +u64 dpa, hpa =ULLONG_MAX ; +struct cxl_region *cxlr;drivers/cxl/core/mbox.c void cxl_event_trace_record(const struct cxl_memdev *cxlmd, · line 866
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+ +/* +* These trace points are annotated with HPA and region +* translations. Take topology mutation locks and lookup +* { HPA, REGION } from { DPA, MEMDEV } in the event record. +*/ +guard (rwsem_read)(&cxl_region_rwsem); +guard (rwsem_read)(&cxl_dpa_rwsem); + + dpa =le64_to_cpu (evt->common.phys_addr) &CXL_DPA_MASK ;A merge issue here for Dave to handle here since
CXL_DPA_MASKis known broken. Do you want to wait until that bikeshedding onCXL_DPA_MASKreplacement name completes, or just take the simple:
Quoted 4 lines
-#define
CXL_DPA_FLAGS_MASK0x3F
+#defineCXL_DPA_FLAGS_MASK0x3FULL...for now?
I'm optimistic we can get that fixes patch in. Don't worry about it ;)
Quoted 4 lines
Otherwise,
Reviewed-by Dan Williams <[email protected]>