Skip to contentSkip to conversation

Add DPA->HPA translation to dram & general_media

PATCH Superseded [email protected] 1 participant 4 messages Apr 25, 2024 CXL

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

[email protected]

author maintainer
#0 lore raw

v3 cover Add DPA->HPA translation to dram & general_media

From: Alison Schofield <[email protected]>

Changes in v3:
- Harvest the cxlr before calling trace handler (Dan)
- Replace macros w inline fast assigns (made easier w available cxlr)
- Drop prior review (Jonathan) on Patch 3 due to churn.
- Remove Patch 4 which was making cxl_poison use the (now gone) macros. (A uuid_t change was tucked in Patch 4 and can be posted as a separate tidy-up patch.)

Link to v2: https://lore.kernel.org/[email protected]/

An update to the cxl_events unit test is in review here:
https://lore.kernel.org/[email protected]/

Begin Cover Letter:
Add HPA translations to CXL events: cxl_dram and cxl_general_media

Patches 1 & 2:
Before adding the new support, do some housekeeping and move related helpers to the region driver because there is no looking up region related info without CONFIG_CXL_REGION.

Patch 3:
The new functionality is introduced - cxl_dram & cxl_general_media events. Lookup and log the DPA->HPA translation along with the region name and region uuid.

Alison Schofield (3):
cxl/region: Move cxl_dpa_to_region() work to the region driver cxl/region: Move cxl_trace_hpa() work to the region driver cxl/core: Add region info to cxl_general_media and cxl_dram events

 drivers/cxl/core/core.h   |  14 ++++
 drivers/cxl/core/mbox.c   |  30 +++++++--
 drivers/cxl/core/memdev.c |  44 -------------
 drivers/cxl/core/region.c | 135 ++++++++++++++++++++++++++++++++++++++
 drivers/cxl/core/trace.c  |  91 -------------------------
 drivers/cxl/core/trace.h  |  48 ++++++++++----
 6 files changed, 210 insertions(+), 152 deletions(-)

base-commit: 4cece764965020c22cff7665b18a012006359095

Signature · 1 line
2.37.3
author maintainer
#1 lore raw

PATCH v3 1/3 cxl/region: Move cxl_dpa_to_region() work to the region driver

From: Alison Schofield <[email protected]>

This helper belongs in the region driver as it is only useful with CONFIG_CXL_REGION. Add a stub in core.h for when the region driver is not built.

---
 drivers/cxl/core/core.h   |  7 +++++++
 drivers/cxl/core/memdev.c | 44 ---------------------------------------
 drivers/cxl/core/region.c | 44 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 44 deletions(-)
drivers/cxl/core/core.h +7 -0
index bc5a95665aa0..87008505f8a9 100644
@@ -27,7 +27,14 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled);
 int cxl_region_init(void);
 void cxl_region_exit(void);
 int cxl_get_poison_by_endpoint(struct cxl_port *port);
+struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa);
+
 #else
+static inline
+struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa)
+{
+	return NULL;
+}
 static inline int cxl_get_poison_by_endpoint(struct cxl_port *port)
 {
 	return 0;
drivers/cxl/core/memdev.c +0 -44

50 lines not on this page · raw message

drivers/cxl/core/region.c +44 -0

50 lines not on this page · raw message

Signature · 1 line
2.37.3
author maintainer
#2 lore raw

PATCH v3 2/3 cxl/region: Move cxl_trace_hpa() work to the region driver

In reply to [email protected]

From: Alison Schofield <[email protected]>

This work belongs in the region driver as it is only useful with CONFIG_CXL_REGION. Add a stub in core.h for when the region driver is not built.

---
 drivers/cxl/core/core.h   |  7 +++
 drivers/cxl/core/region.c | 91 +++++++++++++++++++++++++++++++++++++++
 drivers/cxl/core/trace.c  | 91 ---------------------------------------
 drivers/cxl/core/trace.h  |  2 -
 4 files changed, 98 insertions(+), 93 deletions(-)
drivers/cxl/core/core.h +7 -0
index 87008505f8a9..625394486459 100644
@@ -28,8 +28,15 @@ int cxl_region_init(void);
 void cxl_region_exit(void);
 int cxl_get_poison_by_endpoint(struct cxl_port *port);
 struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa);
+u64 cxl_trace_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd,
+		  u64 dpa);
 
 #else
+static inline u64
+cxl_trace_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd, u64 dpa)
+{
+	return ULLONG_MAX;
+}
 static inline
 struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa)
 {
drivers/cxl/core/region.c +91 -0

97 lines not on this page · raw message

drivers/cxl/core/trace.c +0 -91

94 lines not on this page · raw message

drivers/cxl/core/trace.h +0 -2
index e5f13260fc52..161bdb5734b0 100644
@@ -642,8 +642,6 @@ TRACE_EVENT(cxl_memory_module,
 #define cxl_poison_overflow(flags, time)				\
 	(flags & CXL_POISON_FLAG_OVERFLOW ? le64_to_cpu(time) : 0)
 
-u64 cxl_trace_hpa(struct cxl_region *cxlr, struct cxl_memdev *memdev, u64 dpa);
-
 TRACE_EVENT(cxl_poison,
 
 	TP_PROTO(struct cxl_memdev *cxlmd, struct cxl_region *cxlr,
Signature · 1 line
2.37.3
author maintainer
#3 lore raw

PATCH v3 3/3 cxl/core: Add region info to cxl_general_media and cxl_dram events

In reply to [email protected]

From: Alison Schofield <[email protected]>

User space may need to know which region, if any, maps the DPAs (device physical addresses) reported in a cxl_general_media or cxl_dram event. Since the mapping can change, the kernel provides this information at the time the event occurs. This informs user space that at event <timestamp> this <region> mapped this <DPA> to this <HPA>.

Add the same region info that is included in the cxl_poison trace
event: the DPA->HPA translation, region name, and region uuid.

The new fields are inserted in the trace event and no existing fields are modified. If the DPA is not mapped, user will see:
hpa=ULLONG_MAX, region="", and uuid=0

This work must be protected by dpa_rwsem & region_rwsem since it is looking up region mappings.

---
 drivers/cxl/core/mbox.c  | 30 ++++++++++++++++++++------
 drivers/cxl/core/trace.h | 46 ++++++++++++++++++++++++++++++++--------
 2 files changed, 61 insertions(+), 15 deletions(-)
drivers/cxl/core/mbox.c +24 -6
index 9adda4795eb7..d43ce772d030 100644
@@ -842,14 +842,32 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,
 			    enum cxl_event_type event_type,
 			    const uuid_t *uuid, union cxl_event *evt)
 {
-	if (event_type == CXL_CPER_EVENT_GEN_MEDIA)
-		trace_cxl_general_media(cxlmd, type, &evt->gen_media);
-	else if (event_type == CXL_CPER_EVENT_DRAM)
-		trace_cxl_dram(cxlmd, type, &evt->dram);
-	else if (event_type == CXL_CPER_EVENT_MEM_MODULE)
+	if (event_type == CXL_CPER_EVENT_MEM_MODULE) {
 		trace_cxl_memory_module(cxlmd, type, &evt->mem_module);
-	else
+		return;
+	}
+	if (event_type == CXL_CPER_EVENT_GENERIC) {
 		trace_cxl_generic_event(cxlmd, type, uuid, &evt->generic);
+		return;
+	}
+
+	/* Protect trace events that do DPA->HPA translations */
+	guard(rwsem_read)(&cxl_region_rwsem);
+	guard(rwsem_read)(&cxl_dpa_rwsem);
+
+	if (event_type == CXL_CPER_EVENT_GEN_MEDIA &&
+	    trace_cxl_general_media_enabled()) {
+		u64 dpa = le64_to_cpu(evt->gen_media.phys_addr) & CXL_DPA_MASK;
+		struct cxl_region *cxlr = cxl_dpa_to_region(cxlmd, dpa);
+
+		trace_cxl_general_media(cxlmd, type, &evt->gen_media, cxlr);
+	} else if (event_type == CXL_CPER_EVENT_DRAM &&
+		   trace_cxl_dram_enabled()) {
+		u64 dpa = le64_to_cpu(evt->dram.phys_addr) & CXL_DPA_MASK;
+		struct cxl_region *cxlr = cxl_dpa_to_region(cxlmd, dpa);
+
+		trace_cxl_dram(cxlmd, type, &evt->dram, cxlr);
+	}
 }
 EXPORT_SYMBOL_NS_GPL(cxl_event_trace_record, CXL);
 
drivers/cxl/core/trace.h +37 -9

112 lines not on this page · raw message

Signature · 1 line
2.37.3