Skip to contentSkip to conversation

hw/cxl: add trace-events coverage

PATCH Superseded Junjie Cao 2 participants 8 messages Jun 3 – Jun 11 Arm/Amlogic Meson SoC support

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Junjie Cao

author
#0 lore raw

cover hw/cxl: add trace-events coverage

The CXL subsystem in QEMU currently has no trace coverage: hw/cxl/ does not have its own trace-events file, and there are no trace_*() calls across hw/cxl/ or hw/mem/cxl_type3.c. Debugging mailbox commands, HDM decoder programming, CDAT ingestion, event log activity, and CFMWS address decoding currently relies on temporary printf instrumentation or qemu_log_mask output. Trace events offer a more structured mechanism for observing these paths during development.

This series introduces trace-events infrastructure for the CXL subsystem and adds 12 trace points covering the paths listed above. The events all use the cxl_ prefix so a single -trace 'cxl_*' captures everything. The two cxl_hdm_decoder_* events live in the existing hw/mem/trace-events alongside the file they trace (hw/mem/cxl_type3.c); the cxl_ prefix preserves the unified filtering UX.

Each patch is independent and corresponds to a self-contained debug scenario:

1/3 trace infrastructure + mailbox dispatch
- cxl_mailbox_command, cxl_mailbox_invalid_payload, cxl_mailbox_handler_return

2/3 HDM decoder commit + CDAT load
- cxl_hdm_decoder_write, cxl_hdm_decoder_commit, cxl_cdat_init, cxl_cdat_loaded

3/3 event log + CFMWS HPA -> device decoding
- cxl_event_insert, cxl_event_overflow, cxl_event_clear
- cxl_cfmws_lookup, cxl_cfmws_target_resolved

Junjie Cao (3):
hw/cxl: add trace-events for mailbox dispatch hw/cxl: trace HDM decoder commit and CDAT load hw/cxl: trace event log activity and CFMWS address decoding

 hw/cxl/cxl-cdat.c          |  3 +++
 hw/cxl/cxl-events.c        |  8 +++++++-
 hw/cxl/cxl-host.c          |  4 ++++
 hw/cxl/cxl-mailbox-utils.c |  5 +++++
 hw/cxl/trace-events        | 21 +++++++++++++++++++++
 hw/cxl/trace.h             |  4 ++++
 hw/mem/cxl_type3.c         | 17 ++++++++++++++++-
 hw/mem/trace-events        |  4 ++++
 meson.build                |  1 +
 9 files changed, 65 insertions(+), 2 deletions(-)
 create mode 100644 hw/cxl/trace-events
 create mode 100644 hw/cxl/trace.h

base-commit: 5611a9268dae7b7ff99d478ed134052a9fc7e9f7 --
2.43.0

author
#1 lore raw

PATCH 1/3 hw/cxl: add trace-events for mailbox dispatch

The CXL subsystem currently has no trace coverage. Debugging mailbox command flow requires either printf or qemu_log_mask, neither of which is well suited for following the dispatch path that is exercised by every guest driver probe.

Introduce a dedicated trace-events file under hw/cxl/ and wire it into the build via the trace_events_subdirs list in the top-level meson.build. Add three trace points covering command entry, payload validation, and handler completion in cxl_process_cci_message():

  * cxl_mailbox_command          - command entry (opcode, len_in)
  * cxl_mailbox_invalid_payload  - payload length check rejection
  * cxl_mailbox_handler_return   - handler completion with return code

and len_out

The early-exit paths for unimplemented commands, in-progress background operations, and disabled media skip handler dispatch and are therefore not paired with cxl_mailbox_handler_return; an absent return event for a given cxl_mailbox_command identifies these cases in the trace stream.

Trace points use the cxl_ prefix so future patches in this subsystem share a single namespace, allowing -trace 'cxl_*' to match all of them.

---
 hw/cxl/cxl-mailbox-utils.c | 5 +++++
 hw/cxl/trace-events        | 8 ++++++++
 hw/cxl/trace.h             | 4 ++++
 meson.build                | 1 +
 4 files changed, 18 insertions(+)
 create mode 100644 hw/cxl/trace-events
 create mode 100644 hw/cxl/trace.h
hw/cxl/cxl-mailbox-utils.c +5 -0
index d8ba7e8625..521b8c6334 100644
@@ -27,6 +27,7 @@
 #include "system/hostmem.h"
 #include "qemu/range.h"
 #include "qapi/qapi-types-cxl.h"
+#include "trace.h"
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
 #define CXL_DC_EVENT_LOG_SIZE 8
@@ -4580,6 +4581,7 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
     CXLDeviceState *cxl_dstate;
 
     *len_out = 0;
+    trace_cxl_mailbox_command((set << 8) | cmd, len_in);
     cxl_cmd = &cci->cxl_cmd_set[set][cmd];
     h = cxl_cmd->handler;
     if (!h) {
@@ -4589,6 +4591,8 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
     }
 
     if (len_in != cxl_cmd->in && cxl_cmd->in != ~0) {
+        trace_cxl_mailbox_invalid_payload((set << 8) | cmd, len_in,
+                                          cxl_cmd->in);
         return CXL_MBOX_INVALID_PAYLOAD_LENGTH;
     }
 
@@ -4642,6 +4646,7 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
         timer_mod(cci->bg.timer, now + CXL_MBOX_BG_UPDATE_FREQ);
     }
 
+    trace_cxl_mailbox_handler_return((set << 8) | cmd, ret, *len_out);
     return ret;
 }
 
hw/cxl/trace-events new file +8 -0
new file mode 100644
index 0000000000..abca116ed3
@@ -0,0 +1,8 @@
+# See docs/devel/tracing.rst for syntax documentation.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+# cxl-mailbox-utils.c
+cxl_mailbox_command(uint16_t opcode, size_t len_in) "opcode=0x%04x len_in=%zu"
+cxl_mailbox_invalid_payload(uint16_t opcode, size_t len_in, size_t expected) "opcode=0x%04x len_in=%zu expected=%zu"
+cxl_mailbox_handler_return(uint16_t opcode, int ret, size_t len_out) "opcode=0x%04x ret=%d len_out=%zu"
hw/cxl/trace.h new file +4 -0
new file mode 100644
index 0000000000..84e79d50c0
@@ -0,0 +1,4 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+#include "trace/trace-hw_cxl.h"
meson.build +1 -0
index 19e123423b..c229199d5d 100644
@@ -3596,6 +3596,7 @@ if have_system
     'hw/audio',
     'hw/block',
     'hw/char',
+    'hw/cxl',
     'hw/display',
     'hw/dma',
     'hw/fsi',
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH 1/3] hw/cxl: add trace-events for mailbox dispatch

Quoted 20 lines · On Wed, 3 Jun 2026 09:34:18 +0800 Junjie Cao <[email protected]> wrote:

The CXL subsystem currently has no trace coverage. Debugging mailbox command flow requires either printf or qemu_log_mask, neither of which is well suited for following the dispatch path that is exercised by every guest driver probe.

Introduce a dedicated trace-events file under hw/cxl/ and wire it into the build via the trace_events_subdirs list in the top-level meson.build. Add three trace points covering command entry, payload validation, and handler completion in cxl_process_cci_message():

  * cxl_mailbox_command          - command entry (opcode, len_in)
  * cxl_mailbox_invalid_payload  - payload length check rejection
  * cxl_mailbox_handler_return   - handler completion with return code

and len_out

The early-exit paths for unimplemented commands, in-progress background operations, and disabled media skip handler dispatch and are therefore not paired with cxl_mailbox_handler_return; an absent return event for a given cxl_mailbox_command identifies these cases in the trace stream.

Quoted 4 lines

Trace points use the cxl_ prefix so future patches in this subsystem share a single namespace, allowing -trace 'cxl_*' to match all of them.

Seems likely to be a useful addition. My main request is to make sure all possible places we might exit due to a given condition are covered rather than just the simplest one (for length checks that is!)

Jonathan

Quoted 22 lines
---
 hw/cxl/cxl-mailbox-utils.c | 5 +++++
 hw/cxl/trace-events        | 8 ++++++++
 hw/cxl/trace.h             | 4 ++++
 meson.build                | 1 +
 4 files changed, 18 insertions(+)
 create mode 100644 hw/cxl/trace-events
 create mode 100644 hw/cxl/trace.h
hw/cxl/cxl-mailbox-utils.c +1 -0
index d8ba7e8625..521b8c6334 100644
@@ -27,6 +27,7 @@
 #include "system/hostmem.h"
 #include "qemu/range.h"
 #include "qapi/qapi-types-cxl.h"
+#include "trace.h"
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
 #define CXL_DC_EVENT_LOG_SIZE 8

hw/cxl/cxl-mailbox-utils.c int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd, · line 4584

Quoted 4 lines
hw/cxl/cxl-mailbox-utils.c +1 -0
@@ -4580,6 +4581,7 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
     CXLDeviceState *cxl_dstate;
 
     *len_out = 0;
+    trace_cxl_mailbox_command((set << 8) | cmd, len_in);

Given the opcode is always made up of those two smaller fields, it might be nicer to just pass them as separate parameters? Potentially combine them into the single print in the tracepoint, or do it as set:cmd e.g 0x01:03 or something like that?

hw/cxl/cxl-mailbox-utils.c int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd, · line 4596

Quoted 10 lines
hw/cxl/cxl-mailbox-utils.c +2 -0
     cxl_cmd = &cci->cxl_cmd_set[set][cmd];
     h = cxl_cmd->handler;
     if (!h) {
@@ -4589,6 +4591,8 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
     }
 
     if (len_in != cxl_cmd->in && cxl_cmd->in != ~0) {
+        trace_cxl_mailbox_invalid_payload((set << 8) | cmd, len_in,
+                                          cxl_cmd->in);
         return CXL_MBOX_INVALID_PAYLOAD_LENGTH;

This is one of a lot of places that we fail with that error code - because many commands have variable lengths. I'd want to see this on all of the places where we end up bailing out with that error code.

Quoted 45 lines
hw/cxl/cxl-mailbox-utils.c +1 -0
@@ -4642,6 +4646,7 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
         timer_mod(cci->bg.timer, now + CXL_MBOX_BG_UPDATE_FREQ);
     }
 
+    trace_cxl_mailbox_handler_return((set << 8) | cmd, ret, *len_out);
     return ret;
 }
 
hw/cxl/trace-events new file +8 -0
new file mode 100644
index 0000000000..abca116ed3
@@ -0,0 +1,8 @@
+# See docs/devel/tracing.rst for syntax documentation.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+# cxl-mailbox-utils.c
+cxl_mailbox_command(uint16_t opcode, size_t len_in) "opcode=0x%04x len_in=%zu"
+cxl_mailbox_invalid_payload(uint16_t opcode, size_t len_in, size_t expected) "opcode=0x%04x len_in=%zu expected=%zu"
+cxl_mailbox_handler_return(uint16_t opcode, int ret, size_t len_out) "opcode=0x%04x ret=%d len_out=%zu"
hw/cxl/trace.h new file +4 -0
new file mode 100644
index 0000000000..84e79d50c0
@@ -0,0 +1,4 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+#include "trace/trace-hw_cxl.h"
meson.build +1 -0
index 19e123423b..c229199d5d 100644
@@ -3596,6 +3596,7 @@ if have_system
     'hw/audio',
     'hw/block',
     'hw/char',
+    'hw/cxl',
     'hw/display',
     'hw/dma',
     'hw/fsi',
author
#3 lore raw

PATCH 2/3 hw/cxl: trace HDM decoder commit and CDAT load

In reply to Junjie Cao

HDM decoder programming and CDAT table ingestion are two debugging-prone paths in the CXL stack: the former drives address-decoding behaviour, the latter is parsed before any guest interaction so wrong inputs surface late.

Add four trace points:

* cxl_hdm_decoder_write - guest write to a decoder CTRL register,
capturing the raw value and whether a commit is being requested
* cxl_hdm_decoder_commit - actual commit, with the resolved base,
size, IW and IG fields
* cxl_cdat_init - CDAT initialisation, distinguishing the
file-backed vs built-in source
* cxl_cdat_loaded - successful file load with size and
number of entries parsed

The hw/mem/cxl_type3.c trace points are declared in the existing hw/mem/trace-events. The cxl_ prefix keeps the namespace unified for filtering with -trace 'cxl_*'.

---
 hw/cxl/cxl-cdat.c   |  3 +++
 hw/cxl/trace-events |  4 ++++
 hw/mem/cxl_type3.c  | 17 ++++++++++++++++-
 hw/mem/trace-events |  4 ++++
 4 files changed, 27 insertions(+), 1 deletion(-)
hw/cxl/cxl-cdat.c +3 -0
index 959a55518e..5274ac8c12 100644
@@ -12,6 +12,7 @@
 #include "hw/cxl/cxl.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
+#include "trace.h"
 
 static void cdat_len_check(CDATSubHeader *hdr, Error **errp)
 {
@@ -186,6 +187,7 @@ static bool ct3_load_cdat(CDATObject *cdat, Error **errp)
     cdat->entry_len = num_ent;
     cdat->entry = g_steal_pointer(&cdat_st);
     cdat->buf = g_steal_pointer(&buf);
+    trace_cxl_cdat_loaded(cdat->filename, file_size, num_ent);
     return true;
 }
 
@@ -193,6 +195,7 @@ bool cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
 {
     CDATObject *cdat = &cxl_cstate->cdat;
 
+    trace_cxl_cdat_init(cdat->filename ?: "<built-in>");
     if (cdat->filename) {
         return ct3_load_cdat(cdat, errp);
     } else {
hw/cxl/trace-events +4 -0
index abca116ed3..15977dce6e 100644
@@ -6,3 +6,7 @@
 cxl_mailbox_command(uint16_t opcode, size_t len_in) "opcode=0x%04x len_in=%zu"
 cxl_mailbox_invalid_payload(uint16_t opcode, size_t len_in, size_t expected) "opcode=0x%04x len_in=%zu expected=%zu"
 cxl_mailbox_handler_return(uint16_t opcode, int ret, size_t len_out) "opcode=0x%04x ret=%d len_out=%zu"
+
+# cxl-cdat.c
+cxl_cdat_init(const char *source) "source=%s"
+cxl_cdat_loaded(const char *filename, size_t length, int num_entries) "filename=%s length=%zu entries=%d"
hw/mem/cxl_type3.c +16 -1
index 4739239da3..8130abeb37 100644
@@ -30,6 +30,7 @@
 #include "system/numa.h"
 #include "hw/cxl/cxl.h"
 #include "hw/pci/msix.h"
+#include "trace.h"
 
 /* type3 device private */
 enum CXL_T3_MSIX_VECTOR {
@@ -419,9 +420,20 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
     int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
     ComponentRegisters *cregs = &ct3d->cxl_cstate.crb;
     uint32_t *cache_mem = cregs->cache_mem_registers;
-    uint32_t ctrl;
+    uint32_t ctrl, low, high;
+    uint64_t base, size;
 
     ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc);
+    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + which * hdm_inc);
+    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + which * hdm_inc);
+    base = (low & 0xf0000000) | ((uint64_t)high << 32);
+    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + which * hdm_inc);
+    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + which * hdm_inc);
+    size = (low & 0xf0000000) | ((uint64_t)high << 32);
+    trace_cxl_hdm_decoder_commit(which, base, size,
+                                 FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW),
+                                 FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG));
+
     /* TODO: Sanity checks that the decoder is possible */
     ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0);
     ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
@@ -623,6 +635,9 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, uint64_t value,
     }
 
     stl_le_p((uint8_t *)cache_mem + offset, value);
+    if (which_hdm >= 0) {
+        trace_cxl_hdm_decoder_write(which_hdm, value, should_commit);
+    }
     if (should_commit) {
         hdm_decoder_commit(ct3d, which_hdm);
     } else if (should_uncommit) {
hw/mem/trace-events +4 -0
index 8b6b02b5bf..770831f701 100644
@@ -6,3 +6,7 @@ mhp_pc_dimm_assigned_slot(int slot) "%d"
 memory_device_pre_plug(const char *id, uint64_t addr) "id=%s addr=0x%"PRIx64
 memory_device_plug(const char *id, uint64_t addr) "id=%s addr=0x%"PRIx64
 memory_device_unplug(const char *id, uint64_t addr) "id=%s addr=0x%"PRIx64
+
+# cxl_type3.c
+cxl_hdm_decoder_write(int which, uint32_t value, bool commit) "decoder=%d ctrl=0x%08x commit=%d"
+cxl_hdm_decoder_commit(int which, uint64_t base, uint64_t size, uint32_t iw_enc, uint32_t ig_enc) "decoder=%d base=0x%"PRIx64" size=0x%"PRIx64" iw_enc=0x%x ig_enc=0x%x"
Signature · 1 line
2.43.0
#4 lore raw

Re: [PATCH 2/3] hw/cxl: trace HDM decoder commit and CDAT load

Quoted 7 lines · On Wed, 3 Jun 2026 09:34:19 +0800 Junjie Cao <[email protected]> wrote:

HDM decoder programming and CDAT table ingestion are two debugging-prone paths in the CXL stack: the former drives address-decoding behaviour, the latter is parsed before any guest interaction so wrong inputs surface late.

Add four trace points:

Quoted 9 lines

* cxl_hdm_decoder_write - guest write to a decoder CTRL register,
capturing the raw value and whether a commit is being requested
* cxl_hdm_decoder_commit - actual commit, with the resolved base,
size, IW and IG fields
* cxl_cdat_init - CDAT initialisation, distinguishing the
file-backed vs built-in source
* cxl_cdat_loaded - successful file load with size and
number of entries parsed

The 'and' in the commit log suggests to me this is two largely unrelated things. Can we have one patch for hdm decoder actions and one for cdat actions. I suspect the hdm decoder one will at least have some fuzz due to Alireza's work. Ideally I'd like to get that in first given it's usefulness. (not the DCD case probably as that may need more review but the static fast path case).

I'm not really convinced the cdat ones provide as much value as the others you have added. Perhaps provide more info on why you find that one useful - unless we have a silly bug the source should be obvious as it is commandline controlled for instance.

Quoted 109 lines

The hw/mem/cxl_type3.c trace points are declared in the existing hw/mem/trace-events. The cxl_ prefix keeps the namespace unified for filtering with -trace 'cxl_*'.

---
 hw/cxl/cxl-cdat.c   |  3 +++
 hw/cxl/trace-events |  4 ++++
 hw/mem/cxl_type3.c  | 17 ++++++++++++++++-
 hw/mem/trace-events |  4 ++++
 4 files changed, 27 insertions(+), 1 deletion(-)
hw/cxl/cxl-cdat.c +3 -0
index 959a55518e..5274ac8c12 100644
@@ -12,6 +12,7 @@
 #include "hw/cxl/cxl.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
+#include "trace.h"
 
 static void cdat_len_check(CDATSubHeader *hdr, Error **errp)
 {
@@ -186,6 +187,7 @@ static bool ct3_load_cdat(CDATObject *cdat, Error **errp)
     cdat->entry_len = num_ent;
     cdat->entry = g_steal_pointer(&cdat_st);
     cdat->buf = g_steal_pointer(&buf);
+    trace_cxl_cdat_loaded(cdat->filename, file_size, num_ent);
     return true;
 }
 
@@ -193,6 +195,7 @@ bool cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
 {
     CDATObject *cdat = &cxl_cstate->cdat;
 
+    trace_cxl_cdat_init(cdat->filename ?: "<built-in>");
     if (cdat->filename) {
         return ct3_load_cdat(cdat, errp);
     } else {
hw/cxl/trace-events +4 -0
index abca116ed3..15977dce6e 100644
@@ -6,3 +6,7 @@
 cxl_mailbox_command(uint16_t opcode, size_t len_in) "opcode=0x%04x len_in=%zu"
 cxl_mailbox_invalid_payload(uint16_t opcode, size_t len_in, size_t expected) "opcode=0x%04x len_in=%zu expected=%zu"
 cxl_mailbox_handler_return(uint16_t opcode, int ret, size_t len_out) "opcode=0x%04x ret=%d len_out=%zu"
+
+# cxl-cdat.c
+cxl_cdat_init(const char *source) "source=%s"
+cxl_cdat_loaded(const char *filename, size_t length, int num_entries) "filename=%s length=%zu entries=%d"
hw/mem/cxl_type3.c +16 -1
index 4739239da3..8130abeb37 100644
@@ -30,6 +30,7 @@
 #include "system/numa.h"
 #include "hw/cxl/cxl.h"
 #include "hw/pci/msix.h"
+#include "trace.h"
 
 /* type3 device private */
 enum CXL_T3_MSIX_VECTOR {
@@ -419,9 +420,20 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
     int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
     ComponentRegisters *cregs = &ct3d->cxl_cstate.crb;
     uint32_t *cache_mem = cregs->cache_mem_registers;
-    uint32_t ctrl;
+    uint32_t ctrl, low, high;
+    uint64_t base, size;
 
     ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc);
+    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + which * hdm_inc);
+    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + which * hdm_inc);
+    base = (low & 0xf0000000) | ((uint64_t)high << 32);
+    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + which * hdm_inc);
+    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + which * hdm_inc);
+    size = (low & 0xf0000000) | ((uint64_t)high << 32);
+    trace_cxl_hdm_decoder_commit(which, base, size,
+                                 FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW),
+                                 FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG));
+
     /* TODO: Sanity checks that the decoder is possible */
     ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0);
     ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
@@ -623,6 +635,9 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, uint64_t value,
     }
 
     stl_le_p((uint8_t *)cache_mem + offset, value);
+    if (which_hdm >= 0) {
+        trace_cxl_hdm_decoder_write(which_hdm, value, should_commit);
+    }
     if (should_commit) {
         hdm_decoder_commit(ct3d, which_hdm);
     } else if (should_uncommit) {
hw/mem/trace-events +4 -0
index 8b6b02b5bf..770831f701 100644
@@ -6,3 +6,7 @@ mhp_pc_dimm_assigned_slot(int slot) "%d"
 memory_device_pre_plug(const char *id, uint64_t addr) "id=%s addr=0x%"PRIx64
 memory_device_plug(const char *id, uint64_t addr) "id=%s addr=0x%"PRIx64
 memory_device_unplug(const char *id, uint64_t addr) "id=%s addr=0x%"PRIx64
+
+# cxl_type3.c
+cxl_hdm_decoder_write(int which, uint32_t value, bool commit) "decoder=%d ctrl=0x%08x commit=%d"
+cxl_hdm_decoder_commit(int which, uint64_t base, uint64_t size, uint32_t iw_enc, uint32_t ig_enc) "decoder=%d base=0x%"PRIx64" size=0x%"PRIx64" iw_enc=0x%x ig_enc=0x%x"
author
#5 lore raw

PATCH 3/3 hw/cxl: trace event log activity and CFMWS address decoding

In reply to Junjie Cao

Two more debug-prone paths benefit from trace coverage. Event log insertion and clearing are exercised on every error injection or dynamic capacity event, but their interaction with the per-log overflow counter is currently opaque. CFMWS address decoding translates a host physical address through up to two HDM decoder lookups (host bridge and optional switch USP).

Add five trace points:

* cxl_event_insert - successful enqueue, log type and
resulting depth
* cxl_event_overflow - the artificial overflow threshold
was hit; an event was dropped
* cxl_event_clear - bulk clear, log type and request size

* cxl_cfmws_lookup - entry to address decoding, with
the resolved fixed-window index
* cxl_cfmws_target_resolved - HDM decoder produced a target index
(fires for both the host-bridge and switch USP lookups)

In non-passthrough topologies, a cxl_cfmws_lookup without a following cxl_cfmws_target_resolved indicates the host-bridge HDM decode failed. In passthrough mode the lookup resolves via pcie_find_port_first() without an HDM target step, so no target_resolved event is expected.

In cxl_event_insert(), cxl_event_count() is hoisted into a local variable so its O(n) queue walk is performed once per insert rather than twice (once for the trace point, once for the existing return- value comparison).

---
 hw/cxl/cxl-events.c | 8 +++++++-
 hw/cxl/cxl-host.c   | 4 ++++
 hw/cxl/trace-events | 9 +++++++++
 3 files changed, 20 insertions(+), 1 deletion(-)
hw/cxl/cxl-events.c +7 -1
index 5356dfb5b3..f4aa419953 100644
@@ -13,6 +13,7 @@
 #include "hw/pci/msix.h"
 #include "hw/cxl/cxl.h"
 #include "hw/cxl/cxl_events.h"
+#include "trace.h"
 
 /* Artificial limit on the number of events a log can hold */
 #define CXL_TEST_EVENT_OVERFLOW 8
@@ -98,6 +99,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
     uint64_t time;
     CXLEventLog *log;
     CXLEvent *entry;
+    int depth;
 
     if (log_type >= CXL_EVENT_TYPE_MAX) {
         return false;
@@ -115,6 +117,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
         }
         log->overflow_err_count++;
         log->last_overflow_timestamp = time;
+        trace_cxl_event_overflow(log_type, log->overflow_err_count);
         return false;
     }
 
@@ -133,8 +136,10 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
     QSIMPLEQ_INSERT_TAIL(&log->events, entry, node);
     cxl_event_set_status(cxlds, log_type, true);
 
+    depth = cxl_event_count(log);
+    trace_cxl_event_insert(log_type, depth);
     /* Count went from 0 to 1 */
-    return cxl_event_count(log) == 1;
+    return depth == 1;
 }
 
 void cxl_discard_all_event_records(CXLDeviceState *cxlds)
@@ -234,6 +239,7 @@ CXLRetCode cxl_event_clear_records(CXLDeviceState *cxlds,
         entry = cxl_event_get_head(log);
     }
 
+    trace_cxl_event_clear(log_type, pl->nr_recs);
     return CXL_MBOX_SUCCESS;
 }
 
hw/cxl/cxl-host.c +4 -0
index a94b893e99..4157377a95 100644
@@ -21,6 +21,7 @@
 #include "hw/pci/pci_host.h"
 #include "hw/pci/pcie_port.h"
 #include "hw/pci-bridge/pci_expander_bridge.h"
+#include "trace.h"
 
 static void cxl_fixed_memory_window_config(CXLFixedMemoryWindowOptions *object,
                                            int index, Error **errp)
@@ -168,6 +169,7 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr)
     bool target_found;
     PCIDevice *rp, *d;
 
+    trace_cxl_cfmws_lookup(addr, fw->index);
     rb_index = (addr / cxl_decode_ig(fw->enc_int_gran)) % fw->num_targets;
     hb = PCI_HOST_BRIDGE(fw->target_hbs[rb_index]->cxl_host_bridge);
     if (!hb || !hb->bus || !pci_bus_is_cxl(hb->bus)) {
@@ -191,6 +193,7 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr)
         if (!target_found) {
             return NULL;
         }
+        trace_cxl_cfmws_target_resolved(addr, target);
 
         rp = pcie_find_port_by_pn(hb->bus, target);
         if (!rp) {
@@ -227,6 +230,7 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr)
     if (!target_found) {
         return NULL;
     }
+    trace_cxl_cfmws_target_resolved(addr, target);
 
     d = pcie_find_port_by_pn(&PCI_BRIDGE(d)->sec_bus, target);
     if (!d) {
hw/cxl/trace-events +9 -0
index 15977dce6e..e5f196787d 100644
@@ -10,3 +10,12 @@ cxl_mailbox_handler_return(uint16_t opcode, int ret, size_t len_out) "opcode=0x%
 # cxl-cdat.c
 cxl_cdat_init(const char *source) "source=%s"
 cxl_cdat_loaded(const char *filename, size_t length, int num_entries) "filename=%s length=%zu entries=%d"
+
+# cxl-events.c
+cxl_event_insert(int log_type, int depth) "log_type=%d depth=%d"
+cxl_event_overflow(int log_type, uint16_t overflow_err_count) "log_type=%d overflow_err_count=%u"
+cxl_event_clear(int log_type, int nr_recs) "log_type=%d nr_recs=%d"
+
+# cxl-host.c
+cxl_cfmws_lookup(uint64_t addr, int fw_index) "addr=0x%"PRIx64" fw=%d"
+cxl_cfmws_target_resolved(uint64_t addr, uint8_t target) "addr=0x%"PRIx64" target=0x%02x"
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH 3/3] hw/cxl: trace event log activity and CFMWS address decoding

Quoted 6 lines · On Wed, 3 Jun 2026 09:34:20 +0800 Junjie Cao <[email protected]> wrote:

Two more debug-prone paths benefit from trace coverage. Event log insertion and clearing are exercised on every error injection or dynamic capacity event, but their interaction with the per-log overflow counter is currently opaque. CFMWS address decoding translates a host physical address through up to two HDM decoder lookups (host bridge and optional switch USP).

Again, two largely unrelated things - two patches please.

Quoted 15 lines

Add five trace points:

* cxl_event_insert - successful enqueue, log type and
resulting depth
* cxl_event_overflow - the artificial overflow threshold
was hit; an event was dropped
* cxl_event_clear - bulk clear, log type and request size

* cxl_cfmws_lookup - entry to address decoding, with
the resolved fixed-window index
* cxl_cfmws_target_resolved - HDM decoder produced a target index
(fires for both the host-bridge and switch USP lookups)

Quoted 5 lines

In non-passthrough topologies, a cxl_cfmws_lookup without a following cxl_cfmws_target_resolved indicates the host-bridge HDM decode failed. In passthrough mode the lookup resolves via pcie_find_port_first() without an HDM target step, so no target_resolved event is expected.

Ah. Good to see you have noted these don't apply to pass through case. I think for that we should probably have an event that makes it clear it is enabled and we won't see these. (Need's Ali's patches to go upstream first though).

Quoted 42 lines

In cxl_event_insert(), cxl_event_count() is hoisted into a local variable so its O(n) queue walk is performed once per insert rather than twice (once for the trace point, once for the existing return- value comparison).

---
 hw/cxl/cxl-events.c | 8 +++++++-
 hw/cxl/cxl-host.c   | 4 ++++
 hw/cxl/trace-events | 9 +++++++++
 3 files changed, 20 insertions(+), 1 deletion(-)
hw/cxl/cxl-events.c +3 -0
index 5356dfb5b3..f4aa419953 100644
@@ -13,6 +13,7 @@
 #include "hw/pci/msix.h"
 #include "hw/cxl/cxl.h"
 #include "hw/cxl/cxl_events.h"
+#include "trace.h"
 
 /* Artificial limit on the number of events a log can hold */
 #define CXL_TEST_EVENT_OVERFLOW 8
@@ -98,6 +99,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
     uint64_t time;
     CXLEventLog *log;
     CXLEvent *entry;
+    int depth;
 
     if (log_type >= CXL_EVENT_TYPE_MAX) {
         return false;
@@ -115,6 +117,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
         }
         log->overflow_err_count++;
         log->last_overflow_timestamp = time;
+        trace_cxl_event_overflow(log_type, log->overflow_err_count);
         return false;
     }
 

hw/cxl/cxl-events.c bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type, · line 139

Quoted 4 lines
hw/cxl/cxl-events.c +1 -0
@@ -133,8 +136,10 @@ bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
     QSIMPLEQ_INSERT_TAIL(&log->events, entry, node);
     cxl_event_set_status(cxlds, log_type, true);
 
+    depth = cxl_event_count(log);

Hmm. I wonder if we can name this more clearly. It is kind of queue depth, but I've not really seen that used with and people might
think that's the maximum space. Maybe occupancy is better?
(Assisted by claude ;)

Quoted 68 lines
hw/cxl/cxl-events.c +3 -1
+    trace_cxl_event_insert(log_type, depth);
     /* Count went from 0 to 1 */
-    return cxl_event_count(log) == 1;
+    return depth == 1;
 }
 
 void cxl_discard_all_event_records(CXLDeviceState *cxlds)
@@ -234,6 +239,7 @@ CXLRetCode cxl_event_clear_records(CXLDeviceState *cxlds,
         entry = cxl_event_get_head(log);
     }
 
+    trace_cxl_event_clear(log_type, pl->nr_recs);
     return CXL_MBOX_SUCCESS;
 }
 
hw/cxl/cxl-host.c +4 -0
index a94b893e99..4157377a95 100644
@@ -21,6 +21,7 @@
 #include "hw/pci/pci_host.h"
 #include "hw/pci/pcie_port.h"
 #include "hw/pci-bridge/pci_expander_bridge.h"
+#include "trace.h"
 
 static void cxl_fixed_memory_window_config(CXLFixedMemoryWindowOptions *object,
                                            int index, Error **errp)
@@ -168,6 +169,7 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr)
     bool target_found;
     PCIDevice *rp, *d;
 
+    trace_cxl_cfmws_lookup(addr, fw->index);
     rb_index = (addr / cxl_decode_ig(fw->enc_int_gran)) % fw->num_targets;
     hb = PCI_HOST_BRIDGE(fw->target_hbs[rb_index]->cxl_host_bridge);
     if (!hb || !hb->bus || !pci_bus_is_cxl(hb->bus)) {
@@ -191,6 +193,7 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr)
         if (!target_found) {
             return NULL;
         }
+        trace_cxl_cfmws_target_resolved(addr, target);
 
         rp = pcie_find_port_by_pn(hb->bus, target);
         if (!rp) {
@@ -227,6 +230,7 @@ static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr)
     if (!target_found) {
         return NULL;
     }
+    trace_cxl_cfmws_target_resolved(addr, target);
 
     d = pcie_find_port_by_pn(&PCI_BRIDGE(d)->sec_bus, target);
     if (!d) {
hw/cxl/trace-events +9 -0
index 15977dce6e..e5f196787d 100644
@@ -10,3 +10,12 @@ cxl_mailbox_handler_return(uint16_t opcode, int ret, size_t len_out) "opcode=0x%
 # cxl-cdat.c
 cxl_cdat_init(const char *source) "source=%s"
 cxl_cdat_loaded(const char *filename, size_t length, int num_entries) "filename=%s length=%zu entries=%d"
+
+# cxl-events.c
+cxl_event_insert(int log_type, int depth) "log_type=%d depth=%d"
+cxl_event_overflow(int log_type, uint16_t overflow_err_count) "log_type=%d overflow_err_count=%u"
+cxl_event_clear(int log_type, int nr_recs) "log_type=%d nr_recs=%d"
+
+# cxl-host.c
+cxl_cfmws_lookup(uint64_t addr, int fw_index) "addr=0x%"PRIx64" fw=%d"
+cxl_cfmws_target_resolved(uint64_t addr, uint8_t target) "addr=0x%"PRIx64" target=0x%02x"
author
#7 lore raw

Re: [PATCH 3/3] hw/cxl: trace event log activity and CFMWS address decoding

Hi Jonathan,

Thanks so much for the detailed review. All points addressed in v2.