PCI/CXL: Add CXL reset support for Type 2 devices
Subsystems (3)
v11 cover PCI/CXL: Add CXL reset support for Type 2 devices
Hi folks!
This series adds CXL Reset support for CXL Type 2 devices through the existing PCI reset_method ABI. The reset sequence follows the CXL 4.0 specification [1], including CXL.cache disable, optional cache writeback, CXL Reset initiation, ResetComplete polling, and ResetError reporting.
The userspace ABI is the existing PCI reset interface:
/sys/bus/pci/devices/.../reset_method
/sys/bus/pci/devices/.../resetUserspace can select "cxl_reset" in reset_method and then trigger reset through the existing reset attribute.
CXL Reset is exposed conservatively. The target must be a single-function CXL Type 2 device with CXL.cache and CXL.mem enabled, and the kernel must have cached HDM state sufficient to validate affected ranges and restore decoder programming. The reset preserves device memory; it does not request CXL Reset Memory Clear.
Following Dan's v6 feedback, this uses the existing PCI reset_method interface instead of adding a memdev-specific sysfs ABI.
v11 changes from v10 [2]:
- Rebase onto v7.3-rc1, which includes the CXL Type 2 enabling work, and use the merged public CXL PCI header.
- Rename the built-in support symbol to CONFIG_CXL_RESET and fix built-in/module linkage for CONFIG_CXL_BUS=m.
- Fix tools/testing/cxl linkage without compiling production helper objects twice in the mock cxl_core module.
- Simplify decoder snapshot handling and preserve endpoint DPA skip ownership under the existing DPA lock.
- Keep policy out of shared decoder parsing by accepting spec-legal zero-size decoder state, while skipping zero-size ranges during reset validation.
- Tighten early HDM cache setup, capability checks, BAR bounds, MSE enable/restore handling, and cache publication locking.
- Keep the conservative multifunction-device rejection requested in earlier review.
- Harden post-reset failure handling so MSE and BME remain disabled when HDM state cannot be safely restored.
- Serialize CXL bus-reset HDM restore against region updates and flush posted HDM writes before PCI_COMMAND can restore bus mastering.
- Address the remaining reviewability and naming feedback from v10.
Motivation:
-----------
- Type 2 devices need a CXL-specific reset mechanism beyond existing PCI reset methods.
- FLR does not reset CXL.cache or CXL.mem protocol state. CXL Reset is the architectural reset mechanism for those protocols.
- The PCI reset_method ABI lets userspace select CXL Reset before falling back to broader bus reset methods.
Change Description:
-------------------
Patch 1: cxl: Move HDM decoder programming helpers
- Introduce a decoder settings snapshot collected under the DPA lock.
- Move low-level settings-based HDM programming into resource.c.
Patch 2: cxl: Make HDM commit helpers available to reset code
- Build generic HDM resource helpers under CONFIG_CXL_RESET.
- Export the lock and commit helpers needed when cxl_core is modular.
Patch 3: cxl: Share HDM decoder decode logic
- Share HDM decoder decode and validation between normal CXL core enumeration and early PCI HDM cache setup.
- Decode zero-size committed decoder state without imposing caller policy.
Patch 4: cxl: Cache decoder settings on PCI devices
- Cache CXL core HDM decoder settings in pci_dev->hdm.
- Refresh the cache as decoders are enumerated, committed, or reset.
Patch 5: cxl: Cache endpoint decoder settings during PCI enumeration
- Snapshot endpoint HDM state during PCI enumeration.
- Cache the HDM register locator as BAR-relative metadata and preserve raw decoder state for reset restore.
Patch 6: cxl: Add CXL Device Reset helper
- Add the internal DVSEC reset sequence.
- Disable CXL.cache, perform cache writeback where supported, initiate CXL Reset, and wait for completion.
Patch 7: cxl: Validate HDM ranges before CXL reset
- Collect enabled cached HDM ranges, reject busy ranges, and perform CPU cache invalidation when a runtime backend is available.
Patch 8: PCI/CXL: Reject CXL Reset on multifunction devices
- Conservatively reject CXL Reset where function scope cannot be assumed.
Patch 9: cxl: Restore CXL HDM state after PCI reset
- Restore cached global and per-decoder HDM state after reset while the target IOMMU reset block remains active.
Patch 10: PCI/CXL: Expose CXL Reset as a PCI reset method
- Add "cxl_reset" to the PCI reset-method table ahead of FLR.
Patch 11: Documentation/ABI: Document CXL Reset PCI reset method
- Document the new reset_method value and reset behavior.
Patch 12: PCI/CXL: Restore HDM state after CXL bus reset
- Restore cached CXL HDM decoder state after cxl_bus reset.
The CPU cache invalidation step depends on cpu_cache_invalidate_memregion() support for the affected address ranges. If no runtime provider is available, the kernel emits a warning and continues after the affected HDM ranges have been reserved.
Example:
echo cxl_reset > /sys/bus/pci/devices/0000:bb:dd.f/reset_method
echo 1 > /sys/bus/pci/devices/0000:bb:dd.f/resetTesting:
- Built every intermediate patch, plus the v7.3-rc1 baseline, with CONFIG_CXL_BUS=m. All 12 patches passed a full kernel build and make M=tools/testing/cxl/ independently.
- Built and booted the series on an arm64 NVIDIA NVL72 system and successfully exercised CXL Reset through the PCI reset sysfs ABI.
- Earlier revision testing ran 100 iterations of cxl_reset and exercised cxl_bus separately with HDM state restoration.
References:
[1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[2] https://lore.kernel.org/linux-cxl/[email protected]/
[3] https://lore.kernel.org/linux-cxl/[email protected]/
[4] https://lore.kernel.org/linux-cxl/[email protected]/
[5] https://lore.kernel.org/linux-cxl/[email protected]/
Srirangan Madhavan (12):
cxl: Move HDM decoder programming helpers
cxl: Make HDM commit helpers available to reset code
cxl: Share HDM decoder decode logic
cxl: Cache decoder settings on PCI devices
cxl: Cache endpoint decoder settings during PCI enumeration
cxl: Add CXL Device Reset helper
cxl: Validate HDM ranges before CXL reset PCI/CXL: Reject CXL Reset on multifunction devices
cxl: Restore CXL HDM state after PCI reset PCI/CXL: Expose CXL Reset as a PCI reset method Documentation/ABI: Document CXL Reset PCI reset method PCI/CXL: Restore HDM state after CXL bus reset
Documentation/ABI/testing/sysfs-bus-pci | 15 + drivers/cxl/Kconfig | 4 + drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/core.h | 8 + drivers/cxl/core/hdm.c | 324 +++--- drivers/cxl/core/pci.c | 7 +- drivers/cxl/core/port.c | 19 + drivers/cxl/core/regs.c | 4 + drivers/cxl/core/resource.c | 1351 +++++++++++++++++++++++ drivers/cxl/cxl.h | 41 - drivers/pci/bus.c | 2 + drivers/pci/pci.c | 8 + drivers/pci/probe.c | 2 + include/cxl/cxl.h | 116 +- include/linux/pci.h | 8 +- include/uapi/linux/pci_regs.h | 14 + tools/testing/cxl/Kbuild | 1 - 17 files changed, 1715 insertions(+), 211 deletions(-) create mode 100644 drivers/cxl/core/resource.c
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Signature · 1 line
2.43.0
PATCH v11 1/12 cxl: Move HDM decoder programming helpers
Introduce struct cxl_decoder_settings as a snapshot of HDM decoder programming state. Collect endpoint skip and switch target-list state in hdm.c while holding the DPA lock, then pass the snapshot to generic commit helpers.
Move the low-level settings-based register programming into resource.c so later reset restore code can reuse it without depending on private decoder types.
--- drivers/cxl/core/Makefile | 1 + drivers/cxl/core/core.h | 3 + drivers/cxl/core/hdm.c | 159 +++++++++++------------------------- drivers/cxl/core/resource.c | 112 +++++++++++++++++++++++++ drivers/cxl/cxl.h | 41 ---------- include/cxl/cxl.h | 65 ++++++++++++++- tools/testing/cxl/Kbuild | 1 + 7 files changed, 227 insertions(+), 155 deletions(-) create mode 100644 drivers/cxl/core/resource.c
drivers/cxl/core/Makefile +1 -0
@@ -6,6 +6,7 @@ ccflags-y += -I$(srctree)/drivers/cxl CFLAGS_trace.o = -DTRACE_INCLUDE_PATH=. -I$(src) cxl_core-y := port.o +cxl_core-y += resource.o cxl_core-y += pmem.o cxl_core-y += regs.o cxl_core-y += memdev.o
drivers/cxl/core/core.h +3 -0
@@ -210,8 +210,11 @@ static inline void devm_cxl_dport_ras_setup(struct cxl_dport *dport) { }int cxl_gpf_port_setup (struct cxl_dport *dport);struct cxl_hdm ; +struct cxl_decoder_settings ;int cxl_hdm_decode_init (struct cxl_dev_state *cxlds,struct cxl_hdm *cxlhdm,struct cxl_endpoint_dvsec_info *info); +int cxl_commit_start (void __iomem *hdm,struct cxl_decoder_settings *settings); +int cxl_commit_wait (void __iomem *hdm,struct cxl_decoder_settings *settings);int cxl_port_get_possible_dports (struct cxl_port *port);#ifdef CONFIG_CXL_FEATURES
drivers/cxl/core/hdm.c +46 -113
212 lines not on this page · raw message
drivers/cxl/core/resource.c new file +112 -0
112 lines not on this page · raw message
drivers/cxl/cxl.h +0 -41
49 lines not on this page · raw message
include/cxl/cxl.h +64 -1
79 lines not on this page · raw message
tools/testing/cxl/Kbuild +1 -0
@@ -60,6 +60,7 @@ cxl_core-y += $(CXL_CORE_SRC)/memdev.o cxl_core-y += $(CXL_CORE_SRC)/mbox.o cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o +cxl_core-y += $(CXL_CORE_SRC)/resource.o cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 01/12] cxl: Move HDM decoder programming helpers
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Introduce struct
cxl_decoder_settingsas a snapshot of HDM decoder programming state. Collect endpoint skip and switch target-list state inhdm.cwhile holding the DPA lock, then pass the snapshot to generic commit helpers.Move the low-level settings-based register programming into
resource.cso later reset restore code can reuse it without depending on private decoder types.Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH v11 2/12 cxl: Make HDM commit helpers available to reset code
In reply to Srirangan MadhavanBuild the generic HDM resource helpers under CONFIG_CXL_RESET so PCI reset paths can use them when cxl_core is modular. Export the lock and commit entry points consumed by cxl_core, and leave resource.o out of the mock cxl_core module once it is built into the kernel.
--- drivers/cxl/Kconfig | 4 ++++ drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/resource.c | 3 +++ tools/testing/cxl/Kbuild | 1 - 4 files changed, 8 insertions(+), 2 deletions(-)
drivers/cxl/Kconfig +4 -0
@@ -6,6 +6,7 @@ menuconfig CXL_BUS select FW_UPLOAD select PCI_DOE select FIRMWARE_TABLE + select CXL_RESET select NUMA_KEEP_MEMINFO if NUMA_MEMBLKS select FWCTL if CXL_FEATURES help
@@ -243,4 +244,7 @@ config CXL_ATL depends on CXL_REGION depends on ACPI_PRMT && AMD_NB +config CXL_RESET + bool + endif
drivers/cxl/core/Makefile +1 -1
@@ -1,12 +1,12 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_CXL_BUS) += cxl_core.o +obj-$(CONFIG_CXL_RESET) += resource.o obj-$(CONFIG_CXL_SUSPEND) += suspend.o ccflags-y += -I$(srctree)/drivers/cxl CFLAGS_trace.o = -DTRACE_INCLUDE_PATH=. -I$(src) cxl_core-y := port.o -cxl_core-y += resource.o cxl_core-y += pmem.o cxl_core-y += regs.o cxl_core-y += memdev.o
drivers/cxl/core/resource.c +3 -0
@@ -13,6 +13,7 @@ struct cxl_rwsem cxl_rwsem = { .region = __RWSEM_INITIALIZER(cxl_rwsem.region), .dpa = __RWSEM_INITIALIZER(cxl_rwsem.dpa), }; +EXPORT_SYMBOL_FOR_MODULES (cxl_rwsem,"cxl_core" );static void cxld_set_interleave (struct cxl_decoder_settings *settings,u32 *ctrl) {
@@ -105,8 +106,10 @@ int cxl_commit_start(void __iomem *hdm, struct cxl_decoder_settings *settings)lockdep_assert_held (&cxl_rwsem.dpa);return setup_hw_decoder (hdm, settings); } +EXPORT_SYMBOL_FOR_MODULES (cxl_commit_start,"cxl_core" );int cxl_commit_wait (void __iomem *hdm,struct cxl_decoder_settings *settings) {return cxld_await_commit (hdm, settings->id); } +EXPORT_SYMBOL_FOR_MODULES (cxl_commit_wait,"cxl_core" );
tools/testing/cxl/Kbuild +0 -1
@@ -60,7 +60,6 @@ cxl_core-y += $(CXL_CORE_SRC)/memdev.o cxl_core-y += $(CXL_CORE_SRC)/mbox.o cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o -cxl_core-y += $(CXL_CORE_SRC)/resource.o cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 02/12] cxl: Make HDM commit helpers available to reset code
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
Build the generic HDM resource helpers under
CONFIG_CXL_RESETso PCI reset paths can use them whencxl_coreis modular. Export the lock and commit entry points consumed bycxl_core, and leave resource.o out of the mockcxl_coremodule once it is built into the kernel.Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
PATCH v11 3/12 cxl: Share HDM decoder decode logic
In reply to Srirangan MadhavanMove HDM decoder register decoding into a helper shared by normal CXL core enumeration and early PCI HDM cache setup. This keeps validation of base, range overflow, interleave, target type, and enable state in one place before adding another HDM parser.
Keep caller-owned policy out of the decode helper. A committed zero-size decoder now decodes successfully, while init_hdm_decoder() retains its existing zero-size rejection.
Preserve endpoint DPA state ownership by using the decoded skip value as a local input to devm_cxl_dpa_reserve(). The reservation helper updates cxled->skip under cxl_rwsem.dpa.
--- drivers/cxl/core/core.h | 4 ++ drivers/cxl/core/hdm.c | 75 +++++++++++++------------------------ drivers/cxl/core/port.c | 19 ++++++++++ drivers/cxl/core/resource.c | 43 +++++++++++++++++++++ 4 files changed, 91 insertions(+), 50 deletions(-)
drivers/cxl/core/core.h +4 -0
@@ -157,6 +157,7 @@ long cxl_pci_get_latency(struct pci_dev *pdev);int cxl_pci_get_bandwidth (struct pci_dev *pdev,struct access_coordinate *c);int cxl_port_get_switch_dport_bandwidth (struct cxl_port *port,struct access_coordinate *c); +struct pci_dev *cxl_port_get_uport_pci_dev (struct cxl_port *port);static inline struct device *port_to_host (struct cxl_port *port) {
@@ -215,6 +216,9 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, struct cxl_endpoint_dvsec_info *info);int cxl_commit_start (void __iomem *hdm,struct cxl_decoder_settings *settings);int cxl_commit_wait (void __iomem *hdm,struct cxl_decoder_settings *settings); +int cxl_hdm_decode_decoder (struct cxl_decoder_settings *settings,int id, +u32 ctrl,u64 base,u64 size,u64 target_or_skip, +bool *committed);int cxl_port_get_possible_dports (struct cxl_port *port);#ifdef CONFIG_CXL_FEATURES
drivers/cxl/core/hdm.c +25 -50
116 lines not on this page · raw message
drivers/cxl/core/port.c +19 -0
@@ -34,6 +34,25 @@static DEFINE_IDA (cxl_port_ida);static DEFINE_XARRAY (cxl_root_buses); +struct pci_dev *cxl_port_get_uport_pci_dev (struct cxl_port *port) +{ +struct device *uport = port->uport_dev; +struct device *host; + +if (is_cxl_memdev (uport)) { +struct cxl_memdev *cxlmd =to_cxl_memdev (uport); + + host = cxlmd->dev.parent; + }else { + host = uport; + } + +if (!host || !dev_is_pci (host)) +return NULL ; + +return pci_dev_get (to_pci_dev (host)); +} +/* * The terminal device in PCI is NULL and @platform_bus * for platform devices (for cxl_test)
drivers/cxl/core/resource.c +43 -0
46 lines not on this page · raw message
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 03/12] cxl: Share HDM decoder decode logic
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Move HDM decoder register decoding into a helper shared by normal CXL core enumeration and early PCI HDM cache setup. This keeps validation of base, range overflow, interleave, target type, and enable state in one place before adding another HDM parser.
Keep caller-owned policy out of the decode helper. A committed zero-size decoder now decodes successfully, while
init_hdm_decoder()retains its existing zero-size rejection.
Quoted 6 lines
Preserve endpoint DPA state ownership by using the decoded skip value as a local input to
devm_cxl_dpa_reserve(). The reservation helper updatescxled->skipunder cxl_rwsem.dpa.Reported-by Fenghua Yu <[email protected]>Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
PATCH v11 4/12 cxl: Cache decoder settings on PCI devices
In reply to Srirangan MadhavanAdd CXL core plumbing to refresh a PCI device HDM decoder cache when decoders are enumerated, committed, or reset. PCI reset paths can use this snapshot to restore HDM programming without walking CXL topology during reset recovery.
The cache is populated by PCI-side discovery in a follow-on patch. Until then, the CXL core update path is a no-op when no PCI HDM cache is present.
--- drivers/cxl/core/hdm.c | 68 +++++++++++++++++++++++++++++++++++++++++- include/cxl/cxl.h | 12 ++++++++ include/linux/pci.h | 6 ++++ 3 files changed, 85 insertions(+), 1 deletion(-)
drivers/cxl/core/hdm.c +67 -1
@@ -16,6 +16,9 @@* for enumerating these registers and capabilities. */ +static void cxl_decoder_snapshot (struct cxl_decoder *cxld, +struct cxl_decoder_settings *settings); +static int add_hdm_decoder (struct cxl_port *port,struct cxl_decoder *cxld) {int rc;
@@ -84,6 +87,62 @@ static void parse_hdm_decoder_caps(struct cxl_hdm *cxlhdm) cxlhdm->iw_cap_mask |=BIT (16 ); } +static bool __cxl_pci_hdm_decoder_count_match (struct pci_dev *pdev, +int decoder_count) +{ +struct cxl_hdm_info *info; +bool match = true; + +down_read (&cxl_rwsem.dpa); + info = pdev->hdm; +if (info) { +if (info->decoder_count != decoder_count) { +pci_warn (pdev, +"CXL HDM cache decoder count mismatch: cached=%d hdm=%d\n" , + info->decoder_count, decoder_count); + match = false; + } + } +up_read (&cxl_rwsem.dpa); + +return match; +} + +static bool cxl_pci_hdm_decoder_count_match (struct cxl_hdm *cxlhdm) +{ + struct pci_dev *pdev __free(pci_dev_put) = +cxl_port_get_uport_pci_dev (cxlhdm->port); + +if (!pdev) +return true; + +return __cxl_pci_hdm_decoder_count_match (pdev, cxlhdm->decoder_count); +} + +static void cxl_hdm_save_decoder_info (struct cxl_hdm *cxlhdm, +struct cxl_decoder *cxld) +{ + struct pci_dev *pdev __free(pci_dev_put) = +cxl_port_get_uport_pci_dev (cxlhdm->port); +struct cxl_decoder_settings *settings; +struct cxl_hdm_info *info; + +if (!pdev) +return ; + +guard (rwsem_write)(&cxl_rwsem.dpa); + info = pdev->hdm; +if (!info || cxld->id >= info->decoder_count) +return ; + + settings = &info->settings[cxld->id]; + *settings = (struct cxl_decoder_settings ) { + .id = cxld->id, + }; +if (cxld->flags &CXL_DECODER_F_ENABLE ) +cxl_decoder_snapshot (cxld, settings); +} +static bool should_emulate_decoders (struct cxl_endpoint_dvsec_info *info) {struct cxl_hdm *cxlhdm;
@@ -778,6 +837,7 @@ static int cxl_decoder_commit(struct cxl_decoder *cxld) } port->commit_end++; cxld->flags |=CXL_DECODER_F_ENABLE ; +cxl_hdm_save_decoder_info (cxlhdm, cxld);return 0 ; }
@@ -850,6 +910,7 @@ static void cxl_decoder_reset(struct cxl_decoder *cxld)writel (0 , hdm +CXL_HDM_DECODER0_BASE_LOW_OFFSET (id)); cxld->flags &= ~CXL_DECODER_F_ENABLE ; +cxl_hdm_save_decoder_info (cxlhdm, cxld);/* Userspace is now responsible for reconfiguring this decoder */ if (is_endpoint_decoder (&cxld->dev)) {
@@ -1068,11 +1129,15 @@ static int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm,struct cxl_port *port = cxlhdm->port;int i;u64 dpa_base =0 ; +int rc;cxl_settle_decoders (cxlhdm); +if (!cxl_pci_hdm_decoder_count_match (cxlhdm)) +return -ENXIO ; +for (i =0 ; i < cxlhdm->decoder_count; i++) { -int rc, target_count = cxlhdm->target_count; +int target_count = cxlhdm->target_count;struct cxl_decoder *cxld;if (is_cxl_endpoint (port)) {
@@ -1107,6 +1172,7 @@ static int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm,put_device (&cxld->dev);return rc; } +cxl_hdm_save_decoder_info (cxlhdm, cxld); rc =add_hdm_decoder (port, cxld);if (rc) { dev_warn(&port->dev,
include/cxl/cxl.h +12 -0
@@ -133,6 +133,18 @@ struct cxl_regs { ); }; +#define CXL_HDM_DECODER_MAX_COUNT 32 + +/** +* struct cxl_hdm_info - PCI device HDM decoder programming cache +* @decoder_count: number of decoder settings entries +* @settings: cached per-decoder programming state +*/ +struct cxl_hdm_info { +int decoder_count; +struct cxl_decoder_settings settings[CXL_HDM_DECODER_MAX_COUNT ]; +}; +struct cxl_reg_map {bool valid;int id;
include/linux/pci.h +6 -0
@@ -339,6 +339,9 @@ struct pcie_link_state;struct pci_sriov ;struct pci_p2pdma ;struct rcec_ea ; +#ifdef CONFIG_CXL_RESET +struct cxl_hdm_info ; +#endif /* struct pci_dev - describes a PCI device *
@@ -566,6 +569,9 @@ struct pci_dev {#ifdef CONFIG_PCI_DOE struct xarray doe_mbs;/* Data Object Exchange mailboxes */ #endif +#ifdef CONFIG_CXL_RESET +struct cxl_hdm_info *hdm;/* CXL HDM decoder reset state */ +#endif #ifdef CONFIG_PCI_NPEM struct npem *npem;/* Native PCIe Enclosure Management */ #endif
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 04/12] cxl: Cache decoder settings on PCI devices
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Add CXL core plumbing to refresh a PCI device HDM decoder cache when decoders are enumerated, committed, or reset. PCI reset paths can use this snapshot to restore HDM programming without walking CXL topology during reset recovery.
The cache is populated by PCI-side discovery in a follow-on patch. Until then, the CXL core update path is a no-op when no PCI HDM cache is present.
Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
PATCH v11 5/12 cxl: Cache endpoint decoder settings during PCI enumeration
In reply to Srirangan MadhavanPopulate pci_dev->hdm for CXL.mem functions from pci_bus_add_device(), after final PCI fixups and state save but before driver binding. This gives driver-free reset paths an early HDM snapshot while avoiding the pre-resource-assignment window in PCI capability initialization.
Use the CXL Register Locator BAR Indicator to find the component register BAR, reject unassigned, disabled, or zero memory BAR resources before temporarily enabling Memory Space, and restore the original PCI_COMMAND value before returning.
CXL core reuses and refreshes the same cache as decoders are committed or reset, and retries cache setup during CXL HDM enumeration if the PCI bus-add attempt did not publish a cache. Move the register helpers into the built-in CONFIG_CXL_RESET set so the early cache path is available without cxl_core, and keep the cxl-test mock core from building a duplicate regs.o.
--- drivers/cxl/core/Makefile | 3 +- drivers/cxl/core/core.h | 1 + drivers/cxl/core/hdm.c | 28 ++- drivers/cxl/core/pci.c | 7 +- drivers/cxl/core/regs.c | 4 + drivers/cxl/core/resource.c | 340 ++++++++++++++++++++++++++++++++++++ drivers/pci/bus.c | 2 + drivers/pci/probe.c | 2 + include/cxl/cxl.h | 26 +++ tools/testing/cxl/Kbuild | 1 - 10 files changed, 406 insertions(+), 8 deletions(-)
drivers/cxl/core/Makefile +1 -2
@@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_CXL_BUS) += cxl_core.o -obj-$(CONFIG_CXL_RESET) += resource.o +obj-$(CONFIG_CXL_RESET) += regs.o resource.o obj-$(CONFIG_CXL_SUSPEND) += suspend.o ccflags-y += -I$(srctree)/drivers/cxl
@@ -8,7 +8,6 @@ CFLAGS_trace.o = -DTRACE_INCLUDE_PATH=. -I$(src) cxl_core-y := port.o cxl_core-y += pmem.o -cxl_core-y += regs.o cxl_core-y += memdev.o cxl_core-y += mbox.o cxl_core-y += pci.o
drivers/cxl/core/core.h +1 -0
@@ -158,6 +158,7 @@ int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c);int cxl_port_get_switch_dport_bandwidth (struct cxl_port *port,struct access_coordinate *c);struct pci_dev *cxl_port_get_uport_pci_dev (struct cxl_port *port); +int cxl_pci_get_device_dvsec_cap (struct pci_dev *pdev,int dvsec,u16 *cap);static inline struct device *port_to_host (struct cxl_port *port) {
drivers/cxl/core/hdm.c +26 -2
@@ -88,14 +88,17 @@ static void parse_hdm_decoder_caps(struct cxl_hdm *cxlhdm) }static bool __cxl_pci_hdm_decoder_count_match (struct pci_dev *pdev, -int decoder_count) +int decoder_count, +bool *present) {struct cxl_hdm_info *info;bool match = true; + *present = false;down_read (&cxl_rwsem.dpa); info = pdev->hdm;if (info) { + *present = true;if (info->decoder_count != decoder_count) { pci_warn(pdev, "CXL HDM cache decoder count mismatch: cached=%d hdm=%d\n",
@@ -112,11 +115,21 @@ static bool cxl_pci_hdm_decoder_count_match(struct cxl_hdm *cxlhdm) { struct pci_dev *pdev __free(pci_dev_put) =cxl_port_get_uport_pci_dev (cxlhdm->port); +bool present;if (!pdev)return true; -return __cxl_pci_hdm_decoder_count_match (pdev, cxlhdm->decoder_count); +if (!__cxl_pci_hdm_decoder_count_match (pdev, cxlhdm->decoder_count, + &present)) +return false; +if (present) +return true; + +pci_cxl_hdm_init (pdev); + +return __cxl_pci_hdm_decoder_count_match (pdev, cxlhdm->decoder_count, + &present); } static void cxl_hdm_save_decoder_info(struct cxl_hdm *cxlhdm,
@@ -126,10 +139,17 @@ static void cxl_hdm_save_decoder_info(struct cxl_hdm *cxlhdm,cxl_port_get_uport_pci_dev (cxlhdm->port);struct cxl_decoder_settings *settings;struct cxl_hdm_info *info; +u32 global_ctrl =0 ; +bool has_hdm_decoder;if (!pdev)return ; + has_hdm_decoder = !!cxlhdm->regs.hdm_decoder; +if (has_hdm_decoder) + global_ctrl =readl (cxlhdm->regs.hdm_decoder + +CXL_HDM_DECODER_CTRL_OFFSET ); +guard (rwsem_write)(&cxl_rwsem.dpa); info = pdev->hdm;if (!info || cxld->id >= info->decoder_count)
@@ -139,6 +159,10 @@ static void cxl_hdm_save_decoder_info(struct cxl_hdm *cxlhdm, *settings = (struct cxl_decoder_settings ) { .id = cxld->id, }; + +if (has_hdm_decoder) + info->global_ctrl = global_ctrl; +if (cxld->flags &CXL_DECODER_F_ENABLE )cxl_decoder_snapshot (cxld, settings); }
drivers/cxl/core/pci.c +4 -3
@@ -274,9 +274,10 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,return -ENXIO ; } - rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CAP , &cap); -if (rc) -return pcibios_err_to_errno (rc); + rc =cxl_pci_get_device_dvsec_cap (pdev, d, &cap); +if (rc <0 ) +return rc; + d = rc;if (!(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) {dev_dbg (dev,"Not MEM Capable\n" );
drivers/cxl/core/regs.c +4 -0
@@ -200,6 +200,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,return ret_val; } +EXPORT_SYMBOL_NS_GPL (devm_cxl_iomap_block,"CXL" ); int cxl_map_component_regs(const struct cxl_register_map *map, struct cxl_component_regs *regs,
@@ -518,6 +519,7 @@ u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb)return offset; } +EXPORT_SYMBOL_NS_GPL (cxl_rcrb_to_aer,"CXL" );static resource_size_t cxl_rcrb_to_linkcap (struct device *dev,struct cxl_dport *dport) {
@@ -634,6 +636,7 @@ resource_size_t __rcrb_to_component(struct device *dev, struct cxl_rcrb_info *rireturn component_reg_phys; } +EXPORT_SYMBOL_NS_GPL (__rcrb_to_component,"CXL" );resource_size_t cxl_rcd_component_reg_phys (struct device *dev,struct cxl_dport *dport)
@@ -642,3 +645,4 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev,return CXL_RESOURCE_NONE ;return __rcrb_to_component (dev, &dport->rcrb,CXL_RCRB_UPSTREAM ); } +EXPORT_SYMBOL_NS_GPL (cxl_rcd_component_reg_phys,"CXL" );
drivers/cxl/core/resource.c +340 -0
352 lines not on this page · raw message
drivers/pci/bus.c +2 -0
@@ -18,6 +18,7 @@#include <linux/pm_runtime.h> #include <linux/proc_fs.h> #include <linux/slab.h> +#include <cxl/cxl.h> #include "pci.h"
@@ -359,6 +360,7 @@ void pci_bus_add_device(struct pci_dev *dev)/* Save config space for error recoverability */ pci_save_state (dev); +pci_cxl_hdm_init (dev);/* * Enable runtime PM, which potentially allows the device to
drivers/pci/probe.c +2 -0
@@ -24,6 +24,7 @@#include <linux/pm_runtime.h> #include <linux/bitfield.h> #include <trace/events/pci.h> +#include <cxl/cxl.h> #include "pci.h" static struct resource busn_resource = {
@@ -2484,6 +2485,7 @@ static void pci_release_dev(struct device *dev)struct pci_dev *pci_dev; pci_dev =to_pci_dev (dev); +pci_cxl_hdm_release (pci_dev);pci_release_capabilities (pci_dev);pci_release_of_node (pci_dev);pcibios_release_device (pci_dev);
include/cxl/cxl.h +26 -0
@@ -26,6 +26,7 @@ enum cxl_devtype { };struct cxl_region ; +struct pci_dev ;enum cxl_decoder_type {CXL_DECODER_DEVMEM =2 ,
@@ -135,16 +136,41 @@ struct cxl_regs {#define CXL_HDM_DECODER_MAX_COUNT 32 +struct cxl_hdm_decoder_state ; +/** * struct cxl_hdm_info - PCI device HDM decoder programming cache * @decoder_count: number of decoder settings entries +* @hdm_bar: BAR containing the HDM decoder registers +* @hdm_offset: HDM decoder register offset relative to @hdm_bar +* @hdm_size: HDM decoder register resource size +* @global_ctrl: cached HDM decoder global control register +* @decoder_state: cached raw per-decoder register state * @settings: cached per-decoder programming state */ struct cxl_hdm_info {int decoder_count; +int hdm_bar; +resource_size_t hdm_offset; +resource_size_t hdm_size; +u32 global_ctrl; +struct cxl_hdm_decoder_state *decoder_state;struct cxl_decoder_settings settings[CXL_HDM_DECODER_MAX_COUNT ]; }; +#ifdef CONFIG_CXL_RESET +void pci_cxl_hdm_init (struct pci_dev *pdev); +void pci_cxl_hdm_release (struct pci_dev *pdev); +#else +static inline void pci_cxl_hdm_init (struct pci_dev *pdev) +{ +} + +static inline void pci_cxl_hdm_release (struct pci_dev *pdev) +{ +} +#endif +struct cxl_reg_map {bool valid;int id;
tools/testing/cxl/Kbuild +0 -1
@@ -55,7 +55,6 @@ obj-m += cxl_core.o cxl_core-y := $(CXL_CORE_SRC)/port.o cxl_core-y += $(CXL_CORE_SRC)/pmem.o -cxl_core-y += $(CXL_CORE_SRC)/regs.o cxl_core-y += $(CXL_CORE_SRC)/memdev.o cxl_core-y += $(CXL_CORE_SRC)/mbox.o cxl_core-y += $(CXL_CORE_SRC)/pci.o
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 05/12] cxl: Cache endpoint decoder settings during PCI enumeration
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Mixing goto-based error handling with scope-based cleanup helpers (`scoped_guard`) in…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Mixing goto-based error handling with scope-based cleanup helpers (scoped_guard) in __pci_cxl_hdm_init(). --
commit 240aef0efe56885a67760e149786f74d4a709d41
Author: Srirangan Madhavan <[email protected]>
cxl: Cache endpoint decoder settings during PCI enumeration
This commit populates pci_dev->hdm for CXL.mem functions from pci_bus_add_device(), after final PCI fixups but before driver binding. It uses the CXL Register Locator BAR Indicator to find the component register BAR and reject unassigned or disabled memory BAR resources.
Quoted 4 lines
drivers/cxl/core/resource.c +0
-0
[ ... ]
Quoted 10 lines
drivers/cxl/core/resource.c +10
-0+static int __pci_cxl_hdm_init (struct pci_dev *pdev) +{ +struct cxl_register_map map = {0 }; +struct cxl_hdm_info *info; +int rc; + + scoped_guard(rwsem_read, &cxl_rwsem.dpa) { +if (pdev->hdm) +return 0 ; + }
[ ... ]
Quoted 13 lines
drivers/cxl/core/resource.c +13
-0+ info =kzalloc_obj (*info,GFP_KERNEL ); +if (!info) +return -ENOMEM ; + + rc =cxl_pci_hdm_read_info (pdev, &map, info); +if (rc) + goto out_free_info; + + scoped_guard(rwsem_write, &cxl_rwsem.dpa) { +if (!pdev->hdm) { + pdev->hdm = info; + info =NULL ; + }else {
drivers/cxl/core/resource.c int cxl_hdm_decode_decoder(struct cxl_decoder_settings *settings, int id, · line 488
Quoted 10 lines
drivers/cxl/core/resource.c +10
-0+cxl_pci_hdm_info_free (info); + } + } + +return 0 ; + +out_free_info: +cxl_pci_hdm_info_free (info); +return rc; +}
[Severity: Low]
Does this code violate the cleanup API guidelines by mixing goto-based error handling with scope-based cleanup helpers (scoped_guard) in __pci_cxl_hdm_init()?
The expectation in include/linux/cleanup.h is that goto and cleanup helpers are never mixed in the same function. Could this be converted to use scope-based cleanup for the info allocation as well, to avoid confusing ownership and maintenance hazards in the future?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Li Ming
reviewerRe: [PATCH v11 05/12] cxl: Cache endpoint decoder settings during PCI enumeration
In reply to Srirangan MadhavanQuoted 93 lines · Srirangan Madhavan
Populate
pci_dev->hdmfor CXL.mem functions frompci_bus_add_device(), after final PCI fixups and state save but before driver binding. This gives driver-free reset paths an early HDM snapshot while avoiding the pre-resource-assignment window in PCI capability initialization.Use the CXL Register Locator BAR Indicator to find the component register BAR, reject unassigned, disabled, or zero memory BAR resources before temporarily enabling Memory Space, and restore the original
PCI_COMMANDvalue before returning.CXL core reuses and refreshes the same cache as decoders are committed or reset, and retries cache setup during CXL HDM enumeration if the PCI bus-add attempt did not publish a cache. Move the register helpers into the built-in
CONFIG_CXL_RESETset so the early cache path is available withoutcxl_core, and keep the cxl-test mock core from building a duplicate regs.o.Signed-off-by Srirangan Madhavan <[email protected]>--- drivers/cxl/core/Makefile | 3 +- drivers/cxl/core/core.h | 1 + drivers/cxl/core/hdm.c | 28 ++- drivers/cxl/core/pci.c | 7 +- drivers/cxl/core/regs.c | 4 + drivers/cxl/core/resource.c | 340 ++++++++++++++++++++++++++++++++++++ drivers/pci/bus.c | 2 + drivers/pci/probe.c | 2 + include/cxl/cxl.h | 26 +++ tools/testing/cxl/Kbuild | 1 - 10 files changed, 406 insertions(+), 8 deletions(-)drivers/cxl/core/Makefile +1
-2@@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_CXL_BUS) += cxl_core.o -obj-$(CONFIG_CXL_RESET) += resource.o +obj-$(CONFIG_CXL_RESET) += regs.o resource.o obj-$(CONFIG_CXL_SUSPEND) += suspend.o ccflags-y += -I$(srctree)/drivers/cxl@@ -8,7 +8,6 @@ CFLAGS_trace.o = -DTRACE_INCLUDE_PATH=. -I$(src) cxl_core-y := port.o cxl_core-y += pmem.o -cxl_core-y += regs.o cxl_core-y += memdev.o cxl_core-y += mbox.o cxl_core-y += pci.odrivers/cxl/core/core.h +1
-0@@ -158,6 +158,7 @@ int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c);int cxl_port_get_switch_dport_bandwidth (struct cxl_port *port,struct access_coordinate *c);struct pci_dev *cxl_port_get_uport_pci_dev (struct cxl_port *port); +int cxl_pci_get_device_dvsec_cap (struct pci_dev *pdev,int dvsec,u16 *cap);static inline struct device *port_to_host (struct cxl_port *port) {drivers/cxl/core/hdm.c +5
-1@@ -88,14 +88,17 @@ static void parse_hdm_decoder_caps(struct cxl_hdm *cxlhdm) }static bool __cxl_pci_hdm_decoder_count_match (struct pci_dev *pdev, -int decoder_count) +int decoder_count, +bool *present) {struct cxl_hdm_info *info;bool match = true; + *present = false;down_read (&cxl_rwsem.dpa); info = pdev->hdm;if (info) { + *present = true;if (info->decoder_count != decoder_count) { pci_warn(pdev, "CXL HDM cache decoder count mismatch: cached=%d hdm=%d\n",@@ -112,11 +115,21 @@ static bool cxl_pci_hdm_decoder_count_match(struct cxl_hdm *cxlhdm) { struct pci_dev *pdev __free(pci_dev_put) =cxl_port_get_uport_pci_dev (cxlhdm->port); +bool present;if (!pdev)
drivers/cxl/core/hdm.c static bool cxl_pci_hdm_decoder_count_match(struct cxl_hdm *cxlhdm) · line 129
Quoted 10 lines
drivers/cxl/core/hdm.c +7
-1return true; -return __cxl_pci_hdm_decoder_count_match (pdev, cxlhdm->decoder_count); +if (!__cxl_pci_hdm_decoder_count_match (pdev, cxlhdm->decoder_count, + &present)) +return false; +if (present) +return true; + +pci_cxl_hdm_init (pdev);
I forgot to ask a question in v10, sorry about that.
I am wondering why calling pci_cxl_hdm_init() in cxl_pci_hdm_decoder_count_match() is needed. pci_bus_add_device() already does pci_cxl_hdm_init(), so if my understanding is correct, pdev->hdm is not NULL here, so this pci_cxl_hdm_init() here will not do anything except checking if pdev->hdm is NULL. I am not sure if I miss some cases for that.
Quoted 550 lines
drivers/cxl/core/hdm.c +14
-0+ +return __cxl_pci_hdm_decoder_count_match (pdev, cxlhdm->decoder_count, + &present); } static void cxl_hdm_save_decoder_info(struct cxl_hdm *cxlhdm,@@ -126,10 +139,17 @@ static void cxl_hdm_save_decoder_info(struct cxl_hdm *cxlhdm,cxl_port_get_uport_pci_dev (cxlhdm->port);struct cxl_decoder_settings *settings;struct cxl_hdm_info *info; +u32 global_ctrl =0 ; +bool has_hdm_decoder;if (!pdev)return ; + has_hdm_decoder = !!cxlhdm->regs.hdm_decoder; +if (has_hdm_decoder) + global_ctrl =readl (cxlhdm->regs.hdm_decoder + +CXL_HDM_DECODER_CTRL_OFFSET ); +guard (rwsem_write)(&cxl_rwsem.dpa); info = pdev->hdm;if (!info || cxld->id >= info->decoder_count)@@ -139,6 +159,10 @@ static void cxl_hdm_save_decoder_info(struct cxl_hdm *cxlhdm, *settings = (struct cxl_decoder_settings ) { .id = cxld->id, }; + +if (has_hdm_decoder) + info->global_ctrl = global_ctrl; +if (cxld->flags &CXL_DECODER_F_ENABLE )cxl_decoder_snapshot (cxld, settings); }drivers/cxl/core/pci.c +4
-3@@ -274,9 +274,10 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,return -ENXIO ; } - rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CAP , &cap); -if (rc) -return pcibios_err_to_errno (rc); + rc =cxl_pci_get_device_dvsec_cap (pdev, d, &cap); +if (rc <0 ) +return rc; + d = rc;if (!(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) {dev_dbg (dev,"Not MEM Capable\n" );drivers/cxl/core/regs.c +4
-0@@ -200,6 +200,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,return ret_val; } +EXPORT_SYMBOL_NS_GPL (devm_cxl_iomap_block,"CXL" ); int cxl_map_component_regs(const struct cxl_register_map *map, struct cxl_component_regs *regs,@@ -518,6 +519,7 @@ u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb)return offset; } +EXPORT_SYMBOL_NS_GPL (cxl_rcrb_to_aer,"CXL" );static resource_size_t cxl_rcrb_to_linkcap (struct device *dev,struct cxl_dport *dport) {@@ -634,6 +636,7 @@ resource_size_t __rcrb_to_component(struct device *dev, struct cxl_rcrb_info *rireturn component_reg_phys; } +EXPORT_SYMBOL_NS_GPL (__rcrb_to_component,"CXL" );resource_size_t cxl_rcd_component_reg_phys (struct device *dev,struct cxl_dport *dport)@@ -642,3 +645,4 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev,return CXL_RESOURCE_NONE ;return __rcrb_to_component (dev, &dport->rcrb,CXL_RCRB_UPSTREAM ); } +EXPORT_SYMBOL_NS_GPL (cxl_rcd_component_reg_phys,"CXL" );drivers/cxl/core/resource.c +340
-0@@ -2,9 +2,17 @@/* Copyright (c) 2026 NVIDIA Corporation & Affiliates */ #include <linux/delay.h> #include <linux/bug.h> +#include <linux/bitfield.h> +#include <linux/cleanup.h> #include <linux/errno.h> #include <linux/export.h> +#include <linux/io.h> +#include <linux/ioport.h> #include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/slab.h> + +#include <cxl/pci.h> #include "cxl.h" #include "core.h" @@ -156,3 +164,335 @@ int cxl_hdm_decode_decoder(struct cxl_decoder_settings *settings, int id, &settings->interleave_granularity); }EXPORT_SYMBOL_FOR_MODULES (cxl_hdm_decode_decoder,"cxl_core" ); + +struct cxl_hdm_decoder_state { +u32 ctrl; +u32 base_low; +u32 base_high; +u32 size_low; +u32 size_high; +u32 target_low; +u32 target_high; +}; + +static void cxl_pci_hdm_info_free (struct cxl_hdm_info *info) +{ +if (!info) +return ; + +kfree (info->decoder_state); +kfree (info); +} + +void pci_cxl_hdm_release (struct pci_dev *pdev) +{ +struct cxl_hdm_info *info; + + scoped_guard(rwsem_write, &cxl_rwsem.dpa) { + info = pdev->hdm; + pdev->hdm =NULL ; + } + +cxl_pci_hdm_info_free (info); +} + +static bool cxl_pci_bar_usable (struct pci_dev *pdev,int bar) +{ +struct resource *res = &pdev->resource[bar]; + +if (!pci_resource_len (pdev, bar)) +return false; +if (res->flags & (IORESOURCE_UNSET |IORESOURCE_DISABLED )) +return false; +if (resource_type (res) !=IORESOURCE_MEM ) +return false; +if (!res->start || !res->end) +return false; + +return true; +} + +static int cxl_pci_hdm_find_bar (struct pci_dev *pdev,resource_size_t hdm_start, +resource_size_t hdm_size,int *bar, +resource_size_t *offset) +{ +resource_size_t hdm_end; + +if (!hdm_size) +return -EINVAL ; + + hdm_end = hdm_start + hdm_size -1 ; +if (hdm_end < hdm_start) +return -EINVAL ; + +for (int i =0 ; i <PCI_STD_NUM_BARS ; i++) { +struct resource *res = &pdev->resource[i]; + +if (!cxl_pci_bar_usable (pdev, i)) +continue ; +if (hdm_start < res->start || hdm_end > res->end) +continue ; + +if (bar) + *bar = i; +if (offset) + *offset = hdm_start - res->start; +return 0 ; + } + +return -ENODEV ; +} + +static void __iomem *cxl_pci_hdm_map (struct pci_dev *pdev, +struct cxl_register_map *map, +struct cxl_hdm_info *info) +{ +struct cxl_reg_map *hdm_map = &map->component_map.hdm_decoder; +resource_size_t hdm_start; +void __iomem *hdm; +int rc; + + hdm_start = map->resource + hdm_map->offset; + info->hdm_size = hdm_map->size; + + rc =cxl_pci_hdm_find_bar (pdev, hdm_start, info->hdm_size, + &info->hdm_bar, &info->hdm_offset); +if (rc) +return ERR_PTR (rc); + + hdm =ioremap (hdm_start, info->hdm_size); +if (!hdm) { +pci_err (pdev,"failed to map CXL HDM decoder registers\n" ); +return ERR_PTR (-ENOMEM ); + } + +return hdm; +} + +static void cxl_pci_hdm_read_decoder_state (struct cxl_hdm_decoder_state *state, +void __iomem *hdm,int id) +{ + state->ctrl =readl (hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); + state->base_low =readl (hdm +CXL_HDM_DECODER0_BASE_LOW_OFFSET (id)); + state->base_high =readl (hdm +CXL_HDM_DECODER0_BASE_HIGH_OFFSET (id)); + state->size_low =readl (hdm +CXL_HDM_DECODER0_SIZE_LOW_OFFSET (id)); + state->size_high =readl (hdm +CXL_HDM_DECODER0_SIZE_HIGH_OFFSET (id)); + state->target_low =readl (hdm +CXL_HDM_DECODER0_TL_LOW (id)); + state->target_high =readl (hdm +CXL_HDM_DECODER0_TL_HIGH (id)); +} + +static int cxl_pci_hdm_read_decoder (struct pci_dev *pdev, +struct cxl_hdm_decoder_state *state, +struct cxl_decoder_settings *settings, +void __iomem *hdm,int id) +{ +u64 target_or_skip, base, size; +int rc; + +cxl_pci_hdm_read_decoder_state (state, hdm, id); + + base = ((u64 )state->base_high <<32 ) | state->base_low; + size = ((u64 )state->size_high <<32 ) | state->size_low; + target_or_skip = ((u64 )state->target_high <<32 ) | state->target_low; + + rc =cxl_hdm_decode_decoder (settings, id, state->ctrl, base, size, + target_or_skip,NULL ); +if (rc) { +pci_err (pdev,"CXL HDM decoder %d has invalid configuration: %d\n" , + id, rc); +return rc; + } +return 0 ; +} + +int cxl_pci_get_device_dvsec_cap (struct pci_dev *pdev,int dvsec,u16 *cap) +{ +int rc; + +if (!dvsec) { + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); +if (!dvsec) +return -ENOTTY ; + } + + rc =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CAP , cap); +if (rc) +return pcibios_err_to_errno (rc); + +return dvsec; +} +EXPORT_SYMBOL_FOR_MODULES (cxl_pci_get_device_dvsec_cap,"cxl_core" ); + +DEFINE_FREE(cxl_hdm_iounmap, void __iomem *, if (_T) iounmap(_T)) + +static bool cxl_pci_hdm_capable (struct pci_dev *pdev) +{ +u16 cap; +int dvsec; + + dvsec =cxl_pci_get_device_dvsec_cap (pdev,0 , &cap); +if (dvsec <0 ) { +if (dvsec != -ENOTTY ) +pci_dbg (pdev, +"failed to read CXL DVSEC capability: %d\n" , + dvsec); +return false; + } + +if (!(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) +return false; + +if (!FIELD_GET (PCI_DVSEC_CXL_HDM_COUNT , cap)) +return false; + +return true; +} + +static int __cxl_pci_hdm_read_info (struct pci_dev *pdev, +struct cxl_register_map *map, +struct cxl_hdm_info *info) +{ +struct cxl_decoder_settings *settings; +int decoder_count; +int rc; + + rc =cxl_setup_regs (map); +if (rc) +return rc; + +if (!map->component_map.hdm_decoder.valid) +return -ENODEV ; + + void __iomem __free(cxl_hdm_iounmap) *hdm = +cxl_pci_hdm_map (pdev, map, info); +if (IS_ERR (hdm)) +return PTR_ERR (no_free_ptr (hdm)); + + decoder_count =cxl_hdm_decoder_count (readl (hdm + +CXL_HDM_DECODER_CAP_OFFSET )); +if (decoder_count <0 ) +return decoder_count; + +if (decoder_count >ARRAY_SIZE (info->settings)) +return -ENXIO ; + +if (CXL_HDM_DECODER0_CTRL_OFFSET (decoder_count -1 ) +0x10 > + info->hdm_size) { +pci_err (pdev, +"CXL HDM decoder count exceeds mapped register block\n" ); +return -ENXIO ; + } + + info->decoder_count = decoder_count; + info->global_ctrl =readl (hdm +CXL_HDM_DECODER_CTRL_OFFSET ); + info->decoder_state =kcalloc (decoder_count, +sizeof (*info->decoder_state), +GFP_KERNEL ); +if (!info->decoder_state) +return -ENOMEM ; + + settings = info->settings; +for (int i =0 ; i < info->decoder_count; i++) { + rc =cxl_pci_hdm_read_decoder (pdev, &info->decoder_state[i], + &settings[i], hdm, i); +if (rc) +return rc; + } + +return 0 ; +} + +static int cxl_pci_hdm_read_info (struct pci_dev *pdev, +struct cxl_register_map *map, +struct cxl_hdm_info *info) +{ +bool restore_command; +u16 command; +int rc, rc2; + + rc =pci_read_config_word (pdev,PCI_COMMAND , &command); +if (rc) +return pcibios_err_to_errno (rc); + + restore_command = !(command &PCI_COMMAND_MEMORY ); +if (restore_command) { + rc =pci_write_config_word (pdev,PCI_COMMAND , + command |PCI_COMMAND_MEMORY ); +if (rc) +return pcibios_err_to_errno (rc); + } + + rc =__cxl_pci_hdm_read_info (pdev, map, info); + +if (!restore_command) +return rc; + + rc2 =pci_write_config_word (pdev,PCI_COMMAND , command); +if (rc2) { + rc2 =pcibios_err_to_errno (rc2); +pci_err (pdev, +"failed to restore PCI_COMMAND after CXL HDM cache init: %d\n" , + rc2); +if (!rc) + rc = rc2; + } + +return rc; +} + +static int __pci_cxl_hdm_init (struct pci_dev *pdev) +{ +struct cxl_register_map map = {0 }; +struct cxl_hdm_info *info; +int rc; + + scoped_guard(rwsem_read, &cxl_rwsem.dpa) { +if (pdev->hdm) +return 0 ; + } + +if (!cxl_pci_hdm_capable (pdev)) +return -ENOTTY ; + + rc =cxl_find_regblock (pdev,CXL_REGLOC_RBI_COMPONENT , &map); +if (rc) +return rc; + + rc =cxl_pci_hdm_find_bar (pdev, map.resource, map.max_size,NULL ,NULL ); +if (rc) +return rc; + + info =kzalloc_obj (*info,GFP_KERNEL ); +if (!info) +return -ENOMEM ; + + rc =cxl_pci_hdm_read_info (pdev, &map, info); +if (rc) + goto out_free_info; + + scoped_guard(rwsem_write, &cxl_rwsem.dpa) { +if (!pdev->hdm) { + pdev->hdm = info; + info =NULL ; + }else { +cxl_pci_hdm_info_free (info); + } + } + +return 0 ; + +out_free_info: +cxl_pci_hdm_info_free (info); +return rc; +} + +void pci_cxl_hdm_init (struct pci_dev *pdev) +{ +int rc; + + rc =__pci_cxl_hdm_init (pdev); +if (rc && rc != -ENOTTY && rc != -ENODEV ) +pci_dbg (pdev,"CXL HDM cache init failed: %d\n" , rc); +} +EXPORT_SYMBOL_FOR_MODULES (pci_cxl_hdm_init,"cxl_core" );drivers/pci/bus.c +2
-0@@ -18,6 +18,7 @@#include <linux/pm_runtime.h> #include <linux/proc_fs.h> #include <linux/slab.h> +#include <cxl/cxl.h> #include "pci.h" @@ -359,6 +360,7 @@ void pci_bus_add_device(struct pci_dev *dev)/* Save config space for error recoverability */ pci_save_state (dev); +pci_cxl_hdm_init (dev);/* * Enable runtime PM, which potentially allows the device to drivers/pci/probe.c +2
-0@@ -24,6 +24,7 @@#include <linux/pm_runtime.h> #include <linux/bitfield.h> #include <trace/events/pci.h> +#include <cxl/cxl.h> #include "pci.h" static struct resource busn_resource = {@@ -2484,6 +2485,7 @@ static void pci_release_dev(struct device *dev)struct pci_dev *pci_dev; pci_dev =to_pci_dev (dev); +pci_cxl_hdm_release (pci_dev);pci_release_capabilities (pci_dev);pci_release_of_node (pci_dev);pcibios_release_device (pci_dev);include/cxl/cxl.h +26
-0@@ -26,6 +26,7 @@ enum cxl_devtype { };struct cxl_region ; +struct pci_dev ;enum cxl_decoder_type {CXL_DECODER_DEVMEM =2 ,@@ -135,16 +136,41 @@ struct cxl_regs {#define CXL_HDM_DECODER_MAX_COUNT 32 +struct cxl_hdm_decoder_state ; +/** * struct cxl_hdm_info - PCI device HDM decoder programming cache * @decoder_count: number of decoder settings entries +* @hdm_bar: BAR containing the HDM decoder registers +* @hdm_offset: HDM decoder register offset relative to @hdm_bar +* @hdm_size: HDM decoder register resource size +* @global_ctrl: cached HDM decoder global control register +* @decoder_state: cached raw per-decoder register state * @settings: cached per-decoder programming state */ struct cxl_hdm_info {int decoder_count; +int hdm_bar; +resource_size_t hdm_offset; +resource_size_t hdm_size; +u32 global_ctrl; +struct cxl_hdm_decoder_state *decoder_state;struct cxl_decoder_settings settings[CXL_HDM_DECODER_MAX_COUNT ]; }; +#ifdef CONFIG_CXL_RESET +void pci_cxl_hdm_init (struct pci_dev *pdev); +void pci_cxl_hdm_release (struct pci_dev *pdev); +#else +static inline void pci_cxl_hdm_init (struct pci_dev *pdev) +{ +} + +static inline void pci_cxl_hdm_release (struct pci_dev *pdev) +{ +} +#endif +struct cxl_reg_map {bool valid;int id;tools/testing/cxl/Kbuild +0
-1@@ -55,7 +55,6 @@ obj-m += cxl_core.o cxl_core-y := $(CXL_CORE_SRC)/port.o cxl_core-y += $(CXL_CORE_SRC)/pmem.o -cxl_core-y += $(CXL_CORE_SRC)/regs.o cxl_core-y += $(CXL_CORE_SRC)/memdev.o cxl_core-y += $(CXL_CORE_SRC)/mbox.o cxl_core-y += $(CXL_CORE_SRC)/pci.o
Re: [PATCH v11 05/12] cxl: Cache endpoint decoder settings during PCI enumeration
Quoted 4 lines · Li Ming
Quoted 3 lines
+ return true;
+
+pci_cxl_hdm_init(pdev);
Quoted 8 lines
I forgot to ask a question in v10, sorry about that.
I am wondering why calling
pci_cxl_hdm_init()incxl_pci_hdm_decoder_count_match()is needed.pci_bus_add_device()already doespci_cxl_hdm_init(), so if my understanding is correct,pdev->hdmis not NULL here, so thispci_cxl_hdm_init()here will not do anything except checking ifpdev->hdmis NULL. I am not sure if I miss some cases for that.
Hi Li Ming!
Your question is valid. pci_bus_add_device() is the normal initialization path. The later call is a retry for cases where that attempt did not publish pdev->hdm, for example because allocation or MMIO mapping failed at that point. If initialization succeeded earlier, the existing cache is detected and the retry returns without rebuilding it. I clarified this behavior in the v12 commit message.
Signature · 2 lines
Regards, Srirangan
Li Ming
reviewerRe: [PATCH v11 05/12] cxl: Cache endpoint decoder settings during PCI enumeration
Quoted 14 lines · Srirangan Madhavan
Quoted 4 lines · Li Ming
Quoted 3 lines
+ return true;
+
+pci_cxl_hdm_init(pdev);Quoted 8 lines
I forgot to ask a question in v10, sorry about that.
I am wondering why calling
pci_cxl_hdm_init()incxl_pci_hdm_decoder_count_match()is needed.pci_bus_add_device()already doespci_cxl_hdm_init(), so if my understanding is correct,pdev->hdmis not NULL here, so thispci_cxl_hdm_init()here will not do anything except checking ifpdev->hdmis NULL. I am not sure if I miss some cases for that.
Quoted 4 lines
Hi Li Ming!
Your question is valid.pci_bus_add_device()is the normal initialization path. The later call is a retry for cases where that attempt did not publishpdev->hdm, for example because allocation or MMIO mapping failed at that point. If initialization succeeded earlier, the existing cache is detected and the retry returns without rebuilding it. I clarified this behavior in the v12 commit message.
Hi Srirangan,
Thanks for your explaination.
I think removing this retry should be better if it is only used for the case that allocation or MMIO mapping failure during pci_bus_add_device(). Because these failures are not expected to happen often, and removing it can make pci_cxl_hdm_init() implementation simpler. If only pci_bus_add_device() calls pci_cxl_hdm_init(), you won't have to hold cxl_rwsem.dpa and check whether pdev->hdm is NULL in __pci_cxl_hdm_init(). Besides, hold cxl_rwsem.dpa lock in pci_cxl_hdm_release() is not needed, because the function is called in pci_release_dev() where is at the end of this pci device's lifecycle.
Ming
Re: [PATCH v11 05/12] cxl: Cache endpoint decoder settings during PCI enumeration
Quoted 43 lines · Li Ming
External email: Use caution opening links or attachments
Quoted 14 lines · Srirangan Madhavan
Quoted 4 lines · Li Ming
Quoted 3 lines
+ return true;
+
+pci_cxl_hdm_init(pdev);Quoted 8 lines
I forgot to ask a question in v10, sorry about that.
I am wondering why calling
pci_cxl_hdm_init()incxl_pci_hdm_decoder_count_match()is needed.pci_bus_add_device()already doespci_cxl_hdm_init(), so if my understanding is correct,pdev->hdmis not NULL here, so thispci_cxl_hdm_init()here will not do anything except checking ifpdev->hdmis NULL. I am not sure if I miss some cases for that.Quoted 9 lines
Hi Li Ming!
Your question is valid.pci_bus_add_device()is the normal initialization path. The later call is a retry for cases where that attempt did not publishpdev->hdm, for example because allocation or MMIO mapping failed at that point.
If initialization succeeded earlier, the existing cache is detected and the retry returns without rebuilding it. I clarified this behavior in the v12 commit message.Hi Srirangan,
Thanks for your explaination.
I think removing this retry should be better if it is only used for the case that allocation or MMIO mapping failure during
pci_bus_add_device(). Because these failures are not expected to happen often, and removing it can makepci_cxl_hdm_init()implementation simpler. If onlypci_bus_add_device()callspci_cxl_hdm_init(), you won't have to hold cxl_rwsem.dpa and check whetherpdev->hdmis NULL in__pci_cxl_hdm_init(). Besides, hold cxl_rwsem.dpa lock inpci_cxl_hdm_release()is not needed, because the function is called inpci_release_dev()where is at the end of this pci device's lifecycle.
Quoted 1 line
Ming
Hi Ming!
In v12 I removed the retry from the CXL core path and kept pci_cxl_hdm_init() only in pci_bus_add_device().
Signature · 2 lines
Regards, Srirangan
PATCH v11 6/12 cxl: Add CXL Device Reset helper
In reply to Srirangan MadhavanAdd an internal CXL Device Reset helper for Type 2 functions that advertise CXL Reset in the CXL Device DVSEC. The helper disables CXL.cache, performs cache writeback when supported, initiates reset with Memory Clear disabled, waits for completion, and re-enables CXL.cache on exit.
Leave the helper unregistered until range validation and reset-scope validation are in place.
--- drivers/cxl/core/resource.c | 247 ++++++++++++++++++++++++++++++++++ include/cxl/cxl.h | 7 + include/uapi/linux/pci_regs.h | 14 ++ 3 files changed, 268 insertions(+)
drivers/cxl/core/resource.c +247 -0
@@ -8,6 +8,8 @@#include <linux/export.h> #include <linux/io.h> #include <linux/ioport.h> +#include <linux/iommu.h> +#include <linux/jiffies.h> #include <linux/kernel.h> #include <linux/pci.h> #include <linux/slab.h>
@@ -496,3 +498,248 @@ void pci_cxl_hdm_init(struct pci_dev *pdev)pci_dbg (pdev,"CXL HDM cache init failed: %d\n" , rc); }EXPORT_SYMBOL_FOR_MODULES (pci_cxl_hdm_init,"cxl_core" ); + +/* +* CXL r4.0 sec 9.7.2 defines the reset completion timeout encodings. +* Sec 9.7.3 leaves config-space access behavior undefined for 100 ms after +* initiating CXL Reset, then limits software to CXL Status2 access until +* reset completion, timeout, or error. +*/ +#define CXL_RESET_RRS_WAIT_MS 100 +#define CXL_RESET_STATUS_POLL_MS 20 +static const u32 cxl_reset_timeout_ms[] = { +10 ,100 ,1000 ,10000 ,100000 , +}; + +#define CXL_CACHE_WBI_TIMEOUT_US 100000 +#define CXL_CACHE_WBI_POLL_US 100 + +static int cxl_reset_get_dvsec (struct pci_dev *pdev,u16 *cap_out) +{ +int dvsec, rc; +u16 cap, ctrl; + + dvsec =cxl_pci_get_device_dvsec_cap (pdev,0 , &cap); +if (dvsec <0 ) +return dvsec; + +if (!(cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || + !(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) +return -ENOTTY ; + +if (!(cap &PCI_DVSEC_CXL_RST_CAPABLE )) +return -ENOTTY ; + + rc =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CTRL , &ctrl); +if (rc) +return pcibios_err_to_errno (rc); + +if (!(ctrl &PCI_DVSEC_CXL_CACHE_ENABLE ) || + !(ctrl &PCI_DVSEC_CXL_MEM_ENABLE )) +return -ENOTTY ; + + *cap_out = cap; +return dvsec; +} + +#define CXL_RESET_CTRL2_CMD_MASK \ + (PCI_DVSEC_CXL_INIT_CACHE_WBI | PCI_DVSEC_CXL_INIT_CXL_RST) + +static int cxl_reset_read_ctrl2 (struct pci_dev *pdev,int dvsec,u16 *ctrl2) +{ +int rc; + + rc =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CTRL2 , ctrl2); +if (rc) +return pcibios_err_to_errno (rc); + + *ctrl2 &= ~CXL_RESET_CTRL2_CMD_MASK ; +return 0 ; +} + +static int cxl_reset_write_ctrl2 (struct pci_dev *pdev,int dvsec,u16 ctrl2) +{ +int rc; + + rc =pci_write_config_word (pdev, dvsec +PCI_DVSEC_CXL_CTRL2 , ctrl2); +if (rc) +return pcibios_err_to_errno (rc); + +return 0 ; +} + +static int cxl_reset_modify_ctrl2 (struct pci_dev *pdev,int dvsec,u16 set, +u16 clear) +{ +u16 ctrl2; +int rc; + + rc =cxl_reset_read_ctrl2 (pdev, dvsec, &ctrl2); +if (rc) +return rc; + + ctrl2 &= ~clear; + ctrl2 |= set; +return cxl_reset_write_ctrl2 (pdev, dvsec, ctrl2); +} + +static int cxl_reset_enable_cache (struct pci_dev *pdev,int dvsec) +{ +return cxl_reset_modify_ctrl2 (pdev, dvsec,0 , +PCI_DVSEC_CXL_DISABLE_CACHING ); +} + +static int cxl_reset_initiate (struct pci_dev *pdev,int dvsec) +{ +return cxl_reset_modify_ctrl2 (pdev, dvsec, +PCI_DVSEC_CXL_INIT_CXL_RST , +PCI_DVSEC_CXL_RST_MEM_CLR_EN ); +} + +static int cxl_reset_wait_cache_wbi (struct pci_dev *pdev,int dvsec) +{ +unsigned long deadline; +u16 status2; +int rc; + + rc =cxl_reset_modify_ctrl2 (pdev, dvsec, +PCI_DVSEC_CXL_INIT_CACHE_WBI ,0 ); +if (rc) +return rc; + + deadline = jiffies +usecs_to_jiffies (CXL_CACHE_WBI_TIMEOUT_US ); +do { +usleep_range (CXL_CACHE_WBI_POLL_US ,CXL_CACHE_WBI_POLL_US +1 ); + + rc =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_STATUS2 , + &status2); +if (rc) +return pcibios_err_to_errno (rc); +if (status2 !=U16_MAX && (status2 &PCI_DVSEC_CXL_CACHE_INV )) +return 0 ; + }while (time_before (jiffies, deadline)); + +return -ETIMEDOUT ; +} + +static int cxl_reset_disable_cache (struct pci_dev *pdev,int dvsec,u16 cap) +{ +int rc, rc2; + + rc =cxl_reset_modify_ctrl2 (pdev, dvsec, +PCI_DVSEC_CXL_DISABLE_CACHING ,0 ); +if (rc) +return rc; + +if (!(cap &PCI_DVSEC_CXL_CACHE_WBI_CAPABLE )) +return 0 ; + + rc =cxl_reset_wait_cache_wbi (pdev, dvsec); +if (!rc) +return 0 ; + +/* +* Restore the default CXL.cache policy after error paths that +* successfully set DISABLE_CACHING before reset completion. +*/ + rc2 =cxl_reset_enable_cache (pdev, dvsec); +if (rc2) +pci_warn (pdev,"failed to re-enable CXL caching: %d\n" , rc2); + +return rc; +} + +static int cxl_reset_wait_done (struct pci_dev *pdev,int dvsec,u16 cap) +{ +unsigned long deadline; +u32 timeout_ms; +u16 status2; +bool final = false; +int idx, rc; + + idx =FIELD_GET (PCI_DVSEC_CXL_RST_TIMEOUT , cap); +if (idx >=ARRAY_SIZE (cxl_reset_timeout_ms)) { +int last =ARRAY_SIZE (cxl_reset_timeout_ms) -1 ; + +pci_warn (pdev, +"unknown CXL reset timeout encoding %d; using %u ms\n" , + idx, cxl_reset_timeout_ms[last]); + idx = last; + } + + timeout_ms =max_t (u32, cxl_reset_timeout_ms[idx], +CXL_RESET_RRS_WAIT_MS ); +msleep (CXL_RESET_RRS_WAIT_MS ); + deadline = jiffies +msecs_to_jiffies (timeout_ms - +CXL_RESET_RRS_WAIT_MS ); + +do { + rc =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_STATUS2 , + &status2); +if (!rc && status2 !=U16_MAX ) { +if (status2 &PCI_DVSEC_CXL_RST_ERR ) +return -EIO ; + +if (status2 &PCI_DVSEC_CXL_RST_DONE ) +return 0 ; + } + +if (time_after_eq (jiffies, deadline)) { +if (final) +return -ETIMEDOUT ; + final = true; +continue ; + } + +msleep (CXL_RESET_STATUS_POLL_MS ); + }while (true); +} + +static int cxl_reset_execute (struct pci_dev *pdev,int dvsec,u16 cap) +{ +bool target_prepared = false; +int rc, rc2; + + rc =cxl_reset_disable_cache (pdev, dvsec, cap); +if (rc) +return rc; + +if (!pci_wait_for_pending_transaction (pdev)) +pci_err (pdev,"timed out waiting for pending transactions\n" ); + + rc =pci_dev_reset_iommu_prepare (pdev); +if (rc) +pci_err (pdev,"failed to stop IOMMU for CXL reset: %d\n" , rc); +else + target_prepared = true; + +if (!rc) + rc =cxl_reset_initiate (pdev, dvsec); +if (!rc) + rc =cxl_reset_wait_done (pdev, dvsec, cap); + + rc2 =cxl_reset_enable_cache (pdev, dvsec); +if (rc2 && rc) +pci_warn (pdev,"failed to re-enable CXL caching: %d\n" , rc2); +else if (rc2) + rc = rc2; + +if (target_prepared) +pci_dev_reset_iommu_done (pdev); +return rc; +} + +int cxl_reset_function (struct pci_dev *pdev,bool probe) +{ +int dvsec; +u16 cap; + + dvsec =cxl_reset_get_dvsec (pdev, &cap); +if (dvsec <0 ) +return dvsec; + +if (probe) +return 0 ; + +return cxl_reset_execute (pdev, dvsec, cap); +}
include/cxl/cxl.h +7 -0
@@ -9,6 +9,7 @@#include <linux/node.h> #include <linux/ioport.h> #include <linux/range.h> +#include <linux/errno.h> #include <cxl/mailbox.h> /**
@@ -161,6 +162,7 @@ struct cxl_hdm_info {#ifdef CONFIG_CXL_RESET void pci_cxl_hdm_init (struct pci_dev *pdev);void pci_cxl_hdm_release (struct pci_dev *pdev); +int cxl_reset_function (struct pci_dev *pdev,bool probe);#else static inline void pci_cxl_hdm_init (struct pci_dev *pdev) {
@@ -169,6 +171,11 @@ static inline void pci_cxl_hdm_init(struct pci_dev *pdev)static inline void pci_cxl_hdm_release (struct pci_dev *pdev) { } + +static inline int cxl_reset_function (struct pci_dev *pdev,bool probe) +{ +return -ENOTTY ; +}#endif struct cxl_reg_map {
include/uapi/linux/pci_regs.h +14 -0
@@ -1352,8 +1352,22 @@#define PCI_DVSEC_CXL_CACHE_CAPABLE _BITUL(0)#define PCI_DVSEC_CXL_MEM_CAPABLE _BITUL(2)#define PCI_DVSEC_CXL_HDM_COUNT __GENMASK(5, 4) +#define PCI_DVSEC_CXL_CACHE_WBI_CAPABLE _BITUL(6) +#define PCI_DVSEC_CXL_RST_CAPABLE _BITUL(7) +#define PCI_DVSEC_CXL_RST_TIMEOUT __GENMASK(10, 8) +#define PCI_DVSEC_CXL_RST_MEM_CLR_CAPABLE _BITUL(11)#define PCI_DVSEC_CXL_CTRL 0xC +#define PCI_DVSEC_CXL_CACHE_ENABLE _BITUL(0)#define PCI_DVSEC_CXL_MEM_ENABLE _BITUL(2) +#define PCI_DVSEC_CXL_CTRL2 0x10 +#define PCI_DVSEC_CXL_DISABLE_CACHING _BITUL(0) +#define PCI_DVSEC_CXL_INIT_CACHE_WBI _BITUL(1) +#define PCI_DVSEC_CXL_INIT_CXL_RST _BITUL(2) +#define PCI_DVSEC_CXL_RST_MEM_CLR_EN _BITUL(3) +#define PCI_DVSEC_CXL_STATUS2 0x12 +#define PCI_DVSEC_CXL_CACHE_INV _BITUL(0) +#define PCI_DVSEC_CXL_RST_DONE _BITUL(1) +#define PCI_DVSEC_CXL_RST_ERR _BITUL(2)#define PCI_DVSEC_CXL_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10))#define PCI_DVSEC_CXL_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10))#define PCI_DVSEC_CXL_MEM_INFO_VALID _BITUL(0)
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 06/12] cxl: Add CXL Device Reset helper
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Add an internal CXL Device Reset helper for Type 2 functions that advertise CXL Reset in the CXL Device DVSEC. The helper disables CXL.cache, performs cache writeback when supported, initiates reset with Memory Clear disabled, waits for completion, and re-enables CXL.cache on exit.
Leave the helper unregistered until range validation and reset-scope validation are in place.
Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
PATCH v11 7/12 cxl: Validate HDM ranges before CXL reset
In reply to Srirangan MadhavanBefore reset, require cached HDM decoder state, collect enabled decoder ranges, and reserve them with request_mem_region(). This rejects reset while affected CXL memory is busy and keeps the validation stable through reset.
If CPU cache invalidation support is available, invalidate the affected ranges before reset. If the runtime backend is unavailable, continue after the range reservation succeeds.
Reject CXL Reset when no cached HDM decoder state is available. The reset path needs the cached address map to validate affected ranges and perform CPU cache invalidation. Also reject normalized-addressing decoders for now because the cached decoder range is not a system physical address.
--- drivers/cxl/core/resource.c | 255 +++++++++++++++++++++++++++++++++++- 1 file changed, 254 insertions(+), 1 deletion(-)
drivers/cxl/core/resource.c +254 -1
@@ -11,6 +11,8 @@#include <linux/iommu.h> #include <linux/jiffies.h> #include <linux/kernel.h> +#include <linux/list.h> +#include <linux/memregion.h> #include <linux/pci.h> #include <linux/slab.h>
@@ -514,6 +516,229 @@ static const u32 cxl_reset_timeout_ms[] = {#define CXL_CACHE_WBI_TIMEOUT_US 100000#define CXL_CACHE_WBI_POLL_US 100 +struct cxl_hdm_range { +struct list_head list; +struct pci_dev *pdev; +struct range hpa_range; +u64 len; +struct resource *res; +}; + +struct cxl_hdm_range_context { +struct list_head ranges; +}; + +static void cxl_hdm_range_context_init (struct cxl_hdm_range_context *ctx) +{ +INIT_LIST_HEAD (&ctx->ranges); +} + +static void cxl_hdm_range_context_destroy (struct cxl_hdm_range_context *ctx) +{ +struct cxl_hdm_range *range, *next; + +list_for_each_entry_safe (range, next, &ctx->ranges, list) { +list_del (&range->list); +if (range->res) +release_mem_region (range->hpa_range.start, +resource_size (range->res)); +kfree (range); + } +} + +/* +* Bound the range twice: request_mem_region() takes resource_size_t while +* cpu_cache_invalidate_memregion() takes size_t, and the two differ on +* 32-bit builds with CONFIG_PHYS_ADDR_T_64BIT. range_len() can also reach +* RESOURCE_SIZE_MAX + 1 for a full-width range, and wraps to zero when +* resource_size_t is 64-bit, which the !len test catches. +*/ +static int cxl_hdm_range_validate (struct pci_dev *pdev, +const struct range *hpa_range) +{ +u64 len =range_len (hpa_range); + +if (!len) +return -EINVAL ; + +if (hpa_range->end >RESOURCE_SIZE_MAX ) { +pci_err (pdev, +"CXL reset range [%#llx-%#llx] exceeds resource address size\n" , + hpa_range->start, hpa_range->end); +return -EOVERFLOW ; + } + +if (len >RESOURCE_SIZE_MAX ) { +pci_err (pdev, +"CXL reset range [%#llx-%#llx] exceeds resource size\n" , + hpa_range->start, hpa_range->end); +return -EOVERFLOW ; + } + +if (len >SIZE_MAX ) { +pci_err (pdev, +"CXL reset range [%#llx-%#llx] exceeds cache flush size\n" , + hpa_range->start, hpa_range->end); +return -EOVERFLOW ; + } + +return 0 ; +} + +static int cxl_hdm_range_add (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev,const struct range *hpa_range) +{ +struct cxl_hdm_range *range; +int rc; + + rc =cxl_hdm_range_validate (pdev, hpa_range); +if (rc) +return rc; + +list_for_each_entry (range, &ctx->ranges, list) +if (range->hpa_range.start == hpa_range->start && + range->hpa_range.end == hpa_range->end) +return 0 ; + + range =kzalloc_obj (*range); +if (!range) +return -ENOMEM ; + + range->pdev = pdev; + range->hpa_range = *hpa_range; + range->len =range_len (hpa_range); +list_add_tail (&range->list, &ctx->ranges); + +return 0 ; +} + +static int cxl_hdm_ranges_collect (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev) +{ +struct cxl_hdm_info *info; +int rc; + +guard (rwsem_read)(&cxl_rwsem.dpa); + info = pdev->hdm; +if (!info) { +pci_err (pdev,"CXL HDM decoder state unavailable\n" ); +return -ENXIO ; + } + +for (int i =0 ; i < info->decoder_count; i++) { +struct cxl_decoder_settings *settings = &info->settings[i]; + +if (!(settings->flags &CXL_DECODER_F_ENABLE )) +continue ; + +/* A committed zero-size decoder maps no HPA. */ +if (!range_len (&settings->hpa_range)) +continue ; + +if (settings->flags &CXL_DECODER_F_NORMALIZED_ADDRESSING ) { +pci_err (pdev, +"CXL reset does not support normalized address decoders\n" ); +return -EOPNOTSUPP ; + } + + rc =cxl_hdm_range_add (ctx, pdev, &settings->hpa_range); +if (rc) +return rc; + } + +return 0 ; +} + +static int cxl_hdm_range_request (struct cxl_hdm_range *range) +{ +struct pci_dev *pdev = range->pdev; +const struct range *hpa_range = &range->hpa_range; + + range->res =request_mem_region (hpa_range->start, range->len, +"cxl_reset" ); +if (!range->res) { +pci_err (pdev, +"cannot reset while CXL memory range is busy [%#llx-%#llx]\n" , + hpa_range->start, hpa_range->end); +return -EBUSY ; + } + +return 0 ; +} + +static int cxl_hdm_ranges_request (struct cxl_hdm_range_context *ctx) +{ +struct cxl_hdm_range *range; +int rc; + +lockdep_assert_held_write (&cxl_rwsem.region); + +list_for_each_entry (range, &ctx->ranges, list) { + rc =cxl_hdm_range_request (range); +if (rc) +return rc; + } + +return 0 ; +} + +static int cxl_hdm_range_flush_cache (struct cxl_hdm_range *range) +{ +struct pci_dev *pdev = range->pdev; +const struct range *hpa_range = &range->hpa_range; +int rc; + + rc =cpu_cache_invalidate_memregion (hpa_range->start, range->len); +if (rc) +pci_err (pdev, +"failed to invalidate CPU cache [%#llx-%#llx]: %d\n" , + hpa_range->start, hpa_range->end, rc); + +return rc; +} + +static int cxl_hdm_ranges_flush_cpu_caches (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev) +{ +struct cxl_hdm_range *range; +int rc; + +if (list_empty (&ctx->ranges)) +return 0 ; + +if (!cpu_cache_has_invalidate_memregion ()) { +pci_warn (pdev, +"CPU cache synchronization unavailable; continuing without cache invalidation\n" ); +return 0 ; + } + +list_for_each_entry (range, &ctx->ranges, list) { + rc =cxl_hdm_range_flush_cache (range); +if (rc) +return rc; + } + +return 0 ; +} + +static int cxl_hdm_ranges_prepare (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev) +{ +int rc; + +lockdep_assert_held_write (&cxl_rwsem.region); + + rc =cxl_hdm_ranges_collect (ctx, pdev); +if (rc) +return rc; + + rc =cxl_hdm_ranges_request (ctx); +if (rc) +return rc; + +return cxl_hdm_ranges_flush_cpu_caches (ctx, pdev); +} +static int cxl_reset_get_dvsec (struct pci_dev *pdev,u16 *cap_out) {int dvsec, rc;
@@ -542,6 +767,20 @@ static int cxl_reset_get_dvsec(struct pci_dev *pdev, u16 *cap_out)return dvsec; } +static bool cxl_reset_hdm_available (struct pci_dev *pdev) +{ +struct cxl_hdm_info *info; + +/* +* pdev->hdm is owned by the PCI device and released with pci_dev, so +* reset-method probes and reset requests can test availability without +* a CXL driver bound to the device. +*/ +guard (rwsem_read)(&cxl_rwsem.dpa); + info = pdev->hdm; +return info && info->hdm_size; +} +#define CXL_RESET_CTRL2_CMD_MASK \ (PCI_DVSEC_CXL_INIT_CACHE_WBI | PCI_DVSEC_CXL_INIT_CXL_RST)
@@ -731,7 +970,9 @@ static int cxl_reset_execute(struct pci_dev *pdev, int dvsec, u16 cap)int cxl_reset_function (struct pci_dev *pdev,bool probe) { +struct cxl_hdm_range_context range_ctx;int dvsec; +int rc;u16 cap; dvsec =cxl_reset_get_dvsec (pdev, &cap);
@@ -741,5 +982,17 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe)if (probe)return 0 ; -return cxl_reset_execute (pdev, dvsec, cap); +if (!cxl_reset_hdm_available (pdev)) +return -ENOTTY ; + +cxl_hdm_range_context_init (&range_ctx); + + scoped_guard(rwsem_write, &cxl_rwsem.region) { + rc =cxl_hdm_ranges_prepare (&range_ctx, pdev); +if (!rc) + rc =cxl_reset_execute (pdev, dvsec, cap); +cxl_hdm_range_context_destroy (&range_ctx); + } + +return rc; }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 07/12] cxl: Validate HDM ranges before CXL reset
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Before reset, require cached HDM decoder state, collect enabled decoder ranges, and reserve them with
request_mem_region(). This rejects reset while affected CXL memory is busy and keeps the validation stable through reset.If CPU cache invalidation support is available, invalidate the affected ranges before reset. If the runtime backend is unavailable, continue after the range reservation succeeds.
Quoted 6 lines
Reject CXL Reset when no cached HDM decoder state is available. The reset path needs the cached address map to validate affected ranges and perform CPU cache invalidation. Also reject normalized-addressing decoders for now because the cached decoder range is not a system physical address.
Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Re: [PATCH v11 07/12] cxl: Validate HDM ranges before CXL reset
In reply to Srirangan MadhavanQuoted 166 lines · Srirangan Madhavan
Before reset, require cached HDM decoder state, collect enabled decoder ranges, and reserve them with
request_mem_region(). This rejects reset while affected CXL memory is busy and keeps the validation stable through reset.If CPU cache invalidation support is available, invalidate the affected ranges before reset. If the runtime backend is unavailable, continue after the range reservation succeeds.
Reject CXL Reset when no cached HDM decoder state is available. The reset path needs the cached address map to validate affected ranges and perform CPU cache invalidation. Also reject normalized-addressing decoders for now because the cached decoder range is not a system physical address.
Signed-off-by Srirangan Madhavan <[email protected]>--- drivers/cxl/core/resource.c | 255 +++++++++++++++++++++++++++++++++++- 1 file changed, 254 insertions(+), 1 deletion(-)drivers/cxl/core/resource.c +132
-0@@ -11,6 +11,8 @@#include <linux/iommu.h> #include <linux/jiffies.h> #include <linux/kernel.h> +#include <linux/list.h> +#include <linux/memregion.h> #include <linux/pci.h> #include <linux/slab.h> @@ -514,6 +516,229 @@ static const u32 cxl_reset_timeout_ms[] = {#define CXL_CACHE_WBI_TIMEOUT_US 100000#define CXL_CACHE_WBI_POLL_US 100 +struct cxl_hdm_range { +struct list_head list; +struct pci_dev *pdev; +struct range hpa_range; +u64 len; +struct resource *res; +}; + +struct cxl_hdm_range_context { +struct list_head ranges; +}; + +static void cxl_hdm_range_context_init (struct cxl_hdm_range_context *ctx) +{ +INIT_LIST_HEAD (&ctx->ranges); +} + +static void cxl_hdm_range_context_destroy (struct cxl_hdm_range_context *ctx) +{ +struct cxl_hdm_range *range, *next; + +list_for_each_entry_safe (range, next, &ctx->ranges, list) { +list_del (&range->list); +if (range->res) +release_mem_region (range->hpa_range.start, +resource_size (range->res)); +kfree (range); + } +} + +/* +* Bound the range twice: request_mem_region() takes resource_size_t while +* cpu_cache_invalidate_memregion() takes size_t, and the two differ on +* 32-bit builds with CONFIG_PHYS_ADDR_T_64BIT. range_len() can also reach +* RESOURCE_SIZE_MAX + 1 for a full-width range, and wraps to zero when +* resource_size_t is 64-bit, which the !len test catches. +*/ +static int cxl_hdm_range_validate (struct pci_dev *pdev, +const struct range *hpa_range) +{ +u64 len =range_len (hpa_range); + +if (!len) +return -EINVAL ; + +if (hpa_range->end >RESOURCE_SIZE_MAX ) { +pci_err (pdev, +"CXL reset range [%#llx-%#llx] exceeds resource address size\n" , + hpa_range->start, hpa_range->end); +return -EOVERFLOW ; + } + +if (len >RESOURCE_SIZE_MAX ) { +pci_err (pdev, +"CXL reset range [%#llx-%#llx] exceeds resource size\n" , + hpa_range->start, hpa_range->end); +return -EOVERFLOW ; + } + +if (len >SIZE_MAX ) { +pci_err (pdev, +"CXL reset range [%#llx-%#llx] exceeds cache flush size\n" , + hpa_range->start, hpa_range->end); +return -EOVERFLOW ; + } + +return 0 ; +} + +static int cxl_hdm_range_add (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev,const struct range *hpa_range) +{ +struct cxl_hdm_range *range; +int rc; + + rc =cxl_hdm_range_validate (pdev, hpa_range); +if (rc) +return rc; + +list_for_each_entry (range, &ctx->ranges, list) +if (range->hpa_range.start == hpa_range->start && + range->hpa_range.end == hpa_range->end) +return 0 ; + + range =kzalloc_obj (*range); +if (!range) +return -ENOMEM ; + + range->pdev = pdev; + range->hpa_range = *hpa_range; + range->len =range_len (hpa_range); +list_add_tail (&range->list, &ctx->ranges); + +return 0 ; +} + +static int cxl_hdm_ranges_collect (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev) +{ +struct cxl_hdm_info *info; +int rc; + +guard (rwsem_read)(&cxl_rwsem.dpa); + info = pdev->hdm; +if (!info) { +pci_err (pdev,"CXL HDM decoder state unavailable\n" ); +return -ENXIO ; + } + +for (int i =0 ; i < info->decoder_count; i++) { +struct cxl_decoder_settings *settings = &info->settings[i]; + +if (!(settings->flags &CXL_DECODER_F_ENABLE )) +continue ; + +/* A committed zero-size decoder maps no HPA. */ +if (!range_len (&settings->hpa_range)) +continue ; + +if (settings->flags &CXL_DECODER_F_NORMALIZED_ADDRESSING ) { +pci_err (pdev, +"CXL reset does not support normalized address decoders\n" ); +return -EOPNOTSUPP ; + } + + rc =cxl_hdm_range_add (ctx, pdev, &settings->hpa_range); +if (rc) +return rc; + } +
drivers/cxl/core/resource.c static const u32 cxl_reset_timeout_ms[] = { · line 658
Quoted 10 lines
drivers/cxl/core/resource.c +10
-0+return 0 ; +} + +static int cxl_hdm_range_request (struct cxl_hdm_range *range) +{ +struct pci_dev *pdev = range->pdev; +const struct range *hpa_range = &range->hpa_range; + + range->res =request_mem_region (hpa_range->start, range->len, +"cxl_reset" );
How does this function provide us the required quiescing ?
I do not see it revoking existing PFN/VFIO mapping or preventing a driver from issueing a new access after cpu_cache_invalidate_memregion()
Want to know what prevents new CXL.mem requests between cache invalidation and reset completion ?
It does not require the range to be offline or unmapped first ?
Best regards,
Richard Cheng.
Quoted 139 lines
drivers/cxl/core/resource.c +112
-1+if (!range->res) { +pci_err (pdev, +"cannot reset while CXL memory range is busy [%#llx-%#llx]\n" , + hpa_range->start, hpa_range->end); +return -EBUSY ; + } + +return 0 ; +} + +static int cxl_hdm_ranges_request (struct cxl_hdm_range_context *ctx) +{ +struct cxl_hdm_range *range; +int rc; + +lockdep_assert_held_write (&cxl_rwsem.region); + +list_for_each_entry (range, &ctx->ranges, list) { + rc =cxl_hdm_range_request (range); +if (rc) +return rc; + } + +return 0 ; +} + +static int cxl_hdm_range_flush_cache (struct cxl_hdm_range *range) +{ +struct pci_dev *pdev = range->pdev; +const struct range *hpa_range = &range->hpa_range; +int rc; + + rc =cpu_cache_invalidate_memregion (hpa_range->start, range->len); +if (rc) +pci_err (pdev, +"failed to invalidate CPU cache [%#llx-%#llx]: %d\n" , + hpa_range->start, hpa_range->end, rc); + +return rc; +} + +static int cxl_hdm_ranges_flush_cpu_caches (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev) +{ +struct cxl_hdm_range *range; +int rc; + +if (list_empty (&ctx->ranges)) +return 0 ; + +if (!cpu_cache_has_invalidate_memregion ()) { +pci_warn (pdev, +"CPU cache synchronization unavailable; continuing without cache invalidation\n" ); +return 0 ; + } + +list_for_each_entry (range, &ctx->ranges, list) { + rc =cxl_hdm_range_flush_cache (range); +if (rc) +return rc; + } + +return 0 ; +} + +static int cxl_hdm_ranges_prepare (struct cxl_hdm_range_context *ctx, +struct pci_dev *pdev) +{ +int rc; + +lockdep_assert_held_write (&cxl_rwsem.region); + + rc =cxl_hdm_ranges_collect (ctx, pdev); +if (rc) +return rc; + + rc =cxl_hdm_ranges_request (ctx); +if (rc) +return rc; + +return cxl_hdm_ranges_flush_cpu_caches (ctx, pdev); +} +static int cxl_reset_get_dvsec (struct pci_dev *pdev,u16 *cap_out) {int dvsec, rc;@@ -542,6 +767,20 @@ static int cxl_reset_get_dvsec(struct pci_dev *pdev, u16 *cap_out)return dvsec; } +static bool cxl_reset_hdm_available (struct pci_dev *pdev) +{ +struct cxl_hdm_info *info; + +/* +* pdev->hdm is owned by the PCI device and released with pci_dev, so +* reset-method probes and reset requests can test availability without +* a CXL driver bound to the device. +*/ +guard (rwsem_read)(&cxl_rwsem.dpa); + info = pdev->hdm; +return info && info->hdm_size; +} +#define CXL_RESET_CTRL2_CMD_MASK \ (PCI_DVSEC_CXL_INIT_CACHE_WBI | PCI_DVSEC_CXL_INIT_CXL_RST)@@ -731,7 +970,9 @@ static int cxl_reset_execute(struct pci_dev *pdev, int dvsec, u16 cap)int cxl_reset_function (struct pci_dev *pdev,bool probe) { +struct cxl_hdm_range_context range_ctx;int dvsec; +int rc;u16 cap; dvsec =cxl_reset_get_dvsec (pdev, &cap);@@ -741,5 +982,17 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe)if (probe)return 0 ; -return cxl_reset_execute (pdev, dvsec, cap); +if (!cxl_reset_hdm_available (pdev)) +return -ENOTTY ; + +cxl_hdm_range_context_init (&range_ctx); + + scoped_guard(rwsem_write, &cxl_rwsem.region) { + rc =cxl_hdm_ranges_prepare (&range_ctx, pdev); +if (!rc) + rc =cxl_reset_execute (pdev, dvsec, cap); +cxl_hdm_range_context_destroy (&range_ctx); + } + +return rc; }Signature · 1 line
2.43.0
Re: [PATCH v11 07/12] cxl: Validate HDM ranges before CXL reset
Quoted 6 lines · Richard Cheng
drivers/cxl/core/resource.c static const u32 cxl_reset_timeout_ms[] = { · line 658
Quoted 1 line
drivers/cxl/core/resource.c +1
-0+ "cxl_reset");How does this function provide us the required quiescing ?
I do not see it revoking existing PFN/VFIO mapping or preventing a driver from issueing a new access aftercpu_cache_invalidate_memregion()
Quoted 6 lines
Want to know what prevents new CXL.mem requests between cache invalidation and reset completion ?
It does not require the range to be offline or unmapped first ?Best regards,
Richard Cheng.
Hi Richard,
You are right that request_mem_region() does not revoke PFN or VFIO mappings, and this series does not offline memory itself.
It temporarily reserves every enabled HDM range and rejects the reset with -EBUSY if a conflicting resource owns one of those ranges. I documented this limitation explicitly in v12.
Signature · 2 lines
Regards, Srirangan
PATCH v11 8/12 PCI/CXL: Reject CXL Reset on multifunction devices
In reply to Srirangan MadhavanCXL Reset can be exposed as a PCI function reset method only when the reset is function-scoped. PCI reset-method probing runs during enumeration, so walking sibling functions there can race with functions that have not been added to the bus yet.
Reject multifunction devices for now instead of trying to infer reset scope from sibling state. Systems that can prove a multifunction device has function-scoped CXL Reset can be enabled later.
--- drivers/cxl/core/resource.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/cxl/core/resource.c +3 -0
@@ -979,6 +979,9 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe)if (dvsec <0 )return dvsec; +if (pdev->multifunction) +return -ENOTTY ; +if (probe)return 0 ;
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 08/12] PCI/CXL: Reject CXL Reset on multifunction devices
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
CXL Reset can be exposed as a PCI function reset method only when the reset is function-scoped. PCI reset-method probing runs during enumeration, so walking sibling functions there can race with functions that have not been added to the bus yet.
Reject multifunction devices for now instead of trying to infer reset scope from sibling state. Systems that can prove a multifunction device has function-scoped CXL Reset can be enabled later.
Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Re: [PATCH v11 08/12] PCI/CXL: Reject CXL Reset on multifunction devices
In reply to Srirangan MadhavanQuoted 20 lines · Srirangan Madhavan
CXL Reset can be exposed as a PCI function reset method only when the reset is function-scoped. PCI reset-method probing runs during enumeration, so walking sibling functions there can race with functions that have not been added to the bus yet.
Reject multifunction devices for now instead of trying to infer reset scope from sibling state. Systems that can prove a multifunction device has function-scoped CXL Reset can be enabled later.
Signed-off-by Srirangan Madhavan <[email protected]>Reviewed-by Dave Jiang <[email protected]>--- drivers/cxl/core/resource.c | 3 +++ 1 file changed, 3 insertions(+)drivers/cxl/core/resource.c +0
-0
drivers/cxl/core/resource.c int cxl_reset_function(struct pci_dev *pdev, bool probe) · line 987
Quoted 9 lines
drivers/cxl/core/resource.c +3
-0@@ -979,6 +979,9 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe)if (dvsec <0 )return dvsec; +if (pdev->multifunction) +return -ENOTTY ; +if (probe)return 0 ;
Hmm a PF can have a single-function PCI header while having many enabled VFs, and VFs enablement can change after reset-method probing.
Since CXL Reset can affect VFs, do we have the mechanism to quiescie all VF?
Best regards,
Richard Cheng.
Quoted 3 lines
Signature · 1 line
2.43.0
Re: [PATCH v11 08/12] PCI/CXL: Reject CXL Reset on multifunction devices
Quoted 9 lines · Richard Cheng
drivers/cxl/core/resource.c int cxl_reset_function(struct pci_dev *pdev, bool probe) · line 986
Quoted 6 lines
drivers/cxl/core/resource.c +3
-0+if (pdev->multifunction) +return -ENOTTY ; +if (probe)return 0 ;Hmm a PF can have a single-function PCI header while having many enabled VFs, and VFs enablement can change after reset-method probing.
Quoted 4 lines
Since CXL Reset can affect VFs, do we have the mechanism to quiescie all VF?
Best regards,
Richard Cheng.
I see the issue.
In V12, it now conservatively rejects CXL Reset when either pdev->multifunction is set or pci_num_vf(pdev) reports enabled VFs. The check is repeated when the reset executes, so VFs enabled after reset-method probing are also caught. These changes do not attempt to quiesce active VFs.
Signature · 2 lines
Regards, Srirangan
PATCH v11 9/12 cxl: Restore CXL HDM state after PCI reset
In reply to Srirangan MadhavanAfter CXL reset, restore PCI config state enough to reach HDM MMIO, restore cached global and per-decoder HDM state, and then run the normal PCI restore callbacks.
Keep the target IOMMU reset block active until HDM restore completes so Bus Master Enable cannot reopen DMA before decoder state is valid.
--- drivers/cxl/core/resource.c | 362 ++++++++++++++++++++++++++++++++++-- 1 file changed, 347 insertions(+), 15 deletions(-)
drivers/cxl/core/resource.c +347 -15
@@ -13,6 +13,7 @@#include <linux/kernel.h> #include <linux/list.h> #include <linux/memregion.h> +#include <linux/overflow.h> #include <linux/pci.h> #include <linux/slab.h>
@@ -83,6 +84,26 @@ static int cxld_await_commit(void __iomem *hdm, int id)return -ETIMEDOUT ; } +static int cxld_await_uncommit (void __iomem *hdm,int id) +{ +u32 ctrl; +int i; + +for (i =0 ; i <COMMIT_TIMEOUT_MS ; i++) { + ctrl =readl (hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +if (FIELD_GET (CXL_HDM_DECODER0_CTRL_COMMIT_ERROR , ctrl)) { + ctrl &= ~CXL_HDM_DECODER0_CTRL_COMMIT ; +writel (ctrl, hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +return -EIO ; + } +if (!FIELD_GET (CXL_HDM_DECODER0_CTRL_COMMITTED , ctrl)) +return 0 ; +fsleep (1000 ); + } + +return -ETIMEDOUT ; +} +static int setup_hw_decoder (void __iomem *hdm,struct cxl_decoder_settings *settings) {
@@ -273,6 +294,31 @@ static void __iomem *cxl_pci_hdm_map(struct pci_dev *pdev,return hdm; } +static void __iomem *cxl_pci_hdm_ioremap_current (struct pci_dev *pdev, +int bar, +resource_size_t offset, +resource_size_t size) +{ +resource_size_t hdm_start, bar_len; +void __iomem *hdm; + +if (bar <0 || bar >=PCI_STD_NUM_BARS || !size) +return ERR_PTR (-EINVAL ); + + bar_len =pci_resource_len (pdev, bar); +if (!bar_len || offset > bar_len || size > bar_len - offset) +return ERR_PTR (-ENODEV ); + + hdm_start =pci_resource_start (pdev, bar) + offset; + hdm =ioremap (hdm_start, size); +if (!hdm) { +pci_err (pdev,"failed to remap CXL HDM decoder registers\n" ); +return ERR_PTR (-ENOMEM ); + } + +return hdm; +} +static void cxl_pci_hdm_read_decoder_state (struct cxl_hdm_decoder_state *state,void __iomem *hdm,int id) {
@@ -285,6 +331,40 @@ static void cxl_pci_hdm_read_decoder_state(struct cxl_hdm_decoder_state *state, state->target_high =readl (hdm +CXL_HDM_DECODER0_TL_HIGH (id)); } +static int cxl_hdm_enable_mem (struct pci_dev *pdev,u16 *command, +bool *restore_command) +{ +int rc; + + *restore_command = false; + + rc =pci_read_config_word (pdev,PCI_COMMAND , command); +if (rc) +return pcibios_err_to_errno (rc); + +if (*command &PCI_COMMAND_MEMORY ) +return 0 ; + + rc =pci_write_config_word (pdev,PCI_COMMAND , + *command |PCI_COMMAND_MEMORY ); +if (rc) +return pcibios_err_to_errno (rc); + + *restore_command = true; +return 0 ; +} + +static int cxl_hdm_restore_command (struct pci_dev *pdev,u16 command) +{ +int rc; + + rc =pci_write_config_word (pdev,PCI_COMMAND , command); +if (rc) +return pcibios_err_to_errno (rc); + +return 0 ; +} + static int cxl_pci_hdm_read_decoder(struct pci_dev *pdev, struct cxl_hdm_decoder_state *state, struct cxl_decoder_settings *settings,
@@ -501,6 +581,220 @@ void pci_cxl_hdm_init(struct pci_dev *pdev) }EXPORT_SYMBOL_FOR_MODULES (pci_cxl_hdm_init,"cxl_core" ); +static int cxl_hdm_decoder_uncommit (struct pci_dev *pdev,void __iomem *hdm, +int id) +{ +u32 ctrl; +int rc; + + ctrl =readl (hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +if (ctrl &CXL_HDM_DECODER0_CTRL_LOCK ) { +if (ctrl &CXL_HDM_DECODER0_CTRL_COMMITTED ) { +pci_dbg (pdev, +"CXL HDM decoder %d retained locked committed state\n" , + id); +return -EBUSY ; + } + +pci_err (pdev,"CXL HDM decoder %d is locked and uncommitted\n" , + id); +return -EIO ; + } + +if (!(ctrl &CXL_HDM_DECODER0_CTRL_COMMITTED )) +return 0 ; + + ctrl &= ~CXL_HDM_DECODER0_CTRL_COMMIT ; +writel (ctrl, hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); + + rc =cxld_await_uncommit (hdm, id); +if (rc) +pci_err (pdev,"CXL HDM decoder %d uncommit failed: %d\n" , + id, rc); + +return rc; +} + +static void cxl_restore_hdm_decoder_state (struct cxl_hdm_decoder_state *state, +void __iomem *hdm,int id) +{ +u32 ctrl = state->ctrl; + + ctrl &= ~(CXL_HDM_DECODER0_CTRL_COMMIT | +CXL_HDM_DECODER0_CTRL_COMMITTED | +CXL_HDM_DECODER0_CTRL_COMMIT_ERROR | +CXL_HDM_DECODER0_CTRL_LOCK ); + +writel (state->base_high, hdm +CXL_HDM_DECODER0_BASE_HIGH_OFFSET (id)); +writel (state->base_low, hdm +CXL_HDM_DECODER0_BASE_LOW_OFFSET (id)); +writel (state->size_high, hdm +CXL_HDM_DECODER0_SIZE_HIGH_OFFSET (id)); +writel (state->size_low, hdm +CXL_HDM_DECODER0_SIZE_LOW_OFFSET (id)); +writel (state->target_high, hdm +CXL_HDM_DECODER0_TL_HIGH (id)); +writel (state->target_low, hdm +CXL_HDM_DECODER0_TL_LOW (id)); +writel (ctrl, hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +} + +static int cxl_restore_hdm_decoder (struct pci_dev *pdev, +struct cxl_hdm_decoder_state *state, +struct cxl_decoder_settings *settings, +void __iomem *hdm) +{ +int rc; + + rc =cxl_hdm_decoder_uncommit (pdev, hdm, settings->id); +if (rc == -EBUSY ) +return 0 ; +if (rc) +return rc; + +cxl_restore_hdm_decoder_state (state, hdm, settings->id); + +if (!(settings->flags &CXL_DECODER_F_ENABLE )) +return 0 ; + +scoped_guard (rwsem_read, &cxl_rwsem.dpa) + rc = cxl_commit_start(hdm, settings); +if (!rc) + rc =cxl_commit_wait (hdm, settings); +if (rc) +pci_err (pdev,"CXL HDM decoder %d restore failed: %d\n" , + settings->id, rc); + +return rc; +} + +static struct cxl_hdm_info *cxl_snapshot_hdm (struct pci_dev *pdev) +{ +struct cxl_hdm_info *snap; +struct cxl_hdm_info *info; +size_t state_sz; + +guard (rwsem_read)(&cxl_rwsem.dpa); + + info = pdev->hdm; +if (!info) +return NULL ; +if (info->decoder_count <0 || + info->decoder_count >CXL_HDM_DECODER_MAX_COUNT || + (info->decoder_count && !info->decoder_state)) +return ERR_PTR (-EINVAL ); + + state_sz =array_size (info->decoder_count,sizeof (*info->decoder_state)); + snap =kzalloc (size_add (sizeof (*snap), state_sz),GFP_KERNEL ); +if (!snap) +return ERR_PTR (-ENOMEM ); + + *snap = *info; + snap->decoder_state = (void *)(snap +1 ); +memcpy (snap->decoder_state, info->decoder_state, state_sz); + +return snap; +} + +static void cxl_restore_pci_state_for_hdm_restore (struct pci_dev *pdev, +u16 *command) +{ +u32 saved_config = pdev->saved_config_space[PCI_COMMAND /4 ]; + + pdev->saved_config_space[PCI_COMMAND /4 ] &= ~PCI_COMMAND_MASTER ; + pdev->saved_config_space[PCI_COMMAND /4 ] |=PCI_COMMAND_INTX_DISABLE ; +pci_restore_state (pdev); + pdev->saved_config_space[PCI_COMMAND /4 ] = saved_config; + *command = saved_config &0xffff ; +} + +static int cxl_restore_hdm (struct pci_dev *pdev) +{ +struct cxl_hdm_info *snap =cxl_snapshot_hdm (pdev); +bool restore_command = false; +void __iomem *hdm; +int first_rc =0 ; +u16 command; +int rc; + +if (!snap) +return 0 ; +if (IS_ERR (snap)) +return PTR_ERR (snap); + + rc =cxl_hdm_enable_mem (pdev, &command, &restore_command); +if (rc) { +kfree (snap); +return rc; + } + + hdm =cxl_pci_hdm_ioremap_current (pdev, snap->hdm_bar, + snap->hdm_offset, snap->hdm_size); +if (IS_ERR (hdm)) { + first_rc =PTR_ERR (hdm); + }else { +/* +* Restore global HDM control before per-decoder commit. PCI +* config memory decoding is enabled for MMIO access, but bus +* mastering remains disabled until HDM restore completes. +*/ +writel (snap->global_ctrl, hdm +CXL_HDM_DECODER_CTRL_OFFSET ); + +for (int i =0 ; i < snap->decoder_count; i++) { + rc =cxl_restore_hdm_decoder (pdev, + &snap->decoder_state[i], + &snap->settings[i], hdm); +if (rc && !first_rc) + first_rc = rc; + } + +/* Flush posted HDM writes before PCI_COMMAND can restore BME. */ +readl (hdm +CXL_HDM_DECODER_CTRL_OFFSET ); +iounmap (hdm); + } + +if (restore_command) { + rc =cxl_hdm_restore_command (pdev, command); +if (rc && !first_rc) + first_rc = rc; + } + +kfree (snap); +return first_rc; +} + +static void cxl_reset_save_disabled_state (struct pci_dev *pdev) +{ +int rc; + + rc =pci_write_config_word (pdev,PCI_COMMAND ,PCI_COMMAND_INTX_DISABLE ); +if (rc) { +pci_warn (pdev,"failed to keep device disabled after CXL reset restore failure: %d\n" , +pcibios_err_to_errno (rc)); +return ; + } + + rc =pci_save_state (pdev); +if (rc) +pci_warn (pdev,"failed to save disabled state after CXL reset restore failure: %d\n" , + rc); +} + +static int cxl_reset_save_restored_state (struct pci_dev *pdev,u16 command) +{ +int rc; + + rc =cxl_hdm_restore_command (pdev, command); +if (rc) { +cxl_reset_save_disabled_state (pdev); +return rc; + } + + rc =pci_save_state (pdev); +if (rc) { +pci_warn (pdev,"failed to save restored state after CXL reset: %d\n" , + rc); +cxl_reset_save_disabled_state (pdev); + } + +return rc; +} +/* * CXL r4.0 sec 9.7.2 defines the reset completion timeout encodings. * Sec 9.7.3 leaves config-space access behavior undefined for 100 ms after
@@ -528,6 +822,34 @@ struct cxl_hdm_range_context {struct list_head ranges; }; +static void cxl_pci_target_reset_done (struct pci_dev *pdev, +bool *target_prepared) +{ +if (!*target_prepared) +return ; + +pci_dev_reset_iommu_done (pdev); + *target_prepared = false; +} + +static int cxl_pci_target_reset_prepare (struct pci_dev *pdev, +bool *target_prepared) +{ +int rc; + +if (!pci_wait_for_pending_transaction (pdev)) +pci_err (pdev,"timed out waiting for pending transactions\n" ); + + rc =pci_dev_reset_iommu_prepare (pdev); +if (rc) { +pci_err (pdev,"failed to stop IOMMU for CXL reset: %d\n" , rc); +return rc; + } + + *target_prepared = true; +return 0 ; +} +static void cxl_hdm_range_context_init (struct cxl_hdm_range_context *ctx) {INIT_LIST_HEAD (&ctx->ranges);
@@ -934,26 +1256,21 @@ static int cxl_reset_wait_done(struct pci_dev *pdev, int dvsec, u16 cap) }while (true); } -static int cxl_reset_execute (struct pci_dev *pdev,int dvsec,u16 cap) +static int cxl_reset_execute (struct pci_dev *pdev,bool *target_prepared, +bool *reset_initiated,int dvsec,u16 cap) { -bool target_prepared = false;int rc, rc2; rc =cxl_reset_disable_cache (pdev, dvsec, cap);if (rc)return rc; -if (!pci_wait_for_pending_transaction (pdev)) -pci_err (pdev,"timed out waiting for pending transactions\n" ); - - rc =pci_dev_reset_iommu_prepare (pdev); -if (rc) -pci_err (pdev,"failed to stop IOMMU for CXL reset: %d\n" , rc); -else - target_prepared = true; - -if (!rc) + rc =cxl_pci_target_reset_prepare (pdev, target_prepared); +if (!rc) { rc =cxl_reset_initiate (pdev, dvsec); +if (!rc) + *reset_initiated = true; + }if (!rc) rc =cxl_reset_wait_done (pdev, dvsec, cap);
@@ -963,14 +1280,14 @@ static int cxl_reset_execute(struct pci_dev *pdev, int dvsec, u16 cap)else if (rc2) rc = rc2; -if (target_prepared) -pci_dev_reset_iommu_done (pdev);return rc; }int cxl_reset_function (struct pci_dev *pdev,bool probe) {struct cxl_hdm_range_context range_ctx; +bool target_prepared = false; +bool reset_initiated = false;int dvsec;int rc;u16 cap;
@@ -993,9 +1310,24 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe) scoped_guard(rwsem_write, &cxl_rwsem.region) { rc =cxl_hdm_ranges_prepare (&range_ctx, pdev);if (!rc) - rc =cxl_reset_execute (pdev, dvsec, cap); + rc =cxl_reset_execute (pdev, &target_prepared, + &reset_initiated, dvsec, cap); +if (!rc) { +u16 command; + +cxl_restore_pci_state_for_hdm_restore (pdev, &command); + rc =cxl_restore_hdm (pdev); +if (rc) +cxl_reset_save_disabled_state (pdev); +else + rc =cxl_reset_save_restored_state (pdev, + command); + }else if (reset_initiated) { +cxl_reset_save_disabled_state (pdev); + }cxl_hdm_range_context_destroy (&range_ctx); } +cxl_pci_target_reset_done (pdev, &target_prepared);return rc; }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 09/12] cxl: Restore CXL HDM state after PCI reset
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
After CXL reset, restore PCI config state enough to reach HDM MMIO, restore cached global and per-decoder HDM state, and then run the normal PCI restore callbacks.
Keep the target IOMMU reset block active until HDM restore completes so Bus Master Enable cannot reopen DMA before decoder state is valid.
Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Re: [PATCH v11 09/12] cxl: Restore CXL HDM state after PCI reset
In reply to Srirangan MadhavanQuoted 242 lines · Srirangan Madhavan
After CXL reset, restore PCI config state enough to reach HDM MMIO, restore cached global and per-decoder HDM state, and then run the normal PCI restore callbacks.
Keep the target IOMMU reset block active until HDM restore completes so Bus Master Enable cannot reopen DMA before decoder state is valid.
Signed-off-by Srirangan Madhavan <[email protected]>--- drivers/cxl/core/resource.c | 362 ++++++++++++++++++++++++++++++++++-- 1 file changed, 347 insertions(+), 15 deletions(-)drivers/cxl/core/resource.c +194
-0@@ -13,6 +13,7 @@#include <linux/kernel.h> #include <linux/list.h> #include <linux/memregion.h> +#include <linux/overflow.h> #include <linux/pci.h> #include <linux/slab.h> @@ -83,6 +84,26 @@ static int cxld_await_commit(void __iomem *hdm, int id)return -ETIMEDOUT ; } +static int cxld_await_uncommit (void __iomem *hdm,int id) +{ +u32 ctrl; +int i; + +for (i =0 ; i <COMMIT_TIMEOUT_MS ; i++) { + ctrl =readl (hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +if (FIELD_GET (CXL_HDM_DECODER0_CTRL_COMMIT_ERROR , ctrl)) { + ctrl &= ~CXL_HDM_DECODER0_CTRL_COMMIT ; +writel (ctrl, hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +return -EIO ; + } +if (!FIELD_GET (CXL_HDM_DECODER0_CTRL_COMMITTED , ctrl)) +return 0 ; +fsleep (1000 ); + } + +return -ETIMEDOUT ; +} +static int setup_hw_decoder (void __iomem *hdm,struct cxl_decoder_settings *settings) {@@ -273,6 +294,31 @@ static void __iomem *cxl_pci_hdm_map(struct pci_dev *pdev,return hdm; } +static void __iomem *cxl_pci_hdm_ioremap_current (struct pci_dev *pdev, +int bar, +resource_size_t offset, +resource_size_t size) +{ +resource_size_t hdm_start, bar_len; +void __iomem *hdm; + +if (bar <0 || bar >=PCI_STD_NUM_BARS || !size) +return ERR_PTR (-EINVAL ); + + bar_len =pci_resource_len (pdev, bar); +if (!bar_len || offset > bar_len || size > bar_len - offset) +return ERR_PTR (-ENODEV ); + + hdm_start =pci_resource_start (pdev, bar) + offset; + hdm =ioremap (hdm_start, size); +if (!hdm) { +pci_err (pdev,"failed to remap CXL HDM decoder registers\n" ); +return ERR_PTR (-ENOMEM ); + } + +return hdm; +} +static void cxl_pci_hdm_read_decoder_state (struct cxl_hdm_decoder_state *state,void __iomem *hdm,int id) {@@ -285,6 +331,40 @@ static void cxl_pci_hdm_read_decoder_state(struct cxl_hdm_decoder_state *state, state->target_high =readl (hdm +CXL_HDM_DECODER0_TL_HIGH (id)); } +static int cxl_hdm_enable_mem (struct pci_dev *pdev,u16 *command, +bool *restore_command) +{ +int rc; + + *restore_command = false; + + rc =pci_read_config_word (pdev,PCI_COMMAND , command); +if (rc) +return pcibios_err_to_errno (rc); + +if (*command &PCI_COMMAND_MEMORY ) +return 0 ; + + rc =pci_write_config_word (pdev,PCI_COMMAND , + *command |PCI_COMMAND_MEMORY ); +if (rc) +return pcibios_err_to_errno (rc); + + *restore_command = true; +return 0 ; +} + +static int cxl_hdm_restore_command (struct pci_dev *pdev,u16 command) +{ +int rc; + + rc =pci_write_config_word (pdev,PCI_COMMAND , command); +if (rc) +return pcibios_err_to_errno (rc); + +return 0 ; +} + static int cxl_pci_hdm_read_decoder(struct pci_dev *pdev, struct cxl_hdm_decoder_state *state, struct cxl_decoder_settings *settings,@@ -501,6 +581,220 @@ void pci_cxl_hdm_init(struct pci_dev *pdev) }EXPORT_SYMBOL_FOR_MODULES (pci_cxl_hdm_init,"cxl_core" ); +static int cxl_hdm_decoder_uncommit (struct pci_dev *pdev,void __iomem *hdm, +int id) +{ +u32 ctrl; +int rc; + + ctrl =readl (hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +if (ctrl &CXL_HDM_DECODER0_CTRL_LOCK ) { +if (ctrl &CXL_HDM_DECODER0_CTRL_COMMITTED ) { +pci_dbg (pdev, +"CXL HDM decoder %d retained locked committed state\n" , + id); +return -EBUSY ; + } + +pci_err (pdev,"CXL HDM decoder %d is locked and uncommitted\n" , + id); +return -EIO ; + } + +if (!(ctrl &CXL_HDM_DECODER0_CTRL_COMMITTED )) +return 0 ; + + ctrl &= ~CXL_HDM_DECODER0_CTRL_COMMIT ; +writel (ctrl, hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); + + rc =cxld_await_uncommit (hdm, id); +if (rc) +pci_err (pdev,"CXL HDM decoder %d uncommit failed: %d\n" , + id, rc); + +return rc; +} + +static void cxl_restore_hdm_decoder_state (struct cxl_hdm_decoder_state *state, +void __iomem *hdm,int id) +{ +u32 ctrl = state->ctrl; + + ctrl &= ~(CXL_HDM_DECODER0_CTRL_COMMIT | +CXL_HDM_DECODER0_CTRL_COMMITTED | +CXL_HDM_DECODER0_CTRL_COMMIT_ERROR | +CXL_HDM_DECODER0_CTRL_LOCK ); + +writel (state->base_high, hdm +CXL_HDM_DECODER0_BASE_HIGH_OFFSET (id)); +writel (state->base_low, hdm +CXL_HDM_DECODER0_BASE_LOW_OFFSET (id)); +writel (state->size_high, hdm +CXL_HDM_DECODER0_SIZE_HIGH_OFFSET (id)); +writel (state->size_low, hdm +CXL_HDM_DECODER0_SIZE_LOW_OFFSET (id)); +writel (state->target_high, hdm +CXL_HDM_DECODER0_TL_HIGH (id)); +writel (state->target_low, hdm +CXL_HDM_DECODER0_TL_LOW (id)); +writel (ctrl, hdm +CXL_HDM_DECODER0_CTRL_OFFSET (id)); +} + +static int cxl_restore_hdm_decoder (struct pci_dev *pdev, +struct cxl_hdm_decoder_state *state, +struct cxl_decoder_settings *settings, +void __iomem *hdm) +{ +int rc; + + rc =cxl_hdm_decoder_uncommit (pdev, hdm, settings->id); +if (rc == -EBUSY ) +return 0 ; +if (rc) +return rc; + +cxl_restore_hdm_decoder_state (state, hdm, settings->id); + +if (!(settings->flags &CXL_DECODER_F_ENABLE )) +return 0 ; + +scoped_guard (rwsem_read, &cxl_rwsem.dpa) + rc = cxl_commit_start(hdm, settings); +if (!rc) + rc =cxl_commit_wait (hdm, settings); +if (rc) +pci_err (pdev,"CXL HDM decoder %d restore failed: %d\n" , + settings->id, rc); + +return rc; +} + +static struct cxl_hdm_info *cxl_snapshot_hdm (struct pci_dev *pdev) +{ +struct cxl_hdm_info *snap; +struct cxl_hdm_info *info; +size_t state_sz; + +guard (rwsem_read)(&cxl_rwsem.dpa); + + info = pdev->hdm; +if (!info) +return NULL ; +if (info->decoder_count <0 || + info->decoder_count >CXL_HDM_DECODER_MAX_COUNT || + (info->decoder_count && !info->decoder_state)) +return ERR_PTR (-EINVAL ); + + state_sz =array_size (info->decoder_count,sizeof (*info->decoder_state)); + snap =kzalloc (size_add (sizeof (*snap), state_sz),GFP_KERNEL ); +if (!snap) +return ERR_PTR (-ENOMEM ); + + *snap = *info; + snap->decoder_state = (void *)(snap +1 ); +memcpy (snap->decoder_state, info->decoder_state, state_sz); + +return snap; +} + +static void cxl_restore_pci_state_for_hdm_restore (struct pci_dev *pdev, +u16 *command) +{ +u32 saved_config = pdev->saved_config_space[PCI_COMMAND /4 ];
drivers/cxl/core/resource.c void pci_cxl_hdm_init(struct pci_dev *pdev) · line 707
Quoted 10 lines
drivers/cxl/core/resource.c +10
-0+ + pdev->saved_config_space[PCI_COMMAND /4 ] &= ~PCI_COMMAND_MASTER ; + pdev->saved_config_space[PCI_COMMAND /4 ] |=PCI_COMMAND_INTX_DISABLE ; +pci_restore_state (pdev); + pdev->saved_config_space[PCI_COMMAND /4 ] = saved_config; + *command = saved_config &0xffff ; +} + +static int cxl_restore_hdm (struct pci_dev *pdev) +{
In this function, after reset it will restore PCI state and HDM state, but I see no Device-DVSEC fields being restored.
What if CONFIG_LOCK is not set, can we guarantee they remain the same?
cxl_reset_enable_cache() only clears Control2 DisableCaching, I don't see it to restore Cache_enable or Mem_Enable , or Type-2 device driver is the one responsible for this ?
Best regards,
Richard Cheng.
Quoted 208 lines
drivers/cxl/core/resource.c +143
-15+struct cxl_hdm_info *snap =cxl_snapshot_hdm (pdev); +bool restore_command = false; +void __iomem *hdm; +int first_rc =0 ; +u16 command; +int rc; + +if (!snap) +return 0 ; +if (IS_ERR (snap)) +return PTR_ERR (snap); + + rc =cxl_hdm_enable_mem (pdev, &command, &restore_command); +if (rc) { +kfree (snap); +return rc; + } + + hdm =cxl_pci_hdm_ioremap_current (pdev, snap->hdm_bar, + snap->hdm_offset, snap->hdm_size); +if (IS_ERR (hdm)) { + first_rc =PTR_ERR (hdm); + }else { +/* +* Restore global HDM control before per-decoder commit. PCI +* config memory decoding is enabled for MMIO access, but bus +* mastering remains disabled until HDM restore completes. +*/ +writel (snap->global_ctrl, hdm +CXL_HDM_DECODER_CTRL_OFFSET ); + +for (int i =0 ; i < snap->decoder_count; i++) { + rc =cxl_restore_hdm_decoder (pdev, + &snap->decoder_state[i], + &snap->settings[i], hdm); +if (rc && !first_rc) + first_rc = rc; + } + +/* Flush posted HDM writes before PCI_COMMAND can restore BME. */ +readl (hdm +CXL_HDM_DECODER_CTRL_OFFSET ); +iounmap (hdm); + } + +if (restore_command) { + rc =cxl_hdm_restore_command (pdev, command); +if (rc && !first_rc) + first_rc = rc; + } + +kfree (snap); +return first_rc; +} + +static void cxl_reset_save_disabled_state (struct pci_dev *pdev) +{ +int rc; + + rc =pci_write_config_word (pdev,PCI_COMMAND ,PCI_COMMAND_INTX_DISABLE ); +if (rc) { +pci_warn (pdev,"failed to keep device disabled after CXL reset restore failure: %d\n" , +pcibios_err_to_errno (rc)); +return ; + } + + rc =pci_save_state (pdev); +if (rc) +pci_warn (pdev,"failed to save disabled state after CXL reset restore failure: %d\n" , + rc); +} + +static int cxl_reset_save_restored_state (struct pci_dev *pdev,u16 command) +{ +int rc; + + rc =cxl_hdm_restore_command (pdev, command); +if (rc) { +cxl_reset_save_disabled_state (pdev); +return rc; + } + + rc =pci_save_state (pdev); +if (rc) { +pci_warn (pdev,"failed to save restored state after CXL reset: %d\n" , + rc); +cxl_reset_save_disabled_state (pdev); + } + +return rc; +} +/* * CXL r4.0 sec 9.7.2 defines the reset completion timeout encodings. * Sec 9.7.3 leaves config-space access behavior undefined for 100 ms after @@ -528,6 +822,34 @@ struct cxl_hdm_range_context {struct list_head ranges; }; +static void cxl_pci_target_reset_done (struct pci_dev *pdev, +bool *target_prepared) +{ +if (!*target_prepared) +return ; + +pci_dev_reset_iommu_done (pdev); + *target_prepared = false; +} + +static int cxl_pci_target_reset_prepare (struct pci_dev *pdev, +bool *target_prepared) +{ +int rc; + +if (!pci_wait_for_pending_transaction (pdev)) +pci_err (pdev,"timed out waiting for pending transactions\n" ); + + rc =pci_dev_reset_iommu_prepare (pdev); +if (rc) { +pci_err (pdev,"failed to stop IOMMU for CXL reset: %d\n" , rc); +return rc; + } + + *target_prepared = true; +return 0 ; +} +static void cxl_hdm_range_context_init (struct cxl_hdm_range_context *ctx) {INIT_LIST_HEAD (&ctx->ranges);@@ -934,26 +1256,21 @@ static int cxl_reset_wait_done(struct pci_dev *pdev, int dvsec, u16 cap) }while (true); } -static int cxl_reset_execute (struct pci_dev *pdev,int dvsec,u16 cap) +static int cxl_reset_execute (struct pci_dev *pdev,bool *target_prepared, +bool *reset_initiated,int dvsec,u16 cap) { -bool target_prepared = false;int rc, rc2; rc =cxl_reset_disable_cache (pdev, dvsec, cap);if (rc)return rc; -if (!pci_wait_for_pending_transaction (pdev)) -pci_err (pdev,"timed out waiting for pending transactions\n" ); - - rc =pci_dev_reset_iommu_prepare (pdev); -if (rc) -pci_err (pdev,"failed to stop IOMMU for CXL reset: %d\n" , rc); -else - target_prepared = true; - -if (!rc) + rc =cxl_pci_target_reset_prepare (pdev, target_prepared); +if (!rc) { rc =cxl_reset_initiate (pdev, dvsec); +if (!rc) + *reset_initiated = true; + }if (!rc) rc =cxl_reset_wait_done (pdev, dvsec, cap);@@ -963,14 +1280,14 @@ static int cxl_reset_execute(struct pci_dev *pdev, int dvsec, u16 cap)else if (rc2) rc = rc2; -if (target_prepared) -pci_dev_reset_iommu_done (pdev);return rc; }int cxl_reset_function (struct pci_dev *pdev,bool probe) {struct cxl_hdm_range_context range_ctx; +bool target_prepared = false; +bool reset_initiated = false;int dvsec;int rc;u16 cap;@@ -993,9 +1310,24 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe) scoped_guard(rwsem_write, &cxl_rwsem.region) { rc =cxl_hdm_ranges_prepare (&range_ctx, pdev);if (!rc) - rc =cxl_reset_execute (pdev, dvsec, cap); + rc =cxl_reset_execute (pdev, &target_prepared, + &reset_initiated, dvsec, cap); +if (!rc) { +u16 command; + +cxl_restore_pci_state_for_hdm_restore (pdev, &command); + rc =cxl_restore_hdm (pdev); +if (rc) +cxl_reset_save_disabled_state (pdev); +else + rc =cxl_reset_save_restored_state (pdev, + command); + }else if (reset_initiated) { +cxl_reset_save_disabled_state (pdev); + }cxl_hdm_range_context_destroy (&range_ctx); } +cxl_pci_target_reset_done (pdev, &target_prepared);return rc; }Signature · 1 line
2.43.0
Re: [PATCH v11 09/12] cxl: Restore CXL HDM state after PCI reset
Quoted 5 lines · Richard Cheng
Review
Quoted 2 lines
(quoted diff) +2
-0+static int cxl_restore_hdm(struct pci_dev *pdev) +{In this function, after reset it will restore PCI state and HDM state, but I see no Device-DVSEC fields being restored.
Quoted 8 lines
What if
CONFIG_LOCKis not set, can we guarantee they remain the same?
cxl_reset_enable_cache()only clears Control2 DisableCaching, I don't see it to restoreCache_enableorMem_Enable, or Type-2 device driver is the one responsible for this ?Best regards,
Richard Cheng.
In V12 I've added cache for the CXL Device DVSEC control state during early PCI discovery and restores the Cache Enable and Mem Enable bits after reset.
For decoder locks, a decoder that remains locked and committed after reset is left unchanged. If reset clears that state, in v12 I restore the cached decoder programming, recommit it, then reapply the
cached lock bit.
Signature · 2 lines
Regards, Srirangan
PATCH v11 10/12 PCI/CXL: Expose CXL Reset as a PCI reset method
In reply to Srirangan MadhavanAdd the CXL Reset helper to the PCI reset-method table so userspace can select it through the existing reset_method ABI.
Advertise the method for Type 2 CXL devices that report CXL Reset support in the CXL Device DVSEC and pass the function-scope check. Reset execution requires cached HDM decoder state for the target so affected ranges can be validated and HDM programming can be restored. If that state is unavailable at reset time, return -ENOTTY so PCI can try the next reset method.
--- drivers/pci/pci.c | 2 ++ include/linux/pci.h | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)
drivers/pci/pci.c +2 -0
@@ -35,6 +35,7 @@#include <linux/aer.h> #include <linux/bitfield.h> #include <linux/suspend.h> +#include <cxl/cxl.h> #include "pci.h" DEFINE_MUTEX (pci_slot_mutex);
@@ -5145,6 +5146,7 @@ const struct pci_reset_fn_method pci_reset_fn_methods[] = { { }, { pci_dev_specific_reset, .name = "device_specific" }, { pci_dev_acpi_reset, .name = "acpi" }, + { cxl_reset_function, .name = "cxl_reset" }, { pcie_reset_flr, .name = "flr" }, { pci_af_flr, .name = "af_flr" }, { pci_pm_reset, .name = "pm" },
include/linux/pci.h +1 -1
@@ -52,7 +52,7 @@ PCI_STATUS_PARITY)/* Number of reset methods used in pci_reset_fn_methods array in pci.c */ -#define PCI_NUM_RESET_METHODS 8 +#define PCI_NUM_RESET_METHODS 9#define PCI_RESET_PROBE true#define PCI_RESET_DO_RESET false
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 10/12] PCI/CXL: Expose CXL Reset as a PCI reset method
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Bypass of hardware-enforced CXL decoder lock via reset --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Bypass of hardware-enforced CXL decoder lock via reset
--
commit ac98b5cb3878ca89e0ba4a99f7bf56eb9d00c253
Author: Srirangan Madhavan <[email protected]>
PCI/CXL: Expose CXL Reset as a PCI reset method
Add the CXL Reset helper to the PCI reset-method table so userspace can select it through the existing reset_method ABI.
Advertise the method for Type 2 CXL devices that report CXL Reset support in the CXL Device DVSEC and pass the function-scope check. Reset execution requires cached HDM decoder state for the target so affected ranges can be validated and HDM programming can be restored. If that state is unavailable at reset time, return -ENOTTY so PCI can try the next reset method.
Quoted 4 lines
drivers/pci/pci.c +0
-0
[ ... ]
drivers/pci/pci.c const struct pci_reset_fn_method pci_reset_fn_methods[] = { · line 5149
Quoted 5 lines
drivers/pci/pci.c +1
-0@@ -5145,6 +5146,7 @@ const struct pci_reset_fn_method pci_reset_fn_methods[] = { { }, { pci_dev_specific_reset, .name = "device_specific" }, { pci_dev_acpi_reset, .name = "acpi" }, + { cxl_reset_function, .name = "cxl_reset" },
[Severity: High]
Does exposing this reset method allow bypassing the hardware-enforced CXL decoder lock?
When a privileged user triggers a PCI reset on the CXL device via sysfs, the hardware reset clears the decoder lock bit.
During recovery, the lock bit is explicitly stripped from the saved state before restoration:
drivers/cxl/core/resource.c:cxl_restore_hdm_decoder_state() {
...
ctrl &= ~(CXL_HDM_DECODER0_CTRL_COMMIT | CXL_HDM_DECODER0_CTRL_COMMITTED | CXL_HDM_DECODER0_CTRL_COMMIT_ERROR | CXL_HDM_DECODER0_CTRL_LOCK);
...
}
Then in setup_hw_decoder(), which is called by cxl_commit_start(), the decoder is committed but fails to check settings->flags & CXL_DECODER_F_LOCK to re-apply the lock. The modified control register is written to hardware, leaving it permanently unlocked:
drivers/cxl/core/resource.c:setup_hw_decoder() {
...
writel(ctrl, hdm + CXL_HDM_DECODER0_CTRL_OFFSET(id));
...
}
Since firmware/BIOS sets the CXL decoder lock bit to prevent system memory map modifications, does exposing this reset method expose the system to malicious reprogramming of CXL decoders by bypassing the hardware lock?
Quoted 1 line
drivers/pci/pci.c +0
-0{ pcie_reset_flr, .name = "flr" },
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
PATCH v11 11/12 Documentation/ABI: Document CXL Reset PCI reset method
In reply to Srirangan MadhavanDocument the "cxl_reset" PCI reset_method value for Type 2 CXL devices. CXL Reset is device scoped, requires affected memory to be idle, invalidates CPU caches, restores cached HDM decoder state, and does not request Memory Clear.
--- Documentation/ABI/testing/sysfs-bus-pci | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +15 -0
@@ -153,6 +153,21 @@ Description: "default" enables all supported reset methods in the default ordering. + If present, "cxl_reset" selects CXL Reset for CXL Type 2 + devices that advertise CXL Reset support. The method is only + listed for single-function devices so CXL Reset is not exposed + when the reset could affect another PCI function. + + Before issuing CXL Reset, the kernel rejects the operation if + cached HDM state is unavailable or affected CXL memory is busy. + It reserves affected HDM ranges and, where supported, explicitly + invalidates CPU caches for those ranges. The kernel then disables + CXL.cache, initiates cache write-back where supported, quiesces + the PCI function, and issues CXL Reset. After reset, the kernel + restores PCI config state needed to access HDM MMIO, restores + cached HDM decoder state, and completes PCI reset recovery. + "cxl_reset" does not request CXL Reset Memory Clear. + What: /sys/bus/pci/devices/.../reset Date: July 2009 Contact: Michael S. Tsirkin <[email protected]>
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 11/12] Documentation/ABI: Document CXL Reset PCI reset method
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
Document the "
cxl_reset" PCIreset_methodvalue for Type 2 CXL devices. CXL Reset is device scoped, requires affected memory to be idle, invalidates CPU caches, restores cached HDM decoder state, and does not request Memory Clear.Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
PATCH v11 12/12 PCI/CXL: Restore HDM state after CXL bus reset
In reply to Srirangan MadhavanCXL bus reset can clear HDM global control and decoder programming. Restore cached HDM state after a successful cxl_bus reset while the IOMMU reset block remains active.
Restore PCI config state first so BARs are valid before HDM MMIO access. If HDM restore fails, save a disabled PCI command state so later PCI restore does not reopen bus mastering over invalid decoder state.
Reject cxl_bus reset for multifunction devices, matching cxl_reset.
This restores only HDM component decoder state. It does not restore CXL Device DVSEC range/control registers.
--- drivers/cxl/core/resource.c | 18 ++++++++++++++++++ drivers/pci/pci.c | 6 ++++++ include/cxl/cxl.h | 6 ++++++ 3 files changed, 30 insertions(+)
drivers/cxl/core/resource.c +18 -0
@@ -850,6 +850,24 @@ static int cxl_pci_target_reset_prepare(struct pci_dev *pdev,return 0 ; } +int cxl_restore_hdm_after_pci_reset (struct pci_dev *pdev) +{ +u16 command; +int rc; + +device_lock_assert (&pdev->dev); +guard (rwsem_write)(&cxl_rwsem.region); + +cxl_restore_pci_state_for_hdm_restore (pdev, &command); + rc =cxl_restore_hdm (pdev); +if (rc) { +cxl_reset_save_disabled_state (pdev); +return rc; + } + +return cxl_reset_save_restored_state (pdev, command); +} +static void cxl_hdm_range_context_init (struct cxl_hdm_range_context *ctx) {INIT_LIST_HEAD (&ctx->ranges);
drivers/pci/pci.c +6 -0
@@ -5027,6 +5027,9 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)if (!dvsec)return -ENOTTY ; +if (dev->multifunction) +return -ENOTTY ; +if (probe)return 0 ;
@@ -5054,6 +5057,9 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)pci_write_config_word (bridge, dvsec +PCI_DVSEC_CXL_PORT_CTL , reg); +if (!rc) + rc =cxl_restore_hdm_after_pci_reset (dev); +pci_dev_reset_iommu_done (dev);return rc; }
include/cxl/cxl.h +6 -0
@@ -162,6 +162,7 @@ struct cxl_hdm_info {#ifdef CONFIG_CXL_RESET void pci_cxl_hdm_init (struct pci_dev *pdev);void pci_cxl_hdm_release (struct pci_dev *pdev); +int cxl_restore_hdm_after_pci_reset (struct pci_dev *pdev);int cxl_reset_function (struct pci_dev *pdev,bool probe);#else static inline void pci_cxl_hdm_init (struct pci_dev *pdev)
@@ -172,6 +173,11 @@ static inline void pci_cxl_hdm_release(struct pci_dev *pdev) { } +static inline int cxl_restore_hdm_after_pci_reset (struct pci_dev *pdev) +{ +return 0 ; +} +static inline int cxl_reset_function (struct pci_dev *pdev,bool probe) {return -ENOTTY ;
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v11 12/12] PCI/CXL: Restore HDM state after CXL bus reset
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
CXL bus reset can clear HDM global control and decoder programming. Restore cached HDM state after a successful
cxl_busreset while the IOMMU reset block remains active.Restore PCI config state first so BARs are valid before HDM MMIO access. If HDM restore fails, save a disabled PCI command state so later PCI restore does not reopen bus mastering over invalid decoder state.
Quoted 8 lines
Reject
cxl_busreset for multifunction devices, matchingcxl_reset.This restores only HDM component decoder state. It does not restore CXL Device DVSEC range/control registers.
Reported-by Alex Williamson <[email protected]>Tested-by Richard Cheng <[email protected]>Signed-off-by Srirangan Madhavan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
Quoted 8 lines · Srirangan Madhavan
Hi folks!
This series adds CXL Reset support for CXL Type 2 devices through the existing PCI
reset_methodABI. The reset sequence follows the CXL 4.0 specification [1], including CXL.cache disable, optional cache writeback, CXL Reset initiation, ResetComplete polling, and ResetError reporting.
Hi Srirangan,
Quoted 5 lines
The userspace ABI is the existing PCI reset interface:
/sys/bus/pci/devices/.../reset_method /sys/bus/pci/devices/.../reset
Quoted 9 lines
Userspace can select "
cxl_reset" inreset_methodand then trigger reset through the existing reset attribute.CXL Reset is exposed conservatively. The target must be a single-function CXL Type 2 device with CXL.cache and CXL.mem enabled, and the kernel must have cached HDM state sufficient to validate affected ranges and restore decoder programming. The reset preserves device memory; it does not request CXL Reset Memory Clear.
It seems reasonable to me, the whole approach.
I wonder if we can document the complete reset-safety contract down ?
Especially what guarantees that all CXL.mem users and functions are quiesced ? Besides HDM EP , what are other EP state that should be retained or restored ?
I think it will make people understand the mechanism more easily.
Best regards,
Richard Cheng.
Quoted 150 lines
Following Dan's v6 feedback, this uses the existing PCI
reset_methodinterface instead of adding a memdev-specific sysfs ABI.v11 changes from v10 [2]:
- Rebase onto v7.3-rc1, which includes the CXL Type 2 enabling work, and use the merged public CXL PCI header.
- Rename the built-in support symbol toCONFIG_CXL_RESETand fix built-in/module linkage forCONFIG_CXL_BUS=m.
- Fixtools/testing/cxllinkage without compiling production helper objects twice in the mockcxl_coremodule.
- Simplify decoder snapshot handling and preserve endpoint DPA skip ownership under the existing DPA lock.
- Keep policy out of shared decoder parsing by accepting spec-legal zero-size decoder state, while skipping zero-size ranges during reset validation.
- Tighten early HDM cache setup, capability checks, BAR bounds, MSE enable/restore handling, and cache publication locking.
- Keep the conservative multifunction-device rejection requested in earlier review.
- Harden post-reset failure handling so MSE and BME remain disabled when HDM state cannot be safely restored.
- Serialize CXL bus-reset HDM restore against region updates and flush posted HDM writes beforePCI_COMMANDcan restore bus mastering.
- Address the remaining reviewability and naming feedback from v10.Motivation:
-----------
- Type 2 devices need a CXL-specific reset mechanism beyond existing PCI reset methods.- FLR does not reset CXL.cache or CXL.mem protocol state. CXL Reset is the architectural reset mechanism for those protocols.
- The PCI
reset_methodABI lets userspace select CXL Reset before falling back to broader bus reset methods.Change Description:
-------------------Patch 1: cxl: Move HDM decoder programming helpers
- Introduce a decoder settings snapshot collected under the DPA lock.
- Move low-level settings-based HDM programming intoresource.c.Patch 2: cxl: Make HDM commit helpers available to reset code
- Build generic HDM resource helpers underCONFIG_CXL_RESET.
- Export the lock and commit helpers needed whencxl_coreis modular.Patch 3: cxl: Share HDM decoder decode logic
- Share HDM decoder decode and validation between normal CXL core enumeration and early PCI HDM cache setup.
- Decode zero-size committed decoder state without imposing caller policy.Patch 4: cxl: Cache decoder settings on PCI devices
- Cache CXL core HDM decoder settings inpci_dev->hdm.
- Refresh the cache as decoders are enumerated, committed, or reset.Patch 5: cxl: Cache endpoint decoder settings during PCI enumeration
- Snapshot endpoint HDM state during PCI enumeration.
- Cache the HDM register locator as BAR-relative metadata and preserve raw decoder state for reset restore.Patch 6: cxl: Add CXL Device Reset helper
- Add the internal DVSEC reset sequence.
- Disable CXL.cache, perform cache writeback where supported, initiate CXL Reset, and wait for completion.Patch 7: cxl: Validate HDM ranges before CXL reset
- Collect enabled cached HDM ranges, reject busy ranges, and perform CPU cache invalidation when a runtime backend is available.Patch 8: PCI/CXL: Reject CXL Reset on multifunction devices
- Conservatively reject CXL Reset where function scope cannot be assumed.Patch 9: cxl: Restore CXL HDM state after PCI reset
- Restore cached global and per-decoder HDM state after reset while the target IOMMU reset block remains active.Patch 10: PCI/CXL: Expose CXL Reset as a PCI reset method
- Add "cxl_reset" to the PCI reset-method table ahead of FLR.Patch 11:
Documentation/ABI: Document CXL Reset PCI reset method
- Document the newreset_methodvalue and reset behavior.Patch 12: PCI/CXL: Restore HDM state after CXL bus reset
- Restore cached CXL HDM decoder state aftercxl_busreset.The CPU cache invalidation step depends on
cpu_cache_invalidate_memregion()support for the affected address ranges. If no runtime provider is available, the kernel emits a warning and continues after the affected HDM ranges have been reserved.Example:
echo cxl_reset > /sys/bus/pci/devices/0000:bb:dd.f/reset_method echo 1 > /sys/bus/pci/devices/0000:bb:dd.f/resetTesting:
- Built every intermediate patch, plus the v7.3-rc1 baseline, withCONFIG_CXL_BUS=m. All 12 patches passed a full kernel build andmake M=tools/testing/cxl/independently.
- Built and booted the series on an arm64 NVIDIA NVL72 system and successfully exercised CXL Reset through the PCI reset sysfs ABI.
- Earlier revision testing ran 100 iterations ofcxl_resetand exercisedcxl_busseparately with HDM state restoration.References:
[1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[2] https://lore.kernel.org/linux-cxl/[email protected]/
[3] https://lore.kernel.org/linux-cxl/[email protected]/
[4] https://lore.kernel.org/linux-cxl/[email protected]/
[5] https://lore.kernel.org/linux-cxl/[email protected]/Srirangan Madhavan (12):
cxl: Move HDM decoder programming helpers
cxl: Make HDM commit helpers available to reset code
cxl: Share HDM decoder decode logic
cxl: Cache decoder settings on PCI devices
cxl: Cache endpoint decoder settings during PCI enumeration
cxl: Add CXL Device Reset helper
cxl: Validate HDM ranges before CXL reset PCI/CXL: Reject CXL Reset on multifunction devices
cxl: Restore CXL HDM state after PCI reset PCI/CXL: Expose CXL Reset as a PCI reset methodDocumentation/ABI: Document CXL Reset PCI reset method PCI/CXL: Restore HDM state after CXL bus resetDocumentation/ABI/testing/sysfs-bus-pci | 15 + drivers/cxl/Kconfig | 4 + drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/core.h | 8 + drivers/cxl/core/hdm.c | 324 +++--- drivers/cxl/core/pci.c | 7 +- drivers/cxl/core/port.c | 19 + drivers/cxl/core/regs.c | 4 + drivers/cxl/core/resource.c | 1351 +++++++++++++++++++++++ drivers/cxl/cxl.h | 41 - drivers/pci/bus.c | 2 + drivers/pci/pci.c | 8 + drivers/pci/probe.c | 2 + include/cxl/cxl.h | 116 +- include/linux/pci.h | 8 +- include/uapi/linux/pci_regs.h | 14 + tools/testing/cxl/Kbuild | 1 - 17 files changed, 1715 insertions(+), 211 deletions(-) create mode 100644 drivers/cxl/core/resource.cbase-commit:
cee9395acd8043be0644b25c34bfa86623f2b935Signature · 1 line
2.43.0
Quoted 7 lines · Richard Cheng
Quoted 4 lines
and the kernel must have cached HDM state sufficient to validate affected ranges and restore decoder programming. The reset preserves device memory; it does not request CXL Reset Memory Clear.
It seems reasonable to me, the whole approach.
I wonder if we can document the complete reset-safety contract down ?
Quoted 7 lines
Especially what guarantees that all CXL.mem users and functions are quiesced ? Besides HDM EP , what are other EP state that should be retained or restored ?
I think it will make people understand the mechanism more easily.
Best regards,
Richard Cheng.
Thank you. I expanded the v12 cover letter and ABI documentation to describe the complete reset-safety contract, including the quiescing assumptions, state restoration, and limitations around memory offlining and mapping revocation.
Signature · 2 lines
Regards, Srirangan
Quoted 13 lines · On Wed, 2 Sep 2026 07:27:52 +0000 Srirangan Madhavan <[email protected]>…
Hi folks!
This series adds CXL Reset support for CXL Type 2 devices through the existing PCI
reset_methodABI. The reset sequence follows the CXL 4.0 specification [1], including CXL.cache disable, optional cache writeback, CXL Reset initiation, ResetComplete polling, and ResetError reporting.The userspace ABI is the existing PCI reset interface:
/sys/bus/pci/devices/.../reset_method /sys/bus/pci/devices/.../reset
Quoted 8 lines
Userspace can select "
cxl_reset" inreset_methodand then trigger reset through the existing reset attribute.CXL Reset is exposed conservatively. The target must be a single-function CXL Type 2 device with CXL.cache and CXL.mem enabled, and the kernel must have cached HDM state sufficient to validate affected ranges and restore decoder programming. The reset preserves device memory; it does not request CXL Reset Memory Clear.
How does this provide multi-tenancy then? vfio-pci considers the device to be scrubbed of tenant data after a pci_reset_function() and I don't think Manish's series alters that. Thanks,
Alex
Quoted 5 lines · Alex Williamson
How does this provide multi-tenancy then? vfio-pci considers the device to be scrubbed of tenant data after a
pci_reset_function()and I don't think Manish's series alters that. Thanks,Alex
Hi Alex!
I missed considering this.
In V12 I'm exposing cxl_reset when the device advertises CXL Reset Memory Clear capability and sets RST_MEM_CLR_EN when initiating reset. I added the gate to prevent exposing a non-scrubbing reset to VFIO.
Signature · 2 lines
Regards, Srirangan