Skip to contentSkip to conversation

vfio/pci: Add CXL Type-2 device passthrough support

PATCH Superseded [email protected] 5 participants 31 messages Apr 1 – Apr 17

Subsystems (7)

v2 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v2 cover vfio/pci: Add CXL Type-2 device passthrough support

From: Manish Honap <[email protected]>

CXL Type-2 accelerators (e.g. CXL.mem-capable GPUs) cannot be passed through to virtual machines with stock vfio-pci because the driver has no concept of HDM decoder management, DPA region exposure, or component register emulation. This series wires all of that into vfio-pci-core behind a new CONFIG_VFIO_CXL_CORE optional module, without requiring a variant driver.

When a CXL Device DVSEC (Vendor ID 0x1E98, ID 0x0000) is detected at device open time, the driver:

- Probes the HDM Decoder Capability block in the component registers and allocates a DPA region through the CXL subsystem. On devices where firmware has already committed a decoder, the kernel skips allocation and re-uses the committed range.

- Builds a kernel-owned shadow of the HDM register block. The VMM reads and writes this shadow through a dedicated COMP_REGS VFIO region rather than touching the hardware directly. The kernel enforces CXL 3.1 bit-field rules: reserved bits, read-only bits, the COMMIT/COMMITTED latch, and the LOCK→0 reprogram path for firmware-committed decoders.

- Exposes the DPA range as a second VFIO region (VFIO_REGION_SUBTYPE_CXL) backed by the kernel-assigned HPA. PTEs are inserted lazily on first page fault and torn down atomically under memory_lock during FLR.

- Intercepts writes to the CXL DVSEC configuration-space registers (Control, Status, Control2, Status2, Lock, Range Base) and replays them through a per-device vconfig shadow, enforcing RWL/RW1CS/RWO access semantics and the CONFIG_LOCK one-shot latch.

- Returns a VFIO_DEVICE_INFO_CAP_CXL capability (id=6) carrying the HDM register BAR index and offset, commit flags, and the indices of the DPA and COMP_REGS regions. HDM decoder count and the HDM block offset within COMP_REGS are derivable by the VMM from the CXL Capability Array in the COMP_REGS region itself, so they are not duplicated in the capability struct.

- Builds a sparse-mmap capability for the component register BAR so VMMs can map GPU/accelerator register windows while the kernel protects the CXL component register block. Three physical layouts are handled: component block at the BAR end, at the start, and in the middle.

- Provides a module parameter (disable_cxl=1) and a per-device flag

    (vdev->disable_cxl) for suppressing the feature without recompiling.

- Includes selftests covering device detection, capability parsing, region enumeration, HDM register emulation, DPA mmap with page-fault insertion, FLR invalidation, and DVSEC register emulation.

The series is applied on top of the cxl/next branch using the base specified at the end of this cover letter plus Alejandro's v23 Type-2 device support patches [1].

Series structure
================

Patches 1-5 extend the CXL subsystem with the APIs vfio-pci needs.

Patches 6-8 add the vfio-pci-core plumbing (UAPI, device state, Kconfig/build).

Patches 9-15 implement the core device lifecycle: detection, HDM emulation, media readiness, region management, DPA region, and DVSEC emulation.

Patches 16-18 wire everything together at open/close time and populate the VFIO ioctl paths.

Patches 19-20 add documentation and selftests.

Changes since v1
================

UAPI struct minimization (patch 6)

v1 carried hdm_count, hdm_regs_size, hdm_decoder_offset, dpa_size, and a pad byte in vfio_device_info_cap_cxl. All four fields are derivable from data the VMM already has: hdm_count and the HDM block offset come from the CXL Capability Array in the COMP_REGS region, hdm_regs_size is implicit in the COMP_REGS region size, and dpa_size is the DPA region size. v2 drops them and replaces pad with reserved[3]. The VFIO_CXL_CAP_PRECOMMITTED flag is gone; the single VFIO_CXL_CAP_FIRMWARE_COMMITTED flag covers both the committed and precommitted cases. VFIO_CXL_CAP_CACHE_CAPABLE is added to expose the HDM-DB (CXL.cache) capability bit.

Component BAR access: sparse mmap instead of blanket rejection (patch 17)

v1 returned size=0 for the component BAR and rejected all mmap and r/w access to it. That broke GPU passthrough scenarios where the device puts accelerator register windows in the same BAR as the CXL component registers. v2 replaces the blanket rejection with a sparse-mmap capability that advertises only the GPU register windows, carving out the component register block. vfio_cxl_mmap_overlaps_comp_regs() rejects only the sub-range covering [comp_reg_offset, comp_reg_offset
+ comp_reg_size); everything else in the BAR remains mappable.

CXL register defines moved to uapi/cxl/cxl_regs.h (patch 3)

v1 placed the component register defines in a private header (include/cxl/cxl_regs.h). v2 moves them to include/uapi/cxl/cxl_regs.h so VMMs can include them directly without duplicating definitions.

HDM API simplification (patch 1)

v1 exported cxl_get_hdm_reg_info() which returned a raw struct with offset and size fields. v2 replaces it with cxl_get_hdm_info() which uses the cached count already populated by cxl_probe_component_regs() and returns a single struct with all HDM metadata, removing the need for callers to re-read the hardware.

cxl_await_range_active() split (patch 4)

cxl_await_media_ready() requires a CXLMDEV mailbox register, which Type-2 accelerators may not have. v2 splits out cxl_await_range_active() so the HDM range-active poll can be used independently of the media ready path.

LOCK→0 transition in HDM ctrl write emulation (patch 11)

v1 did not handle the case where a guest tries to clear the LOCK bit to reprogram a firmware-committed decoder. v2 allows this transition and re-programs the hardware accordingly.

Component register buffer allocation (patch 11)

v1 allocated only the HDM register sub-range in the COMP_REGS buffer. v2 allocates the full CXL_COMPONENT_REG_BLOCK_SIZE so future patches can expose other capability blocks (e.g. RAS, CXL.cache) without a structural change.

Register region setup split (patch 16)

v1 tied region registration to the detection/init path. v2 splits it into explicit vfio_cxl_register_cxl_region() and vfio_cxl_register_comp_regs_region() functions called from vfio_pci_open_device(), which is the correct point since vconfig and pci_config_map are valid there.

VLA fix merged into selftest (patch 20)

v1 had a separate patch 20 fixing a VLA initialisation in vfio_pci_irq_set(). v2 folds that fix into the selftest patch to keep the standalone CXL change count at 19 functional patches.

Reviewer feedback addressed
===========================

Dave Jiang:
- Replace open-coded bit shifts with FIELD_GET() / FIELD_PREP() throughout the HDM emulation code.
- Rename flag from VFIO_CXL_CAP_COMMITTED / VFIO_CXL_CAP_PRECOMMITTED to VFIO_CXL_CAP_FIRMWARE_COMMITTED; the old names were ambiguous.
- Use memremap(MEMREMAP_WB) for the DPA kernel mapping instead of ioremap_cache(), which selects the wrong memory-type descriptor on ARM64.
- Use __free() / DEFINE_FREE() scope helpers for CXL resource cleanup in the region management path, replacing the open-coded error unwind.
- Remove the unused abs_off parameter from the HDM accessor.
- Rename cxl_dvsec_control_write() to better reflect its role.

Jonathan Cameron:
- Move CXL register defines to uapi/cxl/cxl_regs.h so VMMs can consume them without a kernel header dependency.
- Use local variables with __free() rather than struct members for intermediate ERR_PTR returns in the region management code; avoids ambiguity about ownership on error paths.
- The assumption that a pre-committed decoder always exists at probe time is too restrictive for hotplug scenarios; v2 makes the precommitted path a fast-track that falls back to dynamic allocation when no committed decoder is found.

Alex Williamson:
- The blanket size=0 / mmap-reject approach for the component BAR prevents VMMs from accessing GPU register windows in the same BAR. v2 implements the sparse-mmap capability described above.

Limitations and future work
===========================

Switched topologies with more than one caching agent are not yet supported; that is planned for a follow-on series.

RAS/ECC handling and CXL core reset integration (cxl_reset support from Srirangan [2]) will be added in subsequent patches.

Dependencies
============

[1] CXL Type-2 device basic support (Alejandro Lucero-Palau, v23):

    https://lore.kernel.org/linux-cxl/[email protected]/

[2] CXL reset support for Type-2 devices (Srirangan Madhavan):

    https://lore.kernel.org/linux-cxl/[email protected]/
Cc Alex Williamson <[email protected]>
Cc Dan Williams <[email protected]>
Cc Jonathan Cameron <[email protected]>
Cc Dave Jiang <[email protected]>
Cc Alejandro Lucero <[email protected]>

base-commit: 3f7938b1aec7f06d5b23adca83e4542fcf027001 --

Manish Honap (20):
cxl: Add cxl_get_hdm_info() for HDM decoder metadata
cxl: Declare cxl_find_regblock and cxl_probe_component_regs in public
header
cxl: Move component/HDM register defines to uapi/cxl/cxl_regs.h
cxl: Split cxl_await_range_active() from media-ready wait
cxl: Record BIR and BAR offset in cxl_register_map
vfio: UAPI for CXL-capable PCI device assignment
vfio/pci: Add CXL state to vfio_pci_core_device
vfio/pci: Add CONFIG_VFIO_CXL_CORE and stub CXL hooks vfio/cxl: Detect CXL DVSEC and probe HDM block
vfio/pci: Export config access helpers
vfio/cxl: Introduce HDM decoder register emulation framework vfio/cxl: Wait for HDM ranges and create memdev
vfio/cxl: CXL region management support
vfio/cxl: DPA VFIO region with demand fault mmap and reset zap vfio/cxl: Virtualize CXL DVSEC config writes
vfio/cxl: Register regions with VFIO layer
vfio/pci: Advertise CXL cap and sparse component BAR to userspace vfio/cxl: Provide opt-out for CXL feature
docs: vfio-pci: Document CXL Type-2 device passthrough selftests/vfio: Add CXL Type-2 VFIO assignment test

 Documentation/driver-api/index.rst            |    1 +
 Documentation/driver-api/vfio-pci-cxl.rst     |  382 +++
 drivers/cxl/core/pci.c                        |   64 +-
 drivers/cxl/core/regs.c                       |   30 +
 drivers/cxl/cxl.h                             |   46 -
 drivers/vfio/pci/Kconfig                      |    2 +
 drivers/vfio/pci/Makefile                     |    1 +
 drivers/vfio/pci/cxl/Kconfig                  |    9 +
 drivers/vfio/pci/cxl/vfio_cxl_config.c        |  306 ++
 drivers/vfio/pci/cxl/vfio_cxl_core.c          |  880 ++++++
 drivers/vfio/pci/cxl/vfio_cxl_emu.c           |  509 ++++
 drivers/vfio/pci/cxl/vfio_cxl_priv.h          |  133 +
 drivers/vfio/pci/vfio_pci.c                   |   32 +
 drivers/vfio/pci/vfio_pci_config.c            |   58 +-
 drivers/vfio/pci/vfio_pci_core.c              |   46 +-
 drivers/vfio/pci/vfio_pci_priv.h              |   66 +
 drivers/vfio/pci/vfio_pci_rdwr.c              |   16 +-
 include/cxl/cxl.h                             |   51 +
 include/linux/vfio_pci_core.h                 |   10 +
 include/uapi/cxl/cxl_regs.h                   |  160 +
 include/uapi/linux/vfio.h                     |   86 +
 tools/testing/selftests/vfio/Makefile         |    1 +
 .../selftests/vfio/lib/vfio_pci_device.c      |    3 +-
 .../selftests/vfio/vfio_cxl_type2_test.c      | 2631 +++++++++++++++++
 24 files changed, 5459 insertions(+), 64 deletions(-)
 create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst
 create mode 100644 drivers/vfio/pci/cxl/Kconfig
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_config.c
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_core.c
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_emu.c
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_priv.h
 create mode 100644 include/uapi/cxl/cxl_regs.h
 create mode 100644 tools/testing/selftests/vfio/vfio_cxl_type2_test.c

--
2.25.1

#1 lore raw

PATCH v2 1/20 cxl: Add cxl_get_hdm_info() for HDM decoder metadata

From: Manish Honap <[email protected]>

cxl_probe_component_regs() finds the HDM decoder block during device probe and caches its location, but does not record the decoder count and does not expose the result outside drivers/cxl/.

vfio-cxl needs the decoder count and the byte offset and size of the HDM block without re-running the probe sequence. Record decoder_cnt in rmap->count when parsing the HDM capability in cxl_probe_component_regs(), extend struct cxl_reg_map with a count member, and add cxl_get_hdm_info() to return offset, size, and count from the cached map.

Export under the CXL namespace; stub to -EOPNOTSUPP when CONFIG_CXL_BUS is off.

---
 drivers/cxl/core/pci.c  | 29 +++++++++++++++++++++++++++++
 drivers/cxl/core/regs.c |  1 +
 include/cxl/cxl.h       | 16 ++++++++++++++++
 3 files changed, 46 insertions(+)
drivers/cxl/core/pci.c +29 -0
index ba2d393c540a..a5147602f91f 100644
@@ -449,6 +449,35 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm,
 }
 EXPORT_SYMBOL_NS_GPL(cxl_hdm_decode_init, "CXL");
 
+/**
+ * cxl_get_hdm_info - Get HDM decoder register block location and count
+ * @cxlds: CXL device state (must have component regs enumerated via
+ *	   cxl_probe_component_regs())
+ * @count:  number of HDM decoders in the block (from HDM Capability bits [3:0])
+ * @offset: byte offset of HDM decoder block within the component register BAR
+ * @size:   size in bytes of the HDM decoder block
+ *
+ * Return: 0 on success. -ENODEV if the HDM decoder block is not present.
+ */
+int cxl_get_hdm_info(struct cxl_dev_state *cxlds, u8 *count,
+		     resource_size_t *offset, resource_size_t *size)
+{
+	struct cxl_reg_map *hdm = &cxlds->reg_map.component_map.hdm_decoder;
+
+	if (WARN_ON(!count || !offset || !size))
+		return -EINVAL;
+
+	if (!hdm->valid)
+		return -ENODEV;
+
+	*count	= hdm->count;
+	*offset = hdm->offset;
+	*size	= hdm->size;
+
+	return 0;
+}
+EXPORT_SYMBOL_NS_GPL(cxl_get_hdm_info, "CXL");
+
 #define CXL_DOE_TABLE_ACCESS_REQ_CODE		0x000000ff
 #define   CXL_DOE_TABLE_ACCESS_REQ_CODE_READ	0
 #define CXL_DOE_TABLE_ACCESS_TABLE_TYPE		0x0000ff00
drivers/cxl/core/regs.c +1 -0
index 20c2d9fbcfe7..e828df0629d0 100644
@@ -85,6 +85,7 @@ void cxl_probe_component_regs(struct device *dev, void __iomem *base,
 			decoder_cnt = cxl_hdm_decoder_count(hdr);
 			length = 0x20 * decoder_cnt + 0x10;
 			rmap = &map->hdm_decoder;
+			rmap->count = decoder_cnt;
 			break;
 		}
 		case CXL_CM_CAP_CAP_ID_RAS:
include/cxl/cxl.h +16 -0
index 50acbd13bcf8..d86faebb99b7 100644
@@ -80,6 +80,7 @@ struct cxl_reg_map {
 	int id;
 	unsigned long offset;
 	unsigned long size;
+	u8 count;
 };
 
 struct cxl_component_reg_map {
@@ -284,4 +285,19 @@ int cxl_dpa_free(struct cxl_endpoint_decoder *cxled);
 struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
 				     struct cxl_endpoint_decoder **cxled,
 				     int ways);
+
+#ifdef CONFIG_CXL_BUS
+
+int cxl_get_hdm_info(struct cxl_dev_state *cxlds, u8 *count,
+		     resource_size_t *offset, resource_size_t *size);
+
+#else
+
+static inline
+int cxl_get_hdm_info(struct cxl_dev_state *cxlds, u8 *count,
+		     resource_size_t *offset, resource_size_t *size)
+{ return -EOPNOTSUPP; }
+
+#endif /* CONFIG_CXL_BUS */
+
 #endif /* __CXL_CXL_H__ */
Signature · 1 line
2.25.1
#2 lore raw

PATCH v2 2/20 cxl: Declare cxl_find_regblock and cxl_probe_component_regs in public header

In reply to [email protected]

From: Manish Honap <[email protected]>

vfio-cxl lives outside drivers/cxl/ but still needs to locate the component register block and fill cxl_component_reg_map. Those prototypes were stuck in the internal drivers/cxl/cxl.h.

Move the declarations to include/cxl/cxl.h next to the other vfio-facing hooks, with stubs when CXL bus support is disabled. Drop the duplicate prototypes from the private header.

---
 drivers/cxl/cxl.h |  4 ----
 include/cxl/cxl.h | 16 ++++++++++++++++
 2 files changed, 16 insertions(+), 4 deletions(-)
drivers/cxl/cxl.h +0 -4
index 2b1f7d687a0e..10ddab3949ee 100644
@@ -198,8 +198,6 @@ static inline int ways_to_eiw(unsigned int ways, u8 *eiw)
 #define   CXLDEV_MBOX_BG_CMD_COMMAND_VENDOR_MASK GENMASK_ULL(63, 48)
 #define CXLDEV_MBOX_PAYLOAD_OFFSET 0x20
 
-void cxl_probe_component_regs(struct device *dev, void __iomem *base,
-			      struct cxl_component_reg_map *map);
 void cxl_probe_device_regs(struct device *dev, void __iomem *base,
 			   struct cxl_device_reg_map *map);
 int cxl_map_device_regs(const struct cxl_register_map *map,
@@ -211,8 +209,6 @@ enum cxl_regloc_type;
 int cxl_count_regblock(struct pci_dev *pdev, enum cxl_regloc_type type);
 int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type,
 			       struct cxl_register_map *map, unsigned int index);
-int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
-		      struct cxl_register_map *map);
 int cxl_setup_regs(struct cxl_register_map *map);
 struct cxl_dport;
 int cxl_dport_map_rcd_linkcap(struct pci_dev *pdev, struct cxl_dport *dport);
include/cxl/cxl.h +16 -0
index d86faebb99b7..8ef7915a51f7 100644
@@ -286,17 +286,33 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd,
 				     struct cxl_endpoint_decoder **cxled,
 				     int ways);
 
+struct pci_dev;
+enum cxl_regloc_type;
+
 #ifdef CONFIG_CXL_BUS
 
 int cxl_get_hdm_info(struct cxl_dev_state *cxlds, u8 *count,
 		     resource_size_t *offset, resource_size_t *size);
 
+int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
+		      struct cxl_register_map *map);
+void cxl_probe_component_regs(struct device *dev, void __iomem *base,
+			      struct cxl_component_reg_map *map);
+
 #else
 
 static inline
 int cxl_get_hdm_info(struct cxl_dev_state *cxlds, u8 *count,
 		     resource_size_t *offset, resource_size_t *size)
 { return -EOPNOTSUPP; }
+static inline int
+cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
+		  struct cxl_register_map *map)
+{ return -EOPNOTSUPP; }
+static inline void
+cxl_probe_component_regs(struct device *dev, void __iomem *base,
+			 struct cxl_component_reg_map *map)
+{ }
 
 #endif /* CONFIG_CXL_BUS */
 
Signature · 1 line
2.25.1
#3 lore raw

PATCH v2 3/20 cxl: Move component/HDM register defines to uapi/cxl/cxl_regs.h

In reply to [email protected]

From: Manish Honap <[email protected]>

VFIO and other code outside the CXL core needs the same offset/mask constants the core uses for the component register block and HDM decoders.

Pull them into a new include/uapi/cxl/cxl_regs.h
(GPL-2.0 WITH Linux-syscall-note) and include it from include/cxl/cxl.h. Use the uapi-friendly __GENMASK helpers where needed. Section comments in the new file reference CXL spec r4.0 numbering.

For UAPI change, replaced the SZ_64K with actual size as the macro will not be available for userspace programs.

---
 drivers/cxl/cxl.h           | 42 ---------------------------
 include/cxl/cxl.h           |  1 +
 include/uapi/cxl/cxl_regs.h | 57 +++++++++++++++++++++++++++++++++++++
 3 files changed, 58 insertions(+), 42 deletions(-)
 create mode 100644 include/uapi/cxl/cxl_regs.h
drivers/cxl/cxl.h +0 -42
index 10ddab3949ee..172e38d58c50 100644
@@ -24,48 +24,6 @@ extern const struct nvdimm_security_ops *cxl_security_ops;
  * (port-driver, region-driver, nvdimm object-drivers... etc).
  */
 
-/* CXL 2.0 8.2.4 CXL Component Register Layout and Definition */
-#define CXL_COMPONENT_REG_BLOCK_SIZE SZ_64K
-
-/* CXL 2.0 8.2.5 CXL.cache and CXL.mem Registers*/
-#define CXL_CM_OFFSET 0x1000
-#define CXL_CM_CAP_HDR_OFFSET 0x0
-#define   CXL_CM_CAP_HDR_ID_MASK GENMASK(15, 0)
-#define     CM_CAP_HDR_CAP_ID 1
-#define   CXL_CM_CAP_HDR_VERSION_MASK GENMASK(19, 16)
-#define     CM_CAP_HDR_CAP_VERSION 1
-#define   CXL_CM_CAP_HDR_CACHE_MEM_VERSION_MASK GENMASK(23, 20)
-#define     CM_CAP_HDR_CACHE_MEM_VERSION 1
-#define   CXL_CM_CAP_HDR_ARRAY_SIZE_MASK GENMASK(31, 24)
-#define CXL_CM_CAP_PTR_MASK GENMASK(31, 20)
-
-/* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */
-#define CXL_HDM_DECODER_CAP_OFFSET 0x0
-#define   CXL_HDM_DECODER_COUNT_MASK GENMASK(3, 0)
-#define   CXL_HDM_DECODER_TARGET_COUNT_MASK GENMASK(7, 4)
-#define   CXL_HDM_DECODER_INTERLEAVE_11_8 BIT(8)
-#define   CXL_HDM_DECODER_INTERLEAVE_14_12 BIT(9)
-#define   CXL_HDM_DECODER_INTERLEAVE_3_6_12_WAY BIT(11)
-#define   CXL_HDM_DECODER_INTERLEAVE_16_WAY BIT(12)
-#define CXL_HDM_DECODER_CTRL_OFFSET 0x4
-#define   CXL_HDM_DECODER_ENABLE BIT(1)
-#define CXL_HDM_DECODER0_BASE_LOW_OFFSET(i) (0x20 * (i) + 0x10)
-#define CXL_HDM_DECODER0_BASE_HIGH_OFFSET(i) (0x20 * (i) + 0x14)
-#define CXL_HDM_DECODER0_SIZE_LOW_OFFSET(i) (0x20 * (i) + 0x18)
-#define CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(i) (0x20 * (i) + 0x1c)
-#define CXL_HDM_DECODER0_CTRL_OFFSET(i) (0x20 * (i) + 0x20)
-#define   CXL_HDM_DECODER0_CTRL_IG_MASK GENMASK(3, 0)
-#define   CXL_HDM_DECODER0_CTRL_IW_MASK GENMASK(7, 4)
-#define   CXL_HDM_DECODER0_CTRL_LOCK BIT(8)
-#define   CXL_HDM_DECODER0_CTRL_COMMIT BIT(9)
-#define   CXL_HDM_DECODER0_CTRL_COMMITTED BIT(10)
-#define   CXL_HDM_DECODER0_CTRL_COMMIT_ERROR BIT(11)
-#define   CXL_HDM_DECODER0_CTRL_HOSTONLY BIT(12)
-#define CXL_HDM_DECODER0_TL_LOW(i) (0x20 * (i) + 0x24)
-#define CXL_HDM_DECODER0_TL_HIGH(i) (0x20 * (i) + 0x28)
-#define CXL_HDM_DECODER0_SKIP_LOW(i) CXL_HDM_DECODER0_TL_LOW(i)
-#define CXL_HDM_DECODER0_SKIP_HIGH(i) CXL_HDM_DECODER0_TL_HIGH(i)
-
 /* HDM decoder control register constants CXL 3.0 8.2.5.19.7 */
 #define CXL_DECODER_MIN_GRANULARITY 256
 #define CXL_DECODER_MAX_ENCODED_IG 6
include/cxl/cxl.h +1 -0
index 8ef7915a51f7..f48274673b1b 100644
@@ -9,6 +9,7 @@
 #include <linux/ioport.h>
 #include <linux/range.h>
 #include <cxl/mailbox.h>
+#include <uapi/cxl/cxl_regs.h>
 
 /**
  * enum cxl_devtype - delineate type-2 from a generic type-3 device
include/uapi/cxl/cxl_regs.h new file +57 -0

57 lines not on this page · raw message

Signature · 1 line
2.25.1
#4 lore raw

PATCH v2 4/20 cxl: Split cxl_await_range_active() from media-ready wait

In reply to [email protected]

From: Manish Honap <[email protected]>

Before accessing CXL device memory after reset/power-on, the driver must ensure media is ready. Not every CXL device implements the CXL Memory Device register group (many Type-2 devices do not). cxl_await_media_ready() reads cxlds->regs.memdev. Access to the memory device registers on a Type-2 device may result in kernel panic.

Split the HDM DVSEC range-active poll out of cxl_await_media_ready() into a new function, cxl_await_range_active(). Type-2 devices often lack the CXLMDEV status register, so they need the range check without the memdev read. cxl_await_media_ready() now calls cxl_await_range_active() for the DVSEC poll, then reads the memory device status as before.

---
 drivers/cxl/core/pci.c | 35 ++++++++++++++++++++++++++++++-----
 include/cxl/cxl.h      |  3 +++
 2 files changed, 33 insertions(+), 5 deletions(-)
drivers/cxl/core/pci.c +30 -5
index a5147602f91f..1fbe3338a0da 100644
@@ -142,16 +142,24 @@ static int cxl_dvsec_mem_range_active(struct cxl_dev_state *cxlds, int id)
 	return 0;
 }
 
-/*
- * Wait up to @media_ready_timeout for the device to report memory
- * active.
+/**
+ * cxl_await_range_active - Wait for all HDM DVSEC memory ranges to be active
+ * @cxlds: CXL device state (DVSEC and HDM count must be valid)
+ *
+ * For each HDM decoder range reported in the CXL DVSEC capability, waits for
+ * the range to report MEM INFO VALID (up to 1s per range), then MEM ACTIVE
+ * (up to media_ready_timeout seconds per range, default 60s). Used by
+ * cxl_await_media_ready() and by callers that only need range readiness
+ * without checking the memory device status register.
+ *
+ * Return: 0 if all ranges become valid and active, -ETIMEDOUT if a timeout
+ * occurs, or a negative errno from config read on failure.
  */
-int cxl_await_media_ready(struct cxl_dev_state *cxlds)
+int cxl_await_range_active(struct cxl_dev_state *cxlds)
 {
 	struct pci_dev *pdev = to_pci_dev(cxlds->dev);
 	int d = cxlds->cxl_dvsec;
 	int rc, i, hdm_count;
-	u64 md_status;
 	u16 cap;
 
 	rc = pci_read_config_word(pdev,
@@ -172,6 +180,23 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds)
 			return rc;
 	}
 
+	return 0;
+}
+EXPORT_SYMBOL_NS_GPL(cxl_await_range_active, "CXL");
+
+/*
+ * Wait up to @media_ready_timeout for the device to report memory
+ * active.
+ */
+int cxl_await_media_ready(struct cxl_dev_state *cxlds)
+{
+	u64 md_status;
+	int rc;
+
+	rc = cxl_await_range_active(cxlds);
+	if (rc)
+		return rc;
+
 	md_status = readq(cxlds->regs.memdev + CXLMDEV_STATUS_OFFSET);
 	if (!CXLMDEV_READY(md_status))
 		return -EIO;
include/cxl/cxl.h +3 -0
index f48274673b1b..45d911735883 100644
@@ -299,6 +299,7 @@ int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
 		      struct cxl_register_map *map);
 void cxl_probe_component_regs(struct device *dev, void __iomem *base,
 			      struct cxl_component_reg_map *map);
+int cxl_await_range_active(struct cxl_dev_state *cxlds);
 
 #else
 
@@ -314,6 +315,8 @@ static inline void
 cxl_probe_component_regs(struct device *dev, void __iomem *base,
 			 struct cxl_component_reg_map *map)
 { }
+static inline int cxl_await_range_active(struct cxl_dev_state *cxlds)
+{ return -EOPNOTSUPP; }
 
 #endif /* CONFIG_CXL_BUS */
 
Signature · 1 line
2.25.1
#5 lore raw

PATCH v2 5/20 cxl: Record BIR and BAR offset in cxl_register_map

In reply to [email protected]

From: Manish Honap <[email protected]>

The Register Locator DVSEC (CXL 4.0 8.1.9) describes register blocks by BAR index (BIR) and offset within the BAR. CXL core currently only stores the resolved HPA (resource + offset) in struct cxl_register_map, so callers that need to use pci_iomap() or report the BAR to userspace must reverse-engineer the BAR from the HPA.

Add bar_index and bar_offset to struct cxl_register_map and fill them in cxl_decode_regblock() when the regblock is BAR-backed (BIR 0-5). Add cxl_regblock_get_bar_info() so callers (e.g. vfio-cxl) can get BAR index and offset directly and use pci_iomap() instead of ioremap(HPA).

Add cxl_regblock_get_bar_info() to return those fields; -EINVAL if the map is not BAR-backed.

---
 drivers/cxl/core/regs.c | 29 +++++++++++++++++++++++++++++
 include/cxl/cxl.h       | 15 +++++++++++++++
 2 files changed, 44 insertions(+)
drivers/cxl/core/regs.c +29 -0
index e828df0629d0..43661e51230a 100644
@@ -288,9 +288,37 @@ static bool cxl_decode_regblock(struct pci_dev *pdev, u32 reg_lo, u32 reg_hi,
 	map->reg_type = reg_type;
 	map->resource = pci_resource_start(pdev, bar) + offset;
 	map->max_size = pci_resource_len(pdev, bar) - offset;
+	map->bar_index = bar;
+	map->bar_offset = offset;
 	return true;
 }
 
+/**
+ * cxl_regblock_get_bar_info() - Get BAR index and offset for a BAR-backed
+ * regblock
+ * @map: Register map from cxl_find_regblock() or cxl_find_regblock_instance()
+ * @bar_index: Output BAR index (0-5). Optional, may be NULL.
+ * @bar_offset: Output offset within the BAR. Optional, may be NULL.
+ *
+ * When the register block was found via the Register Locator DVSEC and
+ * lives in a PCI BAR (BIR 0-5), this returns the BAR index and the offset
+ * within that BAR.
+ *
+ * Return: 0 if the regblock is BAR-backed (bar_index <= 5), -EINVAL otherwise.
+ */
+int cxl_regblock_get_bar_info(const struct cxl_register_map *map, u8 *bar_index,
+			      resource_size_t *bar_offset)
+{
+	if (!map || map->bar_index == 0xff)
+		return -EINVAL;
+	if (bar_index)
+		*bar_index = map->bar_index;
+	if (bar_offset)
+		*bar_offset = map->bar_offset;
+	return 0;
+}
+EXPORT_SYMBOL_NS_GPL(cxl_regblock_get_bar_info, "CXL");
+
 /*
  * __cxl_find_regblock_instance() - Locate a register block or count instances by type / index
  * Use CXL_INSTANCES_COUNT for @index if counting instances.
@@ -309,6 +337,7 @@ static int __cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_ty
 
 	*map = (struct cxl_register_map) {
 		.host = &pdev->dev,
+		.bar_index = 0xFF,
 		.resource = CXL_RESOURCE_NONE,
 	};
 
include/cxl/cxl.h +15 -0
index 45d911735883..52eb40352edc 100644
@@ -106,9 +106,16 @@ struct cxl_pmu_reg_map {
  * @resource: physical resource base of the register block
  * @max_size: maximum mapping size to perform register search
  * @reg_type: see enum cxl_regloc_type
+ * @bar_index: PCI BAR index (0-5) when regblock is BAR-backed; 0xFF otherwise
+ * @bar_offset: offset within the BAR; only valid when bar_index <= 5
  * @component_map: cxl_reg_map for component registers
  * @device_map: cxl_reg_maps for device registers
  * @pmu_map: cxl_reg_maps for CXL Performance Monitoring Units
+ *
+ * When the register block is described by the Register Locator DVSEC with
+ * a BAR Indicator (BIR 0-5), bar_index and bar_offset are set so callers can
+ * use pci_iomap(pdev, bar_index, size) and base + bar_offset instead of
+ * ioremap(resource).
  */
 struct cxl_register_map {
 	struct device *host;
@@ -116,6 +123,8 @@ struct cxl_register_map {
 	resource_size_t resource;
 	resource_size_t max_size;
 	u8 reg_type;
+	u8 bar_index;
+	resource_size_t bar_offset;
 	union {
 		struct cxl_component_reg_map component_map;
 		struct cxl_device_reg_map device_map;
@@ -300,6 +309,8 @@ int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
 void cxl_probe_component_regs(struct device *dev, void __iomem *base,
 			      struct cxl_component_reg_map *map);
 int cxl_await_range_active(struct cxl_dev_state *cxlds);
+int cxl_regblock_get_bar_info(const struct cxl_register_map *map, u8 *bar_index,
+			      resource_size_t *bar_offset);
 
 #else
 
@@ -317,6 +328,10 @@ cxl_probe_component_regs(struct device *dev, void __iomem *base,
 { }
 static inline int cxl_await_range_active(struct cxl_dev_state *cxlds)
 { return -EOPNOTSUPP; }
+static inline int
+cxl_regblock_get_bar_info(const struct cxl_register_map *map, u8 *bar_index,
+			  resource_size_t *bar_offset)
+{ return -EINVAL; }
 
 #endif /* CONFIG_CXL_BUS */
 
Signature · 1 line
2.25.1
#6 lore raw

PATCH v2 6/20 vfio: UAPI for CXL-capable PCI device assignment

In reply to [email protected]

From: Manish Honap <[email protected]>

Vendor GPUs and accelerators can expose CXL.mem (HDM-D or HDM-DB) without using PCI class code 0x0502. VMMs need a stable way to learn DPA sizing, firmware commit state, and where the extra VFIO regions live.

Add VFIO_DEVICE_FLAGS_CXL (bit 9) and VFIO_DEVICE_INFO_CAP_CXL (cap ID 6). The capability struct carries:

hdm_regs_bar_index PCI BAR containing the component register block
hdm_regs_offset byte offset within that BAR to the CXL.mem area
(comp_reg_offset + CXL_CM_OFFSET)
dpa_region_index VFIO region index for the DPA window
comp_regs_region_index VFIO region index for the emulated COMP_REGS

HDM decoder count and the HDM block offset within COMP_REGS are intentionally absent; both are derivable from the CXL Capability Array at COMP_REGS offset 0. Locate cap ID 0x5 (HDM) and read bits[31:20] of its entry for the byte offset. Then read bits[3:0] of the HDM Decoder Capability register for the count: count = (field == 0) ? 1 : field * 2.

Two flags accompany the capability:

VFIO_CXL_CAP_FIRMWARE_COMMITTED
A decoder covering @dpa_size bytes was programmed and committed by platform firmware before device open. The VMM can use the DPA region immediately without re-committing.

VFIO_CXL_CAP_CACHE_CAPABLE
The device is HDM-DB (CXL.mem + CXL.cache). HDM-DB requires a Write-Back Invalidation sequence before FLR to flush dirty cache lines; HDM-D (CXL.mem only) does not. QEMU uses this flag to schedule WBI and to report Back-Invalidation capability accurately in the virtual CXL topology. Mirrors the Cache_Capable bit from the CXL DVSEC Capability register.

---
 include/uapi/linux/vfio.h | 86 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 86 insertions(+)
include/uapi/linux/vfio.h +86 -0
index ac2329f24141..fc07fc50b2e5 100644
@@ -215,6 +215,16 @@ struct vfio_device_info {
 #define VFIO_DEVICE_FLAGS_FSL_MC (1 << 6)	/* vfio-fsl-mc device */
 #define VFIO_DEVICE_FLAGS_CAPS	(1 << 7)	/* Info supports caps */
 #define VFIO_DEVICE_FLAGS_CDX	(1 << 8)	/* vfio-cdx device */
+/*
+ * Vendor-specific CXL device with CXL.mem capability (HDM-D or HDM-DB
+ * decoder, PCI class code != PCI_CLASS_MEMORY_CXL).  Covers CXL Type-2
+ * accelerators and non-class-code Type-3 variants.  When set,
+ * VFIO_DEVICE_FLAGS_PCI is also set (same device is a PCI device). The
+ * capability chain (VFIO_DEVICE_FLAGS_CAPS) contains VFIO_DEVICE_INFO_CAP_CXL
+ * describing HDM decoders, region indices, decoder layout, and CXL-specific
+ * options.
+ */
+#define VFIO_DEVICE_FLAGS_CXL   (1 << 9)        /* Device supports CXL */
 	__u32	num_regions;	/* Max region index + 1 */
 	__u32	num_irqs;	/* Max IRQ index + 1 */
 	__u32   cap_offset;	/* Offset within info struct of first cap */
@@ -257,6 +267,70 @@ struct vfio_device_info_cap_pci_atomic_comp {
 	__u32 reserved;
 };
 
+/*
+ * VFIO_DEVICE_INFO_CAP_CXL - CXL Type-2 device capability
+ *
+ * Present in the device info capability chain when VFIO_DEVICE_FLAGS_CXL
+ * is set. Describes Host Managed Device Memory (HDM) layout and CXL
+ * memory options so that userspace (e.g. QEMU) can expose the CXL region
+ * and component registers correctly to the guest.
+ *
+ * The HDM decoder count and HDM decoder block offset within the COMP_REGS
+ * region are derivable from the COMP_REGS region itself.
+ *
+ * To find the HDM decoder block offset (hdm_decoder_offset), traverse the CXL
+ * Capability Array starting at COMP_REGS region offset 0:
+ *   - Dword 0 bits[31:24] (CXL_CM_CAP_HDR_ARRAY_SIZE_MASK): number of
+ *     capability entries.
+ *   - Each subsequent dword at offset (cap * 4): bits[15:0] = cap ID
+ *     (CXL_CM_CAP_HDR_ID_MASK), bits[31:20] = byte offset from COMP_REGS
+ *     start to that capability's register block (CXL_CM_CAP_PTR_MASK).
+ *   - Locate the entry with cap ID == CXL_CM_CAP_CAP_ID_HDM (0x5); the
+ *     extracted bits[31:20] value is directly the byte offset
+ *     hdm_decoder_offset (no further scaling required).
+ *
+ * To find the HDM decoder count, pread the HDM Decoder Capability register
+ * at hdm_decoder_offset + CXL_HDM_DECODER_CAP_OFFSET within the
+ * COMP_REGS region; bits[3:0] (CXL_HDM_DECODER_COUNT_MASK) encode the count
+ * using the formula: count = (field == 0) ? 1 : field * 2.
+ */
+#define VFIO_DEVICE_INFO_CAP_CXL		6
+struct vfio_device_info_cap_cxl {
+	struct vfio_info_cap_header header;
+	__u8  hdm_regs_bar_index; /* PCI BAR containing HDM registers */
+	__u8  reserved[3];
+	__u32 flags;
+/* Decoder was committed by host firmware/BIOS */
+#define VFIO_CXL_CAP_FIRMWARE_COMMITTED		(1 << 0)
+/*
+ * Device implements an HDM-DB decoder (CXL.cache + CXL.mem).  Reflects
+ * the Cache_Capable bit (bit 0) in the CXL DVSEC Capability register.
+ *
+ * When clear: HDM-D decoder (CXL.mem only, no CXL.cache).  FLR does not
+ * require a Write-Back Invalidation (WBI) sequence; the device holds no
+ * coherent copies of host memory.
+ *
+ * When set: HDM-DB decoder (CXL 3.0+).  The kernel driver does not
+ * perform Write-Back Invalidation (WBI) automatically.  The VMM must
+ * issue a WBI sequence before asserting FLR to flush dirty device cache
+ * lines and prevent coherency violations, and should advertise
+ * Back-Invalidation support in the virtual CXL topology.
+ */
+#define	VFIO_CXL_CAP_CACHE_CAPABLE		(1 << 1)
+	/*
+	 * Byte offset within the BAR to the CXL.mem register area start
+	 * (= comp_reg_offset + CXL_CM_OFFSET).	 This is where the CXL
+	 * Capability Array Header lives.
+	 */
+	__u64 hdm_regs_offset;
+	/*
+	 * Region indices for the two CXL VFIO device regions.
+	 * Avoids forcing userspace to scan all regions by type/subtype.
+	 */
+	__u32  dpa_region_index;       /* VFIO_REGION_SUBTYPE_CXL */
+	__u32  comp_regs_region_index; /* VFIO_REGION_SUBTYPE_CXL_COMP_REGS */
+};
+
 /**
  * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8,
  *				       struct vfio_region_info)
@@ -370,6 +444,18 @@ struct vfio_region_info_cap_type {
  */
 #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD	(1)
 
+/* 1e98 vendor PCI sub-types (CXL Consortium) */
+/*
+ * CXL memory region. Use with region type
+ * (PCI_VENDOR_ID_CXL | VFIO_REGION_TYPE_PCI_VENDOR_TYPE).
+ * DPA memory region (fault+zap mmap)
+ */
+#define VFIO_REGION_SUBTYPE_CXL                 (1)
+/*
+ * HDM decoder register emulation region (read/write only, no mmap).
+ */
+#define VFIO_REGION_SUBTYPE_CXL_COMP_REGS       (2)
+
 /* sub-types for VFIO_REGION_TYPE_GFX */
 #define VFIO_REGION_SUBTYPE_GFX_EDID            (1)
 
Signature · 1 line
2.25.1
#7 lore raw

PATCH v2 7/20 vfio/pci: Add CXL state to vfio_pci_core_device

In reply to [email protected]

From: Manish Honap <[email protected]>

Add CXL-specific state to vfio_pci_core_device structure to support CXL Type-2 device passthrough.

The new vfio_pci_cxl_state structure embeds CXL core objects:
- struct cxl_dev_state: CXL device state (from CXL core)
- struct cxl_memdev: CXL memory device
- struct cxl_region: CXL region object
- Root and endpoint decoders

Key design point: The CXL state pointer is NULL for non-CXL devices, allowing vfio-pci-core to handle both CXL and standard PCI devices with minimal overhead.

This will follow the approach where vfio-pci-core itself gains CXL awareness, rather than requiring a separate variant driver.

---
 drivers/vfio/pci/cxl/vfio_cxl_priv.h | 28 ++++++++++++++++++++++++++++
 include/linux/vfio_pci_core.h        |  3 +++
 2 files changed, 31 insertions(+)
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_priv.h
drivers/vfio/pci/cxl/vfio_cxl_priv.h new file +28 -0
new file mode 100644
index 000000000000..4cecc25db410
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Common infrastructure for CXL Type-2 device variant drivers
+ *
+ * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#ifndef __LINUX_VFIO_CXL_PRIV_H
+#define __LINUX_VFIO_CXL_PRIV_H
+
+#include <cxl/cxl.h>
+#include <linux/types.h>
+
+/* CXL device state embedded in vfio_pci_core_device */
+struct vfio_pci_cxl_state {
+	struct cxl_dev_state         cxlds;
+	struct cxl_memdev           *cxlmd;
+	struct cxl_root_decoder     *cxlrd;
+	struct cxl_endpoint_decoder *cxled;
+	resource_size_t              hdm_reg_offset;
+	size_t                       hdm_reg_size;
+	resource_size_t              comp_reg_offset;
+	size_t                       comp_reg_size;
+	u8                           hdm_count;
+	u8                           comp_reg_bar;
+};
+
+#endif /* __LINUX_VFIO_CXL_PRIV_H */
include/linux/vfio_pci_core.h +3 -0
index 1ac86896875c..cd8ed98a82a3 100644
@@ -30,6 +30,8 @@ struct vfio_pci_region;
 struct p2pdma_provider;
 struct dma_buf_phys_vec;
 struct dma_buf_attachment;
+struct vfio_pci_cxl_state;
+
 
 struct vfio_pci_eventfd {
 	struct eventfd_ctx	*ctx;
@@ -138,6 +140,7 @@ struct vfio_pci_core_device {
 	struct mutex		ioeventfds_lock;
 	struct list_head	ioeventfds_list;
 	struct vfio_pci_vf_token	*vf_token;
+	struct vfio_pci_cxl_state *cxl;
 	struct list_head		sriov_pfs_item;
 	struct vfio_pci_core_device	*sriov_pf_core_dev;
 	struct notifier_block	nb;
Signature · 1 line
2.25.1
#8 lore raw

PATCH v2 8/20 vfio/pci: Add CONFIG_VFIO_CXL_CORE and stub CXL hooks

In reply to [email protected]

From: Manish Honap <[email protected]>

Introduce the Kconfig option CONFIG_VFIO_CXL_CORE and the necessary build rules to compile CXL.mem passthrough infrastructure for vendor-specific CXL devices into the vfio-pci-core module. The new option depends on VFIO_PCI_CORE, CXL_BUS and CXL_MEM.

Wire up the detection and cleanup entry-point stubs in vfio_pci_core_register_device() and vfio_pci_core_unregister_device() so that subsequent patches can fill in the CXL-specific logic without touching the vfio-pci-core flow again.

The vfio_cxl_core.c file added here is an empty skeleton; the actual CXL detection and initialisation code is introduced in the following patch to keep this build-system patch reviewable on its own.

---
 drivers/vfio/pci/Kconfig             |  2 ++
 drivers/vfio/pci/Makefile            |  1 +
 drivers/vfio/pci/cxl/Kconfig         |  9 ++++++
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 41 ++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c     |  4 +++
 drivers/vfio/pci/vfio_pci_priv.h     | 14 ++++++++++
 6 files changed, 71 insertions(+)
 create mode 100644 drivers/vfio/pci/cxl/Kconfig
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_core.c
drivers/vfio/pci/Kconfig +2 -0
index 1e82b44bda1a..b981a7c164ca 100644
@@ -68,6 +68,8 @@ source "drivers/vfio/pci/virtio/Kconfig"
 
 source "drivers/vfio/pci/nvgrace-gpu/Kconfig"
 
+source "drivers/vfio/pci/cxl/Kconfig"
+
 source "drivers/vfio/pci/qat/Kconfig"
 
 source "drivers/vfio/pci/xe/Kconfig"
drivers/vfio/pci/Makefile +1 -0
index e0a0757dd1d2..ecb0eacbc089 100644
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o
+vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) += cxl/vfio_cxl_core.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) += vfio_pci_zdev.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) += vfio_pci_dmabuf.o
 obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o
drivers/vfio/pci/cxl/Kconfig new file +9 -0
new file mode 100644
index 000000000000..fad53300fecf
@@ -0,0 +1,9 @@
+config VFIO_CXL_CORE
+	bool "VFIO CXL core"
+	depends on VFIO_PCI_CORE && CXL_BUS && CXL_MEM
+	help
+	    Extends vfio-pci-core with CXL.mem passthrough for vendor-specific
+	    CXL devices (CXL_DEVTYPE_DEVMEM) that implement HDM-D or HDM-DB
+	    decoders without the standard CXL memory expander class code
+	    (PCI_CLASS_MEMORY_CXL).  Covers CXL Type-2 accelerators and
+	    non-class-code Type-3 variants (e.g. compressed memory devices).
drivers/vfio/pci/cxl/vfio_cxl_core.c new file +41 -0

41 lines not on this page · raw message

drivers/vfio/pci/vfio_pci_core.c +4 -0
index 3a11e6f450f7..b7364178e23d 100644
@@ -2181,6 +2181,8 @@ int vfio_pci_core_register_device(struct vfio_pci_core_device *vdev)
 	if (ret)
 		goto out_vf;
 
+	vfio_pci_cxl_detect_and_init(vdev);
+
 	vfio_pci_probe_power_state(vdev);
 
 	/*
@@ -2224,6 +2226,8 @@ void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev)
 	vfio_pci_vf_uninit(vdev);
 	vfio_pci_vga_uninit(vdev);
 
+	vfio_pci_cxl_cleanup(vdev);
+
 	if (!disable_idle_d3)
 		pm_runtime_get_noresume(&vdev->pdev->dev);
 
drivers/vfio/pci/vfio_pci_priv.h +14 -0
index 27ac280f00b9..d7df5538dcde 100644
@@ -133,4 +133,18 @@ static inline void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev,
 }
 #endif
 
+#if IS_ENABLED(CONFIG_VFIO_CXL_CORE)
+
+void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev);
+void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev);
+
+#else
+
+static inline void
+vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev) { }
+static inline void
+vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev) { }
+
+#endif /* CONFIG_VFIO_CXL_CORE */
+
 #endif
Signature · 1 line
2.25.1
#9 lore raw

PATCH v2 9/20 vfio/cxl: Detect CXL DVSEC and probe HDM block

In reply to [email protected]

From: Manish Honap <[email protected]>

Detect a vendor-specific CXL device at vfio-pci bind time and probe its HDM decoder register block.

vfio_cxl_create_device_state() allocates per-device state via devm and reads MEM_CAPABLE and CACHE_CAPABLE from the CXL DVSEC.

vfio_cxl_setup_regs() locates the component register block, temporarily maps it, calls cxl_probe_component_regs() to find the HDM block, then releases the mapping.

vfio_pci_cxl_detect_and_init() chains these two steps. If either fails, vdev->cxl stays NULL and the device falls back to plain vfio-pci.

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 217 +++++++++++++++++++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_priv.h |  12 ++
 2 files changed, 229 insertions(+)
drivers/vfio/pci/cxl/vfio_cxl_core.c +217 -0
index d12afec82ecd..b1c7603590b5 100644
@@ -21,6 +21,158 @@
 #include "../vfio_pci_priv.h"
 #include "vfio_cxl_priv.h"
 
+/*
+ * vfio_cxl_create_device_state - Allocate and validate CXL device state
+ *
+ * Returns a pointer to the allocated vfio_pci_cxl_state on success, or
+ * ERR_PTR on failure.  The allocation uses devm; the caller must call
+ * devm_kfree(&pdev->dev, cxl) on any subsequent setup failure to release
+ * the resource before device unbind.  Using devm_kfree() to undo a devm
+ * allocation early is explicitly supported by the devres API.
+ *
+ * The caller assigns vdev->cxl only after all setup steps succeed, preventing
+ * partially-initialised state from being visible through vdev->cxl on any
+ * failure path.
+ */
+static struct vfio_pci_cxl_state *
+vfio_cxl_create_device_state(struct pci_dev *pdev, u16 dvsec)
+{
+	struct vfio_pci_cxl_state *cxl;
+	u16 cap_word;
+	u32 hdr1;
+
+	/* Freed automatically when pdev->dev is released. */
+	cxl = devm_cxl_dev_state_create(&pdev->dev,
+					CXL_DEVTYPE_DEVMEM,
+					pdev->dev.id, dvsec,
+					struct vfio_pci_cxl_state,
+					cxlds, false);
+	if (!cxl)
+		return ERR_PTR(-ENOMEM);
+
+	pci_read_config_dword(pdev, dvsec + PCI_DVSEC_HEADER1, &hdr1);
+	cxl->dvsec_len = PCI_DVSEC_HEADER1_LEN(hdr1);
+
+	pci_read_config_word(pdev, dvsec + CXL_DVSEC_CAPABILITY_OFFSET,
+			     &cap_word);
+
+	/*
+	 * Only handle vendor devices (class != 0x0502) with Mem_Capable set.
+	 * CACHE_CAPABLE is forwarded to the VMM so it knows whether a WBI
+	 * sequence is needed before FLR.
+	 */
+	if (!FIELD_GET(CXL_DVSEC_MEM_CAPABLE, cap_word) ||
+	    (pdev->class >> 8) == PCI_CLASS_MEMORY_CXL) {
+		devm_kfree(&pdev->dev, cxl);
+		return ERR_PTR(-ENODEV);
+	}
+
+	cxl->cache_capable = FIELD_GET(CXL_DVSEC_CACHE_CAPABLE, cap_word);
+
+	return cxl;
+}
+
+static int vfio_cxl_setup_regs(struct vfio_pci_core_device *vdev,
+			       struct vfio_pci_cxl_state *cxl)
+{
+	struct cxl_register_map *map = &cxl->cxlds.reg_map;
+	resource_size_t offset, bar_offset, size;
+	struct pci_dev *pdev = vdev->pdev;
+	void __iomem *base;
+	int ret;
+	u8 count;
+	u8 bar;
+
+	if (WARN_ON_ONCE(!pci_is_enabled(pdev)))
+		return -EINVAL;
+
+	/* Find component register block via Register Locator DVSEC */
+	ret = cxl_find_regblock(pdev, CXL_REGLOC_RBI_COMPONENT, map);
+	if (ret)
+		return ret;
+
+	/*
+	 * Request the region and map.  This is a transient mapping
+	 * used only to probe register capabilities; released immediately
+	 * after cxl_probe_component_regs() returns.
+	 */
+	if (!request_mem_region(map->resource, map->max_size, "vfio-cxl-probe"))
+		return -EBUSY;
+
+	base = ioremap(map->resource, map->max_size);
+	if (!base) {
+		ret = -ENOMEM;
+		goto failed_release;
+	}
+
+	/* Probe component register capabilities */
+	cxl_probe_component_regs(&pdev->dev, base, &map->component_map);
+
+	/* Check if HDM decoder was found */
+	if (!map->component_map.hdm_decoder.valid) {
+		ret = -ENODEV;
+		goto failed_unmap;
+	}
+
+	pci_dbg(pdev, "vfio_cxl: HDM decoder at offset=0x%lx, size=0x%lx\n",
+		map->component_map.hdm_decoder.offset,
+		map->component_map.hdm_decoder.size);
+
+	/* Get HDM register info */
+	ret = cxl_get_hdm_info(&cxl->cxlds, &count, &offset, &size);
+	if (ret)
+		goto failed_unmap;
+
+	if (!count || !size) {
+		ret = -ENODEV;
+		goto failed_unmap;
+	}
+
+	cxl->hdm_count = count;
+	/*
+	 * cxl_get_hdm_info() returns rmap->offset = CXL_CM_OFFSET + <hdm_within_cm>
+	 * (see cxl_probe_component_regs() which does base += CXL_CM_OFFSET before
+	 * reading caps and stores CXL_CM_OFFSET + cap_ptr as the offset).
+	 * Subtract CXL_CM_OFFSET so hdm_reg_offset is relative to the CXL.mem
+	 * register area start, which is where comp_reg_virt[0] is anchored.
+	 * The physical BAR address for hdm_iobase is recovered by adding
+	 * CXL_CM_OFFSET back in vfio_cxl_setup_virt_regs().
+	 */
+	cxl->hdm_reg_offset = offset - CXL_CM_OFFSET;
+	cxl->hdm_reg_size = size;
+
+	ret = cxl_regblock_get_bar_info(map, &bar, &bar_offset);
+	if (ret)
+		goto failed_unmap;
+
+	cxl->comp_reg_bar = bar;
+	cxl->comp_reg_offset = bar_offset;
+	cxl->comp_reg_size = CXL_COMPONENT_REG_BLOCK_SIZE;
+
+	iounmap(base);
+	release_mem_region(map->resource, map->max_size);
+
+	return 0;
+
+failed_unmap:
+	iounmap(base);
+failed_release:
+	release_mem_region(map->resource, map->max_size);
+
+	return ret;
+}
+
+/*
+ * Free CXL state early on probe failure.  devm_kfree() on a live devres
+ * allocation removes it from the list immediately, so the normal devres
+ * teardown at unbind time won't double-free it.
+ */
+static void vfio_cxl_dev_state_free(struct pci_dev *pdev,
+				    struct vfio_pci_cxl_state *cxl)
+{
+	devm_kfree(&pdev->dev, cxl);
+}
+
 /**
  * vfio_pci_cxl_detect_and_init - Detect and initialize a vendor-specific
  *                                CXL.mem device
@@ -32,10 +184,75 @@
  */
 void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev)
 {
+	struct pci_dev *pdev = vdev->pdev;
+	struct vfio_pci_cxl_state *cxl;
+	u16 dvsec;
+	int ret;
+
+	if (!pcie_is_cxl(pdev))
+		return;
+
+	dvsec = pci_find_dvsec_capability(pdev,
+					  PCI_VENDOR_ID_CXL,
+					  PCI_DVSEC_CXL_DEVICE);
+	if (!dvsec)
+		return;
+
+	/*
+	 * CXL DVSEC found: any failure from here is a hard probe error on
+	 * a confirmed CXL-capable device, not a silent non-CXL fallback.
+	 * Warn the operator so misconfiguration is visible.
+	 */
+	cxl = vfio_cxl_create_device_state(pdev, dvsec);
+	if (IS_ERR(cxl)) {
+		if (PTR_ERR(cxl) != -ENODEV)
+			pci_warn(pdev,
+				 "vfio-cxl: CXL device state allocation failed: %ld\n",
+				 PTR_ERR(cxl));
+		return;
+	}
+
+	/*
+	 * Required for ioremap of the component register block and
+	 * calls to cxl_probe_component_regs().
+	 */
+	ret = pci_enable_device_mem(pdev);
+	if (ret) {
+		pci_warn(pdev,
+			 "vfio-cxl: pci_enable_device_mem failed: %d\n", ret);
+		goto free_cxl;
+	}
+
+	ret = vfio_cxl_setup_regs(vdev, cxl);
+	if (ret) {
+		pci_warn(pdev,
+			 "vfio-cxl: HDM register probing failed: %d\n", ret);
+		pci_disable_device(pdev);
+		goto free_cxl;
+	}
+
+	pci_disable_device(pdev);
+
+	/*
+	 * Register probing succeeded.  Assign vdev->cxl now so that
+	 * all subsequent helpers can access state via vdev->cxl.
+	 * All failure paths below clear vdev->cxl before calling
+	 * vfio_cxl_dev_state_free().
+	 */
+	vdev->cxl = cxl;
+
+	return;
+
+free_cxl:
+	vfio_cxl_dev_state_free(pdev, cxl);
 }
 
 void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev)
 {
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+
+	if (!cxl)
+		return;
 }
 
 MODULE_IMPORT_NS("CXL");
drivers/vfio/pci/cxl/vfio_cxl_priv.h +12 -0
index 4cecc25db410..54b1f6d885aa 100644
@@ -21,8 +21,20 @@ struct vfio_pci_cxl_state {
 	size_t                       hdm_reg_size;
 	resource_size_t              comp_reg_offset;
 	size_t                       comp_reg_size;
+	u16                          dvsec_len;
 	u8                           hdm_count;
 	u8                           comp_reg_bar;
+	bool                         cache_capable;
 };
 
+/*
+ * CXL DVSEC for CXL Devices - register offsets within the DVSEC
+ * (CXL 4.0 8.1.3).
+ * Offsets are relative to the DVSEC capability base (cxl->dvsec).
+ */
+#define CXL_DVSEC_CAPABILITY_OFFSET 0xa
+#define CXL_DVSEC_MEM_CAPABLE	    BIT(2)
+/* CXL DVSEC Capability register bit 0: device supports CXL.cache (HDM-DB) */
+#define CXL_DVSEC_CACHE_CAPABLE	    BIT(0)
+
 #endif /* __LINUX_VFIO_CXL_PRIV_H */
Signature · 1 line
2.25.1
#10 lore raw

PATCH v2 10/20 vfio/pci: Export config access helpers

In reply to [email protected]

From: Manish Honap <[email protected]>

Promote vfio_raw_config_write() and vfio_raw_config_read() to non-static so that the CXL DVSEC write handler in the next patch can call them.

---
 drivers/vfio/pci/vfio_pci_config.c | 12 ++++++------
 drivers/vfio/pci/vfio_pci_priv.h   |  8 ++++++++
 2 files changed, 14 insertions(+), 6 deletions(-)
drivers/vfio/pci/vfio_pci_config.c +6 -6
index dc4e510e6e1b..79aaf270adb2 100644
@@ -270,9 +270,9 @@ static int vfio_direct_config_read(struct vfio_pci_core_device *vdev, int pos,
 }
 
 /* Raw access skips any kind of virtualization */
-static int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int pos,
-				 int count, struct perm_bits *perm,
-				 int offset, __le32 val)
+int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int pos,
+			  int count, struct perm_bits *perm,
+			  int offset, __le32 val)
 {
 	int ret;
 
@@ -283,9 +283,9 @@ static int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int pos,
 	return count;
 }
 
-static int vfio_raw_config_read(struct vfio_pci_core_device *vdev, int pos,
-				int count, struct perm_bits *perm,
-				int offset, __le32 *val)
+int vfio_raw_config_read(struct vfio_pci_core_device *vdev, int pos,
+			 int count, struct perm_bits *perm,
+			 int offset, __le32 *val)
 {
 	int ret;
 
drivers/vfio/pci/vfio_pci_priv.h +8 -0
index d7df5538dcde..1082ba43bafe 100644
@@ -37,6 +37,14 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags,
 ssize_t vfio_pci_config_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 			   size_t count, loff_t *ppos, bool iswrite);
 
+int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int pos,
+			  int count, struct perm_bits *perm,
+			  int offset, __le32 val);
+
+int vfio_raw_config_read(struct vfio_pci_core_device *vdev, int pos,
+			 int count, struct perm_bits *perm,
+			 int offset, __le32 *val);
+
 ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 			size_t count, loff_t *ppos, bool iswrite);
 
Signature · 1 line
2.25.1
#11 lore raw

PATCH v2 11/20 vfio/cxl: Introduce HDM decoder register emulation framework

In reply to [email protected]

From: Manish Honap <[email protected]>

Add HDM decoder register emulation for CXL devices assigned to a guest.

New file vfio_cxl_emu.c allocates comp_reg_virt[] covering the full component register block (CXL_COMPONENT_REG_BLOCK_SIZE), snapshots it from MMIO after probe, and registers a VFIO device region (VFIO_REGION_SUBTYPE_CXL_COMP_REGS) with read/write ops but no mmap, so every access hits the emulated buffer and write dispatchers.

vfio_cxl_setup_virt_regs() is called from the tail of vfio_cxl_setup_regs(); vfio_cxl_clean_virt_regs() runs on cleanup.

HDM decoder register defines come from include/uapi/cxl/cxl_regs.h. Bits with no hardware equivalent stay in vfio_cxl_priv.h.

hdm_decoder_n_ctrl_write() allows the guest to clear the LOCK bit. A firmware-committed decoder arrives with LOCK=1; the guest driver must clear it before reprogramming BASE and SIZE with the VM's GPA. Such a write clears the bit in the shadow while preserving all other fields.

---
 drivers/vfio/pci/Makefile            |   2 +-
 drivers/vfio/pci/cxl/vfio_cxl_core.c |   5 +
 drivers/vfio/pci/cxl/vfio_cxl_emu.c  | 433 +++++++++++++++++++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_priv.h |  47 +++
 include/uapi/cxl/cxl_regs.h          |   5 +
 5 files changed, 491 insertions(+), 1 deletion(-)
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_emu.c
drivers/vfio/pci/Makefile +1 -1
index ecb0eacbc089..bef916495eae 100644
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o
-vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) += cxl/vfio_cxl_core.o
+vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) += cxl/vfio_cxl_core.o cxl/vfio_cxl_emu.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) += vfio_pci_zdev.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) += vfio_pci_dmabuf.o
 obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o
drivers/vfio/pci/cxl/vfio_cxl_core.c +5 -0
index b1c7603590b5..0b9e4419cd47 100644
@@ -149,8 +149,11 @@ static int vfio_cxl_setup_regs(struct vfio_pci_core_device *vdev,
 	cxl->comp_reg_offset = bar_offset;
 	cxl->comp_reg_size = CXL_COMPONENT_REG_BLOCK_SIZE;
 
+	ret = vfio_cxl_setup_virt_regs(vdev, cxl, base);
 	iounmap(base);
 	release_mem_region(map->resource, map->max_size);
+	if (ret)
+		return ret;
 
 	return 0;
 
@@ -253,6 +256,8 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev)
 
 	if (!cxl)
 		return;
+
+	vfio_cxl_clean_virt_regs(cxl);
 }
 
 MODULE_IMPORT_NS("CXL");
drivers/vfio/pci/cxl/vfio_cxl_emu.c new file +433 -0

433 lines not on this page · raw message

drivers/vfio/pci/cxl/vfio_cxl_priv.h +47 -0

63 lines not on this page · raw message

include/uapi/cxl/cxl_regs.h +5 -0
index 1a48a3805f52..b6fcae91d216 100644
@@ -33,8 +33,13 @@
 #define   CXL_HDM_DECODER_TARGET_COUNT_MASK __GENMASK(7, 4)
 #define   CXL_HDM_DECODER_INTERLEAVE_11_8 _BITUL(8)
 #define   CXL_HDM_DECODER_INTERLEAVE_14_12 _BITUL(9)
+#define   CXL_HDM_DECODER_POISON_ON_DECODE_ERR _BITUL(10)
 #define   CXL_HDM_DECODER_INTERLEAVE_3_6_12_WAY _BITUL(11)
 #define   CXL_HDM_DECODER_INTERLEAVE_16_WAY _BITUL(12)
+#define   CXL_HDM_DECODER_UIO_CAPABLE _BITUL(13)
+#define   CXL_HDM_DECODER_UIO_COUNT_MASK __GENMASK(19, 16)
+#define   CXL_HDM_DECODER_MEMDATA_NXM _BITUL(20)
+#define   CXL_HDM_DECODER_COHERENCY_MODELS_MASK    __GENMASK(22, 21)
 #define CXL_HDM_DECODER_CTRL_OFFSET 0x4
 #define   CXL_HDM_DECODER_ENABLE _BITUL(1)
 #define CXL_HDM_DECODER0_BASE_LOW_OFFSET(i) (0x20 * (i) + 0x10)
Signature · 1 line
2.25.1
#12 lore raw

PATCH v2 12/20 vfio/cxl: Wait for HDM ranges and create memdev

In reply to [email protected]

From: Manish Honap <[email protected]>

After HDM registers are mapped, call cxl_await_range_active() so we only proceed when DVSEC ranges report active without touching the memdev register group Type-2 may lack.

Re-snapshot component regs (vfio_cxl_reinit_comp_regs) once MEM_ACTIVE so firmware final SIZE_HIGH etc. land in comp_reg_virt.

Read committed decoder size from hardware, set capacity via cxl_set_capacity(), and devm_cxl_add_memdev().

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 56 ++++++++++++++++++++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_emu.c  | 42 +++++++++++++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_priv.h |  4 ++
 3 files changed, 102 insertions(+)
drivers/vfio/pci/cxl/vfio_cxl_core.c +56 -0
index 0b9e4419cd47..02755265d530 100644
@@ -165,6 +165,22 @@ static int vfio_cxl_setup_regs(struct vfio_pci_core_device *vdev,
 	return ret;
 }
 
+static int vfio_cxl_create_memdev(struct vfio_pci_cxl_state *cxl,
+				  resource_size_t capacity)
+{
+	int ret;
+
+	ret = cxl_set_capacity(&cxl->cxlds, capacity);
+	if (ret)
+		return ret;
+
+	cxl->cxlmd = devm_cxl_add_memdev(&cxl->cxlds, NULL);
+	if (IS_ERR(cxl->cxlmd))
+		return PTR_ERR(cxl->cxlmd);
+
+	return 0;
+}
+
 /*
  * Free CXL state early on probe failure.  devm_kfree() on a live devres
  * allocation removes it from the list immediately, so the normal devres
@@ -189,6 +205,7 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev)
 {
 	struct pci_dev *pdev = vdev->pdev;
 	struct vfio_pci_cxl_state *cxl;
+	resource_size_t capacity = 0;
 	u16 dvsec;
 	int ret;
 
@@ -234,8 +251,44 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev)
 		goto free_cxl;
 	}
 
+	cxl->cxlds.media_ready = !cxl_await_range_active(&cxl->cxlds);
+	if (!cxl->cxlds.media_ready) {
+		pci_warn(pdev, "CXL media not ready\n");
+		pci_disable_device(pdev);
+		goto regs_failed;
+	}
+
+	/*
+	 * Take the single authoritative HDM decoder snapshot now that
+	 * MEM_ACTIVE is confirmed and BAR memory is still enabled.  Using
+	 * readl() per-dword ensures correct MMIO serialisation and captures
+	 * the final firmware-written values for all fields including SIZE_HIGH,
+	 * which firmware commits to the BAR at MEM_ACTIVE time.
+	 */
+	vfio_cxl_reinit_comp_regs(cxl);
+
 	pci_disable_device(pdev);
 
+	capacity = vfio_cxl_read_committed_decoder_size(vdev, cxl);
+	if (capacity == 0) {
+		/*
+		 * TODO: Add handling for devices which do not have
+		 * firmware pre-committed decoders
+		 */
+		pci_info(pdev, "Uncommitted region size must be configured via sysfs before bind\n");
+		goto regs_failed;
+	}
+
+	cxl->dpa_size = capacity;
+
+	pci_dbg(pdev, "Device capacity: %llu MB\n", capacity >> 20);
+
+	ret = vfio_cxl_create_memdev(cxl, capacity);
+	if (ret) {
+		pci_warn(pdev, "Failed to create memdev\n");
+		goto regs_failed;
+	}
+
 	/*
 	 * Register probing succeeded.  Assign vdev->cxl now so that
 	 * all subsequent helpers can access state via vdev->cxl.
@@ -246,6 +299,9 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev)
 
 	return;
 
+regs_failed:
+	vfio_cxl_clean_virt_regs(cxl);
+
 free_cxl:
 	vfio_cxl_dev_state_free(pdev, cxl);
 }
drivers/vfio/pci/cxl/vfio_cxl_emu.c +42 -0

48 lines not on this page · raw message

drivers/vfio/pci/cxl/vfio_cxl_priv.h +4 -0
index 463a55062144..6359ad260bde 100644
@@ -22,6 +22,7 @@ struct vfio_pci_cxl_state {
 	resource_size_t              comp_reg_offset;
 	size_t                       comp_reg_size;
 	__le32                      *comp_reg_virt;
+	size_t                       dpa_size;
 	void __iomem                *hdm_iobase;
 	u16                          dvsec_len;
 	u8                           hdm_count;
@@ -83,5 +84,8 @@ int vfio_cxl_setup_virt_regs(struct vfio_pci_core_device *vdev,
 			     void __iomem *cap_base);
 void vfio_cxl_clean_virt_regs(struct vfio_pci_cxl_state *cxl);
 void vfio_cxl_reinit_comp_regs(struct vfio_pci_cxl_state *cxl);
+resource_size_t
+vfio_cxl_read_committed_decoder_size(struct vfio_pci_core_device *vdev,
+				     struct vfio_pci_cxl_state *cxl);
 
 #endif /* __LINUX_VFIO_CXL_PRIV_H */
Signature · 1 line
2.25.1
#13 lore raw

PATCH v2 13/20 vfio/cxl: CXL region management support

In reply to [email protected]

From: Manish Honap <[email protected]>

Region Management makes use of APIs provided by CXL_CORE as below:

CREATE_REGION flow:
1. Validate request (size, decoder availability)
2. Allocate HPA via cxl_get_hpa_freespace()
3. Allocate DPA via cxl_request_dpa()
4. Create region via cxl_create_region() - commits HDM decoder
5. Get HPA range via cxl_get_region_range()

DESTROY_REGION flow:
1. Detach decoder via cxl_decoder_detach()
2. Free DPA via cxl_dpa_free()
3. Release root decoder via cxl_put_root_decoder()

Use DEFINE_FREE scope helpers so error paths unwind cleanly.

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 119 +++++++++++++++++++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_priv.h |   8 ++
 2 files changed, 127 insertions(+)
drivers/vfio/pci/cxl/vfio_cxl_core.c +119 -0
index 02755265d530..30b365b91903 100644
@@ -21,6 +21,13 @@
 #include "../vfio_pci_priv.h"
 #include "vfio_cxl_priv.h"
 
+/*
+ * Scope-based cleanup wrappers for the CXL resource APIs
+ */
+DEFINE_FREE(cxl_put_root_decoder, struct cxl_root_decoder *, if (!IS_ERR_OR_NULL(_T)) cxl_put_root_decoder(_T))
+DEFINE_FREE(cxl_dpa_free, struct cxl_endpoint_decoder *, if (!IS_ERR_OR_NULL(_T)) cxl_dpa_free(_T))
+DEFINE_FREE(cxl_unregister_region, struct cxl_region *, if (!IS_ERR_OR_NULL(_T)) cxl_unregister_region(_T))
+
 /*
  * vfio_cxl_create_device_state - Allocate and validate CXL device state
  *
@@ -165,6 +172,112 @@ static int vfio_cxl_setup_regs(struct vfio_pci_core_device *vdev,
 	return ret;
 }
 
+int vfio_cxl_create_cxl_region(struct vfio_pci_cxl_state *cxl,
+			       resource_size_t size)
+{
+	resource_size_t max_size;
+
+	WARN_ON(cxl->precommitted);
+
+	struct cxl_root_decoder *cxlrd __free(cxl_put_root_decoder) =
+		cxl_get_hpa_freespace(cxl->cxlmd, 1,
+				      CXL_DECODER_F_RAM | CXL_DECODER_F_TYPE2,
+				      &max_size);
+	if (IS_ERR(cxlrd))
+		return PTR_ERR(cxlrd);
+
+	/* Insufficient HPA space; cxlrd freed automatically by __free() */
+	if (max_size < size)
+		return -ENOSPC;
+
+	struct cxl_endpoint_decoder *cxled __free(cxl_dpa_free) =
+		cxl_request_dpa(cxl->cxlmd, CXL_PARTMODE_RAM, size);
+	if (IS_ERR(cxled))
+		return PTR_ERR(cxled);
+
+	struct cxl_region *region __free(cxl_unregister_region) =
+		cxl_create_region(cxlrd, &cxled, 1);
+	if (IS_ERR(region))
+		return PTR_ERR(region);
+
+	/* All operations succeeded; transfer ownership to cxl state */
+	cxl->cxlrd  = no_free_ptr(cxlrd);
+	cxl->cxled  = no_free_ptr(cxled);
+	cxl->region = no_free_ptr(region);
+
+	return 0;
+}
+
+void vfio_cxl_destroy_cxl_region(struct vfio_pci_cxl_state *cxl)
+{
+	if (!cxl->region)
+		return;
+
+	cxl_unregister_region(cxl->region);
+	cxl->region = NULL;
+
+	if (!cxl->precommitted) {
+		cxl_dpa_free(cxl->cxled);
+		cxl_put_root_decoder(cxl->cxlrd);
+	}
+
+	cxl->cxled = NULL;
+	cxl->cxlrd = NULL;
+}
+
+static int vfio_cxl_create_region_helper(struct vfio_pci_core_device *vdev,
+					 struct vfio_pci_cxl_state *cxl,
+					 resource_size_t capacity)
+{
+	struct pci_dev *pdev = vdev->pdev;
+	struct range range;
+	int ret;
+
+	if (cxl->precommitted) {
+		struct cxl_endpoint_decoder *cxled;
+		struct cxl_region *region;
+
+		cxled = cxl_get_committed_decoder(cxl->cxlmd, &region);
+		if (IS_ERR(cxled))
+			return PTR_ERR(cxled);
+		cxl->cxled = cxled;
+		cxl->region = region;
+	} else {
+		ret = vfio_cxl_create_cxl_region(cxl, capacity);
+		if (ret)
+			return ret;
+	}
+
+	if (!cxl->region) {
+		pci_err(pdev, "Failed to create CXL region\n");
+		ret = -ENODEV;
+		goto failed;
+	}
+
+	ret = cxl_get_region_range(cxl->region, &range);
+	if (ret)
+		goto failed;
+
+	cxl->region_hpa = range.start;
+	cxl->region_size = range_len(&range);
+
+	pci_dbg(pdev, "CXL region: HPA 0x%llx size %lu MB\n",
+		cxl->region_hpa, cxl->region_size >> 20);
+
+	return 0;
+
+failed:
+	if (cxl->region) {
+		cxl_unregister_region(cxl->region);
+		cxl->region = NULL;
+	}
+
+	cxl->cxled = NULL;
+	cxl->cxlrd = NULL;
+
+	return ret;
+}
+
 static int vfio_cxl_create_memdev(struct vfio_pci_cxl_state *cxl,
 				  resource_size_t capacity)
 {
@@ -279,6 +392,7 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev)
 		goto regs_failed;
 	}
 
+	cxl->precommitted = true;
 	cxl->dpa_size = capacity;
 
 	pci_dbg(pdev, "Device capacity: %llu MB\n", capacity >> 20);
@@ -289,6 +403,10 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev)
 		goto regs_failed;
 	}
 
+	ret = vfio_cxl_create_region_helper(vdev, cxl, capacity);
+	if (ret)
+		goto regs_failed;
+
 	/*
 	 * Register probing succeeded.  Assign vdev->cxl now so that
 	 * all subsequent helpers can access state via vdev->cxl.
@@ -314,6 +432,7 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev)
 		return;
 
 	vfio_cxl_clean_virt_regs(cxl);
+	vfio_cxl_destroy_cxl_region(cxl);
 }
 
 MODULE_IMPORT_NS("CXL");
drivers/vfio/pci/cxl/vfio_cxl_priv.h +8 -0
index 6359ad260bde..72a0d7d7e183 100644
@@ -17,6 +17,10 @@ struct vfio_pci_cxl_state {
 	struct cxl_memdev           *cxlmd;
 	struct cxl_root_decoder     *cxlrd;
 	struct cxl_endpoint_decoder *cxled;
+	struct cxl_region	    *region;
+	resource_size_t		     region_hpa;
+	size_t			     region_size;
+	void			    *region_vaddr;
 	resource_size_t              hdm_reg_offset;
 	size_t                       hdm_reg_size;
 	resource_size_t              comp_reg_offset;
@@ -28,6 +32,7 @@ struct vfio_pci_cxl_state {
 	u8                           hdm_count;
 	u8                           comp_reg_bar;
 	bool                         cache_capable;
+	bool                         precommitted;
 };
 
 /* Register access sizes */
@@ -87,5 +92,8 @@ void vfio_cxl_reinit_comp_regs(struct vfio_pci_cxl_state *cxl);
 resource_size_t
 vfio_cxl_read_committed_decoder_size(struct vfio_pci_core_device *vdev,
 				     struct vfio_pci_cxl_state *cxl);
+int vfio_cxl_create_cxl_region(struct vfio_pci_cxl_state *cxl,
+			       resource_size_t size);
+void vfio_cxl_destroy_cxl_region(struct vfio_pci_cxl_state *cxl);
 
 #endif /* __LINUX_VFIO_CXL_PRIV_H */
Signature · 1 line
2.25.1
#14 lore raw

PATCH v2 14/20 vfio/cxl: DPA VFIO region with demand fault mmap and reset zap

In reply to [email protected]

From: Manish Honap <[email protected]>

Wire the CXL DPA range up as a VFIO demand-paged region so QEMU can mmap guest device memory directly. Faults call vmf_insert_pfn() to insert one PFN at a time rather than mapping the full range upfront.

CXL region lifecycle:
- The CXL memory region is registered with VFIO layer during vfio_pci_open_device
- mmap() establishes the VMA with vm_ops but inserts no PTEs
- Each guest page fault calls vfio_cxl_region_page_fault() which inserts a single PFN under the memory_lock read side
- On device reset, vfio_cxl_zap_region_locked() sets region_active=false and calls unmap_mapping_range() to invalidate all DPA PTEs atomically while holding memory_lock for writing
- Faults racing with reset see region_active==false and return VM_FAULT_SIGBUS
- vfio_cxl_reactivate_region() restores region_active after successful hardware reset

Also integrate the zap/reactivate calls into vfio_pci_ioctl_reset() so that FLR correctly invalidates DPA mappings and restores them on success.

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 187 +++++++++++++++++++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_emu.c  |   2 +-
 drivers/vfio/pci/cxl/vfio_cxl_priv.h |   3 +
 drivers/vfio/pci/vfio_pci_core.c     |  11 ++
 drivers/vfio/pci/vfio_pci_priv.h     |   6 +
 5 files changed, 208 insertions(+), 1 deletion(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +187 -0
index 30b365b91903..19d3dc205f99 100644
@@ -435,4 +435,191 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev)
 	vfio_cxl_destroy_cxl_region(cxl);
 }
 
+static vm_fault_t vfio_cxl_region_vm_fault(struct vm_fault *vmf)
+{
+	struct vfio_pci_region *region = vmf->vma->vm_private_data;
+	struct vfio_pci_cxl_state *cxl = region->data;
+	unsigned long pgoff;
+	unsigned long pfn;
+
+	if (!READ_ONCE(cxl->region_active))
+		return VM_FAULT_SIGBUS;
+
+	pgoff = vmf->pgoff &
+		((1UL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
+
+	if (pgoff >= (cxl->region_size >> PAGE_SHIFT))
+		return VM_FAULT_SIGBUS;
+
+	pfn = PHYS_PFN(cxl->region_hpa) + pgoff;
+
+	return vmf_insert_pfn(vmf->vma, vmf->address, pfn);
+}
+
+static const struct vm_operations_struct vfio_cxl_region_vm_ops = {
+	.fault = vfio_cxl_region_vm_fault,
+};
+
+static int vfio_cxl_region_mmap(struct vfio_pci_core_device *vdev,
+				struct vfio_pci_region *region,
+				struct vm_area_struct *vma)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+	u64 req_len, pgoff, end;
+
+	if (!(region->flags & VFIO_REGION_INFO_FLAG_MMAP))
+		return -EINVAL;
+
+	if (!(region->flags & VFIO_REGION_INFO_FLAG_READ) &&
+	    (vma->vm_flags & VM_READ))
+		return -EPERM;
+
+	if (!(region->flags & VFIO_REGION_INFO_FLAG_WRITE) &&
+	    (vma->vm_flags & VM_WRITE))
+		return -EPERM;
+
+	pgoff = vma->vm_pgoff &
+		((1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
+
+	if (check_sub_overflow(vma->vm_end, vma->vm_start, &req_len) ||
+	    check_add_overflow(PFN_PHYS(pgoff), req_len, &end))
+		return -EOVERFLOW;
+
+	if (end > cxl->region_size)
+		return -EINVAL;
+
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
+	vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED | VM_IO | VM_PFNMAP |
+		     VM_DONTEXPAND | VM_DONTDUMP);
+
+	vma->vm_ops = &vfio_cxl_region_vm_ops;
+	vma->vm_private_data = region;
+
+	return 0;
+}
+
+/*
+ * vfio_cxl_zap_region_locked - Invalidate all DPA region PTEs.
+ *
+ * Must be called with vdev->memory_lock held for writing.  Sets
+ * region_active=false before zapping so any subsequent I/O to the region
+ * sees the inactive state and returns an error rather than accessing
+ * stale mappings.
+ */
+void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+
+	lockdep_assert_held_write(&vdev->memory_lock);
+
+	if (!cxl)
+		return;
+
+	WRITE_ONCE(cxl->region_active, false);
+}
+
+/*
+ * vfio_cxl_reactivate_region - Re-enable DPA region after successful reset.
+ *
+ * Must be called with vdev->memory_lock held for writing.  Re-reads the
+ * HDM decoder state from hardware (FLR cleared it) and sets region_active
+ * so that subsequent I/O to the region is permitted again.
+ */
+void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+
+	lockdep_assert_held_write(&vdev->memory_lock);
+
+	if (!cxl)
+		return;
+	/*
+	 * Re-initialise the emulated HDM comp_reg_virt[] from hardware.
+	 * After FLR the decoder registers read as zero; mirror that in
+	 * the emulated state so QEMU sees a clean slate.
+	 */
+	vfio_cxl_reinit_comp_regs(cxl);
+
+	/*
+	 * Only re-enable the DPA mmap if the hardware has actually
+	 * re-committed decoder 0 after FLR.  Read the COMMITTED bit from the
+	 * freshly-re-snapshotted comp_reg_virt[] so we check the post-FLR
+	 * hardware state, not stale pre-reset state.
+	 *
+	 * If COMMITTED is 0 (slow firmware re-commit path), leave
+	 * region_active=false.	 Guest faults will return VM_FAULT_SIGBUS
+	 * until the decoder is re-committed and the region is re-enabled.
+	 */
+	if (cxl->precommitted && cxl->comp_reg_virt) {
+		/*
+		 * Read CTRL via the full CXL.mem-relative index: hdm_reg_offset
+		 * (now CXL.mem-relative) plus the within-HDM-block offset.
+		 */
+		u32 ctrl = le32_to_cpu(*hdm_reg_ptr(cxl,
+					    CXL_HDM_DECODER0_CTRL_OFFSET(0)));
+
+		if (ctrl & CXL_HDM_DECODER0_CTRL_COMMITTED)
+			WRITE_ONCE(cxl->region_active, true);
+	}
+}
+
+static ssize_t vfio_cxl_region_rw(struct vfio_pci_core_device *core_dev,
+				  char __user *buf, size_t count, loff_t *ppos,
+				  bool iswrite)
+{
+	unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS;
+	struct vfio_pci_cxl_state *cxl = core_dev->region[i].data;
+	loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
+
+	if (!count || pos >= cxl->region_size)
+		return 0;
+
+	/*
+	 * Guard against access after a failed reset (region_active=false)
+	 * or a release race (region_vaddr=NULL).  Either condition means
+	 * the memremap'd window is no longer valid; touching it would produce
+	 * a Synchronous External Abort.  Return -EIO so the caller gets a
+	 * clean error rather than a kernel oops.
+	 */
+	if (!READ_ONCE(cxl->region_active) || !cxl->region_vaddr)
+		return -EIO;
+
+	count = min(count, (size_t)(cxl->region_size - pos));
+
+	if (iswrite) {
+		if (copy_from_user(cxl->region_vaddr + pos, buf, count))
+			return -EFAULT;
+	} else {
+		if (copy_to_user(buf, cxl->region_vaddr + pos, count))
+			return -EFAULT;
+	}
+
+	return count;
+}
+
+static void vfio_cxl_region_release(struct vfio_pci_core_device *vdev,
+				    struct vfio_pci_region *region)
+{
+	struct vfio_pci_cxl_state *cxl = region->data;
+
+	/*
+	 * Deactivate the region before removing user mappings so that any
+	 * fault handler racing the release returns VM_FAULT_SIGBUS rather
+	 * than inserting a PFN into an unmapped region.
+	 */
+	WRITE_ONCE(cxl->region_active, false);
+
+	if (cxl->region_vaddr) {
+		memunmap(cxl->region_vaddr);
+		cxl->region_vaddr = NULL;
+	}
+}
+
+static const struct vfio_pci_regops vfio_cxl_regops = {
+	.rw		= vfio_cxl_region_rw,
+	.mmap		= vfio_cxl_region_mmap,
+	.release	= vfio_cxl_region_release,
+};
+
 MODULE_IMPORT_NS("CXL");
drivers/vfio/pci/cxl/vfio_cxl_emu.c +1 -1
index 11195e8c21d7..781328a79b43 100644
@@ -33,7 +33,7 @@
  *     +0x1c: (reserved)
  */
 
-static inline __le32 *hdm_reg_ptr(struct vfio_pci_cxl_state *cxl, u32 hdm_off)
+__le32 *hdm_reg_ptr(struct vfio_pci_cxl_state *cxl, u32 hdm_off)
 {
 	/*
 	 * hdm_off is a byte offset within the HDM decoder block.
drivers/vfio/pci/cxl/vfio_cxl_priv.h +3 -0
index 72a0d7d7e183..3458768445af 100644
@@ -33,6 +33,7 @@ struct vfio_pci_cxl_state {
 	u8                           comp_reg_bar;
 	bool                         cache_capable;
 	bool                         precommitted;
+	bool                         region_active;
 };
 
 /* Register access sizes */
@@ -96,4 +97,6 @@ int vfio_cxl_create_cxl_region(struct vfio_pci_cxl_state *cxl,
 			       resource_size_t size);
 void vfio_cxl_destroy_cxl_region(struct vfio_pci_cxl_state *cxl);
 
+__le32 *hdm_reg_ptr(struct vfio_pci_cxl_state *cxl, u32 hdm_off);
+
 #endif /* __LINUX_VFIO_CXL_PRIV_H */
drivers/vfio/pci/vfio_pci_core.c +11 -0
index b7364178e23d..48e0274c19aa 100644
@@ -1223,6 +1223,9 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
 
 	vfio_pci_zap_and_down_write_memory_lock(vdev);
 
+	/* Zap CXL DPA region PTEs before hardware reset clears HDM state */
+	vfio_cxl_zap_region_locked(vdev);
+
 	/*
 	 * This function can be invoked while the power state is non-D0. If
 	 * pci_try_reset_function() has been called while the power state is
@@ -1236,6 +1239,14 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
 
 	vfio_pci_dma_buf_move(vdev, true);
 	ret = pci_try_reset_function(vdev->pdev);
+
+	/*
+	 * Re-enable DPA region if reset succeeded; fault handler will
+	 * re-insert PFNs on next access without requiring a new mmap.
+	 */
+	if (!ret)
+		vfio_cxl_reactivate_region(vdev);
+
 	if (__vfio_pci_memory_enabled(vdev))
 		vfio_pci_dma_buf_move(vdev, false);
 	up_write(&vdev->memory_lock);
drivers/vfio/pci/vfio_pci_priv.h +6 -0
index 1082ba43bafe..726063b6ff70 100644
@@ -145,6 +145,8 @@ static inline void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev,
 
 void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev);
 void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev);
+void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev);
+void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev);
 
 #else
 
@@ -152,6 +154,10 @@ static inline void
 vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev) { }
 static inline void
 vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev) { }
+static inline void
+vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev) { }
+static inline void
+vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev) { }
 
 #endif /* CONFIG_VFIO_CXL_CORE */
 
Signature · 1 line
2.25.1
#15 lore raw

PATCH v2 15/20 vfio/cxl: Virtualize CXL DVSEC config writes

In reply to [email protected]

From: Manish Honap <[email protected]>

CXL devices have CXL DVSEC registers in the configuration space. Many of them affect the behaviors of the devices, e.g. enabling CXL.io/CXL.mem/CXL.cache. However, these configurations are owned by the host and a virtualization policy should be applied when handling the access from the guest.

Introduce the emulation of CXL configuration space to handle the access of the virtual CXL configuration space from the guest.

vfio-pci-core already allocates vdev->vconfig as the authoritative virtual config space shadow. Directly use vdev->vconfig:
- DVSEC reads return data from vdev->vconfig (already populated by vfio_config_init() via vfio_ecap_init())
- DVSEC writes go through new CXL-aware write handlers that update

    vdev->vconfig in place

- The writable DVSEC registers are marked virtual in vdev->pci_config_map

---
 drivers/vfio/pci/Makefile              |   2 +-
 drivers/vfio/pci/cxl/vfio_cxl_config.c | 306 +++++++++++++++++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_core.c   |   4 +-
 drivers/vfio/pci/cxl/vfio_cxl_priv.h   |  43 +++-
 drivers/vfio/pci/vfio_pci_config.c     |  46 +++-
 drivers/vfio/pci/vfio_pci_priv.h       |   3 +
 include/linux/vfio_pci_core.h          |   8 +-
 include/uapi/cxl/cxl_regs.h            |  98 ++++++++
 8 files changed, 498 insertions(+), 12 deletions(-)
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_config.c
drivers/vfio/pci/Makefile +1 -1
index bef916495eae..7c86b7845e8f 100644
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o
-vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) += cxl/vfio_cxl_core.o cxl/vfio_cxl_emu.o
+vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) += cxl/vfio_cxl_core.o cxl/vfio_cxl_emu.o cxl/vfio_cxl_config.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) += vfio_pci_zdev.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) += vfio_pci_dmabuf.o
 obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o
drivers/vfio/pci/cxl/vfio_cxl_config.c new file +306 -0

306 lines not on this page · raw message

drivers/vfio/pci/cxl/vfio_cxl_core.c +2 -2
index 19d3dc205f99..a3ff90b7a22c 100644
@@ -68,13 +68,13 @@ vfio_cxl_create_device_state(struct pci_dev *pdev, u16 dvsec)
 	 * CACHE_CAPABLE is forwarded to the VMM so it knows whether a WBI
 	 * sequence is needed before FLR.
 	 */
-	if (!FIELD_GET(CXL_DVSEC_MEM_CAPABLE, cap_word) ||
+	if (!FIELD_GET(CXL_DVSEC_CAP_MEM_CAPABLE, cap_word) ||
 	    (pdev->class >> 8) == PCI_CLASS_MEMORY_CXL) {
 		devm_kfree(&pdev->dev, cxl);
 		return ERR_PTR(-ENODEV);
 	}
 
-	cxl->cache_capable = FIELD_GET(CXL_DVSEC_CACHE_CAPABLE, cap_word);
+	cxl->cache_capable = FIELD_GET(CXL_DVSEC_CAP_CACHE_CAPABLE, cap_word);
 
 	return cxl;
 }
drivers/vfio/pci/cxl/vfio_cxl_priv.h +36 -7

50 lines not on this page · raw message

drivers/vfio/pci/vfio_pci_config.c +45 -1

58 lines not on this page · raw message

drivers/vfio/pci/vfio_pci_priv.h +3 -0
index 726063b6ff70..96f8361ce6f3 100644
@@ -147,6 +147,7 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev);
 void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev);
 void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev);
 void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev);
+void vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev);
 
 #else
 
@@ -158,6 +159,8 @@ static inline void
 vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev) { }
 static inline void
 vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev) { }
+static inline void
+vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev) { }
 
 #endif /* CONFIG_VFIO_CXL_CORE */
 
include/linux/vfio_pci_core.h +7 -1
index cd8ed98a82a3..aa159d0c8da7 100644
@@ -31,7 +31,7 @@ struct p2pdma_provider;
 struct dma_buf_phys_vec;
 struct dma_buf_attachment;
 struct vfio_pci_cxl_state;
-
+struct perm_bits;
 
 struct vfio_pci_eventfd {
 	struct eventfd_ctx	*ctx;
@@ -141,6 +141,12 @@ struct vfio_pci_core_device {
 	struct list_head	ioeventfds_list;
 	struct vfio_pci_vf_token	*vf_token;
 	struct vfio_pci_cxl_state *cxl;
+	int (*dvsec_readfn)(struct vfio_pci_core_device *vdev, int pos,
+			    int count, struct perm_bits *perm,
+			    int offset, __le32 *val);
+	int (*dvsec_writefn)(struct vfio_pci_core_device *vdev, int pos,
+			     int count, struct perm_bits *perm,
+			     int offset, __le32 val);
 	struct list_head		sriov_pfs_item;
 	struct vfio_pci_core_device	*sriov_pf_core_dev;
 	struct notifier_block	nb;
include/uapi/cxl/cxl_regs.h +98 -0

102 lines not on this page · raw message

Signature · 1 line
2.25.1
#16 lore raw

PATCH v2 16/20 vfio/cxl: Register regions with VFIO layer

In reply to [email protected]

From: Manish Honap <[email protected]>

Register the DPA and component register region with VFIO layer. Region indices for both these regions are cached for quick lookup.

vfio_cxl_register_cxl_region()
- memremap(WB) the region HPA (treat CXL.mem as RAM, not MMIO)
- Register VFIO_REGION_SUBTYPE_CXL
- Records dpa_region_idx.

vfio_cxl_register_comp_regs_region()
- Registers VFIO_REGION_SUBTYPE_CXL_COMP_REGS with size hdm_reg_offset + hdm_reg_size
- Records comp_reg_region_idx.

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 98 +++++++++++++++++++++++++++-
 drivers/vfio/pci/cxl/vfio_cxl_emu.c  | 34 ++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_priv.h |  2 +
 drivers/vfio/pci/vfio_pci.c          | 23 +++++++
 drivers/vfio/pci/vfio_pci_priv.h     | 11 ++++
 5 files changed, 167 insertions(+), 1 deletion(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +97 -1
index a3ff90b7a22c..b38a04301660 100644
@@ -75,6 +75,8 @@ vfio_cxl_create_device_state(struct pci_dev *pdev, u16 dvsec)
 	}
 
 	cxl->cache_capable = FIELD_GET(CXL_DVSEC_CAP_CACHE_CAPABLE, cap_word);
+	cxl->dpa_region_idx = -1;
+	cxl->comp_reg_region_idx = -1;
 
 	return cxl;
 }
@@ -509,14 +511,19 @@ static int vfio_cxl_region_mmap(struct vfio_pci_core_device *vdev,
  */
 void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev)
 {
+	struct vfio_device *core_vdev = &vdev->vdev;
 	struct vfio_pci_cxl_state *cxl = vdev->cxl;
 
 	lockdep_assert_held_write(&vdev->memory_lock);
 
-	if (!cxl)
+	if (!cxl || cxl->dpa_region_idx < 0)
 		return;
 
 	WRITE_ONCE(cxl->region_active, false);
+	unmap_mapping_range(core_vdev->inode->i_mapping,
+			    VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_NUM_REGIONS +
+						     cxl->dpa_region_idx),
+			    cxl->region_size, true);
 }
 
 /*
@@ -601,6 +608,7 @@ static ssize_t vfio_cxl_region_rw(struct vfio_pci_core_device *core_dev,
 static void vfio_cxl_region_release(struct vfio_pci_core_device *vdev,
 				    struct vfio_pci_region *region)
 {
+	struct vfio_device *core_vdev = &vdev->vdev;
 	struct vfio_pci_cxl_state *cxl = region->data;
 
 	/*
@@ -610,6 +618,16 @@ static void vfio_cxl_region_release(struct vfio_pci_core_device *vdev,
 	 */
 	WRITE_ONCE(cxl->region_active, false);
 
+	/*
+	 * Remove all user mappings of the DPA region while the device is
+	 * still alive.
+	 */
+	if (cxl->dpa_region_idx >= 0)
+		unmap_mapping_range(core_vdev->inode->i_mapping,
+			    VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_NUM_REGIONS +
+						     cxl->dpa_region_idx),
+				    cxl->region_size, true);
+
 	if (cxl->region_vaddr) {
 		memunmap(cxl->region_vaddr);
 		cxl->region_vaddr = NULL;
@@ -622,4 +640,82 @@ static const struct vfio_pci_regops vfio_cxl_regops = {
 	.release	= vfio_cxl_region_release,
 };
 
+int vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+	u32 flags;
+	int ret;
+
+	if (!cxl)
+		return -ENODEV;
+
+	if (!cxl->region || cxl->region_vaddr)
+		return -ENODEV;
+
+	/*
+	 * CXL device memory is RAM, not MMIO.  Use memremap() rather than
+	 * ioremap_cache() so the correct memory-mapping API is used.
+	 * The WB attribute matches the cache-coherent nature of CXL.mem.
+	 */
+	cxl->region_vaddr = memremap(cxl->region_hpa, cxl->region_size,
+				     MEMREMAP_WB);
+	if (!cxl->region_vaddr)
+		return -ENOMEM;
+
+	flags = VFIO_REGION_INFO_FLAG_READ |
+		VFIO_REGION_INFO_FLAG_WRITE |
+		VFIO_REGION_INFO_FLAG_MMAP;
+
+	ret = vfio_pci_core_register_dev_region(vdev,
+						PCI_VENDOR_ID_CXL |
+						VFIO_REGION_TYPE_PCI_VENDOR_TYPE,
+						VFIO_REGION_SUBTYPE_CXL,
+						&vfio_cxl_regops,
+						cxl->region_size, flags,
+						cxl);
+	if (ret) {
+		memunmap(cxl->region_vaddr);
+		cxl->region_vaddr = NULL;
+		return ret;
+	}
+
+	/*
+	 * Cache the vdev->region[] index before activating the region.
+	 * vfio_pci_core_register_dev_region() placed the new entry at
+	 * vdev->region[num_regions - 1] and incremented num_regions.
+	 * vfio_cxl_zap_region_locked() uses this to avoid scanning
+	 * vdev->region[] on every FLR.
+	 */
+	cxl->dpa_region_idx = vdev->num_regions - 1;
+
+	vfio_cxl_reinit_comp_regs(cxl);
+
+	WRITE_ONCE(cxl->region_active, true);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_cxl_register_cxl_region);
+
+/**
+ * vfio_cxl_unregister_cxl_region - Undo vfio_cxl_register_cxl_region()
+ * @vdev: VFIO PCI device
+ *
+ * Marks the DPA region inactive and resets dpa_region_idx.
+ * Does NOT touch CXL subsystem state (cxl->region, cxl->cxled, cxl->cxlrd).
+ * The caller must call vfio_cxl_destroy_cxl_region() separately to release
+ * those objects.
+ */
+void vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+
+	if (!cxl || cxl->dpa_region_idx < 0)
+		return;
+
+	WRITE_ONCE(cxl->region_active, false);
+
+	cxl->dpa_region_idx = -1;
+}
+EXPORT_SYMBOL_GPL(vfio_cxl_unregister_cxl_region);
+
 MODULE_IMPORT_NS("CXL");
drivers/vfio/pci/cxl/vfio_cxl_emu.c +34 -0
index 781328a79b43..50d3718b101d 100644
@@ -473,3 +473,37 @@ void vfio_cxl_clean_virt_regs(struct vfio_pci_cxl_state *cxl)
 	kfree(cxl->comp_reg_virt);
 	cxl->comp_reg_virt = NULL;
 }
+
+/*
+ * vfio_cxl_register_comp_regs_region - Register the COMP_REGS device region.
+ *
+ * Exposes the emulated HDM decoder register state as a VFIO device region
+ * with type VFIO_REGION_SUBTYPE_CXL_COMP_REGS.	 QEMU attaches a
+ * notify_change callback to this region to intercept HDM COMMIT writes
+ * and map the DPA MemoryRegion at the appropriate GPA.
+ *
+ * The region is read+write only (no mmap) to ensure all accesses pass
+ * through comp_regs_dispatch_write() for proper bit-field enforcement.
+ */
+int vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+	u32 flags = VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE;
+	int ret;
+
+	if (!cxl || !cxl->comp_reg_virt)
+		return -ENODEV;
+
+	ret = vfio_pci_core_register_dev_region(vdev,
+						PCI_VENDOR_ID_CXL |
+						VFIO_REGION_TYPE_PCI_VENDOR_TYPE,
+						VFIO_REGION_SUBTYPE_CXL_COMP_REGS,
+						&vfio_cxl_comp_regs_ops,
+						cxl->hdm_reg_offset +
+						cxl->hdm_reg_size, flags, cxl);
+	if (!ret)
+		cxl->comp_reg_region_idx = vdev->num_regions - 1;
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(vfio_cxl_register_comp_regs_region);
drivers/vfio/pci/cxl/vfio_cxl_priv.h +2 -0
index b86ee691d050..b884689a1226 100644
@@ -28,6 +28,8 @@ struct vfio_pci_cxl_state {
 	__le32                      *comp_reg_virt;
 	size_t                       dpa_size;
 	void __iomem                *hdm_iobase;
+	int                          dpa_region_idx;
+	int                          comp_reg_region_idx;
 	u16                          dvsec_len;
 	u8                           hdm_count;
 	u8                           comp_reg_bar;
drivers/vfio/pci/vfio_pci.c +23 -0
index 0c771064c0b8..22cf9ea831f9 100644
@@ -120,6 +120,29 @@ static int vfio_pci_open_device(struct vfio_device *core_vdev)
 		}
 	}
 
+	if (vdev->cxl) {
+		/*
+		 * pci_config_map and vconfig are valid now (allocated by
+		 * vfio_config_init() inside vfio_pci_core_enable() above).
+		 */
+		vfio_cxl_setup_dvsec_perms(vdev);
+
+		ret = vfio_cxl_register_cxl_region(vdev);
+		if (ret) {
+			pci_warn(pdev, "Failed to setup CXL region\n");
+			vfio_pci_core_disable(vdev);
+			return ret;
+		}
+
+		ret = vfio_cxl_register_comp_regs_region(vdev);
+		if (ret) {
+			pci_warn(pdev, "Failed to register COMP_REGS region\n");
+			vfio_cxl_unregister_cxl_region(vdev);
+			vfio_pci_core_disable(vdev);
+			return ret;
+		}
+	}
+
 	vfio_pci_core_finish_enable(vdev);
 
 	return 0;
drivers/vfio/pci/vfio_pci_priv.h +11 -0
index 96f8361ce6f3..ae0091d5096c 100644
@@ -148,6 +148,9 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev);
 void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev);
 void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev);
 void vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev);
+int vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev);
+void vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev);
+int  vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev);
 
 #else
 
@@ -161,6 +164,14 @@ static inline void
 vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev) { }
 static inline void
 vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev) { }
+static inline int
+vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev)
+{ return 0; }
+static inline void
+vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev) { }
+static inline int
+vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev)
+{ return 0; }
 
 #endif /* CONFIG_VFIO_CXL_CORE */
 
Signature · 1 line
2.25.1
#17 lore raw

Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

Quoted 23 lines · mhonap@ wrote:

From: Manish Honap <[email protected]>

Register the DPA and component register region with VFIO layer. Region indices for both these regions are cached for quick lookup.

vfio_cxl_register_cxl_region()
- memremap(WB) the region HPA (treat CXL.mem as RAM, not MMIO)
- Register VFIO_REGION_SUBTYPE_CXL
- Records dpa_region_idx.

vfio_cxl_register_comp_regs_region()
- Registers VFIO_REGION_SUBTYPE_CXL_COMP_REGS with size hdm_reg_offset + hdm_reg_size
- Records comp_reg_region_idx.

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 98 +++++++++++++++++++++++++++-
 drivers/vfio/pci/cxl/vfio_cxl_emu.c  | 34 ++++++++++
 drivers/vfio/pci/cxl/vfio_cxl_priv.h |  2 +
 drivers/vfio/pci/vfio_pci.c          | 23 +++++++
 drivers/vfio/pci/vfio_pci_priv.h     | 11 ++++
 5 files changed, 167 insertions(+), 1 deletion(-)

[..]

Quoted 8 lines
drivers/vfio/pci/cxl/vfio_cxl_core.c +4 -0
@@ -622,4 +640,82 @@ static const struct vfio_pci_regops vfio_cxl_regops = {
 	.release	= vfio_cxl_region_release,
 };
 
+int vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+	u32 flags;

drivers/vfio/pci/cxl/vfio_cxl_core.c static const struct vfio_pci_regops vfio_cxl_regops = { · line 656

Quoted 10 lines
drivers/vfio/pci/cxl/vfio_cxl_core.c +10 -0
+	int ret;
+
+	if (!cxl)
+		return -ENODEV;
+
+	if (!cxl->region || cxl->region_vaddr)
+		return -ENODEV;
+
+	/*
+	 * CXL device memory is RAM, not MMIO.  Use memremap() rather than

Right, CXL.mem is RAM, not MMIO, so I question why is this not being mapped via a RAM mechanism? Can you explain a bit more about why reusing vfio MMIO mapping mechanisms is suitable here and what happens when we get to questions like guest_memfd integration, page conversions, and large page mapping support? I am looking at proposals like Gregory's as a way to get full MM semantics for CXL.mem but without the memory being consumed for other purposes [1].

Maybe this is the start of the conversation for something simple, but I would actually prefer uncached ioremap() if only to avoid all the coherence management and suitability of MMIO mechanisms questions. ioremap() being a poor fit for CXL is a "feature" to force the "find the right, something better" direction.

[1]: https://lore.kernel.org/all/[email protected]

#18 lore raw

Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

Quoted 8 lines · Dan Williams

drivers/vfio/pci/cxl/vfio_cxl_core.c static const struct vfio_pci_regops vfio_cxl_regops = { · line 656

Quoted 2 lines
drivers/vfio/pci/cxl/vfio_cxl_core.c +2 -0
+	/*
+	 * CXL device memory is RAM, not MMIO.  Use memremap() rather than

Right, CXL.mem is RAM, not MMIO, so I question why is this not being mapped via a RAM mechanism? Can you explain a bit more about why reusing vfio MMIO mapping mechanisms is suitable here and what happens when we get to questions like guest_memfd integration, page conversions, and large page mapping support?

None of that is applicable to VFIO. VFIO owns the entire address space and does not share it with the mm or anything else.

Quoted 3 lines

Maybe this is the start of the conversation for something simple, but I would actually prefer uncached ioremap() if only to avoid all the coherence management and suitability of MMIO mechanisms questions.

The entire thing needs to be mmapable in a cache coherent way since that is what the HW semantic is. If you try to do something else you will break KVM support since it follows the VMA.

Jason

maintainer
#19 lore raw

Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

Quoted 10 lines · Jason Gunthorpe
Quoted 8 lines · Dan Williams

drivers/vfio/pci/cxl/vfio_cxl_core.c static const struct vfio_pci_regops vfio_cxl_regops = { · line 656

Quoted 2 lines
drivers/vfio/pci/cxl/vfio_cxl_core.c +2 -0
+	/*
+	 * CXL device memory is RAM, not MMIO.  Use memremap() rather than

Right, CXL.mem is RAM, not MMIO, so I question why is this not being mapped via a RAM mechanism? Can you explain a bit more about why reusing vfio MMIO mapping mechanisms is suitable here and what happens when we get to questions like guest_memfd integration, page conversions, and large page mapping support?

Quoted 2 lines

None of that is applicable to VFIO. VFIO owns the entire address space and does not share it with the mm or anything else.

I was worried less about sharing and more about how the map eventually gets used, but makes sense all of that is enabled through the VFIO VMA. I have more reading to do in this space.

Quoted 7 lines
Quoted 3 lines

Maybe this is the start of the conversation for something simple, but I would actually prefer uncached ioremap() if only to avoid all the coherence management and suitability of MMIO mechanisms questions.

The entire thing needs to be mmapable in a cache coherent way since that is what the HW semantic is. If you try to do something else you will break KVM support since it follows the VMA.

Then I assume it matters that memremap() sometimes silently falls back to the direct map. The "VFIO owns" expectation needs to guard against some helpful platform firmware mapping accelerator memory as System RAM.

At a minimum having VFIO fail to map in that case helps with the argument I have been making that "no, EFI_CONVENTIONAL_MEMORY type + EFI_SPECIFIC_PURPOSE flag" is not suitable for accelerators with private CXL memory. Those want to be enforcing "EFI_RESERVED".

#20 lore raw

Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

Quoted 6 lines · Dan Williams
Quoted 4 lines · Jason Gunthorpe

The entire thing needs to be mmapable in a cache coherent way since that is what the HW semantic is. If you try to do something else you will break KVM support since it follows the VMA.

Quoted 8 lines

Then I assume it matters that memremap() sometimes silently falls back to the direct map. The "VFIO owns" expectation needs to guard against some helpful platform firmware mapping accelerator memory as System RAM.

At a minimum having VFIO fail to map in that case helps with the argument I have been making that "no, EFI_CONVENTIONAL_MEMORY type + EFI_SPECIFIC_PURPOSE flag" is not suitable for accelerators with private CXL memory. Those want to be enforcing "EFI_RESERVED".

Agree - in fact I would argue any potential user of private nodes should have some kind of splat saying the memory should be marked reserved in the first place, otherwise it's a firmware bug.

I need to read up a little bit on this area, but i don't see the "needs to be mmap'able in a cache coherent way" to be an argument for one particular method or another (hotplug, private node, memremap, etc).

If this extension starts bleeding into special-casing a bunch more mm/ stuff to be aware of and handle special memremap'd memory, that's when I would argue maybe we need to take a look at whether nodes apply. But if this just wants to pass a whole chunk from driver to guest and otherwise the host is supposed to stay out of the way - memremap seems like an ok start / a reasonable existing upstream solution.

~Gregory

#21 lore raw

Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

Quoted 11 lines · Gregory Price
Quoted 10 lines · Dan Williams
Quoted 4 lines · Jason Gunthorpe

The entire thing needs to be mmapable in a cache coherent way since that is what the HW semantic is. If you try to do something else you will break KVM support since it follows the VMA.

Then I assume it matters that memremap() sometimes silently falls back to the direct map. The "VFIO owns" expectation needs to guard against some helpful platform firmware mapping accelerator memory as System RAM.

Quoted 8 lines
Quoted 4 lines

At a minimum having VFIO fail to map in that case helps with the argument I have been making that "no, EFI_CONVENTIONAL_MEMORY type + EFI_SPECIFIC_PURPOSE flag" is not suitable for accelerators with private CXL memory. Those want to be enforcing "EFI_RESERVED".

Agree - in fact I would argue any potential user of private nodes should have some kind of splat saying the memory should be marked reserved in the first place, otherwise it's a firmware bug.

I would expect this to happen via the request_resource mechanism, if MM is using the CXL range it should be locked in the resource tree and vfio should request it and then fail during the initial startup phases.

Quoted 3 lines

I need to read up a little bit on this area, but i don't see the "needs to be mmap'able in a cache coherent way" to be an argument for one particular method or another (hotplug, private node, memremap, etc).

It is speaking to what VFIO has to do. It is the exclusive owner of the physical range, it does not have struct pages, it must be cachable for VFIO and KVM to work - not alot of choices here. Turn the phys_addr_t's into large cachable special PTEs inside a VMA.

Jason

#22 lore raw

Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

Quoted 9 lines · Jason Gunthorpe
Quoted 2 lines · Gregory Price
Quoted 1 line · Dan Williams

Jason Gunthorpe wrote:

I would expect this to happen via the request_resource mechanism, if MM is using the CXL range it should be locked in the resource tree and vfio should request it and then fail during the initial startup phases.

Quoted 9 lines
Quoted 3 lines

I need to read up a little bit on this area, but i don't see the "needs to be mmap'able in a cache coherent way" to be an argument for one particular method or another (hotplug, private node, memremap, etc).

It is speaking to what VFIO has to do. It is the exclusive owner of the physical range, it does not have struct pages, it must be cachable for VFIO and KVM to work - not alot of choices here. Turn the phys_addr_t's into large cachable special PTEs inside a VMA.

Right - the observation here is essentially "if you need struct page and
need to get fancy with them...". But if you just need to plonk a
region into one set of page tables and move on - then this is fine and arguable preferable.

~Gregory

#23 lore raw

Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

In reply to Dan Williams
Quoted 3 lines · Dan Williams

Then I assume it matters that memremap() sometimes silently falls back to the direct map. The "VFIO owns" expectation needs to guard against some helpful platform firmware mapping accelerator memory as System RAM.

I don't think how memremap works under the covers matters to vfio, it takes in a phys_addr_t an gives back a KVA that is cachable kernel memory.

We just have to be mindful to not allow mismatched attributes on virtual aliases.

Quoted 4 lines

At a minimum having VFIO fail to map in that case helps with the argument I have been making that "no, EFI_CONVENTIONAL_MEMORY type + EFI_SPECIFIC_PURPOSE flag" is not suitable for accelerators with private CXL memory. Those want to be enforcing "EFI_RESERVED".

Certainly it should fail the request region if something else is using it, so if those EFI flags plug it into the mm and it is plugged when VFIO starts then it should stop.

The direct map isn't really "plugged into the mm" but it does raise the mismatched attributed issue.

Jason

#24 lore raw

RE: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

Quoted 39 lines

-----Original Message-----
From: Jason Gunthorpe <[email protected]>
Sent: 07 April 2026 03:40
To: Dan Williams <[email protected]>
Cc: Manish Honap <[email protected]>; Alex Williamson <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; Ankit Agrawal <[email protected]>; Vikram Sethi <[email protected]>; Neo Jia <[email protected]>; Tarun Gupta (SW-GPU) <[email protected]>; Zhi Wang <[email protected]>; Krishnakant Jaju <[email protected]>; linux- [email protected]; [email protected]; linux- [email protected]; [email protected]
Subject: Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer

External email: Use caution opening links or attachments

Quoted 3 lines · Dan Williams

Then I assume it matters that memremap() sometimes silently falls back to the direct map. The "VFIO owns" expectation needs to guard against some helpful platform firmware mapping accelerator memory as System RAM.

I don't think how memremap works under the covers matters to vfio, it takes in a phys_addr_t an gives back a KVA that is cachable kernel memory.

We just have to be mindful to not allow mismatched attributes on virtual aliases.

Quoted 4 lines

At a minimum having VFIO fail to map in that case helps with the argument I have been making that "no, EFI_CONVENTIONAL_MEMORY type + EFI_SPECIFIC_PURPOSE flag" is not suitable for accelerators with private CXL memory. Those want to be enforcing "EFI_RESERVED".

Quoted 8 lines

Certainly it should fail the request region if something else is using it, so if those EFI flags plug it into the mm and it is plugged when VFIO starts then it should stop.

The direct map isn't really "plugged into the mm" but it does raise the mismatched attributed issue.

Jason

Thank you all for your suggestions.

As pointed out by Jason/Gregory, the whole design presented here is VFIO owned device regions + mmap + vmf_insert_pfn (patch 14) and registration at vfio_pci_open_device() (patch 16), Lazy fault (vmf_insert_pfn), zapping PTEs on reset.

I am convinced on the memremap usage as discussed. VFIO owns the physical range exclusively: no struct pages are involved (vmf_insert_pfn inserts raw PFNs), and memremap(MEMREMAP_WB) gives the VMA the correct WB cache attributes.

As suggested, I will add request_mem_region for memory range and a failure message if we cannot request mem region at probe.
This will also catch the EFI case: if firmware marks accelerator memory as EFI_CONVENTIONAL_MEMORY, MM will claim it at boot and request_mem_region will return a failure at probe time itself.

I will also add a comment in next cover letter explaining why we have decided to use memremap and not ioremap so that the decision we have taken in v2 is entirely clear at the start of review cycle.

Manish

#25 lore raw

PATCH v2 17/20 vfio/pci: Advertise CXL cap and sparse component BAR to userspace

In reply to [email protected]

From: Manish Honap <[email protected]>

Expose CXL device capability through the VFIO device info ioctl and give userspace access to the GPU/accelerator register windows in the component BAR while protecting the CXL component register block.

vfio_cxl_get_info() fills VFIO_DEVICE_INFO_CAP_CXL with the HDM register BAR index and byte offset, commit flags, and VFIO region indices for the DPA and COMP_REGS regions. HDM decoder count and the HDM block offset within COMP_REGS are not populated; both are derivable from the CXL Capability Array in the COMP_REGS region itself.

vfio_cxl_get_region_info() handles VFIO_DEVICE_GET_REGION_INFO for the component register BAR. It builds a sparse-mmap capability that advertises only the GPU/accelerator register windows, carving out the CXL component register block. Three physical layouts are handled:

  Topology A  comp block at BAR end:    one area [0, comp_reg_offset)
  Topology B  comp block at BAR start:  one area [comp_end, bar_len)
  Topology C  comp block in the middle: two areas, one on each side

vfio_cxl_mmap_overlaps_comp_regs() checks whether an mmap request overlaps [comp_reg_offset, comp_reg_offset + comp_reg_size). vfio_pci_core_mmap() calls it to reject access to the component register block while allowing mmap of the GPU register windows in the sparse capability. This replaces the earlier blanket rejection of any mmap on the component BAR index.

Hook both helpers into vfio_pci_ioctl_get_info() and vfio_pci_ioctl_get_region_info() in vfio_pci_core.c.

The component BAR cannot be claimed exclusively since the CXL subsystem holds persistent sub-range iomem claims during HDM decoder setup. pci_request_selected_regions() returns EBUSY; pass bars=0 to skip the request and map directly via pci_iomap(). Physical ownership is assured by driver binding.

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 155 +++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c     |  31 +++++-
 drivers/vfio/pci/vfio_pci_priv.h     |  24 +++++
 drivers/vfio/pci/vfio_pci_rdwr.c     |  16 ++-
 4 files changed, 221 insertions(+), 5 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +155 -0
index b38a04301660..46430cbfa962 100644
@@ -21,6 +21,161 @@
 #include "../vfio_pci_priv.h"
 #include "vfio_cxl_priv.h"
 
+u8 vfio_cxl_get_component_reg_bar(struct vfio_pci_core_device *vdev)
+{
+	return vdev->cxl->comp_reg_bar;
+}
+
+int vfio_cxl_get_region_info(struct vfio_pci_core_device *vdev,
+			     struct vfio_region_info *info,
+			     struct vfio_info_cap *caps)
+{
+	unsigned long minsz = offsetofend(struct vfio_region_info, offset);
+	struct vfio_region_info_cap_sparse_mmap *sparse;
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+	resource_size_t bar_len, comp_end;
+	u32 nr_areas, cap_size;
+	int ret;
+
+	if (!cxl)
+		return -ENOTTY;
+
+	if (!info)
+		return -ENOTTY;
+
+	if (info->argsz < minsz)
+		return -EINVAL;
+
+	if (info->index != cxl->comp_reg_bar)
+		return -ENOTTY;
+
+	/*
+	 * The device state is not fully initialised;
+	 * fall through to the default BAR handler.
+	 */
+	if (!cxl->comp_reg_size)
+		return -ENOTTY;
+
+	bar_len  = pci_resource_len(vdev->pdev, info->index);
+	comp_end = cxl->comp_reg_offset + cxl->comp_reg_size;
+
+	/*
+	 * Advertise the GPU/accelerator register windows as mmappable by
+	 * carving the CXL component register block out of the BAR.  The
+	 * number of sparse areas depends on where the block sits:
+	 *
+	 *  [A] comp block at BAR end  [gpu_regs | comp_regs]:
+	 *    comp_reg_offset > 0  &&  comp_end == bar_len
+	 *    = 1 area: [0, comp_reg_offset)
+	 *
+	 *  [B] comp block at BAR start [comp_regs | gpu_regs]:
+	 *    comp_reg_offset == 0 &&  comp_end < bar_len
+	 *    = 1 area: [comp_end, bar_len)
+	 *
+	 *  [C] comp block in middle    [gpu_regs | comp_regs | gpu_regs]:
+	 *    comp_reg_offset > 0  &&  comp_end < bar_len
+	 *    = 2 areas: [0, comp_reg_offset) and [comp_end, bar_len)
+	 */
+	if (cxl->comp_reg_offset > 0 && comp_end < bar_len)
+		nr_areas = 2;
+	else
+		nr_areas = 1;
+
+	cap_size = struct_size(sparse, areas, nr_areas);
+	sparse = kzalloc(cap_size, GFP_KERNEL);
+	if (!sparse)
+		return -ENOMEM;
+
+	sparse->header.id = VFIO_REGION_INFO_CAP_SPARSE_MMAP;
+	sparse->header.version = 1;
+	sparse->nr_areas = nr_areas;
+
+	if (nr_areas == 2) {
+		/* [C]: window before and after comp block */
+		sparse->areas[0].offset = 0;
+		sparse->areas[0].size   = cxl->comp_reg_offset;
+		sparse->areas[1].offset = comp_end;
+		sparse->areas[1].size   = bar_len - comp_end;
+	} else if (cxl->comp_reg_offset == 0) {
+		/* [B]: comp block at BAR start, window follows */
+		sparse->areas[0].offset = comp_end;
+		sparse->areas[0].size   = bar_len - comp_end;
+	} else {
+		/* [A]: comp block at BAR end, window precedes */
+		sparse->areas[0].offset = 0;
+		sparse->areas[0].size   = cxl->comp_reg_offset;
+	}
+
+	ret = vfio_info_add_capability(caps, &sparse->header, cap_size);
+	kfree(sparse);
+	if (ret)
+		return ret;
+
+	info->offset = VFIO_PCI_INDEX_TO_OFFSET(info->index);
+	info->size   = bar_len;
+	info->flags  = VFIO_REGION_INFO_FLAG_READ |
+		       VFIO_REGION_INFO_FLAG_WRITE |
+		       VFIO_REGION_INFO_FLAG_MMAP;
+
+	return 0;
+}
+
+bool vfio_cxl_mmap_overlaps_comp_regs(struct vfio_pci_core_device *vdev,
+				       u64 req_start, u64 req_len)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+
+	if (!cxl->comp_reg_size)
+		return false;
+
+	return req_start < cxl->comp_reg_offset + cxl->comp_reg_size &&
+	       req_start + req_len > cxl->comp_reg_offset;
+}
+
+int vfio_cxl_get_info(struct vfio_pci_core_device *vdev,
+		      struct vfio_info_cap *caps)
+{
+	struct vfio_pci_cxl_state *cxl = vdev->cxl;
+	struct vfio_device_info_cap_cxl cxl_cap = {0};
+
+	if (!cxl)
+		return 0;
+
+	/*
+	 * Device is not fully initialised?
+	 */
+	if (WARN_ON(cxl->dpa_region_idx < 0 || cxl->comp_reg_region_idx < 0))
+		return -ENODEV;
+
+	/* Fill in from CXL device structure */
+	cxl_cap.header.id = VFIO_DEVICE_INFO_CAP_CXL;
+	cxl_cap.header.version = 1;
+	/*
+	 * COMP_REGS region starts at comp_reg_offset + CXL_CM_OFFSET within
+	 * the BAR.  This is the byte offset of the CXL.mem register area (where
+	 * the CXL Capability Array Header lives) within the component register
+	 * block. Userspace derives hdm_decoder_offset and hdm_count from the
+	 * COMP_REGS region itself (CXL Capability Array traversal + HDMC read).
+	 */
+	cxl_cap.hdm_regs_offset = cxl->comp_reg_offset + CXL_CM_OFFSET;
+	cxl_cap.hdm_regs_bar_index = cxl->comp_reg_bar;
+
+	if (cxl->precommitted)
+		cxl_cap.flags |= VFIO_CXL_CAP_FIRMWARE_COMMITTED;
+	if (cxl->cache_capable)
+		cxl_cap.flags |= VFIO_CXL_CAP_CACHE_CAPABLE;
+
+	/*
+	 * Populate absolute VFIO region indices so userspace can query them
+	 * directly with VFIO_DEVICE_GET_REGION_INFO.
+	 */
+	cxl_cap.dpa_region_index = VFIO_PCI_NUM_REGIONS + cxl->dpa_region_idx;
+	cxl_cap.comp_regs_region_index =
+		VFIO_PCI_NUM_REGIONS + cxl->comp_reg_region_idx;
+
+	return vfio_info_add_capability(caps, &cxl_cap.header, sizeof(cxl_cap));
+}
+
 /*
  * Scope-based cleanup wrappers for the CXL resource APIs
  */
drivers/vfio/pci/vfio_pci_core.c +29 -2
index 48e0274c19aa..570775cc8711 100644
@@ -591,7 +591,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 	struct pci_dev *pdev = vdev->pdev;
 	struct vfio_pci_dummy_resource *dummy_res, *tmp;
 	struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp;
-	int i, bar;
+	int i, bar, bars;
 
 	/* For needs_reset */
 	lockdep_assert_held(&vdev->vdev.dev_set->lock);
@@ -650,8 +650,10 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 		bar = i + PCI_STD_RESOURCES;
 		if (!vdev->barmap[bar])
 			continue;
+		bars = (vdev->cxl && i == vfio_cxl_get_component_reg_bar(vdev)) ?
+			0 : (1 << bar);
 		pci_iounmap(pdev, vdev->barmap[bar]);
-		pci_release_selected_regions(pdev, 1 << bar);
+		pci_release_selected_regions(pdev, bars);
 		vdev->barmap[bar] = NULL;
 	}
 
@@ -989,6 +991,13 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_core_device *vdev,
 	if (vdev->reset_works)
 		info.flags |= VFIO_DEVICE_FLAGS_RESET;
 
+	if (vdev->cxl) {
+		ret = vfio_cxl_get_info(vdev, &caps);
+		if (ret)
+			return ret;
+		info.flags |= VFIO_DEVICE_FLAGS_CXL;
+	}
+
 	info.num_regions = VFIO_PCI_NUM_REGIONS + vdev->num_regions;
 	info.num_irqs = VFIO_PCI_NUM_IRQS;
 
@@ -1034,6 +1043,12 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev,
 	struct pci_dev *pdev = vdev->pdev;
 	int i, ret;
 
+	if (vdev->cxl) {
+		ret = vfio_cxl_get_region_info(vdev, info, caps);
+		if (ret != -ENOTTY)
+			return ret;
+	}
+
 	switch (info->index) {
 	case VFIO_PCI_CONFIG_REGION_INDEX:
 		info->offset = VFIO_PCI_INDEX_TO_OFFSET(info->index);
@@ -1768,6 +1783,18 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma
 	if (req_start + req_len > phys_len)
 		return -EINVAL;
 
+	/*
+	 * CXL devices: mmap is permitted for the GPU/accelerator register
+	 * windows listed in the sparse-mmap capability.  Block any request
+	 * that overlaps the CXL component register block
+	 * [comp_reg_offset, comp_reg_offset + comp_reg_size); those registers
+	 * must be accessed exclusively through the COMP_REGS device region so
+	 * that the emulation layer (notify_change) intercepts every write.
+	 */
+	if (vdev->cxl && index == vfio_cxl_get_component_reg_bar(vdev) &&
+	    vfio_cxl_mmap_overlaps_comp_regs(vdev, req_start, req_len))
+		return -EINVAL;
+
 	/*
 	 * Even though we don't make use of the barmap for the mmap,
 	 * we need to request the region and the barmap tracks that.
drivers/vfio/pci/vfio_pci_priv.h +24 -0
index ae0091d5096c..2d4aadd1b35a 100644
@@ -151,6 +151,14 @@ void vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev);
 int vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev);
 void vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev);
 int  vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev);
+int vfio_cxl_get_info(struct vfio_pci_core_device *vdev,
+		      struct vfio_info_cap *caps);
+int vfio_cxl_get_region_info(struct vfio_pci_core_device *vdev,
+			     struct vfio_region_info *info,
+			     struct vfio_info_cap *caps);
+u8 vfio_cxl_get_component_reg_bar(struct vfio_pci_core_device *vdev);
+bool vfio_cxl_mmap_overlaps_comp_regs(struct vfio_pci_core_device *vdev,
+				      u64 req_start, u64 req_len);
 
 #else
 
@@ -172,6 +180,22 @@ vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev) { }
 static inline int
 vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev)
 { return 0; }
+static inline int
+vfio_cxl_get_info(struct vfio_pci_core_device *vdev,
+		  struct vfio_info_cap *caps)
+{ return -ENOTTY; }
+static inline int
+vfio_cxl_get_region_info(struct vfio_pci_core_device *vdev,
+			 struct vfio_region_info *info,
+			 struct vfio_info_cap *caps)
+{ return -ENOTTY; }
+static inline u8
+vfio_cxl_get_component_reg_bar(struct vfio_pci_core_device *vdev)
+{ return U8_MAX; }
+static inline bool
+vfio_cxl_mmap_overlaps_comp_regs(struct vfio_pci_core_device *vdev,
+				 u64 req_start, u64 req_len)
+{ return false; }
 
 #endif /* CONFIG_VFIO_CXL_CORE */
 
drivers/vfio/pci/vfio_pci_rdwr.c +13 -3
index b38627b35c35..e95bdbdbcdb2 100644
@@ -201,19 +201,29 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_do_io_rw);
 int vfio_pci_core_setup_barmap(struct vfio_pci_core_device *vdev, int bar)
 {
 	struct pci_dev *pdev = vdev->pdev;
-	int ret;
+	int ret, bars;
 	void __iomem *io;
 
 	if (vdev->barmap[bar])
 		return 0;
 
-	ret = pci_request_selected_regions(pdev, 1 << bar, "vfio");
+	/*
+	 * The CXL component register BAR cannot be claimed exclusively: the
+	 * CXL subsystem holds persistent sub-range iomem claims during HDM
+	 * decoder setup. pci_request_selected_regions() for the full BAR
+	 * fails with EBUSY. Pass bars=0 to make the request a no-op and map
+	 * directly via pci_iomap().
+	 */
+	bars = (vdev->cxl && bar == vfio_cxl_get_component_reg_bar(vdev)) ?
+		0 : (1 << bar);
+
+	ret = pci_request_selected_regions(pdev, bars, "vfio");
 	if (ret)
 		return ret;
 
 	io = pci_iomap(pdev, bar, 0);
 	if (!io) {
-		pci_release_selected_regions(pdev, 1 << bar);
+		pci_release_selected_regions(pdev, bars);
 		return -ENOMEM;
 	}
 
Signature · 1 line
2.25.1
#26 lore raw

PATCH v2 18/20 vfio/cxl: Provide opt-out for CXL feature

In reply to [email protected]

From: Manish Honap <[email protected]>

This commit provides an opt-out mechanism to disable the CXL support from vfio module. The opt-out is provided both build time and module load time.

Build time option CONFIG_VFIO_CXL_CORE is used to enable/disable CXL support in vfio-pci module.

For runtime disabling the CXL support, use the module parameter disable_cxl. This is a per-device opt-out on the core device set by the driver before registration.

---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 4 ++++
 drivers/vfio/pci/vfio_pci.c          | 9 +++++++++
 include/linux/vfio_pci_core.h        | 1 +
 3 files changed, 14 insertions(+)
drivers/vfio/pci/cxl/vfio_cxl_core.c +4 -0
index 46430cbfa962..3ffc3e593d04 100644
@@ -479,6 +479,10 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev)
 	u16 dvsec;
 	int ret;
 
+	/* Honor the user opt-out decision */
+	if (vdev->disable_cxl)
+		return;
+
 	if (!pcie_is_cxl(pdev))
 		return;
 
drivers/vfio/pci/vfio_pci.c +9 -0
index 22cf9ea831f9..a6b0fb882b9f 100644
@@ -60,6 +60,12 @@ static bool disable_denylist;
 module_param(disable_denylist, bool, 0444);
 MODULE_PARM_DESC(disable_denylist, "Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users.");
 
+#if IS_ENABLED(CONFIG_VFIO_CXL_CORE)
+static bool disable_cxl;
+module_param(disable_cxl, bool, 0444);
+MODULE_PARM_DESC(disable_cxl, "Disable CXL Type-2 extensions for all devices bound to vfio-pci. Variant drivers may instead set vdev->disable_cxl in their probe for per-device control without needing this parameter.");
+#endif
+
 static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev)
 {
 	switch (pdev->vendor) {
@@ -189,6 +195,9 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 		return PTR_ERR(vdev);
 
 	dev_set_drvdata(&pdev->dev, vdev);
+#if IS_ENABLED(CONFIG_VFIO_CXL_CORE)
+	vdev->disable_cxl = disable_cxl;
+#endif
 	vdev->pci_ops = &vfio_pci_dev_ops;
 	ret = vfio_pci_core_register_device(vdev);
 	if (ret)
include/linux/vfio_pci_core.h +1 -0
index aa159d0c8da7..48dc69df52fa 100644
@@ -130,6 +130,7 @@ struct vfio_pci_core_device {
 	bool			needs_pm_restore:1;
 	bool			pm_intx_masked:1;
 	bool			pm_runtime_engaged:1;
+	bool                    disable_cxl:1;
 	struct pci_saved_state	*pci_saved_state;
 	struct pci_saved_state	*pm_save;
 	int			ioeventfds_nr;
Signature · 1 line
2.25.1
#27 lore raw

PATCH v2 19/20 docs: vfio-pci: Document CXL Type-2 device passthrough

In reply to [email protected]

From: Manish Honap <[email protected]>

Add Documentation/driver-api/vfio-pci-cxl.rst describing the architecture, VFIO interfaces, and operational constraints for CXL Type-2 (cache-coherent accelerator) passthrough via vfio-pci-core, and link it from the driver-api index.

The document covers:
- VFIO_DEVICE_FLAGS_CXL and VFIO_DEVICE_INFO_CAP_CXL: what the capability struct contains and what the FIRMWARE_COMMITTED and CACHE_CAPABLE flags mean
- How to derive hdm_decoder_offset and hdm_count from the COMP_REGS region by traversing the CXL Capability Array to find cap ID 0x5 and reading the HDM Decoder Capability register
- Topology-aware sparse mmap on the component BAR (topologies A, B, C covering comp block at end, start, or middle of the BAR)
- Two extra VFIO device regions: COMP_REGS for the emulated HDM register state and the DPA memory window
- DVSEC config write virtualization: what the guest sees vs. hardware
- FLR coordination: DPA PTEs zapped before reset, restored after

---
 Documentation/driver-api/index.rst        |   1 +
 Documentation/driver-api/vfio-pci-cxl.rst | 382 ++++++++++++++++++++++
 2 files changed, 383 insertions(+)
 create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst
Documentation/driver-api/index.rst +1 -0
index 1833e6a0687e..7ec661846f6b 100644
@@ -47,6 +47,7 @@ of interest to most developers working on device drivers.
    vfio-mediated-device
    vfio
    vfio-pci-device-specific-driver-acceptance
+   vfio-pci-cxl
 
 Bus-level documentation
 =======================
Documentation/driver-api/vfio-pci-cxl.rst new file +382 -0

382 lines not on this page · raw message

Signature · 1 line
2.25.1
#28 lore raw

PATCH v2 20/20 selftests/vfio: Add CXL Type-2 VFIO assignment test

In reply to [email protected]

From: Manish Honap <[email protected]>

Add vfio_cxl_type2_test and build it from the vfio selftest Makefile. The binary expects a PCI BDF (argv or VFIO_SELFTESTS_BDF) with the device already on vfio-pci and CONFIG_VFIO_CXL_CORE enabled.

It exercises:
- VFIO_DEVICE_GET_INFO,
- GET_REGION_INFO,
- VFIO_DEVICE_INFO_CAP_CXL capability list,
- sparse component-BAR vs DPA/COMP_REG regions,
- HDM decoder emulation (masks, commit, lock),
- DVSEC-backed config where the driver exposes it.

Large region read/write loops and FLR-heavy test cases are still pending; Need to revisit these in next version of patches.

vfio_pci_device_setup() skips auto-mmap for BARs that carry sparse-mmap capabilities; those require the caller to mmap only the windows advertised by the capability.

---
 tools/testing/selftests/vfio/Makefile         |   1 +
 .../selftests/vfio/lib/vfio_pci_device.c      |   3 +-
 .../selftests/vfio/vfio_cxl_type2_test.c      | 920 ++++++++++++++++++
 3 files changed, 923 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/vfio/vfio_cxl_type2_test.c
tools/testing/selftests/vfio/Makefile +1 -0
index 3c796ca99a50..2cac98302609 100644
@@ -4,6 +4,7 @@ TEST_GEN_PROGS += vfio_iommufd_setup_test
 TEST_GEN_PROGS += vfio_pci_device_test
 TEST_GEN_PROGS += vfio_pci_device_init_perf_test
 TEST_GEN_PROGS += vfio_pci_driver_test
+TEST_GEN_PROGS += vfio_cxl_type2_test
 
 TEST_FILES += scripts/cleanup.sh
 TEST_FILES += scripts/lib.sh
tools/testing/selftests/vfio/lib/vfio_pci_device.c +2 -1
index fac4c0ecadef..98832acc31ac 100644
@@ -257,7 +257,8 @@ static void vfio_pci_device_setup(struct vfio_pci_device *device)
 		struct vfio_pci_bar *bar = device->bars + i;
 
 		vfio_pci_region_get(device, i, &bar->info);
-		if (bar->info.flags & VFIO_REGION_INFO_FLAG_MMAP)
+		if ((bar->info.flags & VFIO_REGION_INFO_FLAG_MMAP) &&
+		    !(bar->info.flags & VFIO_REGION_INFO_FLAG_CAPS))
 			vfio_pci_bar_map(device, i);
 	}
 
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +920 -0

920 lines not on this page · raw message

Signature · 1 line
2.25.1
maintainer
#29 lore raw
In reply to [email protected]

Forgive me if any of the commentary below was already hashed out in the v1 discussion. Your excellent changelog notes make catching up much easier, thanks!

Quoted 24 lines · mhonap@ wrote:

From: Manish Honap <[email protected]>

CXL Type-2 accelerators (e.g. CXL.mem-capable GPUs) cannot be passed through to virtual machines with stock vfio-pci because the driver has no concept of HDM decoder management, DPA region exposure, or component register emulation. This series wires all of that into vfio-pci-core behind a new CONFIG_VFIO_CXL_CORE optional module, without requiring a variant driver.

When a CXL Device DVSEC (Vendor ID 0x1E98, ID 0x0000) is detected at device open time, the driver:

- Probes the HDM Decoder Capability block in the component registers and allocates a DPA region through the CXL subsystem. On devices where firmware has already committed a decoder, the kernel skips allocation and re-uses the committed range.

- Builds a kernel-owned shadow of the HDM register block. The VMM reads and writes this shadow through a dedicated COMP_REGS VFIO region rather than touching the hardware directly. The kernel enforces CXL 3.1 bit-field rules: reserved bits, read-only bits, the COMMIT/COMMITTED latch, and the LOCK→0 reprogram path for firmware-committed decoders.

Quoted 3 lines

- Exposes the DPA range as a second VFIO region (VFIO_REGION_SUBTYPE_CXL) backed by the kernel-assigned HPA. PTEs are inserted lazily on first page fault and torn down atomically under memory_lock during FLR.

I assume, or hope this means expose a CXL region as VFIO_REGION_SUBTYPE_CXL, as DPA is a device-internal address space that VFIO probably does not need to worry about. VFIO likely only needs to care about system visible resource.

If / when interleaving arrives for CXL accelerators the 1:1 vfio-pci to DPA to CXL region HPA association breaks. Ok, to assume 1:1 for now.

Quoted 2 lines

- Intercepts writes to the CXL DVSEC configuration-space registers (Control, Status, Control2, Status2, Lock, Range Base) and replays

Range Base is ignored when global HDM Decoder Control is enabled. I would hope that this enabling ditches CXL 1.x legacy wherever possible.

Quoted 2 lines

them through a per-device vconfig shadow, enforcing RWL/RW1CS/RWO access semantics and the CONFIG_LOCK one-shot latch.

Linux should have no need to ever trigger CXL register bit locks. That is only for firmware to make changes immutable if the firmware has requirements that nothing moves for its own purposes.

Now, it makes sense to configure the vCXL device to be locked at setup, but I do not currently see the use case for the vBIOS to mutate and lock the configuration.

[..]

Quoted 1 line

- Includes selftests

Yay!

Quoted 7 lines

covering device detection, capability parsing,
region enumeration, HDM register emulation, DPA mmap with page-fault insertion, FLR invalidation, and DVSEC register emulation.

The series is applied on top of the cxl/next branch using the base specified at the end of this cover letter plus Alejandro's v23 Type-2 device support patches [1].

One of the sticking points of the accelerator series has been how many details of the CXL core internal object lifetime leak out.

My hope / thought experiment is that the initial version of this enabling only needs to facilitate getting a VMM established CXL region into a guest. With that VFIO only needs is the CXL region HPA and MMIO layout so that CXL registers can be trapped and non-CXL registers can be direct mapped.

Quoted 19 lines

Series structure
================

Patches 1-5 extend the CXL subsystem with the APIs vfio-pci needs.

Patches 6-8 add the vfio-pci-core plumbing (UAPI, device state, Kconfig/build).

Patches 9-15 implement the core device lifecycle: detection, HDM emulation, media readiness, region management, DPA region, and DVSEC emulation.

Patches 16-18 wire everything together at open/close time and populate the VFIO ioctl paths.

Patches 19-20 add documentation and selftests.

Changes since v1
================

[..]

Quoted 7 lines

HDM API simplification (patch 1)

v1 exported cxl_get_hdm_reg_info() which returned a raw struct with offset and size fields. v2 replaces it with cxl_get_hdm_info() which uses the cached count already populated by cxl_probe_component_regs() and returns a single struct with all HDM metadata, removing the need for callers to re-read the hardware.

What is the accelerator use case to support multiple CXL regions per device?

In other words, it feels ambitious to support that while simultaneously kicking the "interleave" question down the road. If we are going for initial simplicity that also means single region to start.

Quoted 6 lines

cxl_await_range_active() split (patch 4)

cxl_await_media_ready() requires a CXLMDEV mailbox register, which Type-2 accelerators may not have. v2 splits out cxl_await_range_active() so the HDM range-active poll can be used independently of the media ready path.

This feels like a detail vfio-pci does not need to worry about. The core knows that the device does not have a mailbox and the core knows it needs to await range ready when probing HDM. Something is broken if vfio-pci needs to duplicate this part of the setup.

Quoted 5 lines

LOCK→0 transition in HDM ctrl write emulation (patch 11)

v1 did not handle the case where a guest tries to clear the LOCK bit to reprogram a firmware-committed decoder. v2 allows this transition and re-programs the hardware accordingly.

? Guest has no ability to manipulate Host HPA mappings. A protocol for a guest to work with a host to remap HPA does not sound like a v1 requirement. This would be equivalent to a guest asking to move a host PCI BAR.

#30 lore raw
Quoted 56 lines

-----Original Message-----
From: Dan Williams <[email protected]>
Sent: 14 April 2026 09:39
To: Manish Honap <[email protected]>; Alex Williamson <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; Ankit Agrawal <[email protected]>
Cc: Vikram Sethi <[email protected]>; Neo Jia <[email protected]>; Tarun Gupta (SW-GPU) <[email protected]>; Zhi Wang <[email protected]>; Krishnakant Jaju <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; Manish Honap <[email protected]>; Alex Williamson <[email protected]>; Jonathan Cameron <[email protected]>
Subject: Re: [PATCH v2 00/20] vfio/pci: Add CXL Type-2 device passthrough support

External email: Use caution opening links or attachments

Forgive me if any of the commentary below was already hashed out in the v1 discussion. Your excellent changelog notes make catching up much easier, thanks!

Quoted 17 lines · mhonap@ wrote:

From: Manish Honap <[email protected]>

CXL Type-2 accelerators (e.g. CXL.mem-capable GPUs) cannot be passed through to virtual machines with stock vfio-pci because the driver has no concept of HDM decoder management, DPA region exposure, or component register emulation. This series wires all of that into vfio-pci-core behind a new CONFIG_VFIO_CXL_CORE optional module, without requiring a variant driver.

When a CXL Device DVSEC (Vendor ID 0x1E98, ID 0x0000) is detected at device open time, the driver:

- Probes the HDM Decoder Capability block in the component registers and allocates a DPA region through the CXL subsystem. On devices where firmware has already committed a decoder, the kernel skips allocation and re-uses the committed range.

Quoted 8 lines

- Builds a kernel-owned shadow of the HDM register block. The VMM reads and writes this shadow through a dedicated COMP_REGS VFIO region rather than touching the hardware directly. The kernel enforces CXL 3.1 bit-field rules: reserved bits, read-only bits, the COMMIT/COMMITTED latch, and the LOCK→0 reprogram path for firmware-committed decoders.

- Exposes the DPA range as a second VFIO region

(VFIO_REGION_SUBTYPE_CXL)

Quoted 8 lines
Quoted 1 line

backed by the kernel-assigned HPA. PTEs are inserted lazily on

first

Quoted 1 line

page fault and torn down atomically under memory_lock during FLR.

I assume, or hope this means expose a CXL region as VFIO_REGION_SUBTYPE_CXL, as DPA is a device-internal address space that VFIO probably does not need to worry about. VFIO likely only needs to care about system visible resource.

Fair catch - that was incorrect wording. DPA is just what we hand to the CXL subsystem during allocation setup; guests never see it. I will fix the cover letter and any comments with the same mistake so that docs do not imply VFIO is exporting DPA directly.

Quoted 3 lines

If / when interleaving arrives for CXL accelerators the 1:1 vfio-pci to DPA to CXL region HPA association breaks. Ok, to assume 1:1 for now.

Okay, v2 is designed for single-region / non-interleaved case; I will revisit association model in next round of review and update the changelog to state this decision.

Quoted 6 lines
Quoted 3 lines

- Intercepts writes to the CXL DVSEC configuration-space registers (Control, Status, Control2, Status2, Lock, Range Base) and replays

Range Base is ignored when global HDM Decoder Control is enabled. I would hope that this enabling ditches CXL 1.x legacy wherever possible.

Noted on Range Base being ignored when global HDM decoder control is in play. I will audit the emulation path and the doc for any wording that sounds like we depend on legacy range-base behavior when global HDM is enabled. In the next review round, I will drop the Range Base handling from DVSEC emulation codepath.

Quoted 8 lines
Quoted 3 lines

them through a per-device vconfig shadow, enforcing RWL/RW1CS/RWO access semantics and the CONFIG_LOCK one-shot latch.

Linux should have no need to ever trigger CXL register bit locks. That is only for firmware to make changes immutable if the firmware has requirements that nothing moves for its own purposes.

Quoted 8 lines

Now, it makes sense to configure the vCXL device to be locked at setup, but I do not currently see the use case for the vBIOS to mutate and lock the configuration.

[..]

Quoted 1 line

- Includes selftests

Yay!

Thank you! I will keep extending them as the UAPI surface stabilizes.

Quoted 10 lines
Quoted 3 lines

covering device detection, capability parsing, region enumeration, HDM register emulation, DPA mmap with page-

fault

Quoted 5 lines

insertion, FLR invalidation, and DVSEC register emulation.

The series is applied on top of the cxl/next branch using the base specified at the end of this cover letter plus Alejandro's v23 Type-2 device support patches [1].

Quoted 8 lines

One of the sticking points of the accelerator series has been how many details of the CXL core internal object lifetime leak out.

My hope / thought experiment is that the initial version of this enabling only needs to facilitate getting a VMM established CXL region into a guest. With that VFIO only needs is the CXL region HPA and MMIO layout so that CXL registers can be trapped and non-CXL registers can be direct mapped.

Okay, I will investigate this part.

Quoted 26 lines
Quoted 20 lines

Series structure
================

Patches 1-5 extend the CXL subsystem with the APIs vfio-pci needs.

Patches 6-8 add the vfio-pci-core plumbing (UAPI, device state, Kconfig/build).

Patches 9-15 implement the core device lifecycle: detection, HDM emulation, media readiness, region management, DPA region, and DVSEC emulation.

Patches 16-18 wire everything together at open/close time and populate the VFIO ioctl paths.

Patches 19-20 add documentation and selftests.

Changes since v1
================

[..]

Quoted 5 lines

HDM API simplification (patch 1)

v1 exported cxl_get_hdm_reg_info() which returned a raw struct with offset and size fields. v2 replaces it with cxl_get_hdm_info() which uses the cached count already populated by

Quoted 6 lines

cxl_probe_component_regs()

Quoted 2 lines

and returns a single struct with all HDM metadata, removing the need for callers to re-read the hardware.

What is the accelerator use case to support multiple CXL regions per device?

For this version there isn't one. One committed decoder, one contiguous Region and restricted to decoder 0. I will think about addition of these aspects.

Quoted 9 lines

In other words, it feels ambitious to support that while simultaneously kicking the "interleave" question down the road. If we are going for initial simplicity that also means single region to start.

Quoted 4 lines

cxl_await_range_active() split (patch 4)

cxl_await_media_ready() requires a CXLMDEV mailbox register, which Type-2 accelerators may not have. v2 splits out

Quoted 8 lines

cxl_await_range_active()

Quoted 2 lines

so the HDM range-active poll can be used independently of the media ready path.

This feels like a detail vfio-pci does not need to worry about. The core knows that the device does not have a mailbox and the core knows it needs to await range ready when probing HDM. Something is broken if vfio-pci needs to duplicate this part of the setup.

Okay, I'll send an RFC to linux-cxl for this and refactor the patches in current series.

Quoted 11 lines
Quoted 6 lines

LOCK→0 transition in HDM ctrl write emulation (patch 11)

v1 did not handle the case where a guest tries to clear the LOCK bit to reprogram a firmware-committed decoder. v2 allows this transition and re-programs the hardware accordingly.

? Guest has no ability to manipulate Host HPA mappings. A protocol for a guest to work with a host to remap HPA does not sound like a v1 requirement. This would be equivalent to a guest asking to move a host PCI BAR.

Okay, agreed. For initial support addition, I will drop the lock programming support for guest in next review. If in future we require HDM remapping, we can think of a separate mechanism for this instead of config-space writes.

Manish