vfio/pci: Add CXL Type-2 device passthrough support
Subsystems (8)
v6 cover vfio/pci: Add CXL Type-2 device passthrough support
From: Manish Honap <[email protected]>
Hello CXL Community,
This series adds VFIO passthrough for CXL Type-2 accelerators. The guest drives its own virtual HDM decoder and can reset the device. The host owns the physical decoder and the host physical address the memory lands at. The guest only picks a guest physical address. This series targets a single, non-interleaved, firmware-committed endpoint decoder.
Base and dependencies
---------------------
Base: Linux 7.3-rc5 (72d3fcf802c4), plus:
- Richard's zero-sized HDM decoder series (v10) [1].
- Srirangan's cxl_reset series (v15) [2]. It caches the endpoint decoder settings in pdev->hdm at enumeration and implements CXL Reset as a PCI reset method in drivers/pci/cxl.c.
Patch 4 of this series changes drivers/pci/cxl.c from [2]. v15 exports no reset entry that a module can call, and its reset method falls through to FLR when it cannot run, so patch 4 adds an entry for a driver that holds the HDM range and exports the helpers vfio-cxl needs.
Runtime requirement on arm64: CXL Reset invalidates the CPU caches for the HDM range, which needs a registered cache maintenance driver. The series was tested with the SMCCC cache clean/invalidate provider (v3)
[3] and firmware that implements the SMCCC cache operations; v4 [4] has since been posted. This is not a build dependency. Without a provider the device binds as plain vfio-pci, without the CXL extensions.
Changes since v5
----------------
v5 [5] kept the v4 provider model. v6 keeps it too and folds the v5 review, the rebase onto cxl_reset v15, and fixes found by testing on hardware.
Base and dependency
- Rebased from cxl_reset v12 onto v15. v15 moved the reset into drivers/pci/cxl.c, so the v5 patch that added cxl_reset_dvsec_sequence() in drivers/cxl/core/resource.c is replaced by patch 4: pci_cxl_reset_owned() runs the reset for the driver that holds the HDM range and returns its failure to that driver. A reset that cannot run fails instead of falling through to FLR.
- Bind requires that the host can run CXL Reset for the device, including the CPU cache invalidation. Otherwise the device binds without CXL support, so no guest gets device memory that the host cannot clear on the next reset.
Structural
- Patches 07 and 08 are folded into 06 (now patch 7).
- The excluded-range patches 09 and 10 are now three: per-BAR sorted lists with the read/write path and MSI-X moved onto them, the sparse mmap capability, and refusing ioeventfds and dma-buf exports over an excluded range.
- The HDM region patch is split into read/write (21) and mmap (22).
- New vfio-pci-core patches: one try-reset helper for every reset point (15), revoke and re-arm of one device-specific region with its dma-bufs (16), and provider hooks across low power (17).
- New PCI patch: bus and slot reset are refused below a CXL port with Secondary Bus Reset (SBR) masked (5).
Behavioral
- Low power is supported and the D0 pin is gone. vfio-cxl takes the HDM region out of use before the device leaves D0 and restores the decoder when it returns.
- The HDM region and its dma-bufs follow the committed decoder, not Memory Space Enable. vfio-pci-core no longer knows about HDM: each dma-buf records the region it was exported from, and vfio-cxl revokes and re-arms its own region around reset, hot reset and low power.
- With SBR masked, VFIO_DEVICE_PCI_HOT_RESET fails instead of reporting a reset that never reached the device. With SBR unmasked, reset_done restores the decoder and the ioctl fails if the restore fails.
- A failed CXL Reset returns its error and leaves the HDM region out of use until a later CXL Reset succeeds. A hot reset or a return from low power in between does not reopen it. The first reset at open fails the open only on -EAGAIN.
- AER and DPC recovery take the HDM region out of use, and put it back only if it was in use before the error and the decoder is still committed.
- Reads of the decoder register region fail with -EIO while Memory Space is disabled, as BAR reads do.
- HDM region read and write copy through a page-sized kernel buffer, so memory_lock is not held across a user copy.
- The guest's Initiate CXL Reset doorbell runs the real reset and reports Reset Complete or Reset Error in the shadow Status2.
- Without a provider, vfio-pci still guards the CXL DVSEC of a Type-2
function: Lock and Range writes are dropped, and Initiate CXL Reset runs as a vfio reset.
- The CXL DVSEC is matched by offset range, so a DVSEC right before or after it in config space no longer bypasses the virtualization or indexes past the permission map.
- CXL specification references are updated to CXL r4.0.
Found in hardware testing
vfio-cxl now waits for Memory_Active, up to the Memory_Active_Timeout the device reports, before it puts the HDM region back. It also waits at bind, since a reset just before the bind clears it too.
- The bind-time copy of the HDM state used a helper that asserts the device lock. The PCI core can run probe in a worker on the device's NUMA node, so lockdep warned at every bind. Patch 4 adds pci_cxl_hdm_snapshot(), which needs no device lock.
- The disable_cxl module parameter was never read. It now takes effect.
Reviewer feedback addressed
---------------------------
The v5 [5] thread has the full discussion. This is where each comment landed.
Alex Williamson
- Provider load and its failure handling are one patch (7), and the messages say "continuing without CXL support".
- The excluded-range list is per BAR, sorted, and rejects overlaps; the ROM is left alone; ioeventfds and dma-buf exports over an excluded range are refused (8 to 10).
- cxl_dvsec_len had no user in v5 (11). It now has one: the CXL DVSEC is matched by offset range.
- vfio-pci exports vfio_pci_core_reset_prepare() and vfio_pci_core_reset_done(), and calls them from its own error handlers instead of reaching into drvdata (13).
- Low power is supported instead of pinning D0 (17, 21).
- vfio_pci_core_unregister_dev_region() is gone. vfio-pci-core frees every dynamic region on a failed open as well as on close (12), so vfio-cxl has nothing to unwind.
- hdm_valid is set at bind and changed only by reset, hot reset and low power.
- vfio_pci_zap_bars() covers only the BARs again; a device-specific region is revoked by its provider (16).
- The decoder register read takes memory_lock and checks Memory Space
(24).
- Hot reset with SBR masked and unmasked (5, 26).
- Reset is a mandatory provider op behind one try-reset helper (15), with a success-oriented flow in vfio-cxl (29). .hdm_active is gone.
Jonathan Cameron
- Patch 1 keeps the Reviewed-by tag.
- A BAR-owning driver records one owned resource; a second call warns and fails with -EBUSY (2).
- The uapi header uses __GENMASK() and _BITUL() (3).
- CXL Reset writes back and invalidates the CPU caches for the HDM range before and after the reset (4).
- Allocator and style changes in 9, 11 and 12, and the devres group factored into a helper (19).
Richard Cheng
- The provider module reference is pinned per bound device (7).
- ioeventfds and dma-buf exports over an excluded range are refused
(10).
- A dma-buf of the HDM region is exported with no_peer2peer set (28).
- The PCI state is saved before the reset at open (15).
Gregory Price
- The documentation is reworked (30) as per review comments.
Patch order
-----------
The tree builds at every commit. Five groups:
Part 1 CXL core and PCI (patches 1 to 5)
- Split the BAR block request and ioremap helpers, and let a BAR-owning driver own the component register block, so the HDM and RAS sub-blocks are left for vfio-cxl.
- Move the component register defines to include/uapi/cxl/cxl_regs.h so a VMM and the selftest can use them.
- Let the driver that owns the HDM range run CXL Reset.
- Refuse bus and slot reset below a CXL port with SBR masked.
Part 2 vfio-pci-core (patches 6 to 17)
- The CXL provider ops registration and on-demand provider load.
- The generic excluded-range list, its sparse mmap capability, and the ioeventfd and dma-buf checks.
- The CXL DVSEC virtualization, the open and close hooks, the reset hooks, and the disable_cxl opt-out.
- The try-reset helper, per-region revoke and re-arm, and the low power hooks.
Part 3 vfio-cxl provider and HDM regions (patches 18 to 28)
- The vfio-cxl module, the CXL memdev created at bind, and ownership of the whole component BAR.
- The HDM memory region (read/write, then mmap), memory-failure containment for it, and the read-only decoder register region.
- Excluding the decoder block from the direct BAR, the decoder restore after a hot reset, the device and decoder geometry capability, and the dma-buf export.
Part 4 vfio-cxl reset (patch 29)
- Run CXL Reset at every vfio reset point.
Part 5 Documentation and selftests (patches 30 and 31)
Subsystem boundary
------------------
vfio-pci-core does not implement CXL registers. vfio-cxl is a separate module that registers a struct vfio_cxl_ops at init. vfio-pci-core loads it on demand for a CXL device (request_module() after pcie_is_cxl()) and pins it per bound device. If a modular provider is missing or fails to load, the device is driven as plain vfio-pci. The bind only defers for a built-in provider whose initcall has not run yet.
vfio-pci-core provides generic pieces that a provider calls: the excluded-range list, revoke and re-arm of one device-specific region, the try-reset helper and the low power hooks. The CXL-specific work stays behind the ops. The CXL DVSEC virtualization is the one CXL piece in vfio-pci-core, because it lives in the config space permission hooks.
Memory ownership
----------------
The host resolves the host physical address once, at bind, from the committed decoder. vfio-cxl creates the memdev at bind and tears it down at unbind. It claims the HPA range with request_mem_region_exclusive(), so nothing, /dev/mem included, can map a conflicting cacheable alias.
The guest programs a guest physical address into a virtual decoder that the VMM emulates. It never reaches the physical decoder registers: those are read through a read-only region and are excluded from the direct BAR mapping, so the guest cannot move the host physical window.
HDM region access
-----------------
The HDM region carries the coherent device memory. A VMM mmaps it and maps it into the guest through stage-2. The fault handler inserts the host PFNs, including PMD-sized mappings when the mapping is aligned, only while the committed decoder is in place and the media is ready. Otherwise the fault gets SIGBUS. Region read and write check the same state and return -EIO while the region is out of use. The range has no struct page and is registered with the memory-failure machinery, so a memory error is contained to a SIGBUS.
DMA and iommufd
---------------
Some accelerators, for example a GPU, send ATS-translated DMA to their own HDM, so that range must be in the guest IOAS. IOMMU_IOAS_MAP cannot pin a range with no struct page. VFIO_DEVICE_FEATURE_DMA_BUF on the HDM region returns a dma-buf fd that iommufd maps with IOMMU_IOAS_MAP_FILE. vfio-cxl revokes the dma-buf whenever it takes the region out of use. iommufd then unmaps the range, and the VMM maps the same dma-buf again once the region is back.
Reset
-----
An FLR does not reset CXL.cache or CXL.mem state and does not clear device memory (CXL r4.0 sec 9.5), so it cannot separate one user of the device from the next. vfio-cxl runs CXL Reset with Memory Clear at every vfio reset point: open and close, VFIO_DEVICE_RESET, a guest FLR through config space, and the guest's Initiate CXL Reset doorbell. Under memory_lock, with the HDM mmap zapped and the dma-buf revoked, the host invalidates the CPU caches for the HDM range, resets the function, restores the decoder and the PCI config state, and invalidates the caches again. vfio-cxl then waits for Memory_Active before it puts the region back.
A CXL port masks SBR by default (CXL r4.0 sec 8.1.5.2). The PCI core now refuses a bus or slot reset below such a port, so the vfio hot reset ioctls fail and a VMM uses VFIO_DEVICE_RESET. If the port has SBR unmasked, the hot reset decommits the decoder, and vfio-cxl restores it in reset_done with cxl_restore_state_after_pci_reset(), the helper the PCI core's cxl_bus reset method uses.
Low power
---------
The device may enter D3hot or runtime low power while it is assigned. Low power loses the committed decoder, so vfio-cxl takes the HDM region out of use first and restores the decoder when the device is back in D0. Device memory is not saved or restored across low power.
Deferred
--------
- Switched, multi-decoder and interleaved topologies stay rejected at bind.
- Decoders that firmware did not commit are not supported.
Validation
----------
- Each patch builds with W=1 (drivers/cxl, drivers/pci and
drivers/vfio/pci, x86_64, VFIO_CXL=m).
- scripts/checkpatch.pl --strict --codespell reports no errors or warnings. Patch 16 has one CHECK, a missing blank line between two inline stubs, kept to match the other stubs in vfio_pci_priv.h.
- The selftest, tools/testing/selftests/vfio/vfio_cxl_type2_test.c, has 27 tests: the device flag and regions, the HDM mmap and huge fault, the decoder register region and its -EIO gate, the dma-buf map with IOMMU_IOAS_MAP_FILE, region read and write across pages, the DVSEC reset doorbell, and the decoder after a D3hot cycle. All 27 pass on an arm64 host with 64K pages and lockdep (PROVE_LOCKING) enabled.
- A guest with the device assigned booted, ran CUDA and unified memory workloads, and reset the device twice through guest CXL Reset through guest sysfs reset file and by using the CTRL2_INIT_CXL_RST in the guest. (used with QEMU vfio-cxl support v2 [6]).
AI assistance disclosure
------------------------
This series was developed with substantial AI assistance (Anthropic Claude, through the Claude Code CLI). It was used for design trade-offs, multi-agent regression and adversarial review of the v5 feedback using review-core.md, the port to cxl_reset v15, analysis of the hardware failures, and drafting the selftests and the documentation. Following Documentation/process/coding-assistants.rst, each patch carries an Assisted-by: LLM tag. I ran the tests on hardware, have reviewed every change, and take full DCO responsibility for the series.
References
----------
[1] [PATCH v10 0/3] Support zero-sized HDM decoders
https://lore.kernel.org/all/[email protected]/
[2] [PATCH v15 00/16] PCI/CXL: Add CXL reset support for Type 2 devices
https://lore.kernel.org/all/[email protected]/
[3] [PATCH v3 0/2] Add SMCCC cache clean/invalidate provider
https://lore.kernel.org/all/[email protected]/
[4] [PATCH v4 0/2] Add SMCCC cache clean/invalidate provider
https://lore.kernel.org/all/[email protected]/
[5] [PATCH v5 0/27] vfio/pci: Add CXL Type-2 device passthrough support
https://lore.kernel.org/all/[email protected]/
[6] [PATCH v2 00/10] QEMU: CXL Type-2 device passthrough via vfio-pci
https://lore.kernel.org/linux-cxl/[email protected]/
Manish Honap (31):
cxl/regs: Split the BAR block request and ioremap helpers cxl/regs: Let a BAR-owning driver own the component register block
cxl: Move component register defines to uapi/cxl/cxl_regs.h PCI/CXL: Let the driver that owns the HDM range run CXL Reset
PCI: Refuse bus and slot reset below a CXL port with SBR masked vfio/pci: Add the CXL provider ops registration interface vfio/pci: Detect CXL devices and load the CXL provider on demand vfio/pci: Add a generic excluded-range list
vfio/pci: Advertise the mmap areas around excluded ranges vfio/pci: Refuse ioeventfds and dma-buf exports over excluded ranges vfio/pci: Virtualize the CXL DVSEC in vfio_pci_config.c vfio/pci: Call the CXL open and close hooks around device use vfio/pci: Bracket PCI resets with the CXL reset hooks vfio/pci: Provide an opt-out for the CXL Type-2 extensions vfio/pci: Route every reset point through a try-reset helper vfio/pci: Revoke and re-arm one device-specific region vfio/pci: Call the device provider across low-power transitions vfio/cxl: Add the vfio-cxl provider module skeleton vfio/cxl: Create the CXL memdev and set media ready at bind vfio/cxl: Own the whole component register BAR
vfio/cxl: Expose the HDM memory region to the guest vfio/cxl: Allow mmap of the HDM memory region
vfio/cxl: Contain HDM memory errors with memory_failure() vfio/cxl: Expose the HDM decoder registers read-only to the guest vfio/cxl: Exclude the HDM decoder registers from direct BAR access vfio/cxl: Restore the HDM decoder after a hot reset vfio/cxl: Describe the CXL device and decoder geometry to userspace vfio/cxl: Export the HDM memory region as a dma-buf vfio/cxl: Run the CXL reset at the vfio reset points
Documentation: vfio-pci: Document CXL Type-2 device passthrough selftests/vfio: Add CXL Type-2 passthrough tests
Documentation/driver-api/index.rst | 1 + Documentation/driver-api/vfio-pci-cxl.rst | 312 +++++ MAINTAINERS | 10 + drivers/cxl/core/hdm_state.c | 22 +- drivers/cxl/core/regs.c | 64 +- drivers/cxl/cxl.h | 47 +- drivers/pci/cxl.c | 136 +- drivers/pci/pci.c | 31 +- drivers/pci/pci.h | 18 +- drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/cxl/Kconfig | 11 + drivers/vfio/pci/cxl/Makefile | 3 + drivers/vfio/pci/cxl/vfio_cxl_core.c | 1124 +++++++++++++++++ drivers/vfio/pci/vfio_pci.c | 52 +- drivers/vfio/pci/vfio_pci_config.c | 225 +++- drivers/vfio/pci/vfio_pci_core.c | 747 ++++++++++- drivers/vfio/pci/vfio_pci_dmabuf.c | 183 ++- drivers/vfio/pci/vfio_pci_priv.h | 20 + drivers/vfio/pci/vfio_pci_rdwr.c | 76 +- include/cxl/cxl.h | 2 + include/cxl/hdm.h | 3 +- include/cxl/pci.h | 42 + include/linux/vfio_pci_core.h | 92 ++ include/uapi/cxl/cxl_regs.h | 56 + include/uapi/linux/vfio.h | 24 + tools/testing/selftests/vfio/Makefile | 1 + .../vfio/lib/include/libvfio/iommu.h | 3 + tools/testing/selftests/vfio/lib/iommu.c | 29 + .../selftests/vfio/lib/vfio_pci_device.c | 57 +- .../selftests/vfio/vfio_cxl_type2_test.c | 1095 ++++++++++++++++ 31 files changed, 4306 insertions(+), 184 deletions(-) create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst create mode 100644 drivers/vfio/pci/cxl/Kconfig create mode 100644 drivers/vfio/pci/cxl/Makefile create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_core.c create mode 100644 include/uapi/cxl/cxl_regs.h create mode 100644 tools/testing/selftests/vfio/vfio_cxl_type2_test.c
Signature · 1 line
2.53.0
PATCH v6 1/31 cxl/regs: Split the BAR block request and ioremap helpers
From: Manish Honap <[email protected]>
devm_cxl_iomap_block() claims a register block with devm_request_mem_region() and then maps it with devm_ioremap().
A driver that already owns the parent BAR needs to skip the sub-block request while still mapping it, so the request and the map must be callable independently.
Split them into devm_cxl_request_block() and devm_cxl_ioremap_block(). devm_cxl_iomap_block() keeps calling both in turn, so there is no functional change.
--- drivers/cxl/core/regs.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/cxl/core/regs.c +16 -2
@@ -177,6 +177,20 @@ void cxl_probe_device_regs(struct device *dev, void __iomem *base, }EXPORT_SYMBOL_NS_GPL (cxl_probe_device_regs,"CXL" ); +static struct resource *devm_cxl_request_block (struct device *dev, +resource_size_t addr, +resource_size_t length) +{ +return devm_request_mem_region (dev, addr, length,dev_name (dev)); +} + +static void __iomem *devm_cxl_ioremap_block (struct device *dev, +resource_size_t addr, +resource_size_t length) +{ +return devm_ioremap (dev, addr, length); +} +void __iomem *devm_cxl_iomap_block (struct device *dev,resource_size_t addr,resource_size_t length) {
@@ -186,7 +200,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,if (WARN_ON_ONCE (addr ==CXL_RESOURCE_NONE ))return NULL ; - res =devm_request_mem_region (dev, addr, length,dev_name (dev)); + res =devm_cxl_request_block (dev, addr, length);if (!res) {resource_size_t end = addr + length -1 ;
@@ -194,7 +208,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,return NULL ; } - ret_val =devm_ioremap (dev, addr, length); + ret_val =devm_cxl_ioremap_block (dev, addr, length);if (!ret_val)dev_err (dev,"Failed to map region %pr\n" , res);
Signature · 1 line
2.53.0
PATCH v6 2/31 cxl/regs: Let a BAR-owning driver own the component register block
In reply to [email protected]From: Manish Honap <[email protected]>
cxl_map_component_regs() claims each mapped sub-block with devm_request_mem_region(). A driver that requests the whole component register BAR itself, such as vfio-cxl, would then collide with that claim and fail to map the HDM decoder and RAS blocks.
Add cxl_reg_map_set_owned_resource() so such a driver records the resource it already owns, the whole component register BAR, on the register map.
Skip the sub-block request for a block inside that resource, and fail with -ENXIO for a block outside it.
--- drivers/cxl/core/regs.c | 46 +++++++++++++++++++++++++++++++++++++++-- include/cxl/cxl.h | 2 ++ include/cxl/pci.h | 3 +++ 3 files changed, 49 insertions(+), 2 deletions(-)
drivers/cxl/core/regs.c +44 -2
@@ -216,6 +216,39 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, }EXPORT_SYMBOL_NS_GPL (devm_cxl_iomap_block,"CXL" ); +/** +* cxl_reg_map_set_owned_resource() - record the BAR a driver owns +* @map: register map of the component register block +* @res: resource of the whole component register BAR, which the caller +* requests itself, before or after this call +* +* A driver that requests the whole component register BAR itself, such as +* vfio-cxl, records that BAR here. cxl_map_component_regs() then maps the +* blocks inside @res without requesting them again and refuses any block +* outside it. A register map has at most one owned resource. +* +* Call this after cxl_pci_setup_regs() has filled @map: enumeration rewrites +* the whole map and drops a previously recorded owned resource. +* +* Return: 0 on success, -EBUSY if an owned resource is already set, or +* -EINVAL if @map has not been set up or its block lies outside @res. +*/ +int cxl_reg_map_set_owned_resource (struct cxl_register_map *map, +struct resource *res) +{ +struct resource block =DEFINE_RES_MEM (map->resource, map->max_size); + +if (WARN_ON_ONCE (map->owned)) +return -EBUSY ; +if (WARN_ON_ONCE (map->resource ==CXL_RESOURCE_NONE || + !resource_contains (res, &block))) +return -EINVAL ; + + map->owned = res; +return 0 ; +} +EXPORT_SYMBOL_NS_GPL (cxl_reg_map_set_owned_resource,"CXL" ); +int cxl_map_component_regs (const struct cxl_register_map *map,struct cxl_component_regs *regs,unsigned long map_mask)
@@ -234,6 +267,7 @@ int cxl_map_component_regs(const struct cxl_register_map *map,struct mapinfo *mi = &mapinfo[i];resource_size_t addr;resource_size_t length; +struct resource res;if (!mi->rmap->valid)continue ;
@@ -241,8 +275,16 @@ int cxl_map_component_regs(const struct cxl_register_map *map,continue ; addr = map->resource + mi->rmap->offset; length = mi->rmap->size; - *(mi->addr) =devm_cxl_iomap_block (host, addr, length); -if (!*(mi->addr)) + res =DEFINE_RES_MEM (addr, length); + +if (map->owned) { +if (!resource_contains (map->owned, &res)) +return -ENXIO ; + *mi->addr =devm_cxl_ioremap_block (host, addr, length); + }else { + *mi->addr =devm_cxl_iomap_block (host, addr, length); + } +if (!*mi->addr)return -ENOMEM ; }
include/cxl/cxl.h +2 -0
@@ -153,6 +153,7 @@ struct cxl_pmu_reg_map {* @bar: PCI BAR containing the register block * @bar_offset: register block offset relative to @bar * @reg_type: see enum cxl_regloc_type +* @owned: driver-owned resource whose sub-blocks are mapped without re-requesting * @component_map: cxl_reg_map for component registers * @device_map: cxl_reg_maps for device registers * @pmu_map: cxl_reg_maps for CXL Performance Monitoring Units
@@ -165,6 +166,7 @@ struct cxl_register_map {int bar;resource_size_t bar_offset;u8 reg_type; +struct resource *owned;union {struct cxl_component_reg_map component_map;struct cxl_device_reg_map device_map;
include/cxl/pci.h +3 -0
@@ -18,9 +18,12 @@ enum cxl_regloc_type {struct cxl_register_map ;struct pci_dev ; +struct resource ;int cxl_pci_setup_regs (struct pci_dev *pdev,enum cxl_regloc_type type,struct cxl_register_map *map); +int cxl_reg_map_set_owned_resource (struct cxl_register_map *map, +struct resource *res);int cxl_find_regblock (struct pci_dev *pdev,enum cxl_regloc_type type,struct cxl_register_map *map);int cxl_setup_regs (struct cxl_register_map *map);
Signature · 1 line
2.53.0
PATCH v6 3/31 cxl: Move component register defines to uapi/cxl/cxl_regs.h
In reply to [email protected]From: Manish Honap <[email protected]>
The CXL component and HDM decoder register layout is spec-defined and is needed by a userspace consumer that cannot include the private drivers/cxl/cxl.h
The vfio-cxl selftest (tools/testing/selftests/vfio/vfio_cxl_type2_test.c) builds against installed uapi headers only. Move those defines to a new uapi header, the way PCI register offsets live in uapi pci_regs.h, and include it from drivers/cxl/cxl.h so existing in-kernel users are unchanged.
The section references in the moved comments are updated to CXL r4.0.
No functional change.
--- MAINTAINERS | 1 + drivers/cxl/cxl.h | 47 +------------------------------ include/uapi/cxl/cxl_regs.h | 56 +++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 46 deletions(-) create mode 100644 include/uapi/cxl/cxl_regs.h
MAINTAINERS +1 -0
@@ -6575,6 +6575,7 @@ F: Documentation/userspace-api/fwctl/fwctl-cxl.rst F: drivers/cxl/ F: drivers/pci/cxl.c F: include/cxl/ +F: include/uapi/cxl/ F: include/uapi/linux/cxl_mem.h F: tools/testing/cxl/
drivers/cxl/cxl.h +1 -46
59 lines not on this page · raw message
include/uapi/cxl/cxl_regs.h new file +56 -0
56 lines not on this page · raw message
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 03/31] cxl: Move component register defines to uapi/cxl/cxl_regs.h
Quoted 12 lines · On Fri, 16 Oct 2026 21:48:10 +0530 <[email protected]> wrote:
include/uapi/cxl/cxl_regs.h new file +6
-0@@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* Copyright (c) 2026 NVIDIA Corporation & Affiliates */ + +#ifndef _UAPI_CXL_REGS_H_ +#define _UAPI_CXL_REGS_H_ +
include/uapi/cxl/cxl_regs.h line 16
Quoted 10 lines
include/uapi/cxl/cxl_regs.h new file +10
-0+#include <linux/bits.h> +#include <linux/const.h> + +/* CXL r4.0 sec 8.2.3 Component Register Layout and Definition */ +#define CXL_COMPONENT_REG_BLOCK_SIZE 0x10000 + +/* CXL r4.0 sec 8.2.4 CXL.cache and CXL.mem Registers */ +#define CXL_CM_OFFSET 0x1000 +#define CXL_CM_CAP_HDR_OFFSET 0x0 +#define CXL_CM_CAP_HDR_ID_MASK __GENMASK(15, 0)
A local sashiko review asks if this new uapi header should include asm/bitsperlong.h since __GENMASK requires __BITS_PER_LONG. We include the other two header dependencies, but not this one, so it's a bit self-inconsistent.
BTW, current guidance on the Assisted-by tag: 1e035297ffda ("docs:
remove Assisted-by references"). Thanks,
Alex
RE: [PATCH v6 03/31] cxl: Move component register defines to uapi/cxl/cxl_regs.h
Quoted 44 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Thursday, November 5, 2026 5:23 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 03/31] cxl: Move component register defines touapi/cxl/cxl_regs.hExternal email: Use caution opening links or attachments
Quoted 17 lines · On Fri, 16 Oct 2026 21:48:10 +0530 <[email protected]> wrote:
include/uapi/cxl/cxl_regs.h new file +12
-0@@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* Copyright (c) 2026 NVIDIA Corporation & Affiliates */ + +#ifndef _UAPI_CXL_REGS_H_ +#define _UAPI_CXL_REGS_H_ + +#include <linux/bits.h> +#include <linux/const.h> + +/* CXL r4.0 sec 8.2.3 Component Register Layout and Definition */ +#define CXL_COMPONENT_REG_BLOCK_SIZE 0x10000 +
Quoted 9 lines
include/uapi/cxl/cxl_regs.h line 15
Quoted 3 lines
include/uapi/cxl/cxl_regs.h new file +3
-0+/* CXL r4.0 sec 8.2.4 CXL.cache and CXL.mem Registers */ #define +CXL_CM_OFFSET 0x1000 #define CXL_CM_CAP_HDR_OFFSET 0x0 +#define CXL_CM_CAP_HDR_ID_MASK __GENMASK(15, 0)A local sashiko review asks if this new uapi header should include
asm/bitsperlong.hsince__GENMASKrequires__BITS_PER_LONG. We include the other two header dependencies, but not this one, so it's a bit self- inconsistent.
Okay, I will add it in v7.
Quoted 4 lines
BTW, current guidance on the Assisted-by tag:
1e035297ffda("docs:
remove Assisted-by references"). Thanks,Alex
Okay, I will drop this in v7 and update the cover letter as per new guidance.
Thanks,
Manish
PATCH v6 4/31 PCI/CXL: Let the driver that owns the HDM range run CXL Reset
In reply to [email protected]From: Manish Honap <[email protected]>
vfio-cxl passes a CXL Type-2 device to a guest and must run CXL Reset at the vfio reset points. The cxl_reset PCI reset method does not fit that caller:
- It requests every HPA range the device decodes. vfio-cxl holds that range with request_mem_region_exclusive() while it exposes the memory, so the request fails with -EBUSY.
- When the reset cannot run, the method returns -ENOTTY and the PCI core moves on to the next method. That is FLR, which leaves device memory intact for the next user.
- The snapshot and restore helpers that vfio-cxl needs after a bus reset and on return from low power are not exported.
Let cxl_hdm_quiesce_prepare() take the resource the caller already holds and skip the request for ranges inside it. pci_cxl_reset_owned() accepts only a resource marked busy and exclusive, as request_mem_region_exclusive() leaves it. The CPU caches are still written back and invalidated for every range before and after the reset, as the reset method does.
Add:
- pci_cxl_reset_owned(): runs the reset method's sequence for a caller that holds the device lock and returns any failure to that caller
- pci_cxl_reset_capable(): checks the cached HDM state and the CPU cache invalidation that the reset-method probe skips.
- pci_cxl_hdm_snapshot(): copies the HDM state cached at enumeration without requiring the device lock. A driver needs this copy in probe, which the PCI core may run in a worker on the device's NUMA node, where device_lock_assert() would warn although the binding task holds the lock. pci_cxl_reset_prepare() becomes that copy plus the lock assertion.
Export:
- pci_cxl_reset_owned()
- pci_cxl_reset_capable()
- pci_cxl_hdm_snapshot()
- cxl_restore_state_after_pci_reset()
to the vfio-cxl module only, and declare them in include/cxl/pci.h.
CXL Reset always clears memory, so there is no caller Memory Clear choice.
--- drivers/cxl/core/hdm_state.c | 22 ++++-- drivers/pci/cxl.c | 136 ++++++++++++++++++++++++++++------- drivers/pci/pci.h | 18 +---- include/cxl/hdm.h | 3 +- include/cxl/pci.h | 39 ++++++++++ 5 files changed, 171 insertions(+), 47 deletions(-)
drivers/cxl/core/hdm_state.c +17 -5
@@ -267,7 +267,8 @@ static int cxl_hdm_ranges_collect(struct cxl_hdm_quiesce *q, struct device *dev,return 0 ; } -static int cxl_hdm_ranges_request (struct cxl_hdm_quiesce *q) +static int cxl_hdm_ranges_request (struct cxl_hdm_quiesce *q, +const struct resource *owned) {struct cxl_hdm_range *range;
@@ -275,6 +276,12 @@ static int cxl_hdm_ranges_request(struct cxl_hdm_quiesce *q)list_for_each_entry (range, &q->ranges, list) {const struct range *hpa_range = &range->hpa_range; +struct resource res =DEFINE_RES_MEM (hpa_range->start, + range->len); + +/* The caller already holds this range exclusively. */ +if (owned &&resource_contains (owned, &res)) +continue ; range->res =request_mem_region (hpa_range->start, range->len,"cxl_reset" );
@@ -316,15 +323,20 @@ static int cxl_hdm_ranges_invalidate(struct cxl_hdm_quiesce *q)* @dev: device for error messages * @q: quiesce state to initialize * @snapshot: HDM state from cxl_hdm_cache_snapshot() +* @owned: resource the caller holds exclusively over the ranges, or NULL * * Reserve every enabled system-physical range in @snapshot exclusively, then -* write back and invalidate CPU caches for them. Call -* cxl_hdm_quiesce_destroy() afterwards whether or not this succeeds. +* write back and invalidate CPU caches for them. A range inside @owned is +* not requested again; the caller's reservation already excludes other +* users. +* +* Call cxl_hdm_quiesce_destroy() afterwards whether or not this succeeds. * * Return: 0 on success, or a negative errno. */ int cxl_hdm_quiesce_prepare (struct device *dev,struct cxl_hdm_quiesce *q, -const struct cxl_hdm_info *snapshot) +const struct cxl_hdm_info *snapshot, +const struct resource *owned) {int rc;
@@ -340,7 +352,7 @@ int cxl_hdm_quiesce_prepare(struct device *dev, struct cxl_hdm_quiesce *q,if (rc)return rc; - rc =cxl_hdm_ranges_request (q); + rc =cxl_hdm_ranges_request (q, owned);if (rc)return rc;
drivers/pci/cxl.c +111 -25
197 lines not on this page · raw message
drivers/pci/pci.h +2 -16
@@ -730,14 +730,12 @@ static inline void pci_doe_destroy(struct pci_dev *pdev) { }static inline void pci_doe_disconnected (struct pci_dev *pdev) { }#endif -struct cxl_hdm_info ; +#include <cxl/pci.h> +#ifdef CONFIG_CXL_RESET void pci_cxl_hdm_cache_init (struct pci_dev *pdev);void pci_cxl_hdm_cache_release (struct pci_dev *pdev);int cxl_reset_function (struct pci_dev *pdev,bool probe); -struct cxl_hdm_info *pci_cxl_reset_prepare (struct pci_dev *pdev); -int cxl_restore_state_after_pci_reset (struct pci_dev *pdev, -struct cxl_hdm_info *snapshot);#else static inline void pci_cxl_hdm_cache_init (struct pci_dev *pdev) { }static inline void pci_cxl_hdm_cache_release (struct pci_dev *pdev) { }
@@ -745,18 +743,6 @@ static inline int cxl_reset_function(struct pci_dev *pdev, bool probe) {return -ENOTTY ; } - -static inline struct cxl_hdm_info *pci_cxl_reset_prepare (struct pci_dev *pdev) -{ -return NULL ; -} - -static inline int -cxl_restore_state_after_pci_reset (struct pci_dev *pdev, -struct cxl_hdm_info *snapshot) -{ -return 0 ; -}#endif #ifdef CONFIG_PCI_NPEM
include/cxl/hdm.h +2 -1
@@ -96,7 +96,8 @@ struct cxl_hdm_quiesce {void cxl_region_quiesce_lock (void );void cxl_region_quiesce_unlock (void ); int cxl_hdm_quiesce_prepare(struct device *dev, struct cxl_hdm_quiesce *q, - const struct cxl_hdm_info *snapshot) + const struct cxl_hdm_info *snapshot, + const struct resource *owned) __must_hold(cxl_region_quiesce); int cxl_hdm_quiesce_flush(struct cxl_hdm_quiesce *q) __must_hold(cxl_region_quiesce);
include/cxl/pci.h +39 -0
@@ -4,6 +4,7 @@#ifndef __CXL_CXL_PCI_H__#define __CXL_CXL_PCI_H__ +#include <linux/errno.h> #include <linux/types.h> /* Register Block Identifier (RBI) */
@@ -27,6 +28,44 @@ int cxl_reg_map_set_owned_resource(struct cxl_register_map *map,int cxl_find_regblock (struct pci_dev *pdev,enum cxl_regloc_type type,struct cxl_register_map *map);int cxl_setup_regs (struct cxl_register_map *map); + +struct cxl_hdm_info ; +#ifdef CONFIG_CXL_RESET +struct cxl_hdm_info *pci_cxl_hdm_snapshot (struct pci_dev *pdev); +struct cxl_hdm_info *pci_cxl_reset_prepare (struct pci_dev *pdev); +int cxl_restore_state_after_pci_reset (struct pci_dev *pdev, +struct cxl_hdm_info *snapshot); +bool pci_cxl_reset_capable (struct pci_dev *pdev); +int pci_cxl_reset_owned (struct pci_dev *pdev,const struct resource *owned); +#else +static inline struct cxl_hdm_info *pci_cxl_hdm_snapshot (struct pci_dev *pdev) +{ +return NULL ; +} + +static inline struct cxl_hdm_info *pci_cxl_reset_prepare (struct pci_dev *pdev) +{ +return NULL ; +} + +static inline int +cxl_restore_state_after_pci_reset (struct pci_dev *pdev, +struct cxl_hdm_info *snapshot) +{ +return 0 ; +} + +static inline bool pci_cxl_reset_capable (struct pci_dev *pdev) +{ +return false; +} + +static inline int pci_cxl_reset_owned (struct pci_dev *pdev, +const struct resource *owned) +{ +return -ENOTTY ; +} +#endif /* * CXL r4.0 sec 8.1.3.1: a CXL.mem capable device reports one or two HDM * ranges. 00b is illegal when Mem_Capable is set and 11b is reserved.
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 04/31] PCI/CXL: Let the driver that owns the HDM range run CXL Reset
Quoted 135 lines · On Fri, 16 Oct 2026 21:48:11 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio-cxl passes a CXL Type-2 device to a guest and must run CXL Reset at the vfio reset points. The
cxl_resetPCI reset method does not fit that caller:- It requests every HPA range the device decodes. vfio-cxl holds that range with
request_mem_region_exclusive()while it exposes the memory, so the request fails with-EBUSY.
- When the reset cannot run, the method returns-ENOTTYand the PCI core moves on to the next method. That is FLR, which leaves device memory intact for the next user.
- The snapshot and restore helpers that vfio-cxl needs after a bus reset and on return from low power are not exported.Let
cxl_hdm_quiesce_prepare()take the resource the caller already holds and skip the request for ranges inside it.pci_cxl_reset_owned()accepts only a resource marked busy and exclusive, asrequest_mem_region_exclusive()leaves it. The CPU caches are still written back and invalidated for every range before and after the reset, as the reset method does.Add:
-pci_cxl_reset_owned(): runs the reset method's sequence for a caller that holds the device lock and returns any failure to that caller-
pci_cxl_reset_capable(): checks the cached HDM state and the CPU cache invalidation that the reset-method probe skips.-
pci_cxl_hdm_snapshot(): copies the HDM state cached at enumeration without requiring the device lock. A driver needs this copy in probe, which the PCI core may run in a worker on the device's NUMA node, wheredevice_lock_assert()would warn although the binding task holds the lock.pci_cxl_reset_prepare()becomes that copy plus the lock assertion.Export:
-pci_cxl_reset_owned()
-pci_cxl_reset_capable()
-pci_cxl_hdm_snapshot()
-cxl_restore_state_after_pci_reset()to the vfio-cxl module only, and declare them in
include/cxl/pci.h.CXL Reset always clears memory, so there is no caller Memory Clear choice.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/cxl/core/hdm_state.c | 22 ++++-- drivers/pci/cxl.c | 136 ++++++++++++++++++++++++++++------- drivers/pci/pci.h | 18 +---- include/cxl/hdm.h | 3 +- include/cxl/pci.h | 39 ++++++++++ 5 files changed, 171 insertions(+), 47 deletions(-)drivers/cxl/core/hdm_state.c +17
-5@@ -267,7 +267,8 @@ static int cxl_hdm_ranges_collect(struct cxl_hdm_quiesce *q, struct device *dev,return 0 ; } -static int cxl_hdm_ranges_request (struct cxl_hdm_quiesce *q) +static int cxl_hdm_ranges_request (struct cxl_hdm_quiesce *q, +const struct resource *owned) {struct cxl_hdm_range *range;@@ -275,6 +276,12 @@ static int cxl_hdm_ranges_request(struct cxl_hdm_quiesce *q)list_for_each_entry (range, &q->ranges, list) {const struct range *hpa_range = &range->hpa_range; +struct resource res =DEFINE_RES_MEM (hpa_range->start, + range->len); + +/* The caller already holds this range exclusively. */ +if (owned &&resource_contains (owned, &res)) +continue ; range->res =request_mem_region (hpa_range->start, range->len,"cxl_reset" );@@ -316,15 +323,20 @@ static int cxl_hdm_ranges_invalidate(struct cxl_hdm_quiesce *q)* @dev: device for error messages * @q: quiesce state to initialize * @snapshot: HDM state from cxl_hdm_cache_snapshot() +* @owned: resource the caller holds exclusively over the ranges, or NULL * * Reserve every enabled system-physical range in @snapshot exclusively, then -* write back and invalidate CPU caches for them. Call -* cxl_hdm_quiesce_destroy() afterwards whether or not this succeeds. +* write back and invalidate CPU caches for them. A range inside @owned is +* not requested again; the caller's reservation already excludes other +* users. +* +* Call cxl_hdm_quiesce_destroy() afterwards whether or not this succeeds. * * Return: 0 on success, or a negative errno. */ int cxl_hdm_quiesce_prepare (struct device *dev,struct cxl_hdm_quiesce *q, -const struct cxl_hdm_info *snapshot) +const struct cxl_hdm_info *snapshot, +const struct resource *owned) {int rc;@@ -340,7 +352,7 @@ int cxl_hdm_quiesce_prepare(struct device *dev, struct cxl_hdm_quiesce *q,if (rc)return rc; - rc =cxl_hdm_ranges_request (q); + rc =cxl_hdm_ranges_request (q, owned);if (rc)return rc;drivers/pci/cxl.c +12
-0@@ -372,9 +372,32 @@ static int cxl_reset_save_restored_state(struct pci_dev *pdev, u16 command)return rc; } +/** +* pci_cxl_hdm_snapshot() - copy the HDM state cached at enumeration +* @pdev: CXL device +* +* The cache is read under cxl_rwsem.dpa, so the caller needs no device +* lock. A driver can call this from probe, which the PCI core may run in a +* worker on the device's NUMA node while the binding task holds the lock. +* +* Return: a copy to free with kfree(); NULL when @pdev has no cached HDM +* state; or an ERR_PTR() on failure. +*/ +struct cxl_hdm_info *pci_cxl_hdm_snapshot (struct pci_dev *pdev)
drivers/pci/cxl.c static int cxl_reset_save_restored_state(struct pci_dev *pdev, u16 command) · line 396
Quoted 10 lines
drivers/pci/cxl.c +10
-0+{ +struct cxl_hdm_info *snapshot; + + snapshot =cxl_hdm_cache_snapshot (&pdev->hdm); +if (IS_ERR (snapshot) &&PTR_ERR (snapshot) == -ENXIO ) +return NULL ; + +return snapshot; +} +EXPORT_SYMBOL_FOR_MODULES (pci_cxl_hdm_snapshot,"vfio-cxl" );
Why are any of these module locked? What great risk do they expose that only vfio-cxl should have access to versus just a NS export?
We shouldn't set a precedent for vfio specific APIs and I certainly don't want to be defending such a thing down the road.
It's unfortunate that we're intermixing "snapshot" from CXL but using it like a "save state" from PCI. Maybe we could bridge the gap at the PCI API?
Quoted 167 lines
drivers/pci/cxl.c +83
-23+/** * pci_cxl_reset_prepare() - allocate HDM state storage ahead of a bus reset -* @pdev: device about to be reset +* @pdev: device about to be reset, with its device lock held * * Copy the cached HDM state before the reset, so that restoring cannot fail * for lack of memory. The decoder settings are refreshed when the state is @@ -386,14 +409,8 @@ static int cxl_reset_save_restored_state(struct pci_dev *pdev, u16 command)*/ struct cxl_hdm_info *pci_cxl_reset_prepare (struct pci_dev *pdev) { -struct cxl_hdm_info *snapshot; -device_lock_assert (&pdev->dev); - snapshot =cxl_hdm_cache_snapshot (&pdev->hdm); -if (IS_ERR (snapshot) &&PTR_ERR (snapshot) == -ENXIO ) -return NULL ; - -return snapshot; +return pci_cxl_hdm_snapshot (pdev); }/** @@ -433,6 +450,7 @@ int cxl_restore_state_after_pci_reset(struct pci_dev *pdev,cxl_region_quiesce_unlock ();return rc; } +EXPORT_SYMBOL_FOR_MODULES (cxl_restore_state_after_pci_reset,"vfio-cxl" );/* * CXL r4.0 sec 9.7.2 defines the reset completion timeout encodings. @@ -642,7 +660,8 @@ static int __cxl_reset_execute(struct pci_dev *pdev, int dvsec, u16 cap,return rc; } -static int cxl_reset_quiesced (struct pci_dev *pdev,int dvsec,u16 cap) +static int cxl_reset_quiesced (struct pci_dev *pdev,int dvsec,u16 cap, +const struct resource *owned)__must_hold (cxl_region_quiesce) { struct cxl_hdm_info *snapshot __free(kfree) = NULL;@@ -663,7 +682,7 @@ static int cxl_reset_quiesced(struct pci_dev *pdev, int dvsec, u16 cap)return PTR_ERR (snapshot); } - rc =cxl_hdm_quiesce_prepare (&pdev->dev, &q, snapshot); + rc =cxl_hdm_quiesce_prepare (&pdev->dev, &q, snapshot, owned);if (rc) {cxl_hdm_quiesce_destroy (&q);return rc;@@ -686,32 +705,40 @@ static int cxl_reset_quiesced(struct pci_dev *pdev, int dvsec, u16 cap)return cxl_reset_save_restored_state (pdev, command); } -static int cxl_reset_execute (struct pci_dev *pdev,int dvsec,u16 cap) +static int cxl_reset_execute (struct pci_dev *pdev,int dvsec,u16 cap, +const struct resource *owned) {int rc; +/* Raw reset callers may not have saved the PCI state needed for restore. */ +if (!pdev->state_saved) { +pci_err (pdev,"CXL reset requires saved PCI state\n" ); +return -EINVAL ; + } +cxl_region_quiesce_lock (); - rc =cxl_reset_quiesced (pdev, dvsec, cap); + rc =cxl_reset_quiesced (pdev, dvsec, cap, owned);cxl_region_quiesce_unlock ();return rc; } -int cxl_reset_function (struct pci_dev *pdev,bool probe) +static int cxl_reset_check (struct pci_dev *pdev,bool probe,u16 *dvsec, +u16 *cap) { -u16 dvsec, cap, ctrl; +u16 ctrl;int rc; - rc =cxl_pci_device_dvsec (pdev, &dvsec, &cap); + rc =cxl_pci_device_dvsec (pdev, dvsec, cap);if (rc)return probe ? -ENOTTY : rc; -if (!(cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || - !(cap &PCI_DVSEC_CXL_MEM_CAPABLE ) || - !(cap &PCI_DVSEC_CXL_RST_CAPABLE ) || - !(cap &PCI_DVSEC_CXL_RST_MEM_CLR_CAPABLE )) +if (!(*cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || + !(*cap &PCI_DVSEC_CXL_MEM_CAPABLE ) || + !(*cap &PCI_DVSEC_CXL_RST_CAPABLE ) || + !(*cap &PCI_DVSEC_CXL_RST_MEM_CLR_CAPABLE ))return -ENOTTY ; - rc =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CTRL , &ctrl); + rc =pci_read_config_word (pdev, *dvsec +PCI_DVSEC_CXL_CTRL , &ctrl);if (rc)return probe ? -ENOTTY :pcibios_err_to_errno (rc);if (PCI_POSSIBLE_ERROR (ctrl))@@ -737,11 +764,70 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe)if (!cpu_cache_has_invalidate_memregion ())return -ENOTTY ; -/* Raw reset callers may not have saved the PCI state needed for restore. */ -if (!pdev->state_saved) { -pci_err (pdev,"CXL reset requires saved PCI state\n" ); +return 0 ; +} + +int cxl_reset_function (struct pci_dev *pdev,bool probe) +{ +u16 dvsec, cap; +int rc; + + rc =cxl_reset_check (pdev, probe, &dvsec, &cap); +if (rc || probe) +return rc; + +return cxl_reset_execute (pdev, dvsec, cap,NULL ); +} + +/** +* pci_cxl_reset_capable() - check whether CXL Reset can run on a device now +* @pdev: CXL device +* +* Unlike the reset-method probe, this also checks the cached HDM state and +* the CPU cache invalidation that the reset needs. +* +* Return: true when pci_cxl_reset_owned() can attempt the reset. +*/ +bool pci_cxl_reset_capable (struct pci_dev *pdev) +{ +u16 dvsec, cap; + +return !cxl_reset_check (pdev, false, &dvsec, &cap); +} +EXPORT_SYMBOL_FOR_MODULES (pci_cxl_reset_capable,"vfio-cxl" ); + +/** +* pci_cxl_reset_owned() - CXL Reset for the driver that owns the HDM range +* @pdev: CXL device, with its device lock held and PCI state saved +* @owned: resource the caller reserved over the device's HPA range with +* request_mem_region_exclusive(), or NULL +* +* Run the same sequence as the cxl_reset method. Ranges inside @owned are not +* requested again, so a driver that keeps the HPA range reserved while it +* exposes the memory can reset the device. The CPU caches are still +* invalidated for every range before and after the reset. +* +* Return: 0 on success, or a negative errno. Unlike the reset method, a +* device that cannot be reset returns an error to the caller instead of +* falling through to another reset method. -EINVAL if @owned is not a busy, +* exclusive reservation. +*/ +int pci_cxl_reset_owned (struct pci_dev *pdev,const struct resource *owned) +{ +const unsigned long held =IORESOURCE_BUSY |IORESOURCE_EXCLUSIVE ; +u16 dvsec, cap; +int rc; + +device_lock_assert (&pdev->dev); + +/* Only a busy, exclusive reservation keeps other users off the range. */ +if (owned &&WARN_ON_ONCE ((owned->flags & held) != held))
drivers/pci/cxl.c int cxl_reset_function(struct pci_dev *pdev, bool probe) · line 832
Quoted 10 lines
drivers/pci/cxl.c +5
-2return -EINVAL ; - } -return cxl_reset_execute (pdev, dvsec, cap); + rc =cxl_reset_check (pdev, false, &dvsec, &cap); +if (rc) +return rc; + +return cxl_reset_execute (pdev, dvsec, cap, owned); }
There's a lot of asymmetric sketchiness through this path. pdev->state_saved is required to be set by the caller, but the state gets restored and state_saved invalided deep inside cxl_restore_pci_state_for_hdm_restore() on this path. That's not at all symmetric to the PCI conventions and the split ownership model is poor. Even worse, it's vfio-pci-core doing the pci_save_state() on behalf of the cxl_ops.reset callback.
Where does bus-master get cleared? Seems we've only got half of pci_dev_save_and_disable() implemented here and the save has become the caller's responsibility. Thanks,
Alex
RE: [PATCH v6 04/31] PCI/CXL: Let the driver that owns the HDM range run CXL Reset
Quoted 189 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Thursday, November 5, 2026 5:24 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 04/31] PCI/CXL: Let the driver that owns the HDM range run CXL ResetExternal email: Use caution opening links or attachments
drivers/cxl/core/hdm_state.c static int cxl_hdm_ranges_collect(struct
Quoted 61 lines · On Fri, 16 Oct 2026 21:48:11 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio-cxl passes a CXL Type-2 device to a guest and must run CXL Reset at the vfio reset points. The
cxl_resetPCI reset method does not fit that
caller:- It requests every HPA range the device decodes. vfio-cxl holds that range with
request_mem_region_exclusive()while it exposes the memory, so the request fails with-EBUSY.
- When the reset cannot run, the method returns-ENOTTYand the PCI core moves on to the next method. That is FLR, which leaves device memory intact for the next user.
- The snapshot and restore helpers that vfio-cxl needs after a bus reset and on return from low power are not exported.Let
cxl_hdm_quiesce_prepare()take the resource the caller already holds and skip the request for ranges inside it.pci_cxl_reset_owned()accepts only a resource marked busy and exclusive, asrequest_mem_region_exclusive()leaves it. The CPU caches are still written back and invalidated for every range before and after the reset, as the reset method does.Add:
-pci_cxl_reset_owned(): runs the reset method's sequence for a caller that holds the device lock and returns any failure to that caller-
pci_cxl_reset_capable(): checks the cached HDM state and the CPU cache invalidation that the reset-method probe skips.-
pci_cxl_hdm_snapshot(): copies the HDM state cached at enumeration without requiring the device lock. A driver needs this copy in probe, which the PCI core may run in a worker on the device's NUMA node, wheredevice_lock_assert()would warn although the binding task holds the lock.pci_cxl_reset_prepare()becomes that copy plus the lock assertion.Export:
-pci_cxl_reset_owned()
-pci_cxl_reset_capable()
-pci_cxl_hdm_snapshot()
-cxl_restore_state_after_pci_reset()to the vfio-cxl module only, and declare them in
include/cxl/pci.h.CXL Reset always clears memory, so there is no caller Memory Clear choice.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/cxl/core/hdm_state.c | 22 ++++-- drivers/pci/cxl.c | 136 ++++++++++++++++++++++++++++------- drivers/pci/pci.h | 18 +---- include/cxl/hdm.h | 3 +- include/cxl/pci.h | 39 ++++++++++ 5 files changed, 171 insertions(+), 47 deletions(-)(quoted diff) +0
-0
b/drivers/cxl/core/hdm_state.cindex95daeda0ef80..9452047e3b97100644drivers/cxl/core/hdm_state.c +0
-0@@ -267,7 +267,8 @@ static int cxl_hdm_ranges_collect(struct
cxl_hdm_quiesce*q, struct device *dev,Quoted 23 lines
drivers/cxl/core/hdm_state.c +8
-1return 0 ; } -static int cxl_hdm_ranges_request (struct cxl_hdm_quiesce *q) +static int cxl_hdm_ranges_request (struct cxl_hdm_quiesce *q, +const struct resource *owned) {struct cxl_hdm_range *range;@@ -275,6 +276,12 @@ static int cxl_hdm_ranges_request(struct cxl_hdm_quiesce *q)list_for_each_entry (range, &q->ranges, list) {const struct range *hpa_range = &range->hpa_range; +struct resource res =DEFINE_RES_MEM (hpa_range->start, + range->len); + +/* The caller already holds this range exclusively. */ +if (owned &&resource_contains (owned, &res)) +continue ; range->res = request_mem_region(hpa_range->start, range->len,"cxl_reset"); @@ -316,15Quoted 1 line
+323,20 @@ static int cxl_hdm_ranges_invalidate(struct
cxl_hdm_quiesce*q)
drivers/cxl/core/hdm_state.c int cxl_hdm_quiesce_prepare(struct device *dev,
Quoted 28 lines
* @dev: device for error messages
* @q: quiesce state to initialize
* @snapshot: HDM state fromcxl_hdm_cache_snapshot()
+ * @owned: resource the caller holds exclusively over the ranges, or
+ NULL
*
* Reserve every enabled system-physical range in @snapshot
exclusively, then
- * write back and invalidate CPU caches for them. Call
- *cxl_hdm_quiesce_destroy()afterwards whether or not this succeeds.
+ * write back and invalidate CPU caches for them. A range inside
+ @owned is
+ * not requested again; the caller's reservation already excludes
+ other
+ * users.
+ *
+ * Callcxl_hdm_quiesce_destroy()afterwards whether or not this succeeds. *
* Return: 0 on success, or a negative errno.
*/
int cxl_hdm_quiesce_prepare(struct device *dev, structcxl_hdm_quiesce*q,drivers/cxl/core/hdm_state.c +2
-1- const struct cxl_hdm_info *snapshot) +const struct cxl_hdm_info *snapshot, + const struct resource *owned) { int rc;@@ -340,7 +352,7 @@ int cxl_hdm_quiesce_prepare(struct device *dev,struct
cxl_hdm_quiesce*q,drivers/pci/cxl.c static int cxl_reset_save_restored_state(struct
Quoted 13 lines
drivers/cxl/core/hdm_state.c +1
-1if (rc)return rc; - rc =cxl_hdm_ranges_request (q); + rc =cxl_hdm_ranges_request (q, owned);if (rc)return rc;drivers/pci/cxl.c index renamed from drivers/pci/cxl.c +0
-0
ed0b6f619c70..b4d467506c54100644drivers/pci/cxl.c +0
-0@@ -372,9 +372,32 @@ static int cxl_reset_save_restored_state(struct
pci_dev*pdev, u16 command)Quoted 17 lines
drivers/pci/cxl.c +14
-0return rc; } +/** +* pci_cxl_hdm_snapshot() - copy the HDM state cached at enumeration +* @pdev: CXL device +* +* The cache is read under cxl_rwsem.dpa, so the caller needs no +device +* lock. A driver can call this from probe, which the PCI core may +run in a +* worker on the device's NUMA node while the binding task holds the lock. +* +* Return: a copy to free with kfree(); NULL when @pdev has no cached +HDM +* state; or an ERR_PTR() on failure. +*/ drivers/pci/cxl.c static int cxl_reset_save_restored_state(struct pci_dev *pdev, u16 command) · line 396
Quoted 10 lines
drivers/pci/cxl.c +10
-0+struct cxl_hdm_info *pci_cxl_hdm_snapshot (struct pci_dev *pdev) { +struct cxl_hdm_info *snapshot; + + snapshot =cxl_hdm_cache_snapshot (&pdev->hdm); +if (IS_ERR (snapshot) &&PTR_ERR (snapshot) == -ENXIO ) +return NULL ; + +return snapshot; +} +EXPORT_SYMBOL_FOR_MODULES (pci_cxl_hdm_snapshot,"vfio-cxl" );Why are any of these module locked? What great risk do they expose that only vfio-cxl should have access to versus just a NS export?
Quoted 2 lines
We shouldn't set a precedent for vfio specific APIs and I certainly don't want to be defending such a thing down the road.
Hmm; let me reassess this in v7. I will try to model the reset code through PCI core. This will result in owner-aware reset and its exports to be removed.
Quoted 4 lines
It's unfortunate that we're intermixing "snapshot" from CXL but using it like a "save state" from PCI. Maybe we could bridge the gap at the PCI API?
For v7, I will drop the save state use. Vfio-cxl restores nothing itself. The only restore is the one cxl_reset method does inside its own reset.
Quoted 202 lines
Quoted 10 lines
drivers/pci/cxl.c +1
-0+/** * pci_cxl_reset_prepare() - allocate HDM state storage ahead of a bus reset
- * @pdev: device about to be reset
+ * @pdev: device about to be reset, with its device lock held
*
* Copy the cached HDM state before the reset, so that restoring cannot fail
* for lack of memory. The decoder settings are refreshed when the
state is @@ -386,14 +409,8 @@ static int
cxl_reset_save_restored_state(struct pci_dev *pdev, u16 command)drivers/pci/cxl.c int cxl_restore_state_after_pci_reset(struct pci_dev
Quoted 15 lines
*/
structcxl_hdm_info*pci_cxl_reset_prepare(structpci_dev*pdev) {drivers/pci/cxl.c +1
-7-struct cxl_hdm_info *snapshot; -device_lock_assert (&pdev->dev); - snapshot =cxl_hdm_cache_snapshot (&pdev->hdm); -if (IS_ERR (snapshot) &&PTR_ERR (snapshot) == -ENXIO ) -return NULL ; - -return snapshot; +return pci_cxl_hdm_snapshot (pdev); }/** @@ -433,6 +450,7 @@ int cxl_restore_state_after_pci_reset(struct pci_dev*pdev,
drivers/pci/cxl.c static int __cxl_reset_execute(struct pci_dev *pdev,
Quoted 9 lines
drivers/pci/cxl.c +2
-0cxl_region_quiesce_unlock ();return rc; } +EXPORT_SYMBOL_FOR_MODULES (cxl_restore_state_after_pci_reset, +"vfio-cxl" );/* * CXL r4.0 sec 9.7.2 defines the reset completion timeout encodings. @@ -642,7 +660,8 @@ static int __cxl_reset_execute(struct pci_dev *pdev,int dvsec, u16 cap,
Quoted 11 lines
drivers/pci/cxl.c +2
-1return rc; } -static int cxl_reset_quiesced (struct pci_dev *pdev,int dvsec,u16 cap) +static int cxl_reset_quiesced(struct pci_dev *pdev, int dvsec, u16 cap, +const struct resource *owned)__must_hold (cxl_region_quiesce) {struct cxl_hdm_info *snapshot __free(kfree) = NULL; @@ -663,7+682,7 @@ static int cxl_reset_quiesced(struct
pci_dev*pdev, int dvsec, u16cap)
drivers/pci/cxl.c static int cxl_reset_quiesced(struct pci_dev *pdev,
Quoted 9 lines
return PTR_ERR(snapshot); }drivers/pci/cxl.c +1
-1- rc =cxl_hdm_quiesce_prepare (&pdev->dev, &q, snapshot); + rc =cxl_hdm_quiesce_prepare (&pdev->dev, &q, snapshot, owned);if (rc) {cxl_hdm_quiesce_destroy (&q);return rc;@@ -686,32 +705,40 @@ static int cxl_reset_quiesced(struct pci_dev *pdev,int dvsec, u16 cap)
drivers/pci/cxl.c static int cxl_reset_quiesced(struct pci_dev *pdev, int dvsec, u16 cap)
Quoted 10 lines
drivers/pci/cxl.c +3
-1return cxl_reset_save_restored_state (pdev, command); } -static int cxl_reset_execute (struct pci_dev *pdev,int dvsec,u16 cap) +static int cxl_reset_execute(struct pci_dev *pdev, int dvsec, u16 cap, +const struct resource *owned) {int rc; +/* Raw reset callers may not have saved the PCI state needed for restore. */
drivers/pci/cxl.c int cxl_reset_function(struct pci_dev *pdev, bool
Quoted 42 lines
drivers/pci/cxl.c +16
-9+if (!pdev->state_saved) { +pci_err (pdev,"CXL reset requires saved PCI state\n" ); +return -EINVAL ; + } +cxl_region_quiesce_lock (); - rc =cxl_reset_quiesced (pdev, dvsec, cap); + rc =cxl_reset_quiesced (pdev, dvsec, cap, owned);cxl_region_quiesce_unlock ();return rc; } -int cxl_reset_function (struct pci_dev *pdev,bool probe) +static int cxl_reset_check (struct pci_dev *pdev,bool probe,u16 *dvsec, +u16 *cap) { -u16 dvsec, cap, ctrl; +u16 ctrl;int rc; - rc =cxl_pci_device_dvsec (pdev, &dvsec, &cap); + rc =cxl_pci_device_dvsec (pdev, dvsec, cap);if (rc)return probe ? -ENOTTY : rc; -if (!(cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || - !(cap &PCI_DVSEC_CXL_MEM_CAPABLE ) || - !(cap &PCI_DVSEC_CXL_RST_CAPABLE ) || - !(cap &PCI_DVSEC_CXL_RST_MEM_CLR_CAPABLE )) +if (!(*cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || + !(*cap &PCI_DVSEC_CXL_MEM_CAPABLE ) || + !(*cap &PCI_DVSEC_CXL_RST_CAPABLE ) || + !(*cap &PCI_DVSEC_CXL_RST_MEM_CLR_CAPABLE ))return -ENOTTY ; - rc =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CTRL , &ctrl); + rc =pci_read_config_word (pdev, *dvsec +PCI_DVSEC_CXL_CTRL , + &ctrl);if (rc)return probe ? -ENOTTY :pcibios_err_to_errno (rc);if (PCI_POSSIBLE_ERROR (ctrl))@@ -737,11 +764,70 @@ int cxl_reset_function(struct pci_dev *pdev, boolprobe)
drivers/pci/cxl.c int cxl_reset_function(struct pci_dev *pdev, bool probe)
Quoted 4 lines
drivers/pci/cxl.c +0
-1if (!cpu_cache_has_invalidate_memregion ())return -ENOTTY ; -/* Raw reset callers may not have saved the PCI state needed for restore. */
Quoted 47 lines
drivers/pci/cxl.c +45
-2-if (!pdev->state_saved) { -pci_err (pdev,"CXL reset requires saved PCI state\n" ); +return 0 ; +} + +int cxl_reset_function (struct pci_dev *pdev,bool probe) { +u16 dvsec, cap; +int rc; + + rc =cxl_reset_check (pdev, probe, &dvsec, &cap); +if (rc || probe) +return rc; + +return cxl_reset_execute (pdev, dvsec, cap,NULL ); } + +/** +* pci_cxl_reset_capable() - check whether CXL Reset can run on a +device now +* @pdev: CXL device +* +* Unlike the reset-method probe, this also checks the cached HDM +state and +* the CPU cache invalidation that the reset needs. +* +* Return: true when pci_cxl_reset_owned() can attempt the reset. +*/ +bool pci_cxl_reset_capable (struct pci_dev *pdev) { +u16 dvsec, cap; + +return !cxl_reset_check (pdev, false, &dvsec, &cap); } +EXPORT_SYMBOL_FOR_MODULES (pci_cxl_reset_capable,"vfio-cxl" ); + +/** +* pci_cxl_reset_owned() - CXL Reset for the driver that owns the HDM +range +* @pdev: CXL device, with its device lock held and PCI state saved +* @owned: resource the caller reserved over the device's HPA range with +* request_mem_region_exclusive(), or NULL +* +* Run the same sequence as the cxl_reset method. Ranges inside +@owned are not +* requested again, so a driver that keeps the HPA range reserved +while it +* exposes the memory can reset the device. The CPU caches are still +* invalidated for every range before and after the reset. +* +* Return: 0 on success, or a negative errno. Unlike the reset drivers/pci/cxl.c int cxl_reset_function(struct pci_dev *pdev, bool probe)
Quoted 10 lines
drivers/pci/cxl.c +10
-0+method, a +* device that cannot be reset returns an error to the caller instead +of +* falling through to another reset method. -EINVAL if @owned is not +a busy, +* exclusive reservation. +*/ +int pci_cxl_reset_owned (struct pci_dev *pdev,const struct resource +*owned) { + const unsigned long held = IORESOURCE_BUSY |
IORESOURCE_EXCLUSIVE;Quoted 7 lines
drivers/pci/cxl.c +7
-0+u16 dvsec, cap; +int rc; + +device_lock_assert (&pdev->dev); + +/* Only a busy, exclusive reservation keeps other users off the range. */ +if (owned &&WARN_ON_ONCE ((owned->flags & held) != held))drivers/pci/cxl.c int cxl_reset_function(struct pci_dev *pdev, bool probe) · line 832
Quoted 10 lines
drivers/pci/cxl.c +5
-2return -EINVAL ; - } -return cxl_reset_execute (pdev, dvsec, cap); + rc =cxl_reset_check (pdev, false, &dvsec, &cap); +if (rc) +return rc; + +return cxl_reset_execute (pdev, dvsec, cap, owned); }There's a lot of asymmetric sketchiness through this path.
pdev->state_savedis required to be set by the caller, but the state gets restored andstate_savedinvalided deep insidecxl_restore_pci_state_for_hdm_restore()on this path. That's not at all symmetric to the PCI conventions and the split ownership model is poor. Even worse, it's vfio-pci-core doing thepci_save_state()on behalf of the cxl_ops.reset callback.
Quoted 4 lines
Where does bus-master get cleared? Seems we've only got half of
pci_dev_save_and_disable()implemented here and the save has become the caller's responsibility. Thanks,
It is not getting cleared on this path. I think this a bug in this patch and needs some more time from me. For v7, I will make sure every reset goes through the PCI core as vfio already does for other cases.
Thanks,
Manish
Quoted 1 line
Alex
PATCH v6 5/31 PCI: Refuse bus and slot reset below a CXL port with SBR masked
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL port ignores the Secondary Bus Reset bit in Bridge Control unless Unmask SBR is set in its CXL Port DVSEC (CXL r4.0 sec 8.1.5.2). pci_reset_bus_function() already returns -ENOTTY in that case. pci_reset_bus(), pci_probe_reset_bus() and the slot variants do not, so they report success for a reset that reached no device below the port.
vfio-pci uses these for VFIO_DEVICE_GET_PCI_HOT_RESET_INFO and VFIO_DEVICE_PCI_HOT_RESET. A user is then told a hot reset happened while a CXL device below a masked port kept its state.
Return -ENOTTY from pci_probe_reset_slot() and pci_probe_reset_bus() when the bridge above the bus is a CXL port with SBR masked. pci_reset_bus() and pci_try_reset_bridge() probe before they reset, so they refuse too.
Users see this in three places:
- the vfio hot reset ioctls fail (vfio reports -ENODEV when neither probe succeeds),
- the reset_subordinate sysfs attribute returns -ENOTTY,
- vfio-pci groups a device in a hotplug slot below such a port by bus instead of by slot.
Every other pci_reset_bus() caller, including the vfio bus reset at close, is refused the same way. With one device per CXL port both groupings have the same members.
Leave pci_bus_error_reset() as it is. AER and host bridge error recovery call it without saving or restoring state and only map its result to recovered or disconnect. Failing it below a masked port would turn recovery of every CXL device there, memory expanders included, into a disconnect; that is a CXL error handling decision, not part of this fix.
--- drivers/pci/pci.c | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
drivers/pci/pci.c +28 -3
@@ -4984,6 +4984,20 @@ static bool cxl_sbr_masked(struct pci_dev *dev)return true; } +/* +* A secondary bus reset below a CXL port with SBR masked does not reach the +* devices on the bus, so a bus or slot reset there would report success for +* a reset that did not happen. The probes and the save/restore resets that +* pci_reset_bus() and reset_subordinate use refuse it; error recovery +* (pci_bus_error_reset()) keeps its existing behavior. +*/ +static bool pci_bus_cxl_sbr_masked (struct pci_bus *bus) +{ +struct pci_dev *bridge = bus->self; + +return bridge &&pcie_is_cxl (bridge) &&cxl_sbr_masked (bridge); +} +static int pci_reset_bus_function (struct pci_dev *dev,bool probe) {struct pci_dev *bridge =pci_upstream_bridge (dev);
@@ -5615,6 +5629,9 @@ static int pci_slot_reset(struct pci_slot *slot, bool probe)*/ int pci_probe_reset_slot (struct pci_slot *slot) { +if (slot &&pci_bus_cxl_sbr_masked (slot->bus)) +return -ENOTTY ; +return pci_slot_reset (slot,PCI_RESET_PROBE ); }EXPORT_SYMBOL_GPL (pci_probe_reset_slot);
@@ -5638,7 +5655,7 @@ static int pci_try_reset_slot(struct pci_slot *slot) {int rc; - rc =pci_slot_reset (slot,PCI_RESET_PROBE ); + rc =pci_probe_reset_slot (slot);if (rc)return rc;
@@ -5685,7 +5702,7 @@ static int pci_try_reset_bus(struct pci_bus *bus) {int rc; - rc =pci_bus_reset (bus,PCI_RESET_PROBE ); + rc =pci_probe_reset_bus (bus);if (rc)return rc;
@@ -5726,8 +5743,13 @@ static int pci_reset_bridge(struct pci_dev *bridge, bool restore)if (list_empty (&bus->slots)) goto bus_reset; +/* +* Probe without the CXL SBR mask check: error recovery keeps its +* reset path below a masked CXL port, and pci_try_reset_slot() and +* pci_try_reset_bus() refuse the restore path there. +*/ list_for_each_entry (slot, &bus->slots, list) -if (pci_probe_reset_slot (slot)) +if (pci_slot_reset (slot,PCI_RESET_PROBE )) goto bus_reset;list_for_each_entry (slot, &bus->slots, list) {
@@ -5773,6 +5795,9 @@ int pci_try_reset_bridge(struct pci_dev *bridge)*/ int pci_probe_reset_bus (struct pci_bus *bus) { +if (pci_bus_cxl_sbr_masked (bus)) +return -ENOTTY ; +return pci_bus_reset (bus,PCI_RESET_PROBE ); }EXPORT_SYMBOL_GPL (pci_probe_reset_bus);
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 05/31] PCI: Refuse bus and slot reset below a CXL port with SBR masked
Quoted 45 lines · On Fri, 16 Oct 2026 21:48:12 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL port ignores the Secondary Bus Reset bit in Bridge Control unless Unmask SBR is set in its CXL Port DVSEC (CXL r4.0 sec 8.1.5.2).
pci_reset_bus_function()already returns-ENOTTYin that case.pci_reset_bus(),pci_probe_reset_bus()and the slot variants do not, so they report success for a reset that reached no device below the port.vfio-pci uses these for
VFIO_DEVICE_GET_PCI_HOT_RESET_INFOandVFIO_DEVICE_PCI_HOT_RESET. A user is then told a hot reset happened while a CXL device below a masked port kept its state.Return
-ENOTTYfrompci_probe_reset_slot()andpci_probe_reset_bus()when the bridge above the bus is a CXL port with SBR masked.pci_reset_bus()andpci_try_reset_bridge()probe before they reset, so they refuse too.Users see this in three places:
- the vfio hot reset ioctls fail (vfio reports-ENODEVwhen neither probe succeeds),
- thereset_subordinatesysfs attribute returns-ENOTTY,
- vfio-pci groups a device in a hotplug slot below such a port by bus instead of by slot.Every other
pci_reset_bus()caller, including the vfio bus reset at close, is refused the same way. With one device per CXL port both groupings have the same members.Leave
pci_bus_error_reset()as it is. AER and host bridge error recovery call it without saving or restoring state and only map its result to recovered or disconnect. Failing it below a masked port would turn recovery of every CXL device there, memory expanders included, into a disconnect; that is a CXL error handling decision, not part of this fix.Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/pci/pci.c | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)drivers/pci/pci.c +0
-0
drivers/pci/pci.c static bool cxl_sbr_masked(struct pci_dev *dev) · line 4993
Quoted 10 lines
drivers/pci/pci.c +7
-0@@ -4984,6 +4984,20 @@ static bool cxl_sbr_masked(struct pci_dev *dev)return true; } +/* +* A secondary bus reset below a CXL port with SBR masked does not reach the +* devices on the bus, so a bus or slot reset there would report success for +* a reset that did not happen. The probes and the save/restore resets that +* pci_reset_bus() and reset_subordinate use refuse it; error recovery +* (pci_bus_error_reset()) keeps its existing behavior. +*/
Reads like AI over-explaining a fairly obvious concept.
Quoted 11 lines
drivers/pci/pci.c +7
-0+static bool pci_bus_cxl_sbr_masked (struct pci_bus *bus) +{ +struct pci_dev *bridge = bus->self; + +return bridge &&pcie_is_cxl (bridge) &&cxl_sbr_masked (bridge); +} +static int pci_reset_bus_function (struct pci_dev *dev,bool probe) {struct pci_dev *bridge =pci_upstream_bridge (dev);
drivers/pci/pci.c static int pci_slot_reset(struct pci_slot *slot, bool probe) · line 5633
Quoted 5 lines
drivers/pci/pci.c +2
-0@@ -5615,6 +5629,9 @@ static int pci_slot_reset(struct pci_slot *slot, bool probe)*/ int pci_probe_reset_slot (struct pci_slot *slot) { +if (slot &&pci_bus_cxl_sbr_masked (slot->bus)) +return -ENOTTY ;
Why would this be open coded here instead of pci_slot_resettable()?
drivers/pci/pci.c static int pci_try_reset_slot(struct pci_slot *slot) · line 5658
Quoted 10 lines
drivers/pci/pci.c +2
-1+return pci_slot_reset (slot,PCI_RESET_PROBE ); }EXPORT_SYMBOL_GPL (pci_probe_reset_slot);@@ -5638,7 +5655,7 @@ static int pci_try_reset_slot(struct pci_slot *slot) {int rc; - rc =pci_slot_reset (slot,PCI_RESET_PROBE ); + rc =pci_probe_reset_slot (slot);
Also suggests it's tested in the wrong place.
drivers/pci/pci.c static int pci_try_reset_bus(struct pci_bus *bus) · line 5705
Quoted 9 lines
drivers/pci/pci.c +1
-1if (rc)return rc;@@ -5685,7 +5702,7 @@ static int pci_try_reset_bus(struct pci_bus *bus) {int rc; - rc =pci_bus_reset (bus,PCI_RESET_PROBE ); + rc =pci_probe_reset_bus (bus);
And again.
drivers/pci/pci.c static int pci_reset_bridge(struct pci_dev *bridge, bool restore) · line 5750
Quoted 12 lines
drivers/pci/pci.c +5
-0if (rc)return rc;@@ -5726,8 +5743,13 @@ static int pci_reset_bridge(struct pci_dev *bridge, bool restore)if (list_empty (&bus->slots)) goto bus_reset; +/* +* Probe without the CXL SBR mask check: error recovery keeps its +* reset path below a masked CXL port, and pci_try_reset_slot() and +* pci_try_reset_bus() refuse the restore path there. +*/
More AI? This function is not designated as exclusive to error recovery. If sbr is masked, slot reset doesn't generate an error and doesn't effect a reset, regardless of who called it. If we're including a new condition for a bus/slot reset to be effective, it needs to be applied universally.
Quoted 7 lines
drivers/pci/pci.c +1
-1list_for_each_entry (slot, &bus->slots, list) -if (pci_probe_reset_slot (slot)) +if (pci_slot_reset (slot,PCI_RESET_PROBE )) goto bus_reset;list_for_each_entry (slot, &bus->slots, list) {
drivers/pci/pci.c int pci_try_reset_bridge(struct pci_dev *bridge) · line 5803
Quoted 9 lines
drivers/pci/pci.c +3
-0@@ -5773,6 +5795,9 @@ int pci_try_reset_bridge(struct pci_dev *bridge)*/ int pci_probe_reset_bus (struct pci_bus *bus) { +if (pci_bus_cxl_sbr_masked (bus)) +return -ENOTTY ; +return pci_bus_reset (bus,PCI_RESET_PROBE ); }EXPORT_SYMBOL_GPL (pci_probe_reset_bus);
Redux, pci_bus_resettable(). Thanks,
Alex
RE: [PATCH v6 05/31] PCI: Refuse bus and slot reset below a CXL port with SBR masked
Quoted 77 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Thursday, November 5, 2026 5:23 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 05/31] PCI: Refuse bus and slot reset below a CXL port with SBR maskedExternal email: Use caution opening links or attachments
Quoted 28 lines · On Fri, 16 Oct 2026 21:48:12 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL port ignores the Secondary Bus Reset bit in Bridge Control unless Unmask SBR is set in its CXL Port DVSEC (CXL r4.0 sec 8.1.5.2).
pci_reset_bus_function()already returns-ENOTTYin that case.pci_reset_bus(),pci_probe_reset_bus()and the slot variants do not, so they report success for a reset that reached no device below the port.vfio-pci uses these for
VFIO_DEVICE_GET_PCI_HOT_RESET_INFOandVFIO_DEVICE_PCI_HOT_RESET. A user is then told a hot reset happened while a CXL device below a masked port kept its state.Return
-ENOTTYfrompci_probe_reset_slot()andpci_probe_reset_bus()when the bridge above the bus is a CXL port with SBR masked.pci_reset_bus()andpci_try_reset_bridge()probe before they reset, so they refuse too.Users see this in three places:
- the vfio hot reset ioctls fail (vfio reports-ENODEVwhen neither probe succeeds),
- thereset_subordinatesysfs attribute returns-ENOTTY,
- vfio-pci groups a device in a hotplug slot below such a port by bus instead of by slot.Every other
pci_reset_bus()caller, including the vfio bus reset at close, is refused the same way. With one device per CXL port both groupings have the same members.Quoted 5 lines
Leave
pci_bus_error_reset()as it is. AER and host bridge error recovery call it without saving or restoring state and only map its result to recovered or disconnect. Failing it below a masked port would turn recovery of every CXL device there, memory expanders included, into a disconnect; that is a CXL error handling decision, not part ofthis fix.
Quoted 15 lines
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/pci/pci.c | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)drivers/pci/pci.c index renamed from drivers/pci/pci.c +0
-0
cecf8e681e69..87fe9de81acf100644drivers/pci/pci.c +0
-0@@ -4984,6 +4984,20 @@ static bool cxl_sbr_masked(struct pci_dev *dev)return true; }
Quoted 13 lines
drivers/pci/pci.c static bool cxl_sbr_masked(struct pci_dev *dev) · line 4997
Quoted 11 lines
drivers/pci/pci.c +11
-0+/* +* A secondary bus reset below a CXL port with SBR masked does not +reach the +* devices on the bus, so a bus or slot reset there would report +success for +* a reset that did not happen. The probes and the save/restore +resets that +* pci_reset_bus() and reset_subordinate use refuse it; error +recovery +* (pci_bus_error_reset()) keeps its existing behavior. +*/ Reads like AI over-explaining a fairly obvious concept.
I will drop this.
Quoted 75 lines
Quoted 4 lines
drivers/pci/pci.c +3
-0+static bool pci_bus_cxl_sbr_masked (struct pci_bus *bus) { +struct pci_dev *bridge = bus->self; +drivers/pci/pci.c static bool cxl_sbr_masked(struct pci_dev *dev)
Quoted 10 lines
drivers/pci/pci.c +6
-0+return bridge && pcie_is_cxl(bridge) && cxl_sbr_masked(bridge); +} +static int pci_reset_bus_function(struct pci_dev *dev, bool probe) { struct pci_dev *bridge = pci_upstream_bridge(dev); @@ -5615,6 +5629,9 @@ static int pci_slot_reset(struct pci_slot *slot, bool probe) */ int pci_probe_reset_slot (struct pci_slot *slot) { +if (slot &&pci_bus_cxl_sbr_masked (slot->bus)) +return -ENOTTY ;Why would this be open coded here instead of
pci_slot_resettable()?drivers/pci/pci.c static int pci_try_reset_slot(struct pci_slot · line 5658
Quoted 9 lines
+
return pci_slot_reset(slot, PCI_RESET_PROBE); }
EXPORT_SYMBOL_GPL(pci_probe_reset_slot);drivers/pci/pci.c +1
-1@@ -5638,7 +5655,7 @@ static int pci_try_reset_slot(struct pci_slot *slot) {int rc; - rc =pci_slot_reset (slot,PCI_RESET_PROBE ); + rc =pci_probe_reset_slot (slot);Also suggests it's tested in the wrong place.
drivers/pci/pci.c static int pci_try_reset_bus(struct pci_bus · line 5705
Quoted 9 lines
drivers/pci/pci.c +1
-1if (rc)return rc;@@ -5685,7 +5702,7 @@ static int pci_try_reset_bus(struct pci_bus *bus) {int rc; - rc =pci_bus_reset (bus,PCI_RESET_PROBE ); + rc =pci_probe_reset_bus (bus);And again.
drivers/pci/pci.c static int pci_reset_bridge(struct pci_dev
Quoted 4 lines
drivers/pci/pci.c +0
-0if (rc)return rc;@@ -5726,8 +5743,13 @@ static int pci_reset_bridge(struct pci_dev*bridge, bool restore)
drivers/pci/pci.c static int pci_reset_bridge(struct pci_dev *bridge, bool restore) · line 5750
Quoted 8 lines
drivers/pci/pci.c +5
-0if (list_empty (&bus->slots)) goto bus_reset; +/* +* Probe without the CXL SBR mask check: error recovery keeps its +* reset path below a masked CXL port, and pci_try_reset_slot() and +* pci_try_reset_bus() refuse the restore path there. +*/ More AI? This function is not designated as exclusive to error recovery. If sbr is masked, slot reset doesn't generate an error and doesn't effect a reset, regardless of who called it. If we're including a new condition for a bus/slot reset to be effective, it needs to be applied universally.
Quoted 13 lines
drivers/pci/pci.c +5
-1list_for_each_entry(slot, &bus->slots, list) -if (pci_probe_reset_slot(slot)) +if (pci_slot_reset(slot, PCI_RESET_PROBE)) goto bus_reset; list_for_each_entry(slot, &bus->slots, list) { @@ -5773,6 +5795,9 @@ int pci_try_reset_bridge(struct pci_dev *bridge) */ int pci_probe_reset_bus (struct pci_bus *bus) { +if (pci_bus_cxl_sbr_masked (bus)) +return -ENOTTY ; +return pci_bus_reset (bus,PCI_RESET_PROBE ); }Quoted 1 line
EXPORT_SYMBOL_GPL(pci_probe_reset_bus);
Quoted 1 line
Redux,
pci_bus_resettable(). Thanks,
After giving more thought for this patch, I am inclined towards drop of this patch. AER and host bridge recovery of every CXL device below a port with SBR masked ends in a disconnect instead of a reported recovery.
That is a CXL error handling change I do not want to make inside this series.
This will also change what I wrote on v5 22/27: below a masked port VFIO_DEVICE_PCI_HOT_RESET keeps returning success without resetting the device. vfio-cxl does not rely on it. It resets the device with CXL Reset at open/close/VFIO_DEVICE_RESET.
After any reset vfio-cxl did not start the device memory stays closed until a CXL Reset through vfio succeeds. I will document this part in v7.
Thanks,
Manish
Quoted 2 lines
Alex
PATCH v6 6/31 vfio/pci: Add the CXL provider ops registration interface
In reply to [email protected]From: Manish Honap <[email protected]>
vfio-pci-core must stay free of any CXL header dependency, so CXL Type-2 handling lives in a separate vfio-cxl module that plugs in a set of callbacks.
Add the registration interface:
- vfio-cxl registers a single struct vfio_cxl_ops at module_init
- vfio-pci-core stores it under an rwsem.
A second registration is refused with -EBUSY.
The owner field will be used to pin vfio-cxl for the lifetime of each bound CXL device.
--- drivers/vfio/pci/vfio_pci_core.c | 24 ++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 10 ++++++++++ 2 files changed, 34 insertions(+)
drivers/vfio/pci/vfio_pci_core.c +24 -0
@@ -2178,6 +2178,30 @@ static void vfio_pci_vga_uninit(struct vfio_pci_core_device *vdev) VGA_RSRC_LEGACY_MEM); } +static const struct vfio_cxl_ops *vfio_pci_cxl_ops; +static DECLARE_RWSEM (vfio_pci_cxl_ops_rwsem); + +int vfio_pci_core_register_cxl_ops (const struct vfio_cxl_ops *ops) +{ +guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops) +return -EBUSY ; + + vfio_pci_cxl_ops = ops; +return 0 ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_register_cxl_ops); + +void vfio_pci_core_unregister_cxl_ops (const struct vfio_cxl_ops *ops) +{ +guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops == ops) + vfio_pci_cxl_ops =NULL ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_unregister_cxl_ops); +int vfio_pci_core_init_dev (struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev =
include/linux/vfio_pci_core.h +10 -0
@@ -66,6 +66,16 @@ struct vfio_pci_device_ops { size_t nr_ranges); }; +struct vfio_cxl_ops { +int (*init)(struct vfio_pci_core_device *vdev); +void (*release)(struct vfio_pci_core_device *vdev); +/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ +struct module *owner; +}; + +int vfio_pci_core_register_cxl_ops (const struct vfio_cxl_ops *ops); +void vfio_pci_core_unregister_cxl_ops (const struct vfio_cxl_ops *ops); +#if IS_ENABLED (CONFIG_VFIO_PCI_DMABUF ) int vfio_pci_core_fill_phys_vec(struct phys_vec *phys_vec, struct vfio_region_dma_range *dma_ranges,
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 06/31] vfio/pci: Add the CXL provider ops registration interface
Quoted 62 lines · On Fri, 16 Oct 2026 21:48:13 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio-pci-core must stay free of any CXL header dependency, so CXL Type-2 handling lives in a separate vfio-cxl module that plugs in a set of callbacks.
Add the registration interface:
- vfio-cxl registers a single structvfio_cxl_opsatmodule_init
- vfio-pci-core stores it under an rwsem.A second registration is refused with
-EBUSY.The owner field will be used to pin vfio-cxl for the lifetime of each bound CXL device.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 24 ++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 10 ++++++++++ 2 files changed, 34 insertions(+)drivers/vfio/pci/vfio_pci_core.c +24
-0@@ -2178,6 +2178,30 @@ static void vfio_pci_vga_uninit(struct vfio_pci_core_device *vdev) VGA_RSRC_LEGACY_MEM); } +static const struct vfio_cxl_ops *vfio_pci_cxl_ops; +static DECLARE_RWSEM (vfio_pci_cxl_ops_rwsem); + +int vfio_pci_core_register_cxl_ops (const struct vfio_cxl_ops *ops) +{ +guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops) +return -EBUSY ; + + vfio_pci_cxl_ops = ops; +return 0 ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_register_cxl_ops); + +void vfio_pci_core_unregister_cxl_ops (const struct vfio_cxl_ops *ops) +{ +guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops == ops) + vfio_pci_cxl_ops =NULL ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_unregister_cxl_ops); +int vfio_pci_core_init_dev (struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev =include/linux/vfio_pci_core.h +0
-0
include/linux/vfio_pci_core.h struct vfio_pci_device_ops { · line 74
Quoted 9 lines
include/linux/vfio_pci_core.h +6
-0@@ -66,6 +66,16 @@ struct vfio_pci_device_ops { size_t nr_ranges); }; +struct vfio_cxl_ops { +int (*init)(struct vfio_pci_core_device *vdev); +void (*release)(struct vfio_pci_core_device *vdev); +/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ +struct module *owner; +};
A forward declaration would be sufficient and cleaner here. It doesn't buy us anything to start defining the structure until we begin to use it. Thanks,
Alex
Quoted 7 lines
include/linux/vfio_pci_core.h +4
-0+ +int vfio_pci_core_register_cxl_ops (const struct vfio_cxl_ops *ops); +void vfio_pci_core_unregister_cxl_ops (const struct vfio_cxl_ops *ops); +#if IS_ENABLED (CONFIG_VFIO_PCI_DMABUF ) int vfio_pci_core_fill_phys_vec(struct phys_vec *phys_vec, struct vfio_region_dma_range *dma_ranges,
RE: [PATCH v6 06/31] vfio/pci: Add the CXL provider ops registration interface
Quoted 90 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Thursday, November 5, 2026 5:23 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 06/31] vfio/pci: Add the CXL provider ops registration interfaceExternal email: Use caution opening links or attachments
drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_vga_uninit(struct
Quoted 28 lines · On Fri, 16 Oct 2026 21:48:13 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio-pci-core must stay free of any CXL header dependency, so CXL Type-2 handling lives in a separate vfio-cxl module that plugs in a set of callbacks.
Add the registration interface:
- vfio-cxl registers a single structvfio_cxl_opsatmodule_init
- vfio-pci-core stores it under an rwsem.A second registration is refused with
-EBUSY.The owner field will be used to pin vfio-cxl for the lifetime of each bound CXL device.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 24 ++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 10 ++++++++++ 2 files changed, 34 insertions(+)drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -2178,6 +2178,30 @@ static void vfio_pci_vga_uninit(struct
vfio_pci_core_device*vdev)Quoted 33 lines
drivers/vfio/pci/vfio_pci_core.c +22
-0VGA_RSRC_LEGACY_MEM); } +static const struct vfio_cxl_ops *vfio_pci_cxl_ops;static +DECLARE_RWSEM (vfio_pci_cxl_ops_rwsem); + +int vfio_pci_core_register_cxl_ops (const struct vfio_cxl_ops *ops) { +guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops) +return -EBUSY ; + + vfio_pci_cxl_ops = ops; +return 0 ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_register_cxl_ops); + +void vfio_pci_core_unregister_cxl_ops (const struct vfio_cxl_ops *ops) +{ +guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops == ops) + vfio_pci_cxl_ops =NULL ; } +EXPORT_SYMBOL_GPL (vfio_pci_core_unregister_cxl_ops); +int vfio_pci_core_init_dev (struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev = diff --git
a/include/linux/vfio_pci_core.hb/include/linux/vfio_pci_core.hindex9a1674c152aa..9fe0d1a3a370100644include/linux/vfio_pci_core.h +0
-0@@ -66,6 +66,16 @@ struct vfio_pci_device_ops { size_t nr_ranges); };
Quoted 9 lines
include/linux/vfio_pci_core.h struct vfio_pci_device_ops { · line 74
Quoted 6 lines
include/linux/vfio_pci_core.h +6
-0+struct vfio_cxl_ops { +int (*init)(struct vfio_pci_core_device *vdev); +void (*release)(struct vfio_pci_core_device *vdev); +/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ +struct module *owner; +};A forward declaration would be sufficient and cleaner here. It doesn't buy us anything to start defining the structure until we begin to use it. Thanks,
Okay, Will do in v7. Each member is added in the patch that first uses it.
Thanks,
Manish
Quoted 12 lines
Alex
Quoted 9 lines
include/linux/vfio_pci_core.h +5
-0+ +int vfio_pci_core_register_cxl_ops (const struct vfio_cxl_ops *ops); +void vfio_pci_core_unregister_cxl_ops (const struct vfio_cxl_ops +*ops); +#if IS_ENABLED (CONFIG_VFIO_PCI_DMABUF )int vfio_pci_core_fill_phys_vec (struct phys_vec *phys_vec, struct vfio_region_dma_range
*dma_ranges,
PATCH v6 7/31 vfio/pci: Detect CXL devices and load the CXL provider on demand
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL device needs the vfio-cxl callbacks, but pulling vfio-cxl and the CXL core in unconditionally would bloat every vfio-pci setup. Detect a CXL Type-2 device at bind and request_module("vfio-cxl") only then, then hand the device to the registered ops.
Each bound CXL device pins the provider through vfio_pci_get_cxl_ops() (try_module_get) and drops it with vfio_pci_put_cxl_ops() at release, so vfio-cxl can unload once no CXL device is bound.
If the provider is absent the device is bound without the CXL extensions. A built-in provider whose initcall has not run yet is waited for with -EPROBE_DEFER; the deferred-probe machinery will retry the bind once the provider registers.
The provider .init() can return -EPROBE_DEFER when the device is not ready yet, for example when its CXL port has not enumerated. Propagate it so the bind is retried later. Any other init failure is non-fatal: a CXL Type-2 device is also a valid vfio-pci device, so log the error, drop the provider reference, leave cxl_ops NULL and continue without CXL support.
--- drivers/vfio/pci/vfio_pci_core.c | 101 +++++++++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 3 + 2 files changed, 104 insertions(+)
drivers/vfio/pci/vfio_pci_core.c +101 -0
@@ -2202,6 +2202,101 @@ void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops) }EXPORT_SYMBOL_GPL (vfio_pci_core_unregister_cxl_ops); +static const struct vfio_cxl_ops *vfio_pci_get_cxl_ops (void ) +{ +guard (rwsem_read)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops &&try_module_get (vfio_pci_cxl_ops->owner)) +return vfio_pci_cxl_ops; + +return NULL ; +} + +static void vfio_pci_put_cxl_ops (const struct vfio_cxl_ops *ops) +{ +module_put (ops->owner); +} + +/* +* A CXL Type-2 device advertises both CXL.cache and CXL.mem in its CXL DVSEC. +* pcie_is_cxl() is also true for Type-1 (cache only) and Type-3 (mem only) +* devices, which the vfio-cxl provider does not handle, so confirm the Type-2 +* identity before engaging it. +*/ +static bool vfio_pci_is_cxl_type2 (struct pci_dev *pdev) +{ +u16 dvsec, cap; + +if (!pcie_is_cxl (pdev)) +return false; + + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); +if (!dvsec) +return false; + +if (pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CAP , &cap)) +return false; + +return (cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) && + (cap &PCI_DVSEC_CXL_MEM_CAPABLE ); +} + +/* +* Load vfio-cxl on demand for a CXL Type-2 device and hand the device to its +* ops. If the provider is absent the device is bound without the CXL +* extensions; a built-in provider whose initcall has not run yet is waited for with +* -EPROBE_DEFER. +*/ +static int vfio_pci_core_cxl_init (struct vfio_pci_core_device *vdev) +{ +const struct vfio_cxl_ops *ops; +int ret; + +if (!vfio_pci_is_cxl_type2 (vdev->pdev)) +return 0 ; + +request_module ("vfio-cxl" ); + ops =vfio_pci_get_cxl_ops (); +if (!ops) +return IS_BUILTIN (CONFIG_VFIO_CXL ) ? -EPROBE_DEFER :0 ; + + ret = ops->init (vdev); +/* +* A provider that is not ready yet (for example its CXL port has not +* enumerated) returns -EPROBE_DEFER. Propagate it so the bind retries +* rather than continuing without CXL support. +*/ +if (ret == -EPROBE_DEFER ) { +vfio_pci_put_cxl_ops (ops); +return ret; + } +/* +* Any other failure is non-fatal: a CXL device that cannot be brought +* up as Type-2 still works without the CXL extensions, so log and +* continue rather than failing the bind. +*/ +if (ret) { +pci_warn (vdev->pdev, +"CXL init failed (%d), continuing without CXL support\n" , + ret); +vfio_pci_put_cxl_ops (ops); +return 0 ; + } + + vdev->cxl_ops = ops; +return 0 ; +} + +static void vfio_pci_core_cxl_release (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_ops) +return ; + + vdev->cxl_ops->release (vdev); +vfio_pci_put_cxl_ops (vdev->cxl_ops); +} +int vfio_pci_core_init_dev (struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev =
@@ -2223,6 +2318,10 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)init_rwsem (&vdev->memory_lock);xa_init (&vdev->ctx); + ret =vfio_pci_core_cxl_init (vdev); +if (ret) +return ret; +return 0 ; }EXPORT_SYMBOL_GPL (vfio_pci_core_init_dev);
@@ -2232,6 +2331,8 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev)struct vfio_pci_core_device *vdev = container_of(core_vdev, struct vfio_pci_core_device, vdev); +vfio_pci_core_cxl_release (vdev); +mutex_destroy (&vdev->igate);mutex_destroy (&vdev->ioeventfds_lock);kfree (vdev->region);
include/linux/vfio_pci_core.h +3 -0
@@ -29,6 +29,7 @@ struct vfio_pci_core_device;struct vfio_pci_region ;struct p2pdma_provider ;struct dma_buf_attachment ; +struct vfio_cxl_state ;struct vfio_pci_eventfd {struct eventfd_ctx *ctx;
@@ -109,6 +110,8 @@ struct vfio_pci_core_device {struct vfio_device vdev;struct pci_dev *pdev;const struct vfio_pci_device_ops *pci_ops; +const struct vfio_cxl_ops *cxl_ops; +struct vfio_cxl_state *cxl;void __iomem *barmap[PCI_STD_NUM_BARS ];bool bar_mmap_supported[PCI_STD_NUM_BARS ];/* Flags modified at runtime - dedicated storage unit */
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 07/31] vfio/pci: Detect CXL devices and load the CXL provider on demand
Quoted 50 lines · On Fri, 16 Oct 2026 21:48:14 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL device needs the vfio-cxl callbacks, but pulling vfio-cxl and the CXL core in unconditionally would bloat every vfio-pci setup. Detect a CXL Type-2 device at bind and
request_module("vfio-cxl")only then, then hand the device to the registered ops.Each bound CXL device pins the provider through
vfio_pci_get_cxl_ops()(try_module_get) and drops it withvfio_pci_put_cxl_ops()at release, so vfio-cxl can unload once no CXL device is bound.If the provider is absent the device is bound without the CXL extensions. A built-in provider whose initcall has not run yet is waited for with -EPROBE_DEFER; the deferred-probe machinery will retry the bind once the provider registers.
The provider .init() can return -EPROBE_DEFER when the device is not ready yet, for example when its CXL port has not enumerated. Propagate it so the bind is retried later. Any other init failure is non-fatal: a CXL Type-2 device is also a valid vfio-pci device, so log the error, drop the provider reference, leave
cxl_opsNULL and continue without CXL support.Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 101 +++++++++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 3 + 2 files changed, 104 insertions(+)drivers/vfio/pci/vfio_pci_core.c +11
-0@@ -2202,6 +2202,101 @@ void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops) }EXPORT_SYMBOL_GPL (vfio_pci_core_unregister_cxl_ops); +static const struct vfio_cxl_ops *vfio_pci_get_cxl_ops (void ) +{ +guard (rwsem_read)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops &&try_module_get (vfio_pci_cxl_ops->owner)) +return vfio_pci_cxl_ops; + +return NULL ; +} + +static void vfio_pci_put_cxl_ops (const struct vfio_cxl_ops *ops)
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops) · line 2225
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+{ +module_put (ops->owner); +} + +/* +* A CXL Type-2 device advertises both CXL.cache and CXL.mem in its CXL DVSEC. +* pcie_is_cxl() is also true for Type-1 (cache only) and Type-3 (mem only) +* devices, which the vfio-cxl provider does not handle, so confirm the Type-2 +* identity before engaging it. +*/
Probably another AI comment, it's only describing what the function is obviously doing. Drop it.
Quoted 24 lines
drivers/vfio/pci/vfio_pci_core.c +24
-0+static bool vfio_pci_is_cxl_type2 (struct pci_dev *pdev) +{ +u16 dvsec, cap; + +if (!pcie_is_cxl (pdev)) +return false; + + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); +if (!dvsec) +return false; + +if (pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CAP , &cap)) +return false; + +return (cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) && + (cap &PCI_DVSEC_CXL_MEM_CAPABLE ); +} + +/* +* Load vfio-cxl on demand for a CXL Type-2 device and hand the device to its +* ops. If the provider is absent the device is bound without the CXL +* extensions; a built-in provider whose initcall has not run yet is waited for with +* -EPROBE_DEFER.
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops) · line 2259
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+*/ +static int vfio_pci_core_cxl_init (struct vfio_pci_core_device *vdev) +{ +const struct vfio_cxl_ops *ops; +int ret; + +if (!vfio_pci_is_cxl_type2 (vdev->pdev)) +return 0 ; + +request_module ("vfio-cxl" );
A local sashiko run questions:
---
[Severity: High]
Could request_module() in vfio_pci_core_cxl_init(), in
drivers/vfio/pci/vfio_pci_core.c, deadlock when a matching Type-2 device
is probed asynchronously and vfio-cxl is an unloaded module whose own async_probe setting remains false?
Asynchronous probing can be selected with vfio-pci's async_probe=1 module argument or driver_async_probe=vfio-pci. The resulting wait cycle is:
__driver_attach_async_helper()
vfio_pci_core_cxl_init()
request_module("vfio-cxl")
waits for modprobe to exit
do_init_module() for vfio-cxl
async_synchronize_full()
waits for the outstanding vfio-pci probe
In kernel/module/main.c, do_init_module() performs this synchronization after the provider initializer returns. Successfully registering the operations therefore does not break the cycle. The provider's async_probe setting is independent of vfio-pci's setting.
The warning in __request_module(), in kernel/module/kmod.c, does not abort loading. Dispatch through pci_call_probe(), in
drivers/pci/pci-driver.c, also preserves the cycle because the originating
async callback waits in flush_work(). The global async wait has no timeout.
The final series retains this synchronous request. Could provider loading be moved outside the outstanding async probe, with binding deferred until the operations are available?
---
AIUI, this could be resolved if we do the request_module() on bind and the CXL init on open, but CXL expects that we use the devm interfaces that have a bind->release lifetime. But then...
Quoted 55 lines
drivers/vfio/pci/vfio_pci_core.c +44
-0+ ops =vfio_pci_get_cxl_ops (); +if (!ops) +return IS_BUILTIN (CONFIG_VFIO_CXL ) ? -EPROBE_DEFER :0 ; + + ret = ops->init (vdev); +/* +* A provider that is not ready yet (for example its CXL port has not +* enumerated) returns -EPROBE_DEFER. Propagate it so the bind retries +* rather than continuing without CXL support. +*/ +if (ret == -EPROBE_DEFER ) { +vfio_pci_put_cxl_ops (ops); +return ret; + } +/* +* Any other failure is non-fatal: a CXL device that cannot be brought +* up as Type-2 still works without the CXL extensions, so log and +* continue rather than failing the bind. +*/ +if (ret) { +pci_warn (vdev->pdev, +"CXL init failed (%d), continuing without CXL support\n" , + ret); +vfio_pci_put_cxl_ops (ops); +return 0 ; + } + + vdev->cxl_ops = ops; +return 0 ; +} + +static void vfio_pci_core_cxl_release (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_ops) +return ; + + vdev->cxl_ops->release (vdev); +vfio_pci_put_cxl_ops (vdev->cxl_ops); +} +int vfio_pci_core_init_dev (struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev =@@ -2223,6 +2318,10 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)init_rwsem (&vdev->memory_lock);xa_init (&vdev->ctx); + ret =vfio_pci_core_cxl_init (vdev); +if (ret) +return ret; +return 0 ; }EXPORT_SYMBOL_GPL (vfio_pci_core_init_dev);
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_release_dev(struct vfio_device *core_vdev) · line 2334
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0@@ -2232,6 +2331,8 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev)struct vfio_pci_core_device *vdev = container_of(core_vdev, struct vfio_pci_core_device, vdev); +vfio_pci_core_cxl_release (vdev);
Sashiko has a doozy regarding the resulting lifecycle teardown:
---
[Severity: Critical]
Can vfio_pci_core_release_dev(), in drivers/vfio/pci/vfio_pci_core.c, invoke provider teardown after device-managed cleanup has freed vdev->cxl when final object destruction is delayed during unbind?
In the final series, vfio_cxl_create_memdev(), in
drivers/vfio/pci/cxl/vfio_cxl_core.c, allocates the provider state throughdevm_cxl_dev_state_create() against pdev->dev. This release hook instead runs at final vfio object destruction, which can follow driver removal and device-managed cleanup.
Device-cdev close and unbind can interleave as follows:
vfio_device_fops_release()
drops the last registration reference
completes the unregister wait
vfio_pci_remove()
finishes unregistering
drops its vfio device reference
returns while the cdev still retains the vfio object
device_unbind_cleanup()
devres_release_all() frees the provider state
__fput()
cdev_put() drops the remaining parent reference vfio_pci_core_release_dev() calls provider teardown
Enabling delayed kobject release provides another path without an open file. The module pin preserves the provider callback text, but does not preserve its device-managed state.
vfio_cxl_release_device(), in drivers/vfio/pci/cxl/vfio_cxl_core.c, then reads the freed cxl->hpa_range and cxl->hdm to release the reservation and free the decoder snapshot. Reuse of that allocation can cause an incorrect reservation release or an invalid free. The final series retains both the managed allocation and this final-release hook.
Could provider teardown run before device-managed cleanup, or could the provider state remain allocated through final vfio release?
---
So perhaps trying to shoehorn CXL's devm lifecycle into vfio-pci is biting us anyway. Thanks,
Alex
RE: [PATCH v6 07/31] vfio/pci: Detect CXL devices and load the CXL provider on demand
Quoted 85 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Thursday, November 5, 2026 5:23 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 07/31] vfio/pci: Detect CXL devices and load the CXL provider on demandExternal email: Use caution opening links or attachments
Quoted 24 lines · On Fri, 16 Oct 2026 21:48:14 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL device needs the vfio-cxl callbacks, but pulling vfio-cxl and the CXL core in unconditionally would bloat every vfio-pci setup. Detect a CXL Type-2 device at bind and
request_module("vfio-cxl")only then, then hand the device to the registered ops.Each bound CXL device pins the provider through
vfio_pci_get_cxl_ops()(try_module_get) and drops it withvfio_pci_put_cxl_ops()at release, so vfio-cxl can unload once no CXL device is bound.If the provider is absent the device is bound without the CXL extensions. A built-in provider whose initcall has not run yet is waited for with -EPROBE_DEFER; the deferred-probe machinery will retry the bind once the provider registers.
The provider .init() can return -EPROBE_DEFER when the device is not ready yet, for example when its CXL port has not enumerated. Propagate it so the bind is retried later. Any other init failure is non-fatal:
a CXL Type-2 device is also a valid vfio-pci device, so log the error, drop the provider reference, leavecxl_opsNULL and continue without CXL support.Quoted 4 lines
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 101+++++++++++++++++++++++++++++++
Quoted 17 lines
include/linux/vfio_pci_core.h | 3 + 2 files changed, 104 insertions(+)drivers/vfio/pci/vfio_pci_core.c +5
-0@@ -2202,6 +2202,101 @@ void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops) }EXPORT_SYMBOL_GPL (vfio_pci_core_unregister_cxl_ops); +static const struct vfio_cxl_ops *vfio_pci_get_cxl_ops (void ) { +guard (rwsem_read)(&vfio_pci_cxl_ops_rwsem); + +if (vfio_pci_cxl_ops &&try_module_get (vfio_pci_cxl_ops->owner)) +return vfio_pci_cxl_ops;drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_unregister_cxl_ops(const · line 2219
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+ +return NULL ; +} + +static void vfio_pci_put_cxl_ops (const struct vfio_cxl_ops *ops) { +module_put (ops->owner); +} + +/* +* A CXL Type-2 device advertises both CXL.cache and CXL.mem in its CXL DVSEC.
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_unregister_cxl_ops(const · line 2225
Quoted 6 lines
drivers/vfio/pci/vfio_pci_core.c +6
-0+* pcie_is_cxl() is also true for Type-1 (cache only) and Type-3 (mem +only) +* devices, which the vfio-cxl provider does not handle, so confirm +the Type-2 +* identity before engaging it. +*/ Probably another AI comment, it's only describing what the function is obviously doing. Drop it.
Okay, I will drop this.
Quoted 65 lines
Quoted 24 lines
drivers/vfio/pci/vfio_pci_core.c +24
-0+static bool vfio_pci_is_cxl_type2 (struct pci_dev *pdev) { +u16 dvsec, cap; + +if (!pcie_is_cxl (pdev)) +return false; + + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); +if (!dvsec) +return false; + +if (pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CAP , &cap)) +return false; + +return (cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) && + (cap &PCI_DVSEC_CXL_MEM_CAPABLE ); } + +/* +* Load vfio-cxl on demand for a CXL Type-2 device and hand the +device to its +* ops. If the provider is absent the device is bound without the CXL +* extensions; a built-in provider whose initcall has not run yet is +waited for with +* -EPROBE_DEFER. drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops) · line 2259
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+*/ +static int vfio_pci_core_cxl_init (struct vfio_pci_core_device *vdev) +{ +const struct vfio_cxl_ops *ops; +int ret; + +if (!vfio_pci_is_cxl_type2 (vdev->pdev)) +return 0 ; + +request_module ("vfio-cxl" );A local sashiko run questions:
---
[Severity: High]
Couldrequest_module()invfio_pci_core_cxl_init(), indrivers/vfio/pci/vfio_pci_core.c, deadlock when a matching Type-2 device is probed asynchronously and vfio-cxl is an unloaded module whose ownasync_probesetting remains false?Asynchronous probing can be selected with vfio-pci's async_probe=1 module argument or driver_async_probe=vfio-pci. The resulting wait cycle is:
__driver_attach_async_helper()
vfio_pci_core_cxl_init()
request_module("vfio-cxl")
waits for modprobe to exit
do_init_module()for vfio-cxl
async_synchronize_full()
waits for the outstanding vfio-pci probeIn
kernel/module/main.c,do_init_module()performs this synchronization after the provider initializer returns. Successfully registering the operations therefore does not break the cycle. The provider'sasync_probesetting is independent of vfio-pci's setting.The warning in
__request_module(), inkernel/module/kmod.c, does not abort loading. Dispatch throughpci_call_probe(), indrivers/pci/pci-driver.c, also preserves the cycle because the originating async callback waits inflush_work(). The global async wait has no timeout.
Quoted 9 lines
The final series retains this synchronous request. Could provider loading be moved outside the outstanding async probe, with binding deferred until the operations are available?
---AIUI, this could be resolved if we do the
request_module()on bind and the CXL init on open, but CXL expects that we use the devm interfaces that have a bind-Quoted 1 line
release lifetime. But then...
Hmm, I will need to spend more time on this. With vfio-pci async_probe=1, I think the probe blocks in request_module() while do_init_module() for vfio-cxl waits in async_synchronize_full() for that same probe.
For v7, I will load vfio-cxl from vfio_pci_init() instead before pci_register_driver():
- if a device present at that point has a CXL DVSEC that vfio-cxl can handle, init calls request_module() and waits. Module init is not an async probe, so the cycle suggested above cannot form, and a host without such a device will not load CXL module, as per your suggestion on v3.
- A device that shows up while vfio-cxl is not loaded gets request_module_nowait() and binds without CXL support, as it does today when CXL setup fails. A rebind will pick it up.
I will use MODULE_WEAKDEP("vfio-cxl") to keep the module in the initramfs without loading it.
Quoted 97 lines
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_release_dev(struct vfio_device
Quoted 54 lines
drivers/vfio/pci/vfio_pci_core.c +40
-0+ ops =vfio_pci_get_cxl_ops (); +if (!ops) +return IS_BUILTIN (CONFIG_VFIO_CXL ) ? -EPROBE_DEFER :0 ; + + ret = ops->init (vdev); +/* +* A provider that is not ready yet (for example its CXL port has not +* enumerated) returns -EPROBE_DEFER. Propagate it so the bind retries +* rather than continuing without CXL support. +*/ +if (ret == -EPROBE_DEFER ) { +vfio_pci_put_cxl_ops (ops); +return ret; + } +/* +* Any other failure is non-fatal: a CXL device that cannot be brought +* up as Type-2 still works without the CXL extensions, so log and +* continue rather than failing the bind. +*/ +if (ret) { +pci_warn (vdev->pdev, +"CXL init failed (%d), continuing without CXL support\n" , + ret); +vfio_pci_put_cxl_ops (ops); +return 0 ; + } + + vdev->cxl_ops = ops; +return 0 ; +} + +static void vfio_pci_core_cxl_release (struct vfio_pci_core_device +*vdev) { +if (!vdev->cxl_ops) +return ; + + vdev->cxl_ops->release (vdev); +vfio_pci_put_cxl_ops (vdev->cxl_ops); +} +int vfio_pci_core_init_dev (struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev = @@ -2223,6 +2318,10 @@ int
vfio_pci_core_init_dev(struct vfio_device *core_vdev)init_rwsem(&vdev->memory_lock); xa_init(&vdev->ctx);drivers/vfio/pci/vfio_pci_core.c +4
-0+ ret =vfio_pci_core_cxl_init (vdev); +if (ret) +return ret; +return 0 ; }EXPORT_SYMBOL_GPL (vfio_pci_core_init_dev);@@ -2232,6 +2331,8 @@ void vfio_pci_core_release_dev(struct vfio_device
*core_vdev)drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_release_dev(struct vfio_device *core_vdev) · line 2334
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0struct vfio_pci_core_device *vdev = container_of(core_vdev, struct vfio_pci_core_device, vdev); +vfio_pci_core_cxl_release (vdev);Sashiko has a doozy regarding the resulting lifecycle teardown:
---
[Severity: Critical]
Canvfio_pci_core_release_dev(), indrivers/vfio/pci/vfio_pci_core.c, invoke provider teardown after device-managed cleanup has freedvdev->cxlwhen final object destruction is delayed during unbind?In the final series,
vfio_cxl_create_memdev(), indrivers/vfio/pci/cxl/vfio_cxl_core.c, allocates the provider state throughdevm_cxl_dev_state_create()againstpdev->dev. This release hook instead runs at final vfio object destruction, which can follow driver removal and device-managed cleanup.Device-cdev close and unbind can interleave as follows:
vfio_device_fops_release()
drops the last registration reference
completes the unregister wait
vfio_pci_remove()
finishes unregistering
drops its vfio device reference
returns while the cdev still retains the vfio object
device_unbind_cleanup()
devres_release_all()frees the provider state
__fput()
cdev_put()drops the remaining parent referencevfio_pci_core_release_dev()calls provider teardownEnabling delayed kobject release provides another path without an open file. The module pin preserves the provider callback text, but does not preserve its device-managed state.
Quoted 8 lines
vfio_cxl_release_device(), indrivers/vfio/pci/cxl/vfio_cxl_core.c, then reads the freedcxl->hpa_rangeandcxl->hdmto release the reservation and free the decoder snapshot. Reuse of that allocation can cause an incorrect reservation release or an invalid free. The final series retains both the managed allocation and this final-release hook.Could provider teardown run before device-managed cleanup, or could the provider state remain allocated through final vfio release?
I agree this is a bug in v6. I will handle this in v7. vfio-cxl state is allocated with devm agains the PCI device and freed at unbind. But the teardown happens at final vfio_device release. This can come later if cdev file outlives the unbind.
For v7, I will setup CXL objects in vfio_pci_core_register_device and tear them down in vfio_pci_core_unregister_device. This way, nothing CXL is left in release_dev. CXL lifetime will be from open to close only.
I will make sure this is not introducing any bugs by running a local sashiko run.
Thanks,
Manish
Quoted 6 lines
---
So perhaps trying to shoehorn CXL's devm lifecycle into vfio-pci is biting us anyway. Thanks,
Alex
PATCH v6 8/31 vfio/pci: Add a generic excluded-range list
In reply to [email protected]From: Manish Honap <[email protected]>
A provider such as vfio-cxl needs to keep sub-ranges of a BAR off the direct guest path and serve them through its own trap. vfio-pci-core already has one such window, the MSI-X table, open-coded in vfio_pci_bar_rw().
Add vfio_pci_core_add_excluded_range(). Each BAR has its own list, sorted by start, and an insert that overlaps a range on the same BAR or ends past the BAR fails with -EINVAL. Ranges are added only while the device is being opened and are freed in vfio_pci_core_disable(), so readers on an open device take no lock.
The MSI-X table is virtualized in vfio_pci_bar_rw() by an open-coded x_start/x_end window that fills reads with -1 and drops writes. Register the MSI-X table as a read and write excluded range when the MSI-X capability is parsed in vfio_pci_core_enable(), and walk the BAR's ranges in order in vfio_pci_bar_rw(), so an access that spans several ranges is handled in one pass. Only the part of the table inside its BAR is registered, so a device whose MSI-X capability points outside the BAR still opens, as before. The ROM BAR keeps its trailing -1 window, outside the list.
No behavioral change for an MSI-X table in a standard BAR: an access to it still reads -1 and drops writes.
--- drivers/vfio/pci/vfio_pci_core.c | 97 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 8 +++ drivers/vfio/pci/vfio_pci_rdwr.c | 67 +++++++++++++++++++--- include/linux/vfio_pci_core.h | 20 +++++++ 4 files changed, 185 insertions(+), 7 deletions(-)
drivers/vfio/pci/vfio_pci_core.c +97 -0
@@ -588,6 +588,8 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = { NULL) }; +static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device *vdev); +int vfio_pci_core_enable (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev;
@@ -644,6 +646,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) msix_pos = pdev->msix_cap;if (msix_pos) { +u64 bar_len =0 ;u16 flags;u32 table;
@@ -654,6 +657,30 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) vdev->msix_offset = table &PCI_MSIX_TABLE_OFFSET ; vdev->msix_size = ((flags &PCI_MSIX_FLAGS_QSIZE ) +1 ) *16 ; vdev->has_dyn_msix =pci_msix_can_alloc_dyn (pdev); + +/* +* Virtualize the MSI-X table through the excluded-range list: +* reads fill -1 and writes are dropped so the guest never +* reaches the hardware table directly. BAR access never goes +* past the BAR, so only the part of the table inside the BAR +* needs a range; a table placed outside its BAR needs none. +*/ +if (vdev->msix_bar <PCI_STD_NUM_BARS ) + bar_len =pci_resource_len (pdev, vdev->msix_bar); +if (vdev->msix_offset < bar_len) { +u64 len =min_t (u64, vdev->msix_size, + bar_len - vdev->msix_offset); + + ret =vfio_pci_core_add_excluded_range (vdev, vdev->msix_bar, + vdev->msix_offset, len, +VFIO_PCI_EXCLUDE_READ | +VFIO_PCI_EXCLUDE_WRITE ); +if (ret) { +vfio_pci_free_excluded_ranges (vdev); +vfio_config_free (vdev); + goto out_free_zdev; + } + } }else { vdev->msix_bar =0xFF ; vdev->has_dyn_msix = false;
@@ -740,6 +767,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) vdev->region =NULL ;/* don't krealloc a freed pointer */ vfio_config_free (vdev); +vfio_pci_free_excluded_ranges (vdev);for (i =0 ; i <PCI_STD_NUM_BARS ; i++) { bar = i +PCI_STD_RESOURCES ;
@@ -1012,6 +1040,72 @@ static int msix_mmappable_cap(struct vfio_pci_core_device *vdev,return vfio_info_add_capability (caps, &header,sizeof (header)); } +/** +* vfio_pci_core_add_excluded_range() - keep a BAR sub-range off direct access +* @vdev: vfio PCI device being opened +* @bar: BAR index, below PCI_STD_NUM_BARS +* @start: offset of the range in @bar +* @size: length of the range in bytes +* @flags: VFIO_PCI_EXCLUDE_* paths the range is excluded from +* +* Call only while the device is being opened, from vfio_pci_core_enable() or +* a provider's open hook; vfio_pci_core_disable() frees the ranges. Ranges on +* one BAR must not overlap and must lie inside the BAR. +* +* Return: 0 on success, -EINVAL for an invalid or overlapping range or one +* that ends past the BAR, or -ENOMEM. +*/ +int vfio_pci_core_add_excluded_range (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 size,u32 flags) +{ +const u32 valid =VFIO_PCI_EXCLUDE_MMAP |VFIO_PCI_EXCLUDE_READ | +VFIO_PCI_EXCLUDE_WRITE ; +struct vfio_pci_excluded_range *range, *pos; +struct list_head *head; + +if (bar <0 || bar >=PCI_STD_NUM_BARS || !size || + start + size < start || !flags || (flags & ~valid)) +return -EINVAL ; +if (start + size >pci_resource_len (vdev->pdev, bar)) +return -EINVAL ; + +/* Keep the list sorted by start; find the first range after @start. */ + head = &vdev->excluded_ranges[bar]; +list_for_each_entry (pos, head, entry) { +if (start < pos->start + pos->size && start + size > pos->start) +return -EINVAL ; +if (pos->start > start) +break ; + } + + range =kmalloc_obj (*range); +if (!range) +return -ENOMEM ; + + *range = (struct vfio_pci_excluded_range ) { + .start = start, + .size = size, + .flags = flags, + }; +list_add_tail (&range->entry, &pos->entry); + +return 0 ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_add_excluded_range); + +static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device *vdev) +{ +struct vfio_pci_excluded_range *range, *tmp; + +for (int bar =0 ; bar <PCI_STD_NUM_BARS ; bar++) { +list_for_each_entry_safe (range, tmp, &vdev->excluded_ranges[bar], + entry) { +list_del (&range->entry); +kfree (range); + } + } +} + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,
@@ -2317,6 +2411,8 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)INIT_LIST_HEAD (&vdev->dmabufs);init_rwsem (&vdev->memory_lock);xa_init (&vdev->ctx); +for (int bar =0 ; bar <PCI_STD_NUM_BARS ; bar++) +INIT_LIST_HEAD (&vdev->excluded_ranges[bar]); ret =vfio_pci_core_cxl_init (vdev);if (ret)
@@ -2332,6 +2428,7 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev) container_of(core_vdev, struct vfio_pci_core_device, vdev);vfio_pci_core_cxl_release (vdev); +vfio_pci_free_excluded_ranges (vdev);mutex_destroy (&vdev->igate);mutex_destroy (&vdev->ioeventfds_lock);
drivers/vfio/pci/vfio_pci_priv.h +8 -0
@@ -44,6 +44,14 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev,ssize_t vfio_pci_bar_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite); +/* A BAR sub-range kept off direct access; see vfio_pci_core_add_excluded_range() */ +struct vfio_pci_excluded_range { +struct list_head entry; +u64 start; +u64 size; +u32 flags; +}; +#ifdef CONFIG_VFIO_PCI_VGA ssize_t vfio_pci_vga_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite);
drivers/vfio/pci/vfio_pci_rdwr.c +60 -7
79 lines not on this page · raw message
include/linux/vfio_pci_core.h +20 -0
@@ -162,6 +162,13 @@ struct vfio_pci_core_device {struct notifier_block nb;struct rw_semaphore memory_lock;struct list_head dmabufs; +/* +* Excluded BAR sub-ranges, one list per BAR, sorted by start and not +* overlapping. The lists are filled only while the device is being +* opened and emptied in vfio_pci_core_disable(); every reader runs on +* an open device in between, so readers take no lock. +*/ +struct list_head excluded_ranges[PCI_STD_NUM_BARS ]; };enum vfio_pci_io_width {
@@ -172,6 +179,19 @@ enum vfio_pci_io_width { };/* Will be exported for vfio pci drivers usage */ +/* +* A provider can keep a BAR sub-range off the direct guest path, reached only +* through its own trap. The flags select which paths are excluded: mmap, and +* region read and write (an excluded read fills -1, an excluded write is +* dropped). +*/ +#define VFIO_PCI_EXCLUDE_MMAP BIT(0) +#define VFIO_PCI_EXCLUDE_READ BIT(1) +#define VFIO_PCI_EXCLUDE_WRITE BIT(2) + +int vfio_pci_core_add_excluded_range (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 size,u32 flags); + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 08/31] vfio/pci: Add a generic excluded-range list
Quoted 58 lines · On Fri, 16 Oct 2026 21:48:15 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A provider such as vfio-cxl needs to keep sub-ranges of a BAR off the direct guest path and serve them through its own trap. vfio-pci-core already has one such window, the MSI-X table, open-coded in
vfio_pci_bar_rw().Add
vfio_pci_core_add_excluded_range(). Each BAR has its own list, sorted by start, and an insert that overlaps a range on the same BAR or ends past the BAR fails with-EINVAL. Ranges are added only while the device is being opened and are freed invfio_pci_core_disable(), so readers on an open device take no lock.The MSI-X table is virtualized in
vfio_pci_bar_rw()by an open-coded x_start/x_end window that fills reads with -1 and drops writes. Register the MSI-X table as a read and write excluded range when the MSI-X capability is parsed invfio_pci_core_enable(), and walk the BAR's ranges in order invfio_pci_bar_rw(), so an access that spans several ranges is handled in one pass. Only the part of the table inside its BAR is registered, so a device whose MSI-X capability points outside the BAR still opens, as before. The ROM BAR keeps its trailing -1 window, outside the list.No behavioral change for an MSI-X table in a standard BAR: an access to it still reads -1 and drops writes.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 97 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 8 +++ drivers/vfio/pci/vfio_pci_rdwr.c | 67 +++++++++++++++++++--- include/linux/vfio_pci_core.h | 20 +++++++ 4 files changed, 185 insertions(+), 7 deletions(-)drivers/vfio/pci/vfio_pci_core.c +3
-0@@ -588,6 +588,8 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = { NULL) }; +static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device *vdev); +int vfio_pci_core_enable (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev;@@ -644,6 +646,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) msix_pos = pdev->msix_cap;if (msix_pos) { +u64 bar_len =0 ;u16 flags;u32 table;@@ -654,6 +657,30 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) vdev->msix_offset = table &PCI_MSIX_TABLE_OFFSET ;
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 667
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +8
-0vdev->msix_size = ((flags &PCI_MSIX_FLAGS_QSIZE ) +1 ) *16 ; vdev->has_dyn_msix =pci_msix_can_alloc_dyn (pdev); + +/* +* Virtualize the MSI-X table through the excluded-range list: +* reads fill -1 and writes are dropped so the guest never +* reaches the hardware table directly. BAR access never goes +* past the BAR, so only the part of the table inside the BAR +* needs a range; a table placed outside its BAR needs none. +*/
AI comments.
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 677
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+if (vdev->msix_bar <PCI_STD_NUM_BARS ) + bar_len =pci_resource_len (pdev, vdev->msix_bar); +if (vdev->msix_offset < bar_len) { +u64 len =min_t (u64, vdev->msix_size, + bar_len - vdev->msix_offset); + + ret =vfio_pci_core_add_excluded_range (vdev, vdev->msix_bar, + vdev->msix_offset, len, +VFIO_PCI_EXCLUDE_READ | +VFIO_PCI_EXCLUDE_WRITE );
This is not a terribly elegant flow and I think the AI comment above misses the main point of what the apparently redundant BAR range and size cap test are trying to accomplish, compatibility.
In the current code, it doesn't matter if BIR is bogus or the offset doesn't land in the BAR length or size overflows, those just generate non-matches when we read or write.
I think this would be cleaner as a separate function that wraps the new function, returning success (perhaps even with a pci_warn_once) for an out of range BIR or offset, and performs silent cleanup on a range that exceeds the BAR, with a human generated comment explaining the compatibility argument.
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 681
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0+if (ret) { +vfio_pci_free_excluded_ranges (vdev); +vfio_config_free (vdev); + goto out_free_zdev;
This is mixing local cleanup with an unwind goto and it bites us later in the series when we end up leaving this in place and adding:
out_free_config:
/* A failed first open never reaches vfio_pci_core_disable(). */
vfio_pci_core_free_dev_regions(vdev);
vfio_pci_core_unmap_bars(vdev);
vfio_pci_free_excluded_ranges(vdev);
vfio_config_free(vdev);after the cxl open.
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) · line 770
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0+ } + } }else { vdev->msix_bar =0xFF ; vdev->has_dyn_msix = false;@@ -740,6 +767,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) vdev->region =NULL ;/* don't krealloc a freed pointer */ vfio_config_free (vdev); +vfio_pci_free_excluded_ranges (vdev);
Order is wrong, should be reverse of setup order. Also doesn't match the correct ordering used in the error unwind above.
drivers/vfio/pci/vfio_pci_core.c static int msix_mmappable_cap(struct vfio_pci_core_device *vdev, · line 1044
Quoted 9 lines
drivers/vfio/pci/vfio_pci_core.c +2
-0for (i =0 ; i <PCI_STD_NUM_BARS ; i++) { bar = i +PCI_STD_RESOURCES ;@@ -1012,6 +1040,72 @@ static int msix_mmappable_cap(struct vfio_pci_core_device *vdev,return vfio_info_add_capability (caps, &header,sizeof (header)); } +/** +* vfio_pci_core_add_excluded_range() - keep a BAR sub-range off direct access
"Exclude a BAR sub-range from user access"
drivers/vfio/pci/vfio_pci_core.c static int msix_mmappable_cap(struct vfio_pci_core_device *vdev, · line 1053
Quoted 9 lines
drivers/vfio/pci/vfio_pci_core.c +9
-0+* @vdev: vfio PCI device being opened +* @bar: BAR index, below PCI_STD_NUM_BARS +* @start: offset of the range in @bar +* @size: length of the range in bytes +* @flags: VFIO_PCI_EXCLUDE_* paths the range is excluded from +* +* Call only while the device is being opened, from vfio_pci_core_enable() or +* a provider's open hook; vfio_pci_core_disable() frees the ranges. Ranges on +* one BAR must not overlap and must lie inside the BAR.
Are we pushing an overly complicated model just to align with MSI-X?
I think MSI-X is configured on open_device because it's trivial. Just a couple config reads and extract the info. Now it's becoming more complicated with allocations and lists, and I'm not sure why it doesn't at the same time move to the .init and .release callbacks instead.
The AI comment also skips the important note reminding us that the exclude list is lockless and must be immutable while the device is opened, wherever we decide that boundary should be.
Quoted 80 lines
drivers/vfio/pci/vfio_pci_core.c +58
-0+* +* Return: 0 on success, -EINVAL for an invalid or overlapping range or one +* that ends past the BAR, or -ENOMEM. +*/ +int vfio_pci_core_add_excluded_range (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 size,u32 flags) +{ +const u32 valid =VFIO_PCI_EXCLUDE_MMAP |VFIO_PCI_EXCLUDE_READ | +VFIO_PCI_EXCLUDE_WRITE ; +struct vfio_pci_excluded_range *range, *pos; +struct list_head *head; + +if (bar <0 || bar >=PCI_STD_NUM_BARS || !size || + start + size < start || !flags || (flags & ~valid)) +return -EINVAL ; +if (start + size >pci_resource_len (vdev->pdev, bar)) +return -EINVAL ; + +/* Keep the list sorted by start; find the first range after @start. */ + head = &vdev->excluded_ranges[bar]; +list_for_each_entry (pos, head, entry) { +if (start < pos->start + pos->size && start + size > pos->start) +return -EINVAL ; +if (pos->start > start) +break ; + } + + range =kmalloc_obj (*range); +if (!range) +return -ENOMEM ; + + *range = (struct vfio_pci_excluded_range ) { + .start = start, + .size = size, + .flags = flags, + }; +list_add_tail (&range->entry, &pos->entry); + +return 0 ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_add_excluded_range); + +static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device *vdev) +{ +struct vfio_pci_excluded_range *range, *tmp; + +for (int bar =0 ; bar <PCI_STD_NUM_BARS ; bar++) { +list_for_each_entry_safe (range, tmp, &vdev->excluded_ranges[bar], + entry) { +list_del (&range->entry); +kfree (range); + } + } +} + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,@@ -2317,6 +2411,8 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)INIT_LIST_HEAD (&vdev->dmabufs);init_rwsem (&vdev->memory_lock);xa_init (&vdev->ctx); +for (int bar =0 ; bar <PCI_STD_NUM_BARS ; bar++) +INIT_LIST_HEAD (&vdev->excluded_ranges[bar]); ret =vfio_pci_core_cxl_init (vdev);if (ret)@@ -2332,6 +2428,7 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev) container_of(core_vdev, struct vfio_pci_core_device, vdev);vfio_pci_core_cxl_release (vdev); +vfio_pci_free_excluded_ranges (vdev);mutex_destroy (&vdev->igate);mutex_destroy (&vdev->ioeventfds_lock);drivers/vfio/pci/vfio_pci_priv.h +0
-0
drivers/vfio/pci/vfio_pci_priv.h ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev, · line 53
Quoted 10 lines
drivers/vfio/pci/vfio_pci_priv.h +7
-0@@ -44,6 +44,14 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev,ssize_t vfio_pci_bar_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite); +/* A BAR sub-range kept off direct access; see vfio_pci_core_add_excluded_range() */ +struct vfio_pci_excluded_range { +struct list_head entry; +u64 start; +u64 size; +u32 flags; +};
If we used struct range as a member here we could use range_contains() and range_overlaps(), for finding a matching range below and avoiding conflicts above (mind the inclusive end).
Quoted 52 lines
drivers/vfio/pci/vfio_pci_priv.h +1
-0+#ifdef CONFIG_VFIO_PCI_VGA ssize_t vfio_pci_vga_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite);drivers/vfio/pci/vfio_pci_rdwr.c +40
-0@@ -198,6 +198,57 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem, }EXPORT_SYMBOL_GPL (vfio_pci_core_do_io_rw); +/* +* Access @count bytes of @bar at @pos, walking the BAR's excluded ranges in +* order: a range excluded for this direction reads as -1 and drops writes, +* and the bytes around it reach the device. The MSI-X table is one such +* range, and a BAR can hold several, for example the MSI-X table and a CXL +* HDM decoder block. +*/ +static ssize_t vfio_pci_bar_do_io_rw (struct vfio_pci_core_device *vdev, +int bar,bool test_mem,void __iomem *io, +char __user *buf,loff_t pos, +size_t count,bool iswrite, +enum vfio_pci_io_width max_width) +{ +u32 want = iswrite ?VFIO_PCI_EXCLUDE_WRITE :VFIO_PCI_EXCLUDE_READ ; +struct vfio_pci_excluded_range *range; +ssize_t done =0 ; +ssize_t ret; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +u64 end = range->start + range->size; +size_t chunk; + +if (!(range->flags & want) || end <= pos) +continue ; +if (range->start >= pos + count) +break ; + +/* Up to the end of this range; do_io_rw() fills it. */ + chunk =min_t (u64, count, end - pos); + ret =vfio_pci_core_do_io_rw (vdev, test_mem, io, buf, pos, chunk, + range->start, end, iswrite, + max_width); +if (ret <0 ) +return done ?: ret; + + done += ret; + pos += ret; + buf += ret; + count -= ret; +if (!count)
drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem, · line 250
Quoted 10 lines
drivers/vfio/pci/vfio_pci_rdwr.c +10
-0+return done; + } + + ret =vfio_pci_core_do_io_rw (vdev, test_mem, io, buf, pos, count,0 ,0 , + iswrite, max_width); +if (ret <0 ) +return done ?: ret; + +return done + ret; +}
This is mixing fault semantics. Currently on fault we return the errno, regardless of work that's been done. With this, we potentially return a short read/write on fault if later errors occur. That short return is not necessarily representative of the length of the actual operation, a partial operation in the last call to core_do_io_rw is discarded.
Therefore, I think both of the above (ret < 0) return branches should just return ret, not the potentially incorrect done value.
Quoted 14 lines
drivers/vfio/pci/vfio_pci_rdwr.c +1
-6+ssize_t vfio_pci_bar_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite) {@@ -256,13 +307,15 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf, } } -if (bar == vdev->msix_bar) { - x_start = vdev->msix_offset; - x_end = vdev->msix_offset + vdev->msix_size; - } - - done = vfio_pci_core_do_io_rw(vdev, res->flags & IORESOURCE_MEM, io, buf, pos,
drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf, · line 318
Quoted 10 lines
drivers/vfio/pci/vfio_pci_rdwr.c +9
-1- count, x_start, x_end, iswrite, max_width); +if (bar ==PCI_ROM_RESOURCE ) + done =vfio_pci_core_do_io_rw (vdev, res->flags &IORESOURCE_MEM , + io, buf, pos, count, x_start, + x_end, iswrite, max_width); +else + done =vfio_pci_bar_do_io_rw (vdev, bar, + res->flags &IORESOURCE_MEM , io, + buf, pos, count, iswrite, + max_width);
Seems like the wrong refactor to add a ROM specific branch, we should probably instead precede the excluded range loop by a test for a standard BAR and non-empty excluded range list. That might mean the new function is reduced to only the excluded range handling and all non-excluded range handling falls through to a common call to the core function.
Quoted 8 lines
drivers/vfio/pci/vfio_pci_rdwr.c +0
-0if (done >=0 ) *ppos += done;include/linux/vfio_pci_core.h +0
-0
include/linux/vfio_pci_core.h struct vfio_pci_core_device { · line 170
Quoted 9 lines
include/linux/vfio_pci_core.h +6
-0@@ -162,6 +162,13 @@ struct vfio_pci_core_device {struct notifier_block nb;struct rw_semaphore memory_lock;struct list_head dmabufs; +/* +* Excluded BAR sub-ranges, one list per BAR, sorted by start and not +* overlapping. The lists are filled only while the device is being +* opened and emptied in vfio_pci_core_disable(); every reader runs on +* an open device in between, so readers take no lock. +*/
I'll stop noting these, but I expect all gratuitous AI comments to be removed by default and evaluated as a human whether a comment is warranted. AI assistance doesn't relieve authors of the responsibility to perform a thorough human self review. /dismount-soapbox Thanks,
Alex
Quoted 24 lines
include/linux/vfio_pci_core.h +14
-0+struct list_head excluded_ranges[PCI_STD_NUM_BARS ]; };enum vfio_pci_io_width {@@ -172,6 +179,19 @@ enum vfio_pci_io_width { };/* Will be exported for vfio pci drivers usage */ +/* +* A provider can keep a BAR sub-range off the direct guest path, reached only +* through its own trap. The flags select which paths are excluded: mmap, and +* region read and write (an excluded read fills -1, an excluded write is +* dropped). +*/ +#define VFIO_PCI_EXCLUDE_MMAP BIT(0) +#define VFIO_PCI_EXCLUDE_READ BIT(1) +#define VFIO_PCI_EXCLUDE_WRITE BIT(2) + +int vfio_pci_core_add_excluded_range (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 size,u32 flags); + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,
RE: [PATCH v6 08/31] vfio/pci: Add a generic excluded-range list
Quoted 119 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Friday, November 6, 2026 9:30 PM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 08/31] vfio/pci: Add a generic excluded-range listExternal email: Use caution opening links or attachments
drivers/vfio/pci/vfio_pci_core.c static const struct dev_pm_ops
Quoted 41 lines · On Fri, 16 Oct 2026 21:48:15 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A provider such as vfio-cxl needs to keep sub-ranges of a BAR off the direct guest path and serve them through its own trap. vfio-pci-core already has one such window, the MSI-X table, open-coded in
vfio_pci_bar_rw().Add
vfio_pci_core_add_excluded_range(). Each BAR has its own list, sorted by start, and an insert that overlaps a range on the same BAR or ends past the BAR fails with-EINVAL. Ranges are added only while the device is being opened and are freed invfio_pci_core_disable(), so readers on an open device take no lock.The MSI-X table is virtualized in
vfio_pci_bar_rw()by an open-coded x_start/x_end window that fills reads with -1 and drops writes. Register the MSI-X table as a read and write excluded range when the MSI-X capability is parsed invfio_pci_core_enable(), and walk the BAR's ranges in order invfio_pci_bar_rw(), so an access that spans several ranges is handled in one pass. Only the part of the table inside its BAR is registered, so a device whose MSI-X capability points outside the BAR still opens, as before. The ROM BAR keeps its trailing -1 window, outside the list.No behavioral change for an MSI-X table in a standard BAR: an access to it still reads -1 and drops writes.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 97 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 8 +++ drivers/vfio/pci/vfio_pci_rdwr.c | 67 +++++++++++++++++++--- include/linux/vfio_pci_core.h | 20 +++++++ 4 files changed, 185 insertions(+), 7 deletions(-)drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -588,6 +588,8 @@ static const struct dev_pm_ops
vfio_pci_core_pm_ops= {drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct
Quoted 17 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0NULL) }; +static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device +*vdev); +int vfio_pci_core_enable (struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; @@ -644,6 +646,7 @@ int
vfio_pci_core_enable(struct vfio_pci_core_device *vdev)msix_pos = pdev->msix_cap; if (msix_pos) {drivers/vfio/pci/vfio_pci_core.c +1
-0+u64 bar_len =0 ;u16 flags;u32 table;@@ -654,6 +657,30 @@ int vfio_pci_core_enable(struct
vfio_pci_core_device*vdev)drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 667
Quoted 11 lines
drivers/vfio/pci/vfio_pci_core.c +8
-0vdev->msix_offset = table &PCI_MSIX_TABLE_OFFSET ; vdev->msix_size = ((flags &PCI_MSIX_FLAGS_QSIZE ) +1 ) *16 ; vdev->has_dyn_msix =pci_msix_can_alloc_dyn (pdev); + +/* +* Virtualize the MSI-X table through the excluded-range list: +* reads fill -1 and writes are dropped so the guest never +* reaches the hardware table directly. BAR access never goes +* past the BAR, so only the part of the table inside the BAR +* needs a range; a table placed outside its BAR needs none. +*/ AI comments.
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 677
Quoted 11 lines
drivers/vfio/pci/vfio_pci_core.c +11
-0+if (vdev->msix_bar <PCI_STD_NUM_BARS ) + bar_len =pci_resource_len (pdev, vdev->msix_bar); +if (vdev->msix_offset < bar_len) { +u64 len =min_t (u64, vdev->msix_size, + bar_len - vdev->msix_offset); + + ret =vfio_pci_core_add_excluded_range (vdev, vdev->msix_bar, + vdev->msix_offset, len, +VFIO_PCI_EXCLUDE_READ | + +VFIO_PCI_EXCLUDE_WRITE );This is not a terribly elegant flow and I think the AI comment above misses the main point of what the apparently redundant BAR range and size cap test are trying to accomplish, compatibility.
In the current code, it doesn't matter if BIR is bogus or the offset doesn't land in the BAR length or size overflows, those just generate non-matches when we read or write.
Quoted 5 lines
I think this would be cleaner as a separate function that wraps the new function, returning success (perhaps even with a
pci_warn_once) for an out of range BIR or offset, and performs silent cleanup on a range that exceeds the BAR, with a human generated comment explaining the compatibility argument.
Okay, Agreed. For v7 I will add the MSI-X range in its own helper: a bad BIR or offset will not add and provide a warning. A table that runs past the BAR is clipped. I will also update the comment correctly removing the AI-slop.
Quoted 8 lines
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 681
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0+if (ret) { +vfio_pci_free_excluded_ranges (vdev); +vfio_config_free (vdev); + goto out_free_zdev;This is mixing local cleanup with an unwind goto and it bites us later in the series when we end up leaving this in place and adding:
Okay, I will add a separate label in unwind for this.
Quoted 18 lines
out_free_config:/* A failed first open never reaches vfio_pci_core_disable(). */ vfio_pci_core_free_dev_regions(vdev); vfio_pci_core_unmap_bars(vdev); vfio_pci_free_excluded_ranges(vdev); vfio_config_free(vdev);after the cxl open.
Quoted 8 lines
drivers/vfio/pci/vfio_pci_core.c +2
-0+ } + } }else { vdev->msix_bar =0xFF ; vdev->has_dyn_msix = false; @@ -740,6 +767,7 @@ void
vfio_pci_core_disable(struct vfio_pci_core_device *vdev)vdev->region = NULL; /* don't krealloc a freed pointer */
Quoted 5 lines
Quoted 2 lines
vfio_config_free(vdev);+
vfio_pci_free_excluded_ranges(vdev);Order is wrong, should be reverse of setup order. Also doesn't match the correct ordering used in the error unwind above.
I will fix this in v7.
Quoted 8 lines
Quoted 8 lines
for (i = 0; i < PCI_STD_NUM_BARS; i++) { bar = i + PCI_STD_RESOURCES; @@ -1012,6 +1040,72 @@ static int msix_mmappable_cap(struct vfio_pci_core_device *vdev, return vfio_info_add_capability(caps, &header, sizeof(header)); }
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c
Quoted 3 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0+/** +* vfio_pci_core_add_excluded_range() - keep a BAR sub-range off +direct access "Exclude a BAR sub-range from user access"
Okay, I will update wording.
Quoted 15 lines
Quoted 12 lines
+ * @vdev: vfio PCI device being opened
+ * @bar: BAR index, belowPCI_STD_NUM_BARS
+ * @start: offset of the range in @bar
+ * @size: length of the range in bytes
+ * @flags: VFIO_PCI_EXCLUDE_* paths the range is excluded from
+ *
+ * Call only while the device is being opened, from
+vfio_pci_core_enable()or
+ * a provider's open hook;vfio_pci_core_disable()frees the ranges.
+ Ranges on
+ * one BAR must not overlap and must lie inside the BAR.Are we pushing an overly complicated model just to align with MSI-X?
Quoted 9 lines
I think MSI-X is configured on
open_devicebecause it's trivial. Just a couple config reads and extract the info. Now it's becoming more complicated with allocations and lists, and I'm not sure why it doesn't at the same time move to the .init and .release callbacks instead.The AI comment also skips the important note reminding us that the exclude list is lockless and must be immutable while the device is opened, wherever we decide that boundary should be.
vfio-cxl knows its ranges at bind and the MSI-X table does not move. So, for v7, I will build the list in .init and free it in .release. The list then does not change while the device exists. I will update the comment accordingly.
Quoted 90 lines
drivers/vfio/pci/vfio_pci_core.c static int msix_mmappable_cap(struct vfio_pci_core_device *vdev,
Quoted 6 lines
+ *
+ * Return: 0 on success,-EINVALfor an invalid or overlapping rangedrivers/vfio/pci/vfio_pci_core.c +4
-0+or one +* that ends past the BAR, or -ENOMEM. +*/ +int vfio_pci_core_add_excluded_range(struct vfio_pci_core_device *vdev,int bar,
drivers/vfio/pci/vfio_pci_core.c
Quoted 2 lines
drivers/vfio/pci/vfio_pci_core.c +2
-0+ u64 start, u64 size, u32 flags) { + const u32 valid = VFIO_PCI_EXCLUDE_MMAP |
VFIO_PCI_EXCLUDE_READ|drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_init_dev(struct vfio_device
Quoted 51 lines
drivers/vfio/pci/vfio_pci_core.c +47
-0+VFIO_PCI_EXCLUDE_WRITE ; +struct vfio_pci_excluded_range *range, *pos; +struct list_head *head; + +if (bar <0 || bar >=PCI_STD_NUM_BARS || !size || + start + size < start || !flags || (flags & ~valid)) +return -EINVAL ; +if (start + size >pci_resource_len (vdev->pdev, bar)) +return -EINVAL ; + +/* Keep the list sorted by start; find the first range after @start. */ + head = &vdev->excluded_ranges[bar]; +list_for_each_entry (pos, head, entry) { +if (start < pos->start + pos->size && start + size > pos->start) +return -EINVAL ; +if (pos->start > start) +break ; + } + + range =kmalloc_obj (*range); +if (!range) +return -ENOMEM ; + + *range = (struct vfio_pci_excluded_range ) { + .start = start, + .size = size, + .flags = flags, + }; +list_add_tail (&range->entry, &pos->entry); + +return 0 ; +} +EXPORT_SYMBOL_GPL (vfio_pci_core_add_excluded_range); + +static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device +*vdev) { +struct vfio_pci_excluded_range *range, *tmp; + +for (int bar =0 ; bar <PCI_STD_NUM_BARS ; bar++) { +list_for_each_entry_safe (range, tmp, &vdev->excluded_ranges[bar], + entry) { +list_del (&range->entry); +kfree (range); + } + } +} + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,@@ -2317,6 +2411,8 @@ int vfio_pci_core_init_dev(struct vfio_device
*core_vdev)drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_release_dev(struct
Quoted 9 lines
drivers/vfio/pci/vfio_pci_core.c +2
-0INIT_LIST_HEAD (&vdev->dmabufs);init_rwsem (&vdev->memory_lock);xa_init (&vdev->ctx); +for (int bar =0 ; bar <PCI_STD_NUM_BARS ; bar++) +INIT_LIST_HEAD (&vdev->excluded_ranges[bar]); ret =vfio_pci_core_cxl_init (vdev);if (ret)@@ -2332,6 +2428,7 @@ void vfio_pci_core_release_dev(struct
vfio_device*core_vdev)Quoted 14 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0container_of(core_vdev, struct vfio_pci_core_device, vdev);vfio_pci_core_cxl_release (vdev); +vfio_pci_free_excluded_ranges (vdev);mutex_destroy (&vdev->igate);mutex_destroy(&vdev->ioeventfds_lock);drivers/vfio/pci/vfio_pci_priv.h +0
-0@@ -44,6 +44,14 @@ ssize_t vfio_pci_config_rw_single(structQuoted 1 line
vfio_pci_core_device*vdev,ssize_tvfio_pci_bar_rw(struct
vfio_pci_core_device*vdev, char__user*buf,Quoted 2 lines
size_t count, loff_t *ppos, bool iswrite);
Quoted 11 lines
drivers/vfio/pci/vfio_pci_priv.h ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev, · line 53
Quoted 7 lines
drivers/vfio/pci/vfio_pci_priv.h +7
-0+/* A BAR sub-range kept off direct access; see +vfio_pci_core_add_excluded_range() */ struct vfio_pci_excluded_range { +struct list_head entry; +u64 start; +u64 size; +u32 flags; +};If we used struct range as a member here we could use
range_contains()andrange_overlaps(), for finding a matching range below and avoiding conflicts above (mind the inclusive end).
Noted. I will update this.
Quoted 75 lines
Quoted 21 lines
+
#ifdefCONFIG_VFIO_PCI_VGA
ssize_tvfio_pci_vga_rw(structvfio_pci_core_device*vdev, char__user*buf,
size_tcount,loff_t*ppos, bool iswrite); diff
--gita/drivers/vfio/pci/vfio_pci_rdwr.cb/drivers/vfio/pci/vfio_pci_rdwr.c
index7f14dd46de17..93823c6391f6100644drivers/vfio/pci/vfio_pci_rdwr.c +7
-0@@ -198,6 +198,57 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem, }EXPORT_SYMBOL_GPL (vfio_pci_core_do_io_rw ); +/* +* Access @count bytes of @bar at @pos, walking the BAR's excluded +ranges in +* order: a range excluded for this direction reads as -1 and drops +writes, +* and the bytes around it reach the device. The MSI-X table is one +such drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_core_do_io_rw(struct · line 217
Quoted 10 lines
drivers/vfio/pci/vfio_pci_rdwr.c +10
-0+* range, and a BAR can hold several, for example the MSI-X table and +a CXL +* HDM decoder block. +*/ +static ssize_t vfio_pci_bar_do_io_rw (struct vfio_pci_core_device *vdev, +int bar,bool test_mem,void __iomem *io, +char __user *buf,loff_t pos, +size_t count,bool iswrite, +enum vfio_pci_io_width max_width) { + u32 want = iswrite ? VFIO_PCI_EXCLUDE_WRITE :
VFIO_PCI_EXCLUDE_READ;Quoted 26 lines
drivers/vfio/pci/vfio_pci_rdwr.c +26
-0+struct vfio_pci_excluded_range *range; +ssize_t done =0 ; +ssize_t ret; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +u64 end = range->start + range->size; +size_t chunk; + +if (!(range->flags & want) || end <= pos) +continue ; +if (range->start >= pos + count) +break ; + +/* Up to the end of this range; do_io_rw() fills it. */ + chunk =min_t (u64, count, end - pos); + ret =vfio_pci_core_do_io_rw (vdev, test_mem, io, buf, pos, chunk, + range->start, end, iswrite, + max_width); +if (ret <0 ) +return done ?: ret; + + done += ret; + pos += ret; + buf += ret; + count -= ret; +if (!count)drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem, · line 250
Quoted 10 lines
drivers/vfio/pci/vfio_pci_rdwr.c +10
-0+return done; + } + + ret =vfio_pci_core_do_io_rw (vdev, test_mem, io, buf, pos, count,0 ,0 , + iswrite, max_width); +if (ret <0 ) +return done ?: ret; + +return done + ret; +}This is mixing fault semantics. Currently on fault we return the errno, regardless of work that's been done. With this, we potentially return a short read/write on fault if later errors occur. That short return is not necessarily representative of the length of the actual operation, a partial operation in the last call to
core_do_io_rwis discarded.
Quoted 3 lines
Therefore, I think both of the above (ret < 0) return branches should just return ret, not the potentially incorrect done value.
Okay, I will update this.
Quoted 28 lines
drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
Quoted 5 lines
drivers/vfio/pci/vfio_pci_rdwr.c +1
-1+ssize_t vfio_pci_bar_rw (struct vfio_pci_core_device *vdev,char __user *buf, size_t count, loff_t *ppos, bool iswrite) { @@ -256,13 +307,15 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device*vdev, char
__user*buf,drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
Quoted 9 lines
} }drivers/vfio/pci/vfio_pci_rdwr.c +0
-6-if (bar == vdev->msix_bar) { - x_start = vdev->msix_offset; - x_end = vdev->msix_offset + vdev->msix_size; - } - - done = vfio_pci_core_do_io_rw(vdev, res->flags & IORESOURCE_MEM,io, buf, pos,
drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
Quoted 3 lines
drivers/vfio/pci/vfio_pci_rdwr.c +2
-1- count, x_start, x_end, iswrite, max_width); +if (bar ==PCI_ROM_RESOURCE ) + done = vfio_pci_core_do_io_rw(vdev, res->flags &
IORESOURCE_MEM,drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf, · line 318
Quoted 7 lines
drivers/vfio/pci/vfio_pci_rdwr.c +7
-0+ io, buf, pos, count, x_start, + x_end, iswrite, max_width); + else + done =vfio_pci_bar_do_io_rw (vdev, bar, + res->flags &IORESOURCE_MEM , io, + buf, pos, count, iswrite, + max_width);
Quoted 5 lines
Seems like the wrong refactor to add a ROM specific branch, we should probably instead precede the excluded range loop by a test for a standard BAR and non-empty excluded range list. That might mean the new function is reduced to only the excluded range handling and all non-excluded range handling falls through to a common call to the core function.
Okay, for v7, only a standard BAR with ranges walks the list; everything else falls through to the one vfio_pci_core_do_io_rw() call.
Quoted 16 lines
Quoted 16 lines
if (done >= 0) *ppos += done;(quoted diff) +0
-0
b/include/linux/vfio_pci_core.hindex7f3a2bcb5830..70acb03d466e100644include/linux/vfio_pci_core.h +3
-0@@ -162,6 +162,13 @@ struct vfio_pci_core_device {struct notifier_block nb;struct rw_semaphore memory_lock;struct list_head dmabufs; +/* +* Excluded BAR sub-ranges, one list per BAR, sorted by start and not +* overlapping. The lists are filled only while the device is being
Quoted 8 lines
include/linux/vfio_pci_core.h struct vfio_pci_core_device { · line 170
Quoted 3 lines
include/linux/vfio_pci_core.h +3
-0+* opened and emptied in vfio_pci_core_disable(); every reader runs on +* an open device in between, so readers take no lock. +*/ I'll stop noting these, but I expect all gratuitous AI comments to be removed by default and evaluated as a human whether a comment is warranted. AI assistance doesn't relieve authors of the responsibility to perform a thorough human self review. /dismount-soapbox Thanks,
Agreed. Sorry for this churn. For v7, I will recheck all the comments in the patches before posting.
Thanks,
Manish
Quoted 30 lines
Alex
Quoted 11 lines
include/linux/vfio_pci_core.h +5
-0+struct list_head excluded_ranges[PCI_STD_NUM_BARS ]; };enum vfio_pci_io_width {@@ -172,6 +179,19 @@ enum vfio_pci_io_width { };/* Will be exported for vfio pci drivers usage */ +/* +* A provider can keep a BAR sub-range off the direct guest path, +reached only +* through its own trap. The flags select which paths are excluded: include/linux/vfio_pci_core.h enum vfio_pci_io_width { }; · line 194
Quoted 10 lines
include/linux/vfio_pci_core.h +10
-0+mmap, and +* region read and write (an excluded read fills -1, an excluded +write is +* dropped). +*/ +#define VFIO_PCI_EXCLUDE_MMAP BIT(0) +#define VFIO_PCI_EXCLUDE_READ BIT(1) +#define VFIO_PCI_EXCLUDE_WRITE BIT(2) + +int vfio_pci_core_add_excluded_range(struct vfio_pci_core_device *vdev,int bar,
Quoted 5 lines
include/linux/vfio_pci_core.h +2
-0+ u64 start, u64 size, u32 flags); + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,
PATCH v6 9/31 vfio/pci: Advertise the mmap areas around excluded ranges
In reply to [email protected]From: Manish Honap <[email protected]>
A range excluded from mmap must not be reachable through an mmap of its BAR. Refuse an mmap that overlaps such a range, and advertise the BAR with a sparse mmap capability that lists the areas between the excluded ranges, so a VMM maps those and traps the rest.
mmap is page granular, so each range is rounded out to whole pages, and ranges that meet after rounding form one hole. The sorted list gives the areas in two walks: the first counts them and the second fills them.
A BAR without mmap-excluded ranges gets no new capability. The MSI-X table is excluded from read and write only, so the MSI-X BAR is unchanged.
--- drivers/vfio/pci/vfio_pci_core.c | 91 ++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+)
drivers/vfio/pci/vfio_pci_core.c +91 -0
@@ -1106,6 +1106,88 @@ static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev) } } +/* True when [start, start + len) on @bar overlaps a range with any of @flags. */ +static bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev, +int bar,u64 start,u64 len,u32 flags) +{ +struct vfio_pci_excluded_range *range; + +/* Only BARs have excluded ranges; a device-specific region has none. */ +if (bar >=PCI_STD_NUM_BARS ) +return false; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +if (range->start >= start + len) +break ; +if ((range->flags & flags) && + start < range->start + range->size) +return true; + } + +return false; +} + +/* +* Advertise a BAR with mmap-excluded ranges as the mmap-able areas between +* them. mmap is page granular, so each range is rounded out to whole pages; +* ranges that meet after rounding merge into one hole. The first walk counts +* the areas and the second fills them. +*/ +static int vfio_pci_excluded_sparse_cap (struct vfio_pci_core_device *vdev, +int bar,struct vfio_info_cap *caps) +{ + struct vfio_region_info_cap_sparse_mmap *sparse __free(kfree) = NULL; +u64 bar_len =pci_resource_len (vdev->pdev, bar); +struct vfio_pci_excluded_range *range; +u32 nr_areas =0 ; +bool holes = false; +u64 pos =0 ; +u32 i =0 ; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +if (!(range->flags &VFIO_PCI_EXCLUDE_MMAP )) +continue ; + holes = true; +if (ALIGN_DOWN (range->start,PAGE_SIZE ) > pos) + nr_areas++; + pos =max (pos,ALIGN (range->start + range->size,PAGE_SIZE )); + } +if (!holes) +return 0 ; +if (pos < bar_len) + nr_areas++; + + sparse =kzalloc_flex (*sparse, areas, nr_areas); +if (!sparse) +return -ENOMEM ; + + sparse->header.id =VFIO_REGION_INFO_CAP_SPARSE_MMAP ; + sparse->header.version =1 ; + sparse->nr_areas = nr_areas; + + pos =0 ; +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +u64 hole =ALIGN_DOWN (range->start,PAGE_SIZE ); + +if (!(range->flags &VFIO_PCI_EXCLUDE_MMAP )) +continue ; +if (hole > pos) + sparse->areas[i++] = (struct vfio_region_sparse_mmap_area ) { + .offset = pos, + .size = hole - pos, + }; + pos =max (pos,ALIGN (range->start + range->size,PAGE_SIZE )); + } +if (pos < bar_len) + sparse->areas[i] = (struct vfio_region_sparse_mmap_area ) { + .offset = pos, + .size = bar_len - pos, + }; + +return vfio_info_add_capability (caps, &sparse->header, +struct_size (sparse, areas, nr_areas)); +} + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,
@@ -1254,6 +1336,10 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev,if (ret)return ret; } + ret =vfio_pci_excluded_sparse_cap (vdev, info->index, + caps); +if (ret) +return ret; }break ;
@@ -1947,6 +2033,11 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vmaif (req_start + req_len > phys_len)return -EINVAL ; +/* An excluded sub-range is reachable only through its trap, not mmap. */ +if (vfio_pci_bar_range_excluded (vdev, index, req_start, req_len, +VFIO_PCI_EXCLUDE_MMAP )) +return -EINVAL ; +/* * Ensure the BAR resource region is reserved for use. */
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 09/31] vfio/pci: Advertise the mmap areas around excluded ranges
Quoted 69 lines · On Fri, 16 Oct 2026 21:48:16 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A range excluded from mmap must not be reachable through an mmap of its BAR. Refuse an mmap that overlaps such a range, and advertise the BAR with a sparse mmap capability that lists the areas between the excluded ranges, so a VMM maps those and traps the rest.
mmap is page granular, so each range is rounded out to whole pages, and ranges that meet after rounding form one hole. The sorted list gives the areas in two walks: the first counts them and the second fills them.
A BAR without mmap-excluded ranges gets no new capability. The MSI-X table is excluded from read and write only, so the MSI-X BAR is unchanged.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 91 ++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+)drivers/vfio/pci/vfio_pci_core.c +40
-0@@ -1106,6 +1106,88 @@ static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev) } } +/* True when [start, start + len) on @bar overlaps a range with any of @flags. */ +static bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev, +int bar,u64 start,u64 len,u32 flags) +{ +struct vfio_pci_excluded_range *range; + +/* Only BARs have excluded ranges; a device-specific region has none. */ +if (bar >=PCI_STD_NUM_BARS ) +return false; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +if (range->start >= start + len) +break ; +if ((range->flags & flags) && + start < range->start + range->size) +return true; + } + +return false; +} + +/* +* Advertise a BAR with mmap-excluded ranges as the mmap-able areas between +* them. mmap is page granular, so each range is rounded out to whole pages; +* ranges that meet after rounding merge into one hole. The first walk counts +* the areas and the second fills them. +*/ +static int vfio_pci_excluded_sparse_cap (struct vfio_pci_core_device *vdev, +int bar,struct vfio_info_cap *caps) +{ + struct vfio_region_info_cap_sparse_mmap *sparse __free(kfree) = NULL; +u64 bar_len =pci_resource_len (vdev->pdev, bar); +struct vfio_pci_excluded_range *range; +u32 nr_areas =0 ; +bool holes = false; +u64 pos =0 ; +u32 i =0 ; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +if (!(range->flags &VFIO_PCI_EXCLUDE_MMAP ))
drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev) · line 1158
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+continue ; + holes = true; +if (ALIGN_DOWN (range->start,PAGE_SIZE ) > pos) + nr_areas++; + pos =max (pos,ALIGN (range->start + range->size,PAGE_SIZE )); + } +if (!holes) +return 0 ; +if (pos < bar_len) + nr_areas++;
The number of sparse ranges is bounded by (num_mmap_excluded_ranges + 1). We're dealing with a very low number of excluded ranges, bounded by insertion via kernel drivers and core. It seems worthwhile to avoid the duplication with a simple list walk that counts the mmap excluded ranges, then all of the alignment happens once, and the resulting nr_areas is calculated as we fill.
Note that with the use of __free, it's generally recommended to allocate at the point it's declared and mixed declarations and statements are allowed for this. We can therefore do the declaration and allocation after we've done the simple version of the counting loop. Thanks,
Alex
Quoted 58 lines
drivers/vfio/pci/vfio_pci_core.c +41
-0+ + sparse =kzalloc_flex (*sparse, areas, nr_areas); +if (!sparse) +return -ENOMEM ; + + sparse->header.id =VFIO_REGION_INFO_CAP_SPARSE_MMAP ; + sparse->header.version =1 ; + sparse->nr_areas = nr_areas; + + pos =0 ; +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +u64 hole =ALIGN_DOWN (range->start,PAGE_SIZE ); + +if (!(range->flags &VFIO_PCI_EXCLUDE_MMAP )) +continue ; +if (hole > pos) + sparse->areas[i++] = (struct vfio_region_sparse_mmap_area ) { + .offset = pos, + .size = hole - pos, + }; + pos =max (pos,ALIGN (range->start + range->size,PAGE_SIZE )); + } +if (pos < bar_len) + sparse->areas[i] = (struct vfio_region_sparse_mmap_area ) { + .offset = pos, + .size = bar_len - pos, + }; + +return vfio_info_add_capability (caps, &sparse->header, +struct_size (sparse, areas, nr_areas)); +} + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,@@ -1254,6 +1336,10 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev,if (ret)return ret; } + ret =vfio_pci_excluded_sparse_cap (vdev, info->index, + caps); +if (ret) +return ret; }break ;@@ -1947,6 +2033,11 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vmaif (req_start + req_len > phys_len)return -EINVAL ; +/* An excluded sub-range is reachable only through its trap, not mmap. */ +if (vfio_pci_bar_range_excluded (vdev, index, req_start, req_len, +VFIO_PCI_EXCLUDE_MMAP )) +return -EINVAL ; +/* * Ensure the BAR resource region is reserved for use. */
RE: [PATCH v6 09/31] vfio/pci: Advertise the mmap areas around excluded ranges
Quoted 122 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Saturday, November 7, 2026 2:09 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 09/31] vfio/pci: Advertise the mmap areas around excluded rangesExternal email: Use caution opening links or attachments
drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_free_excluded_ranges(struct
Quoted 28 lines · On Fri, 16 Oct 2026 21:48:16 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A range excluded from mmap must not be reachable through an mmap of its BAR. Refuse an mmap that overlaps such a range, and advertise the BAR with a sparse mmap capability that lists the areas between the excluded ranges, so a VMM maps those and traps the rest.
mmap is page granular, so each range is rounded out to whole pages, and ranges that meet after rounding form one hole. The sorted list gives the areas in two walks: the first counts them and the second fills them.
A BAR without mmap-excluded ranges gets no new capability. The MSI-X table is excluded from read and write only, so the MSI-X BAR is unchanged.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 91 ++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+)drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -1106,6 +1106,88 @@ static void vfio_pci_free_excluded_ranges(struct
vfio_pci_core_device*vdev)drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_free_excluded_ranges(struct · line 1110
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c +2
-0} } +/* True when [start, start + len) on @bar overlaps a range with any +of @flags. */ static bool vfio_pci_bar_range_excluded(struct
vfio_pci_core_device*vdev,Quoted 41 lines
drivers/vfio/pci/vfio_pci_core.c +41
-0+ int bar, u64 start, u64 len, u32 +flags) { + struct vfio_pci_excluded_range *range; + +/* Only BARs have excluded ranges; a device-specific region has none. */ +if (bar >=PCI_STD_NUM_BARS ) +return false; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +if (range->start >= start + len) +break ; +if ((range->flags & flags) && + start < range->start + range->size) +return true; + } + +return false; +} + +/* +* Advertise a BAR with mmap-excluded ranges as the mmap-able areas +between +* them. mmap is page granular, so each range is rounded out to whole +pages; +* ranges that meet after rounding merge into one hole. The first +walk counts +* the areas and the second fills them. +*/ +static int vfio_pci_excluded_sparse_cap (struct vfio_pci_core_device *vdev, +int bar,struct vfio_info_cap +*caps) { + struct vfio_region_info_cap_sparse_mmap *sparse __free(kfree) = NULL; +u64 bar_len =pci_resource_len (vdev->pdev, bar); +struct vfio_pci_excluded_range *range; +u32 nr_areas =0 ; +bool holes = false; +u64 pos =0 ; +u32 i =0 ; + +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +if (!(range->flags &VFIO_PCI_EXCLUDE_MMAP ))drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev) · line 1158
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+continue ; + holes = true; +if (ALIGN_DOWN (range->start,PAGE_SIZE ) > pos) + nr_areas++; + pos =max (pos,ALIGN (range->start + range->size,PAGE_SIZE )); + } +if (!holes) +return 0 ; +if (pos < bar_len) + nr_areas++;The number of sparse ranges is bounded by (
num_mmap_excluded_ranges+ 1). We're dealing with a very low number of excluded ranges, bounded by insertion via kernel drivers and core. It seems worthwhile to avoid the duplication with a simple list walk that counts the mmap excluded ranges, then all of the alignment happens once, and the resultingnr_areasis calculated as we fill.
Quoted 6 lines
Note that with the use of
__free, it's generally recommended to allocate at the point it's declared and mixed declarations and statements are allowed for this. We can therefore do the declaration and allocation after we've done the simple version of the counting loop.
Thanks,
okay, I will update this. count the mmap-excluded ranges, allocate count + 1 areas at the declaration, then align and fill in one walk and take nr_areas from what was filled.
Thanks,
Manish
Quoted 62 lines
Alex
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_ioctl_get_region_info(struct
Quoted 36 lines
drivers/vfio/pci/vfio_pci_core.c +32
-0+ + sparse =kzalloc_flex (*sparse, areas, nr_areas); +if (!sparse) +return -ENOMEM ; + + sparse->header.id =VFIO_REGION_INFO_CAP_SPARSE_MMAP ; + sparse->header.version =1 ; + sparse->nr_areas = nr_areas; + + pos =0 ; +list_for_each_entry (range, &vdev->excluded_ranges[bar], entry) { +u64 hole =ALIGN_DOWN (range->start,PAGE_SIZE ); + +if (!(range->flags &VFIO_PCI_EXCLUDE_MMAP )) +continue ; +if (hole > pos) + sparse->areas[i++] = (struct vfio_region_sparse_mmap_area ) { + .offset = pos, + .size = hole - pos, + }; + pos =max (pos,ALIGN (range->start + range->size,PAGE_SIZE )); + } +if (pos < bar_len) + sparse->areas[i] = (struct vfio_region_sparse_mmap_area ) { + .offset = pos, + .size = bar_len - pos, + }; + +return vfio_info_add_capability (caps, &sparse->header, +struct_size (sparse, areas, +nr_areas)); } + int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops,@@ -1254,6 +1336,10 @@ int vfio_pci_ioctl_get_region_info(struct
vfio_device*core_vdev,drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_mmap(struct vfio_device
Quoted 11 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0if (ret)return ret; } + ret =vfio_pci_excluded_sparse_cap (vdev, info->index, + caps); +if (ret) +return ret; }break ;@@ -1947,6 +2033,11 @@ int vfio_pci_core_mmap(struct vfio_device
*core_vdev, structvm_area_struct*vmaQuoted 11 lines
drivers/vfio/pci/vfio_pci_core.c +5
-0if (req_start + req_len > phys_len)return -EINVAL ; +/* An excluded sub-range is reachable only through its trap, not mmap. */ +if (vfio_pci_bar_range_excluded (vdev, index, req_start, req_len, +VFIO_PCI_EXCLUDE_MMAP )) +return -EINVAL ; +/* * Ensure the BAR resource region is reserved for use. */
PATCH v6 10/31 vfio/pci: Refuse ioeventfds and dma-buf exports over excluded ranges
In reply to [email protected]From: Manish Honap <[email protected]>
vfio_pci_bar_rw() keeps the excluded ranges of a BAR off the device, but two other paths reach BAR MMIO without it.
An ioeventfd writes the BAR directly when it fires, through vfio_pci_core_iowriteN(). Refuse an ioeventfd whose range overlaps a write-excluded range, as one over the MSI-X table is already refused.
A dma-buf export of a BAR range lets an importer map it for DMA, which reaches a range that the BAR keeps off mmap. Refuse an export whose ranges overlap an mmap-excluded range, so the dma-buf follows the same rule as mmap.
--- drivers/vfio/pci/vfio_pci_core.c | 4 ++-- drivers/vfio/pci/vfio_pci_dmabuf.c | 14 ++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 3 +++ drivers/vfio/pci/vfio_pci_rdwr.c | 9 +++++++++ 4 files changed, 28 insertions(+), 2 deletions(-)
drivers/vfio/pci/vfio_pci_core.c +2 -2
@@ -1107,8 +1107,8 @@ static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev) }/* True when [start, start + len) on @bar overlaps a range with any of @flags. */ -static bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev, -int bar,u64 start,u64 len,u32 flags) +bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 len,u32 flags) {struct vfio_pci_excluded_range *range;
drivers/vfio/pci/vfio_pci_dmabuf.c +14 -0
@@ -260,6 +260,20 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,if (ret) goto err_free_ranges; +/* +* A DMA mapping built from the dma-buf would reach a range that the +* BAR keeps off mmap, so refuse such a range here as mmap does. +*/ +for (u32 i =0 ; i < get_dma_buf.nr_ranges; i++) { +if (vfio_pci_bar_range_excluded (vdev, get_dma_buf.region_index, + dma_ranges[i].offset, + dma_ranges[i].length, +VFIO_PCI_EXCLUDE_MMAP )) { + ret = -EINVAL ; + goto err_free_ranges; + } + } + priv =kzalloc_obj (*priv);if (!priv) { ret = -ENOMEM ;
drivers/vfio/pci/vfio_pci_priv.h +3 -0
@@ -52,6 +52,9 @@ struct vfio_pci_excluded_range {u32 flags; }; +bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 len,u32 flags); +#ifdef CONFIG_VFIO_PCI_VGA ssize_t vfio_pci_vga_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite);
drivers/vfio/pci/vfio_pci_rdwr.c +9 -0
@@ -476,6 +476,15 @@ int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset, pos >= vdev->msix_offset + vdev->msix_size))return -EINVAL ; +/* +* The ioeventfd fire path writes the BAR directly, without the +* excluded-range filter of vfio_pci_bar_rw(), so refuse a write-excluded +* range here. +*/ +if (vfio_pci_bar_range_excluded (vdev, bar, pos, count, +VFIO_PCI_EXCLUDE_WRITE )) +return -EINVAL ; +if (count ==8 )return -EINVAL ;
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 10/31] vfio/pci: Refuse ioeventfds and dma-buf exports over excluded ranges
Quoted 50 lines · On Fri, 16 Oct 2026 21:48:17 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio_pci_bar_rw()keeps the excluded ranges of a BAR off the device, but two other paths reach BAR MMIO without it.An ioeventfd writes the BAR directly when it fires, through
vfio_pci_core_iowriteN(). Refuse an ioeventfd whose range overlaps a write-excluded range, as one over the MSI-X table is already refused.A dma-buf export of a BAR range lets an importer map it for DMA, which reaches a range that the BAR keeps off mmap. Refuse an export whose ranges overlap an mmap-excluded range, so the dma-buf follows the same rule as mmap.
Suggested-by Alex Williamson <[email protected]>Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 4 ++-- drivers/vfio/pci/vfio_pci_dmabuf.c | 14 ++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 3 +++ drivers/vfio/pci/vfio_pci_rdwr.c | 9 +++++++++ 4 files changed, 28 insertions(+), 2 deletions(-)drivers/vfio/pci/vfio_pci_core.c +2
-2@@ -1107,8 +1107,8 @@ static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev) }/* True when [start, start + len) on @bar overlaps a range with any of @flags. */ -static bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev, -int bar,u64 start,u64 len,u32 flags) +bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 len,u32 flags) {struct vfio_pci_excluded_range *range;drivers/vfio/pci/vfio_pci_dmabuf.c +3
-0@@ -260,6 +260,20 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,if (ret) goto err_free_ranges; +/* +* A DMA mapping built from the dma-buf would reach a range that the +* BAR keeps off mmap, so refuse such a range here as mmap does.
drivers/vfio/pci/vfio_pci_dmabuf.c int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, · line 275
Quoted 10 lines
drivers/vfio/pci/vfio_pci_dmabuf.c +10
-0+*/ +for (u32 i =0 ; i < get_dma_buf.nr_ranges; i++) { +if (vfio_pci_bar_range_excluded (vdev, get_dma_buf.region_index, + dma_ranges[i].offset, + dma_ranges[i].length, +VFIO_PCI_EXCLUDE_MMAP )) { + ret = -EINVAL ; + goto err_free_ranges; + } + }
This should be rolled into validate_dmabuf_input().
Quoted 25 lines
drivers/vfio/pci/vfio_pci_dmabuf.c +1
-0+ priv =kzalloc_obj (*priv);if (!priv) { ret = -ENOMEM ;drivers/vfio/pci/vfio_pci_priv.h +3
-0@@ -52,6 +52,9 @@ struct vfio_pci_excluded_range {u32 flags; }; +bool vfio_pci_bar_range_excluded (struct vfio_pci_core_device *vdev,int bar, +u64 start,u64 len,u32 flags); +#ifdef CONFIG_VFIO_PCI_VGA ssize_t vfio_pci_vga_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite);drivers/vfio/pci/vfio_pci_rdwr.c +0
-0@@ -476,6 +476,15 @@ int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset, pos >= vdev->msix_offset + vdev->msix_size))return -EINVAL ;
drivers/vfio/pci/vfio_pci_rdwr.c int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset, · line 487
Quoted 10 lines
drivers/vfio/pci/vfio_pci_rdwr.c +9
-0+/* +* The ioeventfd fire path writes the BAR directly, without the +* excluded-range filter of vfio_pci_bar_rw(), so refuse a write-excluded +* range here. +*/ +if (vfio_pci_bar_range_excluded (vdev, bar, pos, count, +VFIO_PCI_EXCLUDE_WRITE )) +return -EINVAL ; +
The code immediately above this is still performing the open coded exclusion based on the MSI-X range, which is now incorporated into the generic range exclusion. It should be removed. Thanks,
Alex
RE: [PATCH v6 10/31] vfio/pci: Refuse ioeventfds and dma-buf exports over excluded ranges
Quoted 94 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Saturday, November 7, 2026 2:23 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 10/31] vfio/pci: Refuse ioeventfds and dma-buf exports over excluded rangesExternal email: Use caution opening links or attachments
Quoted 33 lines · On Fri, 16 Oct 2026 21:48:17 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio_pci_bar_rw()keeps the excluded ranges of a BAR off the device, but two other paths reach BAR MMIO without it.An ioeventfd writes the BAR directly when it fires, through
vfio_pci_core_iowriteN(). Refuse an ioeventfd whose range overlaps a write-excluded range, as one over the MSI-X table is already refused.A dma-buf export of a BAR range lets an importer map it for DMA, which reaches a range that the BAR keeps off mmap. Refuse an export whose ranges overlap an mmap-excluded range, so the dma-buf follows the same rule as mmap.
Suggested-by Alex Williamson <[email protected]>Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 4 ++-- drivers/vfio/pci/vfio_pci_dmabuf.c | 14 ++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 3 +++ drivers/vfio/pci/vfio_pci_rdwr.c | 9 +++++++++ 4 files changed, 28 insertions(+), 2 deletions(-)drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -1107,8 +1107,8 @@ static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev) }/* True when [start, start + len) on @bar overlaps a range with any Quoted 1 line
of @flags. */ -static bool vfio_pci_bar_range_excluded(struct
vfio_pci_core_device*vdev,drivers/vfio/pci/vfio_pci_core.c
Quoted 2 lines
drivers/vfio/pci/vfio_pci_core.c +1
-1- int bar, u64 start, u64 len, u32 flags) +bool vfio_pci_bar_range_excluded(struct vfio_pci_core_device *vdev, intbar,
drivers/vfio/pci/vfio_pci_dmabuf.c int vfio_pci_core_feature_dma_buf(struct
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0+ u64 start, u64 len, u32 flags) {struct vfio_pci_excluded_range *range;drivers/vfio/pci/vfio_pci_dmabuf.c +0
-0@@ -260,6 +260,20 @@ int vfio_pci_core_feature_dma_buf(struct
vfio_pci_core_device*vdev, u32 flags,Quoted 6 lines
drivers/vfio/pci/vfio_pci_dmabuf.c +3
-0if (ret) goto err_free_ranges; +/* +* A DMA mapping built from the dma-buf would reach a range that the +* BAR keeps off mmap, so refuse such a range here as mmap does. drivers/vfio/pci/vfio_pci_dmabuf.c int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, · line 275
Quoted 10 lines
drivers/vfio/pci/vfio_pci_dmabuf.c +10
-0+*/ +for (u32 i =0 ; i < get_dma_buf.nr_ranges; i++) { +if (vfio_pci_bar_range_excluded (vdev, get_dma_buf.region_index, + dma_ranges[i].offset, + dma_ranges[i].length, +VFIO_PCI_EXCLUDE_MMAP )) { + ret = -EINVAL ; + goto err_free_ranges; + } + }
Quoted 1 line
This should be rolled into
validate_dmabuf_input().
Okay, I will update this in the given function.
Quoted 41 lines
Quoted 11 lines
drivers/vfio/pci/vfio_pci_dmabuf.c +1
-0+ priv =kzalloc_obj (*priv);if (!priv) { ret = -ENOMEM ;drivers/vfio/pci/vfio_pci_priv.h +0
-0drivers/vfio/pci/vfio_pci_priv.h struct vfio_pci_excluded_range { · line 55
Quoted 4 lines
drivers/vfio/pci/vfio_pci_priv.h +1
-0@@ -52,6 +52,9 @@ struct vfio_pci_excluded_range {u32 flags; }; +bool vfio_pci_bar_range_excluded(struct vfio_pci_core_device *vdev, intbar,
drivers/vfio/pci/vfio_pci_rdwr.c int vfio_pci_ioeventfd(struct vfio_pci_core_device
Quoted 11 lines
drivers/vfio/pci/vfio_pci_priv.h +2
-1+ u64 start, u64 len, u32 flags); +#ifdef CONFIG_VFIO_PCI_VGA ssize_t vfio_pci_vga_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite); diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c
b/drivers/vfio/pci/vfio_pci_rdwr.c
index93823c6391f6..ef098f0b1401100644drivers/vfio/pci/vfio_pci_rdwr.c +0
-0@@ -476,6 +476,15 @@ int vfio_pci_ioeventfd(struct vfio_pci_core_device*vdev,
loff_toffset,drivers/vfio/pci/vfio_pci_rdwr.c int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset, · line 487
Quoted 12 lines
drivers/vfio/pci/vfio_pci_rdwr.c +9
-0pos >= vdev->msix_offset + vdev->msix_size))return -EINVAL ; +/* +* The ioeventfd fire path writes the BAR directly, without the +* excluded-range filter of vfio_pci_bar_rw(), so refuse a write-excluded +* range here. +*/ +if (vfio_pci_bar_range_excluded (vdev, bar, pos, count, +VFIO_PCI_EXCLUDE_WRITE )) +return -EINVAL ; +
Quoted 4 lines
The code immediately above this is still performing the open coded exclusion based on the MSI-X range, which is now incorporated into the generic range exclusion. It should be removed. Thanks,
Okay, I will remove this in v7.
Thanks,
Manish
Quoted 1 line
Alex
PATCH v6 11/31 vfio/pci: Virtualize the CXL DVSEC in vfio_pci_config.c
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL Type-2 device is reprogrammable through its CXL DVSEC: a guest could set Config Lock, toggle CXL.cache and CXL.mem enable, or rewrite the HDM range registers that govern host memory decode. Virtualize the DVSEC so the guest sees a shadow it cannot use to reprogram the hardware.
Build a per-device cxl_perm permission map, modeled on msi_perm. When the device is bound through the CXL provider (e.g. vfio-cxl), the whole CXL DVSEC is served from the vconfig shadow; only Control and Control2 are guest programmable, while Capability, Status, Lock and the Range registers keep their firmware snapshot. A vendor DVSEC on the same device is unaffected: the map is selected by the CXL DVSEC's range, since every DVSEC has the same ID in the config map and two DVSECs can be adjacent.
A Type-2 function bound without the provider (the provider is absent, its init failed, or the extensions are disabled) gets a guard map instead. The DVSEC reads from hardware, Control and Control2 stay writable and the RW1C status bits stay clearable, but writes to Lock and the Range registers are dropped, since they outlive the user. Initiate CXL Reset uses the same doorbell; a cache write-back goes to the device.
Control2 carries the CXL reset and cache write-back-invalidate initiate bits as self-clearing doorbells. With the provider, vfio never forwards them to hardware, so a custom writefn synthesizes their completion in the shadow: the initiate bit self-clears and the matching Status2 bit (Cache Invalid or Reset Done) is set, so a guest that writes back its cache (CXL r4.0 sec 9.6) and then runs CXL Reset (sec 9.7.2) progresses instead of timing out. Cache Invalid is reported only while Disable Caching is set, as the spec requires. The host performs the real cache write-back and reset at the vfio reset points. Until a later patch in this series runs the real reset from the doorbell, Initiate CXL Reset completes in the shadow only.
--- drivers/vfio/pci/vfio_pci_config.c | 206 ++++++++++++++++++++++++++++- include/linux/vfio_pci_core.h | 3 + 2 files changed, 205 insertions(+), 4 deletions(-)
drivers/vfio/pci/vfio_pci_config.c +202 -4
@@ -1747,6 +1747,194 @@ static const struct pci_device_id known_bogus_vf_intx_pin[] = {* for each area requiring emulated bits, but the array of pointers * would be comparable in size (at least for standard config space). */ +static int vfio_cxl_dvsec_write (struct vfio_pci_core_device *vdev,int pos, +int count,struct perm_bits *perm, +int offset,__le32 val) +{ +u16 start = vdev->cxl_dvsec; +__le16 *pctrl2, *pstatus2; +u16 ctrl2, status2; + + count =vfio_default_config_write (vdev, pos, count, perm, offset, val); +if (count <0 ) +return count; + +/* Only Control2 carries the self-clearing doorbells. */ +if (offset >PCI_DVSEC_CXL_CTRL2 || + offset + count <=PCI_DVSEC_CXL_CTRL2 ) +return count; + + pctrl2 = (__le16 *)&vdev->vconfig[start +PCI_DVSEC_CXL_CTRL2 ]; + pstatus2 = (__le16 *)&vdev->vconfig[start +PCI_DVSEC_CXL_STATUS2 ]; + ctrl2 =le16_to_cpu (*pctrl2); + status2 =le16_to_cpu (*pstatus2); + +/* +* INIT_CACHE_WBI and INIT_CXL_RST are self-clearing doorbells. With a +* provider neither reaches hardware: synthesize their completion in the +* shadow so the guest poll finishes, clearing the initiate bit and +* setting the matching Status2 completion. Cache Invalid is set only +* while caching is disabled and reads 0 otherwise (CXL r4.0 sec 8.1.3.5, +* Table 8-9). Under the guard map INIT_CACHE_WBI and Cache Invalid go to +* the device; only INIT_CXL_RST and its completion are virtualized. +*/ +if (ctrl2 &PCI_DVSEC_CXL_INIT_CACHE_WBI ) { + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CACHE_WBI ; +if (ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING ) + status2 |=PCI_DVSEC_CXL_CACHE_INV ; + } +if (!(ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING )) + status2 &= ~PCI_DVSEC_CXL_CACHE_INV ; +if (ctrl2 &PCI_DVSEC_CXL_INIT_CXL_RST ) { + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST ; + status2 &= ~PCI_DVSEC_CXL_RST_ERR ; + status2 |=PCI_DVSEC_CXL_RST_DONE ; + } + + *pctrl2 =cpu_to_le16 (ctrl2); + *pstatus2 =cpu_to_le16 (status2); + +return count; +} + +static int init_cxl_dvsec_perm (struct perm_bits *perm,int len) +{ +if (alloc_perm_bits (perm, len)) +return -ENOMEM ; + + perm->writefn = vfio_cxl_dvsec_write; + +/* Serve the whole CXL DVSEC from the shadow. */ +for (int i =0 ; i < len; i++) +p_setb (perm, i, (u8 )ALL_VIRT ,NO_WRITE ); + +/* +* Control and Control2 are guest programmable; Capability, Status, +* Lock and the Range registers keep their firmware snapshot, so the +* guest cannot set Config Lock or rewrite the capability and ranges. +*/ +p_setw (perm,PCI_DVSEC_CXL_CTRL , (u16 )ALL_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_CTRL2 , (u16 )ALL_VIRT , (u16 )ALL_WRITE ); + +return 0 ; +} + +/* +* Without a CXL provider the device is bound without the CXL extensions, so +* the DVSEC is read from hardware, Control and Control2 stay writable, and +* the RW1C status bits in Status and Status2 (for example Viral_Status) can +* still be cleared. Drop the writes whose effect outlives the user: Config +* Lock and the Range registers. Initiate CXL Reset goes through the same +* doorbell as with a provider, so the reset runs at a vfio reset point under +* memory_lock. +*/ +static int init_cxl_dvsec_guard_perm (struct perm_bits *perm,int len) +{ +if (alloc_perm_bits (perm, len)) +return -ENOMEM ; + + perm->writefn = vfio_cxl_dvsec_write; + +/* Extended capability header: ID and next pointer from vconfig. */ +p_setd (perm,0 ,ALL_VIRT ,NO_WRITE ); + +p_setw (perm,PCI_DVSEC_CXL_CTRL ,NO_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_CTRL2 ,PCI_DVSEC_CXL_INIT_CXL_RST , + (u16 )ALL_WRITE ); +/* Status (offset 0Eh, between Control and Control2): RW1C to the device. */ +p_setw (perm,PCI_DVSEC_CXL_CTRL +2 ,NO_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_STATUS2 , +PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR , + (u16 )~(PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR )); + +return 0 ; +} + +/* Virtualize the CXL DVSEC so a guest cannot reprogram the device through it. */ +static int vfio_cxl_dvsec_init (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; +u32 dword; +u16 dvsec, cap; +int len, ret; + + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); +if (!dvsec) +return 0 ; + + ret =pci_read_config_dword (pdev, dvsec +PCI_DVSEC_HEADER1 , &dword); +if (ret) +return pcibios_err_to_errno (ret); + len =PCI_DVSEC_HEADER1_LEN (dword); + +/* +* The virtualization writes fixed DVSEC offsets up to Status2 (the reset +* doorbell stamps it). A device that reports a shorter DVSEC is not a +* usable Type-2 function; leave its DVSEC unvirtualized rather than +* index the device-length-sized perm allocation past its end. +*/ +if (len <PCI_DVSEC_CXL_STATUS2 +2 ) +return 0 ; + +/* +* Without a provider, guard only a CXL Type-2 function, with the same +* test vfio-pci-core uses to load the provider. +*/ +if (!vdev->cxl_ops) { +if (!pcie_is_cxl (pdev)) +return 0 ; + ret =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CAP , &cap); +if (ret) +return pcibios_err_to_errno (ret); +if (!(cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || + !(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) +return 0 ; + } + + vdev->cxl_perm =kmalloc_obj (*vdev->cxl_perm,GFP_KERNEL_ACCOUNT ); +if (!vdev->cxl_perm) +return -ENOMEM ; + +if (vdev->cxl_ops) + ret =init_cxl_dvsec_perm (vdev->cxl_perm, len); +else + ret =init_cxl_dvsec_guard_perm (vdev->cxl_perm, len); +if (ret) { +kfree (vdev->cxl_perm); + vdev->cxl_perm =NULL ; +return ret; + } + + vdev->cxl_dvsec = dvsec; + vdev->cxl_dvsec_len = len; + +return 0 ; +} + +static void vfio_cxl_dvsec_exit (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_perm) +return ; + +free_perm_bits (vdev->cxl_perm); +kfree (vdev->cxl_perm); + vdev->cxl_perm =NULL ; + vdev->cxl_dvsec =0 ; + vdev->cxl_dvsec_len =0 ; +} + +/* +* Every DVSEC carries the same ID in pci_config_map, so vfio_find_cap_start() +* can walk from one DVSEC into an adjacent one. Match the CXL DVSEC by range. +*/ +static bool vfio_cxl_dvsec_contains (struct vfio_pci_core_device *vdev, +loff_t pos) +{ +return vdev->cxl_perm && pos >= vdev->cxl_dvsec && + pos < vdev->cxl_dvsec + vdev->cxl_dvsec_len; +} +int vfio_config_init (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev;
@@ -1842,6 +2030,10 @@ int vfio_config_init(struct vfio_pci_core_device *vdev)if (ret) goto out; + ret =vfio_cxl_dvsec_init (vdev); +if (ret) + goto out; +return 0 ; out:
@@ -1863,6 +2055,7 @@ void vfio_config_free(struct vfio_pci_core_device *vdev)kfree (vdev->msi_perm); vdev->msi_perm =NULL ; } +vfio_cxl_dvsec_exit (vdev); }/*
@@ -1926,12 +2119,17 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev,* of the extended capability list. Use default, ro * access, which will virtualize the id and next values. */ -if (cap_id >PCI_EXT_CAP_ID_MAX ) + cap_start =vfio_find_cap_start (vdev, *ppos); + +if (cap_id >PCI_EXT_CAP_ID_MAX ) { perm = &direct_ro_perms; -else + }else if (cap_id ==PCI_EXT_CAP_ID_DVSEC && +vfio_cxl_dvsec_contains (vdev, *ppos)) { + perm = vdev->cxl_perm; + cap_start = vdev->cxl_dvsec; + }else { perm = &ecap_perms[cap_id]; - - cap_start =vfio_find_cap_start (vdev, *ppos); + } }else {WARN_ON (cap_id >PCI_CAP_ID_MAX );
include/linux/vfio_pci_core.h +3 -0
@@ -120,6 +120,9 @@ struct vfio_pci_core_device {u8 *pci_config_map;u8 *vconfig;struct perm_bits *msi_perm; +struct perm_bits *cxl_perm; +u16 cxl_dvsec; +u16 cxl_dvsec_len;spinlock_t irqlock;struct mutex igate;struct xarray ctx;
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 11/31] vfio/pci: Virtualize the CXL DVSEC in vfio_pci_config.c
Quoted 16 lines · On Fri, 16 Oct 2026 21:48:18 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL Type-2 device is reprogrammable through its CXL DVSEC: a guest could set Config Lock, toggle CXL.cache and CXL.mem enable, or rewrite the HDM range registers that govern host memory decode. Virtualize the DVSEC so the guest sees a shadow it cannot use to reprogram the hardware.
Build a per-device
cxl_permpermission map, modeled onmsi_perm. When the device is bound through the CXL provider (e.g. vfio-cxl), the whole CXL DVSEC is served from the vconfig shadow; only Control and Control2 are guest programmable, while Capability, Status, Lock and the Range registers keep their firmware snapshot. A vendor DVSEC on the same device is unaffected: the map is selected by the CXL DVSEC's range, since every DVSEC has the same ID in the config map and two DVSECs can be adjacent.
Quoted 6 lines
A Type-2 function bound without the provider (the provider is absent, its init failed, or the extensions are disabled) gets a guard map instead. The DVSEC reads from hardware, Control and Control2 stay writable and the RW1C status bits stay clearable, but writes to Lock and the Range registers are dropped, since they outlive the user. Initiate CXL Reset uses the same doorbell; a cache write-back goes to the device.
The guard map actually changes the device ABI. This may be necessary, but if so, it should be pulled out into a Fixes: patch. The device aspects that change need to be well justified. Access to CXL reset without coordination is certainly not good. Config lock has side effects that we may not want to deal with. Is there value in exposing the CXL DVSEC at all when we're operating in vfio-pci mode? What do we expect a type-2 driver does with the guarded capability?
Maybe the right answer is to drop it from the chain and make it read-only.
Quoted 24 lines
Control2 carries the CXL reset and cache write-back-invalidate initiate bits as self-clearing doorbells. With the provider, vfio never forwards them to hardware, so a custom writefn synthesizes their completion in the shadow: the initiate bit self-clears and the matching Status2 bit (Cache Invalid or Reset Done) is set, so a guest that writes back its cache (CXL r4.0 sec 9.6) and then runs CXL Reset (sec 9.7.2) progresses instead of timing out. Cache Invalid is reported only while Disable Caching is set, as the spec requires. The host performs the real cache write-back and reset at the vfio reset points. Until a later patch in this series runs the real reset from the doorbell, Initiate CXL Reset completes in the shadow only.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_config.c | 206 ++++++++++++++++++++++++++++- include/linux/vfio_pci_core.h | 3 + 2 files changed, 205 insertions(+), 4 deletions(-)drivers/vfio/pci/vfio_pci_config.c +0
-0
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = { · line 1749
Quoted 3 lines
drivers/vfio/pci/vfio_pci_config.c +0
-0@@ -1747,6 +1747,194 @@ static const struct pci_device_id known_bogus_vf_intx_pin[] = {* for each area requiring emulated bits, but the array of pointers * would be comparable in size (at least for standard config space). */
This comment above used to be attached to vfio_config_init() and now it's miles away!
Quoted 124 lines
drivers/vfio/pci/vfio_pci_config.c +124
-0+static int vfio_cxl_dvsec_write (struct vfio_pci_core_device *vdev,int pos, +int count,struct perm_bits *perm, +int offset,__le32 val) +{ +u16 start = vdev->cxl_dvsec; +__le16 *pctrl2, *pstatus2; +u16 ctrl2, status2; + + count =vfio_default_config_write (vdev, pos, count, perm, offset, val); +if (count <0 ) +return count; + +/* Only Control2 carries the self-clearing doorbells. */ +if (offset >PCI_DVSEC_CXL_CTRL2 || + offset + count <=PCI_DVSEC_CXL_CTRL2 ) +return count; + + pctrl2 = (__le16 *)&vdev->vconfig[start +PCI_DVSEC_CXL_CTRL2 ]; + pstatus2 = (__le16 *)&vdev->vconfig[start +PCI_DVSEC_CXL_STATUS2 ]; + ctrl2 =le16_to_cpu (*pctrl2); + status2 =le16_to_cpu (*pstatus2); + +/* +* INIT_CACHE_WBI and INIT_CXL_RST are self-clearing doorbells. With a +* provider neither reaches hardware: synthesize their completion in the +* shadow so the guest poll finishes, clearing the initiate bit and +* setting the matching Status2 completion. Cache Invalid is set only +* while caching is disabled and reads 0 otherwise (CXL r4.0 sec 8.1.3.5, +* Table 8-9). Under the guard map INIT_CACHE_WBI and Cache Invalid go to +* the device; only INIT_CXL_RST and its completion are virtualized. +*/ +if (ctrl2 &PCI_DVSEC_CXL_INIT_CACHE_WBI ) { + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CACHE_WBI ; +if (ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING ) + status2 |=PCI_DVSEC_CXL_CACHE_INV ; + } +if (!(ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING )) + status2 &= ~PCI_DVSEC_CXL_CACHE_INV ; +if (ctrl2 &PCI_DVSEC_CXL_INIT_CXL_RST ) { + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST ; + status2 &= ~PCI_DVSEC_CXL_RST_ERR ; + status2 |=PCI_DVSEC_CXL_RST_DONE ; + } + + *pctrl2 =cpu_to_le16 (ctrl2); + *pstatus2 =cpu_to_le16 (status2); + +return count; +} + +static int init_cxl_dvsec_perm (struct perm_bits *perm,int len) +{ +if (alloc_perm_bits (perm, len)) +return -ENOMEM ; + + perm->writefn = vfio_cxl_dvsec_write; + +/* Serve the whole CXL DVSEC from the shadow. */ +for (int i =0 ; i < len; i++) +p_setb (perm, i, (u8 )ALL_VIRT ,NO_WRITE ); + +/* +* Control and Control2 are guest programmable; Capability, Status, +* Lock and the Range registers keep their firmware snapshot, so the +* guest cannot set Config Lock or rewrite the capability and ranges. +*/ +p_setw (perm,PCI_DVSEC_CXL_CTRL , (u16 )ALL_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_CTRL2 , (u16 )ALL_VIRT , (u16 )ALL_WRITE ); + +return 0 ; +} + +/* +* Without a CXL provider the device is bound without the CXL extensions, so +* the DVSEC is read from hardware, Control and Control2 stay writable, and +* the RW1C status bits in Status and Status2 (for example Viral_Status) can +* still be cleared. Drop the writes whose effect outlives the user: Config +* Lock and the Range registers. Initiate CXL Reset goes through the same +* doorbell as with a provider, so the reset runs at a vfio reset point under +* memory_lock. +*/ +static int init_cxl_dvsec_guard_perm (struct perm_bits *perm,int len) +{ +if (alloc_perm_bits (perm, len)) +return -ENOMEM ; + + perm->writefn = vfio_cxl_dvsec_write; + +/* Extended capability header: ID and next pointer from vconfig. */ +p_setd (perm,0 ,ALL_VIRT ,NO_WRITE ); + +p_setw (perm,PCI_DVSEC_CXL_CTRL ,NO_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_CTRL2 ,PCI_DVSEC_CXL_INIT_CXL_RST , + (u16 )ALL_WRITE ); +/* Status (offset 0Eh, between Control and Control2): RW1C to the device. */ +p_setw (perm,PCI_DVSEC_CXL_CTRL +2 ,NO_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_STATUS2 , +PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR , + (u16 )~(PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR )); + +return 0 ; +} + +/* Virtualize the CXL DVSEC so a guest cannot reprogram the device through it. */ +static int vfio_cxl_dvsec_init (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; +u32 dword; +u16 dvsec, cap; +int len, ret; + + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); +if (!dvsec) +return 0 ; + + ret =pci_read_config_dword (pdev, dvsec +PCI_DVSEC_HEADER1 , &dword); +if (ret) +return pcibios_err_to_errno (ret); + len =PCI_DVSEC_HEADER1_LEN (dword); + +/* +* The virtualization writes fixed DVSEC offsets up to Status2 (the reset +* doorbell stamps it). A device that reports a shorter DVSEC is not a
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = { · line 1883
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +10
-0+* usable Type-2 function; leave its DVSEC unvirtualized rather than +* index the device-length-sized perm allocation past its end. +*/ +if (len <PCI_DVSEC_CXL_STATUS2 +2 ) +return 0 ; + +/* +* Without a provider, guard only a CXL Type-2 function, with the same +* test vfio-pci-core uses to load the provider. +*/
Cryptic comment. Why don't we expose vfio_pci_is_cxl_type2() and use it here?
Quoted 37 lines
drivers/vfio/pci/vfio_pci_config.c +37
-0+if (!vdev->cxl_ops) { +if (!pcie_is_cxl (pdev)) +return 0 ; + ret =pci_read_config_word (pdev, dvsec +PCI_DVSEC_CXL_CAP , &cap); +if (ret) +return pcibios_err_to_errno (ret); +if (!(cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || + !(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) +return 0 ; + } + + vdev->cxl_perm =kmalloc_obj (*vdev->cxl_perm,GFP_KERNEL_ACCOUNT ); +if (!vdev->cxl_perm) +return -ENOMEM ; + +if (vdev->cxl_ops) + ret =init_cxl_dvsec_perm (vdev->cxl_perm, len); +else + ret =init_cxl_dvsec_guard_perm (vdev->cxl_perm, len); +if (ret) { +kfree (vdev->cxl_perm); + vdev->cxl_perm =NULL ; +return ret; + } + + vdev->cxl_dvsec = dvsec; + vdev->cxl_dvsec_len = len; + +return 0 ; +} + +static void vfio_cxl_dvsec_exit (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_perm) +return ; + +free_perm_bits (vdev->cxl_perm);
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = { · line 1930
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +10
-0+kfree (vdev->cxl_perm); + vdev->cxl_perm =NULL ; + vdev->cxl_dvsec =0 ; + vdev->cxl_dvsec_len =0 ; +} + +/* +* Every DVSEC carries the same ID in pci_config_map, so vfio_find_cap_start() +* can walk from one DVSEC into an adjacent one. Match the CXL DVSEC by range. +*/
IMO, a comment is more relevant below.
Quoted 22 lines
drivers/vfio/pci/vfio_pci_config.c +11
-0+static bool vfio_cxl_dvsec_contains (struct vfio_pci_core_device *vdev, +loff_t pos) +{ +return vdev->cxl_perm && pos >= vdev->cxl_dvsec && + pos < vdev->cxl_dvsec + vdev->cxl_dvsec_len; +} +int vfio_config_init (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev;@@ -1842,6 +2030,10 @@ int vfio_config_init(struct vfio_pci_core_device *vdev)if (ret) goto out; + ret =vfio_cxl_dvsec_init (vdev); +if (ret) + goto out; +return 0 ; out:
drivers/vfio/pci/vfio_pci_config.c void vfio_config_free(struct vfio_pci_core_device *vdev) · line 2058
Quoted 4 lines
drivers/vfio/pci/vfio_pci_config.c +1
-0@@ -1863,6 +2055,7 @@ void vfio_config_free(struct vfio_pci_core_device *vdev)kfree (vdev->msi_perm); vdev->msi_perm =NULL ; } +vfio_cxl_dvsec_exit (vdev);
Teardown in opposite order of init. Last initialized ~ first exit.
Quoted 6 lines
drivers/vfio/pci/vfio_pci_config.c +0
-0}/* @@ -1926,12 +2119,17 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev,* of the extended capability list. Use default, ro * access, which will virtualize the id and next values.
drivers/vfio/pci/vfio_pci_config.c ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev, · line 2128
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +6
-2*/ -if (cap_id >PCI_EXT_CAP_ID_MAX ) + cap_start =vfio_find_cap_start (vdev, *ppos); + +if (cap_id >PCI_EXT_CAP_ID_MAX ) { perm = &direct_ro_perms; -else + }else if (cap_id ==PCI_EXT_CAP_ID_DVSEC && +vfio_cxl_dvsec_contains (vdev, *ppos)) { + perm = vdev->cxl_perm;
/* * Multiple DVSECs may be back-to-back, resulting * in bogus cap_start. */
Thanks,
Alex
Quoted 23 lines
drivers/vfio/pci/vfio_pci_config.c +3
-2+ cap_start = vdev->cxl_dvsec; + }else { perm = &ecap_perms[cap_id]; - - cap_start =vfio_find_cap_start (vdev, *ppos); + } }else {WARN_ON (cap_id >PCI_CAP_ID_MAX );include/linux/vfio_pci_core.h +3
-0@@ -120,6 +120,9 @@ struct vfio_pci_core_device {u8 *pci_config_map;u8 *vconfig;struct perm_bits *msi_perm; +struct perm_bits *cxl_perm; +u16 cxl_dvsec; +u16 cxl_dvsec_len;spinlock_t irqlock;struct mutex igate;struct xarray ctx;
RE: [PATCH v6 11/31] vfio/pci: Virtualize the CXL DVSEC in vfio_pci_config.c
Quoted 52 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Saturday, November 7, 2026 3:41 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 11/31] vfio/pci: Virtualize the CXL DVSEC invfio_pci_config.cExternal email: Use caution opening links or attachments
Quoted 16 lines · On Fri, 16 Oct 2026 21:48:18 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL Type-2 device is reprogrammable through its CXL DVSEC: a guest could set Config Lock, toggle CXL.cache and CXL.mem enable, or rewrite the HDM range registers that govern host memory decode. Virtualize the DVSEC so the guest sees a shadow it cannot use to reprogram the hardware.
Build a per-device
cxl_permpermission map, modeled onmsi_perm. When the device is bound through the CXL provider (e.g. vfio-cxl), the whole CXL DVSEC is served from the vconfig shadow; only Control and Control2 are guest programmable, while Capability, Status, Lock and the Range registers keep their firmware snapshot. A vendor DVSEC on the same device is unaffected: the map is selected by the CXL DVSEC's range, since every DVSEC has the same ID in the config map and two DVSECs can be adjacent.Quoted 6 lines
A Type-2 function bound without the provider (the provider is absent, its init failed, or the extensions are disabled) gets a guard map instead. The DVSEC reads from hardware, Control and Control2 stay writable and the RW1C status bits stay clearable, but writes to Lock and the Range registers are dropped, since they outlive the user. Initiate CXL Reset uses the same doorbell; a cache write-back goes to the
device.
Quoted 9 lines
The guard map actually changes the device ABI. This may be necessary, but if so, it should be pulled out into a Fixes: patch. The device aspects that change need to be well justified. Access to CXL reset without coordination is certainly not good. Config lock has side effects that we may not want to deal with. Is there value in exposing the CXL DVSEC at all when we're operating in vfio-pci mode? What do we expect a type-2 driver does with the guarded capability?
Maybe the right answer is to drop it from the chain and make it read-only.
okay, for v7 I will drop the guard map and make the CXL DVSEC of a Type-2 function read-only when vfio-cxl is not bound, as a separate patch at the front of the series with appropriate fixes: tag.
Quoted 26 lines
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id
Quoted 25 lines
Control2 carries the CXL reset and cache write-back-invalidate initiate bits as self-clearing doorbells. With the provider, vfio never forwards them to hardware, so a custom writefn synthesizes their completion in the shadow: the initiate bit self-clears and the matching Status2 bit (Cache Invalid or Reset Done) is set, so a guest that writes back its cache (CXL r4.0 sec 9.6) and then runs CXL Reset (sec 9.7.2) progresses instead of timing out. Cache Invalid is reported only while Disable Caching is set, as the spec requires. The host performs the real cache write-back and reset at the vfio reset points. Until a later patch in this series runs the real reset from the doorbell, Initiate CXL Reset completes in the shadow only.
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_config.c | 206 ++++++++++++++++++++++++++++- include/linux/vfio_pci_core.h | 3 + 2 files changed, 205 insertions(+), 4 deletions(-)drivers/vfio/pci/vfio_pci_config.c +0
-0@@ -1747,6 +1747,194 @@ static const struct pci_device_idknown_bogus_vf_intx_pin[] = {
Quoted 7 lines
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = { · line 1749
Quoted 3 lines
drivers/vfio/pci/vfio_pci_config.c +0
-0* for each area requiring emulated bits, but the array of pointers * would be comparable in size (at least for standard config space). */ This comment above used to be attached to
vfio_config_init()and now it's miles away!
I will update this.
Quoted 137 lines
Quoted 18 lines
drivers/vfio/pci/vfio_pci_config.c +18
-0+static int vfio_cxl_dvsec_write (struct vfio_pci_core_device *vdev,int pos, +int count,struct perm_bits *perm, +int offset,__le32 val) { +u16 start = vdev->cxl_dvsec; +__le16 *pctrl2, *pstatus2; +u16 ctrl2, status2; + + count =vfio_default_config_write (vdev, pos, count, perm, offset, val); +if (count <0 ) +return count; + +/* Only Control2 carries the self-clearing doorbells. */ +if (offset >PCI_DVSEC_CXL_CTRL2 || + offset + count <=PCI_DVSEC_CXL_CTRL2 ) +return count; + + pctrl2 = (__le16 *)&vdev->vconfig[start +PCI_DVSEC_CXL_CTRL2 ]; + pstatus2 = (__le16 *)&vdev->vconfig[start +PCI_DVSEC_CXL_STATUS2 ];drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = {
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +10
-0+ ctrl2 =le16_to_cpu (*pctrl2); + status2 =le16_to_cpu (*pstatus2); + +/* +* INIT_CACHE_WBI and INIT_CXL_RST are self-clearing doorbells. With a +* provider neither reaches hardware: synthesize their completion in the +* shadow so the guest poll finishes, clearing the initiate bit and +* setting the matching Status2 completion. Cache Invalid is set only +* while caching is disabled and reads 0 otherwise (CXL r4.0 sec 8.1.3.5, +* Table 8-9). Under the guard map INIT_CACHE_WBI and Cache Invalid go to
Quoted 62 lines
drivers/vfio/pci/vfio_pci_config.c +62
-0+* the device; only INIT_CXL_RST and its completion are virtualized. +*/ +if (ctrl2 &PCI_DVSEC_CXL_INIT_CACHE_WBI ) { + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CACHE_WBI ; +if (ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING ) + status2 |=PCI_DVSEC_CXL_CACHE_INV ; + } +if (!(ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING )) + status2 &= ~PCI_DVSEC_CXL_CACHE_INV ; +if (ctrl2 &PCI_DVSEC_CXL_INIT_CXL_RST ) { + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST ; + status2 &= ~PCI_DVSEC_CXL_RST_ERR ; + status2 |=PCI_DVSEC_CXL_RST_DONE ; + } + + *pctrl2 =cpu_to_le16 (ctrl2); + *pstatus2 =cpu_to_le16 (status2); + +return count; +} + +static int init_cxl_dvsec_perm (struct perm_bits *perm,int len) { +if (alloc_perm_bits (perm, len)) +return -ENOMEM ; + + perm->writefn = vfio_cxl_dvsec_write; + +/* Serve the whole CXL DVSEC from the shadow. */ +for (int i =0 ; i < len; i++) +p_setb (perm, i, (u8 )ALL_VIRT ,NO_WRITE ); + +/* +* Control and Control2 are guest programmable; Capability, Status, +* Lock and the Range registers keep their firmware snapshot, so the +* guest cannot set Config Lock or rewrite the capability and ranges. +*/ +p_setw (perm,PCI_DVSEC_CXL_CTRL , (u16 )ALL_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_CTRL2 , (u16 )ALL_VIRT , + (u16 )ALL_WRITE ); + +return 0 ; +} + +/* +* Without a CXL provider the device is bound without the CXL +extensions, so +* the DVSEC is read from hardware, Control and Control2 stay +writable, and +* the RW1C status bits in Status and Status2 (for example +Viral_Status) can +* still be cleared. Drop the writes whose effect outlives the user: +Config +* Lock and the Range registers. Initiate CXL Reset goes through the +same +* doorbell as with a provider, so the reset runs at a vfio reset +point under +* memory_lock. +*/ +static int init_cxl_dvsec_guard_perm (struct perm_bits *perm,int len) +{ +if (alloc_perm_bits (perm, len)) +return -ENOMEM ;drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = {
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +10
-0+ + perm->writefn = vfio_cxl_dvsec_write; + +/* Extended capability header: ID and next pointer from vconfig. */ +p_setd (perm,0 ,ALL_VIRT ,NO_WRITE ); + +p_setw (perm,PCI_DVSEC_CXL_CTRL ,NO_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_CTRL2 ,PCI_DVSEC_CXL_INIT_CXL_RST , + (u16 )ALL_WRITE ); +/* Status (offset 0Eh, between Control and Control2): RW1C to the device. */
Quoted 12 lines
drivers/vfio/pci/vfio_pci_config.c +12
-0+p_setw (perm,PCI_DVSEC_CXL_CTRL +2 ,NO_VIRT , (u16 )ALL_WRITE ); +p_setw (perm,PCI_DVSEC_CXL_STATUS2 , +PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR , + (u16 )~(PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR )); + +return 0 ; +} + +/* Virtualize the CXL DVSEC so a guest cannot reprogram the device +through it. */ static int vfio_cxl_dvsec_init (struct +vfio_pci_core_device *vdev) { +struct pci_dev *pdev = vdev->pdev;drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = {
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +10
-0+u32 dword; +u16 dvsec, cap; +int len, ret; + + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); +if (!dvsec) +return 0 ; + + ret = pci_read_config_dword(pdev, dvsec + PCI_DVSEC_HEADER1,&dword);
Quoted 12 lines
drivers/vfio/pci/vfio_pci_config.c +12
-0+if (ret) +return pcibios_err_to_errno (ret); + len =PCI_DVSEC_HEADER1_LEN (dword); + +/* +* The virtualization writes fixed DVSEC offsets up to Status2 (the reset +* doorbell stamps it). A device that reports a shorter DVSEC is not a +* usable Type-2 function; leave its DVSEC unvirtualized rather than +* index the device-length-sized perm allocation past its end. +*/ +if (len <PCI_DVSEC_CXL_STATUS2 +2 ) +return 0 ;
Quoted 8 lines
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = { · line 1883
Quoted 5 lines
drivers/vfio/pci/vfio_pci_config.c +5
-0+ +/* +* Without a provider, guard only a CXL Type-2 function, with the same +* test vfio-pci-core uses to load the provider. +*/ Cryptic comment. Why don't we expose
vfio_pci_is_cxl_type2()and use it here?
Okay, I will refactor this part.
Quoted 50 lines
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = {
Quoted 5 lines
drivers/vfio/pci/vfio_pci_config.c +4
-0+if (!vdev->cxl_ops) { +if (!pcie_is_cxl (pdev)) +return 0 ; + ret = pci_read_config_word(pdev, dvsec + PCI_DVSEC_CXL_CAP,&cap);
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = {
Quoted 8 lines
drivers/vfio/pci/vfio_pci_config.c +8
-0+if (ret) +return pcibios_err_to_errno (ret); +if (!(cap &PCI_DVSEC_CXL_CACHE_CAPABLE ) || + !(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) +return 0 ; + } + + vdev->cxl_perm = kmalloc_obj(*vdev->cxl_perm,
GFP_KERNEL_ACCOUNT);Quoted 24 lines
drivers/vfio/pci/vfio_pci_config.c +24
-0+if (!vdev->cxl_perm) +return -ENOMEM ; + +if (vdev->cxl_ops) + ret =init_cxl_dvsec_perm (vdev->cxl_perm, len); +else + ret =init_cxl_dvsec_guard_perm (vdev->cxl_perm, len); +if (ret) { +kfree (vdev->cxl_perm); + vdev->cxl_perm =NULL ; +return ret; + } + + vdev->cxl_dvsec = dvsec; + vdev->cxl_dvsec_len = len; + +return 0 ; +} + +static void vfio_cxl_dvsec_exit (struct vfio_pci_core_device *vdev) { +if (!vdev->cxl_perm) +return ; + +free_perm_bits (vdev->cxl_perm);drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id known_bogus_vf_intx_pin[] = {
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +10
-0+kfree (vdev->cxl_perm); + vdev->cxl_perm =NULL ; + vdev->cxl_dvsec =0 ; + vdev->cxl_dvsec_len =0 ; +} + +/* +* Every DVSEC carries the same ID in pci_config_map, so +vfio_find_cap_start() +* can walk from one DVSEC into an adjacent one. Match the CXL DVSEC by range.
Quoted 3 lines
drivers/vfio/pci/vfio_pci_config.c static const struct pci_device_id · line 1935
Quoted 1 line
drivers/vfio/pci/vfio_pci_config.c +1
-0+*/ IMO, a comment is more relevant below.
Okay, I will update this.
Quoted 21 lines
drivers/vfio/pci/vfio_pci_config.c void vfio_config_free(struct vfio_pci_core_device
Quoted 20 lines
drivers/vfio/pci/vfio_pci_config.c +5
-0+static bool vfio_cxl_dvsec_contains (struct vfio_pci_core_device *vdev, +loff_t pos) { +return vdev->cxl_perm && pos >= vdev->cxl_dvsec && + pos < vdev->cxl_dvsec + vdev->cxl_dvsec_len; } +int vfio_config_init (struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; @@ -1842,6 +2030,10 @@ int
vfio_config_init(struct vfio_pci_core_device *vdev)if (ret) goto out;drivers/vfio/pci/vfio_pci_config.c +4
-0+ ret =vfio_cxl_dvsec_init (vdev); +if (ret) + goto out; +return 0 ; out:@@ -1863,6 +2055,7 @@ void vfio_config_free(struct vfio_pci_core_device*vdev)
Quoted 6 lines
drivers/vfio/pci/vfio_pci_config.c void vfio_config_free(struct vfio_pci_core_device *vdev) · line 2058
Quoted 4 lines
drivers/vfio/pci/vfio_pci_config.c +1
-0kfree (vdev->msi_perm); vdev->msi_perm =NULL ; } +vfio_cxl_dvsec_exit (vdev);Teardown in opposite order of init. Last initialized ~ first exit.
Yes, I will fix this and use the comment at the cap_start call.
Thanks,
Manish
Quoted 51 lines
drivers/vfio/pci/vfio_pci_config.c ssize_t vfio_pci_config_rw_single(struct
Quoted 5 lines
drivers/vfio/pci/vfio_pci_config.c +0
-0}/* @@ -1926,12 +2119,17 @@ ssize_t vfio_pci_config_rw_single(struct
vfio_pci_core_device*vdev,drivers/vfio/pci/vfio_pci_config.c ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev, · line 2128
Quoted 12 lines
drivers/vfio/pci/vfio_pci_config.c +6
-2* of the extended capability list. Use default, ro * access, which will virtualize the id and next values. */ -if (cap_id >PCI_EXT_CAP_ID_MAX ) + cap_start =vfio_find_cap_start (vdev, *ppos); + +if (cap_id >PCI_EXT_CAP_ID_MAX ) { perm = &direct_ro_perms; -else + }else if (cap_id ==PCI_EXT_CAP_ID_DVSEC && +vfio_cxl_dvsec_contains (vdev, *ppos)) { + perm = vdev->cxl_perm;/*
* Multiple DVSECs may be back-to-back, resulting
* in boguscap_start.
*/Thanks,
AlexQuoted 24 lines
drivers/vfio/pci/vfio_pci_config.c +3
-2+ cap_start = vdev->cxl_dvsec; + }else { perm = &ecap_perms[cap_id]; - - cap_start =vfio_find_cap_start (vdev, *ppos); + } }else {WARN_ON (cap_id >PCI_CAP_ID_MAX );(quoted diff) +0
-0
b/include/linux/vfio_pci_core.hindex70acb03d466e..fd1ad0baf470100644include/linux/vfio_pci_core.h +3
-0@@ -120,6 +120,9 @@ struct vfio_pci_core_device {u8 *pci_config_map;u8 *vconfig;struct perm_bits *msi_perm; +struct perm_bits *cxl_perm; +u16 cxl_dvsec; +u16 cxl_dvsec_len;spinlock_t irqlock;struct mutex igate;struct xarray ctx;
PATCH v6 12/31 vfio/pci: Call the CXL open and close hooks around device use
In reply to [email protected]From: Manish Honap <[email protected]>
vfio-pci-core frees device regions and excluded ranges on close and on a failed open, so a CXL provider must set them up at each open.
Add open and close to the CXL ops and call them from the common enable and close paths. A failed open unwinds the enable in reverse order:
release the device regions, unmap the BARs, drop the excluded ranges, then free the virtual config.
Factor the device region release and the BAR unmap out of vfio_pci_core_disable() into helpers so the enable failure path and the disable path share them.
--- drivers/vfio/pci/vfio_pci_core.c | 57 +++++++++++++++++++++++--------- include/linux/vfio_pci_core.h | 2 ++ 2 files changed, 44 insertions(+), 15 deletions(-)
drivers/vfio/pci/vfio_pci_core.c +42 -15
@@ -590,6 +590,31 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = {static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device *vdev); +static void vfio_pci_core_unmap_bars (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; + +for (int i =0 ; i <PCI_STD_NUM_BARS ; i++) { +int bar = i +PCI_STD_RESOURCES ; + +if (IS_ERR_OR_NULL (vdev->barmap[bar])) +continue ; +pci_iounmap (pdev, vdev->barmap[bar]); +pci_release_selected_regions (pdev,1 << bar); + vdev->barmap[bar] =NULL ; + } +} + +static void vfio_pci_core_free_dev_regions (struct vfio_pci_core_device *vdev) +{ +for (int i =0 ; i < vdev->num_regions; i++) + vdev->region[i].ops->release (vdev, &vdev->region[i]); + + vdev->num_regions =0 ; +kfree (vdev->region); + vdev->region =NULL ;/* don't krealloc a freed pointer */ +} +int vfio_pci_core_enable (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev;
@@ -691,8 +716,20 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)vfio_pci_core_map_bars (vdev); +if (vdev->cxl_ops) { + ret = vdev->cxl_ops->open_device (vdev); +if (ret) + goto out_free_config; + } +return 0 ; +out_free_config: +/* A failed first open never reaches vfio_pci_core_disable(). */ +vfio_pci_core_free_dev_regions (vdev); +vfio_pci_core_unmap_bars (vdev); +vfio_pci_free_excluded_ranges (vdev); +vfio_config_free (vdev); out_free_zdev:vfio_pci_zdev_close_device (vdev); out_free_state:
@@ -713,7 +750,6 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)struct pci_dev *pdev = vdev->pdev;struct vfio_pci_dummy_resource *dummy_res, *tmp;struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp; -int i, bar;/* For needs_reset */ lockdep_assert_held (&vdev->vdev.dev_set->lock);
@@ -759,24 +795,12 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) vdev->virq_disabled = false; -for (i =0 ; i < vdev->num_regions; i++) - vdev->region[i].ops->release (vdev, &vdev->region[i]); - - vdev->num_regions =0 ; -kfree (vdev->region); - vdev->region =NULL ;/* don't krealloc a freed pointer */ +vfio_pci_core_free_dev_regions (vdev);vfio_config_free (vdev);vfio_pci_free_excluded_ranges (vdev); -for (i =0 ; i <PCI_STD_NUM_BARS ; i++) { - bar = i +PCI_STD_RESOURCES ; -if (IS_ERR_OR_NULL (vdev->barmap[bar])) -continue ; -pci_iounmap (pdev, vdev->barmap[bar]); -pci_release_selected_regions (pdev,1 << bar); - vdev->barmap[bar] =NULL ; - } +vfio_pci_core_unmap_bars (vdev);list_for_each_entry_safe (dummy_res, tmp, &vdev->dummy_resources_list, res_next) {
@@ -859,6 +883,9 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev)#endif vfio_pci_dma_buf_cleanup (vdev); +if (vdev->cxl_ops) + vdev->cxl_ops->close_device (vdev); +vfio_pci_core_disable (vdev);mutex_lock (&vdev->igate);
include/linux/vfio_pci_core.h +2 -0
@@ -70,6 +70,8 @@ struct vfio_pci_device_ops {struct vfio_cxl_ops {int (*init)(struct vfio_pci_core_device *vdev);void (*release)(struct vfio_pci_core_device *vdev); +int (*open_device)(struct vfio_pci_core_device *vdev); +void (*close_device)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 12/31] vfio/pci: Call the CXL open and close hooks around device use
Quoted 43 lines · On Fri, 16 Oct 2026 21:48:19 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio-pci-core frees device regions and excluded ranges on close and on a failed open, so a CXL provider must set them up at each open.
Add open and close to the CXL ops and call them from the common enable and close paths. A failed open unwinds the enable in reverse order:
release the device regions, unmap the BARs, drop the excluded ranges, then free the virtual config.Factor the device region release and the BAR unmap out of
vfio_pci_core_disable()into helpers so the enable failure path and the disable path share them.Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 57 +++++++++++++++++++++++--------- include/linux/vfio_pci_core.h | 2 ++ 2 files changed, 44 insertions(+), 15 deletions(-)drivers/vfio/pci/vfio_pci_core.c +14
-0@@ -590,6 +590,31 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = {static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device *vdev); +static void vfio_pci_core_unmap_bars (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; + +for (int i =0 ; i <PCI_STD_NUM_BARS ; i++) { +int bar = i +PCI_STD_RESOURCES ; + +if (IS_ERR_OR_NULL (vdev->barmap[bar])) +continue ; +pci_iounmap (pdev, vdev->barmap[bar]); +pci_release_selected_regions (pdev,1 << bar); + vdev->barmap[bar] =NULL ; + } +}
drivers/vfio/pci/vfio_pci_core.c static const struct dev_pm_ops vfio_pci_core_pm_ops = { · line 616
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+ +static void vfio_pci_core_free_dev_regions (struct vfio_pci_core_device *vdev) +{ +for (int i =0 ; i < vdev->num_regions; i++) + vdev->region[i].ops->release (vdev, &vdev->region[i]); + + vdev->num_regions =0 ; +kfree (vdev->region); + vdev->region =NULL ;/* don't krealloc a freed pointer */ +}
Mechanical changes like this should be done in a separate patch.
Quoted 12 lines
drivers/vfio/pci/vfio_pci_core.c +5
-0+int vfio_pci_core_enable (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev;@@ -691,8 +716,20 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)vfio_pci_core_map_bars (vdev); +if (vdev->cxl_ops) { + ret = vdev->cxl_ops->open_device (vdev); +if (ret) + goto out_free_config;
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 732
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +8
-0+ } +return 0 ; +out_free_config: +/* A failed first open never reaches vfio_pci_core_disable(). */ +vfio_pci_core_free_dev_regions (vdev); +vfio_pci_core_unmap_bars (vdev); +vfio_pci_free_excluded_ranges (vdev); +vfio_config_free (vdev);
Raising again that this doesn't account for the code previously added to this function that has its own unwind.
There also remains the question of whether excluded range setup should be moved to init/release.
Quoted 39 lines
drivers/vfio/pci/vfio_pci_core.c +2
-15out_free_zdev:vfio_pci_zdev_close_device (vdev); out_free_state:@@ -713,7 +750,6 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)struct pci_dev *pdev = vdev->pdev;struct vfio_pci_dummy_resource *dummy_res, *tmp;struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp; -int i, bar;/* For needs_reset */ lockdep_assert_held (&vdev->vdev.dev_set->lock);@@ -759,24 +795,12 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) vdev->virq_disabled = false; -for (i =0 ; i < vdev->num_regions; i++) - vdev->region[i].ops->release (vdev, &vdev->region[i]); - - vdev->num_regions =0 ; -kfree (vdev->region); - vdev->region =NULL ;/* don't krealloc a freed pointer */ +vfio_pci_core_free_dev_regions (vdev);vfio_config_free (vdev);vfio_pci_free_excluded_ranges (vdev); -for (i =0 ; i <PCI_STD_NUM_BARS ; i++) { - bar = i +PCI_STD_RESOURCES ; -if (IS_ERR_OR_NULL (vdev->barmap[bar])) -continue ; -pci_iounmap (pdev, vdev->barmap[bar]); -pci_release_selected_regions (pdev,1 << bar); - vdev->barmap[bar] =NULL ; - } +vfio_pci_core_unmap_bars (vdev);list_for_each_entry_safe (dummy_res, tmp, &vdev->dummy_resources_list, res_next) {
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_close_device(struct vfio_device *core_vdev) · line 888
Quoted 6 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0@@ -859,6 +883,9 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev)#endif vfio_pci_dma_buf_cleanup (vdev); +if (vdev->cxl_ops) + vdev->cxl_ops->close_device (vdev); +
Mismatched in both naming and conventions.
vfio_pci_core_enable() -> cxl_ops->open_device()
vfio_pci_core_close_device() -> cxl_ops->close_device()
Some variant drivers unwind a failure after core_enable() with core_disable(), so the placement in the close_device() path is not only inconsistent, it's wrong. The hooks should also be consistently named relative to where they're called from the core. Thanks,
Alex
Quoted 16 lines
drivers/vfio/pci/vfio_pci_core.c +0
-0vfio_pci_core_disable (vdev);mutex_lock (&vdev->igate);include/linux/vfio_pci_core.h +2
-0@@ -70,6 +70,8 @@ struct vfio_pci_device_ops {struct vfio_cxl_ops {int (*init)(struct vfio_pci_core_device *vdev);void (*release)(struct vfio_pci_core_device *vdev); +int (*open_device)(struct vfio_pci_core_device *vdev); +void (*close_device)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
RE: [PATCH v6 12/31] vfio/pci: Call the CXL open and close hooks around device use
Quoted 77 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Saturday, November 7, 2026 4:22 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 12/31] vfio/pci: Call the CXL open and close hooks around device useExternal email: Use caution opening links or attachments
Quoted 49 lines · On Fri, 16 Oct 2026 21:48:19 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
vfio-pci-core frees device regions and excluded ranges on close and on a failed open, so a CXL provider must set them up at each open.
Add open and close to the CXL ops and call them from the common enable and close paths. A failed open unwinds the enable in reverse order:
release the device regions, unmap the BARs, drop the excluded ranges, then free the virtual config.Factor the device region release and the BAR unmap out of
vfio_pci_core_disable()into helpers so the enable failure path and the disable path share them.Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 57 +++++++++++++++++++++++--------- include/linux/vfio_pci_core.h | 2 ++ 2 files changed, 44 insertions(+), 15 deletions(-)drivers/vfio/pci/vfio_pci_core.c +17
-0@@ -590,6 +590,31 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = {static void vfio_pci_free_excluded_ranges (struct vfio_pci_core_device *vdev); +static void vfio_pci_core_unmap_bars (struct vfio_pci_core_device +*vdev) { +struct pci_dev *pdev = vdev->pdev; + +for (int i =0 ; i <PCI_STD_NUM_BARS ; i++) { +int bar = i +PCI_STD_RESOURCES ; + +if (IS_ERR_OR_NULL (vdev->barmap[bar])) +continue ; +pci_iounmap (pdev, vdev->barmap[bar]); +pci_release_selected_regions (pdev,1 << bar); + vdev->barmap[bar] =NULL ; + } +} + +static void vfio_pci_core_free_dev_regions (struct +vfio_pci_core_device *vdev) {
Quoted 8 lines
drivers/vfio/pci/vfio_pci_core.c static const struct dev_pm_ops vfio_pci_core_pm_ops = {
Quoted 6 lines
drivers/vfio/pci/vfio_pci_core.c +6
-0+for (int i =0 ; i < vdev->num_regions; i++) + vdev->region[i].ops->release (vdev, &vdev->region[i]); + + vdev->num_regions =0 ; +kfree (vdev->region); + vdev->region =NULL ;/* don't krealloc a freed pointer */ }Mechanical changes like this should be done in a separate patch.
Okay, I will split these out.
Quoted 26 lines
Quoted 12 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0+int vfio_pci_core_enable (struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; @@ -691,8 +716,20 @@ int
vfio_pci_core_enable(struct vfio_pci_core_device *vdev)vfio_pci_core_map_bars(vdev);drivers/vfio/pci/vfio_pci_core.c +4
-0+if (vdev->cxl_ops) { + ret = vdev->cxl_ops->open_device (vdev); +if (ret) + goto out_free_config;drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 732
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +8
-0+ } +return 0 ; +out_free_config: +/* A failed first open never reaches vfio_pci_core_disable(). */ +vfio_pci_core_free_dev_regions (vdev); +vfio_pci_core_unmap_bars (vdev); +vfio_pci_free_excluded_ranges (vdev); +vfio_config_free (vdev);Raising again that this doesn't account for the code previously added to this function that has its own unwind.
Quoted 3 lines
There also remains the question of whether excluded range setup should be moved to init/release.
Yes, for v7, I will move the range setup to init/release.
Quoted 50 lines
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_disable(struct
Quoted 4 lines
out_free_zdev:vfio_pci_zdev_close_device(vdev);
out_free_state:drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -713,7 +750,6 @@ void vfio_pci_core_disable(struct
vfio_pci_core_device*vdev)Quoted 39 lines
drivers/vfio/pci/vfio_pci_core.c +2
-15struct pci_dev *pdev = vdev->pdev;struct vfio_pci_dummy_resource *dummy_res, *tmp;struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp; -int i, bar;/* For needs_reset */ lockdep_assert_held (&vdev->vdev.dev_set->lock);@@ -759,24 +795,12 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) vdev->virq_disabled = false; -for (i =0 ; i < vdev->num_regions; i++) - vdev->region[i].ops->release (vdev, &vdev->region[i]); - - vdev->num_regions =0 ; -kfree (vdev->region); - vdev->region =NULL ;/* don't krealloc a freed pointer */ +vfio_pci_core_free_dev_regions (vdev);vfio_config_free (vdev);vfio_pci_free_excluded_ranges (vdev); -for (i =0 ; i <PCI_STD_NUM_BARS ; i++) { - bar = i +PCI_STD_RESOURCES ; -if (IS_ERR_OR_NULL (vdev->barmap[bar])) -continue ; -pci_iounmap (pdev, vdev->barmap[bar]); -pci_release_selected_regions (pdev,1 << bar); - vdev->barmap[bar] =NULL ; - } +vfio_pci_core_unmap_bars (vdev); list_for_each_entry_safe(dummy_res, tmp,&vdev->dummy_resources_list, res_next) { @@ -859,6 +883,9 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev) #endif vfio_pci_dma_buf_cleanup(vdev);drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_close_device(struct vfio_device *core_vdev) · line 888
Quoted 3 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0+if (vdev->cxl_ops) + vdev->cxl_ops->close_device (vdev); +Mismatched in both naming and conventions.
Quoted 8 lines
vfio_pci_core_enable()->cxl_ops->open_device()
vfio_pci_core_close_device()->cxl_ops->close_device()Some variant drivers unwind a failure after
core_enable()withcore_disable(), so the placement in theclose_device()path is not only inconsistent, it's wrong. The hooks should also be consistently named relative to where they're called from the core. Thanks,
Agreed. For v7, I will name them enable and disable and call them from vfio_pci_core_enable() and vfio_pci_core_disable().
Thanks,
Manish
Quoted 19 lines
Alex
Quoted 16 lines
vfio_pci_core_disable(vdev); mutex_lock(&vdev->igate);(quoted diff) +0
-0
b/include/linux/vfio_pci_core.hindexfd1ad0baf470..6520a2dcfd4a100644include/linux/vfio_pci_core.h +2
-0@@ -70,6 +70,8 @@ struct vfio_pci_device_ops { struct vfio_cxl_ops {int (*init)(struct vfio_pci_core_device *vdev);void (*release)(struct vfio_pci_core_device *vdev); +int (*open_device)(struct vfio_pci_core_device *vdev); +void (*close_device)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
PATCH v6 13/31 vfio/pci: Bracket PCI resets with the CXL reset hooks
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL Type-2 device must quiesce its coherent link before any reset and restore its HDM decoder state afterward, or a reset can leave the device mastering the bus over decoders that are no longer valid. This has to happen around every reset the PCI core drives for the device, not only the ones VFIO issues directly, and around AER and DPC recovery, which can reset the link without the reset callbacks.
Add:
- reset_prepare
- reset_done
- an optional error_resume to the CXL ops
Export:
- vfio_pci_core_reset_prepare()
- vfio_pci_core_reset_done()
- vfio_pci_core_error_detected()
- vfio_pci_core_error_resume()
which take the vfio device.
vfio-pci calls them from its own pci_error_handlers, which resolve the driver data it owns. A variant driver that binds CXL devices calls them from its own handlers the same way; a driver without reset_prepare, reset_done, error_detected and resume callbacks gets no CXL support.
The provider hooks run with memory_lock held for write: vfio-pci-core already holds it across its own hot reset, and vfio_pci_core_reset_prepare() takes it for a reset started anywhere else until vfio_pci_core_reset_done(). The hot reset records its task, so a reset that another task starts meanwhile waits for memory_lock instead of skipping it. pci_dev_lock() blocks user config access, which a PCI_COMMAND write can be waiting for while it holds memory_lock, so reset_prepare lets config access through while it waits. reset_done returns an error when the provider could not restore its device, and VFIO_DEVICE_PCI_HOT_RESET returns that error; for any other reset the provider logs it.
--- drivers/vfio/pci/vfio_pci.c | 39 +++++++- drivers/vfio/pci/vfio_pci_core.c | 154 +++++++++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 22 +++++ 3 files changed, 214 insertions(+), 1 deletion(-)
drivers/vfio/pci/vfio_pci.c +38 -1
@@ -223,13 +223,50 @@ static const struct pci_device_id vfio_pci_table[] = {MODULE_DEVICE_TABLE (pci, vfio_pci_table); +static void vfio_pci_reset_prepare (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_prepare (vdev); +} + +static void vfio_pci_reset_done (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_done (vdev); +} + +static pci_ers_result_t vfio_pci_error_detected (struct pci_dev *pdev, +pci_channel_state_t state) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_detected (vdev); +return vfio_pci_core_aer_err_detected (pdev, state); +} + +static void vfio_pci_error_resume (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_resume (vdev); +} + +static const struct pci_error_handlers vfio_pci_err_handlers = { + .error_detected = vfio_pci_error_detected, + .reset_prepare = vfio_pci_reset_prepare, + .reset_done = vfio_pci_reset_done, + .resume = vfio_pci_error_resume, +}; +static struct pci_driver vfio_pci_driver = { .name ="vfio-pci" , .id_table = vfio_pci_table, .probe = vfio_pci_probe, .remove = vfio_pci_remove, .sriov_configure = vfio_pci_sriov_configure, - .err_handler = &vfio_pci_core_err_handlers, + .err_handler = &vfio_pci_err_handlers, .driver_managed_dma = true, };
drivers/vfio/pci/vfio_pci_core.c +154 -0
190 lines not on this page · raw message
include/linux/vfio_pci_core.h +22 -0
@@ -72,6 +72,21 @@ struct vfio_cxl_ops {void (*release)(struct vfio_pci_core_device *vdev);int (*open_device)(struct vfio_pci_core_device *vdev);void (*close_device)(struct vfio_pci_core_device *vdev); +/* +* Optional. Called with memory_lock held for write when AER or DPC +* recovery resumes the device, after reset_prepare was called for the +* error. Config space is as the recovery left it. +*/ +void (*error_resume)(struct vfio_pci_core_device *vdev); +/* +* Called around a PCI core reset of the device with memory_lock held +* for write; reset_prepare also when AER or DPC recovery reports an +* error. reset_done returns an error if the provider could not +* restore the device; VFIO_DEVICE_PCI_HOT_RESET returns it, and for +* any other reset the provider logs it. +*/ +void (*reset_prepare)(struct vfio_pci_core_device *vdev); +int (*reset_done)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
@@ -166,6 +181,9 @@ struct vfio_pci_core_device {struct vfio_pci_core_device *sriov_pf_core_dev;struct notifier_block nb;struct rw_semaphore memory_lock; +/* Task running VFIO_DEVICE_PCI_HOT_RESET while it holds memory_lock */ +struct task_struct *hot_reset_task; +int reset_done_err;struct list_head dmabufs;/* * Excluded BAR sub-ranges, one list per BAR, sorted by start and not
@@ -207,6 +225,10 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev);int vfio_pci_core_register_device (struct vfio_pci_core_device *vdev);void vfio_pci_core_unregister_device (struct vfio_pci_core_device *vdev);extern const struct pci_error_handlers vfio_pci_core_err_handlers; +void vfio_pci_core_reset_prepare (struct vfio_pci_core_device *vdev); +void vfio_pci_core_reset_done (struct vfio_pci_core_device *vdev); +void vfio_pci_core_error_detected (struct vfio_pci_core_device *vdev); +void vfio_pci_core_error_resume (struct vfio_pci_core_device *vdev);int vfio_pci_core_sriov_configure (struct vfio_pci_core_device *vdev,int nr_virtfn); long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd,
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 13/31] vfio/pci: Bracket PCI resets with the CXL reset hooks
Quoted 26 lines · On Fri, 16 Oct 2026 21:48:20 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL Type-2 device must quiesce its coherent link before any reset and restore its HDM decoder state afterward, or a reset can leave the device mastering the bus over decoders that are no longer valid. This has to happen around every reset the PCI core drives for the device, not only the ones VFIO issues directly, and around AER and DPC recovery, which can reset the link without the reset callbacks.
Add:
-reset_prepare
-reset_done
- an optionalerror_resumeto the CXL opsExport:
-vfio_pci_core_reset_prepare()
-vfio_pci_core_reset_done()
-vfio_pci_core_error_detected()
-vfio_pci_core_error_resume()
which take the vfio device.vfio-pci calls them from its own
pci_error_handlers, which resolve the driver data it owns. A variant driver that binds CXL devices calls them from its own handlers the same way; a driver withoutreset_prepare,reset_done,error_detectedand resume callbacks gets no CXL support.
Quoted 11 lines
The provider hooks run with
memory_lockheld for write: vfio-pci-core already holds it across its own hot reset, andvfio_pci_core_reset_prepare()takes it for a reset started anywhere else untilvfio_pci_core_reset_done(). The hot reset records its task, so a reset that another task starts meanwhile waits formemory_lockinstead of skipping it.pci_dev_lock()blocks user config access, which aPCI_COMMANDwrite can be waiting for while it holdsmemory_lock, soreset_preparelets config access through while it waits.reset_donereturns an error when the provider could not restore its device, andVFIO_DEVICE_PCI_HOT_RESETreturns that error; for any other reset the provider logs it.
Cryptic. Seems like we're skipping updating all the variant drivers to make use of the new reset and error hooks, which seems wrong.
This also needs to be rebased to the the new drvdata handling in v7.4-rc.
I'm also not sure why the "provider" terminology has snuck into this series with this level of prevalence. There's only one cxl ops "provider", it's vfio-cxl.
Quoted 87 lines
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci.c | 39 +++++++- drivers/vfio/pci/vfio_pci_core.c | 154 +++++++++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 22 +++++ 3 files changed, 214 insertions(+), 1 deletion(-)drivers/vfio/pci/vfio_pci.c +38
-1@@ -223,13 +223,50 @@ static const struct pci_device_id vfio_pci_table[] = {MODULE_DEVICE_TABLE (pci, vfio_pci_table); +static void vfio_pci_reset_prepare (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_prepare (vdev); +} + +static void vfio_pci_reset_done (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_done (vdev); +} + +static pci_ers_result_t vfio_pci_error_detected (struct pci_dev *pdev, +pci_channel_state_t state) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_detected (vdev); +return vfio_pci_core_aer_err_detected (pdev, state); +} + +static void vfio_pci_error_resume (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_resume (vdev); +} + +static const struct pci_error_handlers vfio_pci_err_handlers = { + .error_detected = vfio_pci_error_detected, + .reset_prepare = vfio_pci_reset_prepare, + .reset_done = vfio_pci_reset_done, + .resume = vfio_pci_error_resume, +}; +static struct pci_driver vfio_pci_driver = { .name ="vfio-pci" , .id_table = vfio_pci_table, .probe = vfio_pci_probe, .remove = vfio_pci_remove, .sriov_configure = vfio_pci_sriov_configure, - .err_handler = &vfio_pci_core_err_handlers, + .err_handler = &vfio_pci_err_handlers, .driver_managed_dma = true, };drivers/vfio/pci/vfio_pci_core.c +8
-0@@ -2462,6 +2462,7 @@ static bool vfio_pci_is_cxl_type2(struct pci_dev *pdev)*/ static int vfio_pci_core_cxl_init (struct vfio_pci_core_device *vdev) { +const struct pci_error_handlers *err_handler;const struct vfio_cxl_ops *ops;int ret;@@ -2473,6 +2474,23 @@ static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)if (!ops)return IS_BUILTIN (CONFIG_VFIO_CXL ) ? -EPROBE_DEFER :0 ; +/* +* The provider must see every PCI core reset of the device and every +* AER or DPC recovery. A driver without reset_prepare, reset_done, +* error_detected and resume callbacks gets no CXL support; the driver's +* callbacks must call vfio_pci_core_reset_prepare(), _reset_done(), +* _error_detected() and _error_resume(). +*/
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev) · line 2493
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+ err_handler = vdev->pdev->driver ? vdev->pdev->driver->err_handler :NULL ; +if (!err_handler || !err_handler->reset_prepare || + !err_handler->reset_done || !err_handler->error_detected || + !err_handler->resume) { +pci_info (vdev->pdev, +"driver lacks PCI reset or error recovery hooks, continuing without CXL support\n" ); +vfio_pci_put_cxl_ops (ops); +return 0 ; + } +
Just implement them in all the variant drivers and make them validated in vfio_pci_core_register_device().
Quoted 48 lines
drivers/vfio/pci/vfio_pci_core.c +41
-0ret = ops->init (vdev);/* * A provider that is not ready yet (for example its CXL port has not @@ -2759,6 +2777,128 @@ int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, }EXPORT_SYMBOL_GPL (vfio_pci_core_sriov_configure); +/* +* Take memory_lock for a reset that the PCI core runs on a locked device. +* pci_dev_lock() also blocks user config access, and vfio_basic_config_write() +* holds memory_lock while its pci_user_*_config() calls wait for that block +* to clear. Waiting for memory_lock with config access blocked would hang +* both tasks, so let user config access through while waiting: the PCI core +* calls reset_prepare before it saves or touches the device. The caller holds +* the device lock, so a block found set is the caller's own. +*/ +static void vfio_pci_reset_lock_memory (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; + +if (down_write_trylock (&vdev->memory_lock)) +return ; + +/* The caller did not block config access: nothing to release. */ +if (pci_cfg_access_trylock (pdev)) { +pci_cfg_access_unlock (pdev); +down_write (&vdev->memory_lock); +return ; + } + +pci_cfg_access_unlock (pdev); +down_write (&vdev->memory_lock); +pci_cfg_access_lock (pdev); +} + +/** +* vfio_pci_core_reset_prepare() - prepare a device for a PCI core reset +* @vdev: vfio PCI device about to be reset +* +* Call from the driver's pci_error_handlers.reset_prepare callback. +*/ +void vfio_pci_core_reset_prepare (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_ops) +return ; + +/* +* vfio-pci-core holds memory_lock across its own hot reset. Hold it
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, · line 2830
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+* across a reset started anywhere else too, until reset_done. Test +* the owner, not just a flag: another task can reset the device while +* a hot reset holds memory_lock and has not yet locked the device. +*/ +if (READ_ONCE (vdev->hot_reset_task) != current) +vfio_pci_reset_lock_memory (vdev); + + vdev->cxl_ops->reset_prepare (vdev); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_reset_prepare);
This is all super sketchy. Sashiko triggers multiple critical issues. Having different locking behavior across CXL vs non-CXL device is not acceptable. These callbacks occur even if the device is reset through sysfs or AER while the device is not opened. Inferring whether the lock is held or not is very hacky. Thanks,
Alex
Quoted 156 lines
drivers/vfio/pci/vfio_pci_core.c +85
-0+ +/** +* vfio_pci_core_reset_done() - restore a device after a PCI core reset +* @vdev: vfio PCI device that was reset +* +* Call from the driver's pci_error_handlers.reset_done callback. +*/ +void vfio_pci_core_reset_done (struct vfio_pci_core_device *vdev) +{ +int ret; + +if (!vdev->cxl_ops) +return ; + + ret = vdev->cxl_ops->reset_done (vdev); + +/* +* hot_reset_task cannot change since reset_prepare: setting it needs +* memory_lock, which this task or the hot reset task holds. +*/ +if (READ_ONCE (vdev->hot_reset_task) == current) { +/* VFIO_DEVICE_PCI_HOT_RESET returns the error. */ + vdev->reset_done_err = ret; +return ; + } + +/* +* Nobody waits for the result of a reset started elsewhere. A provider +* that failed has logged it and keeps its device memory out of use +* until a later reset restores the device. +*/ +up_write (&vdev->memory_lock); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_reset_done); + +/** +* vfio_pci_core_error_detected() - prepare a device for AER or DPC recovery +* @vdev: vfio PCI device the recovery reports an error on +* +* Call from the driver's pci_error_handlers.error_detected callback. The +* recovery may reset the device without reset_prepare and reset_done, so a +* provider takes its device memory out of use here. +*/ +void vfio_pci_core_error_detected (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_ops) +return ; + +guard (rwsem_write)(&vdev->memory_lock); + vdev->cxl_ops->reset_prepare (vdev); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_error_detected); + +/** +* vfio_pci_core_error_resume() - resume a device after AER or DPC recovery +* @vdev: vfio PCI device the recovery resumes +* +* Call from the driver's pci_error_handlers.resume callback. A provider puts +* its device memory back if the device still decodes it; otherwise the memory +* stays out of use until the user resets the device. +*/ +void vfio_pci_core_error_resume (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_ops || !vdev->cxl_ops->error_resume) +return ; + +guard (rwsem_write)(&vdev->memory_lock); + vdev->cxl_ops->error_resume (vdev); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_error_resume); +const struct pci_error_handlers vfio_pci_core_err_handlers = { .error_detected = vfio_pci_core_aer_err_detected, };@@ -2915,6 +3055,8 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set, ret = -EBUSY ;break ; } +WRITE_ONCE (vdev->hot_reset_task, current); + vdev->reset_done_err =0 ;vfio_pci_dma_buf_move (vdev, true);vfio_pci_zap_bars (vdev);@@ -2940,6 +3082,17 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set, ret =pci_reset_bus (pdev); +/* A provider that could not restore its device fails the hot reset. */ +if (!ret) { +list_for_each_entry (vdev, &dev_set->device_list, + vdev.dev_set_list) { +if (vdev->reset_done_err) { + ret = vdev->reset_done_err; +break ; + } + } + } + vdev =list_last_entry (&dev_set->device_list, struct vfio_pci_core_device, vdev.dev_set_list);@@ -2948,6 +3101,7 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set, vdev.dev_set_list) {if (vdev->vdev.open_count &&__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false); +WRITE_ONCE (vdev->hot_reset_task,NULL );up_write (&vdev->memory_lock); }include/linux/vfio_pci_core.h +22
-0@@ -72,6 +72,21 @@ struct vfio_cxl_ops {void (*release)(struct vfio_pci_core_device *vdev);int (*open_device)(struct vfio_pci_core_device *vdev);void (*close_device)(struct vfio_pci_core_device *vdev); +/* +* Optional. Called with memory_lock held for write when AER or DPC +* recovery resumes the device, after reset_prepare was called for the +* error. Config space is as the recovery left it. +*/ +void (*error_resume)(struct vfio_pci_core_device *vdev); +/* +* Called around a PCI core reset of the device with memory_lock held +* for write; reset_prepare also when AER or DPC recovery reports an +* error. reset_done returns an error if the provider could not +* restore the device; VFIO_DEVICE_PCI_HOT_RESET returns it, and for +* any other reset the provider logs it. +*/ +void (*reset_prepare)(struct vfio_pci_core_device *vdev); +int (*reset_done)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };@@ -166,6 +181,9 @@ struct vfio_pci_core_device {struct vfio_pci_core_device *sriov_pf_core_dev;struct notifier_block nb;struct rw_semaphore memory_lock; +/* Task running VFIO_DEVICE_PCI_HOT_RESET while it holds memory_lock */ +struct task_struct *hot_reset_task; +int reset_done_err;struct list_head dmabufs;/* * Excluded BAR sub-ranges, one list per BAR, sorted by start and not @@ -207,6 +225,10 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev);int vfio_pci_core_register_device (struct vfio_pci_core_device *vdev);void vfio_pci_core_unregister_device (struct vfio_pci_core_device *vdev);extern const struct pci_error_handlers vfio_pci_core_err_handlers; +void vfio_pci_core_reset_prepare (struct vfio_pci_core_device *vdev); +void vfio_pci_core_reset_done (struct vfio_pci_core_device *vdev); +void vfio_pci_core_error_detected (struct vfio_pci_core_device *vdev); +void vfio_pci_core_error_resume (struct vfio_pci_core_device *vdev);int vfio_pci_core_sriov_configure (struct vfio_pci_core_device *vdev,int nr_virtfn); long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd,
RE: [PATCH v6 13/31] vfio/pci: Bracket PCI resets with the CXL reset hooks
Quoted 67 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Saturday, November 7, 2026 5:14 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 13/31] vfio/pci: Bracket PCI resets with the CXL reset hooksExternal email: Use caution opening links or attachments
Quoted 21 lines · On Fri, 16 Oct 2026 21:48:20 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL Type-2 device must quiesce its coherent link before any reset and restore its HDM decoder state afterward, or a reset can leave the device mastering the bus over decoders that are no longer valid. This has to happen around every reset the PCI core drives for the device, not only the ones VFIO issues directly, and around AER and DPC recovery, which can reset the link without the reset callbacks.
Add:
-reset_prepare
-reset_done
- an optionalerror_resumeto the CXL opsExport:
-vfio_pci_core_reset_prepare()
-vfio_pci_core_reset_done()
-vfio_pci_core_error_detected()
-vfio_pci_core_error_resume()
which take the vfio device.Quoted 4 lines
vfio-pci calls them from its own
pci_error_handlers, which resolve the driver data it owns. A variant driver that binds CXL devices calls them from its own handlers the same way; a driver withoutreset_prepare,reset_done,error_detectedand resume callbacks gets no CXLsupport.
Quoted 12 lines
The provider hooks run with
memory_lockheld for write: vfio-pci-core already holds it across its own hot reset, andvfio_pci_core_reset_prepare()takes it for a reset started anywhere else untilvfio_pci_core_reset_done(). The hot reset records its task, so a reset that another task starts meanwhile waits formemory_lockinstead of skipping it.pci_dev_lock()blocks user config access, which aPCI_COMMANDwrite can be waiting for while it holdsmemory_lock, soreset_preparelets config access through while it waits.reset_donereturns an error when the provider could not restore its device, andVFIO_DEVICE_PCI_HOT_RESETreturns that error; for any other reset the provider logs it.
Quoted 4 lines
Cryptic. Seems like we're skipping updating all the variant drivers to make use of the new reset and error hooks, which seems wrong.
This also needs to be rebased to the the new drvdata handling in v7.4-rc.
Okay, I will handle both the flagged items. I will check the new drvdata handling.
Quoted 3 lines
I'm also not sure why the "provider" terminology has snuck into this series with this level of prevalence. There's only one cxl ops "provider", it's vfio-cxl.
Okay, for v7, I will mention this as vfio-cxl.
Quoted 94 lines
Quoted 6 lines
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci.c | 39 +++++++- drivers/vfio/pci/vfio_pci_core.c | 154+++++++++++++++++++++++++++++++
drivers/vfio/pci/vfio_pci_core.c static bool vfio_pci_is_cxl_type2(struct pci_dev
Quoted 60 lines
include/linux/vfio_pci_core.h | 22 +++++ 3 files changed, 214 insertions(+), 1 deletion(-)drivers/vfio/pci/vfio_pci.c +32
-1@@ -223,13 +223,50 @@ static const struct pci_device_id vfio_pci_table[] = {MODULE_DEVICE_TABLE (pci, vfio_pci_table); +static void vfio_pci_reset_prepare (struct pci_dev *pdev) { +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_prepare (vdev); +} + +static void vfio_pci_reset_done (struct pci_dev *pdev) { +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_done (vdev); +} + +static pci_ers_result_t vfio_pci_error_detected (struct pci_dev *pdev, +pci_channel_state_t +state) { +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_detected (vdev); +return vfio_pci_core_aer_err_detected (pdev, state); } + +static void vfio_pci_error_resume (struct pci_dev *pdev) { +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_resume (vdev); } + +static const struct pci_error_handlers vfio_pci_err_handlers = { + .error_detected = vfio_pci_error_detected, + .reset_prepare = vfio_pci_reset_prepare, + .reset_done = vfio_pci_reset_done, + .resume = vfio_pci_error_resume, }; +static struct pci_driver vfio_pci_driver = { .name ="vfio-pci" , .id_table = vfio_pci_table, .probe = vfio_pci_probe, .remove = vfio_pci_remove, .sriov_configure = vfio_pci_sriov_configure, - .err_handler = &vfio_pci_core_err_handlers, + .err_handler = &vfio_pci_err_handlers, .driver_managed_dma = true, };drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -2462,6 +2462,7 @@ static bool vfio_pci_is_cxl_type2(struct pci_dev*pdev)
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct
Quoted 8 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0*/ static int vfio_pci_core_cxl_init (struct vfio_pci_core_device *vdev) { +const struct pci_error_handlers *err_handler;const struct vfio_cxl_ops *ops;int ret;@@ -2473,6 +2474,23 @@ static int vfio_pci_core_cxl_init(struct
vfio_pci_core_device*vdev)drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
Quoted 11 lines
drivers/vfio/pci/vfio_pci_core.c +8
-0if (!ops)return IS_BUILTIN (CONFIG_VFIO_CXL ) ? -EPROBE_DEFER :0 ; +/* +* The provider must see every PCI core reset of the device and every +* AER or DPC recovery. A driver without reset_prepare, reset_done, +* error_detected and resume callbacks gets no CXL support; the driver's +* callbacks must call vfio_pci_core_reset_prepare(), _reset_done(), +* _error_detected() and _error_resume(). +*/ + err_handler = vdev->pdev->driver ? vdev->pdev->driver->err_handler :NULL;
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c +5
-0+if (!err_handler || !err_handler->reset_prepare || + !err_handler->reset_done || !err_handler->error_detected || + !err_handler->resume) { + pci_info(vdev->pdev, + "driver lacks PCI reset or error recovery hooks, continuing
Quoted 8 lines
without CXL support\n");
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev) · line 2493
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0+vfio_pci_put_cxl_ops (ops); +return 0 ; + } +Just implement them in all the variant drivers and make them validated in
vfio_pci_core_register_device().
Okay, I will check this part.
Quoted 63 lines
Quoted 63 lines
drivers/vfio/pci/vfio_pci_core.c +0
-0ret = ops->init (vdev);/* * A provider that is not ready yet (for example its CXL port has not @@ -2759,6 +2777,128 @@ int vfio_pci_core_sriov_configure(struct
vfio_pci_core_device*vdev, }EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure);drivers/vfio/pci/vfio_pci_core.c +55
-0+/* +* Take memory_lock for a reset that the PCI core runs on a locked device. +* pci_dev_lock() also blocks user config access, and +vfio_basic_config_write() +* holds memory_lock while its pci_user_*_config() calls wait for +that block +* to clear. Waiting for memory_lock with config access blocked would +hang +* both tasks, so let user config access through while waiting: the +PCI core +* calls reset_prepare before it saves or touches the device. The +caller holds +* the device lock, so a block found set is the caller's own. +*/ +static void vfio_pci_reset_lock_memory (struct vfio_pci_core_device +*vdev) { +struct pci_dev *pdev = vdev->pdev; + +if (down_write_trylock (&vdev->memory_lock)) +return ; + +/* The caller did not block config access: nothing to release. */ +if (pci_cfg_access_trylock (pdev)) { +pci_cfg_access_unlock (pdev); +down_write (&vdev->memory_lock); +return ; + } + +pci_cfg_access_unlock (pdev); +down_write (&vdev->memory_lock); +pci_cfg_access_lock (pdev); +} + +/** +* vfio_pci_core_reset_prepare() - prepare a device for a PCI core +reset +* @vdev: vfio PCI device about to be reset +* +* Call from the driver's pci_error_handlers.reset_prepare callback. +*/ +void vfio_pci_core_reset_prepare (struct vfio_pci_core_device *vdev) { +if (!vdev->cxl_ops) +return ; + +/* +* vfio-pci-core holds memory_lock across its own hot reset. Hold it +* across a reset started anywhere else too, until reset_done. Test +* the owner, not just a flag: another task can reset the device while +* a hot reset holds memory_lock and has not yet locked the device. +*/ +if (READ_ONCE (vdev->hot_reset_task) != current) +vfio_pci_reset_lock_memory (vdev); + + vdev->cxl_ops->reset_prepare (vdev); +}
Quoted 8 lines
Quoted 1 line
+EXPORT_SYMBOL_GPL(vfio_pci_core_reset_prepare);
This is all super sketchy. Sashiko triggers multiple critical issues. Having different locking behavior across CXL vs non-CXL device is not acceptable. These callbacks occur even if the device is reset through sysfs or AER while the device is not opened. Inferring whether the lock is held or not is very hacky. Thanks,
Okay, I will recheck this part. For my review I was only using review-prompts kernel md file instructions for my LLM review. I will setup local sashiko and verify this with complete runs.
What vfio-cxl needs from these callbacks is to keep the device memory unmapped from reset_prepare until the device decodes it again. memory_lock was the convenient lock for that, not a requirement: CXL memory decode does not depend on the Memory Space bit. So, in v7 I will serialize vfio-cxl memory region with its own lock and take it in its hooks.
vfio-pci-core will not take lock in the hooks, so its locking is the same for every device, and hot_reset_task and reset_done_err will go away.
Thanks,
Manish
Quoted 169 lines
Alex
Quoted 69 lines
drivers/vfio/pci/vfio_pci_core.c +69
-0+ +/** +* vfio_pci_core_reset_done() - restore a device after a PCI core +reset +* @vdev: vfio PCI device that was reset +* +* Call from the driver's pci_error_handlers.reset_done callback. +*/ +void vfio_pci_core_reset_done (struct vfio_pci_core_device *vdev) { +int ret; + +if (!vdev->cxl_ops) +return ; + + ret = vdev->cxl_ops->reset_done (vdev); + +/* +* hot_reset_task cannot change since reset_prepare: setting it needs +* memory_lock, which this task or the hot reset task holds. +*/ +if (READ_ONCE (vdev->hot_reset_task) == current) { +/* VFIO_DEVICE_PCI_HOT_RESET returns the error. */ + vdev->reset_done_err = ret; +return ; + } + +/* +* Nobody waits for the result of a reset started elsewhere. A provider +* that failed has logged it and keeps its device memory out of use +* until a later reset restores the device. +*/ +up_write (&vdev->memory_lock); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_reset_done); + +/** +* vfio_pci_core_error_detected() - prepare a device for AER or DPC +recovery +* @vdev: vfio PCI device the recovery reports an error on +* +* Call from the driver's pci_error_handlers.error_detected callback. +The +* recovery may reset the device without reset_prepare and +reset_done, so a +* provider takes its device memory out of use here. +*/ +void vfio_pci_core_error_detected (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_ops) +return ; + +guard (rwsem_write)(&vdev->memory_lock); + vdev->cxl_ops->reset_prepare (vdev); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_error_detected); + +/** +* vfio_pci_core_error_resume() - resume a device after AER or DPC +recovery +* @vdev: vfio PCI device the recovery resumes +* +* Call from the driver's pci_error_handlers.resume callback. A +provider puts +* its device memory back if the device still decodes it; otherwise +the memory +* stays out of use until the user resets the device. +*/ +void vfio_pci_core_error_resume (struct vfio_pci_core_device *vdev) { +if (!vdev->cxl_ops || !vdev->cxl_ops->error_resume)drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +8
-0+return ; + +guard (rwsem_write)(&vdev->memory_lock); + vdev->cxl_ops->error_resume (vdev); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_error_resume); +const struct pci_error_handlers vfio_pci_core_err_handlers = { .error_detected = vfio_pci_core_aer_err_detected, }; @@ -2915,6 +3055,8 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set
*dev_set,drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_dev_set_hot_reset(struct
Quoted 28 lines
ret = -EBUSY; break; }drivers/vfio/pci/vfio_pci_core.c +2
-0+WRITE_ONCE (vdev->hot_reset_task, current); + vdev->reset_done_err =0 ;vfio_pci_dma_buf_move (vdev, true); vfio_pci_zap_bars(vdev); @@ -2940,6 +3082,17 @@ staticint vfio_pci_dev_set_hot_reset(struct
vfio_device_set*dev_set,ret = pci_reset_bus(pdev);drivers/vfio/pci/vfio_pci_core.c +11
-0+/* A provider that could not restore its device fails the hot reset. */ +if (!ret) { +list_for_each_entry (vdev, &dev_set->device_list, + vdev.dev_set_list) { +if (vdev->reset_done_err) { + ret = vdev->reset_done_err; +break ; + } + } + } + vdev = list_last_entry(&dev_set->device_list, struct vfio_pci_core_device,vdev.dev_set_list);
drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -2948,6 +3101,7 @@ static int vfio_pci_dev_set_hot_reset(struct
vfio_device_set*dev_set,Quoted 35 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0vdev.dev_set_list) {if (vdev->vdev.open_count &&__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false); +WRITE_ONCE (vdev->hot_reset_task,NULL );up_write (&vdev->memory_lock); }(quoted diff) +0
-0
b/include/linux/vfio_pci_core.hindex6520a2dcfd4a..881e51c42414100644include/linux/vfio_pci_core.h +15
-0@@ -72,6 +72,21 @@ struct vfio_cxl_ops {void (*release)(struct vfio_pci_core_device *vdev);int (*open_device)(struct vfio_pci_core_device *vdev);void (*close_device)(struct vfio_pci_core_device *vdev); +/* +* Optional. Called with memory_lock held for write when AER or DPC +* recovery resumes the device, after reset_prepare was called for the +* error. Config space is as the recovery left it. +*/ +void (*error_resume)(struct vfio_pci_core_device *vdev); +/* +* Called around a PCI core reset of the device with memory_lock held +* for write; reset_prepare also when AER or DPC recovery reports an +* error. reset_done returns an error if the provider could not +* restore the device; VFIO_DEVICE_PCI_HOT_RESET returns it, and for +* any other reset the provider logs it. +*/ +void (*reset_prepare)(struct vfio_pci_core_device *vdev); +int (*reset_done)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };include/linux/vfio_pci_core.h struct vfio_pci_core_device { · line 184
Quoted 4 lines
include/linux/vfio_pci_core.h +1
-0@@ -166,6 +181,9 @@ struct vfio_pci_core_device {struct vfio_pci_core_device *sriov_pf_core_dev;struct notifier_block nb;struct rw_semaphore memory_lock; +/* Task running VFIO_DEVICE_PCI_HOT_RESET while it holds
memory_lock*/Quoted 17 lines
include/linux/vfio_pci_core.h +2
-0+struct task_struct *hot_reset_task; +int reset_done_err;struct list_head dmabufs;/* * Excluded BAR sub-ranges, one list per BAR, sorted by start and not @@ -207,6 +225,10 @@ void
vfio_pci_core_release_dev(struct vfio_device *core_vdev); intvfio_pci_core_register_device(struct vfio_pci_core_device *vdev); voidvfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev); extern const structpci_error_handlersvfio_pci_core_err_handlers;include/linux/vfio_pci_core.h +4
-0+void vfio_pci_core_reset_prepare (struct vfio_pci_core_device *vdev); +void vfio_pci_core_reset_done (struct vfio_pci_core_device *vdev); +void vfio_pci_core_error_detected (struct vfio_pci_core_device *vdev); +void vfio_pci_core_error_resume (struct vfio_pci_core_device *vdev);int vfio_pci_core_sriov_configure (struct vfio_pci_core_device *vdev,int nr_virtfn);long vfio_pci_core_ioctl(struct
vfio_device*core_vdev, unsigned int cmd,
RE: [PATCH v6 13/31] vfio/pci: Bracket PCI resets with the CXL reset hooks
Quoted 297 lines
-----Original Message-----
From: Manish Honap <[email protected]>
Sent: Thursday, November 19, 2026 12:54 AM
To: Alex Williamson <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; Manish Honap <[email protected]>
Subject: RE: [PATCH v6 13/31] vfio/pci: Bracket PCI resets with the CXL reset hooksQuoted 68 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Saturday, November 7, 2026 5:14 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 13/31] vfio/pci: Bracket PCI resets with the CXL reset hooksExternal email: Use caution opening links or attachments
Quoted 21 lines · On Fri, 16 Oct 2026 21:48:20 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A CXL Type-2 device must quiesce its coherent link before any reset and restore its HDM decoder state afterward, or a reset can leave the device mastering the bus over decoders that are no longer valid. This has to happen around every reset the PCI core drives for the device, not only the ones VFIO issues directly, and around AER and DPC recovery, which can reset the link without the reset callbacks.
Add:
-reset_prepare
-reset_done
- an optionalerror_resumeto the CXL opsExport:
-vfio_pci_core_reset_prepare()
-vfio_pci_core_reset_done()
-vfio_pci_core_error_detected()
-vfio_pci_core_error_resume()
which take the vfio device.Quoted 5 lines
vfio-pci calls them from its own
pci_error_handlers, which resolve the driver data it owns. A variant driver that binds CXL devices calls them from its own handlers the same way; a driver withoutreset_prepare,reset_done,error_detectedand resume callbacks gets no CXLsupport.
Quoted 12 lines
The provider hooks run with
memory_lockheld for write:
vfio-pci-core already holds it across its own hot reset, andvfio_pci_core_reset_prepare()takes it for a reset started anywhere else untilvfio_pci_core_reset_done(). The hot reset records its task, so a reset that another task starts meanwhile waits formemory_lockinstead of skipping it.pci_dev_lock()blocks user config access, which aPCI_COMMANDwrite can be waiting for while it holdsmemory_lock, soreset_preparelets config access through while it waits.reset_donereturns an error when the provider could not restore its device, andVFIO_DEVICE_PCI_HOT_RESETreturns that error; for any other reset the provider logs it.Quoted 4 lines
Cryptic. Seems like we're skipping updating all the variant drivers to make use of the new reset and error hooks, which seems wrong.
This also needs to be rebased to the the new drvdata handling in v7.4-rc.
Okay, I will handle both the flagged items. I will check the new drvdata handling.
Quoted 3 lines
I'm also not sure why the "provider" terminology has snuck into this series with this level of prevalence. There's only one cxl ops "provider", it's
vfio-cxl.
Okay, for v7, I will mention this as vfio-cxl.
Quoted 80 lines
Quoted 6 lines
Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci.c | 39 +++++++- drivers/vfio/pci/vfio_pci_core.c | 154+++++++++++++++++++++++++++++++
Quoted 64 lines
include/linux/vfio_pci_core.h | 22 +++++ 3 files changed, 214 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/drivers/vfio/pci/vfio_pci.cindex 830369ff878d..286244059569 100644drivers/vfio/pci/vfio_pci.c +35
-1@@ -223,13 +223,50 @@ static const struct pci_device_id vfio_pci_table[] = {MODULE_DEVICE_TABLE (pci, vfio_pci_table); +static void vfio_pci_reset_prepare (struct pci_dev *pdev) { +struct vfio_pci_core_device *vdev = +dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_prepare (vdev); +} + +static void vfio_pci_reset_done (struct pci_dev *pdev) { +struct vfio_pci_core_device *vdev = +dev_get_drvdata (&pdev->dev); + +vfio_pci_core_reset_done (vdev); } + +static pci_ers_result_t vfio_pci_error_detected (struct pci_dev *pdev, +pci_channel_state_t +state) { +struct vfio_pci_core_device *vdev = +dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_detected (vdev); +return vfio_pci_core_aer_err_detected (pdev, state); } + +static void vfio_pci_error_resume (struct pci_dev *pdev) { +struct vfio_pci_core_device *vdev = +dev_get_drvdata (&pdev->dev); + +vfio_pci_core_error_resume (vdev); } + +static const struct pci_error_handlers vfio_pci_err_handlers = { + .error_detected = vfio_pci_error_detected, + .reset_prepare = vfio_pci_reset_prepare, + .reset_done = vfio_pci_reset_done, + .resume = vfio_pci_error_resume, }; +static struct pci_driver vfio_pci_driver = { .name ="vfio-pci" , .id_table = vfio_pci_table, .probe = vfio_pci_probe, .remove = vfio_pci_remove, .sriov_configure = vfio_pci_sriov_configure, - .err_handler = &vfio_pci_core_err_handlers, + .err_handler = &vfio_pci_err_handlers, .driver_managed_dma = true, };drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -2462,6 +2462,7 @@ static bool vfio_pci_is_cxl_type2(structQuoted 1 line
pci_dev*pdev)
Quoted 7 lines
*/
static int vfio_pci_core_cxl_init(structvfio_pci_core_device
*vdev) {+ const struct pci_error_handlers *err_handler; const struct vfio_cxl_ops *ops; int ret;Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct
Quoted 1 line
drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -2473,6 +2474,23 @@ static int vfio_pci_core_cxl_init(struct
vfio_pci_core_device*vdev)Quoted 8 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0if (!ops)return IS_BUILTIN (CONFIG_VFIO_CXL ) ? -EPROBE_DEFER :0 ; +/* +* The provider must see every PCI core reset of the device and every +* AER or DPC recovery. A driver without reset_prepare, reset_done, +* error_detected and resume callbacks gets no CXL support; the driver's
Quoted 11 lines
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0+* callbacks must call vfio_pci_core_reset_prepare(), _reset_done(), +* _error_detected() and _error_resume(). +*/ + err_handler = vdev->pdev->driver ? vdev->pdev->driver->err_handler :NULL;
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
Quoted 6 lines
drivers/vfio/pci/vfio_pci_core.c +6
-0+if (!err_handler || !err_handler->reset_prepare || + !err_handler->reset_done || !err_handler->error_detected || + !err_handler->resume) { +pci_info (vdev->pdev, + "driver lacks PCI reset or error recovery + hooks, continuingQuoted 8 lines
without CXL support\n");
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev) · line 2493
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0+vfio_pci_put_cxl_ops (ops); +return 0 ; + } +Just implement them in all the variant drivers and make them validated in
vfio_pci_core_register_device().Okay, I will check this part.
Quoted 63 lines
Quoted 63 lines
drivers/vfio/pci/vfio_pci_core.c +0
-0ret = ops->init (vdev);/* * A provider that is not ready yet (for example its CXL port has not @@ -2759,6 +2777,128 @@ int vfio_pci_core_sriov_configure(struct
vfio_pci_core_device*vdev, }EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure);drivers/vfio/pci/vfio_pci_core.c +55
-0+/* +* Take memory_lock for a reset that the PCI core runs on a locked device. +* pci_dev_lock() also blocks user config access, and +vfio_basic_config_write() +* holds memory_lock while its pci_user_*_config() calls wait for +that block +* to clear. Waiting for memory_lock with config access blocked +would hang +* both tasks, so let user config access through while waiting: the +PCI core +* calls reset_prepare before it saves or touches the device. The +caller holds +* the device lock, so a block found set is the caller's own. +*/ +static void vfio_pci_reset_lock_memory (struct vfio_pci_core_device +*vdev) { +struct pci_dev *pdev = vdev->pdev; + +if (down_write_trylock (&vdev->memory_lock)) +return ; + +/* The caller did not block config access: nothing to release. */ +if (pci_cfg_access_trylock (pdev)) { +pci_cfg_access_unlock (pdev); +down_write (&vdev->memory_lock); +return ; + } + +pci_cfg_access_unlock (pdev); +down_write (&vdev->memory_lock); +pci_cfg_access_lock (pdev); +} + +/** +* vfio_pci_core_reset_prepare() - prepare a device for a PCI core +reset +* @vdev: vfio PCI device about to be reset +* +* Call from the driver's pci_error_handlers.reset_prepare callback. +*/ +void vfio_pci_core_reset_prepare (struct vfio_pci_core_device *vdev) { +if (!vdev->cxl_ops) +return ; + +/* +* vfio-pci-core holds memory_lock across its own hot reset. Hold it +* across a reset started anywhere else too, until reset_done. Test +* the owner, not just a flag: another task can reset the device while +* a hot reset holds memory_lock and has not yet locked the device. +*/ +if (READ_ONCE (vdev->hot_reset_task) != current) +vfio_pci_reset_lock_memory (vdev); + + vdev->cxl_ops->reset_prepare (vdev); +}Quoted 8 lines
Quoted 1 line
+EXPORT_SYMBOL_GPL(vfio_pci_core_reset_prepare);
This is all super sketchy. Sashiko triggers multiple critical issues. Having different locking behavior across CXL vs non-CXL device is not acceptable. These callbacks occur even if the device is reset through sysfs or AER while the device is not opened. Inferring whether the lock is held or not is very hacky. Thanks,
Okay, I will recheck this part. For my review I was only using review-prompts kernel md file instructions for my LLM review. I will setup local sashiko and verify this with complete runs.
Quoted 7 lines
What vfio-cxl needs from these callbacks is to keep the device memory unmapped from
reset_prepareuntil the device decodes it again.memory_lockwas the convenient lock for that, not a requirement: CXL memory decode does not depend on the Memory Space bit. So, in v7 I will serialize vfio-cxl memory region with its own lock and take it in its hooks. vfio-pci-core will not take lock in the hooks, so its locking is the same for every device, andhot_reset_taskandreset_done_errwill go away.
After giving this more thought, I feel vfio-cxl lock in these hooks will still need ordering rules of its own: it must never be held across the PCI reset calls or a user config access, which wait on locks lockdep does not track.
So, instead of above locking rules, I will follow the nvgrace-gpu approach in v7:
- reset_prepare, reset_done and error_detected only mark the device memory unavailable and count the reset, with no lock.
- New faults, reads and writes fail until a CXL Reset through vfio succeeds.
- The resets vfio starts still revoke the mmap and the dma-bufs under memory_lock. (Mappings that exist when a non-vfio initiated reset happens, are not zapped)
A plain sysfs reset of an open device resets nothing: vfio-cxl holds the HPA range while the device is open, so the cxl_reset method returns -EBUSY before it touches the device.
vfio-pci-core still takes no lock in the hooks, and hot_reset_task and reset_done_err are gone as noted earlier.
Thanks,
Manish
Quoted 177 lines
Thanks,
ManishQuoted 138 lines
Alex
Quoted 71 lines
drivers/vfio/pci/vfio_pci_core.c +71
-0+ +/** +* vfio_pci_core_reset_done() - restore a device after a PCI core +reset +* @vdev: vfio PCI device that was reset +* +* Call from the driver's pci_error_handlers.reset_done callback. +*/ +void vfio_pci_core_reset_done (struct vfio_pci_core_device *vdev) { +int ret; + +if (!vdev->cxl_ops) +return ; + + ret = vdev->cxl_ops->reset_done (vdev); + +/* +* hot_reset_task cannot change since reset_prepare: setting it needs +* memory_lock, which this task or the hot reset task holds. +*/ +if (READ_ONCE (vdev->hot_reset_task) == current) { +/* VFIO_DEVICE_PCI_HOT_RESET returns the error. */ + vdev->reset_done_err = ret; +return ; + } + +/* +* Nobody waits for the result of a reset started elsewhere. A provider +* that failed has logged it and keeps its device memory out of use +* until a later reset restores the device. +*/ +up_write (&vdev->memory_lock); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_reset_done); + +/** +* vfio_pci_core_error_detected() - prepare a device for AER or DPC +recovery +* @vdev: vfio PCI device the recovery reports an error on +* +* Call from the driver's pci_error_handlers.error_detected callback. +The +* recovery may reset the device without reset_prepare and +reset_done, so a +* provider takes its device memory out of use here. +*/ +void vfio_pci_core_error_detected (struct vfio_pci_core_device +*vdev) { +if (!vdev->cxl_ops) +return ; + +guard (rwsem_write)(&vdev->memory_lock); + vdev->cxl_ops->reset_prepare (vdev); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_error_detected); + +/** +* vfio_pci_core_error_resume() - resume a device after AER or DPC +recovery +* @vdev: vfio PCI device the recovery resumes +* +* Call from the driver's pci_error_handlers.resume callback. A +provider puts +* its device memory back if the device still decodes it; otherwise +the memory +* stays out of use until the user resets the device. +*/ +void vfio_pci_core_error_resume (struct vfio_pci_core_device *vdev) { +if (!vdev->cxl_ops || !vdev->cxl_ops->error_resume) +return ; +drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +7
-1+guard (rwsem_write)(&vdev->memory_lock); + vdev->cxl_ops->error_resume (vdev); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_error_resume); +const struct pci_error_handlers vfio_pci_core_err_handlers = { .error_detected = vfio_pci_core_aer_err_detected, }; @@ -2915 ,6 +3055,8 @@ static int vfio_pci_dev_set_hot_reset(struct +vfio_device_set
*dev_set,drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_dev_set_hot_reset(struct
Quoted 28 lines
ret = -EBUSY; break; }drivers/vfio/pci/vfio_pci_core.c +2
-0+WRITE_ONCE (vdev->hot_reset_task, current); + vdev->reset_done_err =0 ;vfio_pci_dma_buf_move (vdev, true); vfio_pci_zap_bars(vdev); @@ -2940,6 +3082,17 @@ staticint vfio_pci_dev_set_hot_reset(struct
vfio_device_set*dev_set,ret = pci_reset_bus(pdev);drivers/vfio/pci/vfio_pci_core.c +11
-0+/* A provider that could not restore its device fails the hot reset. */ +if (!ret) { +list_for_each_entry (vdev, &dev_set->device_list, + vdev.dev_set_list) { +if (vdev->reset_done_err) { + ret = vdev->reset_done_err; +break ; + } + } + } + vdev = list_last_entry(&dev_set->device_list, struct vfio_pci_core_device,vdev.dev_set_list);
drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -2948,6 +3101,7 @@ static int vfio_pci_dev_set_hot_reset(struct
vfio_device_set*dev_set,Quoted 24 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0vdev.dev_set_list) {if (vdev->vdev.open_count &&__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false); +WRITE_ONCE (vdev->hot_reset_task,NULL );up_write (&vdev->memory_lock); }(quoted diff) +0
-0
b/include/linux/vfio_pci_core.hindex6520a2dcfd4a..881e51c42414100644include/linux/vfio_pci_core.h +8
-0@@ -72,6 +72,21 @@ struct vfio_cxl_ops {void (*release)(struct vfio_pci_core_device *vdev);int (*open_device)(struct vfio_pci_core_device *vdev);void (*close_device)(struct vfio_pci_core_device *vdev); +/* +* Optional. Called with memory_lock held for write when AER or DPC +* recovery resumes the device, after reset_prepare was called for the +* error. Config space is as the recovery left it. +*/ +void (*error_resume)(struct vfio_pci_core_device *vdev); +/* +* Called around a PCI core reset of the device with memory_lock held Quoted 8 lines
include/linux/vfio_pci_core.h struct vfio_cxl_ops {
Quoted 8 lines
include/linux/vfio_pci_core.h +7
-0+* for write; reset_prepare also when AER or DPC recovery reports an +* error. reset_done returns an error if the provider could not +* restore the device; VFIO_DEVICE_PCI_HOT_RESET returns it, and for +* any other reset the provider logs it. +*/ +void (*reset_prepare)(struct vfio_pci_core_device *vdev); +int (*reset_done)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */
Quoted 26 lines
include/linux/vfio_pci_core.h struct vfio_pci_core_device { · line 184
Quoted 7 lines
include/linux/vfio_pci_core.h +1
-0struct module *owner; };@@ -166,6 +181,9 @@ struct vfio_pci_core_device {struct vfio_pci_core_device *sriov_pf_core_dev;struct notifier_block nb;struct rw_semaphore memory_lock; +/* Task running VFIO_DEVICE_PCI_HOT_RESET while it holds
memory_lock*/Quoted 18 lines
include/linux/vfio_pci_core.h +2
-0+struct task_struct *hot_reset_task; +int reset_done_err;struct list_head dmabufs;/* * Excluded BAR sub-ranges, one list per BAR, sorted by start and not @@ -207,6 +225,10 @@ void
vfio_pci_core_release_dev(struct vfio_device *core_vdev); intvfio_pci_core_register_device(struct vfio_pci_core_device *vdev); voidvfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev); extern const structpci_error_handlersvfio_pci_core_err_handlers;include/linux/vfio_pci_core.h +5
-0+void vfio_pci_core_reset_prepare (struct vfio_pci_core_device +*vdev);void vfio_pci_core_reset_done (struct vfio_pci_core_device +*vdev);void vfio_pci_core_error_detected (struct +vfio_pci_core_device *vdev);void vfio_pci_core_error_resume (struct +vfio_pci_core_device *vdev);int vfio_pci_core_sriov_configure (struct vfio_pci_core_device *vdev,int nr_virtfn);long vfio_pci_core_ioctl(struct
vfio_device*core_vdev, unsigned int cmd,
PATCH v6 14/31 vfio/pci: Provide an opt-out for the CXL Type-2 extensions
In reply to [email protected]From: Manish Honap <[email protected]>
Let an admin or a variant driver keep the CXL Type-2 extensions out of the path and bind the device without them.
Add a vfio_pci.disable_cxl module parameter, and let a variant driver set vdev->disable_cxl in its own init callback for per-device control. The parameter is built only when the CXL provider is configured.
--- drivers/vfio/pci/vfio_pci.c | 13 +++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 2 +- include/linux/vfio_pci_core.h | 1 + 3 files changed, 15 insertions(+), 1 deletion(-)
drivers/vfio/pci/vfio_pci.c +13 -0
@@ -60,6 +60,16 @@ static bool disable_denylist;module_param (disable_denylist, bool,0444 );MODULE_PARM_DESC (disable_denylist,"Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users." ); +#if IS_ENABLED (CONFIG_VFIO_CXL ) +/* +* A variant driver may instead set vdev->disable_cxl in its own init callback +* for per-device control without this module-wide parameter. +*/ +static bool disable_cxl; +module_param (disable_cxl, bool,0644 ); +MODULE_PARM_DESC (disable_cxl,"Disable CXL Type-2 extensions for all devices bound to vfio-pci." ); +#endif +static bool vfio_pci_dev_in_denylist (struct pci_dev *pdev) {switch (pdev->vendor) {
@@ -142,6 +152,9 @@ static int vfio_pci_init_dev(struct vfio_device *core_vdev)#ifdef CONFIG_VFIO_PCI_VGA vdev->disable_vga = disable_vga;#endif +#if IS_ENABLED (CONFIG_VFIO_CXL ) + vdev->disable_cxl = disable_cxl; +#endif return vfio_pci_core_init_dev (core_vdev); }
drivers/vfio/pci/vfio_pci_core.c +1 -1
@@ -2466,7 +2466,7 @@ static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)const struct vfio_cxl_ops *ops;int ret; -if (!vfio_pci_is_cxl_type2 (vdev->pdev)) +if (vdev->disable_cxl || !vfio_pci_is_cxl_type2 (vdev->pdev))return 0 ;request_module ("vfio-cxl" );
include/linux/vfio_pci_core.h +1 -0
@@ -162,6 +162,7 @@ struct vfio_pci_core_device { bool disable_idle_d3:1; bool nointxmask:1; bool disable_vga:1; + bool disable_cxl:1;/* Flags modified at runtime - dedicated storage unit */ bool needs_reset;bool pm_intx_masked;
Signature · 1 line
2.53.0
PATCH v6 15/31 vfio/pci: Route every reset point through a try-reset helper
In reply to [email protected]From: Manish Honap <[email protected]>
A provider such as vfio-cxl must reset its device itself: a CXL Type-2 function must not take an FLR, and its reset changes what the provider lets the fault path map. vfio-pci-core resets a function at five points:
enable, close, VFIO_DEVICE_RESET, a virtualized PCIe or AF FLR, and the guest's CXL Reset doorbell in the virtualized CXL DVSEC.
Add a mandatory reset op to the provider ops, and refuse to register ops without it. Add vfio_pci_try_reset_function(), which keeps the trylock of pci_try_reset_function(): it returns -EAGAIN when the device lock is contended, and otherwise runs the provider's reset with the device lock and memory_lock held. The close path keeps its bridge and device trylock and calls a locked variant. vfio_pci_try_reset_function() saves the PCI state first, as pci_try_reset_function() does, for the provider's reset to restore. The locked variant does not save: on close the reset must restore the open-time state that vfio_pci_core_disable() has just loaded. A failed save is returned and the reset is not attempted.
The enable path still fails only on -EAGAIN. A provider checks at bind that it can reset the device, so reset_works is set for a provider device whatever the result of the first reset.
The CXL Reset doorbell now runs the reset and reports its result in
Status2: Reset Error on failure, Reset Done on success.
--- drivers/vfio/pci/vfio_pci_config.c | 19 ++++++-- drivers/vfio/pci/vfio_pci_core.c | 71 ++++++++++++++++++++++++++++-- drivers/vfio/pci/vfio_pci_priv.h | 1 + include/linux/vfio_pci_core.h | 7 +++ 4 files changed, 90 insertions(+), 8 deletions(-)
drivers/vfio/pci/vfio_pci_config.c +15 -4
@@ -910,7 +910,7 @@ static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos,if (!ret && (cap &PCI_EXP_DEVCAP_FLR )) {vfio_pci_zap_and_down_write_memory_lock (vdev);vfio_pci_dma_buf_move (vdev, true); -pci_try_reset_function (vdev->pdev); +vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock);
@@ -995,7 +995,7 @@ static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos,if (!ret && (cap &PCI_AF_CAP_FLR ) && (cap &PCI_AF_CAP_TP )) {vfio_pci_zap_and_down_write_memory_lock (vdev);vfio_pci_dma_buf_move (vdev, true); -pci_try_reset_function (vdev->pdev); +vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock);
@@ -1786,9 +1786,20 @@ static int vfio_cxl_dvsec_write(struct vfio_pci_core_device *vdev, int pos,if (!(ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING )) status2 &= ~PCI_DVSEC_CXL_CACHE_INV ;if (ctrl2 &PCI_DVSEC_CXL_INIT_CXL_RST ) { +int ret; + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST ; - status2 &= ~PCI_DVSEC_CXL_RST_ERR ; - status2 |=PCI_DVSEC_CXL_RST_DONE ; + +/* Run the real reset here and report its result in Status2. */ +vfio_pci_zap_and_down_write_memory_lock (vdev); +vfio_pci_dma_buf_move (vdev, true); + ret =vfio_pci_try_reset_function (vdev); +if (__vfio_pci_memory_enabled (vdev)) +vfio_pci_dma_buf_move (vdev, false); +up_write (&vdev->memory_lock); + + status2 &= ~(PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR ); + status2 |= ret ?PCI_DVSEC_CXL_RST_ERR :PCI_DVSEC_CXL_RST_DONE ; } *pctrl2 =cpu_to_le16 (ctrl2);
drivers/vfio/pci/vfio_pci_core.c +67 -4
104 lines not on this page · raw message
drivers/vfio/pci/vfio_pci_priv.h +1 -0
@@ -80,6 +80,7 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t state);void vfio_pci_zap_and_down_write_memory_lock (struct vfio_pci_core_device *vdev); +int vfio_pci_try_reset_function (struct vfio_pci_core_device *vdev);u16 vfio_pci_memory_lock_and_enable (struct vfio_pci_core_device *vdev);void vfio_pci_memory_unlock_and_restore (struct vfio_pci_core_device *vdev,u16 cmd);
include/linux/vfio_pci_core.h +7 -0
@@ -87,6 +87,13 @@ struct vfio_cxl_ops {*/ void (*reset_prepare)(struct vfio_pci_core_device *vdev);int (*reset_done)(struct vfio_pci_core_device *vdev); +/* +* Reset the function in place of the PCI core; mandatory. Called with +* the PCI device lock and memory_lock (for write) held. The caller +* provides the saved PCI state the reset restores: the try-reset path +* saves it first, the close path keeps the open-time copy. +*/ +int (*reset)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
Signature · 1 line
2.53.0
Alex Williamson
maintainerRe: [PATCH v6 15/31] vfio/pci: Route every reset point through a try-reset helper
Quoted 7 lines · On Fri, 16 Oct 2026 21:48:22 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
A provider such as vfio-cxl must reset its device itself: a CXL Type-2 function must not take an FLR, and its reset changes what the provider lets the fault path map. vfio-pci-core resets a function at five points:
enable, close,VFIO_DEVICE_RESET, a virtualized PCIe or AF FLR, and the guest's CXL Reset doorbell in the virtualized CXL DVSEC.
What substantiates the claim that a type-2 device must not take an FLR?
Here we're promoting all resets to a CXL reset, but a CXL reset and an FLR have different scopes. FLR is exposed to the user through config space. A user might reasonably expect an FLR scoped reset through the PCIe/AF capabilities. The CXL spec defines FLR vs CXL reset, so this actually makes the device non CXL compliant through vfio-pci. This is a massive policy imposition.
What makes this necessary?
Quoted 117 lines
Add a mandatory reset op to the provider ops, and refuse to register ops without it. Add
vfio_pci_try_reset_function(), which keeps the trylock ofpci_try_reset_function(): it returns-EAGAINwhen the device lock is contended, and otherwise runs the provider's reset with the device lock andmemory_lockheld. The close path keeps its bridge and device trylock and calls a locked variant.vfio_pci_try_reset_function()saves the PCI state first, aspci_try_reset_function()does, for the provider's reset to restore. The locked variant does not save: on close the reset must restore the open-time state thatvfio_pci_core_disable()has just loaded. A failed save is returned and the reset is not attempted.The enable path still fails only on
-EAGAIN. A provider checks at bind that it can reset the device, soreset_worksis set for a provider device whatever the result of the first reset.The CXL Reset doorbell now runs the reset and reports its result in
Status2: Reset Error on failure, Reset Done on success.Suggested-by Alex Williamson <[email protected]>Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_config.c | 19 ++++++-- drivers/vfio/pci/vfio_pci_core.c | 71 ++++++++++++++++++++++++++++-- drivers/vfio/pci/vfio_pci_priv.h | 1 + include/linux/vfio_pci_core.h | 7 +++ 4 files changed, 90 insertions(+), 8 deletions(-)drivers/vfio/pci/vfio_pci_config.c +15
-4@@ -910,7 +910,7 @@ static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos,if (!ret && (cap &PCI_EXP_DEVCAP_FLR )) {vfio_pci_zap_and_down_write_memory_lock (vdev);vfio_pci_dma_buf_move (vdev, true); -pci_try_reset_function (vdev->pdev); +vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock);@@ -995,7 +995,7 @@ static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos,if (!ret && (cap &PCI_AF_CAP_FLR ) && (cap &PCI_AF_CAP_TP )) {vfio_pci_zap_and_down_write_memory_lock (vdev);vfio_pci_dma_buf_move (vdev, true); -pci_try_reset_function (vdev->pdev); +vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock);@@ -1786,9 +1786,20 @@ static int vfio_cxl_dvsec_write(struct vfio_pci_core_device *vdev, int pos,if (!(ctrl2 &PCI_DVSEC_CXL_DISABLE_CACHING )) status2 &= ~PCI_DVSEC_CXL_CACHE_INV ;if (ctrl2 &PCI_DVSEC_CXL_INIT_CXL_RST ) { +int ret; + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST ; - status2 &= ~PCI_DVSEC_CXL_RST_ERR ; - status2 |=PCI_DVSEC_CXL_RST_DONE ; + +/* Run the real reset here and report its result in Status2. */ +vfio_pci_zap_and_down_write_memory_lock (vdev); +vfio_pci_dma_buf_move (vdev, true); + ret =vfio_pci_try_reset_function (vdev); +if (__vfio_pci_memory_enabled (vdev)) +vfio_pci_dma_buf_move (vdev, false); +up_write (&vdev->memory_lock); + + status2 &= ~(PCI_DVSEC_CXL_RST_DONE |PCI_DVSEC_CXL_RST_ERR ); + status2 |= ret ?PCI_DVSEC_CXL_RST_ERR :PCI_DVSEC_CXL_RST_DONE ; } *pctrl2 =cpu_to_le16 (ctrl2);drivers/vfio/pci/vfio_pci_core.c +36
-0@@ -605,6 +605,62 @@ static void vfio_pci_core_unmap_bars(struct vfio_pci_core_device *vdev) } } +/** +* vfio_pci_try_reset_function() - reset the function without blocking +* @vdev: vfio PCI device, with memory_lock held for write +* +* Reset through the PCI core, or let the provider reset a device it serves. +* The provider resets the device with its PCI device lock held, so take that +* lock here, without waiting, as pci_try_reset_function() does. +* +* Return: 0 on success, -EAGAIN when the device lock is contended, or the +* reset error. For a provider device a failed pci_save_state() is also +* returned, and the reset is not attempted: unlike pci_try_reset_function(), +* the provider's reset needs a valid saved state to restore. +*/ +int vfio_pci_try_reset_function (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; +int ret; + +lockdep_assert_held_write (&vdev->memory_lock); + +if (!vdev->cxl_ops) +return pci_try_reset_function (pdev); + +if (!pci_dev_trylock (pdev)) +return -EAGAIN ; + +/* +* The provider restores PCI state from the saved copy, so save the +* current state first, as pci_try_reset_function() does. The locked +* variant on the close path must not: it resets with the state that +* vfio_pci_core_disable() has just loaded. +*/ + ret =pci_save_state (pdev); +if (!ret) + ret = vdev->cxl_ops->reset (vdev); +pci_dev_unlock (pdev);
drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_core_unmap_bars(struct vfio_pci_core_device *vdev) · line 653
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+ +return ret; +} + +/* +* As vfio_pci_try_reset_function(), for a caller that holds the device lock. +* Only the close path calls this. It takes memory_lock with the device lock +* held, so user config access is blocked meanwhile: the order a sysfs reset +* uses. That cannot deadlock against a PCI_COMMAND write holding memory_lock +* because no user remains at close.
Please tell the AI not to add any comments, these are difficult to parse and often unnecessary. If there are specific locking requirements for this function, they should be annotated. If this function cannot be annotated, and it not general, even within a static context, maybe it shouldn't be a separate function.
Quoted 17 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+*/ +static int vfio_pci_reset_function_locked (struct vfio_pci_core_device *vdev) +{ +if (!vdev->cxl_ops) +return __pci_reset_function_locked (vdev->pdev); + +guard (rwsem_write)(&vdev->memory_lock); +return vdev->cxl_ops->reset (vdev); +} +static void vfio_pci_core_free_dev_regions (struct vfio_pci_core_device *vdev) {for (int i =0 ; i < vdev->num_regions; i++)@@ -636,11 +692,14 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) goto out_power;/* If reset fails because of the device lock, fail this path entirely */
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 702
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +5
-2- ret =pci_try_reset_function (pdev); +down_write (&vdev->memory_lock); + ret =vfio_pci_try_reset_function (vdev); +up_write (&vdev->memory_lock);if (ret == -EAGAIN ) goto out_disable_device; - vdev->reset_works = !ret; +/* A provider checks at bind that it can reset the device. */ + vdev->reset_works = vdev->cxl_ops || !ret;
I suppose, but why is ret not still a valid indicator?
Quoted 22 lines
drivers/vfio/pci/vfio_pci_core.c +2
-2pci_save_state (pdev); vdev->pci_saved_state =pci_store_saved_state (pdev);if (!vdev->pci_saved_state)@@ -846,7 +905,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)if (bridge && !pci_dev_trylock (bridge)) goto out_restore_state;if (pci_dev_trylock (pdev)) { -if (!__pci_reset_function_locked (pdev)) +if (!vfio_pci_reset_function_locked (vdev)) vdev->needs_reset = false;pci_dev_unlock (pdev); }@@ -1541,7 +1600,7 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,vfio_pci_set_power_state (vdev,PCI_D0 );vfio_pci_dma_buf_move (vdev, true); - ret =pci_try_reset_function (vdev->pdev); + ret =vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock);
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_register_cxl_ops(const struct vfio_cxl_ops *ops) · line 2459
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0@@ -2397,6 +2456,10 @@ int vfio_pci_core_register_cxl_ops(const struct vfio_cxl_ops *ops) {guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem); +/* vfio-pci-core never resets a provider's device by itself. */
What a bizarre comment. Relevance?
Quoted 23 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0+if (!ops->reset) +return -EINVAL ; +if (vfio_pci_cxl_ops)return -EBUSY ;drivers/vfio/pci/vfio_pci_priv.h +1
-0@@ -80,6 +80,7 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t state);void vfio_pci_zap_and_down_write_memory_lock (struct vfio_pci_core_device *vdev); +int vfio_pci_try_reset_function (struct vfio_pci_core_device *vdev);u16 vfio_pci_memory_lock_and_enable (struct vfio_pci_core_device *vdev);void vfio_pci_memory_unlock_and_restore (struct vfio_pci_core_device *vdev,u16 cmd);include/linux/vfio_pci_core.h +0
-0
include/linux/vfio_pci_core.h struct vfio_cxl_ops { · line 95
Quoted 9 lines
include/linux/vfio_pci_core.h +6
-0@@ -87,6 +87,13 @@ struct vfio_cxl_ops {*/ void (*reset_prepare)(struct vfio_pci_core_device *vdev);int (*reset_done)(struct vfio_pci_core_device *vdev); +/* +* Reset the function in place of the PCI core; mandatory. Called with +* the PCI device lock and memory_lock (for write) held. The caller +* provides the saved PCI state the reset restores: the try-reset path +* saves it first, the close path keeps the open-time copy. +*/
Reads like more AI generated comments. These things like to talk, please curb them. Look at vfio_device_ops for an example how to briefly and concisely document callback ops like a human. Thanks,
Alex
PS - I'm still working on the remainder of the series.
Quoted 4 lines
include/linux/vfio_pci_core.h +1
-0+int (*reset)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
RE: [PATCH v6 15/31] vfio/pci: Route every reset point through a try-reset helper
Quoted 30 lines
-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Thursday, November 5, 2026 5:23 AM
To: Manish Honap <[email protected]>
Cc: [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 15/31] vfio/pci: Route every reset point through a try- reset helperExternal email: Use caution opening links or attachments
Quoted 2 lines · On Fri, 16 Oct 2026 21:48:22 +0530 <[email protected]> wrote:
From: Manish Honap <[email protected]>
Quoted 7 lines
Quoted 5 lines
A provider such as vfio-cxl must reset its device itself: a CXL Type-2 function must not take an FLR, and its reset changes what the provider lets the fault path map. vfio-pci-core resets a function at five points:
enable, close,VFIO_DEVICE_RESET, a virtualized PCIe or AF FLR, and the guest's CXL Reset doorbell in the virtualized CXL DVSEC.What substantiates the claim that a type-2 device must not take an FLR?
No, this is a wrong claim in this patch.
This statement is wrong and I will drop it in v7.
As per cxl spec 4.0, FLR only affects CXL.io and does not affect the CXL.mem, CXL.cache, CXL DVSEC state or the memory controller behind the HDM.
Quoted 8 lines
Here we're promoting all resets to a CXL reset, but a CXL reset and an FLR have different scopes. FLR is exposed to the user through config space. A user might reasonably expect an FLR scoped reset through the PCIe/AF capabilities. The CXL spec defines FLR vs CXL reset, so this actually makes the device non CXL compliant through vfio-pci. This is a massive policy imposition.
What makes this necessary?
Agreed that this is changing the already established policy in VFIO. For v7, I will address the reset as per scope the spec has defined.
This is my plan:
- A PCIe or AF FLR written to config space is a PCIe FLR in vfio side. For a CXL device, vfio-cxl will run pcie_reset_flr inside the same save, disable and restore steps in pci_reset_function.
- Initiate_CXL_Reset written to the CXL DVSEC from guest will be a CXL reset routed through pci_try_reset_function(). cxl_reset series orders cxl_reset ahead of FLR so that method will be run here. cxl_reset series also performs all cxl_reset's with a mem_clr enable so we will also have the same effect in this case and guest's own Mem Clr Enable will be dropped and not passed on to VFIO.
- For open/close/Reset ioctl, I will call the same PCI core calls done today. For a CXL type-2 device, those will be cxl_reset with Mem Clr Enable.
The cxl_ops reset op, this patch and the owner-aware entry in patch 04 will go away. vfio-cxl keeps only the work around the resets vfio
starts: it revokes its memory mappings and dma-bufs before the reset and reopens the memory after the device reports Memory_Active again.
vfio-cxl also holds the HPA range exclusively while the device is open, and the cxl_reset method refuses to reset a range that someone else holds. So, for v7, I will release the vfio-cxl's claim on the range and release it around the resets vfio starts, after its mappings are revoked. The claim will be taken back before the memory reopens. A sysfs reset of an open device gets -EBUSY from the cxl_reset method.
Quoted 141 lines
Quoted 11 lines
Add a mandatory reset op to the provider ops, and refuse to register ops without it. Add
vfio_pci_try_reset_function(), which keeps the trylock ofpci_try_reset_function(): it returns-EAGAINwhen the device lock is contended, and otherwise runs the provider's reset with the device lock andmemory_lockheld. The close path keeps its bridge and device trylock and calls a locked variant.vfio_pci_try_reset_function()saves the PCI state first, aspci_try_reset_function()does, for the provider's reset to restore. The locked variant does not save: on close the reset must restore the open-time state thatvfio_pci_core_disable()has just loaded. A failed save isreturned and the reset is not attempted.
drivers/vfio/pci/vfio_pci_config.c static int vfio_exp_config_write(struct
Quoted 24 lines
The enable path still fails only on
-EAGAIN. A provider checks at bind that it can reset the device, soreset_worksis set for a provider device whatever the result of the first reset.The CXL Reset doorbell now runs the reset and reports its result in
Status2: Reset Error on failure, Reset Done on success.Suggested-by Alex Williamson <[email protected]>Assisted-by LLMSigned-off-by Manish Honap <[email protected]>--- drivers/vfio/pci/vfio_pci_config.c | 19 ++++++-- drivers/vfio/pci/vfio_pci_core.c | 71 ++++++++++++++++++++++++++++-- drivers/vfio/pci/vfio_pci_priv.h | 1 + include/linux/vfio_pci_core.h | 7 +++ 4 files changed, 90 insertions(+), 8 deletions(-)drivers/vfio/pci/vfio_pci_config.c +0
-0@@ -910,7 +910,7 @@ static int vfio_exp_config_write(struct
vfio_pci_core_device*vdev, int pos,Quoted 9 lines
drivers/vfio/pci/vfio_pci_config.c +1
-1if (!ret && (cap &PCI_EXP_DEVCAP_FLR )) {vfio_pci_zap_and_down_write_memory_lock (vdev);vfio_pci_dma_buf_move (vdev, true); -pci_try_reset_function (vdev->pdev); +vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false); up_write(&vdev->memory_lock); @@ -995,7 +995,7@@ static int vfio_af_config_write(struct
vfio_pci_core_device*vdev, intpos,
drivers/vfio/pci/vfio_pci_config.c static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos,
Quoted 9 lines
if (!ret && (cap & PCI_AF_CAP_FLR) && (cap & PCI_AF_CAP_TP)) { vfio_pci_zap_and_down_write_memory_lock(vdev); vfio_pci_dma_buf_move(vdev, true);drivers/vfio/pci/vfio_pci_config.c +2
-1-pci_try_reset_function (vdev->pdev); +vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false); up_write(&vdev->memory_lock); @@ -1786,9 +1786,20 @@ static int vfio_cxl_dvsec_write(struct vfio_pci_core_device*vdev, int pos,
Quoted 8 lines
if (!(ctrl2 & PCI_DVSEC_CXL_DISABLE_CACHING)) status2 &= ~PCI_DVSEC_CXL_CACHE_INV; if (ctrl2 & PCI_DVSEC_CXL_INIT_CXL_RST) {drivers/vfio/pci/vfio_pci_config.c +2
-2+int ret; + ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST ; - status2 &= ~PCI_DVSEC_CXL_RST_ERR ; - status2 |=PCI_DVSEC_CXL_RST_DONE ;drivers/vfio/pci/vfio_pci_config.c static int vfio_cxl_dvsec_write(struct vfio_pci_core_device *vdev, int pos,
Quoted 10 lines
drivers/vfio/pci/vfio_pci_config.c +10
-0+ +/* Run the real reset here and report its result in Status2. */ +vfio_pci_zap_and_down_write_memory_lock (vdev); +vfio_pci_dma_buf_move (vdev, true); + ret =vfio_pci_try_reset_function (vdev); +if (__vfio_pci_memory_enabled (vdev)) +vfio_pci_dma_buf_move (vdev, false); +up_write (&vdev->memory_lock); + + status2 &= ~(PCI_DVSEC_CXL_RST_DONE |
PCI_DVSEC_CXL_RST_ERR);drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_core_unmap_bars(struct
Quoted 11 lines
drivers/vfio/pci/vfio_pci_config.c +2
-0+ status2 |= ret ?PCI_DVSEC_CXL_RST_ERR : +PCI_DVSEC_CXL_RST_DONE ; } *pctrl2 =cpu_to_le16 (ctrl2);drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -605,6 +605,62 @@ static void vfio_pci_core_unmap_bars(struct
vfio_pci_core_device*vdev)Quoted 53 lines
drivers/vfio/pci/vfio_pci_core.c +50
-0} } +/** +* vfio_pci_try_reset_function() - reset the function without +blocking +* @vdev: vfio PCI device, with memory_lock held for write +* +* Reset through the PCI core, or let the provider reset a device it serves. +* The provider resets the device with its PCI device lock held, so +take that +* lock here, without waiting, as pci_try_reset_function() does. +* +* Return: 0 on success, -EAGAIN when the device lock is contended, +or the +* reset error. For a provider device a failed pci_save_state() is +also +* returned, and the reset is not attempted: unlike +pci_try_reset_function(), +* the provider's reset needs a valid saved state to restore. +*/ +int vfio_pci_try_reset_function (struct vfio_pci_core_device *vdev) { +struct pci_dev *pdev = vdev->pdev; +int ret; + +lockdep_assert_held_write (&vdev->memory_lock); + +if (!vdev->cxl_ops) +return pci_try_reset_function (pdev); + +if (!pci_dev_trylock (pdev)) +return -EAGAIN ; + +/* +* The provider restores PCI state from the saved copy, so save the +* current state first, as pci_try_reset_function() does. The locked +* variant on the close path must not: it resets with the state that +* vfio_pci_core_disable() has just loaded. +*/ + ret =pci_save_state (pdev); +if (!ret) + ret = vdev->cxl_ops->reset (vdev); +pci_dev_unlock (pdev); + +return ret; +} + +/* +* As vfio_pci_try_reset_function(), for a caller that holds the device lock. +* Only the close path calls this. It takes memory_lock with the +device lock +* held, so user config access is blocked meanwhile: the order a +sysfs reset
Quoted 9 lines
drivers/vfio/pci/vfio_pci_core.c static void vfio_pci_core_unmap_bars(struct · line 660
Quoted 3 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0+ * uses. That cannot deadlock against a PCI_COMMAND write holding +memory_lock + * because no user remains at close.Please tell the AI not to add any comments, these are difficult to parse and often unnecessary. If there are specific locking requirements for this function, they should be annotated. If this function cannot be annotated, and it not general, even within a static context, maybe it shouldn't be a separate function.
Understood. I will refactor this part. For v7, this function will be removed and close path will keep calling the __pci_reset_function_locked as it does today.
Quoted 23 lines
Quoted 23 lines
drivers/vfio/pci/vfio_pci_core.c +9
-0+*/ +static int vfio_pci_reset_function_locked (struct vfio_pci_core_device +*vdev) { +if (!vdev->cxl_ops) +return __pci_reset_function_locked (vdev->pdev); + +guard (rwsem_write)(&vdev->memory_lock); +return vdev->cxl_ops->reset (vdev); } + static void vfio_pci_core_free_dev_regions(structvfio_pci_core_device *vdev) { for (int i = 0; i < vdev->num_regions; i++) @@ -636,11 +692,14 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) goto out_power; /* If reset fails because of the device lock, fail this path entirely */drivers/vfio/pci/vfio_pci_core.c +3
-1- ret =pci_try_reset_function (pdev); +down_write (&vdev->memory_lock); + ret =vfio_pci_try_reset_function (vdev); +up_write (&vdev->memory_lock);if (ret == -EAGAIN ) goto out_disable_device;
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 702
Quoted 3 lines
drivers/vfio/pci/vfio_pci_core.c +2
-1- vdev->reset_works = !ret; +/* A provider checks at bind that it can reset the device. */ + vdev->reset_works = vdev->cxl_ops || !ret;I suppose, but why is ret not still a valid indicator?
Hmm, yes, ret is the valid indicator. I will remove the function so this part will be moved to the earlier version.
Quoted 28 lines
drivers/vfio/pci/vfio_pci_core.c void vfio_pci_core_disable(struct
Quoted 5 lines
pci_save_state(pdev); vdev->pci_saved_state = pci_store_saved_state(pdev); if (!vdev->pci_saved_state)drivers/vfio/pci/vfio_pci_core.c +0
-0@@ -846,7 +905,7 @@ void vfio_pci_core_disable(struct
vfio_pci_core_device*vdev)drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_ioctl_reset(struct
Quoted 9 lines
drivers/vfio/pci/vfio_pci_core.c +1
-1if (bridge && !pci_dev_trylock (bridge)) goto out_restore_state;if (pci_dev_trylock (pdev)) { -if (!__pci_reset_function_locked (pdev)) +if (!vfio_pci_reset_function_locked (vdev)) vdev->needs_reset = false;pci_dev_unlock (pdev); }@@ -1541,7 +1600,7 @@ static int vfio_pci_ioctl_reset(struct
vfio_pci_core_device*vdev,Quoted 12 lines
drivers/vfio/pci/vfio_pci_core.c +1
-1vfio_pci_set_power_state (vdev,PCI_D0 );vfio_pci_dma_buf_move (vdev, true); - ret =pci_try_reset_function (vdev->pdev); + ret =vfio_pci_try_reset_function (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock);@@ -2397,6 +2456,10 @@ int vfio_pci_core_register_cxl_ops(const struct vfio_cxl_ops *ops) {guard (rwsem_write)(&vfio_pci_cxl_ops_rwsem);
Quoted 3 lines
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_register_cxl_ops(const struct vfio_cxl_ops *ops) · line 2459
Quoted 1 line
drivers/vfio/pci/vfio_pci_core.c +1
-0+/* vfio-pci-core never resets a provider's device by itself. */ What a bizarre comment. Relevance?
I think this was edited in partial and left out during merge conflict handling. I will remove this.
Quoted 33 lines
drivers/vfio/pci/vfio_pci_priv.h int vfio_pci_set_power_state(struct
Quoted 13 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0+if (!ops->reset) +return -EINVAL ; +if (vfio_pci_cxl_ops)return -EBUSY ;drivers/vfio/pci/vfio_pci_priv.h +0
-0@@ -80,6 +80,7 @@ int vfio_pci_set_power_state(struct
vfio_pci_core_device*vdev,drivers/vfio/pci/vfio_pci_priv.h int vfio_pci_set_power_state(struct · line 86
Quoted 7 lines
drivers/vfio/pci/vfio_pci_priv.h +1
-0pci_power_t state);void vfio_pci_zap_and_down_write_memory_lock (struct vfio_pci_core_device *vdev); +int vfio_pci_try_reset_function (struct vfio_pci_core_device *vdev);u16 vfio_pci_memory_lock_and_enable (struct vfio_pci_core_device *vdev); void vfio_pci_memory_unlock_and_restore(struct
vfio_pci_core_device*vdev,Quoted 11 lines
u16 cmd); diff --git
a/include/linux/vfio_pci_core.hb/include/linux/vfio_pci_core.hindex991a3fcd84eb..cb9ba92b696a100644include/linux/vfio_pci_core.h +2
-0@@ -87,6 +87,13 @@ struct vfio_cxl_ops {*/ void (*reset_prepare)(struct vfio_pci_core_device *vdev);int (*reset_done)(struct vfio_pci_core_device *vdev); +/* +* Reset the function in place of the PCI core; mandatory. Called with
Quoted 8 lines
include/linux/vfio_pci_core.h struct vfio_cxl_ops { · line 95
Quoted 4 lines
include/linux/vfio_pci_core.h +4
-0+* the PCI device lock and memory_lock (for write) held. The caller +* provides the saved PCI state the reset restores: the try-reset path +* saves it first, the close path keeps the open-time copy. +*/ Reads like more AI generated comments. These things like to talk, please curb them. Look at
vfio_device_opsfor an example how to briefly and concisely document callback ops like a human. Thanks,
I will update this. The remaining ops will be edited similar to vfio_device_ops style.
Thanks,
Manish
Quoted 9 lines
Alex
PS - I'm still working on the remainder of the series.
Quoted 4 lines
include/linux/vfio_pci_core.h +1
-0+int (*reset)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
PATCH v6 16/31 vfio/pci: Revoke and re-arm one device-specific region
In reply to [email protected]From: Manish Honap <[email protected]>
The BARs of a vfio PCI device follow the memory enable bit: clearing it zaps their mmaps and revokes their dma-bufs, and setting it re-arms the dma-bufs. A device-specific region does not have to follow that bit. The HDM memory region of a CXL Type-2 device follows the committed HDM decoder, which changes across reset, bus reset and low power, not with Memory Space Enable. Other device-specific regions, such as nvgrace coherent memory or the IGD OpRegion, do not depend on it either.
Record the region index of each dma-buf at export. vfio_pci_dma_buf_move() now moves only the BAR dma-bufs, and vfio_pci_dma_buf_cleanup() still drains every dma-buf of the device.
Export vfio_pci_core_revoke_region(), which zaps the user mmaps of one device-specific region and revokes its dma-bufs, and vfio_pci_core_rearm_region(), which re-arms them. A provider calls them under memory_lock when the state its region depends on changes. Also export __vfio_pci_vmf_insert_pfn(), the pfn insert of vfio_pci_vmf_insert_pfn() without its memory enable and runtime PM checks, for the fault handler of such a region. Export vfio_pci_core_clear_memory_enable() for a provider whose failed operation left Memory Space off in the device: it zaps the BAR mmaps, revokes the BAR dma-bufs and clears vfio's record of the bit.
--- drivers/vfio/pci/vfio_pci_core.c | 109 +++++++++++++++++++++++++++-- drivers/vfio/pci/vfio_pci_dmabuf.c | 98 +++++++++++++++++--------- drivers/vfio/pci/vfio_pci_priv.h | 7 ++ include/linux/vfio_pci_core.h | 8 +++ 4 files changed, 180 insertions(+), 42 deletions(-)
drivers/vfio/pci/vfio_pci_core.c +102 -7
@@ -1982,6 +1982,77 @@ static void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev)unmap_mapping_range (core_vdev->inode->i_mapping, start, len, true); } +/** +* vfio_pci_core_revoke_region() - take a device-specific region out of use +* @vdev: vfio PCI device, with memory_lock held for write +* @index: vfio region index of a device-specific region +* +* Zap the user mmaps of the region and revoke its dma-bufs, for a provider +* whose region depends on device state that is about to change. The BARs +* follow the memory enable bit instead; a device-specific region may not. +* A later fault on the region goes back to the region's own fault handler. +*/ +void vfio_pci_core_revoke_region (struct vfio_pci_core_device *vdev, +unsigned int index) +{ +struct vfio_pci_region *region; + +lockdep_assert_held_write (&vdev->memory_lock); + +if (WARN_ON (index <VFIO_PCI_NUM_REGIONS || + index -VFIO_PCI_NUM_REGIONS >= vdev->num_regions)) +return ; + + region = &vdev->region[index -VFIO_PCI_NUM_REGIONS ]; +unmap_mapping_range (vdev->vdev.inode->i_mapping, +VFIO_PCI_INDEX_TO_OFFSET (index), region->size, true); +vfio_pci_dma_buf_move_region (vdev, index, true); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_revoke_region); + +/** +* vfio_pci_core_rearm_region() - put a revoked device-specific region back +* @vdev: vfio PCI device, with memory_lock held for write +* @index: vfio region index passed to vfio_pci_core_revoke_region() +*/ +void vfio_pci_core_rearm_region (struct vfio_pci_core_device *vdev, +unsigned int index) +{ +lockdep_assert_held_write (&vdev->memory_lock); + +vfio_pci_dma_buf_move_region (vdev, index, false); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_rearm_region); + +/** +* vfio_pci_core_clear_memory_enable() - follow a device that lost Memory Space +* @vdev: vfio PCI device, with memory_lock held for write +* +* For a provider whose failed operation left Memory Space disabled in the +* device. Zap the BAR mmaps, revoke the BAR dma-bufs and clear vfio's record +* of the Memory Space bit, as a user write that clears the bit would. The +* user's next Memory Space write enables it again. +*/ +void vfio_pci_core_clear_memory_enable (struct vfio_pci_core_device *vdev) +{ +__le16 *virt_cmd; + +lockdep_assert_held_write (&vdev->memory_lock); + +/* Closed, or not open yet: vfio_config_init() reads the device. */ +if (!vdev->vconfig) +return ; + + virt_cmd = (__le16 *)&vdev->vconfig[PCI_COMMAND ]; +if (!(*virt_cmd &cpu_to_le16 (PCI_COMMAND_MEMORY ))) +return ; + +vfio_pci_zap_bars (vdev); +vfio_pci_dma_buf_move (vdev, true); + *virt_cmd &=cpu_to_le16 (~PCI_COMMAND_MEMORY ); +} +EXPORT_SYMBOL_GPL (vfio_pci_core_clear_memory_enable); +void vfio_pci_zap_and_down_write_memory_lock (struct vfio_pci_core_device *vdev) {down_write (&vdev->memory_lock);
@@ -2019,16 +2090,26 @@ static unsigned long vma_to_pfn(struct vm_area_struct *vma)return (pci_resource_start (vdev->pdev, index) >>PAGE_SHIFT ) + pgoff; } -vm_fault_t vfio_pci_vmf_insert_pfn (struct vfio_pci_core_device *vdev, -struct vm_fault *vmf, -unsigned long pfn, -unsigned int order) +/** +* __vfio_pci_vmf_insert_pfn() - insert a pfn without the memory enable and +* runtime PM checks +* @vdev: vfio PCI device, with memory_lock held for read +* @vmf: the fault being served +* @pfn: first pfn to insert +* @order: page order of the mapping +* +* For a device-specific region that does not follow the memory enable bit. +* The caller checks the state its region depends on under memory_lock, and +* revokes the region with vfio_pci_core_revoke_region() when it changes or +* before the device leaves D0. +*/ +vm_fault_t __vfio_pci_vmf_insert_pfn (struct vfio_pci_core_device *vdev, +struct vm_fault *vmf, +unsigned long pfn, +unsigned int order) {lockdep_assert_held_read (&vdev->memory_lock); -if (vdev->pm_runtime_engaged || !__vfio_pci_memory_enabled (vdev)) -return VM_FAULT_SIGBUS ; -if (!order)return vmf_insert_pfn (vmf->vma, vmf->address, pfn);
@@ -2040,6 +2121,20 @@ vm_fault_t vfio_pci_vmf_insert_pfn(struct vfio_pci_core_device *vdev,return VM_FAULT_FALLBACK ; } +EXPORT_SYMBOL_GPL (__vfio_pci_vmf_insert_pfn); + +vm_fault_t vfio_pci_vmf_insert_pfn (struct vfio_pci_core_device *vdev, +struct vm_fault *vmf, +unsigned long pfn, +unsigned int order) +{ +lockdep_assert_held_read (&vdev->memory_lock); + +if (vdev->pm_runtime_engaged || !__vfio_pci_memory_enabled (vdev)) +return VM_FAULT_SIGBUS ; + +return __vfio_pci_vmf_insert_pfn (vdev, vmf, pfn, order); +}EXPORT_SYMBOL_GPL (vfio_pci_vmf_insert_pfn); static vm_fault_t vfio_pci_mmap_huge_fault(struct vm_fault *vmf,
drivers/vfio/pci/vfio_pci_dmabuf.c +63 -35
129 lines not on this page · raw message
drivers/vfio/pci/vfio_pci_priv.h +7 -0
@@ -141,6 +141,8 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, size_t argsz);void vfio_pci_dma_buf_cleanup (struct vfio_pci_core_device *vdev);void vfio_pci_dma_buf_move (struct vfio_pci_core_device *vdev,bool revoked); +void vfio_pci_dma_buf_move_region (struct vfio_pci_core_device *vdev, +unsigned int index,bool revoked);#else static inline int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
@@ -156,6 +158,11 @@ static inline void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked) { } +static inline void +vfio_pci_dma_buf_move_region (struct vfio_pci_core_device *vdev, +unsigned int index,bool revoked) +{ +}#endif #endif
include/linux/vfio_pci_core.h +8 -0
@@ -227,6 +227,11 @@ int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops, size_t size, u32 flags, void *data); +void vfio_pci_core_revoke_region (struct vfio_pci_core_device *vdev, +unsigned int index); +void vfio_pci_core_rearm_region (struct vfio_pci_core_device *vdev, +unsigned int index); +void vfio_pci_core_clear_memory_enable (struct vfio_pci_core_device *vdev);void vfio_pci_core_close_device (struct vfio_device *core_vdev);int vfio_pci_core_init_dev (struct vfio_device *core_vdev);void vfio_pci_core_release_dev (struct vfio_device *core_vdev);
@@ -250,6 +255,9 @@ ssize_t vfio_pci_core_read(struct vfio_device *core_vdev, char __user *buf, size_t count, loff_t *ppos);ssize_t vfio_pci_core_write (struct vfio_device *core_vdev,const char __user *buf,size_t count,loff_t *ppos); +vm_fault_t __vfio_pci_vmf_insert_pfn (struct vfio_pci_core_device *vdev, +struct vm_fault *vmf,unsigned long pfn, +unsigned int order);vm_fault_t vfio_pci_vmf_insert_pfn (struct vfio_pci_core_device *vdev,struct vm_fault *vmf,unsigned long pfn,unsigned int order);
Signature · 1 line
2.53.0
PATCH v6 17/31 vfio/pci: Call the device provider across low-power transitions
In reply to [email protected]From: Manish Honap <[email protected]>
A provider may serve state that does not survive low power. A CXL Type-2 device loses its committed HDM decoder in D3hot or D3cold, and the HDM memory region must stay out of use until the host restores the decoder. Without a hook the provider can only keep the device in D0.
Add optional power_down and power_up ops. vfio-pci-core calls power_down with memory_lock held for write before the device leaves D0: in vfio_pci_set_power_state() for a lower state, and in runtime suspend before the PCI core moves the device to low power. It calls power_up with memory_lock held after the device is back in D0 with its config space restored: after a guest PM control write returns the device to D0, and in runtime resume. If a request for a lower state leaves the device in D0, because the transition failed or a quirk keeps it there, vfio-pci-core calls power_up at once.
The other paths that return a device to D0 either reset it afterwards (close, VFIO_DEVICE_RESET and the hot reset ioctl), where the provider restores its state, or tear it down (unregister).
--- drivers/vfio/pci/vfio_pci_config.c | 4 +++ drivers/vfio/pci/vfio_pci_core.c | 40 +++++++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_priv.h | 1 + include/linux/vfio_pci_core.h | 7 ++++++ 4 files changed, 51 insertions(+), 1 deletion(-)
drivers/vfio/pci/vfio_pci_config.c +4 -0
@@ -712,6 +712,8 @@ static int __init init_pci_cap_basic_perm(struct perm_bits *perm)static void vfio_lock_and_set_power_state (struct vfio_pci_core_device *vdev,pci_power_t state) { +pci_power_t prev = vdev->pdev->current_state; +if (state >= PCI_D3hot) {vfio_pci_zap_and_down_write_memory_lock (vdev);vfio_pci_dma_buf_move (vdev, true);
@@ -720,6 +722,8 @@ static void vfio_lock_and_set_power_state(struct vfio_pci_core_device *vdev, }vfio_pci_set_power_state (vdev, state); +if (state ==PCI_D0 && prev >PCI_D0 ) +vfio_pci_provider_power_up (vdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock);
drivers/vfio/pci/vfio_pci_core.c +39 -1
@@ -318,10 +318,31 @@ static void vfio_pci_probe_power_state(struct vfio_pci_core_device *vdev)* by PM capability emulation and separately from pci_dev internal saved state * to avoid it being overwritten and consumed around other resets. */ +/* +* A provider whose device state does not survive low power is told before +* the device leaves D0 and after it is back in D0 with its config space +* restored. Both run with memory_lock held for write. +*/ +static void vfio_pci_provider_power_down (struct vfio_pci_core_device *vdev) +{ +lockdep_assert_held_write (&vdev->memory_lock); + +if (vdev->cxl_ops && vdev->cxl_ops->power_down) + vdev->cxl_ops->power_down (vdev); +} + +void vfio_pci_provider_power_up (struct vfio_pci_core_device *vdev) +{ +lockdep_assert_held_write (&vdev->memory_lock); + +if (vdev->cxl_ops && vdev->cxl_ops->power_up) + vdev->cxl_ops->power_up (vdev); +} +int vfio_pci_set_power_state (struct vfio_pci_core_device *vdev,pci_power_t state) {struct pci_dev *pdev = vdev->pdev; -bool needs_restore = false, needs_save = false; +bool needs_restore = false, needs_save = false, powered_down = false;int ret;/* Prevent changing power state for PFs with VFs enabled */
@@ -329,6 +350,11 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t statlockdep_assert_held_write (&vdev->memory_lock);if (vdev->sriov_active)return -EBUSY ; + +if (pdev->current_state ==PCI_D0 ) { +vfio_pci_provider_power_down (vdev); + powered_down = true; + } }if (vdev->needs_pm_restore) {
@@ -366,6 +392,14 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t stat } } +/* +* The device stayed in D0: the transition failed, or a quirk keeps it +* there. No later D0 transition will call power_up, so undo power_down +* now. +*/ +if (powered_down && pdev->current_state ==PCI_D0 ) +vfio_pci_provider_power_up (vdev); +return ret; }
@@ -502,6 +536,8 @@ static int vfio_pci_core_runtime_suspend(struct device *dev)* (saved locally in 'vdev->pm_save'). */ vfio_pci_set_power_state (vdev,PCI_D0 ); +/* The PCI core moves the device to low power once this returns. */ +vfio_pci_provider_power_down (vdev);up_write (&vdev->memory_lock);/*
@@ -526,6 +562,8 @@ static int vfio_pci_core_runtime_resume(struct device *dev)* low power mode. */ down_write (&vdev->memory_lock); +/* The PCI core has restored the config space of the device. */ +vfio_pci_provider_power_up (vdev);if (vdev->pm_wake_eventfd_ctx) {eventfd_signal (vdev->pm_wake_eventfd_ctx);__vfio_pci_runtime_pm_exit (vdev);
drivers/vfio/pci/vfio_pci_priv.h +1 -0
@@ -78,6 +78,7 @@ void vfio_config_free(struct vfio_pci_core_device *vdev);int vfio_pci_set_power_state (struct vfio_pci_core_device *vdev,pci_power_t state); +void vfio_pci_provider_power_up (struct vfio_pci_core_device *vdev);void vfio_pci_zap_and_down_write_memory_lock (struct vfio_pci_core_device *vdev);int vfio_pci_try_reset_function (struct vfio_pci_core_device *vdev);
include/linux/vfio_pci_core.h +7 -0
@@ -94,6 +94,13 @@ struct vfio_cxl_ops {* saves it first, the close path keeps the open-time copy. */ int (*reset)(struct vfio_pci_core_device *vdev); +/* +* Optional. Called with memory_lock held for write before the device +* leaves D0, and after it returns to D0 with its config space +* restored. power_down may be called again while already powered down. +*/ +void (*power_down)(struct vfio_pci_core_device *vdev); +void (*power_up)(struct vfio_pci_core_device *vdev);/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */ struct module *owner; };
Signature · 1 line
2.53.0
PATCH v6 18/31 vfio/cxl: Add the vfio-cxl provider module skeleton
In reply to [email protected]From: Manish Honap <[email protected]>
Add the vfio-cxl module that vfio-pci-core loads on demand for a CXL Type-2 device, registering its callbacks at module_init. Its reset op runs the PCI core's reset methods until vfio-cxl serves the device memory.
vfio-cxl is a feature of vfio-pci-core rather than a variant driver, so its Kconfig sources above the variant drivers and its objects link right after vfio-pci-core while still being loaded on demand:request_module("vfio-cxl") finds it by module name.
--- MAINTAINERS | 7 +++ drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/cxl/Kconfig | 11 ++++ drivers/vfio/pci/cxl/Makefile | 3 + drivers/vfio/pci/cxl/vfio_cxl_core.c | 82 ++++++++++++++++++++++++++++ 6 files changed, 107 insertions(+) create mode 100644 drivers/vfio/pci/cxl/Kconfig create mode 100644 drivers/vfio/pci/cxl/Makefile create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_core.c
MAINTAINERS +7 -0
@@ -28636,6 +28636,13 @@ L: [email protected] S: Maintained F: drivers/vfio/cdx/* +VFIO CXL DRIVER +M: Manish Honap <[email protected]> +L: [email protected] +L: [email protected] +S: Supported +F: drivers/vfio/pci/cxl/ + VFIO DRIVER M: Alex Williamson <[email protected]> L: [email protected]
drivers/vfio/pci/Kconfig +2 -0
@@ -58,6 +58,8 @@ config VFIO_PCI_ZDEV_KVM config VFIO_PCI_DMABUF def_bool y if VFIO_PCI_CORE && PCI_P2PDMA && DMA_SHARED_BUFFER +source "drivers/vfio/pci/cxl/Kconfig" + source "drivers/vfio/pci/mlx5/Kconfig" source "drivers/vfio/pci/ism/Kconfig"
drivers/vfio/pci/Makefile +2 -0
@@ -5,6 +5,8 @@ vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) += vfio_pci_zdev.o vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) += vfio_pci_dmabuf.o obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o +obj-$(CONFIG_VFIO_CXL) += cxl/ + vfio-pci-y := vfio_pci.o vfio-pci-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
drivers/vfio/pci/cxl/Kconfig new file +11 -0
@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0-only +config VFIO_CXL + tristate "VFIO support for CXL Type-2 devices" + depends on CXL_BUS && CXL_MEM + select VFIO_PCI_CORE + help + Add CXL HDM handling to vfio-pci so a CXL Type-2 device, such as a + CXL-attached accelerator, can be assigned to a virtual machine. + vfio-pci loads this module on demand when it binds a CXL device. + + If you don't know what to do here, say N.
drivers/vfio/pci/cxl/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_VFIO_CXL) += vfio-cxl.o +vfio-cxl-y := vfio_cxl_core.o
drivers/vfio/pci/cxl/vfio_cxl_core.c new file +82 -0
82 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v6 19/31 vfio/cxl: Create the CXL memdev and set media ready at bind
In reply to [email protected]From: Manish Honap <[email protected]>
At bind, build the CXL memory device for the passed-through Type-2 accelerator so it joins the CXL topology and its HDM region resolves to a host physical range. A Type-2 device has no mailbox, so there is no media-ready register to poll: set media ready directly once the component registers validate (mirroring drivers/net/ethernet/sfc/efx_cxl.c)
As per current vfio-cxl support, reject a device with:
- more than one HDM decoder
- interleaving enabled
- whose reset the host cannot service, including the CPU cache invalidation that the reset needs
The CXL-core allocations are grouped with devres, in a helper that returns on the first error, so a failed bind unwinds them: a failed init leaves the device bound without CXL support, so devm would otherwise hold them until unbind.
The device is not kept in D0. Low power loses the committed HDM decoder, and the host restores the decoder configuration when the device returns to D0. The contents of device memory across low power are the user's to keep.
Take the bind-time copy of the cached HDM state with pci_cxl_hdm_snapshot(), which needs no device lock: probe may run in a worker on the device's NUMA node that does not own the lock.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 135 +++++++++++++++++++++++++++ 1 file changed, 135 insertions(+)
drivers/vfio/pci/cxl/vfio_cxl_core.c +135 -0
@@ -5,17 +5,151 @@* Copyright (c) 2026 NVIDIA Corporation & Affiliates */ +#include <linux/cleanup.h> #include <linux/module.h> #include <linux/pci.h> +#include <linux/range.h> +#include <linux/slab.h> #include <linux/vfio_pci_core.h> +#include <cxl/cxl.h> +#include <cxl/hdm.h> +#include <cxl/pci.h> + +/** +* struct vfio_cxl_state - per-device state for a vfio-cxl device +* @cxlds: CXL device state; kept first for devm_cxl_dev_state_create() +* @cxlmd: memory device joined to the CXL topology at bind +* @hpa_range: host physical range of the HDM region +* @hdm: copy of the HDM state cached at PCI enumeration, taken at bind +*/ +struct vfio_cxl_state { +struct cxl_dev_state cxlds; +struct cxl_memdev *cxlmd; +struct range hpa_range; +struct cxl_hdm_info *hdm; +}; + +/* +* Create the CXL device state and join the device to the CXL topology. The +* caller groups these devres allocations, so return on the first error. +*/ +static struct vfio_cxl_state *vfio_cxl_create_memdev (struct pci_dev *pdev, +u64 hdm_size) +{ +struct vfio_cxl_state *cxl; +struct cxl_memdev *cxlmd; +u64 serial; +u16 dvsec; +int ret; + + dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_DEVICE ); + serial =pci_get_dsn (pdev); + + cxl =devm_cxl_dev_state_create (&pdev->dev,CXL_DEVTYPE_DEVMEM , serial, + dvsec, struct vfio_cxl_state, cxlds, + false); +if (!cxl) +return ERR_PTR (-ENOMEM ); + + ret =cxl_pci_setup_regs (pdev,CXL_REGLOC_RBI_COMPONENT , + &cxl->cxlds.reg_map); +if (ret) { +pci_err (pdev,"vfio-cxl: no component registers\n" ); +return ERR_PTR (ret); + } + +if (!cxl->cxlds.reg_map.component_map.hdm_decoder.valid) { +pci_err (pdev,"vfio-cxl: HDM decoder registers not found\n" ); +return ERR_PTR (-ENODEV ); + } + +/* +* A Type-2 accelerator has no mailbox and no media-ready register, so +* set media ready directly. +*/ + cxl->cxlds.media_ready = true; + + ret =cxl_set_capacity (&cxl->cxlds, hdm_size); +if (ret) +return ERR_PTR (ret); + + cxlmd =devm_cxl_probe_mem (&cxl->cxlds, &cxl->hpa_range); +if (IS_ERR (cxlmd)) +return ERR_CAST (cxlmd); + + cxl->cxlmd = cxlmd; +return cxl; +}static int vfio_cxl_init_device (struct vfio_pci_core_device *vdev) { +struct pci_dev *pdev = vdev->pdev; + struct cxl_hdm_info *hdm __free(kfree) = NULL; +struct vfio_cxl_state *cxl; +u64 hdm_size; + +/* +* pdev->hdm is cached at PCI enumeration, before any driver binds, so a +* device without it has no usable HDM decoder. Continue without CXL +* support rather than defer the bind forever. The cache belongs to the +* CXL core, so work from a copy. Probe may run in a worker on the +* device's NUMA node that does not own the device lock, so take the +* copy that needs none. +*/ + hdm =pci_cxl_hdm_snapshot (pdev); +if (IS_ERR (hdm)) +return PTR_ERR (hdm); +if (!hdm) +return -ENODEV ; + +/* The guest drives one virtual decoder; multiple are unsupported. */ +if (hdm->decoder_count !=1 ) +return -EOPNOTSUPP ; + +/* An interleaved decoder cannot be mapped 1:1 to the guest. */ +if (hdm->settings[0 ].config.interleave_ways !=1 ) +return -EOPNOTSUPP ; + +/* +* The guest drives resets through the CXL Device DVSEC and polls the +* shadow for completion. If the host cannot service a function-scoped +* CXL reset, that request could never complete, so refuse the device +* rather than advertise a reset the guest would poll on forever. The +* check includes the CPU cache invalidation that the reset needs. +*/ +if (!pci_cxl_reset_capable (pdev)) +return -EOPNOTSUPP ; + + hdm_size =range_len (&hdm->settings[0 ].config.hpa_range); +if (!hdm_size) +return -ENXIO ; + +/* +* Group the CXL-core allocations so a failure unwinds them here. A +* failed init leaves the device bound without CXL support, so devm +* would otherwise hold them until unbind. +*/ +if (!devres_open_group (&pdev->dev,NULL ,GFP_KERNEL )) +return -ENOMEM ; + + cxl =vfio_cxl_create_memdev (pdev, hdm_size); +if (IS_ERR (cxl)) { +devres_release_group (&pdev->dev,NULL ); +return PTR_ERR (cxl); + } +devres_close_group (&pdev->dev,NULL ); + + cxl->hdm =no_free_ptr (hdm); + vdev->cxl = cxl; +return 0 ; }static void vfio_cxl_release_device (struct vfio_pci_core_device *vdev) { +kfree (vdev->cxl->hdm); + vdev->cxl =NULL ; }static int vfio_cxl_open_device (struct vfio_pci_core_device *vdev)
@@ -80,3 +214,4 @@ module_exit(vfio_cxl_exit);MODULE_LICENSE ("GPL" );MODULE_DESCRIPTION ("VFIO support for CXL Type-2 devices" ); +MODULE_IMPORT_NS ("CXL" );
Signature · 1 line
2.53.0
PATCH v6 20/31 vfio/cxl: Own the whole component register BAR
In reply to [email protected]From: Manish Honap <[email protected]>
devm_cxl_probe_mem() maps the component registers through the CXL core, which by default claims each HDM and RAS sub-block. That collides with the full-BAR request vfio-pci-core makes when the guest opens the device.
After the component registers are discovered, declare the containing BAR owned with cxl_reg_map_set_owned_resource(). The core then ioremaps the sub-blocks without requesting them, so vfio-pci-core owns the whole BAR and the requests do not collide.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +15 -3
@@ -33,8 +33,9 @@ struct vfio_cxl_state {* Create the CXL device state and join the device to the CXL topology. The * caller groups these devres allocations, so return on the first error. */ -static struct vfio_cxl_state *vfio_cxl_create_memdev (struct pci_dev *pdev, -u64 hdm_size) +static struct vfio_cxl_state * +vfio_cxl_create_memdev (struct pci_dev *pdev,const struct cxl_hdm_info *hdm, +u64 hdm_size) {struct vfio_cxl_state *cxl;struct cxl_memdev *cxlmd;
@@ -59,6 +60,17 @@ static struct vfio_cxl_state *vfio_cxl_create_memdev(struct pci_dev *pdev,return ERR_PTR (ret); } +/* +* vfio-pci-core requests the whole component-register BAR when the +* guest opens the device. Declare that BAR owned so the CXL core +* ioremaps the HDM and RAS sub-blocks without claiming them, and the +* full-BAR request does not collide. +*/ + ret =cxl_reg_map_set_owned_resource (&cxl->cxlds.reg_map, +pci_resource_n (pdev, hdm->hdm_bar)); +if (ret) +return ERR_PTR (ret); +if (!cxl->cxlds.reg_map.component_map.hdm_decoder.valid) {pci_err (pdev,"vfio-cxl: HDM decoder registers not found\n" );return ERR_PTR (-ENODEV );
@@ -133,7 +145,7 @@ static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)if (!devres_open_group (&pdev->dev,NULL ,GFP_KERNEL ))return -ENOMEM ; - cxl =vfio_cxl_create_memdev (pdev, hdm_size); + cxl =vfio_cxl_create_memdev (pdev, hdm, hdm_size);if (IS_ERR (cxl)) {devres_release_group (&pdev->dev,NULL );return PTR_ERR (cxl);
Signature · 1 line
2.53.0
PATCH v6 21/31 vfio/cxl: Expose the HDM memory region to the guest
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL Type-2 guest maps the device HDM memory to use its coherent CXL.mem. The HDM memory is a host physical range with no struct page, so the guest and KVM need a write-back mapping of it.
Register the range as a region under VFIO_REGION_TYPE_PCI_VENDOR_TYPE with the CXL vendor id rather than a bespoke region type, per open because vfio-pci-core tears down all dynamic regions on close and on a failed open. This patch serves read and write on the region fd; mmap follows.
Own the resolved host physical range exclusively (IORESOURCE_EXCLUSIVE) so nothing, /dev/mem included, can map a conflicting cacheable alias that would fault the host once the range is mapped write-back. Release it in the provider .release op rather than through a devm action on the PCI device, which could run after vfio-cxl has unloaded. vfio-cxl now holds the range, so its reset op runs CXL Reset with pci_cxl_reset_owned().
Host access to the range is safe only while the committed decoder is in place. Track that in hdm_valid, set at bind once the decoder is verified committed. Low power loses the decoder, so close the gate in power_down and restore the decoder with cxl_restore_state_after_pci_reset() in power_up, reopening the gate on success.
Device memory can stay unavailable for seconds after the decoder is back:
the device clears Memory_Active in DVSEC CXL Range 1 Size Low until the memory is usable (CXL r4.0 sec 8.1.3.8.2).
At bind, wait up to the Memory_Active_Timeout the device reports for Memory_Active, since a reset just before the bind also clears it. A device whose memory then reads active brings it up by itself, and power_up waits for Memory_Active, up to the same timeout, before it reopens the gate.
A device whose memory stays inactive relies on its driver and is not waited for.
CXL.mem decode follows the committed decoder and Mem_Enable, not Memory Space Enable (CXL r4.0 sec 8.2.4.20.2, 8.2.4.20.7 and 8.1.3.2), so the gate does not check the Command register. Read and write copy through a page-sized kernel buffer, so memory_lock is never held across a user copy.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 285 ++++++++++++++++++++++++++- include/uapi/linux/vfio.h | 4 + 2 files changed, 279 insertions(+), 10 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +275 -10
@@ -5,11 +5,15 @@* Copyright (c) 2026 NVIDIA Corporation & Affiliates */ +#include <linux/bitfield.h> #include <linux/cleanup.h> +#include <linux/delay.h> +#include <linux/io.h> #include <linux/module.h> #include <linux/pci.h> #include <linux/range.h> #include <linux/slab.h> +#include <linux/uaccess.h> #include <linux/vfio_pci_core.h> #include <cxl/cxl.h> #include <cxl/hdm.h>
@@ -20,13 +24,114 @@* @cxlds: CXL device state; kept first for devm_cxl_dev_state_create() * @cxlmd: memory device joined to the CXL topology at bind * @hpa_range: host physical range of the HDM region +* @hpa_res: exclusive reservation of @hpa_range, held for the bind * @hdm: copy of the HDM state cached at PCI enumeration, taken at bind +* @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock +* @await_mem_active: the device reported its memory active at bind, so wait +* for Memory_Active before host access reopens after a reset or low power */ struct vfio_cxl_state {struct cxl_dev_state cxlds;struct cxl_memdev *cxlmd;struct range hpa_range; +struct resource *hpa_res;struct cxl_hdm_info *hdm; +bool hdm_valid; +bool await_mem_active; +}; + +/* +* Copy one chunk between the bounce buffer and the HDM range under +* memory_lock. The user copy stays outside the lock: a user buffer that is +* itself an mmap of this device faults through a handler that takes +* memory_lock for read, and a writer queued in between would deadlock the +* two readers. +*/ +static int vfio_cxl_mem_copy (struct vfio_pci_core_device *vdev,void *bounce, +u64 pos,size_t len,bool iswrite) +{ +struct vfio_cxl_state *cxl = vdev->cxl; +void *mem; + +guard (rwsem_read)(&vdev->memory_lock); + +/* +* Only touch the HDM range with the decoder in a known-good state, or +* a host CPU access aborts as a fatal SError. CXL.mem decode follows +* the committed decoder and Mem_Enable, not Memory Space Enable (CXL +* r4.0 sec 8.2.4.20.2, 8.1.3.2). +*/ +if (!cxl->hdm_valid || !cxl->cxlds.media_ready) +return -EIO ; + + mem =memremap (cxl->hpa_range.start + pos, len,MEMREMAP_WB ); +if (!mem) +return -ENOMEM ; +if (iswrite) +memcpy (mem, bounce, len); +else +memcpy (bounce, mem, len); +memunmap (mem); + +return 0 ; +} + +static ssize_t vfio_cxl_mem_rw (struct vfio_pci_core_device *vdev, +char __user *buf,size_t count,loff_t *ppos, +bool iswrite) +{ +struct vfio_cxl_state *cxl = vdev->cxl; +u64 pos = *ppos &VFIO_PCI_OFFSET_MASK ; +ssize_t done =0 ; +void *bounce; +int ret =0 ; + +if (pos >=range_len (&cxl->hpa_range)) +return -EINVAL ; + count =min_t (u64, count,range_len (&cxl->hpa_range) - pos); + + bounce =kmalloc (min_t (size_t, count,PAGE_SIZE ),GFP_KERNEL ); +if (!bounce) +return -ENOMEM ; + +while (count) { +size_t len =min_t (size_t, count,PAGE_SIZE ); + +if (iswrite &©_from_user (bounce, buf + done, len)) { + ret = -EFAULT ; +break ; + } + ret =vfio_cxl_mem_copy (vdev, bounce, pos + done, len, iswrite); +if (ret) +break ; +if (!iswrite &©_to_user (buf + done, bounce, len)) { + ret = -EFAULT ; +break ; + } + done += len; + count -= len; + } +kfree (bounce); + +if (!done) +return ret; + + *ppos += done; +return done; +} + +/* +* The CXL regions carry no per-region state (region->data is the shared, +* devm-managed vfio_cxl_state), so releasing a region is a no-op. +*/ +static void vfio_cxl_region_release (struct vfio_pci_core_device *vdev, +struct vfio_pci_region *region) +{ +} + +static const struct vfio_pci_regops vfio_cxl_mem_regops = { + .rw = vfio_cxl_mem_rw, + .release = vfio_cxl_region_release, };/*
@@ -94,12 +199,75 @@ vfio_cxl_create_memdev(struct pci_dev *pdev, const struct cxl_hdm_info *hdm,return cxl; } +/* CXL r4.0 sec 8.1.3.8.2: Range 1 Size Low Memory_Active_Timeout encodings */ +static const unsigned int vfio_cxl_mem_active_timeout_s[] = {1 ,4 ,16 ,64 ,256 }; + +/* +* Poll Memory_Active in DVSEC CXL Range 1 Size Low (CXL r4.0 sec 8.1.3.8.2) +* for up to the Memory_Active_Timeout the device reports, which is stored in +* @timeout. Return 0 once it is set, -ETIMEDOUT, or -ENODEV if config space +* does not respond. +*/ +static int vfio_cxl_poll_mem_active (struct pci_dev *pdev,u16 dvsec, +unsigned int *timeout) +{ +int pos = dvsec +PCI_DVSEC_CXL_RANGE_SIZE_LOW (0 ); +unsigned long deadline; +u32 size_lo; + +if (pci_read_config_dword (pdev, pos, &size_lo) || +PCI_POSSIBLE_ERROR (size_lo)) +return -ENODEV ; + + *timeout =FIELD_GET (PCI_DVSEC_CXL_MEM_ACTIVE_TIMEOUT , size_lo); + *timeout = vfio_cxl_mem_active_timeout_s[min_t(unsigned int, *timeout, +ARRAY_SIZE (vfio_cxl_mem_active_timeout_s) -1 )]; + deadline = jiffies +secs_to_jiffies (*timeout); + +while (!(size_lo &PCI_DVSEC_CXL_MEM_ACTIVE )) { +if (time_after (jiffies, deadline)) +return -ETIMEDOUT ; +msleep (20 ); +if (pci_read_config_dword (pdev, pos, &size_lo) || +PCI_POSSIBLE_ERROR (size_lo)) +return -ENODEV ; + } + +return 0 ; +} + +/* +* A reset or a return from low power leaves device memory unavailable until +* the device sets Memory_Active. That can take seconds after the reset +* completes and the decoder is restored, and a host access to the HDM range +* before then can abort. Wait for it on a device that brings its memory up +* by itself; a device that relies on its driver for that was found at bind +* and is not waited for. +*/ +static int vfio_cxl_await_mem_active (struct vfio_pci_core_device *vdev) +{ +struct vfio_cxl_state *cxl = vdev->cxl; +unsigned int timeout =0 ; +int ret; + +if (!cxl->await_mem_active) +return 0 ; + + ret =vfio_cxl_poll_mem_active (vdev->pdev, cxl->hdm->dvsec, &timeout); +if (ret == -ETIMEDOUT ) +pci_err (vdev->pdev,"vfio-cxl: device memory not active after %us\n" , + timeout); +return ret; +} +static int vfio_cxl_init_device (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev; struct cxl_hdm_info *hdm __free(kfree) = NULL;struct vfio_cxl_state *cxl; +unsigned int timeout =0 ;u64 hdm_size; +int ret;/* * pdev->hdm is cached at PCI enumeration, before any driver binds, so a
@@ -150,9 +318,44 @@ static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)devres_release_group (&pdev->dev,NULL );return PTR_ERR (cxl); } + +/* +* Claim the range IORESOURCE_EXCLUSIVE so no conflicting cacheable +* alias can fault the host once it is mapped write-back. Release it +* explicitly in vfio_cxl_release_device() rather than through a devm +* action on pdev->dev: the provider module ref is dropped as soon as +* that .release op returns, so a pdev-devres callback could run after +* vfio-cxl has unloaded and jump into freed module text. +*/ + cxl->hpa_res =request_mem_region_exclusive (cxl->hpa_range.start, +range_len (&cxl->hpa_range), +"vfio-cxl-hdm" ); +if (!cxl->hpa_res) { +devres_release_group (&pdev->dev,NULL ); +return -EBUSY ; + }devres_close_group (&pdev->dev,NULL ); cxl->hdm =no_free_ptr (hdm); +/* +* Wait for Memory_Active after a reset only on a device that brings its +* memory up by itself; see vfio_cxl_await_mem_active(). Memory_Active +* also reads clear for seconds after a reset, for example one the host +* ran just before this bind, so give the device its Memory_Active_Timeout +* before deciding. A device that relies on its driver times out here +* once. +*/ + ret =vfio_cxl_poll_mem_active (pdev, cxl->hdm->dvsec, &timeout); + cxl->await_mem_active = !ret; +if (ret == -ETIMEDOUT ) +pci_info (pdev,"vfio-cxl: device memory not active after %us, left to its driver\n" , + timeout); +/* +* The decoder decodes a non-empty range, so it is committed, and host +* access to the HDM range is safe. Reset, bus reset and low power +* close this gate until the decoder is restored. +*/ + cxl->hdm_valid = true; vdev->cxl = cxl;return 0 ;
@@ -160,13 +363,40 @@ static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)static void vfio_cxl_release_device (struct vfio_pci_core_device *vdev) { -kfree (vdev->cxl->hdm); +struct vfio_cxl_state *cxl = vdev->cxl; + +/* +* Drop the exclusive HDM range claim before vfio_pci_core_cxl_release() +* drops the provider module ref; see vfio_cxl_init_device(). +*/ +release_mem_region (cxl->hpa_range.start,range_len (&cxl->hpa_range)); +kfree (cxl->hdm); vdev->cxl =NULL ; } +static int vfio_cxl_add_region (struct vfio_pci_core_device *vdev,u32 subtype, +const struct vfio_pci_regops *ops,size_t size, +u32 flags) +{ +u32 type =VFIO_REGION_TYPE_PCI_VENDOR_TYPE |PCI_VENDOR_ID_CXL ; + +return vfio_pci_core_register_dev_region (vdev, type, subtype, ops, + size, flags, vdev->cxl); +} +static int vfio_cxl_open_device (struct vfio_pci_core_device *vdev) { -return 0 ; +struct vfio_cxl_state *cxl = vdev->cxl; + +/* +* vfio-pci-core frees all dynamic regions on close and on a failed +* open, so register them here per open rather than at bind. +*/ +return vfio_cxl_add_region (vdev,VFIO_REGION_SUBTYPE_CXL_MEM , + &vfio_cxl_mem_regops, +range_len (&cxl->hpa_range), +VFIO_REGION_INFO_FLAG_READ | +VFIO_REGION_INFO_FLAG_WRITE ); }static void vfio_cxl_close_device (struct vfio_pci_core_device *vdev)
@@ -183,21 +413,54 @@ static int vfio_cxl_reset_done(struct vfio_pci_core_device *vdev) }/* -* Until vfio-cxl serves the device memory, reset through the PCI core's -* reset methods, which run CXL Reset ahead of FLR. Restore the PCI state -* here rather than through pci_reset_function_locked(): its reset_prepare -* and reset_done callbacks take memory_lock, which the caller already holds. -* The caller has saved the state, or on close loaded the open-time copy. +* vfio-cxl holds the HDM range, so the PCI core's cxl_reset method would +* fail to request it. Run the same CXL Reset with that claim. */ static int vfio_cxl_reset (struct vfio_pci_core_device *vdev) { +/* +* The reset restores PCI state from the saved copy. The caller provides +* it: the try-reset helper saves the current state, and the close path +* keeps the open-time state that vfio_pci_core_disable() loaded. +*/ +return pci_cxl_reset_owned (vdev->pdev, vdev->cxl->hpa_res); +} + +/* Low power loses the committed decoder: close host access until restored. */ +static void vfio_cxl_power_down (struct vfio_pci_core_device *vdev) +{ +lockdep_assert_held_write (&vdev->memory_lock); + + vdev->cxl->hdm_valid = false; +} + +static void vfio_cxl_power_up (struct vfio_pci_core_device *vdev) +{ +struct vfio_cxl_state *cxl = vdev->cxl;struct pci_dev *pdev = vdev->pdev;int ret; - ret =__pci_reset_function_locked (pdev); -pci_restore_state (pdev); +lockdep_assert_held_write (&vdev->memory_lock); -return ret; +/* +* Restore the decoder that low power lost. The restore needs the device +* lock; if it is contended, keep host access closed until a reset +* restores the decoder. The restore refreshes the decoders in cxl->hdm +* from the cache; global and DVSEC control stay as taken at bind, when +* pci_cxl_reset_capable() required HDM decode and Mem_Enable on. +*/ +if (!pci_dev_trylock (pdev)) +return ; + + ret =pci_save_state (pdev); +if (!ret) + ret =cxl_restore_state_after_pci_reset (pdev, cxl->hdm); +pci_dev_unlock (pdev); + +if (ret) +pci_warn (pdev,"vfio-cxl: HDM decoder restore failed: %d\n" , ret); +else if (!vfio_cxl_await_mem_active (vdev)) + cxl->hdm_valid = true; }static const struct vfio_cxl_ops vfio_cxl_ops = {
@@ -208,6 +471,8 @@ static const struct vfio_cxl_ops vfio_cxl_ops = { .reset_prepare = vfio_cxl_reset_prepare, .reset_done = vfio_cxl_reset_done, .reset = vfio_cxl_reset, + .power_down = vfio_cxl_power_down, + .power_up = vfio_cxl_power_up, .owner = THIS_MODULE, };
include/uapi/linux/vfio.h +4 -0
@@ -370,6 +370,10 @@ struct vfio_region_info_cap_type {*/ #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD (1) +/* CXL Type-2 device (0x1e98) sub-types for VFIO_REGION_TYPE_PCI_VENDOR_TYPE */ +/* CXL.mem HDM region of a Type-2 device, read and write */ +#define VFIO_REGION_SUBTYPE_CXL_MEM (1) +/* sub-types for VFIO_REGION_TYPE_GFX */ #define VFIO_REGION_SUBTYPE_GFX_EDID (1)
Signature · 1 line
2.53.0
PATCH v6 22/31 vfio/cxl: Allow mmap of the HDM memory region
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL Type-2 guest maps the device HDM memory to use its coherent CXL.mem, so the guest and KVM need a write-back mapping of the HDM memory region. Make the region mmap-able. The range has no struct page, so the mapping is VM_PFNMAP and its fault handler inserts the pfn, at huge-page order where the alignment allows.
The fault handler inserts a pfn only while the decoder is known good and the media is ready, and returns SIGBUS otherwise. It inserts with __vfio_pci_vmf_insert_pfn(), so a fault does not depend on memory enable; low power already closes the gate through power_down. When the gate closes, in low power and later in reset and bus reset, revoke the region with vfio_pci_core_revoke_region() so no live mapping outlives the decoder, and re-arm it once the decoder is restored. The BARs keep following the memory enable bit; this region does not.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 131 +++++++++++++++++++++++++-- include/uapi/linux/vfio.h | 2 +- 2 files changed, 123 insertions(+), 10 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +122 -9
@@ -9,6 +9,7 @@#include <linux/cleanup.h> #include <linux/delay.h> #include <linux/io.h> +#include <linux/mm.h> #include <linux/module.h> #include <linux/pci.h> #include <linux/range.h>
@@ -29,6 +30,7 @@* @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock * @await_mem_active: the device reported its memory active at bind, so wait * for Memory_Active before host access reopens after a reset or low power +* @mem_region_index: vfio region index of the HDM memory region while open, else 0 */ struct vfio_cxl_state {struct cxl_dev_state cxlds;
@@ -38,8 +40,82 @@ struct vfio_cxl_state {struct cxl_hdm_info *hdm;bool hdm_valid;bool await_mem_active; +unsigned int mem_region_index; }; +static unsigned long vfio_cxl_mem_pgoff (struct vm_area_struct *vma, +unsigned long addr) +{ +unsigned long mask = (1U << (VFIO_PCI_OFFSET_SHIFT -PAGE_SHIFT )) -1 ; + +return (vma->vm_pgoff & mask) + ((addr - vma->vm_start) >>PAGE_SHIFT ); +} + +static vm_fault_t vfio_cxl_mem_huge_fault (struct vm_fault *vmf, +unsigned int order) +{ +struct vm_area_struct *vma = vmf->vma; +struct vfio_pci_core_device *vdev = vma->vm_private_data; +struct vfio_cxl_state *cxl = vdev->cxl; +unsigned long addr =ALIGN_DOWN (vmf->address,PAGE_SIZE << order); +unsigned long pfn =PHYS_PFN (cxl->hpa_range.start) + +vfio_cxl_mem_pgoff (vma, addr); +vm_fault_t ret =VM_FAULT_FALLBACK ; + +if (is_aligned_for_order (vma, addr, pfn, order)) { + scoped_guard(rwsem_read, &vdev->memory_lock) { +/* +* Insert a PFN only for a known-good decoder whose +* media is ready. CXL.mem decode follows the committed +* decoder and Mem_Enable, not Memory Space Enable (CXL +* r4.0 sec 8.2.4.20.2, 8.1.3.2), so skip the BAR memory +* enable check; low power closes hdm_valid through +* power_down. +*/ +if (cxl->hdm_valid && cxl->cxlds.media_ready) + ret =__vfio_pci_vmf_insert_pfn (vdev, vmf, pfn, + order); +else + ret =VM_FAULT_SIGBUS ; + } + } + +return ret; +} + +static vm_fault_t vfio_cxl_mem_fault (struct vm_fault *vmf) +{ +return vfio_cxl_mem_huge_fault (vmf,0 ); +} + +static const struct vm_operations_struct vfio_cxl_mem_vm_ops = { + .fault = vfio_cxl_mem_fault, +#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP + .huge_fault = vfio_cxl_mem_huge_fault, +#endif +}; + +static int vfio_cxl_mem_mmap (struct vfio_pci_core_device *vdev, +struct vfio_pci_region *region, +struct vm_area_struct *vma) +{ +unsigned long mask = (1U << (VFIO_PCI_OFFSET_SHIFT -PAGE_SHIFT )) -1 ; +u64 req_start = (vma->vm_pgoff & mask) <<PAGE_SHIFT ; +u64 req_len = vma->vm_end - vma->vm_start; + +if (req_start + req_len > region->size) +return -EINVAL ; + +/* +* CXL.mem is coherent memory, so leave the mapping write-back cacheable. +*/ +vm_flags_set (vma,VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP ); + vma->vm_ops = &vfio_cxl_mem_vm_ops; + vma->vm_private_data = vdev; + +return 0 ; +} +/* * Copy one chunk between the bounce buffer and the HDM range under * memory_lock. The user copy stays outside the lock: a user buffer that is
@@ -131,6 +207,7 @@ static void vfio_cxl_region_release(struct vfio_pci_core_device *vdev,static const struct vfio_pci_regops vfio_cxl_mem_regops = { .rw = vfio_cxl_mem_rw, + .mmap = vfio_cxl_mem_mmap, .release = vfio_cxl_region_release, };
@@ -387,20 +464,31 @@ static int vfio_cxl_add_region(struct vfio_pci_core_device *vdev, u32 subtype,static int vfio_cxl_open_device (struct vfio_pci_core_device *vdev) {struct vfio_cxl_state *cxl = vdev->cxl; +int ret;/* * vfio-pci-core frees all dynamic regions on close and on a failed * open, so register them here per open rather than at bind. */ -return vfio_cxl_add_region (vdev,VFIO_REGION_SUBTYPE_CXL_MEM , - &vfio_cxl_mem_regops, -range_len (&cxl->hpa_range), -VFIO_REGION_INFO_FLAG_READ | -VFIO_REGION_INFO_FLAG_WRITE ); + ret =vfio_cxl_add_region (vdev,VFIO_REGION_SUBTYPE_CXL_MEM , + &vfio_cxl_mem_regops, +range_len (&cxl->hpa_range), +VFIO_REGION_INFO_FLAG_READ | +VFIO_REGION_INFO_FLAG_WRITE | +VFIO_REGION_INFO_FLAG_MMAP ); +if (ret) +return ret; + +/* Record where the HDM memory region landed, to revoke it later. */ + cxl->mem_region_index =VFIO_PCI_NUM_REGIONS + vdev->num_regions -1 ; + +return 0 ; }static void vfio_cxl_close_device (struct vfio_pci_core_device *vdev) { +/* vfio-pci-core frees the regions after this returns. */ + vdev->cxl->mem_region_index =0 ; }static void vfio_cxl_reset_prepare (struct vfio_pci_core_device *vdev)
@@ -412,6 +500,33 @@ static int vfio_cxl_reset_done(struct vfio_pci_core_device *vdev)return 0 ; } +/* +* Take the HDM memory region out of use: zap its mmaps and revoke its +* dma-bufs. A fault then gets SIGBUS from vfio_cxl_mem_huge_fault() until +* hdm_valid is set again. +*/ +static void vfio_cxl_revoke_hdm (struct vfio_pci_core_device *vdev) +{ +struct vfio_cxl_state *cxl = vdev->cxl; + +lockdep_assert_held_write (&vdev->memory_lock); + + cxl->hdm_valid = false; +if (cxl->mem_region_index) +vfio_pci_core_revoke_region (vdev, cxl->mem_region_index); +} + +static void vfio_cxl_rearm_hdm (struct vfio_pci_core_device *vdev) +{ +struct vfio_cxl_state *cxl = vdev->cxl; + +lockdep_assert_held_write (&vdev->memory_lock); + + cxl->hdm_valid = true; +if (cxl->mem_region_index) +vfio_pci_core_rearm_region (vdev, cxl->mem_region_index); +} +/* * vfio-cxl holds the HDM range, so the PCI core's cxl_reset method would * fail to request it. Run the same CXL Reset with that claim.
@@ -429,9 +544,7 @@ static int vfio_cxl_reset(struct vfio_pci_core_device *vdev)/* Low power loses the committed decoder: close host access until restored. */ static void vfio_cxl_power_down (struct vfio_pci_core_device *vdev) { -lockdep_assert_held_write (&vdev->memory_lock); - - vdev->cxl->hdm_valid = false; +vfio_cxl_revoke_hdm (vdev); }static void vfio_cxl_power_up (struct vfio_pci_core_device *vdev)
@@ -460,7 +573,7 @@ static void vfio_cxl_power_up(struct vfio_pci_core_device *vdev)if (ret)pci_warn (pdev,"vfio-cxl: HDM decoder restore failed: %d\n" , ret);else if (!vfio_cxl_await_mem_active (vdev)) - cxl->hdm_valid = true; +vfio_cxl_rearm_hdm (vdev); }static const struct vfio_cxl_ops vfio_cxl_ops = {
include/uapi/linux/vfio.h +1 -1
@@ -371,7 +371,7 @@ struct vfio_region_info_cap_type {#define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD (1)/* CXL Type-2 device (0x1e98) sub-types for VFIO_REGION_TYPE_PCI_VENDOR_TYPE */ -/* CXL.mem HDM region of a Type-2 device, read and write */ +/* CXL.mem HDM region of a Type-2 device, read, write and mmap */ #define VFIO_REGION_SUBTYPE_CXL_MEM (1)/* sub-types for VFIO_REGION_TYPE_GFX */
Signature · 1 line
2.53.0
PATCH v6 23/31 vfio/cxl: Contain HDM memory errors with memory_failure()
In reply to [email protected]From: Manish Honap <[email protected]>
The HDM memory region is mmap-able, so an fd holder can fault its struct-page-less device memory in from the host CPU. A memory error on that range cannot be routed through the normal struct-page path and would otherwise escalate to a fatal host SError.
Register the range with memory_failure() via register_pfn_address_space() so such an error is instead contained to a SIGBUS to each process that maps the poisoned page. Provide a pfn-to-vma-offset callback so memory_failure() can map a poisoned pfn back to the file offset of each user mapping; the region is a single linear range at hpa_range.start, so recover the offset the same way the fault handler derived the pfn.
register_pfn_address_space() returns -EOPNOTSUPP when CONFIG_MEMORY_FAILURE is off, in which case expose the region without containment. Unregister the range on close.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 84 +++++++++++++++++++++++++++- 1 file changed, 83 insertions(+), 1 deletion(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +83 -1
@@ -9,6 +9,7 @@#include <linux/cleanup.h> #include <linux/delay.h> #include <linux/io.h> +#include <linux/memory-failure.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/pci.h>
@@ -27,6 +28,7 @@* @hpa_range: host physical range of the HDM region * @hpa_res: exclusive reservation of @hpa_range, held for the bind * @hdm: copy of the HDM state cached at PCI enumeration, taken at bind +* @hdm_pfn_space: HDM-region pfn range registered with memory_failure() * @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock * @await_mem_active: the device reported its memory active at bind, so wait * for Memory_Active before host access reopens after a reset or low power
@@ -38,6 +40,7 @@ struct vfio_cxl_state {struct range hpa_range;struct resource *hpa_res;struct cxl_hdm_info *hdm; +struct pfn_address_space hdm_pfn_space;bool hdm_valid;bool await_mem_active;unsigned int mem_region_index;
@@ -211,6 +214,63 @@ static const struct vfio_pci_regops vfio_cxl_mem_regops = { .release = vfio_cxl_region_release, }; +/* +* Map a poisoned HDM-region pfn back to the file offset of each user mapping so +* memory_failure() can find the mapping and signal its owner. The region is a +* single linear range at hpa_range.start; recover the per-vma file offset the +* same way the fault handler derived the pfn. +*/ +static int vfio_cxl_pfn_to_vma_pgoff (struct vm_area_struct *vma, +unsigned long pfn,pgoff_t *pgoff) +{ +struct vfio_pci_core_device *vdev; +struct vfio_cxl_state *cxl; +pgoff_t vma_off, pfn_off; +unsigned long start_pfn; + +if (vma->vm_ops != &vfio_cxl_mem_vm_ops) +return -ENOENT ; + + vdev = vma->vm_private_data; + cxl = vdev->cxl; + + start_pfn =PHYS_PFN (cxl->hpa_range.start); +if (pfn < start_pfn || + pfn >= start_pfn + (range_len (&cxl->hpa_range) >>PAGE_SHIFT )) +return -EFAULT ; + + pfn_off = pfn - start_pfn; + vma_off = vma->vm_pgoff & + ((1UL << (VFIO_PCI_OFFSET_SHIFT -PAGE_SHIFT )) -1 ); +/* Skip VMAs that do not map the pfn, e.g. a partial mmap of the region. */ +if (pfn_off < vma_off || pfn_off - vma_off >=vma_pages (vma)) +return -EFAULT ; + + *pgoff = vma->vm_pgoff + (pfn_off - vma_off); +return 0 ; +} + +/* +* The HDM region is struct-page-less device memory, so a memory error on it +* cannot be routed through the normal page path. Register the range with +* memory_failure() so such an error is contained to a SIGBUS to each process +* that maps the poisoned page, instead of escalating to a host SError. The +* mapping itself is not torn down. +*/ +static int vfio_cxl_register_pfn_space (struct vfio_pci_core_device *vdev) +{ +struct vfio_cxl_state *cxl = vdev->cxl; +unsigned long start_pfn =PHYS_PFN (cxl->hpa_range.start); + + cxl->hdm_pfn_space.node.start = start_pfn; + cxl->hdm_pfn_space.node.last = + start_pfn + (range_len (&cxl->hpa_range) >>PAGE_SHIFT ) -1 ; + cxl->hdm_pfn_space.mapping = vdev->vdev.inode->i_mapping; + cxl->hdm_pfn_space.pfn_to_vma_pgoff = vfio_cxl_pfn_to_vma_pgoff; + +return register_pfn_address_space (&cxl->hdm_pfn_space); +} +/* * Create the CXL device state and join the device to the CXL topology. The * caller groups these devres allocations, so return on the first error.
@@ -482,13 +542,35 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)/* Record where the HDM memory region landed, to revoke it later. */ cxl->mem_region_index =VFIO_PCI_NUM_REGIONS + vdev->num_regions -1 ; +/* +* The HDM region is advertised mmap-able, so a fd holder can fault its +* struct-page-less device memory in from the host CPU. Register it with +* memory_failure() to contain a memory error. -EOPNOTSUPP means +* CONFIG_MEMORY_FAILURE is off, so run without containment. +*/ + ret =vfio_cxl_register_pfn_space (vdev); +if (ret && ret != -EOPNOTSUPP ) + goto err_index; +return 0 ; + +err_index: +/* +* A failed open never reaches close_device, and vfio-pci-core frees +* the regions: forget the index, or a later revoke of the HDM gate +* (runtime suspend of the closed device) names a region that is gone. +*/ + cxl->mem_region_index =0 ; +return ret; }static void vfio_cxl_close_device (struct vfio_pci_core_device *vdev) { +struct vfio_cxl_state *cxl = vdev->cxl; + +unregister_pfn_address_space (&cxl->hdm_pfn_space);/* vfio-pci-core frees the regions after this returns. */ - vdev->cxl->mem_region_index =0 ; + cxl->mem_region_index =0 ; }static void vfio_cxl_reset_prepare (struct vfio_pci_core_device *vdev)
Signature · 1 line
2.53.0
PATCH v6 24/31 vfio/cxl: Expose the HDM decoder registers read-only to the guest
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL Type-2 guest reads the HDM decoder registers to learn the HDM region it was handed. Those registers live in the component BAR that vfio-pci owns.
Map the decoder block at bind: its location comes from the pdev->hdm enumeration cache, and vfio-pci owns the BAR, so map it without claiming the block and expose it as a second, read-only region under VFIO_REGION_TYPE_PCI_VENDOR_TYPE with the CXL vendor id, registered per open like the HDM memory region.
Serve reads live from the mapped block and absorb writes without forwarding them to hardware. The block is BAR MMIO, so it decodes only with Memory Space enabled: serve reads under memory_lock and fail them with -EIO while memory is disabled, as vfio_pci_bar_rw() does. A failed decoder restore leaves Memory Space off; clear vfio's record to match.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 123 ++++++++++++++++++++++++++- include/uapi/linux/vfio.h | 2 + 2 files changed, 123 insertions(+), 2 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +121 -2
@@ -29,6 +29,8 @@* @hpa_res: exclusive reservation of @hpa_range, held for the bind * @hdm: copy of the HDM state cached at PCI enumeration, taken at bind * @hdm_pfn_space: HDM-region pfn range registered with memory_failure() +* @hdm_regs: mapped HDM decoder registers, read live by the decoder region +* @hdm_len: length of the HDM decoder register block * @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock * @await_mem_active: the device reported its memory active at bind, so wait * for Memory_Active before host access reopens after a reset or low power
@@ -41,6 +43,8 @@ struct vfio_cxl_state {struct resource *hpa_res;struct cxl_hdm_info *hdm;struct pfn_address_space hdm_pfn_space; +void __iomem *hdm_regs; +u32 hdm_len;bool hdm_valid;bool await_mem_active;unsigned int mem_region_index;
@@ -214,6 +218,76 @@ static const struct vfio_pci_regops vfio_cxl_mem_regops = { .release = vfio_cxl_region_release, }; +static ssize_t vfio_cxl_comp_rw (struct vfio_pci_core_device *vdev, +char __user *buf,size_t count,loff_t *ppos, +bool iswrite) +{ +struct vfio_cxl_state *cxl = vdev->cxl; +loff_t pos = *ppos &VFIO_PCI_OFFSET_MASK ; +__le32 *tmp; + +if (pos >= cxl->hdm_len) +return -EINVAL ; + +/* +* Component registers allow only full-width accesses: a 32-bit +* register as one 4-byte read or write (CXL r4.0 sec 8.2.3). +*/ +if (pos %sizeof (u32) || count %sizeof (u32)) +return -EINVAL ; + + count =min_t (size_t, count, cxl->hdm_len - pos); + +/* +* The physical decoder was committed before the guest saw the device +* (firmware may also have locked it), so the guest never drives it: +* absorb writes without forwarding them to hardware. The guest programs +* a GPA that the VMM virtualizes; reads return the live registers, which +* already report the decoder committed. BASE_LOW and BASE_HIGH carry the +* host HPA, visible only to the trusted VMM that virtualizes it away from +* the guest. +*/ +if (iswrite) { + *ppos += count; +return count; + } + + tmp =kmalloc (count,GFP_KERNEL ); +if (!tmp) +return -ENOMEM ; + +/* The block is BAR MMIO, so it decodes only with memory enabled. */ + scoped_guard(rwsem_read, &vdev->memory_lock) { +if (!__vfio_pci_memory_enabled (vdev)) { +kfree (tmp); +return -EIO ; + } +/* +* One readl() per register: memcpy_fromio() may split the +* copy into byte or qword reads, and the decoder registers +* take only aligned dword accesses. Return them in the +* register byte order, as a raw copy would. +*/ +for (size_t i =0 ; i < count /sizeof (u32); i++) + tmp[i] =cpu_to_le32 (readl (cxl->hdm_regs + pos + + i *sizeof (u32))); + } + +if (copy_to_user (buf, tmp, count)) { +kfree (tmp); +return -EFAULT ; + } +kfree (tmp); + + *ppos += count; +return count; +} + +static const struct vfio_pci_regops vfio_cxl_comp_regops = { + .rw = vfio_cxl_comp_rw, + .release = vfio_cxl_region_release, +}; +/* * Map a poisoned HDM-region pfn back to the file offset of each user mapping so * memory_failure() can find the mapping and signal its owner. The region is a
@@ -318,6 +392,20 @@ vfio_cxl_create_memdev(struct pci_dev *pdev, const struct cxl_hdm_info *hdm,return ERR_PTR (-ENODEV ); } +/* +* Map the HDM decoder registers so the decoder region can read them +* live. The block location comes from the enumeration cache in +* pdev->hdm, read through the copy in hdm; vfio-pci owns the BAR, so map +* without claiming the block. +*/ + cxl->hdm_regs =devm_ioremap (&pdev->dev, +pci_resource_start (pdev, hdm->hdm_bar) + + hdm->hdm_offset, hdm->hdm_size); +if (!cxl->hdm_regs) +return ERR_PTR (-ENOMEM ); + + cxl->hdm_len = hdm->hdm_size; +/* * A Type-2 accelerator has no mailbox and no media-ready register, so * set media ready directly.
@@ -542,6 +630,13 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)/* Record where the HDM memory region landed, to revoke it later. */ cxl->mem_region_index =VFIO_PCI_NUM_REGIONS + vdev->num_regions -1 ; + ret =vfio_cxl_add_region (vdev,VFIO_REGION_SUBTYPE_CXL_COMP_REGS , + &vfio_cxl_comp_regops, cxl->hdm_len, +VFIO_REGION_INFO_FLAG_READ | +VFIO_REGION_INFO_FLAG_WRITE ); +if (ret) + goto err_index; +/* * The HDM region is advertised mmap-able, so a fd holder can fault its * struct-page-less device memory in from the host CPU. Register it with
@@ -609,6 +704,28 @@ static void vfio_cxl_rearm_hdm(struct vfio_pci_core_device *vdev)vfio_pci_core_rearm_region (vdev, cxl->mem_region_index); } +/* +* A CXL Reset or decoder restore that fails leaves Memory Space and Bus +* Master disabled (cxl_reset_save_disabled_state()). The user reads Command +* from the device, but vfio-pci-core gates BAR access, BAR mmap faults, BAR +* dma-bufs and the decoder register region on its own record of Memory +* Space. Match that record to the device, zapping the BAR mmaps and +* revoking the BAR dma-bufs as a user clear would; the user's next Memory +* Space write sets it again. +*/ +static void vfio_cxl_sync_mem_enable (struct vfio_pci_core_device *vdev) +{ +u16 cmd; + +lockdep_assert_held_write (&vdev->memory_lock); + +if (!pci_read_config_word (vdev->pdev,PCI_COMMAND , &cmd) && + !PCI_POSSIBLE_ERROR (cmd) && (cmd &PCI_COMMAND_MEMORY )) +return ; + +vfio_pci_core_clear_memory_enable (vdev); +} +/* * vfio-cxl holds the HDM range, so the PCI core's cxl_reset method would * fail to request it. Run the same CXL Reset with that claim.
@@ -652,10 +769,12 @@ static void vfio_cxl_power_up(struct vfio_pci_core_device *vdev) ret =cxl_restore_state_after_pci_reset (pdev, cxl->hdm);pci_dev_unlock (pdev); -if (ret) +if (ret) {pci_warn (pdev,"vfio-cxl: HDM decoder restore failed: %d\n" , ret); -else if (!vfio_cxl_await_mem_active (vdev)) +vfio_cxl_sync_mem_enable (vdev); + }else if (!vfio_cxl_await_mem_active (vdev)) {vfio_cxl_rearm_hdm (vdev); + } }static const struct vfio_cxl_ops vfio_cxl_ops = {
include/uapi/linux/vfio.h +2 -0
@@ -373,6 +373,8 @@ struct vfio_region_info_cap_type {/* CXL Type-2 device (0x1e98) sub-types for VFIO_REGION_TYPE_PCI_VENDOR_TYPE */ /* CXL.mem HDM region of a Type-2 device, read, write and mmap */ #define VFIO_REGION_SUBTYPE_CXL_MEM (1) +/* CXL HDM decoder registers: read live, guest writes are absorbed */ +#define VFIO_REGION_SUBTYPE_CXL_COMP_REGS (2)/* sub-types for VFIO_REGION_TYPE_GFX */ #define VFIO_REGION_SUBTYPE_GFX_EDID (1)
Signature · 1 line
2.53.0
PATCH v6 25/31 vfio/cxl: Exclude the HDM decoder registers from direct BAR access
In reply to [email protected]From: Manish Honap <[email protected]>
The HDM decoder register block lives in the component BAR that vfio-pci owns. The guest reads the decoder only through the trapped, read-only decoder region; it must not reach the physical registers through the BAR directly, and a write through the BAR could uncommit or move an unlocked decoder that the host relies on.
Add the block to the generic excluded-range list with VFIO_PCI_EXCLUDE_MMAP | VFIO_PCI_EXCLUDE_READ | VFIO_PCI_EXCLUDE_WRITE similar to the MSI-X exclusion.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
drivers/vfio/pci/cxl/vfio_cxl_core.c +16 -0
@@ -614,6 +614,22 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)struct vfio_cxl_state *cxl = vdev->cxl;int ret; +/* +* Keep the HDM decoder block out of the guest's direct BAR access: the +* guest reaches it only through the trapped decoder region, and a write +* through the BAR could uncommit or move an unlocked decoder that the +* host relies on. Exclude it from mmap, fill reads with -1 and drop +* writes. The list is cleared on close, so re-add it per open. +*/ + ret =vfio_pci_core_add_excluded_range (vdev, cxl->hdm->hdm_bar, + cxl->hdm->hdm_offset, + cxl->hdm_len, +VFIO_PCI_EXCLUDE_MMAP | +VFIO_PCI_EXCLUDE_READ | +VFIO_PCI_EXCLUDE_WRITE ); +if (ret) +return ret; +/* * vfio-pci-core frees all dynamic regions on close and on a failed * open, so register them here per open rather than at bind.
Signature · 1 line
2.53.0
PATCH v6 26/31 vfio/cxl: Restore the HDM decoder after a hot reset
In reply to [email protected]From: Manish Honap <[email protected]>
The HDM memory fault inserts a PFN only when the decoder is in a known-good state (cxl->hdm_valid), so the host CPU never faults the coherent range into a disabled decoder.
VFIO_DEVICE_PCI_HOT_RESET drives a plain secondary bus reset rather than the CXL reset sequence. A secondary bus reset only reaches a CXL endpoint when the upstream port has SBR unmasked (CXL r4.0 sec 8.1.5.2). With SBR masked the PCI core now refuses the bus reset, so the hot reset fails instead of doing nothing. With SBR unmasked the reset is a Hot Reset, which returns the decoder's RWL fields, Commit included, to their defaults and decommits it (CXL r4.0 sec 8.2.4.20.7, Table 8-1).
Take the HDM memory region out of use in reset_prepare. In reset_done, when the port has SBR unmasked and the decoder is no longer committed, restore it with cxl_restore_state_after_pci_reset(), the helper the PCI core's cxl_bus reset method uses, and put the region back. A decoder that is still committed, after an FLR or a cxl_bus reset that restored it itself, is left alone. If the restore fails, reset_done returns the error, the hot reset ioctl fails with it, and the region stays out of use until a reset succeeds. reset_done also runs after a reset method that failed and after resets that do not reach the decoder; then put the region back only if it was in use when reset_prepare ran. Before putting the region back, reset_done waits for Memory_Active as power_up does, and returns -ETIMEDOUT if the memory does not come back.
AER and DPC recovery can reset the link without reset_done. The HDM region is taken out of use when the error is reported and put back when recovery resumes the device only if it was in use before and the decoder is still committed; recovery does not restore config space, so otherwise the region stays out of use until the user resets the device. The resume path also waits for Memory_Active before it puts the region back.
vfio_cxl_sbr_unmasked() mirrors the cxl_sbr_masked() check the PCI core uses (drivers/pci/pci.c) for its own CXL bus reset. power_up now also skips the restore when the decoder is still committed, as after a D3 request that left the device in D0.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 156 +++++++++++++++++++++++++-- 1 file changed, 147 insertions(+), 9 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +147 -9
@@ -18,6 +18,7 @@#include <linux/uaccess.h> #include <linux/vfio_pci_core.h> #include <cxl/cxl.h> +#include <cxl/cxl_regs.h> #include <cxl/hdm.h> #include <cxl/pci.h>
@@ -35,6 +36,7 @@* @await_mem_active: the device reported its memory active at bind, so wait * for Memory_Active before host access reopens after a reset or low power * @mem_region_index: vfio region index of the HDM memory region while open, else 0 +* @hdm_open_at_prepare: @hdm_valid as reset_prepare found it; under memory_lock */ struct vfio_cxl_state {struct cxl_dev_state cxlds;
@@ -48,6 +50,7 @@ struct vfio_cxl_state {bool hdm_valid;bool await_mem_active;unsigned int mem_region_index; +bool hdm_open_at_prepare; }; static unsigned long vfio_cxl_mem_pgoff(struct vm_area_struct *vma,
@@ -684,15 +687,6 @@ static void vfio_cxl_close_device(struct vfio_pci_core_device *vdev) cxl->mem_region_index =0 ; } -static void vfio_cxl_reset_prepare (struct vfio_pci_core_device *vdev) -{ -} - -static int vfio_cxl_reset_done (struct vfio_pci_core_device *vdev) -{ -return 0 ; -} -/* * Take the HDM memory region out of use: zap its mmaps and revoke its * dma-bufs. A fault then gets SIGBUS from vfio_cxl_mem_huge_fault() until
@@ -756,6 +750,33 @@ static int vfio_cxl_reset(struct vfio_pci_core_device *vdev)return pci_cxl_reset_owned (vdev->pdev, vdev->cxl->hpa_res); } +/* +* Report whether the device still decodes its HDM range: memory decode on, +* CXL.mem enabled in the CXL Device DVSEC and the decoder committed. A +* secondary bus reset or a link down clears all three. Config space is read +* first; the decoder registers sit in a BAR and are read only while memory +* decode is on. +*/ +static bool vfio_cxl_hdm_intact (struct vfio_pci_core_device *vdev) +{ +struct vfio_cxl_state *cxl = vdev->cxl; +struct pci_dev *pdev = vdev->pdev; +u16 cmd, ctrl; +u32 dctrl; + +if (pci_read_config_word (pdev,PCI_COMMAND , &cmd) || +PCI_POSSIBLE_ERROR (cmd) || !(cmd &PCI_COMMAND_MEMORY )) +return false; + +if (pci_read_config_word (pdev, cxl->hdm->dvsec +PCI_DVSEC_CXL_CTRL , + &ctrl) || +PCI_POSSIBLE_ERROR (ctrl) || !(ctrl &PCI_DVSEC_CXL_MEM_ENABLE )) +return false; + + dctrl =readl (cxl->hdm_regs +CXL_HDM_DECODER0_CTRL_OFFSET (0 )); +return dctrl !=U32_MAX && (dctrl &CXL_HDM_DECODER0_CTRL_COMMITTED ); +} +/* Low power loses the committed decoder: close host access until restored. */ static void vfio_cxl_power_down (struct vfio_pci_core_device *vdev) {
@@ -770,6 +791,16 @@ static void vfio_cxl_power_up(struct vfio_pci_core_device *vdev)lockdep_assert_held_write (&vdev->memory_lock); +/* +* A D3 request that failed, or a quirk that kept the device in D0, +* lost nothing: reopen without decommitting and recommitting. +*/ +if (vfio_cxl_hdm_intact (vdev)) { +if (!vfio_cxl_await_mem_active (vdev)) +vfio_cxl_rearm_hdm (vdev); +return ; + } +/* * Restore the decoder that low power lost. The restore needs the device * lock; if it is contended, keep host access closed until a reset
@@ -793,6 +824,112 @@ static void vfio_cxl_power_up(struct vfio_pci_core_device *vdev) } } +/* A PCI core reset may clear the decoder: take the HDM region out of use. */ +static void vfio_cxl_reset_prepare (struct vfio_pci_core_device *vdev) +{ + vdev->cxl->hdm_open_at_prepare = vdev->cxl->hdm_valid; +vfio_cxl_revoke_hdm (vdev); +} + +/* +* A secondary bus reset only reaches a CXL endpoint when the upstream port +* has SBR unmasked (CXL r4.0 sec 8.1.5.2); otherwise the Bridge Control SBR +* bit is ignored and the decoder is left intact. Mirrors the cxl_sbr_masked() +* check the PCI core uses for its own CXL bus reset. +*/ +static bool vfio_cxl_sbr_unmasked (struct pci_dev *pdev) +{ +struct pci_dev *bridge =pci_upstream_bridge (pdev); +u16 dvsec, ctl; + +if (!bridge) +return false; + + dvsec =pci_find_dvsec_capability (bridge,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_PORT ); +if (!dvsec) +return false; + +if (pci_read_config_word (bridge, dvsec +PCI_DVSEC_CXL_PORT_CTL , &ctl)) +return false; + +return ctl &PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR ; +} + +static int vfio_cxl_reset_done (struct vfio_pci_core_device *vdev) +{ +struct vfio_cxl_state *cxl = vdev->cxl; +struct pci_dev *pdev = vdev->pdev; +int ret; + +lockdep_assert_held_write (&vdev->memory_lock); + +/* +* VFIO_DEVICE_PCI_HOT_RESET drives a plain secondary bus reset, not the +* CXL-aware cxl_reset_bus_function(). When the upstream port has SBR +* unmasked the reset decommits the decoder. The PCI core has restored +* config space by now; restore the decoder with the same helper its +* cxl_bus reset method uses. A masked port refuses the bus reset. Other +* resets (FLR, the cxl_reset and cxl_bus methods) leave the decoder +* committed or restore it themselves; restore only a decoder that is +* gone, so a committed one is not decommitted and committed again. +*/ +if (vfio_cxl_sbr_unmasked (pdev) && !vfio_cxl_hdm_intact (vdev)) { + ret =pci_save_state (pdev); +if (!ret) + ret =cxl_restore_state_after_pci_reset (pdev, cxl->hdm); +if (ret) { +pci_warn (pdev,"vfio-cxl: HDM decoder restore failed: %d\n" , + ret); +vfio_cxl_sync_mem_enable (vdev); +return ret; + } + }else if (!cxl->hdm_open_at_prepare) { +/* +* reset_done runs even when the reset method failed. Nothing +* restored the decoder, and it was not known good before this +* reset (a failed reset or a skipped restore closed the gate), +* so keep host access closed. +*/ +return 0 ; + } + + ret =vfio_cxl_await_mem_active (vdev); +if (ret) +return ret; + +vfio_cxl_rearm_hdm (vdev); +return 0 ; +} + +/* +* AER or DPC recovery resumes the device without restoring its config space, +* so do not restore the decoder here. Put the HDM region back only if the +* device still decodes it; otherwise the user's reset restores it. +*/ +static void vfio_cxl_error_resume (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; + +lockdep_assert_held_write (&vdev->memory_lock); + +/* Low power closed the region; vfio_cxl_power_up() reopens it. */ +if (pdev->current_state !=PCI_D0 ) +return ; + +/* The region was closed before the error; keep it that way. */ +if (!vdev->cxl->hdm_open_at_prepare) +return ; + +if (!vfio_cxl_hdm_intact (vdev)) { +pci_warn (pdev,"vfio-cxl: HDM decoder lost in error recovery, reset the device to restore it\n" ); +return ; + } + +if (!vfio_cxl_await_mem_active (vdev)) +vfio_cxl_rearm_hdm (vdev); +} +static const struct vfio_cxl_ops vfio_cxl_ops = { .init = vfio_cxl_init_device, .release = vfio_cxl_release_device,
@@ -800,6 +937,7 @@ static const struct vfio_cxl_ops vfio_cxl_ops = { .close_device = vfio_cxl_close_device, .reset_prepare = vfio_cxl_reset_prepare, .reset_done = vfio_cxl_reset_done, + .error_resume = vfio_cxl_error_resume, .reset = vfio_cxl_reset, .power_down = vfio_cxl_power_down, .power_up = vfio_cxl_power_up,
Signature · 1 line
2.53.0
PATCH v6 27/31 vfio/cxl: Describe the CXL device and decoder geometry to userspace
In reply to [email protected]From: Manish Honap <[email protected]>
Advertise the assignment with a device-info flag so a VMM can tell a CXL Type-2 device apart from a device bound without the CXL extensions.
On the trapped decoder register region, add a capability carrying the BAR and offset of the HDM decoder block so the VMM can place the trapped window where the guest expects to find it.
The size of the guest's memory window comes from the HDM memory region.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 16 ++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 3 +++ include/uapi/linux/vfio.h | 18 ++++++++++++++++++ 3 files changed, 37 insertions(+)
drivers/vfio/pci/cxl/vfio_cxl_core.c +16 -0
@@ -286,9 +286,25 @@ static ssize_t vfio_cxl_comp_rw(struct vfio_pci_core_device *vdev,return count; } +static int vfio_cxl_comp_add_capability (struct vfio_pci_core_device *vdev, +struct vfio_pci_region *region, +struct vfio_info_cap *caps) +{ +struct cxl_hdm_info *hdm = vdev->cxl->hdm; +struct vfio_region_info_cap_cxl_comp_regs cap = { + .header.id =VFIO_REGION_INFO_CAP_CXL_COMP_REGS , + .header.version =1 , + .bar = hdm->hdm_bar, + .offset = hdm->hdm_offset, + }; + +return vfio_info_add_capability (caps, &cap.header,sizeof (cap)); +} +static const struct vfio_pci_regops vfio_cxl_comp_regops = { .rw = vfio_cxl_comp_rw, .release = vfio_cxl_region_release, + .add_capability = vfio_cxl_comp_add_capability, };/*
drivers/vfio/pci/vfio_pci_core.c +3 -0
@@ -1391,6 +1391,9 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_core_device *vdev,if (vdev->reset_works) info.flags |=VFIO_DEVICE_FLAGS_RESET ; +if (vdev->cxl_ops) + info.flags |=VFIO_DEVICE_FLAGS_CXL ; + info.num_regions =VFIO_PCI_NUM_REGIONS + vdev->num_regions; info.num_irqs =VFIO_PCI_NUM_IRQS ;
include/uapi/linux/vfio.h +18 -0
@@ -215,6 +215,7 @@ struct vfio_device_info {#define VFIO_DEVICE_FLAGS_FSL_MC (1 << 6)/* vfio-fsl-mc device */ #define VFIO_DEVICE_FLAGS_CAPS (1 << 7)/* Info supports caps */ #define VFIO_DEVICE_FLAGS_CDX (1 << 8)/* vfio-cdx device */ +#define VFIO_DEVICE_FLAGS_CXL (1 << 9)/* vfio-cxl device */ __u32 num_regions;/* Max region index + 1 */ __u32 num_irqs;/* Max IRQ index + 1 */ __u32 cap_offset;/* Offset within info struct of first cap */
@@ -503,6 +504,23 @@ struct vfio_region_info_cap_nvlink2_lnkspd {__u32 __pad; }; +/* +* Geometry of a CXL Type-2 device's HDM decoder registers, so a VMM can place +* the trapped component register window where the guest expects it. The trapped +* region spans the whole HDM decoder block (every decoder), not just decoder 0: +* a VMM reads the decoder count and each decoder's committed base from the block +* itself. Additional trapped component capabilities, such as CXL RAS, are +* exposed as their own region subtypes rather than by extending this cap. +*/ +#define VFIO_REGION_INFO_CAP_CXL_COMP_REGS 6 + +struct vfio_region_info_cap_cxl_comp_regs { +struct vfio_info_cap_header header; +__u32 bar; +__u32 __resv; +__aligned_u64 offset; +}; +/** * VFIO_DEVICE_GET_IRQ_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 9, * struct vfio_irq_info)
Signature · 1 line
2.53.0
PATCH v6 28/31 vfio/cxl: Export the HDM memory region as a dma-buf
In reply to [email protected]From: Manish Honap <[email protected]>
A Type-2 accelerator issues ATS-translated DMA to addresses inside its own HDM window, so that coherent host range must be present in the guest's IOAS (the iommufd IOAS backing the nested SMMU stage-2). iommufd maps a struct-page-less range only by fd, via IOMMU_IOAS_MAP_FILE over a dma-buf; a userspace-VA IOMMU_IOAS_MAP of the HDM mmap is rejected because the VMA is VM_IO | VM_PFNMAP. Without a dma-buf the range could only be mapped through an out-of-tree PFNMAP work-around.
vfio-pci already exports BAR memory as a P2P dma-buf, but the exporter is BAR-only: vfio_pci_core_feature_dma_buf() rejects any region index at or above the ROM index, and vfio_pci_core_get_dmabuf_phys() resolves the physical range from a PCI BAR. The HDM memory region is a dynamic device-specific region, not a BAR.
Let a device-specific region reach the device's get_dmabuf_phys(): a region index at or above VFIO_PCI_NUM_REGIONS skips the BAR-resource check and is validated by the driver instead, bounded to the regions that exist. Install a CXL-aware get_dmabuf_phys() in the vfio-cxl provider that returns cxl->hpa_range for the HDM memory region and delegates real BARs to the core, keeping the BAR path unchanged and the core free of CXL knowledge.
The HDM window is coherent host memory with no p2pdma provider of its own, so borrow BAR 0's, matching nvgrace-gpu's handling of its non-BAR device memory. The iommufd importer does not consume the provider; the scatterlist map path (real peer DMA) is left to a follow-up once upstream grows a negotiated interconnect for coherent CXL memory. Mark a device-specific dma-buf so vfio_pci_dma_buf_map() refuses that path, which would translate the range with the borrowed provider's bus offset.
The HDM dma-buf belongs to its region, not to the BARs: a memory enable change leaves it alone, and vfio-cxl revokes and re-arms it with the region when the decoder state changes. A new dma-buf of a device-specific region starts in the state its region reports through a new optional revoked() region op, so an HDM export made with memory disabled starts armed, and one made after a failed reset starts revoked.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 76 +++++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_dmabuf.c | 73 +++++++++++++++++++++++--- include/linux/vfio_pci_core.h | 9 ++++ 3 files changed, 151 insertions(+), 7 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +75 -1
@@ -13,6 +13,7 @@#include <linux/mm.h> #include <linux/module.h> #include <linux/pci.h> +#include <linux/pci-p2pdma.h> #include <linux/range.h> #include <linux/slab.h> #include <linux/uaccess.h>
@@ -206,6 +207,21 @@ static ssize_t vfio_cxl_mem_rw(struct vfio_pci_core_device *vdev,return done; } +/* +* The HDM region follows hdm_valid, not Memory Space Enable, so a dma-buf +* exported from it starts revoked while host access is unsafe and is +* re-armed with the region in vfio_cxl_rearm_hdm(). +*/ +static bool vfio_cxl_mem_revoked (struct vfio_pci_core_device *vdev, +struct vfio_pci_region *region) +{ +struct vfio_cxl_state *cxl = vdev->cxl; + +lockdep_assert_held_write (&vdev->memory_lock); + +return !cxl->hdm_valid || !cxl->cxlds.media_ready; +} +/* * The CXL regions carry no per-region state (region->data is the shared, * devm-managed vfio_cxl_state), so releasing a region is a no-op.
@@ -219,6 +235,7 @@ static const struct vfio_pci_regops vfio_cxl_mem_regops = { .rw = vfio_cxl_mem_rw, .mmap = vfio_cxl_mem_mmap, .release = vfio_cxl_region_release, + .revoked = vfio_cxl_mem_revoked, }; static ssize_t vfio_cxl_comp_rw(struct vfio_pci_core_device *vdev,
@@ -443,6 +460,50 @@ vfio_cxl_create_memdev(struct pci_dev *pdev, const struct cxl_hdm_info *hdm,return cxl; } +/* +* Resolve the physical range that backs a dma-buf export. The core exporter +* only knows BARs; teach it the HDM memory region so a guest IOAS can map the +* coherent window by fd (IOMMU_IOAS_MAP_FILE) instead of the removed PFNMAP +* work-around. Real BARs stay on the byte-identical core path. +*/ +static int vfio_cxl_get_dmabuf_phys (struct vfio_pci_core_device *vdev, +struct p2pdma_provider **provider, +unsigned int region_index, +struct phys_vec *phys_vec, +struct vfio_region_dma_range *dma_ranges, +size_t nr_ranges) +{ +struct vfio_cxl_state *cxl = vdev->cxl; + +/* Real BARs go through the core P2P exporter unchanged. */ +if (region_index <VFIO_PCI_NUM_REGIONS ) +return vfio_pci_core_get_dmabuf_phys (vdev, provider, + region_index, phys_vec, + dma_ranges, nr_ranges); + +/* Of the device regions, only the HDM memory window is exportable. */ +if (region_index != cxl->mem_region_index) +return -EINVAL ; + +/* +* The HDM window is coherent host memory, not BAR MMIO, so it has no +* p2pdma provider of its own. Borrow BAR 0's: the P2P properties match +* and the iommufd importer does not consume the provider. The sgt map +* path (real peer DMA) is not supported for the HDM window. +*/ + *provider =pcim_p2pdma_provider (vdev->pdev,0 ); +if (!*provider) +return -EINVAL ; + +return vfio_pci_core_fill_phys_vec (phys_vec, dma_ranges, nr_ranges, + cxl->hpa_range.start, +range_len (&cxl->hpa_range)); +} + +static const struct vfio_pci_device_ops vfio_cxl_pci_dev_ops = { + .get_dmabuf_phys = vfio_cxl_get_dmabuf_phys, +}; +/* CXL r4.0 sec 8.1.3.8.2: Range 1 Size Low Memory_Active_Timeout encodings */ static const unsigned int vfio_cxl_mem_active_timeout_s[] = {1 ,4 ,16 ,64 ,256 };
@@ -662,9 +723,22 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)if (ret)return ret; -/* Record where the HDM memory region landed, to revoke it later. */ +/* +* Record where the HDM memory region landed, to revoke it and to +* resolve its dma-buf export. +*/ cxl->mem_region_index =VFIO_PCI_NUM_REGIONS + vdev->num_regions -1 ; +/* +* Override the device ops so a dma-buf export of the HDM memory region +* resolves to the coherent host range. This is done at open, not init: +* vfio_pci_probe() resets pci_ops after vfio_alloc_device() returns, so +* an override installed during init would be clobbered. Only a CXL device +* reaches this hook (cxl_ops is set on init success), so a device bound +* without CXL support keeps the core ops. +*/ + vdev->pci_ops = &vfio_cxl_pci_dev_ops; + ret = vfio_cxl_add_region(vdev, VFIO_REGION_SUBTYPE_CXL_COMP_REGS, &vfio_cxl_comp_regops, cxl->hdm_len, VFIO_REGION_INFO_FLAG_READ |
drivers/vfio/pci/vfio_pci_dmabuf.c +67 -6
130 lines not on this page · raw message
include/linux/vfio_pci_core.h +9 -0
@@ -47,6 +47,15 @@ struct vfio_pci_regops {int (*add_capability)(struct vfio_pci_core_device *vdev,struct vfio_pci_region *region,struct vfio_info_cap *caps); +/* +* Optional, for a device-specific region exported as a dma-buf. +* Called with memory_lock held for write; returns true while the +* region is revoked, so a new dma-buf starts in the state that +* vfio_pci_core_revoke_region() and vfio_pci_core_rearm_region() +* maintain. Without it, the dma-buf starts armed. +*/ +bool (*revoked)(struct vfio_pci_core_device *vdev, +struct vfio_pci_region *region); };struct vfio_pci_region {
Signature · 1 line
2.53.0
PATCH v6 29/31 vfio/cxl: Run the CXL reset at the vfio reset points
In reply to [email protected]From: Manish Honap <[email protected]>
A CXL Type-2 function must not take an FLR in place of CXL Reset: an FLR leaves CXL.cache and CXL.mem state, the CXL DVSEC and the HDM contents untouched (CXL r4.0 sec 9.5), so device memory and cache would carry one user's data to the next. CXL Reset resets that state as well (sec 9.7). The PCI core orders cxl_reset ahead of flr in pci_reset_fn_methods[], but falls through to FLR when cxl_reset returns -ENOTTY.
vfio-pci-core routes its reset points through the provider reset op. In vfio-cxl that op runs pci_cxl_reset_owned() with the HDM range claim. The reset resets the function, clearing or randomizing volatile device memory, and restores the HDM decoder and PCI config state, so it is a complete replacement for pci_try_reset_function() on a CXL device. The op runs with memory_lock and the PCI device lock held.
Take the HDM memory region out of use for the duration of the reset, so a fault cannot insert a PFN into a decoder that is being torn down, and put it back only once the reset has restored the decoder and the device reports its memory active again. After CXL Reset with Memory Clear the device can keep Memory_Active clear for seconds (about 3 s on the device this was tested on), and a host access to the HDM range in that window raised an SError. A reset whose memory does not come back within the Memory_Active_Timeout fails like any other. A failed reset returns its error and leaves the region out of use until a later CXL Reset succeeds. Neither a decoder restore after low power nor a hot reset reopens it in between, since device memory may still hold data the reset was meant to clear. If a failed reset left CXL.cache or CXL.mem disabled in the DVSEC Control register, which makes cxl_reset refuse the device, put back the Control value taken at bind before the next reset. A failed reset leaves Memory Space disabled; clear vfio-pci-core's record of it to match. AER recovery does not reopen HDM after a failed CXL Reset either.
--- drivers/vfio/pci/cxl/vfio_cxl_core.c | 82 ++++++++++++++++++++++++++-- 1 file changed, 77 insertions(+), 5 deletions(-)
drivers/vfio/pci/cxl/vfio_cxl_core.c +77 -5
@@ -38,6 +38,8 @@* for Memory_Active before host access reopens after a reset or low power * @mem_region_index: vfio region index of the HDM memory region while open, else 0 * @hdm_open_at_prepare: @hdm_valid as reset_prepare found it; under memory_lock +* @hdm_needs_reset: a CXL Reset failed, so only a CXL Reset that succeeds +* reopens host access; under memory_lock */ struct vfio_cxl_state {struct cxl_dev_state cxlds;
@@ -52,6 +54,7 @@ struct vfio_cxl_state {bool await_mem_active;unsigned int mem_region_index;bool hdm_open_at_prepare; +bool hdm_needs_reset; }; static unsigned long vfio_cxl_mem_pgoff(struct vm_area_struct *vma,
@@ -827,17 +830,75 @@ static void vfio_cxl_sync_mem_enable(struct vfio_pci_core_device *vdev) }/* -* vfio-cxl holds the HDM range, so the PCI core's cxl_reset method would -* fail to request it. Run the same CXL Reset with that claim. +* cxl_reset refuses a device whose CXL DVSEC Control has Cache_Enable or +* Mem_Enable clear (-ENOTTY). A CXL Reset that failed after the device reset +* may leave Control at its reset value, so every later reset would refuse and +* host access would never reopen. Put back the Control value taken at bind, +* when pci_cxl_reset_capable() required both enables, before the reset. +*/ +static void vfio_cxl_restore_dvsec_ctrl (struct vfio_pci_core_device *vdev) +{ +const u16 enables =PCI_DVSEC_CXL_CACHE_ENABLE |PCI_DVSEC_CXL_MEM_ENABLE ; +const struct cxl_hdm_info *hdm = vdev->cxl->hdm; +struct pci_dev *pdev = vdev->pdev; +u16 ctrl; + +if (!hdm->dvsec_ctrl_valid || (hdm->dvsec_ctrl & enables) != enables) +return ; + +if (pci_read_config_word (pdev, hdm->dvsec +PCI_DVSEC_CXL_CTRL , &ctrl) || +PCI_POSSIBLE_ERROR (ctrl) || (ctrl & enables) == enables) +return ; + +pci_warn (pdev,"vfio-cxl: restoring CXL DVSEC control %#06x (was %#06x)\n" , + hdm->dvsec_ctrl, ctrl); +pci_write_config_word (pdev, hdm->dvsec +PCI_DVSEC_CXL_CTRL , + hdm->dvsec_ctrl); +} + +/* +* Run CXL Reset in place of a PCI function reset. An FLR leaves CXL.mem +* state and HDM contents intact (CXL r4.0 sec 9.5), so the vfio reset points +* route here. The reset clears or randomizes volatile device memory and +* restores the HDM decoder. The caller holds memory_lock and the PCI device +* lock. vfio-cxl holds the HDM range, so the PCI core's cxl_reset method +* would fail to request it; run the same CXL Reset with that claim. */ static int vfio_cxl_reset (struct vfio_pci_core_device *vdev) { +struct pci_dev *pdev = vdev->pdev; +int ret; + +/* Host CPU access to the HDM range is unsafe until the decoder is back. */ +vfio_cxl_revoke_hdm (vdev); + +/* Only after a failure: see vfio_cxl_restore_dvsec_ctrl(). */ +if (vdev->cxl->hdm_needs_reset) +vfio_cxl_restore_dvsec_ctrl (vdev); +/* * The reset restores PCI state from the saved copy. The caller provides * it: the try-reset helper saves the current state, and the close path * keeps the open-time state that vfio_pci_core_disable() loaded. */ -return pci_cxl_reset_owned (vdev->pdev, vdev->cxl->hpa_res); + ret =pci_cxl_reset_owned (pdev, vdev->cxl->hpa_res); +if (!ret) + ret =vfio_cxl_await_mem_active (vdev); +if (ret) { +/* +* Device memory may hold data the reset was meant to clear. Only +* a later CXL Reset that succeeds reopens host access; a decoder +* restore alone does not. +*/ + vdev->cxl->hdm_needs_reset = true; +vfio_cxl_sync_mem_enable (vdev); +return ret; + } + + vdev->cxl->hdm_needs_reset = false; +vfio_cxl_rearm_hdm (vdev); + +return 0 ; }/*
@@ -881,6 +942,10 @@ static void vfio_cxl_power_up(struct vfio_pci_core_device *vdev)lockdep_assert_held_write (&vdev->memory_lock); +/* After a failed CXL Reset, only a CXL Reset reopens host access. */ +if (cxl->hdm_needs_reset) +return ; +/* * A D3 request that failed, or a quirk that kept the device in D0, * lost nothing: reopen without decommitting and recommitting.
@@ -984,6 +1049,10 @@ static int vfio_cxl_reset_done(struct vfio_pci_core_device *vdev)return 0 ; } +/* After a failed CXL Reset, only a CXL Reset reopens host access. */ +if (cxl->hdm_needs_reset) +return 0 ; + ret =vfio_cxl_await_mem_active (vdev);if (ret)return ret;
@@ -1007,8 +1076,11 @@ static void vfio_cxl_error_resume(struct vfio_pci_core_device *vdev)if (pdev->current_state !=PCI_D0 )return ; -/* The region was closed before the error; keep it that way. */ -if (!vdev->cxl->hdm_open_at_prepare) +/* +* The region was closed before the error, or a CXL Reset failed while +* recovery ran; keep it closed until a CXL Reset succeeds. +*/ +if (!vdev->cxl->hdm_open_at_prepare || vdev->cxl->hdm_needs_reset)return ;if (!vfio_cxl_hdm_intact (vdev)) {
Signature · 1 line
2.53.0
PATCH v6 30/31 Documentation: vfio-pci: Document CXL Type-2 device passthrough
In reply to [email protected]From: Manish Honap <[email protected]>
Describe the vfio-cxl provider module:
- Address model,
- Two regions,
- Live decoder region and the guest commit,
- DVSEC virtualization,
- The dma-buf export for iommufd stage-2 mapping,
- The reset contract, including the hot reset,
- Low power
so the supported topology and the kernel/VMM split are on record.
--- Documentation/driver-api/index.rst | 1 + Documentation/driver-api/vfio-pci-cxl.rst | 312 ++++++++++++++++++++++ MAINTAINERS | 1 + 3 files changed, 314 insertions(+) create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst
Documentation/driver-api/index.rst +1 -0
@@ -46,6 +46,7 @@ of interest to most developers working on device drivers. uio-howto vfio-mediated-device vfio + vfio-pci-cxl vfio-pci-device-specific-driver-acceptance Bus-level documentation
Documentation/driver-api/vfio-pci-cxl.rst new file +312 -0
312 lines not on this page · raw message
MAINTAINERS +1 -0
@@ -28641,6 +28641,7 @@ M: Manish Honap <[email protected]> L: [email protected] L: [email protected] S: Supported +F: Documentation/driver-api/vfio-pci-cxl.rst F: drivers/vfio/pci/cxl/ VFIO DRIVER
Signature · 1 line
2.53.0
PATCH v6 31/31 selftests/vfio: Add CXL Type-2 passthrough tests
In reply to [email protected]From: Manish Honap <[email protected]>
Exercise the vfio-cxl contract on a bound Type-2 device: the CXL device-info flag and the two vendor-type regions, the HDM memory mmap (including a 2 MB huge fault), the read-only trapped decoder region against the live host-committed decoder, and the guest's DVSEC reset doorbell and manual HDM discovery.
Check that the decoder register region reads fail with EIO while Memory Space is disabled, and that the decoder reads back committed and HDM access works after low power entry and exit.
Map the HDM memory the way a VMM does: export it as a dma-buf with VFIO_DEVICE_FEATURE_DMA_BUF and map the fd into the IOAS with IOMMU_IOAS_MAP_FILE. The struct-page-less range cannot be pinned through a userspace VA, so the by-fd path is the one that works. Skip when the kernel has no CXL dma-buf exporter, so the test becomes a real pass once that support is present. Check that a dma-buf exported with Memory Space disabled maps once it is enabled, that read and write on the HDM region span several pages and may target a mapping of the region itself, and that a multi-dword read of the decoder registers matches single-dword reads.
Also check that close restores the config state saved at open, and that a D3hot to D0 cycle restores the decoder, also when the device stays in D0. The sparse mmap test accepts zero areas when the page-rounded decoder hole covers the whole BAR. The multi-chunk read and write test is sized in pages of the running kernel.
Add a partial-mmap path to the vfio selftest library so the trapped component BAR maps its sparse ranges, and an __iommu_map_file() helper for the dma-buf mapping.
--- MAINTAINERS | 1 + tools/testing/selftests/vfio/Makefile | 1 + .../vfio/lib/include/libvfio/iommu.h | 3 + tools/testing/selftests/vfio/lib/iommu.c | 29 + .../selftests/vfio/lib/vfio_pci_device.c | 57 +- .../selftests/vfio/vfio_cxl_type2_test.c | 1095 +++++++++++++++++ 6 files changed, 1183 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/vfio/vfio_cxl_type2_test.c
MAINTAINERS +1 -0
@@ -28643,6 +28643,7 @@ L: [email protected] S: Supported F: Documentation/driver-api/vfio-pci-cxl.rst F: drivers/vfio/pci/cxl/ +F: tools/testing/selftests/vfio/vfio_cxl_type2_test.c VFIO DRIVER M: Alex Williamson <[email protected]>
tools/testing/selftests/vfio/Makefile +1 -0
@@ -13,6 +13,7 @@ TEST_GEN_PROGS += vfio_pci_device_test TEST_GEN_PROGS += vfio_pci_device_init_perf_test TEST_GEN_PROGS += vfio_pci_driver_test TEST_GEN_PROGS += vfio_pci_sriov_uapi_test +TEST_GEN_PROGS += vfio_cxl_type2_test TEST_FILES += scripts/cleanup.sh TEST_FILES += scripts/lib.sh
tools/testing/selftests/vfio/lib/include/libvfio/iommu.h +3 -0
@@ -42,6 +42,9 @@ static inline void iommu_map(struct iommu *iommu, struct dma_region *region)VFIO_ASSERT_EQ (__iommu_map (iommu, region),0 ); } +int __iommu_map_file (struct iommu *iommu,int fd,u64 start,u64 length, +iova_t iova); +int __iommu_unmap (struct iommu *iommu,struct dma_region *region,u64 *unmapped);static inline void iommu_unmap (struct iommu *iommu,struct dma_region *region)
tools/testing/selftests/vfio/lib/iommu.c +29 -0
@@ -149,6 +149,35 @@ int __iommu_map(struct iommu *iommu, struct dma_region *region)return 0 ; } +/* +* Map a range of a file (a memfd or a supported dma-buf, such as a VFIO PCI +* dma-buf from VFIO_DEVICE_FEATURE_DMA_BUF) into the IOAS by fd. This is an +* iommufd-only ioctl; the legacy VFIO container has no equivalent. +*/ +int __iommu_map_file (struct iommu *iommu,int fd,u64 start,u64 length, +iova_t iova) +{ +struct iommu_ioas_map_file args = { + .size =sizeof (args), + .flags =IOMMU_IOAS_MAP_READABLE | +IOMMU_IOAS_MAP_WRITEABLE | +IOMMU_IOAS_MAP_FIXED_IOVA , + .ioas_id = iommu->ioas_id, + .fd = fd, + .start = start, + .length = length, + .iova = iova, + }; + +if (!iommu->iommufd) +return -EINVAL ; + +if (ioctl (iommu->iommufd,IOMMU_IOAS_MAP_FILE , &args)) +return -errno; + +return 0 ; +} +static int __vfio_iommu_unmap (int fd,u64 iova,u64 size,u32 flags,u64 *unmapped) {struct vfio_iommu_type1_dma_unmap args = {
tools/testing/selftests/vfio/lib/vfio_pci_device.c +54 -3
72 lines not on this page · raw message
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +1095 -0
1,095 lines not on this page · raw message
Signature · 1 line
2.53.0
David Matlack
maintainerRe: [PATCH v6 31/31] selftests/vfio: Add CXL Type-2 passthrough tests
Quoted 5 lines · mhonap
Exercise the vfio-cxl contract on a bound Type-2 device: the CXL device-info flag and the two vendor-type regions, the HDM memory mmap (including a 2 MB huge fault), the read-only trapped decoder region against the live host-committed decoder, and the guest's DVSEC reset doorbell and manual HDM discovery.
I tried running this test in a VM with the emulated cxl-type2 device from your QEMU RFC [1], on an x86 q35 machine with intel-iommu. Out of the box the device binds as plain vfio-pci and the fixture skips everything. v6 now requires that the host can do CPU cache invalidation for CXL Reset, and cpu_cache_has_invalidate_memregion() returns false in a VM (x86 refuses it under a hypervisor, and arm64 has no cache maintenance provider in a guest).
The QEMU RFC reports 19/19 passing in an Arm guest, but that was against v5, which didn't have this requirement. How did you run the v6 version of this test without hardware? Did you use something like CONFIG_CXL_REGION_INVALIDATION_TEST? That option only covers region management in drivers/cxl/core/region.c, and depends on CXL_REGION, so it doesn't help here.
I added a local test-only bypass (a CONFIG_CXL_RESET_INVALIDATION_TEST that skips the invalidation in cxl_reset_check() and cxl_hdm_quiesce_prepare()/cxl_hdm_ranges_invalidate()), I get 26 passes and 1 skip (pm_d3hot_restores_decoder, since cxl-type2 has no PM capability in QEMU). I think something like that belongs in the cxl_reset series. Otherwise this test can only be run by people with real Type-2 hardware.
[1] https://lore.kernel.org/linux-cxl/[email protected]/
Quoted 3 lines
Check that the decoder register region reads fail with EIO while Memory Space is disabled, and that the decoder reads back committed and HDM access works after low power entry and exit.
The series says a Type-2 device loses its committed HDM decoder in D3hot and D3cold (patches 17 and 30), but the emulated cxl-type2 device in your QEMU RFC has no PM capability, so on it these tests never take the decoder restore path in vfio_cxl_power_up(). Please make the emulated device behave the same way as the real hardware across D3hot (PM capability, No_Soft_Reset, and whether the decoder is lost), so that the selftest exercises the same vfio-cxl code paths in a VM as it does on hardware.
To help get that right: what does lspci -vv show for the PM capability of the device you tested with (NoSoftRst+ or NoSoftRst-), and did you observe the decoder actually being lost?
Quoted 3 lines
Add a partial-mmap path to the vfio selftest library so the trapped component BAR maps its sparse ranges, and an
__iommu_map_file()helper for the dma-buf mapping.
Please split the library changes into separate commits ahead of the
test: __iommu_map_file(), the sparse mmap support in vfio_pci_bar_map(), and each of the new libvfio helpers suggested below. That way each one can be reviewed on its own, and the final commit only adds the test, its Makefile entry, and the MAINTAINERS line.
This series is already quite long (31 patches), and with the library changes split out the selftest changes will grow to several patches. Consider sending all the VFIO selftests patches as a separate series that depends on this one.
Quoted 8 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c +8
-0+/* Return the sparse-mmap capability in @info, or NULL if the region has none. */ +static struct vfio_region_info_cap_sparse_mmap * +vfio_pci_sparse_mmap_cap (struct vfio_region_info *info) +{ +struct vfio_info_cap_header *hdr; +u32 offset; + +if (!(info->flags &VFIO_REGION_INFO_FLAG_CAPS ))
tools/testing/selftests/vfio/lib/vfio_pci_device.c static void vfio_pci_region_get(struct vfio_pci_device *device, int index, · line 207
Quoted 10 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c +10
-0+return NULL ; + +for (offset = info->cap_offset; offset; offset = hdr->next) { + hdr = (void *)info + offset; +if (hdr->id ==VFIO_REGION_INFO_CAP_SPARSE_MMAP ) +return (struct vfio_region_info_cap_sparse_mmap *)hdr; + } + +return NULL ; +}
This walks the region info capability chain looking for one ID, which is the same thing find_region_cap() does in the test. Please add one generic libvfio helper that looks up a region info capability by ID, and use it in both places.
Quoted 2 lines
+ * Usage: ./vfio_cxl_type2_test <BDF> (or export VFIO_SELFTESTS_BDF=<BDF>).
+ * The device must be bound to vfio-pci with the vfio-cxl module available.
Please document how to run this test in more detail: the hardware requirements, how to set up the emulated QEMU device, and the kernel config needed (VFIO_CXL, IOMMUFD, any test-only bypass), so others can reproduce your results.
tools/testing/selftests/vfio/vfio_cxl_type2_test.c line 58
Quoted 7 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +7
-0+/* vfio-pci region-offset packing is kernel-internal, not UAPI; define locally. */ +#ifndef VFIO_PCI_OFFSET_SHIFT +#define VFIO_PCI_OFFSET_SHIFT 40 +#endif +#ifndef VFIO_PCI_INDEX_TO_OFFSET +#define VFIO_PCI_INDEX_TO_OFFSET (i) ((uint64_t)(i) << VFIO_PCI_OFFSET_SHIFT) +#endif
As the comment says, this is kernel-internal and not uAPI, so the test shouldn't depend on it. Please use vfio_region_info.offset from VFIO_DEVICE_GET_REGION_INFO instead, as comp_bar_sparse_mmap already does for the BAR. Better yet, use region access helpers in libvfio that take a region index and apply the offset internally (see below), so the test never computes region offsets itself.
Quoted 6 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +6
-0+/* Find a CXL region by subtype; returns the region index or -1, @buf left holding its info. */ +static int find_cxl_region (int fd,uint32_t nregions,uint32_t subtype, +void *buf,size_t bufsz) +{ +uint32_t i; +
tools/testing/selftests/vfio/vfio_cxl_type2_test.c line 94
Quoted 10 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +10
-0+for (i =0 ; i < nregions; i++) { +struct vfio_region_info *ri = buf; +const struct vfio_region_info_cap_type *t; +const struct vfio_info_cap_header *hdr; + +memset (buf,0 , bufsz); + ri->argsz = bufsz; + ri->index = i; +if (ioctl (fd,VFIO_DEVICE_GET_REGION_INFO , ri)) +continue ;
This is a generic lookup of a region by type and subtype. Only the CXL vendor ID is specific. Please make it a libvfio helper (e.g. vfio_pci_find_region(device, type, subtype)) built on the existing region info code in libvfio.
More generally, please replace the raw ioctl() calls throughout the test with libvfio helpers: VFIO_DEVICE_GET_INFO, VFIO_DEVICE_GET_REGION_INFO, and VFIO_DEVICE_FEATURE for DMA_BUF and LOW_POWER_ENTRY/EXIT. Please follow the existing convention, e.g. __vfio_pci_device_reset() and vfio_pci_device_reset(): a double-underscore variant that returns -errno for the tests that expect failures or skip when something isn't supported, and a plain variant that asserts success.
tools/testing/selftests/vfio/vfio_cxl_type2_test.c line 112
Quoted 3 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +3
-0+/* Walk the PCI extended capability list for the CXL Device DVSEC. */ +static uint16_t find_cxl_dvsec (struct vfio_pci_device *dev) +{
Same here, the extended capability walk and the DVSEC match aren't CXL-specific. Please consider vfio_pci_find_ext_capability() and vfio_pci_find_dvsec(device, vendor, id) helpers in libvfio, with the test passing in the CXL IDs.
Quoted 6 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +6
-0+/* Read config space from sysfs, which is the hardware, not the vfio view. */ +static int sysfs_config_read (uint32_t off,void *buf,size_t len) +{ +char path[128 ]; +ssize_t n; +int fd;
tools/testing/selftests/vfio/vfio_cxl_type2_test.c line 149
Quoted 10 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +10
-0+ +snprintf (path,sizeof (path),"/sys/bus/pci/devices/%s/config" , + device_bdf); + fd =open (path,O_RDONLY ); +if (fd <0 ) +return -errno; + n =pread (fd, buf, len, off); +close (fd); +return n == (ssize_t )len ?0 : -EIO ; +}
This and sysfs_config() further down do the same thing. Please replace both with one sysfs config access helper in lib/sysfs.c, modeled after the existing vfio config helpers: a sysfs_config_access(bdf, write, offset, size, data) core with sysfs_config_read{b,w,l}() and sysfs_config_write{b,w,l}() wrappers that assert on failure, like vfio_pci_config_access() and vfio_pci_config_read{b,w,l}().
Taking the BDF as a parameter also gets helpers off the device_bdf global. Please only use device_bdf where the device is opened (FIXTURE_SETUP() and close_restores_open_time_command). Everywhere else can use self->dev->bdf.
tools/testing/selftests/vfio/vfio_cxl_type2_test.c line 507
Quoted 10 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +10
-0+/* The trapped block starts at the HDM decoder registers; CTRL 0 reads back. */ +TEST_F(vfio_cxl, comp_regs_hdm_read) +{ +uint64_t ctrl = self->comp_off +CXL_HDM_DECODER0_CTRL_OFFSET (0 ); +uint32_t val =0 ; + +ASSERT_GE (self->comp_size,CXL_HDM_DECODER0_CTRL_OFFSET (0 ) +4 ); +ASSERT_EQ ((ssize_t )sizeof (val), +pread (self->dev->fd, &val,sizeof (val), ctrl)); +}
Please add region access helpers to libvfio instead of open-coding pread()/pwrite()/mmap() on the device fd, which this test does in about 50 places. Something modeled on the config space helpers:__vfio_pci_region_access(device, index, write, offset, size, data) returning the number of bytes or -errno, vfio_pci_region_access() asserting success, and typed vfio_pci_region_read{b,w,l,q}() and vfio_pci_region_write{b,w,l,q}() wrappers. Then this becomes:
val = vfio_pci_region_readl(self->dev, self->comp_idx, CXL_HDM_DECODER0_CTRL_OFFSET(0));
The double-underscore variant covers the expected failures, like comp_regs_read_needs_memory and the unaligned and out of range tests. vfio_pci_config_access() could become a wrapper around the same code using VFIO_PCI_CONFIG_REGION_INDEX. libvfio would need to cache the region info for all regions, not just config space and the BARs, so it can apply the offset reported by VFIO_DEVICE_GET_REGION_INFO.
Similarly, __vfio_pci_region_mmap() and vfio_pci_region_mmap() could cover the HDM region mappings and the expected mmap failures over the decoder block, and vfio_pci_bar_map() could be built on them.
tools/testing/selftests/vfio/vfio_cxl_type2_test.c line 1055
Quoted 2 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +2
-0+/* Find a standard capability through the vfio config view. */ +static uint8_t find_std_cap (struct vfio_pci_device *dev,uint8_t id)
Please make this a libvfio helper too, e.g.vfio_pci_find_capability(device, cap_id).
RE: [PATCH v6 31/31] selftests/vfio: Add CXL Type-2 passthrough tests
Quoted 38 lines
-----Original Message-----
From: David Matlack <[email protected]>
Sent: Monday, November 16, 2026 11:23 PM
To: Manish Honap <[email protected]>; [email protected]; [email protected]; Ankit Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; Srirangan Madhavan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yishai Hadas <[email protected]>; Shameer Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: Neo Jia <[email protected]>; Krishnakant Jaju <[email protected]>; Vikram Sethi <[email protected]>; Zhi Wang <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]
Subject: Re: [PATCH v6 31/31] selftests/vfio: Add CXL Type-2 passthrough testsExternal email: Use caution opening links or attachments
Quoted 5 lines · mhonap
Exercise the vfio-cxl contract on a bound Type-2 device: the CXL device-info flag and the two vendor-type regions, the HDM memory mmap (including a 2 MB huge fault), the read-only trapped decoder region against the live host-committed decoder, and the guest's DVSEC reset doorbell and manual HDM discovery.
I tried running this test in a VM with the emulated cxl-type2 device from your QEMU RFC [1], on an x86 q35 machine with intel-iommu. Out of the box the device binds as plain vfio-pci and the fixture skips everything. v6 now requires that the host can do CPU cache invalidation for CXL Reset, and
cpu_cache_has_invalidate_memregion()returns false in a VM (x86 refuses it under a hypervisor, and arm64 has no cache maintenance provider in a guest).
Quoted 6 lines
The QEMU RFC reports 19/19 passing in an Arm guest, but that was against v5, which didn't have this requirement. How did you run the v6 version of this test without hardware? Did you use something like
CONFIG_CXL_REGION_INVALIDATION_TEST? That option only covers region management indrivers/cxl/core/region.c, and depends onCXL_REGION, so it doesn't help here.
Thanks for running it. I did not: the 27 results in the v6 cover letter are from hardware only, an arm64 host with the SMCCC cache provider and firmware that implements the cache operations. You are right that the 19/19 in the QEMU RFC was the v5 test.
Quoted 8 lines
I added a local test-only bypass (a
CONFIG_CXL_RESET_INVALIDATION_TESTthat skips the invalidation incxl_reset_check()and cxl_hdm_quiesce_prepare()/cxl_hdm_ranges_invalidate()), I get 26 passes and 1 skip (pm_d3hot_restores_decoder, since cxl-type2 has no PM capability in QEMU). I think something like that belongs in thecxl_resetseries. Otherwise this test can only be run by people with real Type-2 hardware.
Good to see 26 pass on x86. cxl_reset v17 already has this: its patch 13 lets CONFIG_CXL_REGION_INVALIDATION_TEST skip the missing CPU cache invalidation for CXL Reset too, and its cover letter runs the cxl-type2 device in a q35 guest with that option set:
https://lore.kernel.org/r/[email protected]
v7 of this series will be based on v17 (or possibly v18), so a guest kernel with CXL_REGION and that option should run the test without a local bypass. I will check that before posting and put it in the test documentation.
Quoted 7 lines
[1] https://lore.kernel.org/linux-cxl/20261106035911.1280066-1-
[email protected]/Quoted 3 lines
Check that the decoder register region reads fail with EIO while Memory Space is disabled, and that the decoder reads back committed and HDM access works after low power entry and exit.
Quoted 7 lines
The series says a Type-2 device loses its committed HDM decoder in D3hot and D3cold (patches 17 and 30), but the emulated cxl-type2 device in your QEMU RFC has no PM capability, so on it these tests never take the decoder restore path in
vfio_cxl_power_up(). Please make the emulated device behave the same way as the real hardware across D3hot (PM capability,No_Soft_Reset, and whether the decoder is lost), so that the selftest exercises the same vfio-cxl code paths in a VM as it does on hardware.
Okay, I will handle this in next cxl-type2 version to have a more faithful hardware validation.
Quoted 5 lines
To help get that right: what does lspci -vv show for the PM capability of the device you tested with (NoSoftRst+ or NoSoftRst-), and did you observe the decoder actually being lost?
NoSoftRst+, and no. The statement in 17 and 30 is wrong for the device I tested, and I will correct this in v7.
lspci shows "Status: D0 NoSoftRst+ PME-Enable-".
On hardware the test only took the path where the decoder is still committed; the restore path in vfio_cxl_power_up() has not run there either.
Quoted 9 lines
Quoted 3 lines
Add a partial-mmap path to the vfio selftest library so the trapped component BAR maps its sparse ranges, and an
__iommu_map_file()helper for the dma-buf mapping.Please split the library changes into separate commits ahead of the
test:__iommu_map_file(), the sparse mmap support invfio_pci_bar_map(), and each of the new libvfio helpers suggested below. That way each one can be reviewed on its own, and the final commit only adds the test, itsMakefileentry, and theMAINTAINERSline.
Okay, for v7, the library change will be split out ahead of the test, one per helper.
Quoted 5 lines
This series is already quite long (31 patches), and with the library changes split out the selftest changes will grow to several patches.
Consider sending all the VFIO selftests patches as a separate series that depends on this one.
I would like to keep the library and the test at the end of this series for v7, so the test arrives with the code it covers and each version can be checked against it.
Quoted 14 lines
Quoted 14 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c +13
-0+/* Return the sparse-mmap capability in @info, or NULL if the region +has none. */ static struct vfio_region_info_cap_sparse_mmap * +vfio_pci_sparse_mmap_cap (struct vfio_region_info *info) { +struct vfio_info_cap_header *hdr; +u32 offset; + +if (!(info->flags &VFIO_REGION_INFO_FLAG_CAPS )) +return NULL ; + +for (offset = info->cap_offset; offset; offset = hdr->next) { + hdr = (void *)info + offset; +if (hdr->id ==VFIO_REGION_INFO_CAP_SPARSE_MMAP ) +return (struct vfio_region_info_cap_sparse_mmap *)hdr;
Quoted 9 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c static void vfio_pci_region_get(struct vfio_pci_device *device, int index, · line 207
Quoted 4 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c +4
-0+ } + +return NULL ; +}This walks the region info capability chain looking for one ID, which is the same thing
find_region_cap()does in the test. Please add one generic libvfio helper that looks up a region info capability by ID, and use it in both places.
Okay, I will update this.
Quoted 9 lines
Quoted 1 line
+ * Usage: ./vfio_cxl_type2_test <BDF> (or export
VFIO_SELFTESTS_BDF=<BDF>).
Quoted 1 line
+ * The device must be bound to vfio-pci with the vfio-cxl module available.
Please document how to run this test in more detail: the hardware requirements, how to set up the emulated QEMU device, and the kernel config needed (
VFIO_CXL, IOMMUFD, any test-only bypass), so others can reproduce your results.
Okay, I will update this with the qemu command line and guest kernel config.
Quoted 6 lines
Quoted 6 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +6
-0+/* vfio-pci region-offset packing is kernel-internal, not UAPI; +define locally. */ #ifndef VFIO_PCI_OFFSET_SHIFT +#define VFIO_PCI_OFFSET_SHIFT 40 +#endif +#ifndef VFIO_PCI_INDEX_TO_OFFSET +#define VFIO_PCI_INDEX_TO_OFFSET (i) ((uint64_t)(i) <<
Quoted 8 lines
Quoted 1 line
+VFIO_PCI_OFFSET_SHIFT) #endif
As the comment says, this is kernel-internal and not uAPI, so the test shouldn't depend on it. Please use vfio_region_info.offset from
VFIO_DEVICE_GET_REGION_INFOinstead, ascomp_bar_sparse_mmapalready does for the BAR. Better yet, use region access helpers in libvfio that take a region index and apply the offset internally (see below), so the test never computes region offsets itself.
Okay, I will use the region helper for this.
Quoted 23 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+/* Find a CXL region by subtype; returns the region index or -1, @buf +left holding its info. */ static int find_cxl_region(int fd, uint32_t nregions,
uint32_tsubtype,tools/testing/selftests/vfio/vfio_cxl_type2_test.c line 94
Quoted 13 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +13
-0+ void *buf, size_t bufsz) { + uint32_t i; + +for (i =0 ; i < nregions; i++) { +struct vfio_region_info *ri = buf; +const struct vfio_region_info_cap_type *t; +const struct vfio_info_cap_header *hdr; + +memset (buf,0 , bufsz); + ri->argsz = bufsz; + ri->index = i; +if (ioctl (fd,VFIO_DEVICE_GET_REGION_INFO , ri)) +continue ;This is a generic lookup of a region by type and subtype. Only the CXL vendor ID is specific. Please make it a libvfio helper (e.g.
vfio_pci_find_region(device, type, subtype)) built on the existing region info code in libvfio.
Quoted 7 lines
More generally, please replace the raw
ioctl()calls throughout the test with libvfio helpers:VFIO_DEVICE_GET_INFO,VFIO_DEVICE_GET_REGION_INFO, andVFIO_DEVICE_FEATUREforDMA_BUFand LOW_POWER_ENTRY/EXIT. Please follow the existing convention, e.g.__vfio_pci_device_reset()andvfio_pci_device_reset(): a double-underscore variant that returns -errno for the tests that expect failures or skip when something isn't supported, and a plain variant that asserts success.
Ack. I will update this.
Quoted 8 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+/* Walk the PCI extended capability list for the CXL Device DVSEC. */ +static uint16_t find_cxl_dvsec(struct vfio_pci_device *dev) {Same here, the extended capability walk and the DVSEC match aren't CXL- specific. Please consider
vfio_pci_find_ext_capability()andvfio_pci_find_dvsec(device, vendor, id)helpers in libvfio, with the test passing in the CXL IDs.
Okay, will do.
Quoted 24 lines
Quoted 6 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +5
-0+/* Read config space from sysfs, which is the hardware, not the vfio +view. */ static int sysfs_config_read (uint32_t off,void *buf,size_t +len) { +char path[128 ]; +ssize_t n;tools/testing/selftests/vfio/vfio_cxl_type2_test.c
Quoted 10 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +10
-0+int fd; + +snprintf (path,sizeof (path),"/sys/bus/pci/devices/%s/config" , + device_bdf); + fd =open (path,O_RDONLY ); +if (fd <0 ) +return -errno; + n =pread (fd, buf, len, off); +close (fd); +return n == (ssize_t )len ?0 : -EIO ; }This and
sysfs_config()further down do the same thing. Please replace both with one sysfs config access helper inlib/sysfs.c, modeled after the existing vfio config helpers: asysfs_config_access(bdf, write, offset, size, data)core with sysfs_config_read{b,w,l}() and
sysfs_config_write{b,w,l}() wrappers that assert on failure, likevfio_pci_config_access()and vfio_pci_config_read{b,w,l}().
Quoted 4 lines
Taking the BDF as a parameter also gets helpers off the
device_bdfglobal. Please only usedevice_bdfwhere the device is opened (FIXTURE_SETUP()andclose_restores_open_time_command). Everywhere else can useself->dev->bdf.
Agree for both. I will update this.
Quoted 33 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c
Quoted 7 lines
tools/testing/selftests/vfio/vfio_cxl_type2_test.c new file +6
-0+/* The trapped block starts at the HDM decoder registers; CTRL 0 +reads back. */ TEST_F(vfio_cxl, comp_regs_hdm_read) { +uint64_t ctrl = self->comp_off +CXL_HDM_DECODER0_CTRL_OFFSET (0 ); +uint32_t val =0 ; + + ASSERT_GE(self->comp_size, CXL_HDM_DECODER0_CTRL_OFFSET(0) +4);
Review
Quoted 2 lines
(quoted diff) +2
-0+ ASSERT_EQ((ssize_t)sizeof(val), + pread(self->dev->fd, &val, sizeof(val), ctrl)); }Please add region access helpers to libvfio instead of open-coding pread()/pwrite()/mmap() on the device fd, which this test does in about 50 places. Something modeled on the config space helpers:
__vfio_pci_region_access(device, index, write, offset, size, data)returning the number of bytes or -errno,vfio_pci_region_access()asserting success, and typed vfio_pci_region_read{b,w,l,q}() and vfio_pci_region_write{b,w,l,q}() wrappers. Then this becomes:val = vfio_pci_region_readl(self->dev, self->comp_idx, CXL_HDM_DECODER0_CTRL_OFFSET(0));The double-underscore variant covers the expected failures, like
comp_regs_read_needs_memoryand the unaligned and out of range tests.vfio_pci_config_access()could become a wrapper around the same code usingVFIO_PCI_CONFIG_REGION_INDEX. libvfio would need to cache the region info for all regions, not just config space and the BARs, so it can apply the offset reported byVFIO_DEVICE_GET_REGION_INFO.Similarly,
__vfio_pci_region_mmap()andvfio_pci_region_mmap()could cover the HDM region mappings and the expected mmap failures over the decoder block, andvfio_pci_bar_map()could be built on them.
Quoted 5 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+/* Find a standard capability through the vfio config view. */ static +uint8_t find_std_cap(struct vfio_pci_device *dev, uint8_t id)Please make this a libvfio helper too, e.g.
vfio_pci_find_capability(device, cap_id).
Okay; I will update this.
Region access and mmap helpers with double-underscore variants for the expected failures, region info cached for every region, and vfio_pci_config_access() and vfio_pci_bar_map() built on them.
Thanks,
Manish