vfio/pci: Allow MMIO regions to be exported through dma-buf

PATCH Superseded Leon Romanovsky 3 participants 24 messages Sep 11, 2025 – Sep 28, 2025

Subsystems (11)

Memory management Media (V4L/DVB) KVM Block layer +7 more subsystems

lore

Share & tools

lore

Leon Romanovsky

author

2025-09-11 11:33 UTC #0 lore raw

v2 cover vfio/pci: Allow MMIO regions to be exported through dma-buf

Changelog:
v2:
* Added extra patch which adds new CONFIG, so next patches can reuse it.
* Squashed "PCI/P2PDMA: Remove redundant bus_offset from map state" into the other patch.
* Fixed revoke calls to be aligned with true->false semantics.
* Extended p2pdma_providers to be per-BAR and not global to whole device.
* Fixed possible race between dmabuf states and revoke.
* Moved revoke to PCI BAR zap block.
v1: https://lore.kernel.org/all/[email protected]
* Changed commit messages.
* Reused DMA_ATTR_MMIO attribute.
* Returned support for multiple DMA ranges per-dMABUF.
v0: https://lore.kernel.org/all/[email protected]

---------------------------------------------------------------------------
Based on "[PATCH v6 00/16] dma-mapping: migrate to physical address-based API"
https://lore.kernel.org/all/[email protected]/ series.
---------------------------------------------------------------------------

This series extends the VFIO PCI subsystem to support exporting MMIO regions from PCI device BARs as dma-buf objects, enabling safe sharing of non-struct page memory with controlled lifetime management. This allows RDMA and other subsystems to import dma-buf FDs and build them into memory regions for PCI P2P operations.

The series supports a use case for SPDK where a NVMe device will be owned by SPDK through VFIO but interacting with a RDMA device. The RDMA device may directly access the NVMe CMB or directly manipulate the NVMe device's doorbell using PCI P2P.

However, as a general mechanism, it can support many other scenarios with VFIO. This dmabuf approach can be usable by iommufd as well for generic and safe P2P mappings.

In addition to the SPDK use-case mentioned above, the capability added in this patch series can also be useful when a buffer (located in device memory such as VRAM) needs to be shared between any two dGPU devices or instances (assuming one of them is bound to VFIO PCI) as long as they are P2P DMA compatible.

The implementation provides a revocable attachment mechanism using dma-buf move operations. MMIO regions are normally pinned as BARs don't change physical addresses, but access is revoked when the VFIO device is closed or a PCI reset is issued. This ensures kernel self-defense against potentially hostile userspace.

The series includes significant refactoring of the PCI P2PDMA subsystem to separate core P2P functionality from memory allocation features, making it more modular and suitable for VFIO use cases that don't need struct page support.

-----------------------------------------------------------------------
The series is based originally on
https://lore.kernel.org/all/[email protected]/
but heavily rewritten to be based on DMA physical API.
-----------------------------------------------------------------------
The WIP branch can be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git/log/?h=dmabuf-vfio-v2

Thanks

Leon Romanovsky (8):
PCI/P2PDMA: Separate the mmap() support from the core logic PCI/P2PDMA: Simplify bus address mapping API
PCI/P2PDMA: Refactor to separate core P2P functionality from memory
allocation
PCI/P2PDMA: Export pci_p2pdma_map_type() function
types: move phys_vec definition to common header vfio/pci: Add dma-buf export config for MMIO regions vfio/pci: Enable peer-to-peer DMA transactions by default vfio/pci: Add dma-buf export support for MMIO regions
:wqa
Vivek Kasireddy (2):
vfio: Export vfio device get and put registration helpers vfio/pci: Share the core device pointer while invoking feature
functions

 block/blk-mq-dma.c                 |   7 +-
 drivers/iommu/dma-iommu.c          |   4 +-
 drivers/pci/p2pdma.c               | 165 ++++++++----
 drivers/vfio/pci/Kconfig           |  20 ++
 drivers/vfio/pci/Makefile          |   2 +
 drivers/vfio/pci/vfio_pci_config.c |  22 +-
 drivers/vfio/pci/vfio_pci_core.c   |  59 +++--
 drivers/vfio/pci/vfio_pci_dmabuf.c | 398 +++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_priv.h   |  23 ++
 drivers/vfio/vfio_main.c           |   2 +
 include/linux/pci-p2pdma.h         | 114 +++++----
 include/linux/types.h              |   5 +
 include/linux/vfio.h               |   2 +
 include/linux/vfio_pci_core.h      |   4 +
 include/uapi/linux/vfio.h          |  25 ++
 kernel/dma/direct.c                |   4 +-
 mm/hmm.c                           |   2 +-
 17 files changed, 734 insertions(+), 124 deletions(-)
 create mode 100644 drivers/vfio/pci/vfio_pci_dmabuf.c

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:33 UTC #1 lore raw

PATCH v2 1/10 PCI/P2PDMA: Separate the mmap() support from the core logic

From: Leon Romanovsky <[email protected]>

Currently the P2PDMA code requires a pgmap and a struct page to function. The was serving three important purposes:

- DMA API compatibility, where scatterlist required a struct page as input

- Life cycle management, the percpu_ref is used to prevent UAF during device hot unplug

- A way to get the P2P provider data through the pci_p2pdma_pagemap

The DMA API now has a new flow, and has gained phys_addr_t support, so it no longer needs struct pages to perform P2P mapping.

Lifecycle management can be delegated to the user, DMABUF for instance has a suitable invalidation protocol that does not require struct page.

Finding the P2P provider data can also be managed by the caller without need to look it up from the phys_addr.

Split the P2PDMA code into two layers. The optional upper layer, effectively, provides a way to mmap() P2P memory into a VMA by providing struct page, pgmap, a genalloc and sysfs.

The lower layer provides the actual P2P infrastructure and is wrapped up in a new struct p2pdma_provider. Rework the mmap layer to use new p2pdma_provider based APIs.

Drivers that do not want to put P2P memory into VMA's can allocate a struct p2pdma_provider after probe() starts and free it before remove() completes. When DMA mapping the driver must convey the struct p2pdma_provider to the DMA mapping code along with a phys_addr of the MMIO BAR slice to map. The driver must ensure that no DMA mapping outlives the lifetime of the struct p2pdma_provider.

The intended target of this new API layer is DMABUF. There is usually only a single p2pdma_provider for a DMABUF exporter. Most drivers can establish the p2pdma_provider during probe, access the single instance during DMABUF attach and use that to drive the DMA mapping.

DMABUF provides an invalidation mechanism that can guarantee all DMA is halted and the DMA mappings are undone prior to destroying the struct p2pdma_provider. This ensures there is no UAF through DMABUFs that are lingering past driver removal.

The new p2pdma_provider layer cannot be used to create P2P memory that can be mapped into VMA's, be used with pin_user_pages(), O_DIRECT, and so on. These use cases must still use the mmap() layer. The p2pdma_provider layer is principally for DMABUF-like use cases where DMABUF natively manages the life cycle and access instead of vmas/pin_user_pages()/struct page.

In addition, remove the bus_off field from pci_p2pdma_map_state since it duplicates information already available in the pgmap structure. The bus_offset is only used in one location (pci_p2pdma_bus_addr_map) and is always identical to pgmap->bus_offset.

Signed-off-by Jason Gunthorpe <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/pci/p2pdma.c       | 43 ++++++++++++++++++++------------------
 include/linux/pci-p2pdma.h | 19 ++++++++++++-----
 2 files changed, 37 insertions(+), 25 deletions(-)

drivers/pci/p2pdma.c +23 ~~-20~~

index da5657a020074..176a99232fdca 100644
@@ -28,9 +28,8 @@ struct pci_p2pdma {
 };
 
 struct pci_p2pdma_pagemap {
-	struct pci_dev *provider;
-	u64 bus_offset;
 	struct dev_pagemap pgmap;
+	struct p2pdma_provider mem;
 };
 
 static struct pci_p2pdma_pagemap *to_p2p_pgmap(struct dev_pagemap *pgmap)

@@ -204,8 +203,8 @@ static void p2pdma_page_free(struct page *page)
 {
 	struct pci_p2pdma_pagemap *pgmap = to_p2p_pgmap(page_pgmap(page));
 	/* safe to dereference while a reference is held to the percpu ref */
-	struct pci_p2pdma *p2pdma =
-		rcu_dereference_protected(pgmap->provider->p2pdma, 1);
+	struct pci_p2pdma *p2pdma = rcu_dereference_protected(
+		to_pci_dev(pgmap->mem.owner)->p2pdma, 1);
 	struct percpu_ref *ref;
 
 	gen_pool_free_owner(p2pdma->pool, (uintptr_t)page_to_virt(page),

@@ -270,14 +269,15 @@ static int pci_p2pdma_setup(struct pci_dev *pdev)
 
 static void pci_p2pdma_unmap_mappings(void *data)
 {
-	struct pci_dev *pdev = data;
+	struct pci_p2pdma_pagemap *p2p_pgmap = data;
 
 	/*
 	 * Removing the alloc attribute from sysfs will call
 	 * unmap_mapping_range() on the inode, teardown any existing userspace
 	 * mappings and prevent new ones from being created.
 	 */
-	sysfs_remove_file_from_group(&pdev->dev.kobj, &p2pmem_alloc_attr.attr,
+	sysfs_remove_file_from_group(&p2p_pgmap->mem.owner->kobj,
+				     &p2pmem_alloc_attr.attr,
 				     p2pmem_group.name);
 }

@@ -328,10 +328,9 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	pgmap->nr_range = 1;
 	pgmap->type = MEMORY_DEVICE_PCI_P2PDMA;
 	pgmap->ops = &p2pdma_pgmap_ops;
-
-	p2p_pgmap->provider = pdev;
-	p2p_pgmap->bus_offset = pci_bus_address(pdev, bar) -
-		pci_resource_start(pdev, bar);
+	p2p_pgmap->mem.owner = &pdev->dev;
+	p2p_pgmap->mem.bus_offset =
+		pci_bus_address(pdev, bar) - pci_resource_start(pdev, bar);
 
 	addr = devm_memremap_pages(&pdev->dev, pgmap);
 	if (IS_ERR(addr)) {

@@ -340,7 +339,7 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	}
 
 	error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_unmap_mappings,
-					 pdev);
+					 p2p_pgmap);
 	if (error)
 		goto pages_free;

@@ -973,16 +972,16 @@ void pci_p2pmem_publish(struct pci_dev *pdev, bool publish)
 }
 EXPORT_SYMBOL_GPL(pci_p2pmem_publish);
 
-static enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap,
-						    struct device *dev)
+static enum pci_p2pdma_map_type
+pci_p2pdma_map_type(struct p2pdma_provider *provider, struct device *dev)
 {
 	enum pci_p2pdma_map_type type = PCI_P2PDMA_MAP_NOT_SUPPORTED;
-	struct pci_dev *provider = to_p2p_pgmap(pgmap)->provider;
+	struct pci_dev *pdev = to_pci_dev(provider->owner);
 	struct pci_dev *client;
 	struct pci_p2pdma *p2pdma;
 	int dist;
 
-	if (!provider->p2pdma)
+	if (!pdev->p2pdma)
 		return PCI_P2PDMA_MAP_NOT_SUPPORTED;
 
 	if (!dev_is_pci(dev))

@@ -991,7 +990,7 @@ static enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap,
 	client = to_pci_dev(dev);
 
 	rcu_read_lock();
-	p2pdma = rcu_dereference(provider->p2pdma);
+	p2pdma = rcu_dereference(pdev->p2pdma);
 
 	if (p2pdma)
 		type = xa_to_value(xa_load(&p2pdma->map_types,

@@ -999,7 +998,7 @@ static enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap,
 	rcu_read_unlock();
 
 	if (type == PCI_P2PDMA_MAP_UNKNOWN)
-		return calc_map_type_and_dist(provider, client, &dist, true);
+		return calc_map_type_and_dist(pdev, client, &dist, true);
 
 	return type;
 }

@@ -1007,9 +1006,13 @@ static enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap,
 void __pci_p2pdma_update_state(struct pci_p2pdma_map_state *state,
 		struct device *dev, struct page *page)
 {
-	state->pgmap = page_pgmap(page);
-	state->map = pci_p2pdma_map_type(state->pgmap, dev);
-	state->bus_off = to_p2p_pgmap(state->pgmap)->bus_offset;
+	struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(page_pgmap(page));
+
+	if (state->mem == &p2p_pgmap->mem)
+		return;
+
+	state->mem = &p2p_pgmap->mem;
+	state->map = pci_p2pdma_map_type(&p2p_pgmap->mem, dev);
 }
 
 /**

include/linux/pci-p2pdma.h +14 -5

index 075c20b161d98..27a2c399f47da 100644
@@ -16,6 +16,16 @@
 struct block_device;
 struct scatterlist;
 
+/**
+ * struct p2pdma_provider
+ *
+ * A p2pdma provider is a range of MMIO address space available to the CPU.
+ */
+struct p2pdma_provider {
+	struct device *owner;
+	u64 bus_offset;
+};
+
 #ifdef CONFIG_PCI_P2PDMA
 int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 		u64 offset);

@@ -144,11 +154,11 @@ enum pci_p2pdma_map_type {
 };
 
 struct pci_p2pdma_map_state {
-	struct dev_pagemap *pgmap;
+	struct p2pdma_provider *mem;
 	enum pci_p2pdma_map_type map;
-	u64 bus_off;
 };
 
+
 /* helper for pci_p2pdma_state(), do not use directly */
 void __pci_p2pdma_update_state(struct pci_p2pdma_map_state *state,
 		struct device *dev, struct page *page);

@@ -167,8 +177,7 @@ pci_p2pdma_state(struct pci_p2pdma_map_state *state, struct device *dev,
 		struct page *page)
 {
 	if (IS_ENABLED(CONFIG_PCI_P2PDMA) && is_pci_p2pdma_page(page)) {
-		if (state->pgmap != page_pgmap(page))
-			__pci_p2pdma_update_state(state, dev, page);
+		__pci_p2pdma_update_state(state, dev, page);
 		return state->map;
 	}
 	return PCI_P2PDMA_MAP_NONE;

@@ -186,7 +195,7 @@ static inline dma_addr_t
 pci_p2pdma_bus_addr_map(struct pci_p2pdma_map_state *state, phys_addr_t paddr)
 {
 	WARN_ON_ONCE(state->map != PCI_P2PDMA_MAP_BUS_ADDR);
-	return paddr + state->bus_off;
+	return paddr + state->mem->bus_offset;
 }
 
 #endif /* _LINUX_PCI_P2P_H */

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:33 UTC #2 lore raw

PATCH v2 2/10 PCI/P2PDMA: Simplify bus address mapping API

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Update the pci_p2pdma_bus_addr_map() function to take a direct pointer to the p2pdma_provider structure instead of the pci_p2pdma_map_state. This simplifies the API by removing the need for callers to extract the provider from the state structure.

The change updates all callers across the kernel (block layer, IOMMU, DMA direct, and HMM) to pass the provider pointer directly, making the code more explicit and reducing unnecessary indirection. This also removes the runtime warning check since callers now have direct control over which provider they use.

Signed-off-by Leon Romanovsky <[email protected]>

---
 block/blk-mq-dma.c         | 2 +-
 drivers/iommu/dma-iommu.c  | 4 ++--
 include/linux/pci-p2pdma.h | 7 +++----
 kernel/dma/direct.c        | 4 ++--
 mm/hmm.c                   | 2 +-
 5 files changed, 9 insertions(+), 10 deletions(-)

block/blk-mq-dma.c +1 -1

index d415088ed9fd2..430e51ec494a6 100644
@@ -79,7 +79,7 @@ static inline bool blk_can_dma_map_iova(struct request *req,
 
 static bool blk_dma_map_bus(struct blk_dma_iter *iter, struct phys_vec *vec)
 {
-	iter->addr = pci_p2pdma_bus_addr_map(&iter->p2pdma, vec->paddr);
+	iter->addr = pci_p2pdma_bus_addr_map(iter->p2pdma.mem, vec->paddr);
 	iter->len = vec->len;
 	return true;
 }

drivers/iommu/dma-iommu.c +2 -2

index 7944a3af4545e..e52d19d2e8334 100644
@@ -1439,8 +1439,8 @@ int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
 			 * as a bus address, __finalise_sg() will copy the dma
 			 * address into the output segment.
 			 */
-			s->dma_address = pci_p2pdma_bus_addr_map(&p2pdma_state,
-						sg_phys(s));
+			s->dma_address = pci_p2pdma_bus_addr_map(
+				p2pdma_state.mem, sg_phys(s));
 			sg_dma_len(s) = sg->length;
 			sg_dma_mark_bus_address(s);
 			continue;

include/linux/pci-p2pdma.h +3 -4

index 27a2c399f47da..eef96636c67e6 100644
@@ -186,16 +186,15 @@ pci_p2pdma_state(struct pci_p2pdma_map_state *state, struct device *dev,
 /**
  * pci_p2pdma_bus_addr_map - Translate a physical address to a bus address
  *			     for a PCI_P2PDMA_MAP_BUS_ADDR transfer.
- * @state:	P2P state structure
+ * @provider:	P2P provider structure
  * @paddr:	physical address to map
  *
  * Map a physically contiguous PCI_P2PDMA_MAP_BUS_ADDR transfer.
  */
 static inline dma_addr_t
-pci_p2pdma_bus_addr_map(struct pci_p2pdma_map_state *state, phys_addr_t paddr)
+pci_p2pdma_bus_addr_map(struct p2pdma_provider *provider, phys_addr_t paddr)
 {
-	WARN_ON_ONCE(state->map != PCI_P2PDMA_MAP_BUS_ADDR);
-	return paddr + state->mem->bus_offset;
+	return paddr + provider->bus_offset;
 }
 
 #endif /* _LINUX_PCI_P2P_H */

kernel/dma/direct.c +2 -2

index 1062caac47e7b..3e058c99fe856 100644
@@ -484,8 +484,8 @@ int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
 			}
 			break;
 		case PCI_P2PDMA_MAP_BUS_ADDR:
-			sg->dma_address = pci_p2pdma_bus_addr_map(&p2pdma_state,
-					sg_phys(sg));
+			sg->dma_address = pci_p2pdma_bus_addr_map(
+				p2pdma_state.mem, sg_phys(sg));
 			sg_dma_mark_bus_address(sg);
 			continue;
 		default:

mm/hmm.c +1 -1

index 6556c0e074ba8..012b78688fa18 100644
@@ -751,7 +751,7 @@ dma_addr_t hmm_dma_map_pfn(struct device *dev, struct hmm_dma_map *map,
 		break;
 	case PCI_P2PDMA_MAP_BUS_ADDR:
 		pfns[idx] |= HMM_PFN_P2PDMA_BUS | HMM_PFN_DMA_MAPPED;
-		return pci_p2pdma_bus_addr_map(p2pdma_state, paddr);
+		return pci_p2pdma_bus_addr_map(p2pdma_state->mem, paddr);
 	default:
 		return DMA_MAPPING_ERROR;
 	}

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:33 UTC #3 lore raw

PATCH v2 3/10 PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Refactor the PCI P2PDMA subsystem to separate the core peer-to-peer DMA functionality from the optional memory allocation layer. This creates a two-tier architecture:

The core layer provides P2P mapping functionality for physical addresses based on PCI device MMIO BARs and integrates with the DMA API for mapping operations. This layer is required for all P2PDMA users.

The optional upper layer provides memory allocation capabilities including gen_pool allocator, struct page support, and sysfs interface for user space access.

This separation allows subsystems like VFIO to use only the core P2P mapping functionality without the overhead of memory allocation features they don't need. The core functionality is now available through the new pci_p2pdma_enable() function that returns a p2pdma_provider structure.

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/pci/p2pdma.c       | 129 +++++++++++++++++++++++++++----------
 include/linux/pci-p2pdma.h |   5 ++
 2 files changed, 100 insertions(+), 34 deletions(-)

drivers/pci/p2pdma.c +95 ~~-34~~

index 176a99232fdca..c22cbb3a26030 100644
@@ -25,11 +25,12 @@ struct pci_p2pdma {
 	struct gen_pool *pool;
 	bool p2pmem_published;
 	struct xarray map_types;
+	struct p2pdma_provider mem[PCI_STD_NUM_BARS];
 };
 
 struct pci_p2pdma_pagemap {
 	struct dev_pagemap pgmap;
-	struct p2pdma_provider mem;
+	struct p2pdma_provider *mem;
 };
 
 static struct pci_p2pdma_pagemap *to_p2p_pgmap(struct dev_pagemap *pgmap)

@@ -204,7 +205,7 @@ static void p2pdma_page_free(struct page *page)
 	struct pci_p2pdma_pagemap *pgmap = to_p2p_pgmap(page_pgmap(page));
 	/* safe to dereference while a reference is held to the percpu ref */
 	struct pci_p2pdma *p2pdma = rcu_dereference_protected(
-		to_pci_dev(pgmap->mem.owner)->p2pdma, 1);
+		to_pci_dev(pgmap->mem->owner)->p2pdma, 1);
 	struct percpu_ref *ref;
 
 	gen_pool_free_owner(p2pdma->pool, (uintptr_t)page_to_virt(page),

@@ -227,44 +228,93 @@ static void pci_p2pdma_release(void *data)
 
 	/* Flush and disable pci_alloc_p2p_mem() */
 	pdev->p2pdma = NULL;
-	synchronize_rcu();
+	if (p2pdma->pool)
+		synchronize_rcu();
+	xa_destroy(&p2pdma->map_types);
+
+	if (!p2pdma->pool)
+		return;
 
 	gen_pool_destroy(p2pdma->pool);
 	sysfs_remove_group(&pdev->dev.kobj, &p2pmem_group);
-	xa_destroy(&p2pdma->map_types);
 }
 
-static int pci_p2pdma_setup(struct pci_dev *pdev)
+/**
+ * pcim_p2pdma_enable - Enable peer-to-peer DMA support for a PCI device
+ * @pdev: The PCI device to enable P2PDMA for
+ * @bar: BAR index to get provider
+ *
+ * This function initializes the peer-to-peer DMA infrastructure for a PCI
+ * device. It allocates and sets up the necessary data structures to support
+ * P2PDMA operations, including mapping type tracking.
+ */
+struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar)
 {
-	int error = -ENOMEM;
 	struct pci_p2pdma *p2p;
+	int i, ret;
+
+	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (p2p)
+		/* PCI device was "rebound" to the driver */
+		return &p2p->mem[bar];
 
 	p2p = devm_kzalloc(&pdev->dev, sizeof(*p2p), GFP_KERNEL);
 	if (!p2p)
-		return -ENOMEM;
+		return ERR_PTR(-ENOMEM);
 
 	xa_init(&p2p->map_types);
+	/*
+	 * Iterate over all standard PCI BARs and record only those that
+	 * correspond to MMIO regions. Skip non-memory resources (e.g. I/O
+	 * port BARs) since they cannot be used for peer-to-peer (P2P)
+	 * transactions.
+	 */
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		if (!(pci_resource_flags(pdev, i) & IORESOURCE_MEM))
+			continue;
 
-	p2p->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev));
-	if (!p2p->pool)
-		goto out;
+		p2p->mem[i].owner = &pdev->dev;
+		p2p->mem[i].bus_offset =
+			pci_bus_address(pdev, i) - pci_resource_start(pdev, i);
+	}
 
-	error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev);
-	if (error)
-		goto out_pool_destroy;
+	ret = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev);
+	if (ret)
+		goto out_p2p;
 
-	error = sysfs_create_group(&pdev->dev.kobj, &p2pmem_group);
-	if (error)
+	rcu_assign_pointer(pdev->p2pdma, p2p);
+	return &p2p->mem[bar];
+
+out_p2p:
+	devm_kfree(&pdev->dev, p2p);
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(pcim_p2pdma_enable);
+
+static int pci_p2pdma_setup_pool(struct pci_dev *pdev)
+{
+	struct pci_p2pdma *p2pdma;
+	int ret;
+
+	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (p2pdma->pool)
+		/* We already setup pools, do nothing, */
+		return 0;
+
+	p2pdma->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev));
+	if (!p2pdma->pool)
+		return -ENOMEM;
+
+	ret = sysfs_create_group(&pdev->dev.kobj, &p2pmem_group);
+	if (ret)
 		goto out_pool_destroy;
 
-	rcu_assign_pointer(pdev->p2pdma, p2p);
 	return 0;
 
 out_pool_destroy:
-	gen_pool_destroy(p2p->pool);
-out:
-	devm_kfree(&pdev->dev, p2p);
-	return error;
+	gen_pool_destroy(p2pdma->pool);
+	p2pdma->pool = NULL;
+	return ret;
 }
 
 static void pci_p2pdma_unmap_mappings(void *data)

@@ -276,7 +326,7 @@ static void pci_p2pdma_unmap_mappings(void *data)
 	 * unmap_mapping_range() on the inode, teardown any existing userspace
 	 * mappings and prevent new ones from being created.
 	 */
-	sysfs_remove_file_from_group(&p2p_pgmap->mem.owner->kobj,
+	sysfs_remove_file_from_group(&p2p_pgmap->mem->owner->kobj,
 				     &p2pmem_alloc_attr.attr,
 				     p2pmem_group.name);
 }

@@ -295,6 +345,7 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 			    u64 offset)
 {
 	struct pci_p2pdma_pagemap *p2p_pgmap;
+	struct p2pdma_provider *mem;
 	struct dev_pagemap *pgmap;
 	struct pci_p2pdma *p2pdma;
 	void *addr;

@@ -312,15 +363,25 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	if (size + offset > pci_resource_len(pdev, bar))
 		return -EINVAL;
 
-	if (!pdev->p2pdma) {
-		error = pci_p2pdma_setup(pdev);
+	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (!p2pdma) {
+		mem = pcim_p2pdma_enable(pdev, bar);
+		if (IS_ERR(mem))
+			return PTR_ERR(mem);
+
+		error = pci_p2pdma_setup_pool(pdev);
 		if (error)
 			return error;
-	}
+
+		p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
+	} else
+		mem = &p2pdma->mem[bar];
 
 	p2p_pgmap = devm_kzalloc(&pdev->dev, sizeof(*p2p_pgmap), GFP_KERNEL);
-	if (!p2p_pgmap)
-		return -ENOMEM;
+	if (!p2p_pgmap) {
+		error = -ENOMEM;
+		goto free_pool;
+	}
 
 	pgmap = &p2p_pgmap->pgmap;
 	pgmap->range.start = pci_resource_start(pdev, bar) + offset;

@@ -328,9 +389,7 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	pgmap->nr_range = 1;
 	pgmap->type = MEMORY_DEVICE_PCI_P2PDMA;
 	pgmap->ops = &p2pdma_pgmap_ops;
-	p2p_pgmap->mem.owner = &pdev->dev;
-	p2p_pgmap->mem.bus_offset =
-		pci_bus_address(pdev, bar) - pci_resource_start(pdev, bar);
+	p2p_pgmap->mem = mem;
 
 	addr = devm_memremap_pages(&pdev->dev, pgmap);
 	if (IS_ERR(addr)) {

@@ -343,7 +402,6 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	if (error)
 		goto pages_free;
 
-	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
 	error = gen_pool_add_owner(p2pdma->pool, (unsigned long)addr,
 			pci_bus_address(pdev, bar) + offset,
 			range_len(&pgmap->range), dev_to_node(&pdev->dev),

@@ -359,7 +417,10 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 pages_free:
 	devm_memunmap_pages(&pdev->dev, pgmap);
 pgmap_free:
-	devm_kfree(&pdev->dev, pgmap);
+	devm_kfree(&pdev->dev, p2p_pgmap);
+free_pool:
+	sysfs_remove_group(&pdev->dev.kobj, &p2pmem_group);
+	gen_pool_destroy(p2pdma->pool);
 	return error;
 }
 EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource);

@@ -1008,11 +1069,11 @@ void __pci_p2pdma_update_state(struct pci_p2pdma_map_state *state,
 {
 	struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(page_pgmap(page));
 
-	if (state->mem == &p2p_pgmap->mem)
+	if (state->mem == p2p_pgmap->mem)
 		return;
 
-	state->mem = &p2p_pgmap->mem;
-	state->map = pci_p2pdma_map_type(&p2p_pgmap->mem, dev);
+	state->mem = p2p_pgmap->mem;
+	state->map = pci_p2pdma_map_type(p2p_pgmap->mem, dev);
 }
 
 /**

include/linux/pci-p2pdma.h +5 -0

index eef96636c67e6..888ad7b0c54cf 100644
@@ -27,6 +27,7 @@ struct p2pdma_provider {
 };
 
 #ifdef CONFIG_PCI_P2PDMA
+struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar);
 int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 		u64 offset);
 int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients,

@@ -45,6 +46,10 @@ int pci_p2pdma_enable_store(const char *page, struct pci_dev **p2p_dev,
 ssize_t pci_p2pdma_enable_show(char *page, struct pci_dev *p2p_dev,
 			       bool use_p2pdma);
 #else /* CONFIG_PCI_P2PDMA */
+static inline struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
 static inline int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar,
 		size_t size, u64 offset)
 {

Signature · 1 line

2.51.0

Alex Williamson

2025-09-22 21:00 UTC #4 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 81 lines · On Thu, 11 Sep 2025 14:33:07 +0300 Leon Romanovsky <[email protected]> wrote:

From: Leon Romanovsky <[email protected]>

Refactor the PCI P2PDMA subsystem to separate the core peer-to-peer DMA functionality from the optional memory allocation layer. This creates a two-tier architecture:

The optional upper layer provides memory allocation capabilities including gen_pool allocator, struct page support, and sysfs interface for user space access.

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/pci/p2pdma.c       | 129 +++++++++++++++++++++++++++----------
 include/linux/pci-p2pdma.h |   5 ++
 2 files changed, 100 insertions(+), 34 deletions(-)

drivers/pci/p2pdma.c +19 -5

index 176a99232fdca..c22cbb3a26030 100644
@@ -25,11 +25,12 @@ struct pci_p2pdma {
 	struct gen_pool *pool;
 	bool p2pmem_published;
 	struct xarray map_types;
+	struct p2pdma_provider mem[PCI_STD_NUM_BARS];
 };
 
 struct pci_p2pdma_pagemap {
 	struct dev_pagemap pgmap;
-	struct p2pdma_provider mem;
+	struct p2pdma_provider *mem;
 };
 
 static struct pci_p2pdma_pagemap *to_p2p_pgmap(struct dev_pagemap *pgmap)

@@ -204,7 +205,7 @@ static void p2pdma_page_free(struct page *page)
 	struct pci_p2pdma_pagemap *pgmap = to_p2p_pgmap(page_pgmap(page));
 	/* safe to dereference while a reference is held to the percpu ref */
 	struct pci_p2pdma *p2pdma = rcu_dereference_protected(
-		to_pci_dev(pgmap->mem.owner)->p2pdma, 1);
+		to_pci_dev(pgmap->mem->owner)->p2pdma, 1);
 	struct percpu_ref *ref;
 
 	gen_pool_free_owner(p2pdma->pool, (uintptr_t)page_to_virt(page),

@@ -227,44 +228,93 @@ static void pci_p2pdma_release(void *data)
 
 	/* Flush and disable pci_alloc_p2p_mem() */
 	pdev->p2pdma = NULL;
-	synchronize_rcu();
+	if (p2pdma->pool)
+		synchronize_rcu();
+	xa_destroy(&p2pdma->map_types);
+
+	if (!p2pdma->pool)
+		return;
 
 	gen_pool_destroy(p2pdma->pool);
 	sysfs_remove_group(&pdev->dev.kobj, &p2pmem_group);
-	xa_destroy(&p2pdma->map_types);
 }
 
-static int pci_p2pdma_setup(struct pci_dev *pdev)
+/**
+ * pcim_p2pdma_enable - Enable peer-to-peer DMA support for a PCI device
+ * @pdev: The PCI device to enable P2PDMA for
+ * @bar: BAR index to get provider
+ *
+ * This function initializes the peer-to-peer DMA infrastructure for a PCI
+ * device. It allocates and sets up the necessary data structures to support
+ * P2PDMA operations, including mapping type tracking.
+ */
+struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar)

drivers/pci/p2pdma.c static void pci_p2pdma_release(void *data) · line 260

Quoted 10 lines

drivers/pci/p2pdma.c +6 -1

 {
-	int error = -ENOMEM;
 	struct pci_p2pdma *p2p;
+	int i, ret;
+
+	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (p2p)
+		/* PCI device was "rebound" to the driver */
+		return &p2p->mem[bar];

This seems like two separate functions rolled into one, an 'initialize providers' and a 'get provider for BAR'. The comment above even makes it sound like only a driver re-probing a device would encounter this branch, but the use case later in vfio-pci shows it to be the common case to iterate BARs for a device.

But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?

It also seems like the coordination of a valid provider is ad-hoc between p2pdma and vfio-pci. For example, this only fills providers for MMIO BARs and vfio-pci validates that dmabuf operations are for MMIO BARs, but it would be more consistent if vfio-pci relied on p2pdma to give it a valid provider for a given BAR. Thanks,

Alex

Quoted 191 lines

drivers/pci/p2pdma.c +70 ~~-28~~

 	p2p = devm_kzalloc(&pdev->dev, sizeof(*p2p), GFP_KERNEL);
 	if (!p2p)
-		return -ENOMEM;
+		return ERR_PTR(-ENOMEM);
 
 	xa_init(&p2p->map_types);
+	/*
+	 * Iterate over all standard PCI BARs and record only those that
+	 * correspond to MMIO regions. Skip non-memory resources (e.g. I/O
+	 * port BARs) since they cannot be used for peer-to-peer (P2P)
+	 * transactions.
+	 */
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		if (!(pci_resource_flags(pdev, i) & IORESOURCE_MEM))
+			continue;
 
-	p2p->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev));
-	if (!p2p->pool)
-		goto out;
+		p2p->mem[i].owner = &pdev->dev;
+		p2p->mem[i].bus_offset =
+			pci_bus_address(pdev, i) - pci_resource_start(pdev, i);
+	}
 
-	error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev);
-	if (error)
-		goto out_pool_destroy;
+	ret = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev);
+	if (ret)
+		goto out_p2p;
 
-	error = sysfs_create_group(&pdev->dev.kobj, &p2pmem_group);
-	if (error)
+	rcu_assign_pointer(pdev->p2pdma, p2p);
+	return &p2p->mem[bar];
+
+out_p2p:
+	devm_kfree(&pdev->dev, p2p);
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(pcim_p2pdma_enable);
+
+static int pci_p2pdma_setup_pool(struct pci_dev *pdev)
+{
+	struct pci_p2pdma *p2pdma;
+	int ret;
+
+	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (p2pdma->pool)
+		/* We already setup pools, do nothing, */
+		return 0;
+
+	p2pdma->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev));
+	if (!p2pdma->pool)
+		return -ENOMEM;
+
+	ret = sysfs_create_group(&pdev->dev.kobj, &p2pmem_group);
+	if (ret)
 		goto out_pool_destroy;
 
-	rcu_assign_pointer(pdev->p2pdma, p2p);
 	return 0;
 
 out_pool_destroy:
-	gen_pool_destroy(p2p->pool);
-out:
-	devm_kfree(&pdev->dev, p2p);
-	return error;
+	gen_pool_destroy(p2pdma->pool);
+	p2pdma->pool = NULL;
+	return ret;
 }
 
 static void pci_p2pdma_unmap_mappings(void *data)

@@ -276,7 +326,7 @@ static void pci_p2pdma_unmap_mappings(void *data)
 	 * unmap_mapping_range() on the inode, teardown any existing userspace
 	 * mappings and prevent new ones from being created.
 	 */
-	sysfs_remove_file_from_group(&p2p_pgmap->mem.owner->kobj,
+	sysfs_remove_file_from_group(&p2p_pgmap->mem->owner->kobj,
 				     &p2pmem_alloc_attr.attr,
 				     p2pmem_group.name);
 }

@@ -295,6 +345,7 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 			    u64 offset)
 {
 	struct pci_p2pdma_pagemap *p2p_pgmap;
+	struct p2pdma_provider *mem;
 	struct dev_pagemap *pgmap;
 	struct pci_p2pdma *p2pdma;
 	void *addr;

@@ -312,15 +363,25 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	if (size + offset > pci_resource_len(pdev, bar))
 		return -EINVAL;
 
-	if (!pdev->p2pdma) {
-		error = pci_p2pdma_setup(pdev);
+	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (!p2pdma) {
+		mem = pcim_p2pdma_enable(pdev, bar);
+		if (IS_ERR(mem))
+			return PTR_ERR(mem);
+
+		error = pci_p2pdma_setup_pool(pdev);
 		if (error)
 			return error;
-	}
+
+		p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
+	} else
+		mem = &p2pdma->mem[bar];
 
 	p2p_pgmap = devm_kzalloc(&pdev->dev, sizeof(*p2p_pgmap), GFP_KERNEL);
-	if (!p2p_pgmap)
-		return -ENOMEM;
+	if (!p2p_pgmap) {
+		error = -ENOMEM;
+		goto free_pool;
+	}
 
 	pgmap = &p2p_pgmap->pgmap;
 	pgmap->range.start = pci_resource_start(pdev, bar) + offset;

@@ -328,9 +389,7 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	pgmap->nr_range = 1;
 	pgmap->type = MEMORY_DEVICE_PCI_P2PDMA;
 	pgmap->ops = &p2pdma_pgmap_ops;
-	p2p_pgmap->mem.owner = &pdev->dev;
-	p2p_pgmap->mem.bus_offset =
-		pci_bus_address(pdev, bar) - pci_resource_start(pdev, bar);
+	p2p_pgmap->mem = mem;
 
 	addr = devm_memremap_pages(&pdev->dev, pgmap);
 	if (IS_ERR(addr)) {

@@ -343,7 +402,6 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 	if (error)
 		goto pages_free;
 
-	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
 	error = gen_pool_add_owner(p2pdma->pool, (unsigned long)addr,
 			pci_bus_address(pdev, bar) + offset,
 			range_len(&pgmap->range), dev_to_node(&pdev->dev),

@@ -359,7 +417,10 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 pages_free:
 	devm_memunmap_pages(&pdev->dev, pgmap);
 pgmap_free:
-	devm_kfree(&pdev->dev, pgmap);
+	devm_kfree(&pdev->dev, p2p_pgmap);
+free_pool:
+	sysfs_remove_group(&pdev->dev.kobj, &p2pmem_group);
+	gen_pool_destroy(p2pdma->pool);
 	return error;
 }
 EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource);

@@ -1008,11 +1069,11 @@ void __pci_p2pdma_update_state(struct pci_p2pdma_map_state *state,
 {
 	struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(page_pgmap(page));
 
-	if (state->mem == &p2p_pgmap->mem)
+	if (state->mem == p2p_pgmap->mem)
 		return;
 
-	state->mem = &p2p_pgmap->mem;
-	state->map = pci_p2pdma_map_type(&p2p_pgmap->mem, dev);
+	state->mem = p2p_pgmap->mem;
+	state->map = pci_p2pdma_map_type(p2p_pgmap->mem, dev);
 }
 
 /**

include/linux/pci-p2pdma.h +5 -0

index eef96636c67e6..888ad7b0c54cf 100644
@@ -27,6 +27,7 @@ struct p2pdma_provider {
 };
 
 #ifdef CONFIG_PCI_P2PDMA
+struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar);
 int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 		u64 offset);
 int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients,

@@ -45,6 +46,10 @@ int pci_p2pdma_enable_store(const char *page, struct pci_dev **p2p_dev,
 ssize_t pci_p2pdma_enable_show(char *page, struct pci_dev *p2p_dev,
 			       bool use_p2pdma);
 #else /* CONFIG_PCI_P2PDMA */
+static inline struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
 static inline int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar,
 		size_t size, u64 offset)
 {

Jason Gunthorpe

maintainer

2025-09-23 15:04 UTC #5 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 3 lines · Alex Williamson

But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?

It makes the most sense if the P2P is activated once during probe(), it is just a cheap memory allocation, so no reason not to.

If you try to do it on-demand then it will require more locking.

Quoted 5 lines

It also seems like the coordination of a valid provider is ad-hoc between p2pdma and vfio-pci. For example, this only fills providers for MMIO BARs and vfio-pci validates that dmabuf operations are for MMIO BARs, but it would be more consistent if vfio-pci relied on p2pdma to give it a valid provider for a given BAR. Thanks,

Yeah, validate_dmabuf_input() should check priv->vdev->provider[priv->bar] for NULL and I think we should directly store the non-NUL: provider in the dmabuf priv struct instead of the bar index and replace these:

+ provider = priv->vdev->provider[priv->bar];
+ provider = priv->vdev->provider[priv->bar];

Jason

Alex Williamson

2025-09-23 17:30 UTC #6 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 9 lines · On Tue, 23 Sep 2025 12:04:14 -0300 Jason Gunthorpe <[email protected]> wrote:

Quoted 3 lines · Alex Williamson
But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?
It makes the most sense if the P2P is activated once during probe(), it is just a cheap memory allocation, so no reason not to.
If you try to do it on-demand then it will require more locking.

I'm only wondering about splitting to an "initialize/setup" function where providers for each BAR are setup, and a "get provider" interface, which doesn't really seem to be a hot path anyway. Batching could still be done to setup all BAR providers at once.

However, the setup isn't really once per probe(), even in the case of a new driver probing we re-use the previously setup providers. Doesn't that introduce a problem that the provider bus_offset can be stale if something like a BAR resize has occurred between drivers?

Possibly the providers should be setup in PCI core, a re-init triggered for resource updates, and the driver interface is only to get the provider. Thanks,

Alex

Jason Gunthorpe

maintainer

2025-09-23 17:43 UTC #7 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 11 lines · Alex Williamson

Quoted 8 lines · On Tue, 23 Sep 2025 12:04:14 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 3 lines · Alex Williamson
But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?
It makes the most sense if the P2P is activated once during probe(), it is just a cheap memory allocation, so no reason not to.

Quoted 6 lines

Quoted 1 line
If you try to do it on-demand then it will require more locking.
I'm only wondering about splitting to an "initialize/setup" function where providers for each BAR are setup, and a "get provider" interface, which doesn't really seem to be a hot path anyway. Batching could still be done to setup all BAR providers at once.

I agree it is a weird interface, but it is close to the existing weird interface :\

Quoted 2 lines

However, the setup isn't really once per probe(), even in the case of a new driver probing we re-use the previously setup providers.

It uses devm to call pci_p2pdma_release() which NULL's pdev->p2pdma.

Jason

Alex Williamson

2025-09-23 18:09 UTC #8 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 14 lines · On Tue, 23 Sep 2025 14:43:33 -0300 Jason Gunthorpe <[email protected]> wrote:

Quoted 13 lines · Alex Williamson
Quoted 9 lines · On Tue, 23 Sep 2025 12:04:14 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 3 lines · Alex Williamson
But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?
It makes the most sense if the P2P is activated once during probe(), it is just a cheap memory allocation, so no reason not to.
If you try to do it on-demand then it will require more locking.

Quoted 7 lines

Quoted 4 lines
I'm only wondering about splitting to an "initialize/setup" function where providers for each BAR are setup, and a "get provider" interface, which doesn't really seem to be a hot path anyway. Batching could still be done to setup all BAR providers at once.
I agree it is a weird interface, but it is close to the existing weird interface :\

Seems like it would help if we just positioned it as a "get provider for BAR" function that happens to initialize all the providers on the first call, rather than an "enable" function with some strange BAR argument and provider return. pcim_p2pdma_provider(pdev, bar)?

It would at least make sense to me then to store the provider on the vfio_pci_dma_buf object at the time of the get feature call rather than vfio_pci_core_init_dev() though. That would eliminate patch 08/ and the inline #ifdefs.

Quoted 4 lines

Quoted 2 lines
However, the setup isn't really once per probe(), even in the case of a new driver probing we re-use the previously setup providers.
It uses devm to call pci_p2pdma_release() which NULL's pdev->p2pdma.

Ah, right. So the /* PCI device was "rebound" to the driver */ comment is further misleading, a new probe would do a new setup. Thanks,

Alex

Leon Romanovsky

author

2025-09-25 07:03 UTC #9 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 30 lines · Alex Williamson

Quoted 14 lines · On Tue, 23 Sep 2025 14:43:33 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 13 lines · Alex Williamson
Quoted 9 lines · On Tue, 23 Sep 2025 12:04:14 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 3 lines · Alex Williamson
But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?
It makes the most sense if the P2P is activated once during probe(), it is just a cheap memory allocation, so no reason not to.
If you try to do it on-demand then it will require more locking.
Quoted 7 lines
Quoted 4 lines
I'm only wondering about splitting to an "initialize/setup" function where providers for each BAR are setup, and a "get provider" interface, which doesn't really seem to be a hot path anyway. Batching could still be done to setup all BAR providers at once.
I agree it is a weird interface, but it is close to the existing weird interface :\
Seems like it would help if we just positioned it as a "get provider for BAR" function that happens to initialize all the providers on the first call, rather than an "enable" function with some strange BAR argument and provider return. pcim_p2pdma_provider(pdev, bar)?

Quoted 4 lines

It would at least make sense to me then to store the provider on the vfio_pci_dma_buf object at the time of the get feature call rather than vfio_pci_core_init_dev() though. That would eliminate patch 08/ and the inline #ifdefs.

I'll change it now. If "enable" function goes to be "get" function, we
won't need to store anything in vfio_pci_dma_buf too. At the end, we
have exactly two lines "provider = priv->vdev->provider[priv->bar];",
which can easily be changed to be "provider = pcim_p2pdma_provider(priv->vdev->pdev, priv->bar)"

Quoted 8 lines

Quoted 5 lines
Quoted 3 lines
However, the setup isn't really once per probe(), even in the case of a new driver probing we re-use the previously setup providers.
It uses devm to call pci_p2pdma_release() which NULL's pdev->p2pdma.
Ah, right. So the /* PCI device was "rebound" to the driver */ comment is further misleading, a new probe would do a new setup. Thanks,

I will fix the comment.

Thanks

Quoted 3 lines

Alex

Jason Gunthorpe

maintainer

2025-09-25 11:53 UTC #10 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 9 lines · Leon Romanovsky

Quoted 4 lines
It would at least make sense to me then to store the provider on the vfio_pci_dma_buf object at the time of the get feature call rather than vfio_pci_core_init_dev() though. That would eliminate patch 08/ and the inline #ifdefs.
I'll change it now. If "enable" function goes to be "get" function, we
won't need to store anything in vfio_pci_dma_buf too. At the end, we
have exactly two lines "provider = priv->vdev->provider[priv->bar];",
which can easily be changed to be "provider = pcim_p2pdma_provider(priv->vdev->pdev, priv->bar)"

Not without some kind of locking change. I'd keep the priv->vdev->provider[priv->bar] because setup during probe doesn't need special locking.

Jason

Alex Williamson

2025-09-25 22:31 UTC #11 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 7 lines · On Thu, 25 Sep 2025 08:53:08 -0300 Jason Gunthorpe <[email protected]> wrote:

Quoted 5 lines · Leon Romanovsky
Quoted 4 lines
It would at least make sense to me then to store the provider on the vfio_pci_dma_buf object at the time of the get feature call rather than vfio_pci_core_init_dev() though. That would eliminate patch 08/ and the inline #ifdefs.

Quoted 8 lines

Quoted 4 lines
I'll change it now. If "enable" function goes to be "get" function, we
won't need to store anything in vfio_pci_dma_buf too. At the end, we
have exactly two lines "provider = priv->vdev->provider[priv->bar];",
which can easily be changed to be "provider = pcim_p2pdma_provider(priv->vdev->pdev, priv->bar)"
Not without some kind of locking change. I'd keep the priv->vdev->provider[priv->bar] because setup during probe doesn't need special locking.

Why do we need to store the provider on the vfio_pci_core_device at probe though, we can get it later via pcim_p2pdma_provider(). Ideally we'd take the opportunity to pull out the setup part of the _provider function to give us an initialization interface to use at probe time without an unnecessary BAR# arg. Thanks,

Alex

Jason Gunthorpe

maintainer

2025-09-25 23:02 UTC #12 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 15 lines · Alex Williamson

Quoted 12 lines · On Thu, 25 Sep 2025 08:53:08 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 9 lines · Leon Romanovsky
Quoted 4 lines
It would at least make sense to me then to store the provider on the vfio_pci_dma_buf object at the time of the get feature call rather than vfio_pci_core_init_dev() though. That would eliminate patch 08/ and the inline #ifdefs.
I'll change it now. If "enable" function goes to be "get" function, we
won't need to store anything in vfio_pci_dma_buf too. At the end, we
have exactly two lines "provider = priv->vdev->provider[priv->bar];",
which can easily be changed to be "provider = pcim_p2pdma_provider(priv->vdev->pdev, priv->bar)"

Quoted 6 lines

Quoted 3 lines
Not without some kind of locking change. I'd keep the priv->vdev->provider[priv->bar] because setup during probe doesn't need special locking.
Why do we need to store the provider on the vfio_pci_core_device at probe though, we can get it later via pcim_p2pdma_provider().

Because you'd need some new locking to prevent races.

Besides, the model here should be to call the function once during probe and get back the allocated provider. The fact internally it is kind of nutzo still shouldn't leak out as a property of the ABI.

I would like to remove this weird behavior where it caches things inside the struct device. That's not normal for an API to do that, it is only done for the genalloc path that this doesn't use.

Jason

Alex Williamson

2025-09-26 14:13 UTC #13 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 16 lines · On Thu, 25 Sep 2025 20:02:36 -0300 Jason Gunthorpe <[email protected]> wrote:

Quoted 15 lines · Alex Williamson
Quoted 12 lines · On Thu, 25 Sep 2025 08:53:08 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 9 lines · Leon Romanovsky
Quoted 4 lines
It would at least make sense to me then to store the provider on the vfio_pci_dma_buf object at the time of the get feature call rather than vfio_pci_core_init_dev() though. That would eliminate patch 08/ and the inline #ifdefs.
I'll change it now. If "enable" function goes to be "get" function, we
won't need to store anything in vfio_pci_dma_buf too. At the end, we
have exactly two lines "provider = priv->vdev->provider[priv->bar];",
which can easily be changed to be "provider = pcim_p2pdma_provider(priv->vdev->pdev, priv->bar)"

Quoted 8 lines

Quoted 6 lines
Quoted 3 lines
Not without some kind of locking change. I'd keep the priv->vdev->provider[priv->bar] because setup during probe doesn't need special locking.
Why do we need to store the provider on the vfio_pci_core_device at probe though, we can get it later via pcim_p2pdma_provider().
Because you'd need some new locking to prevent races.

The race is avoided if we simply call pcim_p2pdma_provider() during probe. We don't need to save the returned provider. That's where it seems like pulling the setup out to a separate function would eliminate this annoying BAR# arg.

Quoted 7 lines

Besides, the model here should be to call the function once during probe and get back the allocated provider. The fact internally it is kind of nutzo still shouldn't leak out as a property of the ABI.
I would like to remove this weird behavior where it caches things inside the struct device. That's not normal for an API to do that, it is only done for the genalloc path that this doesn't use.

My goal in caching the provider on the vfio p2pdma object was to avoid caching it on the vfio_pci_core_device, but now we're storing it on the struct device, the vfio_pci_core_device, AND the vfio p2pdma object. Given the current state that it's stored on the struct device, I think we only need a setup call during probe (that could be stubbed out rather than #ifdef'd), then cache the provider on the vfio p2pdma object when a dmabuf is configured. Thanks,

Alex

Leon Romanovsky

author

2025-09-28 08:15 UTC #14 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 41 lines · Alex Williamson

Quoted 16 lines · On Thu, 25 Sep 2025 20:02:36 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 15 lines · Alex Williamson
Quoted 12 lines · On Thu, 25 Sep 2025 08:53:08 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 9 lines · Leon Romanovsky
Quoted 4 lines
It would at least make sense to me then to store the provider on the vfio_pci_dma_buf object at the time of the get feature call rather than vfio_pci_core_init_dev() though. That would eliminate patch 08/ and the inline #ifdefs.
I'll change it now. If "enable" function goes to be "get" function, we
won't need to store anything in vfio_pci_dma_buf too. At the end, we
have exactly two lines "provider = priv->vdev->provider[priv->bar];",
which can easily be changed to be "provider = pcim_p2pdma_provider(priv->vdev->pdev, priv->bar)"
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Not without some kind of locking change. I'd keep the priv->vdev->provider[priv->bar] because setup during probe doesn't need special locking.
Why do we need to store the provider on the vfio_pci_core_device at probe though, we can get it later via pcim_p2pdma_provider().
Because you'd need some new locking to prevent races.
The race is avoided if we simply call pcim_p2pdma_provider() during probe. We don't need to save the returned provider. That's where it seems like pulling the setup out to a separate function would eliminate this annoying BAR# arg.
Quoted 7 lines
Besides, the model here should be to call the function once during probe and get back the allocated provider. The fact internally it is kind of nutzo still shouldn't leak out as a property of the ABI.
I would like to remove this weird behavior where it caches things inside the struct device. That's not normal for an API to do that, it is only done for the genalloc path that this doesn't use.

Quoted 7 lines

My goal in caching the provider on the vfio p2pdma object was to avoid caching it on the vfio_pci_core_device, but now we're storing it on the struct device, the vfio_pci_core_device, AND the vfio p2pdma object. Given the current state that it's stored on the struct device, I think we only need a setup call during probe (that could be stubbed out rather than #ifdef'd), then cache the provider on the vfio p2pdma object when a dmabuf is configured. Thanks,

I can do it.

Thanks

Quoted 3 lines

Alex

Leon Romanovsky

author

2025-09-23 17:12 UTC #15 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

In reply to Alex Williamson

Quoted 28 lines · Alex Williamson

Quoted 25 lines · On Thu, 11 Sep 2025 14:33:07 +0300 Leon Romanovsky <[email protected]> wrote:
From: Leon Romanovsky <[email protected]>
Refactor the PCI P2PDMA subsystem to separate the core peer-to-peer DMA functionality from the optional memory allocation layer. This creates a two-tier architecture:
The core layer provides P2P mapping functionality for physical addresses based on PCI device MMIO BARs and integrates with the DMA API for mapping operations. This layer is required for all P2PDMA users.
The optional upper layer provides memory allocation capabilities including gen_pool allocator, struct page support, and sysfs interface for user space access.
This separation allows subsystems like VFIO to use only the core P2P mapping functionality without the overhead of memory allocation features they don't need. The core functionality is now available through the new pci_p2pdma_enable() function that returns a p2pdma_provider structure.
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/pci/p2pdma.c       | 129 +++++++++++++++++++++++++++----------
 include/linux/pci-p2pdma.h |   5 ++
 2 files changed, 100 insertions(+), 34 deletions(-)

<...>

Quoted 28 lines

Quoted 11 lines
drivers/pci/p2pdma.c +9 -1
-static int pci_p2pdma_setup(struct pci_dev *pdev)
+/**
+ * pcim_p2pdma_enable - Enable peer-to-peer DMA support for a PCI device
+ * @pdev: The PCI device to enable P2PDMA for
+ * @bar: BAR index to get provider
+ *
+ * This function initializes the peer-to-peer DMA infrastructure for a PCI
+ * device. It allocates and sets up the necessary data structures to support
+ * P2PDMA operations, including mapping type tracking.
+ */
drivers/pci/p2pdma.c static void pci_p2pdma_release(void *data) · line 259
Quoted 10 lines
drivers/pci/p2pdma.c +7 -1
+struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar)
 {
-	int error = -ENOMEM;
 	struct pci_p2pdma *p2p;
+	int i, ret;
+
+	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (p2p)
+		/* PCI device was "rebound" to the driver */
+		return &p2p->mem[bar];
This seems like two separate functions rolled into one, an 'initialize providers' and a 'get provider for BAR'. The comment above even makes it sound like only a driver re-probing a device would encounter this branch, but the use case later in vfio-pci shows it to be the common case to iterate BARs for a device.

Quoted 3 lines

But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?

In addition to what Jason said about locking. The whole p2pdma.c is written with assumption that "pdev->p2pdma" pointer is assigned only once during PCI device lifetime. For example, see how sysfs files are exposed and accessed in p2pdma.c.

Once you initialize p2pdma, it is much easier to initialize all BARs at the same time.

Thanks

Alex Williamson

2025-09-23 20:07 UTC #16 lore raw

Re: [PATCH v2 03/10] PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation

Quoted 60 lines · On Tue, 23 Sep 2025 20:12:28 +0300 Leon Romanovsky <[email protected]> wrote:

Quoted 28 lines · Alex Williamson
Quoted 25 lines · On Thu, 11 Sep 2025 14:33:07 +0300 Leon Romanovsky <[email protected]> wrote:
From: Leon Romanovsky <[email protected]>
Refactor the PCI P2PDMA subsystem to separate the core peer-to-peer DMA functionality from the optional memory allocation layer. This creates a two-tier architecture:
The core layer provides P2P mapping functionality for physical addresses based on PCI device MMIO BARs and integrates with the DMA API for mapping operations. This layer is required for all P2PDMA users.
The optional upper layer provides memory allocation capabilities including gen_pool allocator, struct page support, and sysfs interface for user space access.
This separation allows subsystems like VFIO to use only the core P2P mapping functionality without the overhead of memory allocation features they don't need. The core functionality is now available through the new pci_p2pdma_enable() function that returns a p2pdma_provider structure.
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/pci/p2pdma.c       | 129 +++++++++++++++++++++++++++----------
 include/linux/pci-p2pdma.h |   5 ++
 2 files changed, 100 insertions(+), 34 deletions(-)  
<...>
Quoted 28 lines
Quoted 11 lines
drivers/pci/p2pdma.c +9 -1
-static int pci_p2pdma_setup(struct pci_dev *pdev)
+/**
+ * pcim_p2pdma_enable - Enable peer-to-peer DMA support for a PCI device
+ * @pdev: The PCI device to enable P2PDMA for
+ * @bar: BAR index to get provider
+ *
+ * This function initializes the peer-to-peer DMA infrastructure for a PCI
+ * device. It allocates and sets up the necessary data structures to support
+ * P2PDMA operations, including mapping type tracking.
+ */
drivers/pci/p2pdma.c static void pci_p2pdma_release(void *data) · line 259
Quoted 10 lines
drivers/pci/p2pdma.c +7 -1
+struct p2pdma_provider *pcim_p2pdma_enable(struct pci_dev *pdev, int bar)
 {
-	int error = -ENOMEM;
 	struct pci_p2pdma *p2p;
+	int i, ret;
+
+	p2p = rcu_dereference_protected(pdev->p2pdma, 1);
+	if (p2p)
+		/* PCI device was "rebound" to the driver */
+		return &p2p->mem[bar];
This seems like two separate functions rolled into one, an 'initialize providers' and a 'get provider for BAR'. The comment above even makes it sound like only a driver re-probing a device would encounter this branch, but the use case later in vfio-pci shows it to be the common case to iterate BARs for a device.

Quoted 8 lines

Quoted 3 lines
But then later in patch 8/ and again in 10/ why exactly do we cache the provider on the vfio_pci_core_device rather than ask for it on demand from the p2pdma?
In addition to what Jason said about locking. The whole p2pdma.c is written with assumption that "pdev->p2pdma" pointer is assigned only once during PCI device lifetime. For example, see how sysfs files are exposed and accessed in p2pdma.c.

Except as Jason identifies in the other thread, the p2pdma is a devm object, so it's assigned once during the lifetime of the driver, not the device. It seems that to get the sysfs attributes exposed, a driver would need to call pci_p2pdma_add_resource() to setup a pool, but that pool setup is only done if pci_p2pdma_add_resource() itself calls pcim_p2pdma_enable():

        p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
        if (!p2pdma) {
                mem = pcim_p2pdma_enable(pdev, bar);
                if (IS_ERR(mem))
                        return PTR_ERR(mem);

                error = pci_p2pdma_setup_pool(pdev);
		...
        } else
                mem = &p2pdma->mem[bar];

Therefore as proposed here a device bound to vfio-pci would never have these sysfs attributes.

Quoted 2 lines

Once you initialize p2pdma, it is much easier to initialize all BARs at the same time.

I didn't phrase my question above well. We can setup all the providers on the p2pdma at once, that's fine. My comment is related to the awkward API we're creating and what seems to be gratuitously caching the providers on the vfio_pci_core_device when it seems much more logical to get the provider for a specific dmabuf and cache it on the vfio_pci_dma_buf object in the device feature ioctl. We could also validate the provider at that point rather than the ad-hoc, parallel checks for MMIO BARs. Thanks,

Alex

Leon Romanovsky

author

2025-09-11 11:34 UTC #17 lore raw

PATCH v2 4/10 PCI/P2PDMA: Export pci_p2pdma_map_type() function

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Export the pci_p2pdma_map_type() function to allow external modules and subsystems to determine the appropriate mapping type for P2PDMA transfers between a provider and target device.

The function determines whether peer-to-peer DMA transfers can be done directly through PCI switches (PCI_P2PDMA_MAP_BUS_ADDR) or must go through the host bridge (PCI_P2PDMA_MAP_THRU_HOST_BRIDGE), or if the transfer is not supported at all.

This export enables subsystems like VFIO to properly handle P2PDMA operations by querying the mapping type before attempting transfers, ensuring correct DMA address programming and error handling.

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/pci/p2pdma.c       | 15 ++++++-
 include/linux/pci-p2pdma.h | 85 +++++++++++++++++++++-----------------
 2 files changed, 59 insertions(+), 41 deletions(-)

drivers/pci/p2pdma.c +13 -2

index c22cbb3a26030..7b899e2f82456 100644
@@ -1033,8 +1033,18 @@ void pci_p2pmem_publish(struct pci_dev *pdev, bool publish)
 }
 EXPORT_SYMBOL_GPL(pci_p2pmem_publish);
 
-static enum pci_p2pdma_map_type
-pci_p2pdma_map_type(struct p2pdma_provider *provider, struct device *dev)
+/**
+ * pci_p2pdma_map_type - Determine the mapping type for P2PDMA transfers
+ * @provider: P2PDMA provider structure
+ * @dev: Target device for the transfer
+ *
+ * Determines how peer-to-peer DMA transfers should be mapped between
+ * the provider and the target device. The mapping type indicates whether
+ * the transfer can be done directly through PCI switches or must go
+ * through the host bridge.
+ */
+enum pci_p2pdma_map_type pci_p2pdma_map_type(struct p2pdma_provider *provider,
+					     struct device *dev)
 {
 	enum pci_p2pdma_map_type type = PCI_P2PDMA_MAP_NOT_SUPPORTED;
 	struct pci_dev *pdev = to_pci_dev(provider->owner);

@@ -1063,6 +1073,7 @@ pci_p2pdma_map_type(struct p2pdma_provider *provider, struct device *dev)
 
 	return type;
 }
+EXPORT_SYMBOL_GPL(pci_p2pdma_map_type);
 
 void __pci_p2pdma_update_state(struct pci_p2pdma_map_state *state,
 		struct device *dev, struct page *page)

include/linux/pci-p2pdma.h +46 ~~-39~~

109 lines not on this page · raw message

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:34 UTC #18 lore raw

PATCH v2 5/10 types: move phys_vec definition to common header

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Move the struct phys_vec definition from block/blk-mq-dma.c to include/linux/types.h to make it available for use across the kernel.

The phys_vec structure represents a physical address range with a length, which is used by the new physical address-based DMA mapping API. This structure is already used by the block layer and will be needed by upcoming VFIO patches for dma-buf operations.

Moving this definition to types.h provides a centralized location for this common data structure and eliminates code duplication across subsystems that need to work with physical address ranges.

Signed-off-by Leon Romanovsky <[email protected]>

---
 block/blk-mq-dma.c    | 5 -----
 include/linux/types.h | 5 +++++
 2 files changed, 5 insertions(+), 5 deletions(-)

block/blk-mq-dma.c +0 -5

index 430e51ec494a6..8d2646ab27953 100644
@@ -5,11 +5,6 @@
 #include <linux/blk-mq-dma.h>
 #include "blk.h"
 
-struct phys_vec {
-	phys_addr_t	paddr;
-	u32		len;
-};
-
 static bool blk_map_iter_next(struct request *req, struct req_iterator *iter,
 			      struct phys_vec *vec)
 {

include/linux/types.h +5 -0

index 6dfdb8e8e4c35..2bc56681b2e62 100644
@@ -170,6 +170,11 @@ typedef u64 phys_addr_t;
 typedef u32 phys_addr_t;
 #endif
 
+struct phys_vec {
+	phys_addr_t	paddr;
+	u32		len;
+};
+
 typedef phys_addr_t resource_size_t;
 
 /*

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:34 UTC #19 lore raw

PATCH v2 6/10 vfio: Export vfio device get and put registration helpers

In reply to Leon Romanovsky

From: Vivek Kasireddy <[email protected]>

These helpers are useful for managing additional references taken on the device from other associated VFIO modules.

Original-patch-by Jason Gunthorpe <[email protected]>

Signed-off-by Vivek Kasireddy <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/vfio/vfio_main.c | 2 ++
 include/linux/vfio.h     | 2 ++
 2 files changed, 4 insertions(+)

drivers/vfio/vfio_main.c +2 -0

index 5046cae052224..2f0dcec67ffe3 100644
@@ -171,11 +171,13 @@ void vfio_device_put_registration(struct vfio_device *device)
 	if (refcount_dec_and_test(&device->refcount))
 		complete(&device->comp);
 }
+EXPORT_SYMBOL_GPL(vfio_device_put_registration);
 
 bool vfio_device_try_get_registration(struct vfio_device *device)
 {
 	return refcount_inc_not_zero(&device->refcount);
 }
+EXPORT_SYMBOL_GPL(vfio_device_try_get_registration);
 
 /*
  * VFIO driver API

include/linux/vfio.h +2 -0

index eb563f538dee5..217ba4ef17522 100644
@@ -297,6 +297,8 @@ static inline void vfio_put_device(struct vfio_device *device)
 int vfio_register_group_dev(struct vfio_device *device);
 int vfio_register_emulated_iommu_dev(struct vfio_device *device);
 void vfio_unregister_group_dev(struct vfio_device *device);
+bool vfio_device_try_get_registration(struct vfio_device *device);
+void vfio_device_put_registration(struct vfio_device *device);
 
 int vfio_assign_device_set(struct vfio_device *device, void *set_id);
 unsigned int vfio_device_set_open_count(struct vfio_device_set *dev_set);

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:34 UTC #20 lore raw

PATCH v2 7/10 vfio/pci: Add dma-buf export config for MMIO regions

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Add new kernel config which indicates support for dma-buf export of MMIO regions, which implementation is provided in next patches.

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/vfio/pci/Kconfig | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

drivers/vfio/pci/Kconfig +20 -0

index 2b0172f546652..55ae888bf26ae 100644
@@ -55,6 +55,26 @@ config VFIO_PCI_ZDEV_KVM
 
 	  To enable s390x KVM vfio-pci extensions, say Y.
 
+config VFIO_PCI_DMABUF
+	bool "VFIO PCI extensions for DMA-BUF"
+	depends on VFIO_PCI_CORE
+	depends on PCI_P2PDMA && DMA_SHARED_BUFFER
+	default y
+	help
+	  Enable support for VFIO PCI extensions that allow exporting
+	  device MMIO regions as DMA-BUFs for peer devices to access via
+	  peer-to-peer (P2P) DMA.
+
+	  This feature enables a VFIO-managed PCI device to export a portion
+	  of its MMIO BAR as a DMA-BUF file descriptor, which can be passed
+	  to other userspace drivers or kernel subsystems capable of
+	  initiating DMA to that region.
+
+	  Say Y here if you want to enable VFIO DMABUF-based MMIO export
+	  support for peer-to-peer DMA use cases.
+
+	  If unsure, say N.
+
 source "drivers/vfio/pci/mlx5/Kconfig"
 
 source "drivers/vfio/pci/hisilicon/Kconfig"

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:34 UTC #21 lore raw

PATCH v2 8/10 vfio/pci: Enable peer-to-peer DMA transactions by default

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Make sure that all VFIO PCI devices have peer-to-peer capabilities enables, so we would be able to export their MMIO memory through DMABUF,

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/vfio/pci/vfio_pci_core.c | 11 +++++++++++
 include/linux/vfio_pci_core.h    |  3 +++
 2 files changed, 14 insertions(+)

drivers/vfio/pci/vfio_pci_core.c +11 -0

index 7dcf5439dedc9..b02dda8c96341 100644
@@ -28,6 +28,9 @@
 #include <linux/nospec.h>
 #include <linux/sched/mm.h>
 #include <linux/iommufd.h>
+#ifdef CONFIG_VFIO_PCI_DMABUF
+#include <linux/pci-p2pdma.h>
+#endif
 #if IS_ENABLED(CONFIG_EEH)
 #include <asm/eeh.h>
 #endif

@@ -2085,6 +2088,7 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 {
 	struct vfio_pci_core_device *vdev =
 		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	int i;
 
 	vdev->pdev = to_pci_dev(core_vdev->dev);
 	vdev->irq_type = VFIO_PCI_NUM_IRQS;

@@ -2094,6 +2098,13 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 	INIT_LIST_HEAD(&vdev->dummy_resources_list);
 	INIT_LIST_HEAD(&vdev->ioeventfds_list);
 	INIT_LIST_HEAD(&vdev->sriov_pfs_item);
+#ifdef CONFIG_VFIO_PCI_DMABUF
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		vdev->provider[i] = pcim_p2pdma_enable(vdev->pdev, i);
+		if (IS_ERR(vdev->provider[i]))
+			return PTR_ERR(vdev->provider[i]);
+	}
+#endif
 	init_rwsem(&vdev->memory_lock);
 	xa_init(&vdev->ctx);

include/linux/vfio_pci_core.h +3 -0

index f541044e42a2a..2184ba65348b8 100644
@@ -94,6 +94,9 @@ struct vfio_pci_core_device {
 	struct vfio_pci_core_device	*sriov_pf_core_dev;
 	struct notifier_block	nb;
 	struct rw_semaphore	memory_lock;
+#ifdef CONFIG_VFIO_PCI_DMABUF
+	struct p2pdma_provider  *provider[PCI_STD_NUM_BARS];
+#endif
 };
 
 /* Will be exported for vfio pci drivers usage */

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:34 UTC #22 lore raw

PATCH v2 9/10 vfio/pci: Share the core device pointer while invoking feature functions

In reply to Leon Romanovsky

From: Vivek Kasireddy <[email protected]>

There is no need to share the main device pointer (struct vfio_device *) with all the feature functions as they only need the core device pointer. Therefore, extract the core device pointer once in the caller (vfio_pci_core_ioctl_feature) and share it instead.

Signed-off-by Vivek Kasireddy <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/vfio/pci/vfio_pci_core.c | 30 +++++++++++++-----------------
 1 file changed, 13 insertions(+), 17 deletions(-)

drivers/vfio/pci/vfio_pci_core.c +13 ~~-17~~

index b02dda8c96341..93b6cecfcab49 100644
@@ -302,11 +302,9 @@ static int vfio_pci_runtime_pm_entry(struct vfio_pci_core_device *vdev,
 	return 0;
 }
 
-static int vfio_pci_core_pm_entry(struct vfio_device *device, u32 flags,
+static int vfio_pci_core_pm_entry(struct vfio_pci_core_device *vdev, u32 flags,
 				  void __user *arg, size_t argsz)
 {
-	struct vfio_pci_core_device *vdev =
-		container_of(device, struct vfio_pci_core_device, vdev);
 	int ret;
 
 	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_SET, 0);

@@ -323,12 +321,10 @@ static int vfio_pci_core_pm_entry(struct vfio_device *device, u32 flags,
 }
 
 static int vfio_pci_core_pm_entry_with_wakeup(
-	struct vfio_device *device, u32 flags,
+	struct vfio_pci_core_device *vdev, u32 flags,
 	struct vfio_device_low_power_entry_with_wakeup __user *arg,
 	size_t argsz)
 {
-	struct vfio_pci_core_device *vdev =
-		container_of(device, struct vfio_pci_core_device, vdev);
 	struct vfio_device_low_power_entry_with_wakeup entry;
 	struct eventfd_ctx *efdctx;
 	int ret;

@@ -379,11 +375,9 @@ static void vfio_pci_runtime_pm_exit(struct vfio_pci_core_device *vdev)
 	up_write(&vdev->memory_lock);
 }
 
-static int vfio_pci_core_pm_exit(struct vfio_device *device, u32 flags,
+static int vfio_pci_core_pm_exit(struct vfio_pci_core_device *vdev, u32 flags,
 				 void __user *arg, size_t argsz)
 {
-	struct vfio_pci_core_device *vdev =
-		container_of(device, struct vfio_pci_core_device, vdev);
 	int ret;
 
 	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_SET, 0);

@@ -1476,11 +1470,10 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd,
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_ioctl);
 
-static int vfio_pci_core_feature_token(struct vfio_device *device, u32 flags,
-				       uuid_t __user *arg, size_t argsz)
+static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
+				       u32 flags, uuid_t __user *arg,
+				       size_t argsz)
 {
-	struct vfio_pci_core_device *vdev =
-		container_of(device, struct vfio_pci_core_device, vdev);
 	uuid_t uuid;
 	int ret;

@@ -1507,16 +1500,19 @@ static int vfio_pci_core_feature_token(struct vfio_device *device, u32 flags,
 int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 				void __user *arg, size_t argsz)
 {
+	struct vfio_pci_core_device *vdev =
+		container_of(device, struct vfio_pci_core_device, vdev);
+
 	switch (flags & VFIO_DEVICE_FEATURE_MASK) {
 	case VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY:
-		return vfio_pci_core_pm_entry(device, flags, arg, argsz);
+		return vfio_pci_core_pm_entry(vdev, flags, arg, argsz);
 	case VFIO_DEVICE_FEATURE_LOW_POWER_ENTRY_WITH_WAKEUP:
-		return vfio_pci_core_pm_entry_with_wakeup(device, flags,
+		return vfio_pci_core_pm_entry_with_wakeup(vdev, flags,
 							  arg, argsz);
 	case VFIO_DEVICE_FEATURE_LOW_POWER_EXIT:
-		return vfio_pci_core_pm_exit(device, flags, arg, argsz);
+		return vfio_pci_core_pm_exit(vdev, flags, arg, argsz);
 	case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN:
-		return vfio_pci_core_feature_token(device, flags, arg, argsz);
+		return vfio_pci_core_feature_token(vdev, flags, arg, argsz);
 	default:
 		return -ENOTTY;
 	}

Signature · 1 line

2.51.0

Leon Romanovsky

author

2025-09-11 11:34 UTC #23 lore raw

PATCH v2 10/10 vfio/pci: Add dma-buf export support for MMIO regions

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Add support for exporting PCI device MMIO regions through dma-buf, enabling safe sharing of non-struct page memory with controlled lifetime management. This allows RDMA and other subsystems to import dma-buf FDs and build them into memory regions for PCI P2P operations.

Signed-off-by Jason Gunthorpe <[email protected]>

Signed-off-by Vivek Kasireddy <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/vfio/pci/Makefile          |   2 +
 drivers/vfio/pci/vfio_pci_config.c |  22 +-
 drivers/vfio/pci/vfio_pci_core.c   |  18 ++
 drivers/vfio/pci/vfio_pci_dmabuf.c | 398 +++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_priv.h   |  23 ++
 include/linux/vfio_pci_core.h      |   1 +
 include/uapi/linux/vfio.h          |  25 ++
 7 files changed, 485 insertions(+), 4 deletions(-)
 create mode 100644 drivers/vfio/pci/vfio_pci_dmabuf.c

drivers/vfio/pci/Makefile +2 -0

index cf00c0a7e55c8..f9155e9c5f630 100644
@@ -2,7 +2,9 @@
 
 vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) += vfio_pci_zdev.o
+
 obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o
+vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) += vfio_pci_dmabuf.o
 
 vfio-pci-y := vfio_pci.o
 vfio-pci-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o

drivers/vfio/pci/vfio_pci_config.c +18 -4

index 8f02f236b5b4b..1f6008eabf236 100644
@@ -589,10 +589,12 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos,
 		virt_mem = !!(le16_to_cpu(*virt_cmd) & PCI_COMMAND_MEMORY);
 		new_mem = !!(new_cmd & PCI_COMMAND_MEMORY);
 
-		if (!new_mem)
+		if (!new_mem) {
 			vfio_pci_zap_and_down_write_memory_lock(vdev);
-		else
+			vfio_pci_dma_buf_move(vdev, true);
+		} else {
 			down_write(&vdev->memory_lock);
+		}
 
 		/*
 		 * If the user is writing mem/io enable (new_mem/io) and we

@@ -627,6 +629,8 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos,
 		*virt_cmd &= cpu_to_le16(~mask);
 		*virt_cmd |= cpu_to_le16(new_cmd & mask);
 
+		if (__vfio_pci_memory_enabled(vdev))
+			vfio_pci_dma_buf_move(vdev, false);
 		up_write(&vdev->memory_lock);
 	}

@@ -707,12 +711,16 @@ static int __init init_pci_cap_basic_perm(struct perm_bits *perm)
 static void vfio_lock_and_set_power_state(struct vfio_pci_core_device *vdev,
 					  pci_power_t state)
 {
-	if (state >= PCI_D3hot)
+	if (state >= PCI_D3hot) {
 		vfio_pci_zap_and_down_write_memory_lock(vdev);
-	else
+		vfio_pci_dma_buf_move(vdev, true);
+	} else {
 		down_write(&vdev->memory_lock);
+	}
 
 	vfio_pci_set_power_state(vdev, state);
+	if (__vfio_pci_memory_enabled(vdev))
+		vfio_pci_dma_buf_move(vdev, false);
 	up_write(&vdev->memory_lock);
 }

@@ -900,7 +908,10 @@ static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos,
 
 		if (!ret && (cap & PCI_EXP_DEVCAP_FLR)) {
 			vfio_pci_zap_and_down_write_memory_lock(vdev);
+			vfio_pci_dma_buf_move(vdev, true);
 			pci_try_reset_function(vdev->pdev);
+			if (__vfio_pci_memory_enabled(vdev))
+				vfio_pci_dma_buf_move(vdev, false);
 			up_write(&vdev->memory_lock);
 		}
 	}

@@ -982,7 +993,10 @@ static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos,
 
 		if (!ret && (cap & PCI_AF_CAP_FLR) && (cap & PCI_AF_CAP_TP)) {
 			vfio_pci_zap_and_down_write_memory_lock(vdev);
+			vfio_pci_dma_buf_move(vdev, true);
 			pci_try_reset_function(vdev->pdev);
+			if (__vfio_pci_memory_enabled(vdev))
+				vfio_pci_dma_buf_move(vdev, false);
 			up_write(&vdev->memory_lock);
 		}
 	}

drivers/vfio/pci/vfio_pci_core.c +18 -0

index 93b6cecfcab49..6d466faddb3a9 100644
@@ -289,6 +289,8 @@ static int vfio_pci_runtime_pm_entry(struct vfio_pci_core_device *vdev,
 	 * semaphore.
 	 */
 	vfio_pci_zap_and_down_write_memory_lock(vdev);
+	vfio_pci_dma_buf_move(vdev, true);
+
 	if (vdev->pm_runtime_engaged) {
 		up_write(&vdev->memory_lock);
 		return -EINVAL;

@@ -372,6 +374,8 @@ static void vfio_pci_runtime_pm_exit(struct vfio_pci_core_device *vdev)
 	 */
 	down_write(&vdev->memory_lock);
 	__vfio_pci_runtime_pm_exit(vdev);
+	if (__vfio_pci_memory_enabled(vdev))
+		vfio_pci_dma_buf_move(vdev, false);
 	up_write(&vdev->memory_lock);
 }

@@ -692,6 +696,8 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev)
 #endif
 	vfio_pci_core_disable(vdev);
 
+	vfio_pci_dma_buf_cleanup(vdev);
+
 	mutex_lock(&vdev->igate);
 	if (vdev->err_trigger) {
 		eventfd_ctx_put(vdev->err_trigger);

@@ -1224,7 +1230,10 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
 	 */
 	vfio_pci_set_power_state(vdev, PCI_D0);
 
+	vfio_pci_dma_buf_move(vdev, true);
 	ret = pci_try_reset_function(vdev->pdev);
+	if (__vfio_pci_memory_enabled(vdev))
+		vfio_pci_dma_buf_move(vdev, false);
 	up_write(&vdev->memory_lock);
 
 	return ret;

@@ -1513,6 +1522,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 		return vfio_pci_core_pm_exit(vdev, flags, arg, argsz);
 	case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN:
 		return vfio_pci_core_feature_token(vdev, flags, arg, argsz);
+	case VFIO_DEVICE_FEATURE_DMA_BUF:
+		return vfio_pci_core_feature_dma_buf(vdev, flags, arg, argsz);
 	default:
 		return -ENOTTY;
 	}

@@ -2100,6 +2111,8 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 		if (IS_ERR(vdev->provider[i]))
 			return PTR_ERR(vdev->provider[i]);
 	}
+
+	INIT_LIST_HEAD(&vdev->dmabufs);
 #endif
 	init_rwsem(&vdev->memory_lock);
 	xa_init(&vdev->ctx);

@@ -2465,6 +2478,7 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set,
 			break;
 		}
 
+		vfio_pci_dma_buf_move(vdev, true);
 		vfio_pci_zap_bars(vdev);
 	}

@@ -2488,6 +2502,10 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set,
 
 	ret = pci_reset_bus(pdev);
 
+	list_for_each_entry(vdev, &dev_set->device_list, vdev.dev_set_list)
+		if (__vfio_pci_memory_enabled(vdev))
+			vfio_pci_dma_buf_move(vdev, false);
+
 	vdev = list_last_entry(&dev_set->device_list,
 			       struct vfio_pci_core_device, vdev.dev_set_list);

drivers/vfio/pci/vfio_pci_dmabuf.c new file +398 -0

398 lines not on this page · raw message

drivers/vfio/pci/vfio_pci_priv.h +23 -0

index a9972eacb2936..28a405f8b97c9 100644
@@ -107,4 +107,27 @@ static inline bool vfio_pci_is_vga(struct pci_dev *pdev)
 	return (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA;
 }
 
+#ifdef CONFIG_VFIO_PCI_DMABUF
+int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
+				  struct vfio_device_feature_dma_buf __user *arg,
+				  size_t argsz);
+void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev);
+void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked);
+#else
+static inline int
+vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
+			      struct vfio_device_feature_dma_buf __user *arg,
+			      size_t argsz)
+{
+	return -ENOTTY;
+}
+static inline void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev)
+{
+}
+static inline void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev,
+					 bool revoked)
+{
+}
+#endif
+
 #endif

include/linux/vfio_pci_core.h +1 -0

index 2184ba65348b8..2d612c14d3cb3 100644
@@ -96,6 +96,7 @@ struct vfio_pci_core_device {
 	struct rw_semaphore	memory_lock;
 #ifdef CONFIG_VFIO_PCI_DMABUF
 	struct p2pdma_provider  *provider[PCI_STD_NUM_BARS];
+	struct list_head	dmabufs;
 #endif
 };

include/uapi/linux/vfio.h +25 -0

index 75100bf009baf..63214467c875a 100644
@@ -1478,6 +1478,31 @@ struct vfio_device_feature_bus_master {
 };
 #define VFIO_DEVICE_FEATURE_BUS_MASTER 10
 
+/**
+ * Upon VFIO_DEVICE_FEATURE_GET create a dma_buf fd for the
+ * regions selected.
+ *
+ * open_flags are the typical flags passed to open(2), eg O_RDWR, O_CLOEXEC,
+ * etc. offset/length specify a slice of the region to create the dmabuf from.
+ * nr_ranges is the total number of (P2P DMA) ranges that comprise the dmabuf.
+ *
+ * Return: The fd number on success, -1 and errno is set on failure.
+ */
+#define VFIO_DEVICE_FEATURE_DMA_BUF 11
+
+struct vfio_region_dma_range {
+	__u64 offset;
+	__u64 length;
+};
+
+struct vfio_device_feature_dma_buf {
+	__u32	region_index;
+	__u32	open_flags;
+	__u32   flags;
+	__u32   nr_ranges;
+	struct vfio_region_dma_range dma_ranges[];
+};
+
 /* -------- API for Type1 VFIO IOMMU -------- */
 
 /**

Signature · 1 line

2.51.0