Skip to contentSkip to conversation

Batch of PCI peer-to-peer fixes

PATCH Applied Leon Romanovsky 3 participants 15 messages Aug 30 – Sep 10

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Batch of PCI peer-to-peer fixes

Hi Bjorn,

These fixes come from my larger series that aligns the code with the PCI specification regarding the handling of ACS bits [1].

This series has already been reviewed by Logan and Jason.

Thanks,

[1] https://lore.kernel.org/all/[email protected]/

---
Leon Romanovsky (5):
PCI/P2PDMA: Do not tear down the allocate attribute on registration failure PCI/P2PDMA: Wait for RCU readers before freeing state PCI/P2PDMA: Restrict the p2pmem search to pool backed providers PCI/P2PDMA: Safely terminate ACS redirect lists
PCI/P2PDMA: Gate the host bridge whitelist warning on verbose

 drivers/pci/p2pdma.c | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)

---
base-commit: 08dbfad3f5040f5bdb6c529da20d6d4e81fefd72
change-id: 20260830-batch-p2p-fixes-b73c10cdeff8

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>
#1 lore raw

PATCH 1/5 PCI/P2PDMA: Do not tear down the allocate attribute on registration failure

From: Leon Romanovsky <[email protected]>

pci_p2pdma_add_resource() installs pci_p2pdma_unmap_mappings() as a devres action with the devres allocated p2p_pgmap as its data, and only then adds the range to the pool:

	error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_unmap_mappings,
					 p2p_pgmap);
	if (error)
		goto pages_free;

	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
	error = gen_pool_add_owner(p2pdma->pool, ...);
	if (error)
		goto pages_free;

The action removes the allocate attribute for the whole device, which tears down existing userspace mappings of every BAR already registered on it. Both failures here get that wrong, in opposite ways.

devm_add_action_or_reset() runs the action when it cannot allocate its devres node, so an -ENOMEM while registering a second BAR unmaps the first one. Use devm_add_action() and let the error path unwind only what this call created.

gen_pool_add_owner() allocates a chunk and can also fail with -ENOMEM. There the action is registered, and the error path frees p2p_pgmap with devm_kfree() while leaving the action pointing at it. On unbind devres runs the action and pci_p2pdma_unmap_mappings() dereferences p2p_pgmap->mem->owner->kobj, which is freed memory. Give that failure its own label and drop the action with devm_remove_action(), which removes it without running it.

Fixes 7e9c7ef83d78 ("PCI/P2PDMA: Allow userspace VMA allocations through sysfs")
Fixes f58ef9d1d135 ("PCI/P2PDMA: Separate the mmap() support from the core logic")
---
 drivers/pci/p2pdma.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/pci/p2pdma.c +5 -3
index 9334eb314663..8124bcebfa2d 100644
@@ -440,8 +440,8 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 		goto pgmap_free;
 	}
 
-	error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_unmap_mappings,
-					 p2p_pgmap);
+	error = devm_add_action(&pdev->dev, pci_p2pdma_unmap_mappings,
+				p2p_pgmap);
 	if (error)
 		goto pages_free;
 
@@ -451,13 +451,15 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size,
 			range_len(&pgmap->range), dev_to_node(&pdev->dev),
 			&pgmap->ref);
 	if (error)
-		goto pages_free;
+		goto mappings_remove;
 
 	pci_info(pdev, "added peer-to-peer DMA memory %#llx-%#llx\n",
 		 pgmap->range.start, pgmap->range.end);
 
 	return 0;
 
+mappings_remove:
+	devm_remove_action(&pdev->dev, pci_p2pdma_unmap_mappings, p2p_pgmap);
 pages_free:
 	devm_memunmap_pages(&pdev->dev, pgmap);
 pgmap_free:
Signature · 1 line
2.55.0
#2 lore raw

Re: [PATCH 1/5] PCI/P2PDMA: Do not tear down the allocate attribute on registration failure

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines

From: Leon Romanovsky <[email protected]>

pci_p2pdma_add_resource() installs pci_p2pdma_unmap_mappings() as a devres action with the devres allocated p2p_pgmap as its data, and only then adds the range to the pool:

	error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_unmap_mappings,
					 p2p_pgmap);
	if (error)
		goto pages_free;

	p2pdma = rcu_dereference_protected(pdev->p2pdma, 1);
	error = gen_pool_add_owner(p2pdma->pool, ...);
	if (error)
		goto pages_free;
Quoted 6 lines
[ ... ]
Fixes f58ef9d1d135 ("PCI/P2PDMA: Separate the mmap() support from the core logic")

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

PATCH 2/5 PCI/P2PDMA: Wait for RCU readers before freeing state

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

pci_p2pmem_find_many() scans all PCI devices without locking or protection against driver unbind, including devices with poolless P2PDMA state. pci_has_p2pmem() may observe pdev->p2pdma just before driver unbind clears it, while pci_p2pdma_release() skips the grace period when no pool is present. This allows devres to free the object while it is still in use.

Clear the pointer with RCU_INIT_POINTER() and always wait for pre-existing RCU readers before returning. The same grace period continues to protect gen_pool users for pool-backed providers.

Cc Alex Williamson <[email protected]>
Cc Matt Evans <[email protected]>
Fixes 372d6d1b8ae3 ("PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation")
---
 drivers/pci/p2pdma.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/pci/p2pdma.c +2 -3
index 8124bcebfa2d..52974809e1e1 100644
@@ -236,9 +236,8 @@ static void pci_p2pdma_release(void *data)
 		return;
 
 	/* Flush and disable pci_alloc_p2p_mem() */
-	pdev->p2pdma = NULL;
-	if (p2pdma->pool)
-		synchronize_rcu();
+	RCU_INIT_POINTER(pdev->p2pdma, NULL);
+	synchronize_rcu();
 	xa_destroy(&p2pdma->map_types);
 
 	if (!p2pdma->pool)
Signature · 1 line
2.55.0
#4 lore raw

Re: [PATCH 2/5] PCI/P2PDMA: Wait for RCU readers before freeing state

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines

From: Leon Romanovsky <[email protected]>

pci_p2pmem_find_many() scans all PCI devices without locking or protection against driver unbind, including devices with poolless P2PDMA state. pci_has_p2pmem() may observe pdev->p2pdma just before driver unbind clears it, while pci_p2pdma_release() skips the grace period when no pool is present. This allows devres to free the object while it is still in use.

Clear the pointer with RCU_INIT_POINTER() and always wait for pre-existing RCU readers before returning. The same grace period continues to protect gen_pool users for pool-backed providers.

Quoted 7 lines
Cc Alex Williamson <[email protected]>
Cc Matt Evans <[email protected]>
Fixes 372d6d1b8ae3 ("PCI/P2PDMA: Refactor to separate core P2P functionality from memory allocation")

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw

PATCH 3/5 PCI/P2PDMA: Restrict the p2pmem search to pool backed providers

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

pci_p2pmem_find_many() exists to pick a provider that the caller will then allocate from with pci_alloc_p2pmem(), which goes straight to the gen_pool:

	ret = (void *)gen_pool_alloc_owner(p2pdma->pool, size, (void **) &ref);

pci_has_p2pmem() does not ask for that pool, only for the published flag. The two used to be equivalent, because a provider could only exist by way of pci_p2pdma_add_resource(), which always creates the pool.

pcim_p2pdma_init() broke that. It registers a provider for the DMABUF path and never creates a pool, so pdev->p2pdma is set while p2pdma->pool stays NULL. Nothing publishes such a provider today, so the search cannot return one yet, but the flag alone no longer says what the caller needs.

Ask for the pool as well, so the search covers the providers its result is used for. A later patch documents the pdev->p2pdma lifetime and RCU rules.

---
 drivers/pci/p2pdma.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/pci/p2pdma.c +6 -1
index 52974809e1e1..914848a993ba 100644
@@ -868,7 +868,12 @@ static bool pci_has_p2pmem(struct pci_dev *pdev)
 
 	rcu_read_lock();
 	p2pdma = rcu_dereference(pdev->p2pdma);
-	res = p2pdma && p2pdma->p2pmem_published;
+	/*
+	 * The callers hand the result to pci_alloc_p2pmem(), so only a
+	 * provider backed by a pool is of any use here. pcim_p2pdma_init()
+	 * creates providers without one.
+	 */
+	res = p2pdma && p2pdma->pool && p2pdma->p2pmem_published;
 	rcu_read_unlock();
 
 	return res;
Signature · 1 line
2.55.0
#6 lore raw

Re: [PATCH 3/5] PCI/P2PDMA: Restrict the p2pmem search to pool backed providers

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 18 lines

From: Leon Romanovsky <[email protected]>

pci_p2pmem_find_many() exists to pick a provider that the caller will then allocate from with pci_alloc_p2pmem(), which goes straight to the gen_pool:

	ret = (void *)gen_pool_alloc_owner(p2pdma->pool, size, (void **) &ref);

pci_has_p2pmem() does not ask for that pool, only for the published flag. The two used to be equivalent, because a provider could only exist by way of pci_p2pdma_add_resource(), which always creates the pool.

pcim_p2pdma_init() broke that. It registers a provider for the DMABUF path and never creates a pool, so pdev->p2pdma is set while p2pdma->pool stays NULL. Nothing publishes such a provider today, so the search cannot return one yet, but the flag alone no longer says what the caller needs.

Quoted 8 lines

Ask for the pool as well, so the search covers the providers its result is used for. A later patch documents the pdev->p2pdma lifetime and RCU rules.

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#7 lore raw

PATCH 4/5 PCI/P2PDMA: Safely terminate ACS redirect lists

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

seq_buf marks an overflow by setting len to size + 1. The ACS diagnostic path unconditionally writes a terminator to buffer[len - 1], so a path with enough ACS ports to fill the 128-byte buffer writes one byte beyond the buffer when verbose diagnostics are requested.

Use seq_buf_str() to terminate truncated output safely and remove the final semicolon only when the buffer did not overflow.

Fixes 52916982af48 ("PCI/P2PDMA: Support peer-to-peer memory")
---
 drivers/pci/p2pdma.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/pci/p2pdma.c +4 -2
index 914848a993ba..becc869cbc17 100644
@@ -772,11 +772,13 @@ calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client,
 	}
 
 	if (verbose) {
-		acs_list.buffer[acs_list.len-1] = 0; /* drop final semicolon */
+		/* Drop the final semicolon; the list is not empty here. */
+		if (!seq_buf_has_overflowed(&acs_list))
+			acs_list.buffer[acs_list.len - 1] = '\0';
 		pci_warn(client, "ACS redirect is set between the client and provider (%s)\n",
 			 pci_name(provider));
 		pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n",
-			 acs_list.buffer);
+			 seq_buf_str(&acs_list));
 	}
 	acs_redirects = true;
 
Signature · 1 line
2.55.0
#8 lore raw

Re: [PATCH 4/5] PCI/P2PDMA: Safely terminate ACS redirect lists

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 7 lines

From: Leon Romanovsky <[email protected]>

seq_buf marks an overflow by setting len to size + 1. The ACS diagnostic path unconditionally writes a terminator to buffer[len - 1], so a path with enough ACS ports to fill the 128-byte buffer writes one byte beyond the buffer when verbose diagnostics are requested.

Quoted 8 lines

Use seq_buf_str() to terminate truncated output safely and remove the final semicolon only when the buffer did not overflow.

Fixes 52916982af48 ("PCI/P2PDMA: Support peer-to-peer memory")

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#9 lore raw

PATCH 5/5 PCI/P2PDMA: Gate the host bridge whitelist warning on verbose

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

calc_map_type_and_dist() prints every other diagnostic under its verbose argument, but reaches the "Host bridge not in P2PDMA whitelist" warning through host_bridge_whitelist(), which it hands acs_redirects instead. A caller that asked for a silent answer still gets the warning whenever any port on the path has an ACS redirect bit set, the CPU is not whitelisted by cpu_supports_p2pdma(), and the host bridge is not in pci_p2pdma_whitelist[].

pci_p2pmem_find_many() is such a caller. It sweeps every device with published p2pmem and asks for the distance to each client with verbose=false, and pci_p2pdma_distance_many() recomputes rather than consulting the map_types cache, so the warning repeats on every sweep.

The argument was never meant to say "ACS redirects were found". When commit cf201bfe8cdc ("PCI/P2PDMA: Warn if host bridge not in whitelist") added it, acs_redirects was a bool pointer that the quiet entry point passed as NULL:

	if (verbose)
		map = calc_map_type_and_dist_warn(provider, pci_client,
						  &distance);
	else
		map = calc_map_type_and_dist(provider, pci_client,
					     &distance, NULL, NULL);

so the argument was true on exactly the path that commit describes. Folding the two entry points into one verbose flag turned the pointer into a value and left the call site alone, silently narrowing the warning to paths that carry an ACS redirect.

Pass verbose. This also restores the warning for a verbose caller that takes the host bridge route with no ACS redirect on the path, which until now was told it could not use peer-to-peer DMA without being told which vendor and device would have to be added to the whitelist.

Fixes d1b8dc09dd71 ("PCI/P2PDMA: Simplify distance calculation")
---
 drivers/pci/p2pdma.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/pci/p2pdma.c +1 -3
index becc869cbc17..349537b4fa42 100644
@@ -709,7 +709,6 @@ calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client,
 {
 	enum pci_p2pdma_map_type map_type = PCI_P2PDMA_MAP_THRU_HOST_BRIDGE;
 	struct pci_dev *a = provider, *b = client, *bb;
-	bool acs_redirects = false;
 	struct pci_p2pdma *p2pdma;
 	struct seq_buf acs_list;
 	int acs_cnt = 0;
@@ -780,11 +779,10 @@ calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client,
 		pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n",
 			 seq_buf_str(&acs_list));
 	}
-	acs_redirects = true;
 
 map_through_host_bridge:
 	if (!cpu_supports_p2pdma() &&
-	    !host_bridge_whitelist(provider, client, acs_redirects)) {
+	    !host_bridge_whitelist(provider, client, verbose)) {
 		if (verbose)
 			pci_warn(client, "cannot be used for peer-to-peer DMA as the client and provider (%s) do not share an upstream bridge or whitelisted host bridge\n",
 				 pci_name(provider));
Signature · 1 line
2.55.0
#10 lore raw

Re: [PATCH 5/5] PCI/P2PDMA: Gate the host bridge whitelist warning on verbose

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines

From: Leon Romanovsky <[email protected]>

calc_map_type_and_dist() prints every other diagnostic under its verbose argument, but reaches the "Host bridge not in P2PDMA whitelist" warning through host_bridge_whitelist(), which it hands acs_redirects instead. A caller that asked for a silent answer still gets the warning whenever any port on the path has an ACS redirect bit set, the CPU is not whitelisted by cpu_supports_p2pdma(), and the host bridge is not in pci_p2pdma_whitelist[].

pci_p2pmem_find_many() is such a caller. It sweeps every device with published p2pmem and asks for the distance to each client with verbose=false, and pci_p2pdma_distance_many() recomputes rather than consulting the map_types cache, so the warning repeats on every sweep.

Quoted 6 lines
[ ... ]
Fixes d1b8dc09dd71 ("PCI/P2PDMA: Simplify distance calculation")

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
#11 lore raw

Re: [PATCH 5/5] PCI/P2PDMA: Gate the host bridge whitelist warning on verbose

In reply to Leon Romanovsky
Quoted 9 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

calc_map_type_and_dist() prints every other diagnostic under its verbose argument, but reaches the "Host bridge not in P2PDMA whitelist" warning through host_bridge_whitelist(), which it hands acs_redirects instead. A caller that asked for a silent answer still gets the warning whenever any port on the path has an ACS redirect bit set, the CPU is not whitelisted by cpu_supports_p2pdma(), and the host bridge is not in pci_p2pdma_whitelist[].

Not in this series, but I would like to see the "whitelist" term removed from drivers/pci. p2pdma.c and pci-pf-stub.c are the only current uses, and I don't think a change would cause any API issue.

There are similar "blacklist" uses in pci.c, aspm.c, and vpd.c.

#12 lore raw

Re: [PATCH 5/5] PCI/P2PDMA: Gate the host bridge whitelist warning on verbose

Quoted 11 lines · Bjorn Helgaas
Quoted 9 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

calc_map_type_and_dist() prints every other diagnostic under its verbose argument, but reaches the "Host bridge not in P2PDMA whitelist" warning through host_bridge_whitelist(), which it hands acs_redirects instead. A caller that asked for a silent answer still gets the warning whenever any port on the path has an ACS redirect bit set, the CPU is not whitelisted by cpu_supports_p2pdma(), and the host bridge is not in pci_p2pdma_whitelist[].

Quoted 3 lines

Not in this series, but I would like to see the "whitelist" term removed from drivers/pci. p2pdma.c and pci-pf-stub.c are the only current uses, and I don't think a change would cause any API issue.

Let's do it after the "Fix ACS ..." series, please, just to reduce the number of rebases on my side.

Thanks

Quoted 3 lines

There are similar "blacklist" uses in pci.c, aspm.c, and vpd.c.

#13 lore raw
In reply to Leon Romanovsky
Quoted 20 lines · Leon Romanovsky

Hi Bjorn,

These fixes come from my larger series that aligns the code with the PCI specification regarding the handling of ACS bits [1].

This series has already been reviewed by Logan and Jason.

Thanks,

[1] https://lore.kernel.org/all/[email protected]/

---
Leon Romanovsky (5):
PCI/P2PDMA: Do not tear down the allocate attribute on registration failure PCI/P2PDMA: Wait for RCU readers before freeing state PCI/P2PDMA: Restrict the p2pmem search to pool backed providers PCI/P2PDMA: Safely terminate ACS redirect lists
PCI/P2PDMA: Gate the host bridge whitelist warning on verbose

Quoted 5 lines
 drivers/pci/p2pdma.c | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)

---
base-commit: 08dbfad3f5040f5bdb6c529da20d6d4e81fefd72
change-id: 20260830-batch-p2p-fixes-b73c10cdeff8

Bjorn,

Can we please make progress on this series? I need it to be applied so that I can send my "Fix ACS ..." series with an upstream base commit. This will allow Sashiko to apply and review it.

Thanks

Quoted 6 lines

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>
#14 lore raw
In reply to Leon Romanovsky
Quoted 20 lines · Leon Romanovsky

Hi Bjorn,

These fixes come from my larger series that aligns the code with the PCI specification regarding the handling of ACS bits [1].

This series has already been reviewed by Logan and Jason.

Thanks,

[1] https://lore.kernel.org/all/[email protected]/

---
Leon Romanovsky (5):
PCI/P2PDMA: Do not tear down the allocate attribute on registration failure PCI/P2PDMA: Wait for RCU readers before freeing state PCI/P2PDMA: Restrict the p2pmem search to pool backed providers PCI/P2PDMA: Safely terminate ACS redirect lists
PCI/P2PDMA: Gate the host bridge whitelist warning on verbose

Quoted 2 lines
 drivers/pci/p2pdma.c | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)

Applied to pci/p2pdma for v7.4, thanks!