Skip to contentSkip to conversation

PCI: Clean up match_driver flag usage

PATCH In review Lukas Wunner 3 participants 6 messages Apr 25, 2025 – May 13, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover PCI: Clean up match_driver flag usage

A small puzzle piece to improve maintainability of the PCI core:

The match_driver flag in struct pci_dev is used to postpone driver binding until all PCI devices have been enumerated.

The AMD IOMMU driver fiddles with the flag to work around breakage introduced 10 years ago. The breaking change has since been reverted, so accessing the flag appears to be superfluous and is hereby dropped (patch [1/2]). The patch needs an ack from AMD IOMMU maintainers.

This clears the way for moving the flag to struct pci_dev's priv_flags and thus prevent any further abuse outside the PCI core (patch [2/2]).

There are already two patches queued up in this cycle which amend priv_flags with new definitions for bits 4, 5 and 6 (on the pci/hotplug and pci/bwctrl topic branches), hence the bit number used here is 7.

Lukas Wunner (2):
Revert "iommu/amd: Prevent binding other PCI drivers to IOMMU PCI
devices"
PCI: Limit visibility of match_driver flag to PCI core

 drivers/iommu/amd/init.c |  3 ---
 drivers/pci/bus.c        |  4 +++-
 drivers/pci/pci-driver.c |  2 +-
 drivers/pci/pci.h        | 11 +++++++++++
 drivers/pci/probe.c      |  1 -
 include/linux/pci.h      |  2 --
 6 files changed, 15 insertions(+), 8 deletions(-)
Signature · 1 line
2.47.2
#1 lore raw

PATCH 1/2 Revert "iommu/amd: Prevent binding other PCI drivers to IOMMU PCI devices"

Commit 991de2e59090 ("PCI, x86: Implement pcibios_alloc_irq() and pcibios_free_irq()") changed IRQ handling on PCI driver probing. It inadvertently broke resume from system sleep on AMD platforms:

https://lore.kernel.org/r/[email protected]/

This was fixed by two independent commits:

* 8affb487d4a4 ("x86/PCI: Don't alloc pcibios-irq when MSI is enabled")
* cbbc00be2ce3 ("iommu/amd: Prevent binding other PCI drivers to IOMMU PCI devices")

The breaking change and one of these two fixes were subsequently reverted:

* fe25d078874f ("Revert "x86/PCI: Don't alloc pcibios-irq when MSI is

                 enabled"")
* 6c777e8799a9 ("Revert "PCI, x86: Implement pcibios_alloc_irq() and
                 pcibios_free_irq()"")

This rendered the second fix unnecessary, so revert it as well. It used the match_driver flag in struct pci_dev, which is internal to the PCI core and not supposed to be touched by arbitrary drivers.

---
I would have cc'ed Jiang Liu (author of the commit reverted here) but his Intel e-mail address appears to no longer be working. Someone with the same name has recently started to contribute using an Alibaba e-mail address, but I'm not sure it's the same person.

 drivers/iommu/amd/init.c | 3 ---
 1 file changed, 3 deletions(-)
drivers/iommu/amd/init.c +0 -3
index dd9e26b..33b6e12 100644
@@ -2030,9 +2030,6 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
 	if (!iommu->dev)
 		return -ENODEV;
 
-	/* Prevent binding other PCI device drivers to IOMMU devices */
-	iommu->dev->match_driver = false;
-
 	/* ACPI _PRT won't have an IRQ for IOMMU */
 	iommu->dev->irq_managed = 1;
 
Signature · 1 line
2.47.2
#2 lore raw

Re: [PATCH 1/2] Revert "iommu/amd: Prevent binding other PCI drivers to IOMMU PCI devices"

Dear AMD IOMMU maintainers,

Just a gentle reminder:

Please consider ack'ing the patch below, for merging through the pci tree.

The patch removes code which prevents driver binding to the pci_dev exposed by an AMD IOMMU. No other IOMMU driver does that or needs that.

The code was added to work around breakage introduced by 991de2e59090:
With that commit, resume from system sleep was broken when a driver was bound to the AMD IOMMU (e.g. vfio-pci). The commit has since been reverted, so there is no apparent reason to continue preventing driver binding to the AMD IOMMU.

Random drivers are not supposed to fiddle with the match_driver flag in struct pci_dev and having the AMD IOMMU driver do that is a maintance burden.

So unless there's a reason to keep this code, please ack the patch below.

Thank you!

Lukas

Quoted 50 lines · Lukas Wunner

Commit 991de2e59090 ("PCI, x86: Implement pcibios_alloc_irq() and pcibios_free_irq()") changed IRQ handling on PCI driver probing. It inadvertently broke resume from system sleep on AMD platforms:

https://lore.kernel.org/r/[email protected]/

This was fixed by two independent commits:

* 8affb487d4a4 ("x86/PCI: Don't alloc pcibios-irq when MSI is enabled")
* cbbc00be2ce3 ("iommu/amd: Prevent binding other PCI drivers to IOMMU PCI devices")

The breaking change and one of these two fixes were subsequently reverted:

* fe25d078874f ("Revert "x86/PCI: Don't alloc pcibios-irq when MSI is

                 enabled"")
* 6c777e8799a9 ("Revert "PCI, x86: Implement pcibios_alloc_irq() and
                 pcibios_free_irq()"")

This rendered the second fix unnecessary, so revert it as well. It used the match_driver flag in struct pci_dev, which is internal to the PCI core and not supposed to be touched by arbitrary drivers.

---
I would have cc'ed Jiang Liu (author of the commit reverted here) but his Intel e-mail address appears to no longer be working. Someone with the same name has recently started to contribute using an Alibaba e-mail address, but I'm not sure it's the same person.

 drivers/iommu/amd/init.c | 3 ---
 1 file changed, 3 deletions(-)
drivers/iommu/amd/init.c +0 -3
index dd9e26b..33b6e12 100644
@@ -2030,9 +2030,6 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
 	if (!iommu->dev)
 		return -ENODEV;
 
-	/* Prevent binding other PCI device drivers to IOMMU devices */
-	iommu->dev->match_driver = false;
-
 	/* ACPI _PRT won't have an IRQ for IOMMU */
 	iommu->dev->irq_managed = 1;
 
Signature · 1 line
2.47.2
maintainer Acked-by on v1 1/2
#3 lore raw

Re: [PATCH 1/2] Revert "iommu/amd: Prevent binding other PCI drivers to IOMMU PCI devices"

In reply to Lukas Wunner
Quoted 19 lines · Lukas Wunner

Commit 991de2e59090 ("PCI, x86: Implement pcibios_alloc_irq() and pcibios_free_irq()") changed IRQ handling on PCI driver probing. It inadvertently broke resume from system sleep on AMD platforms:

https://lore.kernel.org/r/[email protected]/

This was fixed by two independent commits:

* 8affb487d4a4 ("x86/PCI: Don't alloc pcibios-irq when MSI is enabled")
* cbbc00be2ce3 ("iommu/amd: Prevent binding other PCI drivers to IOMMU PCI devices")

The breaking change and one of these two fixes were subsequently reverted:

* fe25d078874f ("Revert "x86/PCI: Don't alloc pcibios-irq when MSI is

                 enabled"")
* 6c777e8799a9 ("Revert "PCI, x86: Implement pcibios_alloc_irq() and
                 pcibios_free_irq()"")
Quoted 5 lines

This rendered the second fix unnecessary, so revert it as well. It used the match_driver flag in struct pci_dev, which is internal to the PCI core and not supposed to be touched by arbitrary drivers.

#4 lore raw

PATCH 2/2 PCI: Limit visibility of match_driver flag to PCI core

In reply to Lukas Wunner

Since commit 58d9a38f6fac ("PCI: Skip attaching driver in device_add()"), PCI enumeration is split into two steps: In the first step, all devices are published in sysfs with device_add(). In the second step, drivers are bound to the devices with device_attach(). To delay driver binding until the second step, a "bool match_driver" in struct pci_dev is used.

Instead of a bool, use a bit in the "unsigned long priv_flags" to shrink struct pci_dev a little and prevent use of the bool outside the PCI core (as has happened with commit cbbc00be2ce3 ("iommu/amd: Prevent binding other PCI drivers to IOMMU PCI devices")).

---
 drivers/pci/bus.c        |  4 +++-
 drivers/pci/pci-driver.c |  2 +-
 drivers/pci/pci.h        | 11 +++++++++++
 drivers/pci/probe.c      |  1 -
 include/linux/pci.h      |  2 --
 5 files changed, 15 insertions(+), 5 deletions(-)
drivers/pci/bus.c +3 -1
index b685110..6904886 100644
@@ -369,7 +369,9 @@ void pci_bus_add_device(struct pci_dev *dev)
 				pdev->name);
 	}
 
-	dev->match_driver = !dn || of_device_is_available(dn);
+	if (!dn || of_device_is_available(dn))
+		pci_dev_allow_binding(dev);
+
 	retval = device_attach(&dev->dev);
 	if (retval < 0 && retval != -EPROBE_DEFER)
 		pci_warn(dev, "device attach failed (%d)\n", retval);
drivers/pci/pci-driver.c +1 -1
index c8bd71a..0c5bdb8 100644
@@ -1507,7 +1507,7 @@ static int pci_bus_match(struct device *dev, const struct device_driver *drv)
 	struct pci_driver *pci_drv;
 	const struct pci_device_id *found_id;
 
-	if (!pci_dev->match_driver)
+	if (pci_dev_binding_disallowed(pci_dev))
 		return 0;
 
 	pci_drv = (struct pci_driver *)to_pci_driver(drv);
drivers/pci/pci.h +11 -0
index b81e99c..de5d4ef 100644
@@ -557,6 +557,7 @@ static inline int pci_dev_set_disconnected(struct pci_dev *dev, void *unused)
 #define PCI_DPC_RECOVERED 1
 #define PCI_DPC_RECOVERING 2
 #define PCI_DEV_REMOVED 3
+#define PCI_DEV_ALLOW_BINDING 7
 
 static inline void pci_dev_assign_added(struct pci_dev *dev)
 {
@@ -580,6 +581,16 @@ static inline bool pci_dev_test_and_set_removed(struct pci_dev *dev)
 	return test_and_set_bit(PCI_DEV_REMOVED, &dev->priv_flags);
 }
 
+static inline void pci_dev_allow_binding(struct pci_dev *dev)
+{
+	set_bit(PCI_DEV_ALLOW_BINDING, &dev->priv_flags);
+}
+
+static inline bool pci_dev_binding_disallowed(struct pci_dev *dev)
+{
+	return !test_bit(PCI_DEV_ALLOW_BINDING, &dev->priv_flags);
+}
+
 #ifdef CONFIG_PCIEAER
 #include <linux/aer.h>
 
drivers/pci/probe.c +0 -1
index 364fa2a..126ae1e 100644
@@ -2711,7 +2711,6 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 	pci_set_msi_domain(dev);
 
 	/* Notifier could use PCI capabilities */
-	dev->match_driver = false;
 	ret = device_add(&dev->dev);
 	WARN_ON(ret < 0);
 
include/linux/pci.h +0 -2
index 51e2bd6..e29d7d4 100644
@@ -425,8 +425,6 @@ struct pci_dev {
 	struct resource resource[DEVICE_COUNT_RESOURCE]; /* I/O and memory regions + expansion ROMs */
 	struct resource driver_exclusive_resource;	 /* driver exclusive resource ranges */
 
-	bool		match_driver;		/* Skip attaching driver */
-
 	unsigned int	transparent:1;		/* Subtractive decode bridge */
 	unsigned int	io_window:1;		/* Bridge has I/O window */
 	unsigned int	pref_window:1;		/* Bridge has pref mem window */
Signature · 1 line
2.47.2
#5 lore raw
In reply to Lukas Wunner
Quoted 22 lines · Lukas Wunner

A small puzzle piece to improve maintainability of the PCI core:

The match_driver flag in struct pci_dev is used to postpone driver binding until all PCI devices have been enumerated.

The AMD IOMMU driver fiddles with the flag to work around breakage introduced 10 years ago. The breaking change has since been reverted, so accessing the flag appears to be superfluous and is hereby dropped (patch [1/2]). The patch needs an ack from AMD IOMMU maintainers.

This clears the way for moving the flag to struct pci_dev's priv_flags and thus prevent any further abuse outside the PCI core (patch [2/2]).

There are already two patches queued up in this cycle which amend priv_flags with new definitions for bits 4, 5 and 6 (on the pci/hotplug and pci/bwctrl topic branches), hence the bit number used here is 7.

Lukas Wunner (2):
Revert "iommu/amd: Prevent binding other PCI drivers to IOMMU PCI
devices"
PCI: Limit visibility of match_driver flag to PCI core

Quoted 7 lines
 drivers/iommu/amd/init.c |  3 ---
 drivers/pci/bus.c        |  4 +++-
 drivers/pci/pci-driver.c |  2 +-
 drivers/pci/pci.h        | 11 +++++++++++
 drivers/pci/probe.c      |  1 -
 include/linux/pci.h      |  2 --
 6 files changed, 15 insertions(+), 8 deletions(-)

Provisionally applied to pci/enumeration for build testing, pending ack from AMD IOMMU folks.