hw/pci/pci: Enforce pci_setup_iommu_per_bus() is called only once per bus
PATCH 1/1 hw/pci/pci: Enforce pci_setup_iommu_per_bus() is called only once per bus
Currently it is possible to attach several arm-smmuv3 devices to the same bus although it is a wrong setup.
Change the prototype of pci_setup_iommu_per_bus to pass an error handle. This latter is set when iommu_per_bus is already set and used by the single caller (smmu_base_realize) to report a useful error to the end-user.
While at it document pci_setup_iommu_per_bus callback in the header.
--- include/hw/pci/pci.h | 16 +++++++++++++++- hw/arm/smmu-common.c | 2 +- hw/pci/pci.c | 9 +++++++-- 3 files changed, 23 insertions(+), 4 deletions(-)
include/hw/pci/pci.h +15 -1
@@ -863,7 +863,21 @@ int pci_iommu_unregister_iotlb_notifier(PCIDevice *dev, uint32_t pasid,*/ void pci_setup_iommu (PCIBus *bus,const PCIIOMMUOps *ops,void *opaque); -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops,void *opaque); +/** +* pci_setup_iommu_per_bus: Initialize specific IOMMU handlers for a PCIBus +* +* Similar to pci_setup_iommu but enforces that the iommu only protects +* @bus downstream end points and no other bus hierarchy +* +* @bus: the #PCIBus being updated. +* @ops: the #PCIIOMMUOps +* @opaque: passed to callbacks of the @ops structure. +* @errp: error handle +* +* Returns false on failure with @errp set, true on success +*/ +bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp);pcibus_t pci_bar_address (PCIDevice *d,int reg,uint8_t type,pcibus_t size);
hw/arm/smmu-common.c +1 -1
@@ -981,7 +981,7 @@ static void smmu_base_realize(DeviceState *dev, Error **errp) }if (s->smmu_per_bus) { -pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s); +pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s, errp); }else {pci_setup_iommu (pci_bus, s->iommu_ops, s); }
hw/pci/pci.c +7 -2
@@ -3307,11 +3307,16 @@ void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque)* IOMMU ops are returned, avoiding the use of the parent’s IOMMU when * it's not appropriate. */ -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, -void *opaque) +bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp) { +if (bus->iommu_per_bus) { +error_setg (errp,"An iommu is already attached to this bus" ); +return false; + }pci_setup_iommu (bus, ops, opaque); bus->iommu_per_bus = true; +return true; }static void pci_dev_get_w64 (PCIBus *b,PCIDevice *dev,void *opaque)
Signature · 1 line
2.53.0
Hi Eric,
Quoted 71 lines
Currently it is possible to attach several arm-smmuv3 devices to the same bus although it is a wrong setup.
Change the prototype of
pci_setup_iommu_per_busto pass an error handle. This latter is set wheniommu_per_busis already set and used by the single caller (smmu_base_realize) to report a useful error to the end-user.While at it document
pci_setup_iommu_per_buscallback in the header.Fixes 66d2f665e163 ("hw/arm/virt: Allow user-creatable SMMUv3 dev instantiation")Signed-off-by Eric Auger <[email protected]>--- include/hw/pci/pci.h | 16 +++++++++++++++- hw/arm/smmu-common.c | 2 +- hw/pci/pci.c | 9 +++++++-- 3 files changed, 23 insertions(+), 4 deletions(-)include/hw/pci/pci.h +15
-1@@ -863,7 +863,21 @@ int pci_iommu_unregister_iotlb_notifier(PCIDevice *dev, uint32_t pasid,*/ void pci_setup_iommu (PCIBus *bus,const PCIIOMMUOps *ops,void *opaque); -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops,void *opaque); +/** +* pci_setup_iommu_per_bus: Initialize specific IOMMU handlers for a PCIBus +* +* Similar to pci_setup_iommu but enforces that the iommu only protects +* @bus downstream end points and no other bus hierarchy +* +* @bus: the #PCIBus being updated. +* @ops: the #PCIIOMMUOps +* @opaque: passed to callbacks of the @ops structure. +* @errp: error handle +* +* Returns false on failure with @errp set, true on success +*/ +bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp);pcibus_t pci_bar_address (PCIDevice *d,int reg,uint8_t type,pcibus_t size);hw/arm/smmu-common.c +1
-1@@ -981,7 +981,7 @@ static void smmu_base_realize(DeviceState *dev, Error **errp) }if (s->smmu_per_bus) { -pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s); +pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s, errp); }else {pci_setup_iommu (pci_bus, s->iommu_ops, s); }hw/pci/pci.c +2
-2@@ -3307,11 +3307,16 @@ void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque)* IOMMU ops are returned, avoiding the use of the parent’s IOMMU when * it's not appropriate. */ -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, -void *opaque) +bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp) {
hw/pci/pci.c void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque) · line 3322
Quoted 10 lines
hw/pci/pci.c +5
-0+if (bus->iommu_per_bus) { +error_setg (errp,"An iommu is already attached to this bus" ); +return false; + }pci_setup_iommu (bus, ops, opaque); bus->iommu_per_bus = true; +return true; }static void pci_dev_get_w64 (PCIBus *b,PCIDevice *dev,void *opaque)
I verified that assigning multiple SMMUv3 to the same bus is prevented by this patch, and confirmed that basic sanity testing is unaffected in the normal case with one SMMUv3 per bus (GPU passthrough with CUDA test apps).
Thanks,
Nathan
Quoted 23 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 23 March 2026 18:30
To: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>
Subject: [PATCH] hw/pci/pci: Enforcepci_setup_iommu_per_bus()is called only once per busExternal email: Use caution opening links or attachments
Currently it is possible to attach several arm-smmuv3 devices to the same bus although it is a wrong setup.
Change the prototype of
pci_setup_iommu_per_busto pass an error handle. This latter is set wheniommu_per_busis already set and used by the single caller (smmu_base_realize) to report a useful error to the end-user.
Quoted 5 lines
While at it document
pci_setup_iommu_per_buscallback in the header.Fixes 66d2f665e163 ("hw/arm/virt: Allow user-creatable SMMUv3 devinstantiation")
Signed-off-by Eric Auger <[email protected]>
LGTM,
Thanks,
Shameer
Quoted 78 lines
--- include/hw/pci/pci.h | 16 +++++++++++++++- hw/arm/smmu-common.c | 2 +- hw/pci/pci.c | 9 +++++++-- 3 files changed, 23 insertions(+), 4 deletions(-)include/hw/pci/pci.h +2
-1@@ -863,7 +863,21 @@ int pci_iommu_unregister_iotlb_notifier(PCIDevice*dev, uint32_t pasid, */ void pci_setup_iommu (PCIBus *bus,const PCIIOMMUOps *ops,void *opaque); -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, void *opaque); +/** +* pci_setup_iommu_per_bus: Initialize specific IOMMU handlers for a PCIBus
+ *
+ * Similar topci_setup_iommubut enforces that the iommu only protects
+ * @bus downstream end points and no other bus hierarchy
+ *
+ * @bus: the #PCIBus being updated.
+ * @ops: the #PCIIOMMUOps
+ * @opaque: passed to callbacks of the @ops structure.
+ * @errp: error handle
+ *
+ * Returns false on failure with @errp set, true on success
+ */include/hw/pci/pci.h +2
-0+bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp);pcibus_t pci_bar_address (PCIDevice *d,int reg,uint8_t type,pcibus_t size);hw/arm/smmu-common.c +1
-1@@ -981,7 +981,7 @@ static void smmu_base_realize(DeviceState *dev, Error **errp) }if (s->smmu_per_bus) { -pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s); +pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s, errp); }else {pci_setup_iommu (pci_bus, s->iommu_ops, s);}hw/pci/pci.c +0
-0@@ -3307,11 +3307,16 @@ void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque) * IOMMU ops are returned, avoiding the use of the parentâEUR(tm)sIOMMU when
* it's not appropriate.
*/hw/pci/pci.c +7
-3-void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, -void *opaque) +bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp) { +if (bus->iommu_per_bus) { +error_setg (errp,"An iommu is already attached to this bus" ); +return false; + }pci_setup_iommu (bus, ops, opaque); bus->iommu_per_bus = true; +return true; }static void pci_dev_get_w64 (PCIBus *b,PCIDevice *dev,void *opaque) --2.53.0
Quoted 6 lines · Eric Auger
hw/pci/pci.c +0
-2@@ -3307,11 +3307,16 @@ void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque)* IOMMU ops are returned, avoiding the use of the parent’s IOMMU when * it's not appropriate. */ -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, -void *opaque)
hw/pci/pci.c void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque) · line 3319
Quoted 10 lines
hw/pci/pci.c +7
-0+bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp) { +if (bus->iommu_per_bus) { +error_setg (errp,"An iommu is already attached to this bus" ); +return false; + }pci_setup_iommu (bus, ops, opaque); bus->iommu_per_bus = true; +return true;
Do we need the same check in pci_setup_iommu()?
Apart from this question,
Quoted 10 lines · Nicolin Chen
Quoted 9 lines · Eric Auger
hw/pci/pci.c +2
-2@@ -3307,11 +3307,16 @@ void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque)* IOMMU ops are returned, avoiding the use of the parent’s IOMMU when * it's not appropriate. */ -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, -void *opaque) +bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp) {
Quoted 8 lines
hw/pci/pci.c void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque) · line 3319
Quoted 7 lines
hw/pci/pci.c +5
-0+if (bus->iommu_per_bus) { +error_setg (errp,"An iommu is already attached to this bus" ); +return false; + }pci_setup_iommu (bus, ops, opaque); bus->iommu_per_bus = true; +return true;Do we need the same check in
pci_setup_iommu()?
For machine wide IOMMU the check is handled in machine code (at least in arm virt).
Nevertheless a more generic solution could be to check in pci_setup_iommu() that bus->iommu_ops is not already set. If set, return an error, cascaded up through pci_setup_iommu() and pci_setup_iommu_per_bus().
However the change is much more invasive and impacts several archs. What do you think?
I don't think this can be a valid use case to call pci_setup_iommu() several times on the same bus?
Quoted 4 lines
Apart from this question,
Reviewed-by Nicolin Chen <[email protected]>
Thanks
Eric
Quoted 1 line
Hi Nicolin,
Quoted 13 lines · Eric Auger
Quoted 10 lines · Nicolin Chen
Quoted 9 lines · Eric Auger
hw/pci/pci.c +2
-2@@ -3307,11 +3307,16 @@ void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque)* IOMMU ops are returned, avoiding the use of the parent’s IOMMU when * it's not appropriate. */ -void pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, -void *opaque) +bool pci_setup_iommu_per_bus (PCIBus *bus,const PCIIOMMUOps *ops, +void *opaque,Error **errp) {
Quoted 8 lines
Quoted 8 lines
hw/pci/pci.c void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque) · line 3319
Quoted 7 lines
hw/pci/pci.c +5
-0+if (bus->iommu_per_bus) { +error_setg (errp,"An iommu is already attached to this bus" ); +return false; + }pci_setup_iommu (bus, ops, opaque); bus->iommu_per_bus = true; +return true;Do we need the same check in
pci_setup_iommu()?
I further looked at it. pci_setup_iommu registers a machine wide iommu. Only ARM uses pci_setup_iommu_per_bus for now and usage of both pci_setup_iommu and pci_setup_iommu_per_bus are exclusive and check is done at machine level.
So I think it is reasonable for now to keep that change limited to pci_setup_iommu_per_bus.
Needs a rebase though, including Philippe's suggestion.
Thanks
Eric
Quoted 23 lines
For machine wide IOMMU the check is handled in machine code (at least in arm virt).
Nevertheless a more generic solution could be to check in
pci_setup_iommu()thatbus->iommu_opsis not already set. If set, return an error, cascaded up throughpci_setup_iommu()andpci_setup_iommu_per_bus().
However the change is much more invasive and impacts several archs. What do you think?I don't think this can be a valid use case to call
pci_setup_iommu()several times on the same bus?Quoted 4 lines
Apart from this question,
Reviewed-by Nicolin Chen <[email protected]>Thanks
Eric
Quoted 1 line
Quoted 24 lines · Eric Auger
Currently it is possible to attach several arm-smmuv3 devices to the same bus although it is a wrong setup.
Change the prototype of
pci_setup_iommu_per_busto pass an error handle. This latter is set wheniommu_per_busis already set and used by the single caller (smmu_base_realize) to report a useful error to the end-user.While at it document
pci_setup_iommu_per_buscallback in the header.Fixes 66d2f665e163 ("hw/arm/virt: Allow user-creatable SMMUv3 dev instantiation")Signed-off-by Eric Auger <[email protected]>--- include/hw/pci/pci.h | 16 +++++++++++++++- hw/arm/smmu-common.c | 2 +- hw/pci/pci.c | 9 +++++++-- 3 files changed, 23 insertions(+), 4 deletions(-)hw/arm/smmu-common.c +0
-0
hw/arm/smmu-common.c static void smmu_base_realize(DeviceState *dev, Error **errp) · line 984
Quoted 5 lines
hw/arm/smmu-common.c +1
-1@@ -981,7 +981,7 @@ static void smmu_base_realize(DeviceState *dev, Error **errp) }if (s->smmu_per_bus) { -pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s); +pci_setup_iommu_per_bus (pci_bus, s->iommu_ops, s, errp);
Should we return if the caller returned false? More code could be added at the end of this function.
Quoted 3 lines
hw/arm/smmu-common.c +0
-0}else {pci_setup_iommu (pci_bus, s->iommu_ops, s); }