Skip to contentSkip to conversation

SMMU handling for PCIe Passthrough on ARM

PATCH Superseded Mykyta Poturai 2 participants 7 messages Jul 16, 2025 – Jul 18, 2025

v13 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#0 lore raw

v13 cover SMMU handling for PCIe Passthrough on ARM

This series introduces SMMU handling for PCIe passthrough on ARM. These patches should be able to be upstreamed independently from the vPCI series [1]. See [2] for notes about test cases.

[1] https://lists.xenproject.org/archives/html/xen-devel/2023-10/msg00660.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2023-06/msg01135.html

v12->v13:
* removed merged patches
* move addition of pci-passthrough enabled check in reset physdev op to a separate patch

v11->v12:
* see individual patches

v10->v11:
* see individual patches

v9->v10:
* drop iommu/arm: Add iommu_dt_xlate()
* see individual patches

v8->v9:
* see individual patches

v7->v8:
* no changes

v6->v7:
* drop ("xen/arm: don't pass iommu properties to hwdom for iommu-map")

v5->v6:
* don't revert ("xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"")
* add ("xen/arm: enable dom0 to use PCI devices with pci-passthrough=no")

v4->v5:
* drop ("xen/arm: Improve readability of check for registered devices")
* drop ("xen/arm: Move is_protected flag to struct device")
* add ("xen/arm: don't pass iommu properties to hwdom for iommu-map")
* add ("xen/arm: Fix mapping for PCI bridge mmio region")
* revert ("xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"")
* add ("xen/arm: Map ITS doorbell register to IOMMU page tables.")
* fix test case #1 with PCI device in dom0

v3->v4:
* split a change from ("xen/arm: Move is_protected flag to struct device") into a new separate patch
* see individual patches for further details

v2->v3:
* drop "pci/arm: Use iommu_add_dt_pci_device()"
* drop "RFC: pci/arm: don't do iommu call for phantom functions"
* move invocation of sideband ID mapping function to add_device() platform_ops/iommu_ops hook

Mykyta Poturai (1):
arm/pci: add pci_passhtrough_enabled check to pci_device_reset

Stewart Hildebrand (1):
xen/arm: enable dom0 to use PCI devices with pci-passthrough=no

 xen/arch/arm/include/asm/pci.h |  2 ++
 xen/arch/arm/pci/pci.c         | 14 +++++++++++++-
 xen/arch/x86/include/asm/pci.h |  6 ++++++
 xen/drivers/pci/physdev.c      |  7 +++++--
 xen/include/xen/pci.h          |  5 +++++
 5 files changed, 31 insertions(+), 3 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v13 1/2 arm/pci: add pci_passhtrough_enabled check to pci_device_reset

Without pci-passthrough=on Xen does not know anything about present PCI devices due to PHYSDEVOP_pci_device_add not executing. This causes PHYSDEVOP_pci_device_reset to fail when trying to find the device that is being reset and return ENODEV.

Add explicit is_pci_passthrough_enabled check to return a more meaningful EOPNOTSUPP code.

---
 xen/drivers/pci/physdev.c | 3 +++
 1 file changed, 3 insertions(+)
xen/drivers/pci/physdev.c +3 -0
index 0161a85e1e..78de67ec64 100644
@@ -74,6 +74,9 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         struct pci_dev *pdev;
         pci_sbdf_t sbdf;
 
+        if ( !is_pci_passthrough_enabled() )
+            return -EOPNOTSUPP;
+
         ret = -EFAULT;
         if ( copy_from_guest(&dev_reset, arg, 1) != 0 )
             break;
Signature · 1 line
2.34.1
#5 lore raw

PATCH v13 2/2 xen/arm: enable dom0 to use PCI devices with pci-passthrough=no

In reply to Mykyta Poturai

From: Stewart Hildebrand <[email protected]>

Enable the use of IOMMU + PCI in dom0 without having to specify "pci-passthrough=yes". Due to possible platform specific dependencies of the PCI host, we rely on dom0 to initialize it and perform a PCI PHYSDEVOP calls to add each device to SMMU.

Because pci_passthrough is not always enabled on all architectures, add a new function arch_pci_device_physdevop that checks if we need to enable a subset of the PCI subsystem related to managing IOMMU configuration for PCI devices.

Enable pci_init() for initializing Xen's internal PCI subsystem, and allow PCI PHYSDEV ops when pci-passthrough is disabled.

---
This is the last patch from SMMU-PCIe on ARM series
https://patchew.org/Xen/[email protected]/
hmm. Since
dec9e02f3190 ("xen: avoid generation of stub <asm/pci.h> header")
Should we also move is_pci_passthrough_enabled() back to xen/arch/arm/include/asm/pci.h ? Not sure if PPC/RISC-V will plan on using this check.

v12->v13:
* move introduction of reset op check to a separate patch
* update the commit message

v11->v12:
* add enabled checks to pci_device_reset
* fix style issues

v10->v11:
* always_inline -> inline
* add comments
* clarify reset sub-op handling in the commit message

v9->v10:
* move iommu_enabled check in a separate arch function
* add Stefano's RB

v8->v9:
* move iommu_enabled check inside is_pci_passthrough_enabled()

v7->v8:
* bring back x86 definition of is_pci_passthrough_enabled()

v6->v7:
* remove x86 definition of is_pci_passthrough_enabled()
* update comments
* make pci_physdev_op checks stricter

v5->v6:
* new patch - this effectively replaces
("Revert "xen/arm: Add cmdline boot option "pci-passthrough = <boolean>""")

---
 xen/arch/arm/include/asm/pci.h |  2 ++
 xen/arch/arm/pci/pci.c         | 14 +++++++++++++-
 xen/arch/x86/include/asm/pci.h |  6 ++++++
 xen/drivers/pci/physdev.c      |  6 +++---
 xen/include/xen/pci.h          |  5 +++++
 5 files changed, 29 insertions(+), 4 deletions(-)
xen/arch/arm/include/asm/pci.h +2 -0
index 37a6f14dd4..08ffcd4438 100644
@@ -151,6 +151,8 @@ void pci_generic_init_bus_range_child(struct dt_device_node *dev,
                                       struct pci_host_bridge *bridge,
                                       struct pci_config_window *cfg);
 
+bool arch_pci_device_physdevop(void);
+
 #else   /*!CONFIG_HAS_PCI*/
 
 struct pci_dev;
xen/arch/arm/pci/pci.c +13 -1
index 8d9692c92e..beb1f971fa 100644
@@ -16,6 +16,7 @@
 #include <xen/device_tree.h>
 #include <xen/errno.h>
 #include <xen/init.h>
+#include <xen/iommu.h>
 #include <xen/param.h>
 #include <xen/pci.h>
 
@@ -75,6 +76,17 @@ static int __init acpi_pci_init(void)
 }
 #endif
 
+/*
+ * Platform-specific PCI host dependencies require dom0 to handle
+ * initialization and issue PHYSDEVOP_pci_device_add/remove calls for SMMU
+ * device registration. This check is used to enable the minimal PCI
+ * subsystem required for dom0 operation when PCI passthrough is disabled.
+ */
+bool arch_pci_device_physdevop(void)
+{
+    return iommu_enabled;
+}
+
 /* By default pci passthrough is disabled. */
 bool __read_mostly pci_passthrough_enabled;
 boolean_param("pci-passthrough", pci_passthrough_enabled);
@@ -85,7 +97,7 @@ static int __init pci_init(void)
      * Enable PCI passthrough when has been enabled explicitly
      * (pci-passthrough=on).
      */
-    if ( !pci_passthrough_enabled )
+    if ( !is_pci_passthrough_enabled() && !arch_pci_device_physdevop() )
         return 0;
 
     if ( pci_add_segment(0) )
xen/arch/x86/include/asm/pci.h +6 -0
index 2e67cba8b9..3830232246 100644
@@ -64,4 +64,10 @@ bool pci_check_bar(const struct pci_dev *pdev, mfn_t start, mfn_t end);
 struct rangeset;
 int pci_sanitize_bar_memory(struct rangeset *r);
 
+/* PCI passthrough is always enabled on x86 so no special handling is needed */
+static inline bool arch_pci_device_physdevop(void)
+{
+    return false;
+}
+
 #endif /* __X86_PCI_H__ */
xen/drivers/pci/physdev.c +3 -3
index 78de67ec64..d46501b884 100644
@@ -19,7 +19,7 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         struct pci_dev_info pdev_info;
         nodeid_t node = NUMA_NO_NODE;
 
-        if ( !is_pci_passthrough_enabled() )
+        if ( !is_pci_passthrough_enabled() && !arch_pci_device_physdevop() )
             return -EOPNOTSUPP;
 
         ret = -EFAULT;
@@ -57,7 +57,7 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
     case PHYSDEVOP_pci_device_remove: {
         struct physdev_pci_device dev;
 
-        if ( !is_pci_passthrough_enabled() )
+        if ( !is_pci_passthrough_enabled() && !arch_pci_device_physdevop() )
             return -EOPNOTSUPP;
 
         ret = -EFAULT;
@@ -74,7 +74,7 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         struct pci_dev *pdev;
         pci_sbdf_t sbdf;
 
-        if ( !is_pci_passthrough_enabled() )
+        if ( !is_pci_passthrough_enabled() && !arch_pci_device_physdevop() )
             return -EOPNOTSUPP;
 
         ret = -EFAULT;
xen/include/xen/pci.h +5 -0
index ef60196653..130c2a8c1a 100644
@@ -79,6 +79,11 @@ static inline bool is_pci_passthrough_enabled(void)
     return false;
 }
 
+static inline bool arch_pci_device_physdevop(void)
+{
+    return false;
+}
+
 #endif
 
 struct pci_dev_info {
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v13 1/2] arm/pci: add pci_passhtrough_enabled check to pci_device_reset

In reply to Mykyta Poturai
Quoted 2 lines · Mykyta Poturai

Without pci-passthrough=on Xen does not know anything about present PCI devices due to PHYSDEVOP_pci_device_add not executing.

While the latter half of the sentence is true, Xen may know of PCI devices be other means. It scans the bus itself after all on x86, under certain conditions. Furthermore "pci-passthrough" is an Arm-only option, when the change here affects x86 as much. The description therefore will need some improvement; the code change itself is okay (on the assumption that it can be justified properly).

Jan

#6 lore raw

Re: [PATCH v13 2/2] xen/arm: enable dom0 to use PCI devices with pci-passthrough=no

In reply to Mykyta Poturai

xen/arch/x86/include/asm/pci.h bool pci_check_bar(const struct pci_dev *pdev, mfn_t start, mfn_t end); · line 71

Quoted 11 lines · Mykyta Poturai
xen/arch/x86/include/asm/pci.h +5 -0
@@ -64,4 +64,10 @@ bool pci_check_bar(const struct pci_dev *pdev, mfn_t start, mfn_t end);
 struct rangeset;
 int pci_sanitize_bar_memory(struct rangeset *r);
 
+/* PCI passthrough is always enabled on x86 so no special handling is needed */
+static inline bool arch_pci_device_physdevop(void)
+{
+    return false;
+}

The comment is somewhat odd, as it talks about pass-through when the function is all about physdevop. A connection wants making imo. Plus isn't it benign right now whether the function returned false or true? From an abstract perspective, it returning true would perhaps make more sense (as opposed to the generic stub in include/xen/pci.h). IOW I think the return value wants changing _and_ the comment wants to clarify that either value could be used with the present call sites (thus helping people finding themselves in need of altering the return value, in order to use the function elsewhere).

I also think this would better go directly below is_pci_passthrough_enabled().

Jan

#3 lore raw

Re: [PATCH v13 1/2] arm/pci: add pci_passhtrough_enabled check to pci_device_reset

In reply to Jan Beulich
Quoted 4 lines · Jan Beulich
Quoted 2 lines · Mykyta Poturai

Without pci-passthrough=on Xen does not know anything about present PCI devices due to PHYSDEVOP_pci_device_add not executing.

Quoted 8 lines

While the latter half of the sentence is true, Xen may know of PCI devices be other means. It scans the bus itself after all on x86, under certain conditions. Furthermore "pci-passthrough" is an Arm-only option, when the change here affects x86 as much. The description therefore will need some improvement; the code change itself is okay (on the assumption that it can be justified properly).

Jan

Will adding an additional note that all of this only relates to Arm and will not change x86 functionality be sufficient? As far as I understand, this sentence is completely true for Arm, even the yet-to-be-upstreamed bus scanning routines in Xen are guarded by is_pci_passthroug_enabled in the latest patches I've seen them in.

Signature · 1 line
Mykyta
#4 lore raw

Re: [PATCH v13 1/2] arm/pci: add pci_passhtrough_enabled check to pci_device_reset

Quoted 12 lines · Mykyta Poturai
Quoted 10 lines · Jan Beulich
Quoted 2 lines · Mykyta Poturai

Without pci-passthrough=on Xen does not know anything about present PCI devices due to PHYSDEVOP_pci_device_add not executing.

While the latter half of the sentence is true, Xen may know of PCI devices be other means. It scans the bus itself after all on x86, under certain conditions. Furthermore "pci-passthrough" is an Arm-only option, when the change here affects x86 as much. The description therefore will need some improvement; the code change itself is okay (on the assumption that it can be justified properly).

Quoted 2 lines

Will adding an additional note that all of this only relates to Arm and will not change x86 functionality be sufficient?

Probably.

Jan

Quoted 5 lines

As far as I understand,
this sentence is completely true for Arm, even the yet-to-be-upstreamed bus scanning routines in Xen are guarded by is_pci_passthroug_enabled in the latest patches I've seen them in.