Skip to contentSkip to conversation

hw/arm/smmuv3-accel: Resolve AUTO properties

PATCH Superseded Nathan Chen 4 participants 41 messages Apr 1 – Apr 21

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

cover hw/arm/smmuv3-accel: Resolve AUTO properties

Hi,

This series introduces support for resolving 'auto' for arm-smmuv3 accelerated mode's ATS, RIL, SSIDSIZE, and OAS feature properties based on host IOMMU capabilities. This is dependent on the series [1] for changing these property types to accept 'auto' values.

Accelerated SMMUv3 Address Translation Services support is derived from IDR0, Range Invalidation support is derived from IDR3, Substream ID size is derived from IDR1, and output address space is derived from IDR5.

Additionally, an OnOffAuto "ats" property is added for vfio-pci devices, where setting 'auto' detects the per-device presence of IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED from the kernel, and the ATS cap can be advertised or hidden by setting 'on' or 'off'. This is dependent on Shameer's recent kernel series for reporting effective ATS support status [2].

The default values are set to 'auto' for all properties.

A complete branch can be found here:
https://github.com/NathanChenNVIDIA/qemu/tree/smmuv3-accel-auto-resolve

Please take a look and let me know your feedback.

Thanks,
Nathan

[0] https://lore.kernel.org/qemu-devel/[email protected]/
[1] https://lore.kernel.org/qemu-arm/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Example usage:
qemu-system-aarch64 \
-object iommufd,id=iommufd0 \
-machine virt,accel=kvm,gic-version=3,ras=on,highmem-mmio-size=4T \
-cpu host -smp cpus=4 -m size=16G -nographic \
-object memory-backend-ram,size=16G,id=m0 \
-numa node,memdev=m0,cpus=0-3,nodeid=0 \
-numa node,nodeid=1 -numa node,nodeid=2 -numa node,nodeid=3 -numa node,nodeid=4 \
-numa node,nodeid=5 -numa node,nodeid=6 -numa node,nodeid=7 -numa node,nodeid=8 \
-device pxb-pcie,id=pcie.1,bus_nr=1,bus=pcie.0,numa_node=0 \
-device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1,accel=on,ats=auto,ssidsize=auto,ril=auto,oas=auto \
-device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0 \
-device vfio-pci-nohotplug,host=0009:06:00.0,bus=pcie.port1,rombar=0,id=dev0,iommufd=iommufd0,ats=auto \
-object acpi-generic-initiator,id=gi0,pci-dev=dev0,node=1 \
-object acpi-generic-initiator,id=gi1,pci-dev=dev0,node=2 \
-object acpi-generic-initiator,id=gi2,pci-dev=dev0,node=3 \
-object acpi-generic-initiator,id=gi3,pci-dev=dev0,node=4 \
-object acpi-generic-initiator,id=gi4,pci-dev=dev0,node=5 \
-object acpi-generic-initiator,id=gi5,pci-dev=dev0,node=6 \
-object acpi-generic-initiator,id=gi6,pci-dev=dev0,node=7 \
-object acpi-generic-initiator,id=gi7,pci-dev=dev0,node=8 \
-bios /usr/share/AAVMF/AAVMF_CODE.fd \
-device nvme,drive=nvme0,serial=deadbeaf1,bus=pcie.0 \
-drive file=/var/lib/libvirt/images/guest.qcow2,index=0,media=disk,format=qcow2,if=none,id=nvme0 \
-device e1000,romfile=/usr/local/share/qemu/efi-e1000.rom,netdev=net0,bus=pcie.0 \
-netdev user,id=net0,hostfwd=tcp::5558-:22,hostfwd=tcp::5586-:5586

Testing:
Basic sanity testing was performed on an NVIDIA Grace platform with GPU device assignment and running CUDA test apps on the guest. Observed the feature properties being set based on host IOMMU capabilities and the ATS capability for a vfio-pci device reported based on what was reported from the host. Verified that the VM boot will fail without a cold-plugged device, and that a hot-plugged device re-uses the resolved values from the initial cold-plug. Additional testing and feedback are welcome.

Nathan Chen (11):
hw/arm/smmuv3-accel: Add helper for resolving auto parameters hw/arm/smmuv3-accel: Implement "auto" value for "ats" hw/arm/smmuv3: Change the default ats support to match the host vfio/pci: Add ats property and mask ATS cap when not exposed hw/arm/smmuv3-accel: Implement "auto" value for "ril" hw/arm/smmuv3: Change the default ril support to match the host hw/arm/smmuv3-accel: Implement "auto" value for "ssidsize" hw/arm/smmuv3: Change the default ssidsize to match the host hw/arm/smmuv3-accel: Implement "auto" value for "oas" hw/arm/smmuv3: Change the default oas to match the host qemu-options.hx: Support "auto" for accel SMMUv3 properties

 backends/iommufd.c                 | 15 +++++++
 hw/arm/smmuv3-accel.c              | 50 +++++++++++++++++++++++-
 hw/arm/smmuv3-accel.h              |  2 +
 hw/arm/smmuv3.c                    | 61 +++++++++++++++++------------
 hw/core/machine.c                  |  8 ++++
 hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++++++
 hw/vfio/pci.h                      |  1 +
 include/hw/arm/smmuv3.h            |  2 +
 include/system/host_iommu_device.h | 10 +++++
 qemu-options.hx                    |  1 -
 10 files changed, 185 insertions(+), 28 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/11 hw/arm/smmuv3-accel: Add helper for resolving auto parameters

From: Nathan Chen <[email protected]>

Introduce smmuv3_accel_auto_finalise() to resolve properties that are set to 'auto' for accelerated SMMUv3. This helper function allows properties such as ats, ril, ssidsize, and oas support to be resolved from host IOMMU capabilities via IOMMU_GET_HW_INFO.

Auto mode requires at least one cold-plugged device to retrieve and finalise these properties. Register a machine_init_done notifier to verify this requirement and fail boot if it is not met.

Hot-plugged devices into an accel SMMUv3-associated bus will re-use the resolved host values from the initial cold-plug.

Subsequent patches will make use of this helper to resolve 'auto' to what is reported by host IOMMU capabilities.

---
 hw/arm/smmuv3-accel.c   | 14 ++++++++++++++
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/smmuv3.c         | 20 ++++++++++++++++++++
 include/hw/arm/smmuv3.h |  2 ++
 4 files changed, 38 insertions(+)
hw/arm/smmuv3-accel.c +14 -0
index 65c2f44880..a0146c8d31 100644
@@ -35,11 +35,25 @@ static int smmuv3_oas_bits(uint32_t oas)
     return map[oas];
 }
 
+static void smmuv3_accel_auto_finalise(SMMUv3State *s,
+                                       struct iommu_hw_info_arm_smmuv3 *info) {
+    SMMUv3AccelState *accel = s->s_accel;
+
+    /* Return if no auto for any or finalised already */
+    if (!accel->auto_mode || accel->auto_finalised) {
+        return;
+    }
+
+    accel->auto_finalised = true;
+}
+
 static bool
 smmuv3_accel_check_hw_compatible(SMMUv3State *s,
                                  struct iommu_hw_info_arm_smmuv3 *info,
                                  Error **errp)
 {
+    smmuv3_accel_auto_finalise(s, info);
+
     /* QEMU SMMUv3 supports both linear and 2-level stream tables */
     if (FIELD_EX32(info->idr[0], IDR0, STLEVEL) !=
                 FIELD_EX32(s->idr[0], IDR0, STLEVEL)) {
hw/arm/smmuv3-accel.h +2 -0
index dba6c71de5..3c1cd55714 100644
@@ -26,6 +26,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    bool auto_mode;
+    bool auto_finalised;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
hw/arm/smmuv3.c +20 -0
index 7fead1c3cf..09ea08eb18 100644
@@ -37,6 +37,7 @@
 #include "smmuv3-accel.h"
 #include "smmuv3-internal.h"
 #include "smmu-internal.h"
+#include "system/system.h"
 
 #define PTW_RECORD_FAULT(ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \
                                         (cfg)->record_faults) || \
@@ -2020,6 +2021,22 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     return true;
 }
 
+static void smmuv3_machine_done(Notifier *notifier, void *data)
+{
+    SMMUv3State *s = container_of(notifier, SMMUv3State, machine_done);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!s->accel) {
+        return;
+    }
+
+    if (accel->auto_mode && !accel->auto_finalised) {
+        error_report("arm-smmuv3 accel=on with 'auto' properties requires "
+                     "at least one cold-plugged VFIO device");
+        exit(1);
+    }
+}
+
 static void smmu_realize(DeviceState *d, Error **errp)
 {
     SMMUState *sys = ARM_SMMU(d);
@@ -2058,6 +2075,9 @@ static void smmu_realize(DeviceState *d, Error **errp)
 
     smmu_init_irq(s, dev);
     smmuv3_init_id_regs(s);
+
+    s->machine_done.notify = smmuv3_machine_done;
+    qemu_add_machine_init_done_notifier(&s->machine_done);
 }
 
 static const VMStateDescription vmstate_smmuv3_queue = {
include/hw/arm/smmuv3.h +2 -0
index 82f18eb090..fe0493c1aa 100644
@@ -74,6 +74,8 @@ struct SMMUv3State {
     OnOffAuto ats;
     OasMode oas;
     SsidSizeMode ssidsize;
+
+    Notifier machine_done;
 };
 
 typedef enum {
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH 01/11] hw/arm/smmuv3-accel: Add helper for resolving auto parameters

Hi Nathan,

Quoted 31 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Introduce smmuv3_accel_auto_finalise() to resolve properties that are set to 'auto' for accelerated SMMUv3. This helper function allows properties such as ats, ril, ssidsize, and oas support to be resolved from host IOMMU capabilities via IOMMU_GET_HW_INFO.

Auto mode requires at least one cold-plugged device to retrieve and finalise these properties. Register a machine_init_done notifier to verify this requirement and fail boot if it is not met.

Hot-plugged devices into an accel SMMUv3-associated bus will re-use the resolved host values from the initial cold-plug.

Subsequent patches will make use of this helper to resolve 'auto' to what is reported by host IOMMU capabilities.

---
 hw/arm/smmuv3-accel.c   | 14 ++++++++++++++
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/smmuv3.c         | 20 ++++++++++++++++++++
 include/hw/arm/smmuv3.h |  2 ++
 4 files changed, 38 insertions(+)
hw/arm/smmuv3-accel.c +0 -0
index 65c2f44880..a0146c8d31 100644

hw/arm/smmuv3-accel.c static int smmuv3_oas_bits(uint32_t oas) · line 43

Quoted 9 lines
hw/arm/smmuv3-accel.c +6 -0
@@ -35,11 +35,25 @@ static int smmuv3_oas_bits(uint32_t oas)
     return map[oas];
 }
 
+static void smmuv3_accel_auto_finalise(SMMUv3State *s,
+                                       struct iommu_hw_info_arm_smmuv3 *info) {
+    SMMUv3AccelState *accel = s->s_accel;
+
+    /* Return if no auto for any or finalised already */
+    if (!accel->auto_mode || accel->auto_finalised) {

I think you need to explain the commit message when auto_mode is going to be set (I guess at least once an accel  prop is set to auto mode).

Also can you detail this case where smmuv3_accel_auto_finalise() would be called several times. Here we return directly without looking again at the prop values. Would it be meaningful to continue and check the collected info matches the ones already collected? Can it happen?

An alternative would have been to fall back to default values in case no cold plugged host dev could be found. Can you motivate that choice in the commit msg?

Quoted 65 lines
hw/arm/smmuv3-accel.c +8 -0
+        return;
+    }
+
+    accel->auto_finalised = true;
+}
+
 static bool
 smmuv3_accel_check_hw_compatible(SMMUv3State *s,
                                  struct iommu_hw_info_arm_smmuv3 *info,
                                  Error **errp)
 {
+    smmuv3_accel_auto_finalise(s, info);
+
     /* QEMU SMMUv3 supports both linear and 2-level stream tables */
     if (FIELD_EX32(info->idr[0], IDR0, STLEVEL) !=
                 FIELD_EX32(s->idr[0], IDR0, STLEVEL)) {
hw/arm/smmuv3-accel.h +2 -0
index dba6c71de5..3c1cd55714 100644
@@ -26,6 +26,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    bool auto_mode;
+    bool auto_finalised;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
hw/arm/smmuv3.c +17 -0
index 7fead1c3cf..09ea08eb18 100644
@@ -37,6 +37,7 @@
 #include "smmuv3-accel.h"
 #include "smmuv3-internal.h"
 #include "smmu-internal.h"
+#include "system/system.h"
 
 #define PTW_RECORD_FAULT(ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \
                                         (cfg)->record_faults) || \
@@ -2020,6 +2021,22 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     return true;
 }
 
+static void smmuv3_machine_done(Notifier *notifier, void *data)
+{
+    SMMUv3State *s = container_of(notifier, SMMUv3State, machine_done);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!s->accel) {
+        return;
+    }
+
+    if (accel->auto_mode && !accel->auto_finalised) {
+        error_report("arm-smmuv3 accel=on with 'auto' properties requires "
+                     "at least one cold-plugged VFIO device");
+        exit(1);
+    }
+}
+
 static void smmu_realize(DeviceState *d, Error **errp)
 {
     SMMUState *sys = ARM_SMMU(d);

hw/arm/smmuv3.c static void smmu_realize(DeviceState *d, Error **errp) · line 2080

Quoted 6 lines
hw/arm/smmuv3.c +3 -0
@@ -2058,6 +2075,9 @@ static void smmu_realize(DeviceState *d, Error **errp)
 
     smmu_init_irq(s, dev);
     smmuv3_init_id_regs(s);
+
+    s->machine_done.notify = smmuv3_machine_done;
+    qemu_add_machine_init_done_notifier(&s->machine_done);

can't you move smmuv3_machine_done in smmuv3-accel.c and also call

qemu_add_machine_init_done_notifier from smmuv3_accel_init()?

Eric

Quoted 16 lines
hw/arm/smmuv3.c +0 -0
 }
 
 static const VMStateDescription vmstate_smmuv3_queue = {
include/hw/arm/smmuv3.h +2 -0
index 82f18eb090..fe0493c1aa 100644
@@ -74,6 +74,8 @@ struct SMMUv3State {
     OnOffAuto ats;
     OasMode oas;
     SsidSizeMode ssidsize;
+
+    Notifier machine_done;
 };
 
 typedef enum {
#3 lore raw

Re: [PATCH 01/11] hw/arm/smmuv3-accel: Add helper for resolving auto parameters

Hi Eric,

Quoted 44 lines · Eric Auger
Quoted 31 lines · Nathan Chen

From: Nathan Chen<[email protected]>

Introduce smmuv3_accel_auto_finalise() to resolve properties that are set to 'auto' for accelerated SMMUv3. This helper function allows properties such as ats, ril, ssidsize, and oas support to be resolved from host IOMMU capabilities via IOMMU_GET_HW_INFO.

Auto mode requires at least one cold-plugged device to retrieve and finalise these properties. Register a machine_init_done notifier to verify this requirement and fail boot if it is not met.

Hot-plugged devices into an accel SMMUv3-associated bus will re-use the resolved host values from the initial cold-plug.

Subsequent patches will make use of this helper to resolve 'auto' to what is reported by host IOMMU capabilities.

---
  hw/arm/smmuv3-accel.c   | 14 ++++++++++++++
  hw/arm/smmuv3-accel.h   |  2 ++
  hw/arm/smmuv3.c         | 20 ++++++++++++++++++++
  include/hw/arm/smmuv3.h |  2 ++
  4 files changed, 38 insertions(+)
hw/arm/smmuv3-accel.c +0 -0
index 65c2f44880..a0146c8d31 100644

hw/arm/smmuv3-accel.c static int smmuv3_oas_bits(uint32_t oas) · line 43

Quoted 9 lines
hw/arm/smmuv3-accel.c +6 -0
@@ -35,11 +35,25 @@ static int smmuv3_oas_bits(uint32_t oas)
      return map[oas];
  }
 
+static void smmuv3_accel_auto_finalise(SMMUv3State *s,
+                                       struct iommu_hw_info_arm_smmuv3 *info) {
+    SMMUv3AccelState *accel = s->s_accel;
+
+    /* Return if no auto for any or finalised already */
+    if (!accel->auto_mode || accel->auto_finalised) {

I think you need to explain the commit message when auto_mode is going to be set (I guess at least once an accel  prop is set to auto mode).

Quoted 8 lines

Also can you detail this case where smmuv3_accel_auto_finalise() would be called several times. Here we return directly without looking again at the prop values. Would it be meaningful to continue and check the collected info matches the ones already collected? Can it happen?

An alternative would have been to fall back to default values in case no cold plugged host dev could be found. Can you motivate that choice in the commit msg?

Ok, I will explain in the commit description auto_mode is set when an accel prop is explicitly set to auto mode, or if the accel prop is not set and defaults to auto mode per the later commits in this series.

smmuv3_accel_auto_finalise() would be called more than once in the hot plug case. We do not accept new property values in that case and re-use the values determined from initial cold plug.

arm-smmuv3 accel=on with 'auto' properties requires at least one cold-plugged VFIO device, otherwise initial VM boot will fail and it would not be possible to call smmuv3_accel_auto_finalise() more than once in that case.

Quoted 66 lines
Quoted 66 lines
hw/arm/smmuv3-accel.c +8 -0
+        return;
+    }
+
+    accel->auto_finalised = true;
+}
+
  static bool
  smmuv3_accel_check_hw_compatible(SMMUv3State *s,
                                   struct iommu_hw_info_arm_smmuv3 *info,
                                   Error **errp)
  {
+    smmuv3_accel_auto_finalise(s, info);
+
      /* QEMU SMMUv3 supports both linear and 2-level stream tables */
      if (FIELD_EX32(info->idr[0], IDR0, STLEVEL) !=
                  FIELD_EX32(s->idr[0], IDR0, STLEVEL)) {
hw/arm/smmuv3-accel.h +2 -0
index dba6c71de5..3c1cd55714 100644
@@ -26,6 +26,8 @@ typedef struct SMMUv3AccelState {
      uint32_t bypass_hwpt_id;
      uint32_t abort_hwpt_id;
      QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    bool auto_mode;
+    bool auto_finalised;
  } SMMUv3AccelState;
 
  typedef struct SMMUS1Hwpt {
hw/arm/smmuv3.c +17 -0
index 7fead1c3cf..09ea08eb18 100644
@@ -37,6 +37,7 @@
  #include "smmuv3-accel.h"
  #include "smmuv3-internal.h"
  #include "smmu-internal.h"
+#include "system/system.h"
 
  #define PTW_RECORD_FAULT(ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \
                                          (cfg)->record_faults) || \
@@ -2020,6 +2021,22 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
      return true;
  }
 
+static void smmuv3_machine_done(Notifier *notifier, void *data)
+{
+    SMMUv3State *s = container_of(notifier, SMMUv3State, machine_done);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!s->accel) {
+        return;
+    }
+
+    if (accel->auto_mode && !accel->auto_finalised) {
+        error_report("arm-smmuv3 accel=on with 'auto' properties requires "
+                     "at least one cold-plugged VFIO device");
+        exit(1);
+    }
+}
+
  static void smmu_realize(DeviceState *d, Error **errp)
  {
      SMMUState *sys = ARM_SMMU(d);
@@ -2058,6 +2075,9 @@ static void smmu_realize(DeviceState *d, Error **errp)
 
Quoted 8 lines

hw/arm/smmuv3.c static void smmu_realize(DeviceState *d, Error **errp) · line 2080

Quoted 5 lines
hw/arm/smmuv3.c +3 -0
      smmu_init_irq(s, dev);
      smmuv3_init_id_regs(s);
+
+    s->machine_done.notify = smmuv3_machine_done;
+    qemu_add_machine_init_done_notifier(&s->machine_done);

can't you move smmuv3_machine_done in smmuv3-accel.c and also call

qemu_add_machine_init_done_notifier from smmuv3_accel_init()?

That makes sense, we can move the machine_done Notifier to SMMUv3AccelState since it's relevant to accel cases. I'll look into updating this.

Thanks,
Nathan

#4 lore raw

Re: [PATCH 01/11] hw/arm/smmuv3-accel: Add helper for resolving auto parameters

In reply to Nathan Chen
Quoted 105 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Introduce smmuv3_accel_auto_finalise() to resolve properties that are set to 'auto' for accelerated SMMUv3. This helper function allows properties such as ats, ril, ssidsize, and oas support to be resolved from host IOMMU capabilities via IOMMU_GET_HW_INFO.

Auto mode requires at least one cold-plugged device to retrieve and finalise these properties. Register a machine_init_done notifier to verify this requirement and fail boot if it is not met.

Hot-plugged devices into an accel SMMUv3-associated bus will re-use the resolved host values from the initial cold-plug.

Subsequent patches will make use of this helper to resolve 'auto' to what is reported by host IOMMU capabilities.

---
 hw/arm/smmuv3-accel.c   | 14 ++++++++++++++
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/smmuv3.c         | 20 ++++++++++++++++++++
 include/hw/arm/smmuv3.h |  2 ++
 4 files changed, 38 insertions(+)
hw/arm/smmuv3-accel.c +14 -0
index 65c2f44880..a0146c8d31 100644
@@ -35,11 +35,25 @@ static int smmuv3_oas_bits(uint32_t oas)
     return map[oas];
 }
 
+static void smmuv3_accel_auto_finalise(SMMUv3State *s,
+                                       struct iommu_hw_info_arm_smmuv3 *info) {
+    SMMUv3AccelState *accel = s->s_accel;
+
+    /* Return if no auto for any or finalised already */
+    if (!accel->auto_mode || accel->auto_finalised) {
+        return;
+    }
+
+    accel->auto_finalised = true;
+}
+
 static bool
 smmuv3_accel_check_hw_compatible(SMMUv3State *s,
                                  struct iommu_hw_info_arm_smmuv3 *info,
                                  Error **errp)
 {
+    smmuv3_accel_auto_finalise(s, info);
+
     /* QEMU SMMUv3 supports both linear and 2-level stream tables */
     if (FIELD_EX32(info->idr[0], IDR0, STLEVEL) !=
                 FIELD_EX32(s->idr[0], IDR0, STLEVEL)) {
hw/arm/smmuv3-accel.h +2 -0
index dba6c71de5..3c1cd55714 100644
@@ -26,6 +26,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    bool auto_mode;
+    bool auto_finalised;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
hw/arm/smmuv3.c +17 -0
index 7fead1c3cf..09ea08eb18 100644
@@ -37,6 +37,7 @@
 #include "smmuv3-accel.h"
 #include "smmuv3-internal.h"
 #include "smmu-internal.h"
+#include "system/system.h"
 
 #define PTW_RECORD_FAULT(ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \
                                         (cfg)->record_faults) || \
@@ -2020,6 +2021,22 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     return true;
 }
 
+static void smmuv3_machine_done(Notifier *notifier, void *data)
+{
+    SMMUv3State *s = container_of(notifier, SMMUv3State, machine_done);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!s->accel) {
+        return;
+    }
+
+    if (accel->auto_mode && !accel->auto_finalised) {
+        error_report("arm-smmuv3 accel=on with 'auto' properties requires "
+                     "at least one cold-plugged VFIO device");
+        exit(1);
+    }
+}
+
 static void smmu_realize(DeviceState *d, Error **errp)
 {
     SMMUState *sys = ARM_SMMU(d);

hw/arm/smmuv3.c static void smmu_realize(DeviceState *d, Error **errp) · line 2080

Quoted 6 lines
hw/arm/smmuv3.c +3 -0
@@ -2058,6 +2075,9 @@ static void smmu_realize(DeviceState *d, Error **errp)
 
     smmu_init_irq(s, dev);
     smmuv3_init_id_regs(s);
+
+    s->machine_done.notify = smmuv3_machine_done;
+    qemu_add_machine_init_done_notifier(&s->machine_done);

you might also register the machine done notifier only in case auto_mode is set

Eric

Quoted 16 lines
hw/arm/smmuv3.c +0 -0
 }
 
 static const VMStateDescription vmstate_smmuv3_queue = {
include/hw/arm/smmuv3.h +2 -0
index 82f18eb090..fe0493c1aa 100644
@@ -74,6 +74,8 @@ struct SMMUv3State {
     OnOffAuto ats;
     OasMode oas;
     SsidSizeMode ssidsize;
+
+    Notifier machine_done;
 };
 
 typedef enum {
#5 lore raw

Re: [PATCH 01/11] hw/arm/smmuv3-accel: Add helper for resolving auto parameters

Quoted 108 lines · Eric Auger
Quoted 106 lines · Nathan Chen

From: Nathan Chen<[email protected]>

Introduce smmuv3_accel_auto_finalise() to resolve properties that are set to 'auto' for accelerated SMMUv3. This helper function allows properties such as ats, ril, ssidsize, and oas support to be resolved from host IOMMU capabilities via IOMMU_GET_HW_INFO.

Auto mode requires at least one cold-plugged device to retrieve and finalise these properties. Register a machine_init_done notifier to verify this requirement and fail boot if it is not met.

Hot-plugged devices into an accel SMMUv3-associated bus will re-use the resolved host values from the initial cold-plug.

Subsequent patches will make use of this helper to resolve 'auto' to what is reported by host IOMMU capabilities.

---
  hw/arm/smmuv3-accel.c   | 14 ++++++++++++++
  hw/arm/smmuv3-accel.h   |  2 ++
  hw/arm/smmuv3.c         | 20 ++++++++++++++++++++
  include/hw/arm/smmuv3.h |  2 ++
  4 files changed, 38 insertions(+)
hw/arm/smmuv3-accel.c +14 -0
index 65c2f44880..a0146c8d31 100644
@@ -35,11 +35,25 @@ static int smmuv3_oas_bits(uint32_t oas)
      return map[oas];
  }
 
+static void smmuv3_accel_auto_finalise(SMMUv3State *s,
+                                       struct iommu_hw_info_arm_smmuv3 *info) {
+    SMMUv3AccelState *accel = s->s_accel;
+
+    /* Return if no auto for any or finalised already */
+    if (!accel->auto_mode || accel->auto_finalised) {
+        return;
+    }
+
+    accel->auto_finalised = true;
+}
+
  static bool
  smmuv3_accel_check_hw_compatible(SMMUv3State *s,
                                   struct iommu_hw_info_arm_smmuv3 *info,
                                   Error **errp)
  {
+    smmuv3_accel_auto_finalise(s, info);
+
      /* QEMU SMMUv3 supports both linear and 2-level stream tables */
      if (FIELD_EX32(info->idr[0], IDR0, STLEVEL) !=
                  FIELD_EX32(s->idr[0], IDR0, STLEVEL)) {
hw/arm/smmuv3-accel.h +2 -0
index dba6c71de5..3c1cd55714 100644
@@ -26,6 +26,8 @@ typedef struct SMMUv3AccelState {
      uint32_t bypass_hwpt_id;
      uint32_t abort_hwpt_id;
      QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    bool auto_mode;
+    bool auto_finalised;
  } SMMUv3AccelState;
 
  typedef struct SMMUS1Hwpt {
hw/arm/smmuv3.c +17 -0
index 7fead1c3cf..09ea08eb18 100644
@@ -37,6 +37,7 @@
  #include "smmuv3-accel.h"
  #include "smmuv3-internal.h"
  #include "smmu-internal.h"
+#include "system/system.h"
 
  #define PTW_RECORD_FAULT(ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \
                                          (cfg)->record_faults) || \
@@ -2020,6 +2021,22 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
      return true;
  }
 
+static void smmuv3_machine_done(Notifier *notifier, void *data)
+{
+    SMMUv3State *s = container_of(notifier, SMMUv3State, machine_done);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!s->accel) {
+        return;
+    }
+
+    if (accel->auto_mode && !accel->auto_finalised) {
+        error_report("arm-smmuv3 accel=on with 'auto' properties requires "
+                     "at least one cold-plugged VFIO device");
+        exit(1);
+    }
+}
+
  static void smmu_realize(DeviceState *d, Error **errp)
  {
      SMMUState *sys = ARM_SMMU(d);
@@ -2058,6 +2075,9 @@ static void smmu_realize(DeviceState *d, Error **errp)
 
Quoted 7 lines

hw/arm/smmuv3.c static void smmu_realize(DeviceState *d, Error **errp) · line 2080

Quoted 5 lines
hw/arm/smmuv3.c +3 -0
      smmu_init_irq(s, dev);
      smmuv3_init_id_regs(s);
+
+    s->machine_done.notify = smmuv3_machine_done;
+    qemu_add_machine_init_done_notifier(&s->machine_done);

you might also register the machine done notifier only in case auto_mode is set

Yes, I'll incorporate this in the next revision.

Thanks,
Nathan

#6 lore raw

PATCH 2/11 hw/arm/smmuv3-accel: Implement "auto" value for "ats"

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 Address Translation Services support property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving ATS capability from IDR0.

---
 hw/arm/smmuv3-accel.c | 10 ++++++++++
 hw/arm/smmuv3.c       |  7 +------
 2 files changed, 11 insertions(+), 6 deletions(-)
hw/arm/smmuv3-accel.c +10 -0
index a0146c8d31..a835c8f220 100644
@@ -44,6 +44,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
         return;
     }
 
+    /* Update ATS if auto from info */
+    if (s->ats == ON_OFF_AUTO_AUTO) {
+        s->idr[0] = FIELD_DP32(s->idr[0], IDR0, ATS,
+                               FIELD_EX32(info->idr[0], IDR0, ATS));
+    }
+
     accel->auto_finalised = true;
 }
 
@@ -938,4 +944,8 @@ void smmuv3_accel_init(SMMUv3State *s)
     s->s_accel = g_new0(SMMUv3AccelState, 1);
     bs->iommu_ops = &smmuv3_accel_ops;
     smmuv3_accel_as_init(s);
+
+    if (s->ats == ON_OFF_AUTO_AUTO) {
+        s->s_accel->auto_mode = true;
+    }
 }
hw/arm/smmuv3.c +1 -6
index 09ea08eb18..f3603554e3 100644
@@ -1973,10 +1973,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->ats == ON_OFF_AUTO_AUTO) {
-        error_setg(errp, "ats auto mode is not supported");
-        return false;
-    }
     if (s->ril == ON_OFF_AUTO_AUTO) {
         error_setg(errp, "ril auto mode is not supported");
         return false;
@@ -2196,8 +2192,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "is not supported.");
     object_class_property_set_description(klass, "ats",
         "Enable/disable ATS support (for accel=on). Please ensure host "
-        "platform has ATS support before enabling this. ats=auto is not "
-        "supported.");
+        "platform has ATS support before enabling this.");
     object_class_property_set_description(klass, "oas",
         "Specify Output Address Size (for accel=on). Supported values "
         "are 44 or 48 bits. Defaults to 44 bits. oas=auto is not "
Signature · 1 line
2.43.0
#7 lore raw

Re: [PATCH 02/11] hw/arm/smmuv3-accel: Implement "auto" value for "ats"

Quoted 54 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 Address Translation Services support property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving ATS capability from IDR0.

---
 hw/arm/smmuv3-accel.c | 10 ++++++++++
 hw/arm/smmuv3.c       |  7 +------
 2 files changed, 11 insertions(+), 6 deletions(-)
hw/arm/smmuv3-accel.c +10 -0
index a0146c8d31..a835c8f220 100644
@@ -44,6 +44,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
         return;
     }
 
+    /* Update ATS if auto from info */
+    if (s->ats == ON_OFF_AUTO_AUTO) {
+        s->idr[0] = FIELD_DP32(s->idr[0], IDR0, ATS,
+                               FIELD_EX32(info->idr[0], IDR0, ATS));
+    }
+
     accel->auto_finalised = true;
 }
 
@@ -938,4 +944,8 @@ void smmuv3_accel_init(SMMUv3State *s)
     s->s_accel = g_new0(SMMUv3AccelState, 1);
     bs->iommu_ops = &smmuv3_accel_ops;
     smmuv3_accel_as_init(s);
+
+    if (s->ats == ON_OFF_AUTO_AUTO) {
+        s->s_accel->auto_mode = true;
+    }
 }
hw/arm/smmuv3.c +0 -4
index 09ea08eb18..f3603554e3 100644
@@ -1973,10 +1973,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->ats == ON_OFF_AUTO_AUTO) {
-        error_setg(errp, "ats auto mode is not supported");
-        return false;
-    }
     if (s->ril == ON_OFF_AUTO_AUTO) {
         error_setg(errp, "ril auto mode is not supported");
         return false;

hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2200

Quoted 5 lines
hw/arm/smmuv3.c +0 -2
@@ -2196,8 +2192,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "is not supported.");
     object_class_property_set_description(klass, "ats",
         "Enable/disable ATS support (for accel=on). Please ensure host "
-        "platform has ATS support before enabling this. ats=auto is not "
-        "supported.");

I would explicitly mention that auto value is supported. Maybe:
Set ATS support. Valid values are off/on/auto. If set to on, make sure the host platform supports ATS.

You have examples in hw/arm/virt.c. For instance msi option.

Also note that the doc in qemu-options.hx must also be updated.

Eric

Quoted 4 lines
hw/arm/smmuv3.c +1 -0
+        "platform has ATS support before enabling this.");
     object_class_property_set_description(klass, "oas",
         "Specify Output Address Size (for accel=on). Supported values "
         "are 44 or 48 bits. Defaults to 44 bits. oas=auto is not "
#8 lore raw

Re: [PATCH 02/11] hw/arm/smmuv3-accel: Implement "auto" value for "ats"

Quoted 66 lines · Eric Auger
Quoted 54 lines · Nathan Chen

From: Nathan Chen<[email protected]>

Allow accelerated SMMUv3 Address Translation Services support property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving ATS capability from IDR0.

---
  hw/arm/smmuv3-accel.c | 10 ++++++++++
  hw/arm/smmuv3.c       |  7 +------
  2 files changed, 11 insertions(+), 6 deletions(-)
hw/arm/smmuv3-accel.c +10 -0
index a0146c8d31..a835c8f220 100644
@@ -44,6 +44,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
          return;
      }
 
+    /* Update ATS if auto from info */
+    if (s->ats == ON_OFF_AUTO_AUTO) {
+        s->idr[0] = FIELD_DP32(s->idr[0], IDR0, ATS,
+                               FIELD_EX32(info->idr[0], IDR0, ATS));
+    }
+
      accel->auto_finalised = true;
  }
 
@@ -938,4 +944,8 @@ void smmuv3_accel_init(SMMUv3State *s)
      s->s_accel = g_new0(SMMUv3AccelState, 1);
      bs->iommu_ops = &smmuv3_accel_ops;
      smmuv3_accel_as_init(s);
+
+    if (s->ats == ON_OFF_AUTO_AUTO) {
+        s->s_accel->auto_mode = true;
+    }
  }
hw/arm/smmuv3.c +0 -4
index 09ea08eb18..f3603554e3 100644
@@ -1973,10 +1973,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
      }
  #endif
 
-    if (s->ats == ON_OFF_AUTO_AUTO) {
-        error_setg(errp, "ats auto mode is not supported");
-        return false;
-    }
      if (s->ril == ON_OFF_AUTO_AUTO) {
          error_setg(errp, "ril auto mode is not supported");
          return false;

hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2200

Quoted 5 lines
hw/arm/smmuv3.c +0 -2
@@ -2196,8 +2192,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
          "is not supported.");
      object_class_property_set_description(klass, "ats",
          "Enable/disable ATS support (for accel=on). Please ensure host "
-        "platform has ATS support before enabling this. ats=auto is not "
-        "supported.");

I would explicitly mention that auto value is supported. Maybe:
Set ATS support. Valid values are off/on/auto. If set to on, make sure the host platform supports ATS.

Quoted 3 lines

You have examples in hw/arm/virt.c. For instance msi option.

Also note that the doc in qemu-options.hx must also be updated.

Ok, I'll explicitly mention it's supported here and in qemu-options.hx.

Thanks,
Nathan

#9 lore raw

PATCH 3/11 hw/arm/smmuv3: Change the default ats support to match the host

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Set the default value of ATS to auto. The default for ATS support used to be set to off, but we change it to match what the host IOMMU properties report. Introduce a new compat for the changed default.

---
 hw/arm/smmuv3.c   | 2 +-
 hw/core/machine.c | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index f3603554e3..f353865187 100644
@@ -2158,7 +2158,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
     /* RIL can be turned off for accel cases */
     DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_ON),
-    DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_OFF),
+    DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),
     DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_44),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
                               SSID_SIZE_MODE_0),
hw/core/machine.c +5 -0
index 0aa77a57e9..6796ab63cc 100644
@@ -37,6 +37,11 @@
 #include "hw/virtio/virtio-iommu.h"
 #include "hw/acpi/generic_event_device.h"
 #include "qemu/audio.h"
+#include "hw/arm/smmuv3.h"
+
+GlobalProperty hw_compat_11_0[] = {
+    { TYPE_ARM_SMMUV3, "ats", "off" },
+};
 
 GlobalProperty hw_compat_10_2[] = {
     { "scsi-block", "migrate-pr", "off" },
Signature · 1 line
2.43.0
#10 lore raw

Re: [PATCH 03/11] hw/arm/smmuv3: Change the default ats support to match the host

Quoted 17 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Set the default value of ATS to auto. The default for ATS support used to be set to off, but we change it to match what the host IOMMU properties report. Introduce a new compat for the changed default.

---
 hw/arm/smmuv3.c   | 2 +-
 hw/core/machine.c | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +0 -0
index f3603554e3..f353865187 100644

hw/arm/smmuv3.c static const Property smmuv3_properties[] = { · line 2161

Quoted 5 lines
hw/arm/smmuv3.c +1 -1
@@ -2158,7 +2158,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
     /* RIL can be turned off for accel cases */
     DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_ON),
-    DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_OFF),
+    DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),

see my previous comment about squashing all those patches. Besides you need to update qemu-options.hx where the default value is documented as non auto.

Eric

Quoted 19 lines
hw/arm/smmuv3.c +0 -0
     DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_44),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
                               SSID_SIZE_MODE_0),
hw/core/machine.c +5 -0
index 0aa77a57e9..6796ab63cc 100644
@@ -37,6 +37,11 @@
 #include "hw/virtio/virtio-iommu.h"
 #include "hw/acpi/generic_event_device.h"
 #include "qemu/audio.h"
+#include "hw/arm/smmuv3.h"
+
+GlobalProperty hw_compat_11_0[] = {
+    { TYPE_ARM_SMMUV3, "ats", "off" },
+};
 
 GlobalProperty hw_compat_10_2[] = {
     { "scsi-block", "migrate-pr", "off" },
#11 lore raw

Re: [PATCH 03/11] hw/arm/smmuv3: Change the default ats support to match the host

Quoted 14 lines · Eric Auger
Quoted 12 lines · Nathan Chen

From: Nathan Chen<[email protected]>

Set the default value of ATS to auto. The default for ATS support used to be set to off, but we change it to match what the host IOMMU properties report. Introduce a new compat for the changed default.

---
  hw/arm/smmuv3.c   | 2 +-
  hw/core/machine.c | 5 +++++
  2 files changed, 6 insertions(+), 1 deletion(-)
Quoted 13 lines

hw/arm/smmuv3.c static const Property smmuv3_properties[] = { · line 2161

Quoted 10 lines
hw/arm/smmuv3.c +1 -1
index f3603554e3..f353865187 100644
@@ -2158,7 +2158,7 @@ static const Property smmuv3_properties[] = {
      DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
      /* RIL can be turned off for accel cases */
      DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_ON),
-    DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_OFF),
+    DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),

see my previous comment about squashing all those patches. Besides you need to update qemu-options.hx where the default value is documented as non auto.

Ok, will squash all those patches into a single one and fix qemu-options.hx.

Thanks,
Nathan

#12 lore raw

PATCH 4/11 vfio/pci: Add ats property and mask ATS cap when not exposed

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

This aligns with the kernel's per-device effective ATS reporting and allows omitting ATS capability when the vIOMMU has ats=off.

---
 backends/iommufd.c                 | 15 +++++++
 hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++++++
 hw/vfio/pci.h                      |  1 +
 include/system/host_iommu_device.h | 10 +++++
 4 files changed, 89 insertions(+)
backends/iommufd.c +15 -0
index e1fee16acf..52cb060454 100644
@@ -22,6 +22,13 @@
 #include "hw/vfio/vfio-device.h"
 #include <sys/ioctl.h>
 #include <linux/iommufd.h>
+/*
+ * Until kernel UAPI is synced via scripts;
+ * matches include/uapi/linux/iommufd.h
+ */
+#ifndef IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+#define IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED (1 << 3)
+#endif
 
 static const char *iommufd_fd_name(IOMMUFDBackend *be)
 {
@@ -573,6 +580,13 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp)
     }
 }
 
+static bool hiod_iommufd_support_ats(HostIOMMUDevice *hiod)
+{
+    HostIOMMUDeviceCaps *caps = &hiod->caps;
+
+    return !(caps->hw_caps & IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED);
+}
+
 static bool hiod_iommufd_get_pasid_info(HostIOMMUDevice *hiod,
                                         PasidInfo *pasid_info)
 {
@@ -595,6 +609,7 @@ static void hiod_iommufd_class_init(ObjectClass *oc, const void *data)
 
     hioc->get_cap = hiod_iommufd_get_cap;
     hioc->get_pasid_info = hiod_iommufd_get_pasid_info;
+    hioc->support_ats = hiod_iommufd_support_ats;
 };
 
 static const TypeInfo types[] = {
hw/vfio/pci.c +63 -0

97 lines not on this page · raw message

hw/vfio/pci.h +1 -0
index d6495d7f29..514a9197ce 100644
@@ -191,6 +191,7 @@ struct VFIOPCIDevice {
     VFIODisplay *dpy;
     Notifier irqchip_change_notifier;
     VFIOPCICPR cpr;
+    OnOffAuto ats;
 };
 
 /* Use uin32_t for vendor & device so PCI_ANY_ID expands and cannot match hw */
include/system/host_iommu_device.h +10 -0
index f000301583..44c56e87bb 100644
@@ -133,6 +133,16 @@ struct HostIOMMUDeviceClass {
      * Returns: true on success, false on failure.
      */
     bool (*get_pasid_info)(HostIOMMUDevice *hiod, PasidInfo *pasid_info);
+    /**
+     *  @support_ats: Return whether ATS is supported for the device
+     *  associated with @hiod host IOMMU device, checking if the
+     *  IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED capability bit is set.
+     *
+     *  @hiod: handle to the host IOMMU device
+     *
+     *  Returns: true on success, false on failure
+     */
+    bool (*support_ats)(HostIOMMUDevice *hiod);
 };
 
 /*
Signature · 1 line
2.43.0
#13 lore raw

Re: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

Hello Nathan,

Quoted 7 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

Quoted 5 lines

This aligns with the kernel's per-device effective ATS reporting and allows omitting ATS capability when the vIOMMU has ats=off.

This looks like a standalone VFIO change that could be sent independently. Could you please rebase on vfio-next [*] ? There are a few conflicting changes in this branch.

[*] https://github.com/legoater/qemu/commits/vfio-next

You will need to add a compat machine property too. This is a guest-visible ABI change. A separate patch would be preferred.

Quoted 12 lines
---
  backends/iommufd.c                 | 15 +++++++
  hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++++++
  hw/vfio/pci.h                      |  1 +
  include/system/host_iommu_device.h | 10 +++++
  4 files changed, 89 insertions(+)
backends/iommufd.c +0 -0
index e1fee16acf..52cb060454 100644

backends/iommufd.c line 31

Quoted 10 lines
backends/iommufd.c +7 -0
@@ -22,6 +22,13 @@
  #include "hw/vfio/vfio-device.h"
  #include <sys/ioctl.h>
  #include <linux/iommufd.h>
+/*
+ * Until kernel UAPI is synced via scripts;
+ * matches include/uapi/linux/iommufd.h
+ */
+#ifndef IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+#define IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED (1 << 3)
+#endif

This definition is introduced by :

https://lore.kernel.org/all/[email protected]/

It should reach Linux v7.1 since it is in linux-next. Anyhow, until then, please isolate this change in a temporary patch updating linux-headers/

Quoted 29 lines
backends/iommufd.c +8 -0
  static const char *iommufd_fd_name(IOMMUFDBackend *be)
  {
@@ -573,6 +580,13 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp)
      }
  }
 
+static bool hiod_iommufd_support_ats(HostIOMMUDevice *hiod)
+{
+    HostIOMMUDeviceCaps *caps = &hiod->caps;
+
+    return !(caps->hw_caps & IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED);
+}
+
  static bool hiod_iommufd_get_pasid_info(HostIOMMUDevice *hiod,
                                          PasidInfo *pasid_info)
  {
@@ -595,6 +609,7 @@ static void hiod_iommufd_class_init(ObjectClass *oc, const void *data)
 
      hioc->get_cap = hiod_iommufd_get_cap;
      hioc->get_pasid_info = hiod_iommufd_get_pasid_info;
+    hioc->support_ats = hiod_iommufd_support_ats;
  };
 
  static const TypeInfo types[] = {
hw/vfio/pci.c +0 -0
index 1945751ffd..2d408e1d9a 100644

hw/vfio/pci.c line 56

Quoted 8 lines
hw/vfio/pci.c +4 -0
@@ -49,6 +49,10 @@
  #include "system/iommufd.h"
  #include "vfio-migration-internal.h"
  #include "vfio-helpers.h"
+#ifdef CONFIG_IOMMUFD
+#include "system/host_iommu_device.h"
+#include "linux/iommufd.h"
+#endif
 

These includes are not necessary.

Quoted 8 lines
hw/vfio/pci.c +2 -0
  /* Protected by BQL */
  static KVMRouteChange vfio_route_change;
@@ -2550,10 +2554,53 @@ static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp)
      return true;
  }
 
+/*
+ * Determine whether ATS capability should be advertised for @vdev, based on

hw/vfio/pci.c static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp) · line 2568

Quoted 10 lines
hw/vfio/pci.c +10 -0
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel's IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED bit.
+ *
+ * Store whether ATS capability should be advertised in @ats_need.
+ *
+ * Return false if kernel enables IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+ * and ATS is effectively unsupported.
+ */
+static bool vfio_pci_ats_requested_and_supported(VFIOPCIDevice *vdev,
+                                                 bool *ats_need, Error **errp)

May be rename 'ats_need' to 'ats' or 'ats_needed'

Quoted 12 lines
hw/vfio/pci.c +12 -0
+{
+    HostIOMMUDevice *hiod = vdev->vbasedev.hiod;
+    HostIOMMUDeviceClass *hiodc;
+    bool ats_supported;
+
+    if (vdev->ats == ON_OFF_AUTO_OFF) {
+        *ats_need = false;
+        return true;
+    }
+
+    *ats_need = true;
+    if (!hiod) {

hw/vfio/pci.c static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp) · line 2590

Quoted 10 lines
hw/vfio/pci.c +10 -0
+        return true;
+    }
+    hiodc = HOST_IOMMU_DEVICE_GET_CLASS(hiod);
+    if (!hiodc || !hiodc->support_ats) {
+        return true;
+    }
+
+    ats_supported = hiodc->support_ats(hiod);
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_supported) {
+        error_setg(errp, "vfio: ATS requested but not supported by kernel");

hmm, I wonder if we shouldn't fail to realize the device and stop the VM in this case (ats requested + no support).

Quoted 17 lines
hw/vfio/pci.c +9 -0
+        *ats_need = false;
+        return false;
+    }
+
+    *ats_need = ats_supported;
+    return true;
+}
+
  static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
  {
      PCIDevice *pdev = PCI_DEVICE(vdev);
      bool pasid_cap_added = false;
+    bool ats_needed = false;
      Error *err = NULL;
      uint32_t header;
      uint16_t cap_id, next, size;

hw/vfio/pci.c static void vfio_add_ext_cap(VFIOPCIDevice *vdev) · line 2655

Quoted 6 lines
hw/vfio/pci.c +3 -0
@@ -2603,6 +2650,11 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
      pci_set_long(pdev->wmask + PCI_CONFIG_SPACE_SIZE, 0);
      pci_set_long(vdev->emulated_config_bits + PCI_CONFIG_SPACE_SIZE, ~0);
 
+    if (!vfio_pci_ats_requested_and_supported(vdev, &ats_needed, &err)) {
+        error_report_err(err);
+       err = NULL;

indent error

hw/vfio/pci.c static void vfio_add_ext_cap(VFIOPCIDevice *vdev) · line 2694

Quoted 9 lines
hw/vfio/pci.c +2 -0
+    }
+
      for (next = PCI_CONFIG_SPACE_SIZE; next;
           next = PCI_EXT_CAP_NEXT(pci_get_long(config + next))) {
          header = pci_get_long(config + next);
@@ -2640,6 +2692,16 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
          case PCI_EXT_CAP_ID_PASID:
              pasid_cap_added = true;
              /* fallthrough */

This change is coupling the PASID and ATS capabilities. Is this intended ?

Quoted 14 lines
hw/vfio/pci.c +10 -0
+        case PCI_EXT_CAP_ID_ATS:
+            /*
+             * If ATS is requested and supported according to the kernel, add
+             * the ATS capability. If not supported according to the kernel or
+             * disabled on the qemu command line, omit the ATS cap.
+             */
+            if (ats_needed) {
+                pcie_add_capability(pdev, cap_id, cap_ver, next, size);
+            }
+            break;
          default:
              pcie_add_capability(pdev, cap_id, cap_ver, next, size);
          }

hw/vfio/pci.c static const Property vfio_pci_properties[] = { · line 3884

Quoted 4 lines
hw/vfio/pci.c +1 -0
@@ -3819,6 +3881,7 @@ static const Property vfio_pci_properties[] = {
  #ifdef CONFIG_IOMMUFD
      DEFINE_PROP_LINK("iommufd", VFIOPCIDevice, vbasedev.iommufd,
                       TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *),
+    DEFINE_PROP_ON_OFF_AUTO("ats", VFIOPCIDevice, ats, ON_OFF_AUTO_AUTO),

conflicts with vfio-next

include/system/host_iommu_device.h

Quoted 19 lines
hw/vfio/pci.c +0 -0
  #endif
      DEFINE_PROP_BOOL("skip-vsc-check", VFIOPCIDevice, skip_vsc_check, true),
      DEFINE_PROP_UINT16("x-vpasid-cap-offset", VFIOPCIDevice,
hw/vfio/pci.h +1 -0
index d6495d7f29..514a9197ce 100644
@@ -191,6 +191,7 @@ struct VFIOPCIDevice {
      VFIODisplay *dpy;
      Notifier irqchip_change_notifier;
      VFIOPCICPR cpr;
+    OnOffAuto ats;
  };
 
  /* Use uin32_t for vendor & device so PCI_ANY_ID expands and cannot match hw */
include/system/host_iommu_device.h +0 -0
index f000301583..44c56e87bb 100644

Please add to your .git/config :

[diff]
orderFile = /path/to/qemu/scripts/git.orderfile

include/system/host_iommu_device.h struct HostIOMMUDeviceClass { · line 143

Quoted 12 lines
include/system/host_iommu_device.h +8 -0
@@ -133,6 +133,16 @@ struct HostIOMMUDeviceClass {
       * Returns: true on success, false on failure.
       */
      bool (*get_pasid_info)(HostIOMMUDevice *hiod, PasidInfo *pasid_info);
+    /**
+     *  @support_ats: Return whether ATS is supported for the device
+     *  associated with @hiod host IOMMU device, checking if the
+     *  IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED capability bit is set.
+     *
+     *  @hiod: handle to the host IOMMU device
+     *
+     *  Returns: true on success, false on failure

"Returns: true if ATS is supported, false otherwise"

include/system/host_iommu_device.h struct HostIOMMUDeviceClass { · line 148

Quoted 5 lines
include/system/host_iommu_device.h +2 -0
+     */
+    bool (*support_ats)(HostIOMMUDevice *hiod);
  };
 
  /*

I would split this patch in 4 parts :

- linux-headers/ v7.1 update
- support_ats() handler introduction and iommufd implementation
- "ats" property addition
- compat property fixup

Thanks,

C.

#14 lore raw

Re: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

Hi Cédric,

Quoted 16 lines · Cédric Le Goater

Hello Nathan,

Quoted 7 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

Quoted 5 lines

This aligns with the kernel's per-device effective ATS reporting and allows omitting ATS capability when the vIOMMU has ats=off.

Quoted 9 lines

This looks like a standalone VFIO change that could be sent independently. Could you please rebase on vfio-next [*] ? There are a few conflicting changes in this branch.

[*] https://github.com/legoater/qemu/commits/vfio-next

You will need to add a compat machine property too. This is a guest-visible ABI change. A separate patch would be preferred.

Yes, I will separate this out and rebase on vfio-next, with the compat machine property added.

Quoted 25 lines
Quoted 12 lines

---
  backends/iommufd.c                 | 15 +++++++ hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++++++ hw/vfio/pci.h                      |  1 + include/system/host_iommu_device.h | 10 +++++

  4 files changed, 89 insertions(+)
backends/iommufd.c +0 -0
index e1fee16acf..52cb060454 100644

backends/iommufd.c line 31

Quoted 10 lines
backends/iommufd.c +7 -0
@@ -22,6 +22,13 @@
   #include "hw/vfio/vfio-device.h"
   #include <sys/ioctl.h>
   #include <linux/iommufd.h>
+/*
+ * Until kernel UAPI is synced via scripts;
+ * matches include/uapi/linux/iommufd.h
+ */
+#ifndef IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+#define IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED (1 << 3)
+#endif

This definition is introduced by :

Quoted 5 lines

https://lore.kernel.org/all/20260303150348.233997-1- [email protected]/

It should reach Linux v7.1 since it is in linux-next. Anyhow, until then, please isolate this change in a temporary patch updating linux-headers/

Ok, I will separate it out.

Quoted 28 lines
Quoted 28 lines
backends/iommufd.c +8 -0
   static const char *iommufd_fd_name(IOMMUFDBackend *be)
   {
@@ -573,6 +580,13 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice 
 *hiod, int cap, Error **errp)
       }
   }
+static bool hiod_iommufd_support_ats(HostIOMMUDevice *hiod)
+{
+    HostIOMMUDeviceCaps *caps = &hiod->caps;
+
+    return !(caps->hw_caps & IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED);
+}
+
   static bool hiod_iommufd_get_pasid_info(HostIOMMUDevice *hiod,
                                           PasidInfo *pasid_info)
   {
@@ -595,6 +609,7 @@ static void hiod_iommufd_class_init(ObjectClass 
 *oc, const void *data)
       hioc->get_cap = hiod_iommufd_get_cap;
       hioc->get_pasid_info = hiod_iommufd_get_pasid_info;
+    hioc->support_ats = hiod_iommufd_support_ats;
   };
   static const TypeInfo types[] = {
hw/vfio/pci.c +0 -0
index 1945751ffd..2d408e1d9a 100644
@@ -49,6 +49,10 @@
Quoted 8 lines

hw/vfio/pci.c line 55

Quoted 7 lines
hw/vfio/pci.c +4 -0
   #include "system/iommufd.h"
   #include "vfio-migration-internal.h"
   #include "vfio-helpers.h"
+#ifdef CONFIG_IOMMUFD
+#include "system/host_iommu_device.h"
+#include "linux/iommufd.h"
+#endif

These includes are not necessary.

Got it, I will remove these. I see that system/host_iommu_device.h is included in system/iommufd.h already. And linux/iommufd.h was leftover from a previous revision.

Quoted 16 lines
Quoted 16 lines
hw/vfio/pci.c +2 -0
   /* Protected by BQL */
   static KVMRouteChange vfio_route_change;
@@ -2550,10 +2554,53 @@ static bool 
 vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp)
       return true;
   }
+/*
+ * Determine whether ATS capability should be advertised for @vdev, 

based on
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel's IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED bit.
+ *
+ * Store whether ATS capability should be advertised in @ats_need.
+ *

Quoted 9 lines
Quoted 6 lines

+ * Return false if kernel enables IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+ * and ATS is effectively unsupported.
+ */
+static bool vfio_pci_ats_requested_and_supported(VFIOPCIDevice *vdev, +                                                 bool *ats_need, Error **errp)

May be rename 'ats_need' to 'ats' or 'ats_needed'

Ok, I will rename this to ats_needed.

Quoted 18 lines
Quoted 18 lines
hw/vfio/pci.c +18 -0
+{
+    HostIOMMUDevice *hiod = vdev->vbasedev.hiod;
+    HostIOMMUDeviceClass *hiodc;
+    bool ats_supported;
+
+    if (vdev->ats == ON_OFF_AUTO_OFF) {
+        *ats_need = false;
+        return true;
+    }
+
+    *ats_need = true;
+    if (!hiod) {
+        return true;
+    }
+    hiodc = HOST_IOMMU_DEVICE_GET_CLASS(hiod);
+    if (!hiodc || !hiodc->support_ats) {
+        return true;
+    }
Quoted 8 lines
Quoted 5 lines

+
+    ats_supported = hiodc->support_ats(hiod);
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_supported) { +        error_setg(errp, "vfio: ATS requested but not supported by kernel");

hmm, I wonder if we shouldn't fail to realize the device and stop the VM in this case (ats requested + no support).

Agreed. I can move the vfio_pci_ats_requested_and_supported() call to vfio_pci_add_capabilities() so we can propagate the Error **, and then pass ats_needed as a new input arg to vfio_add_ext_cap. Would that be alright?

Quoted 18 lines
Quoted 18 lines

+        *ats_need = false;
+        return false;
+    }

hw/vfio/pci.c +5 -0
+
+    *ats_need = ats_supported;
+    return true;
+}
+

  static void vfio_add_ext_cap(VFIOPCIDevice *vdev) {
      PCIDevice *pdev = PCI_DEVICE(vdev); bool pasid_cap_added = false;
+    bool ats_needed = false;
      Error *err = NULL;
      uint32_t header;
      uint16_t cap_id, next, size;

hw/vfio/pci.c +0 -0
@@ -2603,6 +2650,11 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
Quoted 9 lines

hw/vfio/pci.c static void vfio_add_ext_cap(VFIOPCIDevice *vdev) · line 2655

Quoted 6 lines
hw/vfio/pci.c +3 -0
       pci_set_long(pdev->wmask + PCI_CONFIG_SPACE_SIZE, 0);
       pci_set_long(vdev->emulated_config_bits + PCI_CONFIG_SPACE_SIZE, ~0);
+    if (!vfio_pci_ats_requested_and_supported(vdev, &ats_needed, 
 &err)) {
+        error_report_err(err);
+       err = NULL;

indent error

Thanks, will fix this.

Quoted 12 lines
Quoted 9 lines
hw/vfio/pci.c +2 -0
+    }
+
       for (next = PCI_CONFIG_SPACE_SIZE; next;
            next = PCI_EXT_CAP_NEXT(pci_get_long(config + next))) {
           header = pci_get_long(config + next);
@@ -2640,6 +2692,16 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)

          case PCI_EXT_CAP_ID_PASID:
              pasid_cap_added = true;
              /* fallthrough */

This change is coupling the PASID and ATS capabilities. Is this intended ?

This was not intentional, I will add a pcie_add_capability() call and break for the PASID case.

Quoted 16 lines
Quoted 16 lines

+        case PCI_EXT_CAP_ID_ATS:
+            /*
+             * If ATS is requested and supported according to the kernel, add
+             * the ATS capability. If not supported according to the kernel or
+             * disabled on the qemu command line, omit the ATS cap. +             */
+            if (ats_needed) {
+                pcie_add_capability(pdev, cap_id, cap_ver, next, size); +            }
+            break;
          default:
              pcie_add_capability(pdev, cap_id, cap_ver, next, size); }

Quoted 9 lines

hw/vfio/pci.c static const Property vfio_pci_properties[] = { · line 3885

Quoted 6 lines
hw/vfio/pci.c +1 -0
@@ -3819,6 +3881,7 @@ static const Property vfio_pci_properties[] = {
   #ifdef CONFIG_IOMMUFD
       DEFINE_PROP_LINK("iommufd", VFIOPCIDevice, vbasedev.iommufd,
                        TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *),
+    DEFINE_PROP_ON_OFF_AUTO("ats", VFIOPCIDevice, ats, 
 ON_OFF_AUTO_AUTO),

conflicts with vfio-next

Ok, will take a look and rebase on vfio-next.

Quoted 21 lines

include/system/host_iommu_device.h

Quoted 18 lines
hw/vfio/pci.c +0 -0
   #endif
       DEFINE_PROP_BOOL("skip-vsc-check", VFIOPCIDevice, skip_vsc_check, true),

      DEFINE_PROP_UINT16("x-vpasid-cap-offset", VFIOPCIDevice,

hw/vfio/pci.h +1 -0
index d6495d7f29..514a9197ce 100644
@@ -191,6 +191,7 @@ struct VFIOPCIDevice {
       VFIODisplay *dpy;
       Notifier irqchip_change_notifier;
       VFIOPCICPR cpr;
+    OnOffAuto ats;
   };
   /* Use uin32_t for vendor & device so PCI_ANY_ID expands and cannot match hw */
include/system/host_iommu_device.h +0 -0
index f000301583..44c56e87bb 100644

Please add to your .git/config :

Quoted 4 lines

[diff]
    orderFile = /path/to/qemu/scripts/git.orderfile

Ok, I will add this for the next refresh.

Quoted 6 lines
Quoted 6 lines
include/system/host_iommu_device.h +2 -0
@@ -133,6 +133,16 @@ struct HostIOMMUDeviceClass {
        * Returns: true on success, false on failure.
        */
       bool (*get_pasid_info)(HostIOMMUDevice *hiod, PasidInfo *pasid_info);
+    /**
+     *  @support_ats: Return whether ATS is supported for the device
Quoted 9 lines

include/system/host_iommu_device.h struct HostIOMMUDeviceClass { · line 143

Quoted 6 lines
include/system/host_iommu_device.h +6 -0
+     *  associated with @hiod host IOMMU device, checking if the
+     *  IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED capability bit is set.
+     *
+     *  @hiod: handle to the host IOMMU device
+     *
+     *  Returns: true on success, false on failure

 "Returns: true if ATS is supported, false otherwise"

Ok, I will make this more clear.

Quoted 6 lines

include/system/host_iommu_device.h struct HostIOMMUDeviceClass { · line 148

Quoted 4 lines
include/system/host_iommu_device.h +2 -0
+     */
+    bool (*support_ats)(HostIOMMUDevice *hiod);
   };
   /*
Quoted 6 lines

I would split this patch in 4 parts :

- linux-headers/ v7.1 update
- support_ats() handler introduction and iommufd implementation
- "ats" property addition
- compat property fixup

Got it, thanks for the guidance on organizing the split. I will send separately from this series.

Thanks,
Nathan

#15 lore raw

Re: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

Quoted 207 lines · Nathan Chen

Hi Cédric,

Quoted 16 lines · Cédric Le Goater

Hello Nathan,

Quoted 7 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

Quoted 5 lines

This aligns with the kernel's per-device effective ATS reporting and allows omitting ATS capability when the vIOMMU has ats=off.

Quoted 9 lines

This looks like a standalone VFIO change that could be sent independently. Could you please rebase on vfio-next [*] ? There are a few conflicting changes in this branch.

[*] https://github.com/legoater/qemu/commits/vfio-next

You will need to add a compat machine property too. This is a guest-visible ABI change. A separate patch would be preferred.

Yes, I will separate this out and rebase on vfio-next, with the compat machine property added.

Quoted 25 lines
Quoted 12 lines

---
  backends/iommufd.c                 | 15 +++++++ hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++++++ hw/vfio/pci.h                      |  1 + include/system/host_iommu_device.h | 10 +++++

  4 files changed, 89 insertions(+)
backends/iommufd.c +0 -0
index e1fee16acf..52cb060454 100644

backends/iommufd.c line 31

Quoted 10 lines
backends/iommufd.c +7 -0
@@ -22,6 +22,13 @@
   #include "hw/vfio/vfio-device.h"
   #include <sys/ioctl.h>
   #include <linux/iommufd.h>
+/*
+ * Until kernel UAPI is synced via scripts;
+ * matches include/uapi/linux/iommufd.h
+ */
+#ifndef IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+#define IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED (1 << 3)
+#endif

This definition is introduced by :

Quoted 5 lines

https://lore.kernel.org/all/20260303150348.233997-1- [email protected]/

It should reach Linux v7.1 since it is in linux-next. Anyhow, until then, please isolate this change in a temporary patch updating linux-headers/

Ok, I will separate it out.

Quoted 26 lines
Quoted 26 lines
backends/iommufd.c +8 -0
   static const char *iommufd_fd_name(IOMMUFDBackend *be)
   {
@@ -573,6 +580,13 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp)
       }
   }
+static bool hiod_iommufd_support_ats(HostIOMMUDevice *hiod)
+{
+    HostIOMMUDeviceCaps *caps = &hiod->caps;
+
+    return !(caps->hw_caps & IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED);
+}
+
   static bool hiod_iommufd_get_pasid_info(HostIOMMUDevice *hiod,
                                           PasidInfo *pasid_info)
   {
@@ -595,6 +609,7 @@ static void hiod_iommufd_class_init(ObjectClass *oc, const void *data)
       hioc->get_cap = hiod_iommufd_get_cap;
       hioc->get_pasid_info = hiod_iommufd_get_pasid_info;
+    hioc->support_ats = hiod_iommufd_support_ats;
   };
   static const TypeInfo types[] = {
hw/vfio/pci.c +0 -0
index 1945751ffd..2d408e1d9a 100644
@@ -49,6 +49,10 @@
Quoted 8 lines

hw/vfio/pci.c line 55

Quoted 7 lines
hw/vfio/pci.c +4 -0
   #include "system/iommufd.h"
   #include "vfio-migration-internal.h"
   #include "vfio-helpers.h"
+#ifdef CONFIG_IOMMUFD
+#include "system/host_iommu_device.h"
+#include "linux/iommufd.h"
+#endif

These includes are not necessary.

Got it, I will remove these. I see that system/host_iommu_device.h is included in system/iommufd.h already. And linux/iommufd.h was leftover from a previous revision.

Quoted 12 lines
Quoted 12 lines
hw/vfio/pci.c +6 -0
   /* Protected by BQL */
   static KVMRouteChange vfio_route_change;
@@ -2550,10 +2554,53 @@ static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp)
       return true;
   }
+/*
+ * Determine whether ATS capability should be advertised for @vdev, based on
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel's IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED bit.
+ *
+ * Store whether ATS capability should be advertised in @ats_need.
Quoted 9 lines

hw/vfio/pci.c static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp) · line 2568

Quoted 6 lines
hw/vfio/pci.c +6 -0
+ *
+ * Return false if kernel enables IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+ * and ATS is effectively unsupported.
+ */
+static bool vfio_pci_ats_requested_and_supported(VFIOPCIDevice *vdev,
+                                                 bool *ats_need, Error **errp)

May be rename 'ats_need' to 'ats' or 'ats_needed'

Ok, I will rename this to ats_needed.

Quoted 17 lines
Quoted 17 lines
hw/vfio/pci.c +17 -0
+{
+    HostIOMMUDevice *hiod = vdev->vbasedev.hiod;
+    HostIOMMUDeviceClass *hiodc;
+    bool ats_supported;
+
+    if (vdev->ats == ON_OFF_AUTO_OFF) {
+        *ats_need = false;
+        return true;
+    }
+
+    *ats_need = true;
+    if (!hiod) {
+        return true;
+    }
+    hiodc = HOST_IOMMU_DEVICE_GET_CLASS(hiod);
+    if (!hiodc || !hiodc->support_ats) {
+        return true;
Quoted 8 lines

hw/vfio/pci.c static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp) · line 2590

Quoted 5 lines
hw/vfio/pci.c +5 -0
+    }
+
+    ats_supported = hiodc->support_ats(hiod);
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_supported) {
+        error_setg(errp, "vfio: ATS requested but not supported by kernel");

hmm, I wonder if we shouldn't fail to realize the device and stop the VM in this case (ats requested + no support).

Agreed. I can move the vfio_pci_ats_requested_and_supported() call to vfio_pci_add_capabilities() so we can propagate the Error **, and then pass ats_needed as a new input arg to vfio_add_ext_cap. Would that be alright?

Quoted 17 lines
Quoted 17 lines
hw/vfio/pci.c +9 -0
+        *ats_need = false;
+        return false;
+    }
+
+    *ats_need = ats_supported;
+    return true;
+}
+
   static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
   {
       PCIDevice *pdev = PCI_DEVICE(vdev);
       bool pasid_cap_added = false;
+    bool ats_needed = false;
       Error *err = NULL;
       uint32_t header;
       uint16_t cap_id, next, size;
Quoted 9 lines

hw/vfio/pci.c static void vfio_add_ext_cap(VFIOPCIDevice *vdev) · line 2654

Quoted 6 lines
hw/vfio/pci.c +3 -0
@@ -2603,6 +2650,11 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
       pci_set_long(pdev->wmask + PCI_CONFIG_SPACE_SIZE, 0);
       pci_set_long(vdev->emulated_config_bits + PCI_CONFIG_SPACE_SIZE, ~0);
+    if (!vfio_pci_ats_requested_and_supported(vdev, &ats_needed, &err)) {
+        error_report_err(err);
+       err = NULL;

indent error

Thanks, will fix this.

Quoted 12 lines
Quoted 9 lines
hw/vfio/pci.c +2 -0
+    }
+
       for (next = PCI_CONFIG_SPACE_SIZE; next;
            next = PCI_EXT_CAP_NEXT(pci_get_long(config + next))) {
           header = pci_get_long(config + next);
@@ -2640,6 +2692,16 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)

          case PCI_EXT_CAP_ID_PASID:
              pasid_cap_added = true;
              /* fallthrough */

This change is coupling the PASID and ATS capabilities. Is this intended ?

This was not intentional, I will add a pcie_add_capability() call and break for the PASID case.

Quoted 13 lines
Quoted 13 lines

+        case PCI_EXT_CAP_ID_ATS:
+            /*
+             * If ATS is requested and supported according to the kernel, add +             * the ATS capability. If not supported according to the kernel or +             * disabled on the qemu command line, omit the ATS cap. +             */
+            if (ats_needed) {
+                pcie_add_capability(pdev, cap_id, cap_ver, next, size); +            }
+            break;
          default:
              pcie_add_capability(pdev, cap_id, cap_ver, next, size);

Quoted 9 lines

hw/vfio/pci.c static const Property vfio_pci_properties[] = { · line 3884

Quoted 6 lines

          }

hw/vfio/pci.c +1 -0
@@ -3819,6 +3881,7 @@ static const Property vfio_pci_properties[] = {
   #ifdef CONFIG_IOMMUFD
       DEFINE_PROP_LINK("iommufd", VFIOPCIDevice, vbasedev.iommufd,
                        TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *),
+    DEFINE_PROP_ON_OFF_AUTO("ats", VFIOPCIDevice, ats, ON_OFF_AUTO_AUTO),

conflicts with vfio-next

Quoted 1 line

Ok, will take a look and rebase on vfio-next.

when you rebase, please put the "ats" property at the end of the list and update accordingly vfio_pci_class_init().

Thanks,

C.

Quoted 59 lines
Quoted 21 lines

include/system/host_iommu_device.h

Quoted 18 lines
hw/vfio/pci.c +0 -0
   #endif
       DEFINE_PROP_BOOL("skip-vsc-check", VFIOPCIDevice, skip_vsc_check, true),
       DEFINE_PROP_UINT16("x-vpasid-cap-offset", VFIOPCIDevice,
hw/vfio/pci.h +1 -0
index d6495d7f29..514a9197ce 100644
@@ -191,6 +191,7 @@ struct VFIOPCIDevice {
       VFIODisplay *dpy;
       Notifier irqchip_change_notifier;
       VFIOPCICPR cpr;
+    OnOffAuto ats;
   };
   /* Use uin32_t for vendor & device so PCI_ANY_ID expands and cannot match hw */
include/system/host_iommu_device.h +0 -0
index f000301583..44c56e87bb 100644

Please add to your .git/config :

Quoted 4 lines

[diff]
     orderFile = /path/to/qemu/scripts/git.orderfile

Ok, I will add this for the next refresh.

Quoted 6 lines
Quoted 6 lines
include/system/host_iommu_device.h +2 -0
@@ -133,6 +133,16 @@ struct HostIOMMUDeviceClass {
        * Returns: true on success, false on failure.
        */
       bool (*get_pasid_info)(HostIOMMUDevice *hiod, PasidInfo *pasid_info);
+    /**
+     *  @support_ats: Return whether ATS is supported for the device
Quoted 9 lines

include/system/host_iommu_device.h struct HostIOMMUDeviceClass { · line 143

Quoted 6 lines
include/system/host_iommu_device.h +6 -0
+     *  associated with @hiod host IOMMU device, checking if the
+     *  IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED capability bit is set.
+     *
+     *  @hiod: handle to the host IOMMU device
+     *
+     *  Returns: true on success, false on failure

  "Returns: true if ATS is supported, false otherwise"

Ok, I will make this more clear.

Quoted 6 lines

include/system/host_iommu_device.h struct HostIOMMUDeviceClass { · line 148

Quoted 4 lines
include/system/host_iommu_device.h +2 -0
+     */
+    bool (*support_ats)(HostIOMMUDevice *hiod);
   };
   /*
Quoted 6 lines

I would split this patch in 4 parts :

- linux-headers/ v7.1 update
- support_ats() handler introduction and iommufd implementation
- "ats" property addition
- compat property fixup

Got it, thanks for the guidance on organizing the split. I will send separately from this series.

Thanks,
Nathan

#16 lore raw

Re: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

Quoted 223 lines · Cédric Le Goater
Quoted 222 lines · Nathan Chen

Hi Cédric,

Quoted 16 lines · Cédric Le Goater

Hello Nathan,

Quoted 7 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

Quoted 5 lines

This aligns with the kernel's per-device effective ATS reporting and allows omitting ATS capability when the vIOMMU has ats=off.

Quoted 9 lines

This looks like a standalone VFIO change that could be sent independently. Could you please rebase on vfio-next [*] ? There are a few conflicting changes in this branch.

[*] https://github.com/legoater/qemu/commits/vfio-next

You will need to add a compat machine property too. This is a guest- visible ABI change. A separate patch would be preferred.

Yes, I will separate this out and rebase on vfio-next, with the compat machine property added.

Quoted 26 lines
Quoted 13 lines

---
  backends/iommufd.c                 | 15 +++++++ hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++
++++
  hw/vfio/pci.h                      |  1 + include/system/host_iommu_device.h | 10 +++++

  4 files changed, 89 insertions(+)
backends/iommufd.c +0 -0
index e1fee16acf..52cb060454 100644

backends/iommufd.c line 31

Quoted 10 lines
backends/iommufd.c +7 -0
@@ -22,6 +22,13 @@
   #include "hw/vfio/vfio-device.h"
   #include <sys/ioctl.h>
   #include <linux/iommufd.h>
+/*
+ * Until kernel UAPI is synced via scripts;
+ * matches include/uapi/linux/iommufd.h
+ */
+#ifndef IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+#define IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED (1 << 3)
+#endif

This definition is introduced by :

Quoted 5 lines

https://lore.kernel.org/all/20260303150348.233997-1- [email protected]/

It should reach Linux v7.1 since it is in linux-next. Anyhow, until then, please isolate this change in a temporary patch updating linux-headers/

Ok, I will separate it out.

Quoted 28 lines
Quoted 28 lines
backends/iommufd.c +8 -0
   static const char *iommufd_fd_name(IOMMUFDBackend *be)
   {
@@ -573,6 +580,13 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice 
 *hiod, int cap, Error **errp)
       }
   }
+static bool hiod_iommufd_support_ats(HostIOMMUDevice *hiod)
+{
+    HostIOMMUDeviceCaps *caps = &hiod->caps;
+
+    return !(caps->hw_caps & IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED);
+}
+
   static bool hiod_iommufd_get_pasid_info(HostIOMMUDevice *hiod,
                                           PasidInfo *pasid_info)
   {
@@ -595,6 +609,7 @@ static void hiod_iommufd_class_init(ObjectClass 
 *oc, const void *data)
       hioc->get_cap = hiod_iommufd_get_cap;
       hioc->get_pasid_info = hiod_iommufd_get_pasid_info;
+    hioc->support_ats = hiod_iommufd_support_ats;
   };
   static const TypeInfo types[] = {
hw/vfio/pci.c +0 -0
index 1945751ffd..2d408e1d9a 100644
@@ -49,6 +49,10 @@
Quoted 8 lines

hw/vfio/pci.c line 55

Quoted 7 lines
hw/vfio/pci.c +4 -0
   #include "system/iommufd.h"
   #include "vfio-migration-internal.h"
   #include "vfio-helpers.h"
+#ifdef CONFIG_IOMMUFD
+#include "system/host_iommu_device.h"
+#include "linux/iommufd.h"
+#endif

These includes are not necessary.

Got it, I will remove these. I see that system/host_iommu_device.h is included in system/iommufd.h already. And linux/iommufd.h was leftover from a previous revision.

Quoted 16 lines
Quoted 16 lines
hw/vfio/pci.c +2 -0
   /* Protected by BQL */
   static KVMRouteChange vfio_route_change;
@@ -2550,10 +2554,53 @@ static bool 
 vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp)
       return true;
   }
+/*
+ * Determine whether ATS capability should be advertised for @vdev, 

based on
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel's IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED bit.
+ *
+ * Store whether ATS capability should be advertised in @ats_need.
+ *

Quoted 9 lines
Quoted 6 lines

+ * Return false if kernel enables IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+ * and ATS is effectively unsupported.
+ */
+static bool vfio_pci_ats_requested_and_supported(VFIOPCIDevice *vdev, +                                                 bool *ats_need, Error **errp)

May be rename 'ats_need' to 'ats' or 'ats_needed'

Ok, I will rename this to ats_needed.

Quoted 18 lines
Quoted 18 lines
hw/vfio/pci.c +18 -0
+{
+    HostIOMMUDevice *hiod = vdev->vbasedev.hiod;
+    HostIOMMUDeviceClass *hiodc;
+    bool ats_supported;
+
+    if (vdev->ats == ON_OFF_AUTO_OFF) {
+        *ats_need = false;
+        return true;
+    }
+
+    *ats_need = true;
+    if (!hiod) {
+        return true;
+    }
+    hiodc = HOST_IOMMU_DEVICE_GET_CLASS(hiod);
+    if (!hiodc || !hiodc->support_ats) {
+        return true;
+    }
Quoted 8 lines
Quoted 5 lines

+
+    ats_supported = hiodc->support_ats(hiod);
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_supported) { +        error_setg(errp, "vfio: ATS requested but not supported by kernel");

hmm, I wonder if we shouldn't fail to realize the device and stop the VM in this case (ats requested + no support).

Agreed. I can move the vfio_pci_ats_requested_and_supported() call to vfio_pci_add_capabilities() so we can propagate the Error **, and then pass ats_needed as a new input arg to vfio_add_ext_cap. Would that be alright?

Quoted 19 lines
Quoted 19 lines

+        *ats_need = false;
+        return false;
+    }

hw/vfio/pci.c +5 -0
+
+    *ats_need = ats_supported;
+    return true;
+}
+

  static void vfio_add_ext_cap(VFIOPCIDevice *vdev) {
      PCIDevice *pdev = PCI_DEVICE(vdev); bool pasid_cap_added = false;
+    bool ats_needed = false;
      Error *err = NULL;
      uint32_t header;
      uint16_t cap_id, next, size;

hw/vfio/pci.c +0 -0
@@ -2603,6 +2650,11 @@ static void vfio_add_ext_cap(VFIOPCIDevice 
 *vdev)
Quoted 9 lines

hw/vfio/pci.c static void vfio_add_ext_cap(VFIOPCIDevice *vdev) · line 2655

Quoted 6 lines
hw/vfio/pci.c +3 -0
       pci_set_long(pdev->wmask + PCI_CONFIG_SPACE_SIZE, 0);
       pci_set_long(vdev->emulated_config_bits + PCI_CONFIG_SPACE_SIZE, ~0);
+    if (!vfio_pci_ats_requested_and_supported(vdev, &ats_needed, 
 &err)) {
+        error_report_err(err);
+       err = NULL;

indent error

Thanks, will fix this.

Quoted 4 lines
Quoted 4 lines
hw/vfio/pci.c +2 -0
+    }
+
       for (next = PCI_CONFIG_SPACE_SIZE; next;
            next = PCI_EXT_CAP_NEXT(pci_get_long(config + next))) {
Quoted 9 lines
Quoted 6 lines

          header = pci_get_long(config + next);

hw/vfio/pci.c +0 -0
@@ -2640,6 +2692,16 @@ static void vfio_add_ext_cap(VFIOPCIDevice 
 *vdev)

          case PCI_EXT_CAP_ID_PASID:
              pasid_cap_added = true;
              /* fallthrough */

This change is coupling the PASID and ATS capabilities. Is this intended ?

This was not intentional, I will add a pcie_add_capability() call and break for the PASID case.

Quoted 26 lines
Quoted 18 lines

+        case PCI_EXT_CAP_ID_ATS:
+            /*
+             * If ATS is requested and supported according to the kernel, add
+             * the ATS capability. If not supported according to the kernel or
+             * disabled on the qemu command line, omit the ATS cap. +             */
+            if (ats_needed) {
+                pcie_add_capability(pdev, cap_id, cap_ver, next, size);
+            }
+            break;
          default:
              pcie_add_capability(pdev, cap_id, cap_ver, next, size); }

hw/vfio/pci.c static const Property vfio_pci_properties[] = { · line 3885

Quoted 5 lines
hw/vfio/pci.c +1 -0
@@ -3819,6 +3881,7 @@ static const Property vfio_pci_properties[] = {
   #ifdef CONFIG_IOMMUFD
       DEFINE_PROP_LINK("iommufd", VFIOPCIDevice, vbasedev.iommufd,
                        TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *),
+    DEFINE_PROP_ON_OFF_AUTO("ats", VFIOPCIDevice, ats, 
 ON_OFF_AUTO_AUTO),

conflicts with vfio-next

Quoted 4 lines
Quoted 1 line

Ok, will take a look and rebase on vfio-next.

when you rebase, please put the "ats" property at the end of the list and update accordingly vfio_pci_class_init().

Ok, I will keep this in mind, thanks!

Nathan

#17 lore raw

Re: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

In reply to Cédric Le Goater
Quoted 50 lines · Cédric Le Goater

Hello Nathan,

Quoted 7 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

Quoted 5 lines

This aligns with the kernel's per-device effective ATS reporting and allows omitting ATS capability when the vIOMMU has ats=off.

This looks like a standalone VFIO change that could be sent independently. Could you please rebase on vfio-next [*] ? There are a few conflicting changes in this branch.

[*] https://github.com/legoater/qemu/commits/vfio-next

You will need to add a compat machine property too. This is a guest-visible ABI change. A separate patch would be preferred.

Quoted 12 lines

---
  backends/iommufd.c                 | 15 +++++++ hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++++++ hw/vfio/pci.h                      |  1 + include/system/host_iommu_device.h | 10 +++++

  4 files changed, 89 insertions(+)
backends/iommufd.c +0 -0
index e1fee16acf..52cb060454 100644

backends/iommufd.c line 31

Quoted 10 lines
backends/iommufd.c +7 -0
@@ -22,6 +22,13 @@
   #include "hw/vfio/vfio-device.h"
   #include <sys/ioctl.h>
   #include <linux/iommufd.h>
+/*
+ * Until kernel UAPI is synced via scripts;
+ * matches include/uapi/linux/iommufd.h
+ */
+#ifndef IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+#define IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED (1 << 3)
+#endif

This definition is introduced by :

Quoted 1 line

  https://lore.kernel.org/all/[email protected]/

This is now upstream.

Quoted 2 lines

It should reach Linux v7.1 since it is in linux-next. Anyhow, until then, please isolate this change in a temporary patch updating linux-headers/

Please use script ./scripts/update-linux-headers.sh. I would wait for v7.1-rc1 to be tagged though.

Thanks,

C.

#18 lore raw

Re: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

Quoted 53 lines · Cédric Le Goater
Quoted 50 lines · Cédric Le Goater

Hello Nathan,

Quoted 7 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

Quoted 5 lines

This aligns with the kernel's per-device effective ATS reporting and allows omitting ATS capability when the vIOMMU has ats=off.

This looks like a standalone VFIO change that could be sent independently. Could you please rebase on vfio-next [*] ? There are a few conflicting changes in this branch.

[*] https://github.com/legoater/qemu/commits/vfio-next

You will need to add a compat machine property too. This is a guest- visible ABI change. A separate patch would be preferred.

Quoted 12 lines

---
  backends/iommufd.c                 | 15 +++++++ hw/vfio/pci.c                      | 63 ++++++++++++++++++++++++++++++ hw/vfio/pci.h                      |  1 + include/system/host_iommu_device.h | 10 +++++

  4 files changed, 89 insertions(+)
backends/iommufd.c +0 -0
index e1fee16acf..52cb060454 100644

backends/iommufd.c line 31

Quoted 10 lines
backends/iommufd.c +7 -0
@@ -22,6 +22,13 @@
   #include "hw/vfio/vfio-device.h"
   #include <sys/ioctl.h>
   #include <linux/iommufd.h>
+/*
+ * Until kernel UAPI is synced via scripts;
+ * matches include/uapi/linux/iommufd.h
+ */
+#ifndef IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+#define IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED (1 << 3)
+#endif

This definition is introduced by :

Quoted 1 line

https://lore.kernel.org/all/20260303150348.233997-1- [email protected]/

Quoted 7 lines

This is now upstream.

Quoted 2 lines

It should reach Linux v7.1 since it is in linux-next. Anyhow, until then, please isolate this change in a temporary patch updating linux-headers/

Please use script ./scripts/update-linux-headers.sh. I would wait for v7.1-rc1 to be tagged though.

Ok, I will use this script after v7.1-rc1 is tagged.

Thanks,
Nathan

#19 lore raw

RE: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

In reply to Nathan Chen
Quoted 18 lines

-----Original Message-----
From: Nathan Chen <[email protected]>
Sent: 01 April 2026 02:02
To: [email protected]; [email protected]
Cc: Yi Liu <[email protected]>; Eric Auger <[email protected]>; Zhenzhong Duan <[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang <[email protected]>; Zhao Liu <[email protected]>; Alex Williamson <[email protected]>; Cédric Le Goater <[email protected]>; Shameer Kolothum Thodi <[email protected]>; Matt Ochs <[email protected]>; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>
Subject: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

[...]

Quoted 5 lines

+/*
+ * Determine whether ATS capability should be advertised for @vdev, based on
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel's IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED

Quoted 8 lines

bit.
+ *
+ * Store whether ATS capability should be advertised in @ats_need.
+ *
+ * Return false if kernel enables IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+ * and ATS is effectively unsupported.
+ */

Returns false only when ats=on is explicitly requested but the kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED. Returns true in all other cases.

Isn't it?

Thanks,
Shameer

#20 lore raw

Re: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

Quoted 32 lines · Shameer Kolothum Thodi
Quoted 17 lines

-----Original Message-----
From: Nathan Chen<[email protected]>
Sent: 01 April 2026 02:02 To:[email protected];[email protected]
Cc: Yi Liu<[email protected]>; Eric Auger<[email protected]>; Zhenzhong Duan<[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang<[email protected]>; Zhao Liu <[email protected]>; Alex Williamson<[email protected]>; Cédric Le Goater <[email protected]>; Shameer Kolothum Thodi<[email protected]>; Matt Ochs<[email protected]>; Nicolin Chen<[email protected]>; Nathan Chen<[email protected]>
Subject: [PATCH 04/11] vfio/pci: Add ats property and mask ATS cap when not exposed

[...]

Quoted 5 lines

+/*
+ * Determine whether ATS capability should be advertised for @vdev, based on
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel's IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED

Quoted 8 lines

bit.
+ *
+ * Store whether ATS capability should be advertised in @ats_need.
+ *
+ * Return false if kernel enables IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
+ * and ATS is effectively unsupported.
+ */

Quoted 4 lines

Returns false only when ats=on is explicitly requested but the kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED. Returns true in all other cases.

Isn't it?

Yes, I will fix the comment to say it returns false only when ats=on is requested but kernel reports it's not supported. If kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED but ats=off or ats=auto, then we still return true but set ats_need to false.

Thanks,
Nathan

#21 lore raw

PATCH 5/11 hw/arm/smmuv3-accel: Implement "auto" value for "ril"

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 Range Invalidation support property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving RIL capability from IDR3.

---
 hw/arm/smmuv3-accel.c | 9 ++++++++-
 hw/arm/smmuv3.c       | 7 +------
 2 files changed, 9 insertions(+), 7 deletions(-)
hw/arm/smmuv3-accel.c +8 -1
index a835c8f220..9e7ec6a930 100644
@@ -50,6 +50,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                FIELD_EX32(info->idr[0], IDR0, ATS));
     }
 
+    /* Update RIL if auto from info */
+    if (s->ril == ON_OFF_AUTO_AUTO) {
+        s->idr[3] = FIELD_DP32(s->idr[3], IDR3, RIL,
+                               FIELD_EX32(info->idr[3], IDR3, RIL));
+    }
+
     accel->auto_finalised = true;
 }
 
@@ -945,7 +951,8 @@ void smmuv3_accel_init(SMMUv3State *s)
     bs->iommu_ops = &smmuv3_accel_ops;
     smmuv3_accel_as_init(s);
 
-    if (s->ats == ON_OFF_AUTO_AUTO) {
+    if (s->ats == ON_OFF_AUTO_AUTO ||
+        s->ril == ON_OFF_AUTO_AUTO) {
         s->s_accel->auto_mode = true;
     }
 }
hw/arm/smmuv3.c +1 -6
index f353865187..a2d2f0e3bc 100644
@@ -1973,10 +1973,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->ril == ON_OFF_AUTO_AUTO) {
-        error_setg(errp, "ril auto mode is not supported");
-        return false;
-    }
     if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
         error_setg(errp, "ssidsize auto mode is not supported");
         return false;
@@ -2188,8 +2184,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Enable SMMUv3 accelerator support. Allows host SMMUv3 to be "
         "configured in nested mode for vfio-pci dev assignment");
     object_class_property_set_description(klass, "ril",
-        "Disable range invalidation support (for accel=on). ril=auto "
-        "is not supported.");
+        "Disable range invalidation support (for accel=on).");
     object_class_property_set_description(klass, "ats",
         "Enable/disable ATS support (for accel=on). Please ensure host "
         "platform has ATS support before enabling this.");
Signature · 1 line
2.43.0
#22 lore raw

Re: [PATCH 05/11] hw/arm/smmuv3-accel: Implement "auto" value for "ril"

Quoted 17 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 Range Invalidation support property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving RIL capability from IDR3.

---
 hw/arm/smmuv3-accel.c | 9 ++++++++-
 hw/arm/smmuv3.c       | 7 +------
 2 files changed, 9 insertions(+), 7 deletions(-)
hw/arm/smmuv3-accel.c +0 -0
index a835c8f220..9e7ec6a930 100644

hw/arm/smmuv3-accel.c static void smmuv3_accel_auto_finalise(SMMUv3State *s, · line 53

Quoted 4 lines
hw/arm/smmuv3-accel.c +1 -0
@@ -50,6 +50,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                FIELD_EX32(info->idr[0], IDR0, ATS));
     }
 
+    /* Update RIL if auto from info */

not sure that kind of comment is really useful.

Quoted 34 lines
hw/arm/smmuv3-accel.c +7 -1
+    if (s->ril == ON_OFF_AUTO_AUTO) {
+        s->idr[3] = FIELD_DP32(s->idr[3], IDR3, RIL,
+                               FIELD_EX32(info->idr[3], IDR3, RIL));
+    }
+
     accel->auto_finalised = true;
 }
 
@@ -945,7 +951,8 @@ void smmuv3_accel_init(SMMUv3State *s)
     bs->iommu_ops = &smmuv3_accel_ops;
     smmuv3_accel_as_init(s);
 
-    if (s->ats == ON_OFF_AUTO_AUTO) {
+    if (s->ats == ON_OFF_AUTO_AUTO ||
+        s->ril == ON_OFF_AUTO_AUTO) {
         s->s_accel->auto_mode = true;
     }
 }
hw/arm/smmuv3.c +0 -4
index f353865187..a2d2f0e3bc 100644
@@ -1973,10 +1973,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->ril == ON_OFF_AUTO_AUTO) {
-        error_setg(errp, "ril auto mode is not supported");
-        return false;
-    }
     if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
         error_setg(errp, "ssidsize auto mode is not supported");
         return false;

hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2187

Quoted 6 lines
hw/arm/smmuv3.c +1 -2
@@ -2188,8 +2184,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Enable SMMUv3 accelerator support. Allows host SMMUv3 to be "
         "configured in nested mode for vfio-pci dev assignment");
     object_class_property_set_description(klass, "ril",
-        "Disable range invalidation support (for accel=on). ril=auto "
-        "is not supported.");
+        "Disable range invalidation support (for accel=on).");

same comment as before. Suggesting "Set RIL value. Valid values ..."
+ qemu-options.hx updates.

Eric

Quoted 3 lines
hw/arm/smmuv3.c +0 -0
     object_class_property_set_description(klass, "ats",
         "Enable/disable ATS support (for accel=on). Please ensure host "
         "platform has ATS support before enabling this.");
#23 lore raw

Re: [PATCH 05/11] hw/arm/smmuv3-accel: Implement "auto" value for "ril"

Quoted 22 lines · Eric Auger
Quoted 20 lines · Nathan Chen

From: Nathan Chen<[email protected]>

Allow accelerated SMMUv3 Range Invalidation support property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving RIL capability from IDR3.

---
  hw/arm/smmuv3-accel.c | 9 ++++++++-
  hw/arm/smmuv3.c       | 7 +------
  2 files changed, 9 insertions(+), 7 deletions(-)
hw/arm/smmuv3-accel.c +0 -0
index a835c8f220..9e7ec6a930 100644
@@ -50,6 +50,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                 FIELD_EX32(info->idr[0], IDR0, ATS));
      }
 
Quoted 2 lines

hw/arm/smmuv3-accel.c static void smmuv3_accel_auto_finalise(SMMUv3State *s, · line 53

Quoted 1 line
hw/arm/smmuv3-accel.c +1 -0
+    /* Update RIL if auto from info */

not sure that kind of comment is really useful.

I'll remove it for these properties and add some more details to the comment above about "Return if no auto for any or finalised already".

Quoted 26 lines
Quoted 26 lines
hw/arm/smmuv3-accel.c +7 -1
+    if (s->ril == ON_OFF_AUTO_AUTO) {
+        s->idr[3] = FIELD_DP32(s->idr[3], IDR3, RIL,
+                               FIELD_EX32(info->idr[3], IDR3, RIL));
+    }
+
      accel->auto_finalised = true;
  }
 
@@ -945,7 +951,8 @@ void smmuv3_accel_init(SMMUv3State *s)
      bs->iommu_ops = &smmuv3_accel_ops;
      smmuv3_accel_as_init(s);
 
-    if (s->ats == ON_OFF_AUTO_AUTO) {
+    if (s->ats == ON_OFF_AUTO_AUTO ||
+        s->ril == ON_OFF_AUTO_AUTO) {
          s->s_accel->auto_mode = true;
      }
  }
hw/arm/smmuv3.c +0 -0
index f353865187..a2d2f0e3bc 100644
@@ -1973,10 +1973,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
      }
  #endif
 
Quoted 16 lines
Quoted 8 lines
hw/arm/smmuv3.c +0 -4
-    if (s->ril == ON_OFF_AUTO_AUTO) {
-        error_setg(errp, "ril auto mode is not supported");
-        return false;
-    }
      if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
          error_setg(errp, "ssidsize auto mode is not supported");
          return false;

hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2187

Quoted 6 lines
hw/arm/smmuv3.c +1 -2
@@ -2188,8 +2184,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
          "Enable SMMUv3 accelerator support. Allows host SMMUv3 to be "
          "configured in nested mode for vfio-pci dev assignment");
      object_class_property_set_description(klass, "ril",
-        "Disable range invalidation support (for accel=on). ril=auto "
-        "is not supported.");
+        "Disable range invalidation support (for accel=on).");

same comment as before. Suggesting "Set RIL value. Valid values ..."
+ qemu-options.hx updates.

Yes, will update for RIL and the other values as well.

Thanks,
Nathan

#24 lore raw

PATCH 6/11 hw/arm/smmuv3: Change the default ril support to match the host

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Set the default value of RIL to auto. The default for RIL support used to be set to on, but we change it to match what the host IOMMU properties report so that users do not have to introspect host IDR3 for Range Invalidation support. Include the previous default for ril in the 11.0 compat.

---
 hw/arm/smmuv3.c   | 2 +-
 hw/core/machine.c | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index a2d2f0e3bc..74cc81ae32 100644
@@ -2153,7 +2153,7 @@ static const Property smmuv3_properties[] = {
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
     /* RIL can be turned off for accel cases */
-    DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_ON),
+    DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_AUTO),
     DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),
     DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_44),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
hw/core/machine.c +1 -0
index 6796ab63cc..8f8c26bbe8 100644
@@ -41,6 +41,7 @@
 
 GlobalProperty hw_compat_11_0[] = {
     { TYPE_ARM_SMMUV3, "ats", "off" },
+    { TYPE_ARM_SMMUV3, "ril", "on" },
 };
 
 GlobalProperty hw_compat_10_2[] = {
Signature · 1 line
2.43.0
#25 lore raw

Re: [PATCH 06/11] hw/arm/smmuv3: Change the default ril support to match the host

Hi Nathan,

Quoted 32 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Set the default value of RIL to auto. The default for RIL support used to be set to on, but we change it to match what the host IOMMU properties report so that users do not have to introspect host IDR3 for Range Invalidation support. Include the previous default for ril in the 11.0 compat.

---
 hw/arm/smmuv3.c   | 2 +-
 hw/core/machine.c | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index a2d2f0e3bc..74cc81ae32 100644
@@ -2153,7 +2153,7 @@ static const Property smmuv3_properties[] = {
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
     /* RIL can be turned off for accel cases */
-    DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_ON),
+    DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_AUTO),
     DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),
     DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_44),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
hw/core/machine.c +0 -0
index 6796ab63cc..8f8c26bbe8 100644

hw/core/machine.c line 44

Quoted 4 lines
hw/core/machine.c +1 -0
@@ -41,6 +41,7 @@
 
 GlobalProperty hw_compat_11_0[] = {
     { TYPE_ARM_SMMUV3, "ats", "off" },
+    { TYPE_ARM_SMMUV3, "ril", "on" },

I wonder if you shouldn't squash all the patches that change the default values of accel props to auto mode into a single patch, at the end of the series.

Eric

Quoted 3 lines
hw/core/machine.c +0 -0
 };
 
 GlobalProperty hw_compat_10_2[] = {
#26 lore raw

Re: [PATCH 06/11] hw/arm/smmuv3: Change the default ril support to match the host

Hi Eric,

Quoted 36 lines · Eric Auger

Hi Nathan,

Quoted 33 lines · Nathan Chen

From: Nathan Chen<[email protected]>

Set the default value of RIL to auto. The default for RIL support used to be set to on, but we change it to match what the host IOMMU properties report so that users do not have to introspect host IDR3 for Range Invalidation support. Include the previous default for ril in the 11.0 compat.

---
  hw/arm/smmuv3.c   | 2 +-
  hw/core/machine.c | 1 +
  2 files changed, 2 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index a2d2f0e3bc..74cc81ae32 100644
@@ -2153,7 +2153,7 @@ static const Property smmuv3_properties[] = {
      /* GPA of MSI doorbell, for SMMUv3 accel use. */
      DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
      /* RIL can be turned off for accel cases */
-    DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_ON),
+    DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_AUTO),
      DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),
      DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_44),
      DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
hw/core/machine.c +0 -0
index 6796ab63cc..8f8c26bbe8 100644
@@ -41,6 +41,7 @@
 
Quoted 6 lines

hw/core/machine.c line 44

Quoted 3 lines
hw/core/machine.c +1 -0
  GlobalProperty hw_compat_11_0[] = {
      { TYPE_ARM_SMMUV3, "ats", "off" },
+    { TYPE_ARM_SMMUV3, "ril", "on" },

I wonder if you shouldn't squash all the patches that change the default values of accel props to auto mode into a single patch, at the end of the series.

Yes that sounds good to me. I will consolidate them in the next refresh.

Thanks,
Nathan

#27 lore raw

PATCH 7/11 hw/arm/smmuv3-accel: Implement "auto" value for "ssidsize"

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 SSID size property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving SSID size from IDR1. When the auto SSID size is resolved to a non-zero value, PASID capability is advertised to the vIOMMU and accelerated use cases such as Shared Virtual Addressing (SVA) are supported.

---
 hw/arm/smmuv3-accel.c | 14 ++++++++++++--
 hw/arm/smmuv3.c       | 11 +++++------
 2 files changed, 17 insertions(+), 8 deletions(-)
hw/arm/smmuv3-accel.c +12 -2
index 9e7ec6a930..03393fd7ba 100644
@@ -56,6 +56,13 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                FIELD_EX32(info->idr[3], IDR3, RIL));
     }
 
+    /* Update SSIDSIZE if auto from info */
+    if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        /* Store for get_viommu_flags() to determine PASID support */
+        s->idr[1] = FIELD_DP32(s->idr[1], IDR1, SSIDSIZE,
+                               FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
+    }
+
     accel->auto_finalised = true;
 }
 
@@ -828,7 +835,9 @@ static uint64_t smmuv3_accel_get_viommu_flags(void *opaque)
     SMMUState *bs = opaque;
     SMMUv3State *s = ARM_SMMUV3(bs);
 
-    if (s->ssidsize > SSID_SIZE_MODE_0) {
+    if (s->ssidsize > SSID_SIZE_MODE_0 ||
+        (s->ssidsize == SSID_SIZE_MODE_AUTO &&
+         FIELD_EX32(s->idr[1], IDR1, SSIDSIZE))) {
         flags |= VIOMMU_FLAG_PASID_SUPPORTED;
     }
     return flags;
@@ -952,7 +961,8 @@ void smmuv3_accel_init(SMMUv3State *s)
     smmuv3_accel_as_init(s);
 
     if (s->ats == ON_OFF_AUTO_AUTO ||
-        s->ril == ON_OFF_AUTO_AUTO) {
+        s->ril == ON_OFF_AUTO_AUTO ||
+        s->ssidsize == SSID_SIZE_MODE_AUTO) {
         s->s_accel->auto_mode = true;
     }
 }
hw/arm/smmuv3.c +5 -6
index 74cc81ae32..67c499d22b 100644
@@ -627,7 +627,10 @@ static int decode_ste(SMMUv3State *s, SMMUTransCfg *cfg,
     }
 
     /* Multiple context descriptors require SubstreamID support */
-    if (s->ssidsize == SSID_SIZE_MODE_0 && STE_S1CDMAX(ste) != 0) {
+    if ((s->ssidsize == SSID_SIZE_MODE_0 ||
+         (s->ssidsize == SSID_SIZE_MODE_AUTO &&
+          !FIELD_EX32(s->idr[1], IDR1, SSIDSIZE))) &&
+        STE_S1CDMAX(ste) != 0) {
         qemu_log_mask(LOG_UNIMP,
                 "SMMUv3: multiple S1 context descriptors require SubstreamID support. "
                 "Configure ssidsize > 0 (requires accel=on)\n");
@@ -1973,10 +1976,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
-        error_setg(errp, "ssidsize auto mode is not supported");
-        return false;
-    }
     if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
         error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48 bit"
                    "OAS; other OasMode values are not supported");
@@ -2197,7 +2196,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "A value of N allows SSIDs in the range [0 .. 2^N - 1]. "
         "Valid range is 0-20, where 0 disables SubstreamID support. "
         "Defaults to 0. A value greater than 0 is required to enable "
-        "PASID support. ssidsize=auto is not supported.");
+        "PASID support.");
 }
 
 static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
Signature · 1 line
2.43.0
#28 lore raw

Re: [PATCH 07/11] hw/arm/smmuv3-accel: Implement "auto" value for "ssidsize"

Quoted 34 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 SSID size property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving SSID size from IDR1. When the auto SSID size is resolved to a non-zero value, PASID capability is advertised to the vIOMMU and accelerated use cases such as Shared Virtual Addressing (SVA) are supported.

---
 hw/arm/smmuv3-accel.c | 14 ++++++++++++--
 hw/arm/smmuv3.c       | 11 +++++------
 2 files changed, 17 insertions(+), 8 deletions(-)
hw/arm/smmuv3-accel.c +7 -0
index 9e7ec6a930..03393fd7ba 100644
@@ -56,6 +56,13 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                FIELD_EX32(info->idr[3], IDR3, RIL));
     }
 
+    /* Update SSIDSIZE if auto from info */
+    if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        /* Store for get_viommu_flags() to determine PASID support */
+        s->idr[1] = FIELD_DP32(s->idr[1], IDR1, SSIDSIZE,
+                               FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
+    }
+
     accel->auto_finalised = true;
 }
 

hw/arm/smmuv3-accel.c static uint64_t smmuv3_accel_get_viommu_flags(void *opaque) · line 839

Quoted 6 lines
hw/arm/smmuv3-accel.c +2 -1
@@ -828,7 +835,9 @@ static uint64_t smmuv3_accel_get_viommu_flags(void *opaque)
     SMMUState *bs = opaque;
     SMMUv3State *s = ARM_SMMUV3(bs);
 
-    if (s->ssidsize > SSID_SIZE_MODE_0) {
+    if (s->ssidsize > SSID_SIZE_MODE_0 ||
+        (s->ssidsize == SSID_SIZE_MODE_AUTO &&

maybe worth adding a short static inline helper pasid_is_supported(s)?

Eric

Quoted 50 lines
hw/arm/smmuv3-accel.c +3 -1
+         FIELD_EX32(s->idr[1], IDR1, SSIDSIZE))) {
         flags |= VIOMMU_FLAG_PASID_SUPPORTED;
     }
     return flags;
@@ -952,7 +961,8 @@ void smmuv3_accel_init(SMMUv3State *s)
     smmuv3_accel_as_init(s);
 
     if (s->ats == ON_OFF_AUTO_AUTO ||
-        s->ril == ON_OFF_AUTO_AUTO) {
+        s->ril == ON_OFF_AUTO_AUTO ||
+        s->ssidsize == SSID_SIZE_MODE_AUTO) {
         s->s_accel->auto_mode = true;
     }
 }
hw/arm/smmuv3.c +5 -6
index 74cc81ae32..67c499d22b 100644
@@ -627,7 +627,10 @@ static int decode_ste(SMMUv3State *s, SMMUTransCfg *cfg,
     }
 
     /* Multiple context descriptors require SubstreamID support */
-    if (s->ssidsize == SSID_SIZE_MODE_0 && STE_S1CDMAX(ste) != 0) {
+    if ((s->ssidsize == SSID_SIZE_MODE_0 ||
+         (s->ssidsize == SSID_SIZE_MODE_AUTO &&
+          !FIELD_EX32(s->idr[1], IDR1, SSIDSIZE))) &&
+        STE_S1CDMAX(ste) != 0) {
         qemu_log_mask(LOG_UNIMP,
                 "SMMUv3: multiple S1 context descriptors require SubstreamID support. "
                 "Configure ssidsize > 0 (requires accel=on)\n");
@@ -1973,10 +1976,6 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
-        error_setg(errp, "ssidsize auto mode is not supported");
-        return false;
-    }
     if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
         error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48 bit"
                    "OAS; other OasMode values are not supported");
@@ -2197,7 +2196,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "A value of N allows SSIDs in the range [0 .. 2^N - 1]. "
         "Valid range is 0-20, where 0 disables SubstreamID support. "
         "Defaults to 0. A value greater than 0 is required to enable "
-        "PASID support. ssidsize=auto is not supported.");
+        "PASID support.");
 }
 
 static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
#29 lore raw

Re: [PATCH 07/11] hw/arm/smmuv3-accel: Implement "auto" value for "ssidsize"

Quoted 35 lines · Eric Auger
Quoted 33 lines · Nathan Chen

From: Nathan Chen<[email protected]>

Allow accelerated SMMUv3 SSID size property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving SSID size from IDR1. When the auto SSID size is resolved to a non-zero value, PASID capability is advertised to the vIOMMU and accelerated use cases such as Shared Virtual Addressing (SVA) are supported.

---
  hw/arm/smmuv3-accel.c | 14 ++++++++++++--
  hw/arm/smmuv3.c       | 11 +++++------
  2 files changed, 17 insertions(+), 8 deletions(-)
hw/arm/smmuv3-accel.c +7 -0
index 9e7ec6a930..03393fd7ba 100644
@@ -56,6 +56,13 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                 FIELD_EX32(info->idr[3], IDR3, RIL));
      }
 
+    /* Update SSIDSIZE if auto from info */
+    if (s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        /* Store for get_viommu_flags() to determine PASID support */
+        s->idr[1] = FIELD_DP32(s->idr[1], IDR1, SSIDSIZE,
+                               FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
+    }
+
      accel->auto_finalised = true;
  }
 
Quoted 8 lines

hw/arm/smmuv3-accel.c static uint64_t smmuv3_accel_get_viommu_flags(void *opaque) · line 839

Quoted 7 lines
hw/arm/smmuv3-accel.c +2 -1
@@ -828,7 +835,9 @@ static uint64_t smmuv3_accel_get_viommu_flags(void *opaque)
      SMMUState *bs = opaque;
      SMMUv3State *s = ARM_SMMUV3(bs);
 
-    if (s->ssidsize > SSID_SIZE_MODE_0) {
+    if (s->ssidsize > SSID_SIZE_MODE_0 ||
+        (s->ssidsize == SSID_SIZE_MODE_AUTO &&

maybe worth adding a short static inline helper pasid_is_supported(s)?

Yes, I will include that in the next refresh. It should look cleaner in smmuv3_accel_get_viommu_flags() that way.

Thanks,
Nathan

#30 lore raw

PATCH 8/11 hw/arm/smmuv3: Change the default ssidsize to match the host

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Set the default ssidsize value to auto. The default SSID size used to be 0, but we change it to match what the host IOMMU properties report so that users do not have to introspect host IDR1 for the Substream ID support.

---
 hw/arm/smmuv3.c   | 4 ++--
 hw/core/machine.c | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)
hw/arm/smmuv3.c +2 -2
index 67c499d22b..cb14a08a66 100644
@@ -2156,7 +2156,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),
     DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_44),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
-                              SSID_SIZE_MODE_0),
+                              SSID_SIZE_MODE_AUTO),
 };
 
 static void smmuv3_instance_init(Object *obj)
@@ -2195,7 +2195,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Number of bits used to represent SubstreamIDs (SSIDs). "
         "A value of N allows SSIDs in the range [0 .. 2^N - 1]. "
         "Valid range is 0-20, where 0 disables SubstreamID support. "
-        "Defaults to 0. A value greater than 0 is required to enable "
+        "Defaults to auto. A value greater than 0 is required to enable "
         "PASID support.");
 }
 
hw/core/machine.c +1 -0
index 8f8c26bbe8..aa208a2d9a 100644
@@ -42,6 +42,7 @@
 GlobalProperty hw_compat_11_0[] = {
     { TYPE_ARM_SMMUV3, "ats", "off" },
     { TYPE_ARM_SMMUV3, "ril", "on" },
+    { TYPE_ARM_SMMUV3, "ssidsize", "0" },
 };
 
 GlobalProperty hw_compat_10_2[] = {
Signature · 1 line
2.43.0
#31 lore raw

PATCH 9/11 hw/arm/smmuv3-accel: Implement "auto" value for "oas"

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 OAS property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving OAS from IDR5.

This keeps the OAS value advertised by the virtual SMMU compatible with the capabilities of the host SMMUv3, so that the intermediate physical addresses (IPA) consumed by host SMMU for stage-2 translation do not exceed the host's max supported IPA size.

---
 hw/arm/smmuv3-accel.c | 9 ++++++++-
 hw/arm/smmuv3.c       | 6 +++---
 2 files changed, 11 insertions(+), 4 deletions(-)
hw/arm/smmuv3-accel.c +8 -1
index 03393fd7ba..105f6d26bf 100644
@@ -63,6 +63,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
     }
 
+    /* Update OAS if auto from info */
+    if (s->oas == OAS_MODE_AUTO) {
+        s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS,
+                               FIELD_EX32(info->idr[5], IDR5, OAS));
+    }
+
     accel->auto_finalised = true;
 }
 
@@ -962,7 +968,8 @@ void smmuv3_accel_init(SMMUv3State *s)
 
     if (s->ats == ON_OFF_AUTO_AUTO ||
         s->ril == ON_OFF_AUTO_AUTO ||
-        s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        s->ssidsize == SSID_SIZE_MODE_AUTO ||
+        s->oas == OAS_MODE_AUTO) {
         s->s_accel->auto_mode = true;
     }
 }
hw/arm/smmuv3.c +3 -3
index cb14a08a66..0ab6d1e762 100644
@@ -1976,7 +1976,8 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
+    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48 &&
+        s->oas != OAS_MODE_AUTO) {
         error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48 bit"
                    "OAS; other OasMode values are not supported");
         return false;
@@ -2189,8 +2190,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "platform has ATS support before enabling this.");
     object_class_property_set_description(klass, "oas",
         "Specify Output Address Size (for accel=on). Supported values "
-        "are 44 or 48 bits. Defaults to 44 bits. oas=auto is not "
-        "supported.");
+        "are 44 or 48 bits. Defaults to 44 bits.");
     object_class_property_set_description(klass, "ssidsize",
         "Number of bits used to represent SubstreamIDs (SSIDs). "
         "A value of N allows SSIDs in the range [0 .. 2^N - 1]. "
Signature · 1 line
2.43.0
#32 lore raw

Re: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

Quoted 49 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 OAS property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving OAS from IDR5.

This keeps the OAS value advertised by the virtual SMMU compatible with the capabilities of the host SMMUv3, so that the intermediate physical addresses (IPA) consumed by host SMMU for stage-2 translation do not exceed the host's max supported IPA size.

---
 hw/arm/smmuv3-accel.c | 9 ++++++++-
 hw/arm/smmuv3.c       | 6 +++---
 2 files changed, 11 insertions(+), 4 deletions(-)
hw/arm/smmuv3-accel.c +8 -1
index 03393fd7ba..105f6d26bf 100644
@@ -63,6 +63,12 @@ static void smmuv3_accel_auto_finalise(SMMUv3State *s,
                                FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
     }
 
+    /* Update OAS if auto from info */
+    if (s->oas == OAS_MODE_AUTO) {
+        s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS,
+                               FIELD_EX32(info->idr[5], IDR5, OAS));
+    }
+
     accel->auto_finalised = true;
 }
 
@@ -962,7 +968,8 @@ void smmuv3_accel_init(SMMUv3State *s)
 
     if (s->ats == ON_OFF_AUTO_AUTO ||
         s->ril == ON_OFF_AUTO_AUTO ||
-        s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        s->ssidsize == SSID_SIZE_MODE_AUTO ||
+        s->oas == OAS_MODE_AUTO) {
         s->s_accel->auto_mode = true;
     }
 }
hw/arm/smmuv3.c +0 -0
index cb14a08a66..0ab6d1e762 100644

hw/arm/smmuv3.c static bool smmu_validate_property(SMMUv3State *s, Error **errp) · line 1982

Quoted 8 lines
hw/arm/smmuv3.c +2 -1
@@ -1976,7 +1976,8 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
+    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48 &&
+        s->oas != OAS_MODE_AUTO) {
         error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48 bit"
                    "OAS; other OasMode values are not supported");

and auto mode

hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2193

Quoted 8 lines
hw/arm/smmuv3.c +1 -2
         return false;
@@ -2189,8 +2190,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "platform has ATS support before enabling this.");
     object_class_property_set_description(klass, "oas",
         "Specify Output Address Size (for accel=on). Supported values "
-        "are 44 or 48 bits. Defaults to 44 bits. oas=auto is not "
-        "supported.");
+        "are 44 or 48 bits. Defaults to 44 bits.");

see previous comments

Eric

Quoted 3 lines
hw/arm/smmuv3.c +0 -0
     object_class_property_set_description(klass, "ssidsize",
         "Number of bits used to represent SubstreamIDs (SSIDs). "
         "A value of N allows SSIDs in the range [0 .. 2^N - 1]. "
#33 lore raw

RE: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

In reply to Nathan Chen
Quoted 77 lines

-----Original Message-----
From: Nathan Chen <[email protected]>
Sent: 01 April 2026 02:02
To: [email protected]; [email protected]
Cc: Yi Liu <[email protected]>; Eric Auger <[email protected]>; Zhenzhong Duan <[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang <[email protected]>; Zhao Liu <[email protected]>; Alex Williamson <[email protected]>; Cédric Le Goater <[email protected]>; Shameer Kolothum Thodi <[email protected]>; Matt Ochs <[email protected]>; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>
Subject: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

From: Nathan Chen <[email protected]>

Allow accelerated SMMUv3 OAS property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving OAS from IDR5.

This keeps the OAS value advertised by the virtual SMMU compatible with the capabilities of the host SMMUv3, so that the intermediate physical addresses (IPA) consumed by host SMMU for stage-2 translation do not exceed the host's max supported IPA size.

---
 hw/arm/smmuv3-accel.c | 9 ++++++++-
 hw/arm/smmuv3.c       | 6 +++---
 2 files changed, 11 insertions(+), 4 deletions(-)
hw/arm/smmuv3-accel.c +8 -1
index 03393fd7ba..105f6d26bf 100644
@@ -63,6 +63,12 @@ static void
 smmuv3_accel_auto_finalise(SMMUv3State *s,
                                FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
     }
 
+    /* Update OAS if auto from info */
+    if (s->oas == OAS_MODE_AUTO) {
+        s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS,
+                               FIELD_EX32(info->idr[5], IDR5, OAS));
+    }
+
     accel->auto_finalised = true;
 }
@@ -962,7 +968,8 @@ void smmuv3_accel_init(SMMUv3State *s)
 
     if (s->ats == ON_OFF_AUTO_AUTO ||
         s->ril == ON_OFF_AUTO_AUTO ||
-        s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        s->ssidsize == SSID_SIZE_MODE_AUTO ||
+        s->oas == OAS_MODE_AUTO) {
         s->s_accel->auto_mode = true;
     }
 }
hw/arm/smmuv3.c +2 -1
index cb14a08a66..0ab6d1e762 100644
@@ -1976,7 +1976,8 @@ static bool
 smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
-    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
+    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48 &&
+        s->oas != OAS_MODE_AUTO) {
         error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48 bit"
                    "OAS; other OasMode values are not supported");
Quoted 1 line
         return false;

Do we end up now allowing OAS_MODE_AUTO for non-accel mode?

I think we may need to adjust as below:

    if (!s->accel) {
         if (s->ril == ON_OFF_AUTO_OFF) {
              error_setg();
              return false;
          }
          if (s->ats != ON_OFF_AUTO_OFF) {
              error_setg();
              return false;
          }
          if (s->oas != OAS_MODE_44) {
              error_setg();
              return false;
          }
          if (s->ssidsize != SSID_SIZE_MODE_0) {
              error_setg();
              return false;
          }
          return true;
      }

Please check.

Thanks,
Shameer

#34 lore raw

Re: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

Quoted 101 lines · Shameer Kolothum Thodi
Quoted 76 lines

-----Original Message-----
From: Nathan Chen<[email protected]>
Sent: 01 April 2026 02:02 To:[email protected];[email protected]
Cc: Yi Liu<[email protected]>; Eric Auger<[email protected]>; Zhenzhong Duan<[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang<[email protected]>; Zhao Liu <[email protected]>; Alex Williamson<[email protected]>; Cédric Le Goater <[email protected]>; Shameer Kolothum Thodi<[email protected]>; Matt Ochs<[email protected]>; Nicolin Chen<[email protected]>; Nathan Chen<[email protected]>
Subject: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

From: Nathan Chen<[email protected]>

Allow accelerated SMMUv3 OAS property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving OAS from IDR5.

This keeps the OAS value advertised by the virtual SMMU compatible with the capabilities of the host SMMUv3, so that the intermediate physical addresses (IPA) consumed by host SMMU for stage-2 translation do not exceed the host's max supported IPA size.

---
  hw/arm/smmuv3-accel.c | 9 ++++++++-
  hw/arm/smmuv3.c       | 6 +++---
  2 files changed, 11 insertions(+), 4 deletions(-)
hw/arm/smmuv3-accel.c +8 -1
index 03393fd7ba..105f6d26bf 100644
@@ -63,6 +63,12 @@ static void
 smmuv3_accel_auto_finalise(SMMUv3State *s,
                                 FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
      }
 
+    /* Update OAS if auto from info */
+    if (s->oas == OAS_MODE_AUTO) {
+        s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS,
+                               FIELD_EX32(info->idr[5], IDR5, OAS));
+    }
+
      accel->auto_finalised = true;
  }
@@ -962,7 +968,8 @@ void smmuv3_accel_init(SMMUv3State *s)
 
      if (s->ats == ON_OFF_AUTO_AUTO ||
          s->ril == ON_OFF_AUTO_AUTO ||
-        s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        s->ssidsize == SSID_SIZE_MODE_AUTO ||
+        s->oas == OAS_MODE_AUTO) {
          s->s_accel->auto_mode = true;
      }
  }
hw/arm/smmuv3.c +2 -1
index cb14a08a66..0ab6d1e762 100644
@@ -1976,7 +1976,8 @@ static bool
 smmu_validate_property(SMMUv3State *s, Error **errp)
      }
  #endif
 
-    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
+    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48 &&
+        s->oas != OAS_MODE_AUTO) {
          error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48 bit"
                     "OAS; other OasMode values are not supported");
Quoted 1 line
          return false;

Do we end up now allowing OAS_MODE_AUTO for non-accel mode?

I think we may need to adjust as below:

     if (!s->accel) {
          if (s->ril == ON_OFF_AUTO_OFF) {
               error_setg();
               return false;
           }
           if (s->ats != ON_OFF_AUTO_OFF) {
               error_setg();
               return false;
           }
           if (s->oas != OAS_MODE_44) {
               error_setg();
               return false;
           }
           if (s->ssidsize != SSID_SIZE_MODE_0) {
               error_setg();
               return false;
           }
           return true;
       }
Quoted 1 line

Please check.

Yes, we need to adjust the check in smmu_validate_property() accordingly otherwise we can pass 'auto' in the non-accel case. I will include it on the next refresh.

Thanks,
Nathan

#35 lore raw

Re: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

In reply to Shameer Kolothum Thodi

Hi Shameer,

Quoted 101 lines · Shameer Kolothum Thodi
Quoted 76 lines

-----Original Message-----
From: Nathan Chen<[email protected]>
Sent: 01 April 2026 02:02 To:[email protected];[email protected]
Cc: Yi Liu<[email protected]>; Eric Auger<[email protected]>; Zhenzhong Duan<[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang<[email protected]>; Zhao Liu <[email protected]>; Alex Williamson<[email protected]>; Cédric Le Goater <[email protected]>; Shameer Kolothum Thodi<[email protected]>; Matt Ochs<[email protected]>; Nicolin Chen<[email protected]>; Nathan Chen<[email protected]>
Subject: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

From: Nathan Chen<[email protected]>

Allow accelerated SMMUv3 OAS property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving OAS from IDR5.

This keeps the OAS value advertised by the virtual SMMU compatible with the capabilities of the host SMMUv3, so that the intermediate physical addresses (IPA) consumed by host SMMU for stage-2 translation do not exceed the host's max supported IPA size.

---
  hw/arm/smmuv3-accel.c | 9 ++++++++-
  hw/arm/smmuv3.c       | 6 +++---
  2 files changed, 11 insertions(+), 4 deletions(-)
hw/arm/smmuv3-accel.c +8 -1
index 03393fd7ba..105f6d26bf 100644
@@ -63,6 +63,12 @@ static void
 smmuv3_accel_auto_finalise(SMMUv3State *s,
                                 FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
      }
 
+    /* Update OAS if auto from info */
+    if (s->oas == OAS_MODE_AUTO) {
+        s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS,
+                               FIELD_EX32(info->idr[5], IDR5, OAS));
+    }
+
      accel->auto_finalised = true;
  }
@@ -962,7 +968,8 @@ void smmuv3_accel_init(SMMUv3State *s)
 
      if (s->ats == ON_OFF_AUTO_AUTO ||
          s->ril == ON_OFF_AUTO_AUTO ||
-        s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        s->ssidsize == SSID_SIZE_MODE_AUTO ||
+        s->oas == OAS_MODE_AUTO) {
          s->s_accel->auto_mode = true;
      }
  }
hw/arm/smmuv3.c +2 -1
index cb14a08a66..0ab6d1e762 100644
@@ -1976,7 +1976,8 @@ static bool
 smmu_validate_property(SMMUv3State *s, Error **errp)
      }
  #endif
 
-    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
+    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48 &&
+        s->oas != OAS_MODE_AUTO) {
          error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48 bit"
                     "OAS; other OasMode values are not supported");
Quoted 1 line
          return false;

Do we end up now allowing OAS_MODE_AUTO for non-accel mode?

I think we may need to adjust as below:

     if (!s->accel) {
          if (s->ril == ON_OFF_AUTO_OFF) {
               error_setg();
               return false;
           }
           if (s->ats != ON_OFF_AUTO_OFF) {
               error_setg();
               return false;
           }
           if (s->oas != OAS_MODE_44) {
               error_setg();
               return false;
           }
           if (s->ssidsize != SSID_SIZE_MODE_0) {
               error_setg()
               return false;
           }
           return true;
       }
Quoted 1 line

Please check.

We do allow passing 'auto' if accel=off, but in that case smmuv3_accel_init() will not get called, so auto_mode = false and smmuv3_accel_auto_finalise() will not change the OAS.

If we adjust the check like above, users will have to manually set ril=on, ats=off, oas=44, ssidsize=0 when accel=off because the default is auto.

Would you prefer we do one of the following?
1. Add the adjustment above. Users have to manually set ril=on, ats=off, oas=44, ssidsize=0 when accel=off. Prevents setting auto for these properties when accel=off.

2. Add the adjustment above, but if accel=off and auto is set, resolve auto to previous defaults before we reach smmu_validate_property(). No functional change from current implementation.

3. Keep current implementation.

4. Add the adjustment above. Do not change defaults to auto. Users have to manually set auto for accel=on.

Thanks,
Nathan

#36 lore raw

RE: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

Quoted 126 lines

-----Original Message-----
From: Nathan Chen <[email protected]>
Sent: 20 April 2026 23:32
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: Yi Liu <[email protected]>; Eric Auger <[email protected]>; Zhenzhong Duan <[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang <[email protected]>; Zhao Liu <[email protected]>; Alex Williamson <[email protected]>; Cédric Le Goater <[email protected]>; Matt Ochs <[email protected]>; Nicolin Chen <[email protected]>
Subject: Re: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value for "oas"

Hi Shameer,

Quoted 11 lines · Shameer Kolothum Thodi
Quoted 11 lines

-----Original Message-----
From: Nathan Chen<[email protected]>
Sent: 01 April 2026 02:02 To:[email protected];[email protected]
Cc: Yi Liu<[email protected]>; Eric Auger<[email protected]>; Zhenzhong Duan<[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang<[email protected]>; Zhao Liu <[email protected]>; Alex Williamson<[email protected]>; Cédric Le

Goater

Quoted 1 line
Quoted 1 line

<[email protected]>; Shameer Kolothum

Thodi<[email protected]>;

Quoted 3 lines
Quoted 3 lines

Matt Ochs<[email protected]>; Nicolin Chen<[email protected]>; Nathan Chen<[email protected]>
Subject: [PATCH 09/11] hw/arm/smmuv3-accel: Implement "auto" value

for

Quoted 56 lines
Quoted 56 lines

"oas"

From: Nathan Chen<[email protected]>

Allow accelerated SMMUv3 OAS property to be derived from host IOMMU capabilities. Derive host values using IOMMU_GET_HW_INFO, retrieving OAS from IDR5.

This keeps the OAS value advertised by the virtual SMMU compatible with the capabilities of the host SMMUv3, so that the intermediate physical addresses (IPA) consumed by host SMMU for stage-2 translation do not exceed the host's max supported IPA size.

---
  hw/arm/smmuv3-accel.c | 9 ++++++++-
  hw/arm/smmuv3.c       | 6 +++---
  2 files changed, 11 insertions(+), 4 deletions(-)
hw/arm/smmuv3-accel.c +8 -1
index 03393fd7ba..105f6d26bf 100644
@@ -63,6 +63,12 @@ static void
 smmuv3_accel_auto_finalise(SMMUv3State *s,
                                 FIELD_EX32(info->idr[1], IDR1, SSIDSIZE));
      }
 
+    /* Update OAS if auto from info */
+    if (s->oas == OAS_MODE_AUTO) {
+        s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS,
+                               FIELD_EX32(info->idr[5], IDR5, OAS));
+    }
+
      accel->auto_finalised = true;
  }
@@ -962,7 +968,8 @@ void smmuv3_accel_init(SMMUv3State *s)
 
      if (s->ats == ON_OFF_AUTO_AUTO ||
          s->ril == ON_OFF_AUTO_AUTO ||
-        s->ssidsize == SSID_SIZE_MODE_AUTO) {
+        s->ssidsize == SSID_SIZE_MODE_AUTO ||
+        s->oas == OAS_MODE_AUTO) {
          s->s_accel->auto_mode = true;
      }
  }
hw/arm/smmuv3.c +0 -0
index cb14a08a66..0ab6d1e762 100644
@@ -1976,7 +1976,8 @@ static bool
 smmu_validate_property(SMMUv3State *s, Error **errp)
      }
  #endif
 
Quoted 4 lines

hw/arm/smmuv3.c static bool smmu_validate_property(SMMUv3State *s, Error **errp)

Quoted 4 lines
hw/arm/smmuv3.c +2 -1
-    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48) {
+    if (s->oas != OAS_MODE_44 && s->oas != OAS_MODE_48 &&
+        s->oas != OAS_MODE_AUTO) {
          error_setg(errp, "QEMU SMMUv3 model only implements 44 and 48

bit"

Quoted 26 lines
Quoted 2 lines
hw/arm/smmuv3.c +0 -0
                     "OAS; other OasMode values are not supported");
          return false;

Do we end up now allowing OAS_MODE_AUTO for non-accel mode?

I think we may need to adjust as below:

     if (!s->accel) {
          if (s->ril == ON_OFF_AUTO_OFF) {
               error_setg();
               return false;
           }
           if (s->ats != ON_OFF_AUTO_OFF) {
               error_setg();
               return false;
           }
           if (s->oas != OAS_MODE_44) {
               error_setg();
               return false;
           }
           if (s->ssidsize != SSID_SIZE_MODE_0) {
               error_setg()
               return false;
           }
           return true;
       }
Quoted 5 lines
Quoted 1 line

Please check.

We do allow passing 'auto' if accel=off, but in that case smmuv3_accel_init() will not get called, so auto_mode = false and smmuv3_accel_auto_finalise() will not change the OAS.

Ok.

Quoted 10 lines

If we adjust the check like above, users will have to manually set ril=on, ats=off, oas=44, ssidsize=0 when accel=off because the default is auto.

Would you prefer we do one of the following?
1. Add the adjustment above. Users have to manually set ril=on, ats=off, oas=44, ssidsize=0 when accel=off. Prevents setting auto for these properties when accel=off.

Quoted 5 lines

2. Add the adjustment above, but if accel=off and auto is set, resolve auto to previous defaults before we reach smmu_validate_property(). No functional change from current implementation.

3. Keep current implementation.

The problem I am seeing is we set the default for these properties as _AUTO but for non-accel case the default is actually something else.

I am fine to keep it as it is, but we need to capture this in commit log/comments and probably need mention in qemu-options.hx.

Thanks,
Shameer

#37 lore raw

PATCH 10/11 hw/arm/smmuv3: Change the default oas to match the host

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Set the default oas value to auto. The default Output Address Size used to be 44-bit, but we change it to match what the host IOMMU properties report so that users do not have to introspect host IDR5 for the OAS.

---
 hw/arm/smmuv3.c   | 4 ++--
 hw/core/machine.c | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)
hw/arm/smmuv3.c +2 -2
index 0ab6d1e762..7d537b7947 100644
@@ -2155,7 +2155,7 @@ static const Property smmuv3_properties[] = {
     /* RIL can be turned off for accel cases */
     DEFINE_PROP_ON_OFF_AUTO("ril", SMMUv3State, ril, ON_OFF_AUTO_AUTO),
     DEFINE_PROP_ON_OFF_AUTO("ats", SMMUv3State, ats, ON_OFF_AUTO_AUTO),
-    DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_44),
+    DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_AUTO),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
                               SSID_SIZE_MODE_AUTO),
 };
@@ -2190,7 +2190,7 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "platform has ATS support before enabling this.");
     object_class_property_set_description(klass, "oas",
         "Specify Output Address Size (for accel=on). Supported values "
-        "are 44 or 48 bits. Defaults to 44 bits.");
+        "are 44 or 48 bits. Defaults to auto.");
     object_class_property_set_description(klass, "ssidsize",
         "Number of bits used to represent SubstreamIDs (SSIDs). "
         "A value of N allows SSIDs in the range [0 .. 2^N - 1]. "
hw/core/machine.c +1 -0
index aa208a2d9a..a668bb2ec3 100644
@@ -43,6 +43,7 @@ GlobalProperty hw_compat_11_0[] = {
     { TYPE_ARM_SMMUV3, "ats", "off" },
     { TYPE_ARM_SMMUV3, "ril", "on" },
     { TYPE_ARM_SMMUV3, "ssidsize", "0" },
+    { TYPE_ARM_SMMUV3, "oas", "44" },
 };
 
 GlobalProperty hw_compat_10_2[] = {
Signature · 1 line
2.43.0
#38 lore raw

PATCH 11/11 qemu-options.hx: Support "auto" for accel SMMUv3 properties

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Update documentation now that "auto" is supported for accelerated SMMUv3 properties.

---
 qemu-options.hx | 1 -
 1 file changed, 1 deletion(-)
qemu-options.hx +0 -1
index 21972f8326..bd43258445 100644
@@ -1292,7 +1292,6 @@ SRST
         vfio-pci passthrough.
 
      The following options are available when accel=on.
-     Note: 'auto' mode is not currently supported.
 
     ``ril=on|off`` (default: on)
         Support for Range Invalidation, which allows the SMMUv3 driver to
Signature · 1 line
2.43.0
#39 lore raw

RE: [PATCH 11/11] qemu-options.hx: Support "auto" for accel SMMUv3 properties

Quoted 33 lines

-----Original Message-----
From: Nathan Chen <[email protected]>
Sent: 01 April 2026 02:03
To: [email protected]; [email protected]
Cc: Yi Liu <[email protected]>; Eric Auger <[email protected]>; Zhenzhong Duan <[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang <[email protected]>; Zhao Liu <[email protected]>; Alex Williamson <[email protected]>; Cédric Le Goater <[email protected]>; Shameer Kolothum Thodi <[email protected]>; Matt Ochs <[email protected]>; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>
Subject: [PATCH 11/11] qemu-options.hx: Support "auto" for accel SMMUv3 properties

From: Nathan Chen <[email protected]>

Update documentation now that "auto" is supported for accelerated SMMUv3 properties.

---
 qemu-options.hx | 1 -
 1 file changed, 1 deletion(-)
qemu-options.hx index renamed from qemu-options.hx +0 -0

21972f8326..bd43258445 100644

qemu-options.hx SRST · line 1297

Quoted 7 lines
qemu-options.hx +0 -1
@@ -1292,7 +1292,6 @@ SRST
         vfio-pci passthrough.
 
      The following options are available when accel=on.
-     Note: 'auto' mode is not currently supported.
 
     ``ril=on|off`` (default: on)
         Support for Range Invalidation, which allows the SMMUv3 driver to

default: auto ?

This applies to all other properties as well.

Also, it may be useful to clarify the "auto" behaviour, something like:

When set to auto, the value is derived from the host SMMUv3 capabilities via IOMMU_GET_HW_INFO. This requires at least one
cold-plugged vfio-pci device; if none is present at machine init,
QEMU will abort.

Thanks,
Shameer

#40 lore raw

Re: [PATCH 11/11] qemu-options.hx: Support "auto" for accel SMMUv3 properties

Quoted 41 lines · Shameer Kolothum Thodi
Quoted 32 lines

-----Original Message-----
From: Nathan Chen<[email protected]>
Sent: 01 April 2026 02:03 To:[email protected];[email protected]
Cc: Yi Liu<[email protected]>; Eric Auger<[email protected]>; Zhenzhong Duan<[email protected]>; Peter Maydell <[email protected]>; Marcel Apfelbaum <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Yanan Wang<[email protected]>; Zhao Liu <[email protected]>; Alex Williamson<[email protected]>; Cédric Le Goater <[email protected]>; Shameer Kolothum Thodi<[email protected]>; Matt Ochs<[email protected]>; Nicolin Chen<[email protected]>; Nathan Chen<[email protected]>
Subject: [PATCH 11/11] qemu-options.hx: Support "auto" for accel SMMUv3 properties

From: Nathan Chen<[email protected]>

Update documentation now that "auto" is supported for accelerated SMMUv3 properties.

---
  qemu-options.hx | 1 -
  1 file changed, 1 deletion(-)
qemu-options.hx index renamed from qemu-options.hx +0 -0

21972f8326..bd43258445 100644

qemu-options.hx SRST · line 1297

Quoted 7 lines
qemu-options.hx +0 -1
@@ -1292,7 +1292,6 @@ SRST
          vfio-pci passthrough.
 
       The following options are available when accel=on.
-     Note: 'auto' mode is not currently supported.
 
      ``ril=on|off`` (default: on)
          Support for Range Invalidation, which allows the SMMUv3 driver to

default: auto ?

Quoted 8 lines

This applies to all other properties as well.

Also, it may be useful to clarify the "auto" behaviour, something like:

When set to auto, the value is derived from the host SMMUv3 capabilities via IOMMU_GET_HW_INFO. This requires at least one
cold-plugged vfio-pci device; if none is present at machine init,
QEMU will abort.

I will update the default to auto and clarify the auto behavior, thanks for the suggestion.

Thanks,
Nathan