Skip to contentSkip to conversation

vfio/pci: add vfio-pci ATS control property

PATCH In review Nathan Chen 3 participants 6 messages Jun 23 – Jun 26

lore
Share & tools
lore
#0 lore raw

v4 cover vfio/pci: add vfio-pci ATS control property

Hi,

This is a follow-up to the series [0] that adds an ATS control property to vfio-pci and wires iommufd support reporting so vfio-pci can decide whether to expose the ATS PCIe extended capability to the guest.

The new vfio-pci property is:
ats=on|off|auto (default: auto)

Behavior:
- off: always masks ATS from guest view
- on: requires ATS support; fails realize if kernel reports no ATS support via IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
- auto: masks ATS only when kernel reports no ATS support via IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED

If the physical device does not have ATS, the ATS cap will not be present for the device in the guest, and a warning will be emitted when ats is set to on.

We intentionally omit a legacy ats=on compat override because forcing the value to "on" on old machine types can turn a host-kernel capability update (IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED) into a new realize-time boot failure. So we keep the uniform default ats=auto to preserve bootability and follow effective kernel ATS support rather than preserving the old behavior of always advertising ATS when available.

Please note that no linux-headers sync patch is included in this series because IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED is already present in linux-headers/linux/iommufd.h in the current base.

Testing:
Verified ATS capability visibility changes with vfio-pci ats=off/on/auto on an NVIDIA Grace system:
- ats=on + ATS supported by kernel => propagate ATS extended cap to guest view
- ats=auto + ATS supported by kernel => propagate ATS extended cap to guest view
- ats=on + hiod_iommufd_support_ats modified to return false (mimic kernel-reported unsupported ATS) => realize failure
- ats=auto + hiod_iommufd_support_ats modified to return false (mimic kernel-reported unsupported ATS) => ATS masked
- ats=off => ATS masked
- ats=on + no ATS support in physical device => warning + no ATS ext cap

Changes from v3:
- Move warning for missing kernel ATS support when ats=auto to after checking the presence of host device ATS cap. Only emit the warning about masking ATS cap when ATS cap is present on the host device.
- Emit warning about no ATS support in physical device only when ats=on instead of both ats=auto and ats=on cases (via ats_needed).

Changes from v2:
- Added R-by tags, thanks all!
- Fixed alignment of two object_class_property_set_description() blocks in vfio_pci_properties[]
- Removed compat override setting ats=on and added a warning to the case where ats=auto and kernel reports ATS is not supported.
- Squash patch for warning when ats=on and there is no ATS support in physical device into patch #2
- Emit warning when ats=auto and no ATS support in physical device

Changes from v1:
- Set ats_needed to false by default in vfio_pci_ats_requested_and_supported()
- Revise log prefix from "vfio" to "vfio-pci" in vfio_pci_ats_requested_and_supported()
- Squash compat update patch into patch for introducing ats property for bisectibility
- Revise comments and descriptions for support_ats(), vfio_pci_ats_requested_and_supported(), and vfio_pci_class_init() to be more generic, removing references to IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
- Emit a warning when ats=on and no ATS support in physical device

Changes from "Resolve AUTO properties" series:
- Split into separate patches for support_ats() handler introduction and iommufd implementation, "ats" property addition, and compat property fix
- Rebase on vfio-next
- Remove unnecessary includes for hw/vfio/pci.c
- Rename ats_need to ats_needed for vfio_pci_ats_requested_and_supported()
- Move the vfio_pci_ats_requested_and_supported() call to vfio_pci_add_capabilities() so we can propagate the Error **, and then pass ats_needed as a new input arg to vfio_add_ext_cap
- Fix indent error in vfio_add_ext_cap()
- Break and add a pcie_add_capability() call for the PASID case in vfio_add_ext_cap()
- Update comment about return value for support_ats()

Thanks,
Nathan

[0] https://lore.kernel.org/qemu-devel/[email protected]/

Nathan Chen (2):
iommufd: Introduce handler for device ATS support vfio/pci: Add ats property

 hw/vfio/pci.h                      |  1 +
 include/system/host_iommu_device.h |  9 +++
 backends/iommufd.c                 |  8 +++
 hw/vfio/pci.c                      | 88 ++++++++++++++++++++++++++++--
 4 files changed, 102 insertions(+), 4 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v4 1/2 iommufd: Introduce handler for device ATS support

From: Nathan Chen <[email protected]>

Introduce a support_ats() handler that returns whether ATS is supported for a device associated with a host IOMMU device, checking if the IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED capability bit is set.

Implement this handler in a new function for iommufd.

---
 include/system/host_iommu_device.h | 9 +++++++++
 backends/iommufd.c                 | 8 ++++++++
 2 files changed, 17 insertions(+)
include/system/host_iommu_device.h +9 -0
index f000301583..2cb5365340 100644
@@ -133,6 +133,15 @@ struct HostIOMMUDeviceClass {
      * Returns: true on success, false on failure.
      */
     bool (*get_pasid_info)(HostIOMMUDevice *hiod, PasidInfo *pasid_info);
+    /**
+     *  @support_ats: Returns true if ATS can be used by the device,
+     *  false if the host IOMMU reports it is unavailable.
+     *
+     *  @hiod: handle to the host IOMMU device
+     *
+     *  Returns: true if ATS is supported, false otherwise
+     */
+    bool (*support_ats)(HostIOMMUDevice *hiod);
 };
 
 /*
backends/iommufd.c +8 -0
index 09624cd652..4b40aaebe6 100644
@@ -638,6 +638,13 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp)
     }
 }
 
+static bool hiod_iommufd_support_ats(HostIOMMUDevice *hiod)
+{
+    HostIOMMUDeviceCaps *caps = &hiod->caps;
+
+    return !(caps->hw_caps & IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED);
+}
+
 static bool hiod_iommufd_get_pasid_info(HostIOMMUDevice *hiod,
                                         PasidInfo *pasid_info)
 {
@@ -660,6 +667,7 @@ static void hiod_iommufd_class_init(ObjectClass *oc, const void *data)
 
     hiodc->get_cap = hiod_iommufd_get_cap;
     hiodc->get_pasid_info = hiod_iommufd_get_pasid_info;
+    hiodc->support_ats = hiod_iommufd_support_ats;
 };
 
 static const TypeInfo types[] = {
Signature · 1 line
2.43.0
#2 lore raw

PATCH v4 2/2 vfio/pci: Add ats property

In reply to Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

If ATS is explicitly requested but not supported by the kernel, fail device realize.

This aligns with the kernel's per-device effective ATS reporting and allows vfio-pci to mask ATS when the host kernel reports ATS as unsupported.

Emit a warning when ats=on is requested but the physical device does not advertise ATS, since ATS cannot be exposed to the guest in this case.

Emit a warning when ats=auto, ats cap is present on the physical device, but kernel reports ATS as unsupported.

---
 hw/vfio/pci.h |  1 +
 hw/vfio/pci.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 85 insertions(+), 4 deletions(-)
hw/vfio/pci.h +1 -0
index c3a1f53d35..f2934f2d84 100644
@@ -188,6 +188,7 @@ struct VFIOPCIDevice {
     bool clear_parent_atomics_on_exit;
     bool skip_vsc_check;
     uint16_t vpasid_cap_offset;
+    OnOffAuto ats;
     VFIODisplay *dpy;
     Notifier irqchip_change_notifier;
     VFIOPCICPR cpr;
hw/vfio/pci.c +84 -4

139 lines not on this page · raw message

Signature · 1 line
2.43.0
Reviewed-by on v4 2/2
#3 lore raw

Re: [PATCH v4 2/2] vfio/pci: Add ats property

Quoted 178 lines · Nathan Chen

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask the capability so the guest does not see it.

If ATS is explicitly requested but not supported by the kernel, fail device realize.

This aligns with the kernel's per-device effective ATS reporting and allows vfio-pci to mask ATS when the host kernel reports ATS as unsupported.

Emit a warning when ats=on is requested but the physical device does not advertise ATS, since ATS cannot be exposed to the guest in this case.

Emit a warning when ats=auto, ats cap is present on the physical device, but kernel reports ATS as unsupported.

---
  hw/vfio/pci.h |  1 +
  hw/vfio/pci.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++---
  2 files changed, 85 insertions(+), 4 deletions(-)
hw/vfio/pci.h +1 -0
index c3a1f53d35..f2934f2d84 100644
@@ -188,6 +188,7 @@ struct VFIOPCIDevice {
      bool clear_parent_atomics_on_exit;
      bool skip_vsc_check;
      uint16_t vpasid_cap_offset;
+    OnOffAuto ats;
      VFIODisplay *dpy;
      Notifier irqchip_change_notifier;
      VFIOPCICPR cpr;
hw/vfio/pci.c +77 -4
index 9c06b25e63..fbec41f0f5 100644
@@ -2546,10 +2546,53 @@ static bool vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp)
      return true;
  }
 
-static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
+/*
+ * Determine whether ATS capability should be advertised for @vdev, based on
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel.
+ *
+ * Store whether ATS capability should be advertised in @ats_needed.
+ *
+ * Returns false only when ats=on is explicitly requested but the kernel
+ * reports it is not supported. Returns true in all other cases.
+ */
+static bool vfio_pci_ats_requested_and_supported(VFIOPCIDevice *vdev,
+                                                 bool *ats_needed, Error **errp)
+{
+    HostIOMMUDevice *hiod = vdev->vbasedev.hiod;
+    HostIOMMUDeviceClass *hiodc;
+    bool ats_supported;
+    *ats_needed = false;
+
+    if (vdev->ats == ON_OFF_AUTO_OFF) {
+        return true;
+    }
+
+    *ats_needed = true;
+    if (!hiod) {
+        return true;
+    }
+    hiodc = HOST_IOMMU_DEVICE_GET_CLASS(hiod);
+    if (!hiodc || !hiodc->support_ats) {
+        return true;
+    }
+
+    ats_supported = hiodc->support_ats(hiod);
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_supported) {
+        error_setg(errp, "vfio-pci: ATS requested but not supported by kernel");
+        *ats_needed = false;
+        return false;
+    }
+
+    *ats_needed = ats_supported;
+    return true;
+}
+
+static void vfio_add_ext_cap(VFIOPCIDevice *vdev, bool ats_needed)
  {
      PCIDevice *pdev = PCI_DEVICE(vdev);
      bool pasid_cap_added = false;
+    bool ats_cap_present = false;
      Error *err = NULL;
      uint32_t header;
      uint16_t cap_id, next, size;
@@ -2635,7 +2678,19 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
           */
          case PCI_EXT_CAP_ID_PASID:
              pasid_cap_added = true;
-            /* fallthrough */
+            pcie_add_capability(pdev, cap_id, cap_ver, next, size);
+            break;
+        case PCI_EXT_CAP_ID_ATS:
+            ats_cap_present = true;
+            /*
+             * If ATS is requested and supported according to the kernel, add
+             * the ATS capability. If not supported according to the kernel or
+             * disabled on the qemu command line, omit the ATS cap.
+             */
+            if (ats_needed) {
+                pcie_add_capability(pdev, cap_id, cap_ver, next, size);
+            }
+            break;
          default:
              pcie_add_capability(pdev, cap_id, cap_ver, next, size);
          }
@@ -2646,6 +2701,16 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
          error_report_err(err);
      }
 
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_cap_present) {
+        warn_report("vfio-pci: ats=on requested, but host device has no "
+                    "ATS extended capability");
+    }
+
+    if (vdev->ats == ON_OFF_AUTO_AUTO && ats_cap_present && !ats_needed) {
+        warn_report("vfio-pci: host kernel reports ATS unsupported; "
+                    "ATS capability will be masked");
+    }
+
      /* Cleanup chain head ID if necessary */
      if (pci_get_word(pdev->config + PCI_CONFIG_SPACE_SIZE) == 0xFFFF) {
          pci_set_word(pdev->config + PCI_CONFIG_SPACE_SIZE, 0);
@@ -2657,6 +2722,7 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
  bool vfio_pci_add_capabilities(VFIOPCIDevice *vdev, Error **errp)
  {
      PCIDevice *pdev = PCI_DEVICE(vdev);
+    bool ats_needed = false;
 
      if (!(pdev->config[PCI_STATUS] & PCI_STATUS_CAP_LIST) ||
          !pdev->config[PCI_CAPABILITY_LIST]) {
@@ -2667,7 +2733,11 @@ bool vfio_pci_add_capabilities(VFIOPCIDevice *vdev, Error **errp)
          return false;
      }
 
-    vfio_add_ext_cap(vdev);
+    if (!vfio_pci_ats_requested_and_supported(vdev, &ats_needed, errp)) {
+        return false;
+    }
+
+    vfio_add_ext_cap(vdev, ats_needed);
      return true;
  }
 
@@ -3815,6 +3885,7 @@ static const Property vfio_pci_properties[] = {
      DEFINE_PROP_BOOL("skip-vsc-check", VFIOPCIDevice, skip_vsc_check, true),
      DEFINE_PROP_UINT16("x-vpasid-cap-offset", VFIOPCIDevice,
                         vpasid_cap_offset, 0),
+    DEFINE_PROP_ON_OFF_AUTO("ats", VFIOPCIDevice, ats, ON_OFF_AUTO_AUTO),
  };
 
  static void vfio_pci_set_fd(Object *obj, const char *str, Error **errp)
@@ -3966,13 +4037,22 @@ static void vfio_pci_class_init(ObjectClass *klass, const void *data)
                                            "destination when doing live "
                                            "migration of device state via "
                                            "multifd channels");
-   object_class_property_set_description(klass, /* 11.0 */
+    object_class_property_set_description(klass, /* 11.0 */
                                            "x-vpasid-cap-offset",
                                            "PCIe extended configuration space offset at which to place a "
                                            "synthetic PASID extended capability when PASID is enabled via "
                                            "a vIOMMU. A value of 0 (default) places the capability at the "
                                            "end of the extended configuration space. The offset must be "
                                            "4-byte aligned and within the PCIe extended configuration space");
+    object_class_property_set_description(klass, /* 11.1 */
+                                          "ats",

hw/vfio/pci.c static void vfio_pci_class_init(ObjectClass *klass, const void *data) · line 4058

Quoted 10 lines
hw/vfio/pci.c +7 -0
+                                          "Control guest visibility of the ATS PCIe extended capability. "
+                                          "Valid values are on, off, and auto (default). "
+                                          "'off' always masks ATS. "
+                                          "'on' requires ATS support for the device and fails realize if the "
+                                          "host kernel reports ATS as unavailable for this device. "
+                                          "'auto' masks ATS only when the host kernel reports "
+                                          "ATS as unavailable");
  }
 
  static const TypeInfo vfio_pci_info = {

Thanks,

C.

Reviewed-by on v4 2/2
#4 lore raw

RE: [PATCH v4 2/2] vfio/pci: Add ats property

In reply to Nathan Chen
Quoted 28 lines

-----Original Message-----
From: Nathan Chen <[email protected]>
Sent: 23 June 2026 21:50
To: [email protected]
Cc: Yi Liu <[email protected]>; Eric Auger <[email protected]>; Zhenzhong Duan <[email protected]>; Alex Williamson <[email protected]>; Cédric Le Goater <[email protected]>; Matt Ochs <[email protected]>; Nicolin Chen <[email protected]>; Shameer Kolothum Thodi <[email protected]>; Nathan Chen <[email protected]>
Subject: [PATCH v4 2/2] vfio/pci: Add ats property

From: Nathan Chen <[email protected]>

Add an "ats" OnOffAuto property to vfio-pci. When the device has an ATS extended capability in config space but we should not expose it (ats=off, or ats=auto and kernel reports IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED), mask
the capability so the guest does not see it.

If ATS is explicitly requested but not supported by the kernel, fail device realize.

This aligns with the kernel's per-device effective ATS reporting and allows vfio-pci to mask ATS when the host kernel reports ATS as unsupported.

Quoted 8 lines

Emit a warning when ats=on is requested but the physical device does not advertise ATS, since ATS cannot be exposed to the guest in this case.

Emit a warning when ats=auto, ats cap is present on the physical device, but kernel reports ATS as unsupported.

Thanks,
Shameer

Quoted 187 lines
---
 hw/vfio/pci.h |  1 +
 hw/vfio/pci.c | 88
++++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 85 insertions(+), 4 deletions(-)
hw/vfio/pci.h +1 -0
index c3a1f53d35..f2934f2d84 100644
@@ -188,6 +188,7 @@ struct VFIOPCIDevice {
     bool clear_parent_atomics_on_exit;
     bool skip_vsc_check;
     uint16_t vpasid_cap_offset;
+    OnOffAuto ats;
     VFIODisplay *dpy;
     Notifier irqchip_change_notifier;
     VFIOPCICPR cpr;
hw/vfio/pci.c +2 -1
index 9c06b25e63..fbec41f0f5 100644
@@ -2546,10 +2546,53 @@ static bool
 vfio_pci_synthesize_pasid_cap(VFIOPCIDevice *vdev, Error **errp)
     return true;
 }
 
-static void vfio_add_ext_cap(VFIOPCIDevice *vdev)
+/*
+ * Determine whether ATS capability should be advertised for @vdev, based

on
+ * whether it was enabled on the command line and whether it is supported
+ * according to the kernel.
+ *
+ * Store whether ATS capability should be advertised in @ats_needed.
+ *
+ * Returns false only when ats=on is explicitly requested but the kernel
+ * reports it is not supported. Returns true in all other cases.
+ */

hw/vfio/pci.c +47 -1
+static bool vfio_pci_ats_requested_and_supported(VFIOPCIDevice *vdev,
+                                                 bool *ats_needed, Error **errp)
+{
+    HostIOMMUDevice *hiod = vdev->vbasedev.hiod;
+    HostIOMMUDeviceClass *hiodc;
+    bool ats_supported;
+    *ats_needed = false;
+
+    if (vdev->ats == ON_OFF_AUTO_OFF) {
+        return true;
+    }
+
+    *ats_needed = true;
+    if (!hiod) {
+        return true;
+    }
+    hiodc = HOST_IOMMU_DEVICE_GET_CLASS(hiod);
+    if (!hiodc || !hiodc->support_ats) {
+        return true;
+    }
+
+    ats_supported = hiodc->support_ats(hiod);
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_supported) {
+        error_setg(errp, "vfio-pci: ATS requested but not supported by kernel");
+        *ats_needed = false;
+        return false;
+    }
+
+    *ats_needed = ats_supported;
+    return true;
+}
+
+static void vfio_add_ext_cap(VFIOPCIDevice *vdev, bool ats_needed)
 {
     PCIDevice *pdev = PCI_DEVICE(vdev);
     bool pasid_cap_added = false;
+    bool ats_cap_present = false;
     Error *err = NULL;
     uint32_t header;
     uint16_t cap_id, next, size;
@@ -2635,7 +2678,19 @@ static void vfio_add_ext_cap(VFIOPCIDevice
 *vdev)
          */
         case PCI_EXT_CAP_ID_PASID:
             pasid_cap_added = true;
-            /* fallthrough */
+            pcie_add_capability(pdev, cap_id, cap_ver, next, size);
+            break;
+        case PCI_EXT_CAP_ID_ATS:
+            ats_cap_present = true;
+            /*
+             * If ATS is requested and supported according to the kernel, add
+             * the ATS capability. If not supported according to the kernel or
+             * disabled on the qemu command line, omit the ATS cap.
+             */
+            if (ats_needed) {
+                pcie_add_capability(pdev, cap_id, cap_ver, next, size);
+            }
+            break;
         default:
             pcie_add_capability(pdev, cap_id, cap_ver, next, size);
         }
hw/vfio/pci.c +10 -0
@@ -2646,6 +2701,16 @@ static void vfio_add_ext_cap(VFIOPCIDevice
 *vdev)
         error_report_err(err);
     }
 
+    if (vdev->ats == ON_OFF_AUTO_ON && !ats_cap_present) {
+        warn_report("vfio-pci: ats=on requested, but host device has no "
+                    "ATS extended capability");
+    }
+
+    if (vdev->ats == ON_OFF_AUTO_AUTO && ats_cap_present &&
 !ats_needed) {
+        warn_report("vfio-pci: host kernel reports ATS unsupported; "
+                    "ATS capability will be masked");
+    }
+
     /* Cleanup chain head ID if necessary */
     if (pci_get_word(pdev->config + PCI_CONFIG_SPACE_SIZE) == 0xFFFF) {
         pci_set_word(pdev->config + PCI_CONFIG_SPACE_SIZE, 0);
hw/vfio/pci.c +1 -0
@@ -2657,6 +2722,7 @@ static void vfio_add_ext_cap(VFIOPCIDevice
 *vdev)
 bool vfio_pci_add_capabilities(VFIOPCIDevice *vdev, Error **errp)
 {
     PCIDevice *pdev = PCI_DEVICE(vdev);
+    bool ats_needed = false;
 
     if (!(pdev->config[PCI_STATUS] & PCI_STATUS_CAP_LIST) ||
         !pdev->config[PCI_CAPABILITY_LIST]) {
hw/vfio/pci.c +6 -1
@@ -2667,7 +2733,11 @@ bool vfio_pci_add_capabilities(VFIOPCIDevice
 *vdev, Error **errp)
         return false;
     }
 
-    vfio_add_ext_cap(vdev);
+    if (!vfio_pci_ats_requested_and_supported(vdev, &ats_needed, errp)) {
+        return false;
+    }
+
+    vfio_add_ext_cap(vdev, ats_needed);
     return true;
 }
@@ -3815,6 +3885,7 @@ static const Property vfio_pci_properties[] = {
     DEFINE_PROP_BOOL("skip-vsc-check", VFIOPCIDevice, skip_vsc_check,
 true),
     DEFINE_PROP_UINT16("x-vpasid-cap-offset", VFIOPCIDevice,
                        vpasid_cap_offset, 0),
+    DEFINE_PROP_ON_OFF_AUTO("ats", VFIOPCIDevice, ats,
 ON_OFF_AUTO_AUTO),
 };

static void vfio_pci_set_fd(Object *obj, const char *str, Error **errp)

hw/vfio/pci.c +1 -1
@@ -3966,13 +4037,22 @@ static void vfio_pci_class_init(ObjectClass *klass,
 const void *data)
                                           "destination when doing live "
                                           "migration of device state via "
                                           "multifd channels");
-   object_class_property_set_description(klass, /* 11.0 */
+    object_class_property_set_description(klass, /* 11.0 */
                                           "x-vpasid-cap-offset",
                                           "PCIe extended configuration space offset at which to

place a "
"synthetic PASID extended capability when PASID is
enabled via "
"a vIOMMU. A value of 0 (default) places the capability
at the "
"end of the extended configuration space. The offset
must be "
"4-byte aligned and within the PCIe extended
configuration space");

hw/vfio/pci.c +3 -0
+    object_class_property_set_description(klass, /* 11.1 */
+                                          "ats",
+                                          "Control guest visibility of the ATS PCIe extended

capability. "

hw/vfio/pci.c +3 -0
+                                          "Valid values are on, off, and auto (default). "
+                                          "'off' always masks ATS. "
+                                          "'on' requires ATS support for the device and fails realize

if the "

hw/vfio/pci.c +3 -1
+                                          "host kernel reports ATS as unavailable for this device. "
+                                          "'auto' masks ATS only when the host kernel reports "
+                                          "ATS as unavailable");
 }
 
 static const TypeInfo vfio_pci_info = {
--

2.43.0

#5 lore raw
In reply to Nathan Chen
Quoted 98 lines · Nathan Chen

Hi,

This is a follow-up to the series [0] that adds an ATS control property to vfio-pci and wires iommufd support reporting so vfio-pci can decide whether to expose the ATS PCIe extended capability to the guest.

The new vfio-pci property is:
ats=on|off|auto (default: auto)

Behavior:
- off: always masks ATS from guest view
- on: requires ATS support; fails realize if kernel reports no ATS support via IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
- auto: masks ATS only when kernel reports no ATS support via IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED

If the physical device does not have ATS, the ATS cap will not be present for the device in the guest, and a warning will be emitted when ats is set to on.

We intentionally omit a legacy ats=on compat override because forcing the value to "on" on old machine types can turn a host-kernel capability update (IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED) into a new realize-time boot failure. So we keep the uniform default ats=auto to preserve bootability and follow effective kernel ATS support rather than preserving the old behavior of always advertising ATS when available.

Please note that no linux-headers sync patch is included in this series because IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED is already present in linux-headers/linux/iommufd.h in the current base.

Testing:
Verified ATS capability visibility changes with vfio-pci ats=off/on/auto on an NVIDIA Grace system:
- ats=on + ATS supported by kernel => propagate ATS extended cap to guest view
- ats=auto + ATS supported by kernel => propagate ATS extended cap to guest view
- ats=on + hiod_iommufd_support_ats modified to return false (mimic kernel-reported unsupported ATS) => realize failure
- ats=auto + hiod_iommufd_support_ats modified to return false (mimic kernel-reported unsupported ATS) => ATS masked
- ats=off => ATS masked
- ats=on + no ATS support in physical device => warning + no ATS ext cap

Changes from v3:
- Move warning for missing kernel ATS support when ats=auto to after checking the presence of host device ATS cap. Only emit the warning about masking ATS cap when ATS cap is present on the host device.
- Emit warning about no ATS support in physical device only when ats=on instead of both ats=auto and ats=on cases (via ats_needed).

Changes from v2:
- Added R-by tags, thanks all!
- Fixed alignment of two object_class_property_set_description() blocks in vfio_pci_properties[]
- Removed compat override setting ats=on and added a warning to the case where ats=auto and kernel reports ATS is not supported.
- Squash patch for warning when ats=on and there is no ATS support in physical device into patch #2
- Emit warning when ats=auto and no ATS support in physical device

Changes from v1:
- Set ats_needed to false by default in vfio_pci_ats_requested_and_supported()
- Revise log prefix from "vfio" to "vfio-pci" in vfio_pci_ats_requested_and_supported()
- Squash compat update patch into patch for introducing ats property for bisectibility
- Revise comments and descriptions for support_ats(), vfio_pci_ats_requested_and_supported(), and vfio_pci_class_init() to be more generic, removing references to IOMMU_HW_CAP_PCI_ATS_NOT_SUPPORTED
- Emit a warning when ats=on and no ATS support in physical device

Changes from "Resolve AUTO properties" series:
- Split into separate patches for support_ats() handler introduction and iommufd implementation, "ats" property addition, and compat property fix
- Rebase on vfio-next
- Remove unnecessary includes for hw/vfio/pci.c
- Rename ats_need to ats_needed for vfio_pci_ats_requested_and_supported()
- Move the vfio_pci_ats_requested_and_supported() call to vfio_pci_add_capabilities() so we can propagate the Error **, and then pass ats_needed as a new input arg to vfio_add_ext_cap
- Fix indent error in vfio_add_ext_cap()
- Break and add a pcie_add_capability() call for the PASID case in vfio_add_ext_cap()
- Update comment about return value for support_ats()

Thanks,
Nathan

[0] https://lore.kernel.org/qemu-devel/[email protected]/

Nathan Chen (2):
iommufd: Introduce handler for device ATS support vfio/pci: Add ats property

Quoted 6 lines
  hw/vfio/pci.h                      |  1 +
  include/system/host_iommu_device.h |  9 +++
  backends/iommufd.c                 |  8 +++
  hw/vfio/pci.c                      | 88 ++++++++++++++++++++++++++++--
  4 files changed, 102 insertions(+), 4 deletions(-)

Applied to

https://github.com/legoater/qemu vfio-next

Thanks,

C.