VIRTIO-IOMMU/VFIO: Fix host iommu geometry handling for hotplugged devices
v3 cover VIRTIO-IOMMU/VFIO: Fix host iommu geometry handling for hotplugged devices
This series is based on Zhenzhong HostIOMMUDevice:
[PATCH v7 00/17] Add a host IOMMU device abstraction to check with vIOMMU
https://lore.kernel.org/all/[email protected]/
It allows to convey host IOVA reserved regions to the virtio-iommu and uses the HostIOMMUDevice infrastructure. This replaces the usage of IOMMU MR ops which fail to satisfy this need for hotplugged devices.
See below for additional background.
In [1] we attempted to fix a case where a VFIO-PCI device protected with a virtio-iommu was assigned to an x86 guest. On x86 the physical IOMMU may have an address width (gaw) of 39 or 48 bits whereas the virtio-iommu used to expose a 64b address space by default. Hence the guest was trying to use the full 64b space and we hit DMA MAP failures. To work around this issue we managed to pass usable IOVA regions (excluding the out of range space) from VFIO to the virtio-iommu device. This was made feasible by introducing a new IOMMU Memory Region callback dubbed iommu_set_iova_regions(). This latter gets called when the IOMMU MR is enabled which causes the vfio_listener_region_add() to be called.
For coldplugged devices the technique works because we make sure all the IOMMU MR are enabled once on the machine init done: 94df5b2180 ("virtio-iommu: Fix 64kB host page size VFIO device assignment") for granule freeze. But I would be keen to get rid of this trick.
However with VFIO-PCI hotplug, this technique fails due to the race between the call to the callback in the add memory listener and the virtio-iommu probe request. Indeed the probe request gets called before the attach to the domain. So in that case the usable regions are communicated after the probe request and fail to be conveyed to the guest.
Using an IOMMU MR Ops is unpractical because this relies on the IOMMU MR to have been enabled and the corresponding vfio_listener_region_add() to be executed. Instead this series proposes to replace the usage of this API by the recently introduced PCIIOMMUOps: ba7d12eb8c ("hw/pci: modify pci_setup_iommu() to set PCIIOMMUOps"). That way, the callback can be called earlier, once the usable IOVA regions have been collected by VFIO, without the need for the IOMMU MR to be enabled.
This series also removes the spurious message:
qemu-system-aarch64: warning: virtio-iommu-memory-region-7-0: Notified about new host reserved regions after probe
In the short term this may also be used for passing the page size mask, which would allow to get rid of the hacky transient IOMMU MR enablement mentionned above.
[1] [PATCH v4 00/12] VIRTIO-IOMMU/VFIO: Don't assume 64b IOVA space
https://lore.kernel.org/all/[email protected]/
Extra Notes:
With that series, the reserved memory regions are communicated on time so that the virtio-iommu probe request grabs them. However this is not sufficient. In some cases (my case), I still see some DMA MAP failures and the guest keeps on using IOVA ranges outside the geometry of the physical IOMMU. This is due to the fact the VFIO-PCI device is in the same iommu group as the pcie root port. Normally the kernel iova_reserve_iommu_regions (dma-iommu.c) is supposed to call reserve_iova() for each reserved IOVA, which carves them out of the allocator. When iommu_dma_init_domain() gets called for the hotplugged vfio-pci device the iova domain is already allocated and set and we don't call iova_reserve_iommu_regions() again for the vfio-pci device. So its corresponding reserved regions are not properly taken into account.
This is not trivial to fix because theoretically the 1st attached devices could already have allocated IOVAs within the reserved regions of the second device. Also we are somehow hijacking the reserved memory regions to model the geometry of the physical IOMMU so not sure any attempt to fix that upstream will be accepted. At the moment one solution is to make sure assigned devices end up in singleton group. Another solution is to work on a different approach where the gaw can be passed as an option to the virtio-iommu device, similarly at what is done with intel iommu.
This series can be found at:
https://github.com/eauger/qemu/tree/iommufd_nesting_preq_v7_resv_regions_v3
History:
v2 -> v3:
- moved the series from RFC to patch
- collected Zhenzhong's R-bs and took into account most of his comments (see replies on on v2)
Eric Auger (7):
HostIOMMUDevice: Store the VFIO/VDPA agent
virtio-iommu: Implement set|unset]_iommu_device() callbacks
HostIOMMUDevice: Introduce get_iova_ranges callback
virtio-iommu: Compute host reserved regions
virtio-iommu: Remove the implementation of iommu_set_iova_range hw/vfio: Remove memory_region_iommu_set_iova_ranges() call
memory: Remove IOMMU MR iommu_set_iova_range API
include/exec/memory.h | 32 --- include/hw/virtio/virtio-iommu.h | 10 + include/sysemu/host_iommu_device.h | 9 + hw/vfio/common.c | 10 - hw/vfio/container.c | 17 ++ hw/vfio/iommufd.c | 18 ++ hw/virtio/virtio-iommu.c | 303 +++++++++++++++++++---------- system/memory.c | 13 -- 8 files changed, 256 insertions(+), 156 deletions(-)
Signature · 1 line
2.41.0
PATCH v3 1/7 HostIOMMUDevice: Store the VFIO/VDPA agent
Store the agent device (VFIO or VDPA) in the host IOMMU device. This will allow easy access to some of its resources.
--- include/sysemu/host_iommu_device.h | 1 + hw/vfio/container.c | 1 + hw/vfio/iommufd.c | 2 ++ 3 files changed, 4 insertions(+)
include/sysemu/host_iommu_device.h +1 -0
@@ -34,6 +34,7 @@ struct HostIOMMUDevice {Object parent_obj;char *name; +void *agent;/* pointer to agent device, ie. VFIO or VDPA device */ HostIOMMUDeviceCaps caps; };
hw/vfio/container.c +1 -0
@@ -1145,6 +1145,7 @@ static bool hiod_legacy_vfio_realize(HostIOMMUDevice *hiod, void *opaque, hiod->name =g_strdup (vdev->name); hiod->caps.aw_bits =vfio_device_get_aw_bits (vdev); + hiod->agent = opaque;return true; }
hw/vfio/iommufd.c +2 -0
@@ -631,6 +631,8 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,struct iommu_hw_info_vtd vtd; } data; + hiod->agent = opaque; +if (!iommufd_backend_get_device_info (vdev->iommufd, vdev->devid, &type, &data,sizeof (data), errp)) {return false;
Signature · 1 line
2.41.0
Re: [PATCH v3 1/7] HostIOMMUDevice: Store the VFIO/VDPA agent
Quoted 40 lines · Eric Auger
Store the agent device (VFIO or VDPA) in the host IOMMU device. This will allow easy access to some of its resources.
Signed-off-by Eric Auger <[email protected]>Reviewed-by Zhenzhong Duan <[email protected]>--- include/sysemu/host_iommu_device.h | 1 + hw/vfio/container.c | 1 + hw/vfio/iommufd.c | 2 ++ 3 files changed, 4 insertions(+)include/sysemu/host_iommu_device.h +1
-0@@ -34,6 +34,7 @@ struct HostIOMMUDevice {Object parent_obj;char *name; +void *agent;/* pointer to agent device, ie. VFIO or VDPA device */ HostIOMMUDeviceCaps caps; };hw/vfio/container.c +1
-0@@ -1145,6 +1145,7 @@ static bool hiod_legacy_vfio_realize(HostIOMMUDevice *hiod, void *opaque, hiod->name =g_strdup (vdev->name); hiod->caps.aw_bits =vfio_device_get_aw_bits (vdev); + hiod->agent = opaque;return true; }hw/vfio/iommufd.c +0
-0
hw/vfio/iommufd.c static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, · line 635
Quoted 5 lines
hw/vfio/iommufd.c +2
-0@@ -631,6 +631,8 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,struct iommu_hw_info_vtd vtd; } data; + hiod->agent = opaque; +
This opaque pointer could be assigned in vfio_attach_device().
Talking of which, why are we passing a 'VFIODevice *' parameter to HostIOMMUDeviceClass::realize ? I don't see a good reason
I think a 'VFIOContainerBase *' would be more appropriate since 'HostIOMMUDevice' represents a device on the host which is common to all VFIO devices.
In that case, HostIOMMUDevice::agent wouldn't need to be opaque anymore. It could simply be a 'VFIOContainerBase *' and hiod_legacy_vfio_get_iova_ranges() in patch 3 would grab the 'iova_ranges' from the 'VFIOContainerBase *' directly.
This means some rework :
* vfio_device_get_aw_bits() would use a 'VFIOContainerBase *' instead.
* HostIOMMUDevice::name would be removed. This is just for error messages.
* hiod_iommufd_vfio_realize() would use VFIOIOMMUFDContainer::be.
That said, I think we need the QOMification changes first.
Thanks,
C.
Quoted 3 lines
hw/vfio/iommufd.c +0
-0if (!iommufd_backend_get_device_info (vdev->iommufd, vdev->devid, &type, &data,sizeof (data), errp)) {return false;
Re: [PATCH v3 1/7] HostIOMMUDevice: Store the VFIO/VDPA agent
Hi Cédric,
Quoted 59 lines · Cédric Le Goater
Quoted 38 lines · Eric Auger
Store the agent device (VFIO or VDPA) in the host IOMMU device. This will allow easy access to some of its resources.
Signed-off-by Eric Auger <[email protected]>Reviewed-by Zhenzhong Duan <[email protected]>---
include/sysemu/host_iommu_device.h| 1 +
hw/vfio/container.c | 1 +
hw/vfio/iommufd.c | 2 ++3 files changed, 4 insertions(+)include/sysemu/host_iommu_device.h +1
-0@@ -34,6 +34,7 @@ struct HostIOMMUDevice { Object parent_obj; char *name; + void *agent; /* pointer to agent device, ie. VFIO or VDPA device */ HostIOMMUDeviceCaps caps; };diff --git
a/hw/vfio/container.cb/hw/vfio/container.cindex26e6f7fb4f..b728b978a2100644hw/vfio/container.c +1
-0@@ -1145,6 +1145,7 @@ static bool hiod_legacy_vfio_realize(HostIOMMUDevice *hiod, void *opaque, hiod->name = g_strdup(vdev->name); hiod->caps.aw_bits = vfio_device_get_aw_bits(vdev); + hiod->agent = opaque; return true; }hw/vfio/iommufd.c +0
-0hw/vfio/iommufd.c static bool · line 635
Quoted 5 lines
hw/vfio/iommufd.c +1
-0@@ -631,6 +631,8 @@ static boolhiod_iommufd_vfio_realize (HostIOMMUDevice *hiod, void *opaque, struct iommu_hw_info_vtd vtd; } data; + hiod->agent = opaque; +This opaque pointer could be assigned in
vfio_attach_device().Talking of which, why are we passing a 'VFIODevice *' parameter to
HostIOMMUDeviceClass::realize? I don't see a good reasonI think a 'VFIOContainerBase *' would be more appropriate since 'HostIOMMUDevice' represents a device on the host which is common to all VFIO devices.
In that case,HostIOMMUDevice::agentwouldn't need to be opaque anymore. It could simply be a 'VFIOContainerBase *' andhiod_legacy_vfio_get_iova_ranges()in patch 3 would grab the 'iova_ranges' from the 'VFIOContainerBase *' directly.
Quoted 8 lines
This means some rework :
*
vfio_device_get_aw_bits()would use a 'VFIOContainerBase *' instead.
*HostIOMMUDevice::namewould be removed. This is just for error messages.
*hiod_iommufd_vfio_realize()would useVFIOIOMMUFDContainer::be.That said, I think we need the QOMification changes first.
OK I need to review your series first. At the moment I have just addressed Zhenzhong's comment in v4, just sent.
Thanks
Eric
Quoted 13 lines
Thanks,
C.
Quoted 4 lines
hw/vfio/iommufd.c +0
-0if (!iommufd_backend_get_device_info(vdev->iommufd, vdev->devid, &type, &data,sizeof(data), errp)) {
return false;
Re: [PATCH v3 1/7] HostIOMMUDevice: Store the VFIO/VDPA agent
Quoted 20 lines
Quoted 20 lines
Talking of which, why are we passing a 'VFIODevice *' parameter to
HostIOMMUDeviceClass::realize? I don't see a good reasonI think a 'VFIOContainerBase *' would be more appropriate since 'HostIOMMUDevice' represents a device on the host which is common to all VFIO devices.
In that case,
HostIOMMUDevice::agentwouldn't need to be opaque anymore. It could simply be a 'VFIOContainerBase *' andhiod_legacy_vfio_get_iova_ranges()in patch 3 would grab the 'iova_ranges' from the 'VFIOContainerBase *' directly.This means some rework :
*
vfio_device_get_aw_bits()would use a 'VFIOContainerBase *' instead.
*HostIOMMUDevice::namewould be removed. This is just for error messages.
*hiod_iommufd_vfio_realize()would useVFIOIOMMUFDContainer::be.
Quoted 4 lines
Quoted 1 line
That said, I think we need the QOMification changes first.
OK I need to review your series first. At the moment I have just addressed Zhenzhong's comment in v4, just sent.
Yep. Just take a look at mine. If both of you agree with above proposal, I can care of it and resend all 3. It's a small change.
Thanks,
C.
RE: [PATCH v3 1/7] HostIOMMUDevice: Store the VFIO/VDPA agent
Hi Cédric,
Quoted 31 lines
-----Original Message-----
From: Cédric Le Goater <[email protected]>
Sent: Friday, June 14, 2024 6:05 PM
To: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; jean- [email protected]; [email protected]; [email protected]; Duan, Zhenzhong <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v3 1/7] HostIOMMUDevice: Store the VFIO/VDPA agentQuoted 19 lines
Quoted 19 lines
Talking of which, why are we passing a 'VFIODevice *' parameter to
HostIOMMUDeviceClass::realize? I don't see a good reasonI think a 'VFIOContainerBase *' would be more appropriate since 'HostIOMMUDevice' represents a device on the host which is common to all VFIO devices.
In that case,
HostIOMMUDevice::agentwouldn't need to be opaque anymore. It could simply be a 'VFIOContainerBase *' andhiod_legacy_vfio_get_iova_ranges()in patch 3 would grab the 'iova_ranges' from the 'VFIOContainerBase *' directly.This means some rework :
*
vfio_device_get_aw_bits()would use a 'VFIOContainerBase *' instead.
*HostIOMMUDevice::namewould be removed. This is just for error messages.
*hiod_iommufd_vfio_realize()would useVFIOIOMMUFDContainer::be.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
That said, I think we need the QOMification changes first.
OK I need to review your series first. At the moment I have just addressed Zhenzhong's comment in v4, just sent.
Yep. Just take a look at mine. If both of you agree with above proposal, I can care of it and resend all 3. It's a small change.
I would suggest using opaque pointer and VFIODevice for two reasons,
1. in nesting series vIOMMU needs to attach/detaching hwpt which is VFIODevice operations. See https://github.com/yiliu1765/qemu/commit/3ca559d35adc9840555e361a56708af4c6338b3d
2. If we plan to support VDPA Device in future, the opaque pointer can also point to an VDPADevice structure.
Thanks
Zhenzhong
Re: [PATCH v3 1/7] HostIOMMUDevice: Store the VFIO/VDPA agent
Quoted 38 lines · Zhenzhong
Hi Cédric,
Quoted 36 lines
-----Original Message-----
From: Cédric Le Goater <[email protected]>
Sent: Friday, June 14, 2024 6:05 PM
To: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; jean- [email protected]; [email protected]; [email protected]; Duan, Zhenzhong <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v3 1/7] HostIOMMUDevice: Store the VFIO/VDPA agentQuoted 23 lines
Quoted 12 lines
Talking of which, why are we passing a 'VFIODevice *' parameter to
HostIOMMUDeviceClass::realize? I don't see a good reasonI think a 'VFIOContainerBase *' would be more appropriate since 'HostIOMMUDevice' represents a device on the host which is common to all VFIO devices.
In that case,
HostIOMMUDevice::agentwouldn't need to be opaque anymore. It could simply be a 'VFIOContainerBase *' andhiod_legacy_vfio_get_iova_ranges()in patch 3 would grab the 'iova_ranges' from the 'VFIOContainerBase *' directly.Quoted 8 lines
This means some rework :
*
vfio_device_get_aw_bits()would use a 'VFIOContainerBase *' instead.
*HostIOMMUDevice::namewould be removed. This is just for error messages.
*hiod_iommufd_vfio_realize()would useVFIOIOMMUFDContainer::be.That said, I think we need the QOMification changes first.
OK I need to review your series first. At the moment I have just addressed Zhenzhong's comment in v4, just sent.
Quoted 6 lines
Quoted 2 lines
Yep. Just take a look at mine. If both of you agree with above proposal, I can care of it and resend all 3. It's a small change.
I would suggest using opaque pointer and VFIODevice for two reasons,
1. in nesting series vIOMMU needs to attach/detaching hwpt which is VFIODevice operations. See https://github.com/yiliu1765/qemu/commit/3ca559d35adc9840555e361a56708af4c6338b3d
OK. I realized later on that we needed the device id anyhow.
Quoted 3 lines
2. If we plan to support VDPA Device in future, the opaque pointer can also point to an VDPADevice structure.
Thanks,
C.
PATCH v3 2/7 virtio-iommu: Implement set|unset]_iommu_device() callbacks
In reply to Eric AugerImplement PCIIOMMUOPs [set|unset]_iommu_device() callbacks. In set(), a VirtioHostIOMMUDevice is allocated which holds a reference to the HostIOMMUDevice. This object is stored in a hash table indexed by PCI BDF. The handle to the Host IOMMU device will allow to retrieve information related to the physical IOMMU.
---
v2 -> v3:
- include host_iommu_device.h in virtio-iommu.h header
- introduce hiod_destroy() and fix UAF in virtio_iommu_unset_iommu_device()
--- include/hw/virtio/virtio-iommu.h | 10 ++++ hw/virtio/virtio-iommu.c | 90 ++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+)
include/hw/virtio/virtio-iommu.h +10 -0
@@ -25,6 +25,7 @@#include "hw/pci/pci.h" #include "qom/object.h" #include "qapi/qapi-types-virtio.h" +#include "sysemu/host_iommu_device.h" #define TYPE_VIRTIO_IOMMU "virtio-iommu-device"#define TYPE_VIRTIO_IOMMU_PCI "virtio-iommu-pci"
@@ -45,6 +46,14 @@ typedef struct IOMMUDevice {bool probe_done; } IOMMUDevice; +typedef struct VirtioHostIOMMUDevice { +void *viommu; +PCIBus *bus; +uint8_t devfn; +HostIOMMUDevice *dev; +QLIST_ENTRY (VirtioHostIOMMUDevice) next; +}VirtioHostIOMMUDevice ; + typedef struct IOMMUPciBus { PCIBus *bus; IOMMUDevice *pbdev[]; /* Parent array is sparse, so dynamically alloc */
@@ -57,6 +66,7 @@ struct VirtIOIOMMU {struct virtio_iommu_config config;uint64_t features;GHashTable *as_by_busptr; +GHashTable *host_iommu_devices;IOMMUPciBus *iommu_pcibus_by_bus_num[PCI_BUS_MAX ];PCIBus *primary_bus;ReservedRegion *prop_resv_regions;
hw/virtio/virtio-iommu.c +90 -0
110 lines not on this page · raw message
Signature · 1 line
2.41.0
RE: [PATCH v3 2/7] virtio-iommu: Implement set|unset]_iommu_device() callbacks
Quoted 41 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 2/7] virtio-iommu: Implement
set|unset]_iommu_device() callbacksImplement PCIIOMMUOPs [set|unset]_iommu_device() callbacks. In
set(), a VirtioHostIOMMUDevice is allocated which holds a reference to the HostIOMMUDevice. This object is stored in a hash table indexed by PCI BDF. The handle to the Host IOMMU device will allow to retrieve information related to the physical IOMMU.Signed-off-by Eric Auger <[email protected]>---
v2 -> v3:
- includehost_iommu_device.hinvirtio-iommu.hheader
- introducehiod_destroy()and fix UAF invirtio_iommu_unset_iommu_device()
---include/hw/virtio/virtio-iommu.h| 10 ++++hw/virtio/virtio-iommu.c| 90
++++++++++++++++++++++++++++++++2 files changed, 100 insertions(+)include/hw/virtio/virtio- renamed from include/hw/virtio/virtio-iommu.h +0
-0
iommu.h
index83a52cc446..a5926de947100644include/hw/virtio/virtio-iommu.h +1
-0@@ -25,6 +25,7 @@#include "hw/pci/pci.h" #include "qom/object.h" #include "qapi/qapi-types-virtio.h" +#include "sysemu/host_iommu_device.h" #define TYPE_VIRTIO_IOMMU "virtio-iommu-device"#define TYPE_VIRTIO_IOMMU_PCI "virtio-iommu-pci"
include/hw/virtio/virtio-iommu.h typedef struct IOMMUDevice { · line 55
Quoted 10 lines
include/hw/virtio/virtio-iommu.h +7
-0@@ -45,6 +46,14 @@ typedef struct IOMMUDevice {bool probe_done; } IOMMUDevice; +typedef struct VirtioHostIOMMUDevice { +void *viommu; +PCIBus *bus; +uint8_t devfn; +HostIOMMUDevice *dev; +QLIST_ENTRY (VirtioHostIOMMUDevice) next; +}VirtioHostIOMMUDevice ;
Do you have plan to use the elements in VirtioHostIOMMUDevice?
Quoted 43 lines
include/hw/virtio/virtio-iommu.h +1
-0+ typedef struct IOMMUPciBus { PCIBus *bus; IOMMUDevice *pbdev[]; /* Parent array is sparse, so dynamically alloc*/
include/hw/virtio/virtio-iommu.h +1
-0@@ -57,6 +66,7 @@ struct VirtIOIOMMU {struct virtio_iommu_config config;uint64_t features;GHashTable *as_by_busptr; +GHashTable *host_iommu_devices;IOMMUPciBus *iommu_pcibus_by_bus_num[PCI_BUS_MAX ];PCIBus *primary_bus;ReservedRegion *prop_resv_regions;hw/virtio/virtio-iommu.c +14
-0@@ -69,6 +69,11 @@ typedef struct VirtIOIOMMUMapping {uint32_t flags; } VirtIOIOMMUMapping; +struct hiod_key { +PCIBus *bus; +uint8_t devfn; +}; +static inline uint16_t virtio_iommu_get_bdf (IOMMUDevice *dev) {return PCI_BUILD_BDF (pci_bus_num (dev->bus), dev->devfn);@@ -462,8 +467,90 @@ static AddressSpace *virtio_iommu_find_add_as (PCIBus *bus, void *opaque, return &sdev->as; } +static gboolean hiod_equal (gconstpointer v1,gconstpointer v2) +{ +const struct hiod_key *key1 = v1; +const struct hiod_key *key2 = v2; + +return (key1->bus == key2->bus) && (key1->devfn == key2->devfn); +} + +static guint hiod_hash (gconstpointer v)
hw/virtio/virtio-iommu.c static AddressSpace · line 489
Quoted 10 lines
hw/virtio/virtio-iommu.c +10
-0+{ +const struct hiod_key *key = v; +guint value = (guint )(uintptr_t )key->bus; + +return (guint)(value <<8 | key->devfn); +} + +static void hiod_destroy (gpointer v) +{ +object_unref (v);
This is a bit different from intel_iommu, here v represents VirtioHostIOMMUDevice *.
Thanks
Zhenzhong
Quoted 83 lines
hw/virtio/virtio-iommu.c +63
-0+} + +static VirtioHostIOMMUDevice * +get_host_iommu_device (VirtIOIOMMU *viommu,PCIBus *bus,int devfn) { +struct hiod_key key = { + .bus = bus, + .devfn = devfn, + }; + +return g_hash_table_lookup (viommu->host_iommu_devices, &key); +} + +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn, +HostIOMMUDevice *hiod,Error **errp) +{ +VirtIOIOMMU *viommu = opaque; +VirtioHostIOMMUDevice *vhiod; +struct hiod_key *new_key; + +assert (hiod); + + vhiod =get_host_iommu_device (viommu, bus, devfn); +if (vhiod) { +error_setg (errp,"VirtioHostIOMMUDevice already exists" ); +return false; + } + + vhiod =g_malloc0 (sizeof (VirtioHostIOMMUDevice)); + vhiod->bus = bus; + vhiod->devfn = (uint8_t )devfn; + vhiod->viommu = viommu; + vhiod->dev = hiod; + + new_key =g_malloc (sizeof (*new_key)); + new_key->bus = bus; + new_key->devfn = devfn; + +object_ref (hiod); +g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); + +return true; +} + +static void +virtio_iommu_unset_iommu_device (PCIBus *bus,void *opaque,int devfn) +{ +VirtIOIOMMU *viommu = opaque; +VirtioHostIOMMUDevice *vhiod; +struct hiod_key key = { + .bus = bus, + .devfn = devfn, + }; + + vhiod =g_hash_table_lookup (viommu->host_iommu_devices, &key); +if (!vhiod) { +return ; + } + +g_hash_table_remove (viommu->host_iommu_devices, &key); +} +static const PCIIOMMUOps virtio_iommu_ops = { .get_address_space = virtio_iommu_find_add_as, + .set_iommu_device = virtio_iommu_set_iommu_device, + .unset_iommu_device = virtio_iommu_unset_iommu_device, };static int virtio_iommu_attach(VirtIOIOMMU *s,
hw/virtio/virtio-iommu.c +3
-0@@ -1357,6 +1444,9 @@ static voidvirtio_iommu_device_realize (DeviceState *dev, Error **errp) s->as_by_busptr = g_hash_table_new_full(NULL, NULL, NULL, g_free); + s->host_iommu_devices =g_hash_table_new_full (hiod_hash, hiod_equal, + g_free, hiod_destroy); +if (s->primary_bus) {pci_setup_iommu(s->primary_bus, &virtio_iommu_ops, s); } else {--
2.41.0
Re: [PATCH v3 2/7] virtio-iommu: Implement set|unset]_iommu_device() callbacks
Quoted 44 lines · Zhenzhong
Quoted 44 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 2/7] virtio-iommu: Implement
set|unset]_iommu_device() callbacksImplement PCIIOMMUOPs [set|unset]_iommu_device() callbacks. In
set(), a VirtioHostIOMMUDevice is allocated which holds a reference to the HostIOMMUDevice. This object is stored in a hash table indexed by PCI BDF. The handle to the Host IOMMU device will allow to retrieve information related to the physical IOMMU.Signed-off-by Eric Auger <[email protected]>---
v2 -> v3:
- includehost_iommu_device.hinvirtio-iommu.hheader
- introducehiod_destroy()and fix UAF invirtio_iommu_unset_iommu_device()
---include/hw/virtio/virtio-iommu.h| 10 ++++hw/virtio/virtio-iommu.c| 90
++++++++++++++++++++++++++++++++2 files changed, 100 insertions(+)include/hw/virtio/virtio- renamed from include/hw/virtio/virtio-iommu.h +0
-0
iommu.h
index83a52cc446..a5926de947100644include/hw/virtio/virtio-iommu.h +1
-0@@ -25,6 +25,7 @@#include "hw/pci/pci.h" #include "qom/object.h" #include "qapi/qapi-types-virtio.h" +#include "sysemu/host_iommu_device.h" #define TYPE_VIRTIO_IOMMU "virtio-iommu-device"#define TYPE_VIRTIO_IOMMU_PCI "virtio-iommu-pci"@@ -45,6 +46,14 @@ typedef struct IOMMUDevice {bool probe_done; } IOMMUDevice;
Quoted 9 lines
include/hw/virtio/virtio-iommu.h typedef struct IOMMUDevice { · line 55
Quoted 7 lines
include/hw/virtio/virtio-iommu.h +7
-0+typedef struct VirtioHostIOMMUDevice { +void *viommu; +PCIBus *bus; +uint8_t devfn; +HostIOMMUDevice *dev; +QLIST_ENTRY (VirtioHostIOMMUDevice) next; +}VirtioHostIOMMUDevice ;Do you have plan to use the elements in VirtioHostIOMMUDevice?
Indeed.
It would be better to avoid the unused struct definition. If removed, the new vtd and virtio un/set_iommu_device ops become very similar and there could be a common shared framework. Some of it could be handled in the PCI subsystem probably ?
Thanks,
C.
Quoted 144 lines
Quoted 44 lines
include/hw/virtio/virtio-iommu.h +1
-0+ typedef struct IOMMUPciBus { PCIBus *bus; IOMMUDevice *pbdev[]; /* Parent array is sparse, so dynamically alloc*/
include/hw/virtio/virtio-iommu.h +1
-0@@ -57,6 +66,7 @@ struct VirtIOIOMMU {struct virtio_iommu_config config;uint64_t features;GHashTable *as_by_busptr; +GHashTable *host_iommu_devices;IOMMUPciBus *iommu_pcibus_by_bus_num[PCI_BUS_MAX ];PCIBus *primary_bus;ReservedRegion *prop_resv_regions;hw/virtio/virtio-iommu.c +14
-0@@ -69,6 +69,11 @@ typedef struct VirtIOIOMMUMapping {uint32_t flags; } VirtIOIOMMUMapping; +struct hiod_key { +PCIBus *bus; +uint8_t devfn; +}; +static inline uint16_t virtio_iommu_get_bdf (IOMMUDevice *dev) {return PCI_BUILD_BDF (pci_bus_num (dev->bus), dev->devfn);@@ -462,8 +467,90 @@ static AddressSpace *virtio_iommu_find_add_as (PCIBus *bus, void *opaque, return &sdev->as; } +static gboolean hiod_equal (gconstpointer v1,gconstpointer v2) +{ +const struct hiod_key *key1 = v1; +const struct hiod_key *key2 = v2; + +return (key1->bus == key2->bus) && (key1->devfn == key2->devfn); +} + +static guint hiod_hash (gconstpointer v)hw/virtio/virtio-iommu.c static AddressSpace · line 489
Quoted 10 lines
hw/virtio/virtio-iommu.c +10
-0+{ +const struct hiod_key *key = v; +guint value = (guint )(uintptr_t )key->bus; + +return (guint)(value <<8 | key->devfn); +} + +static void hiod_destroy (gpointer v) +{ +object_unref (v);This is a bit different from
intel_iommu, here v represents VirtioHostIOMMUDevice *.Thanks
ZhenzhongQuoted 83 lines
hw/virtio/virtio-iommu.c +63
-0+} + +static VirtioHostIOMMUDevice * +get_host_iommu_device (VirtIOIOMMU *viommu,PCIBus *bus,int devfn) { +struct hiod_key key = { + .bus = bus, + .devfn = devfn, + }; + +return g_hash_table_lookup (viommu->host_iommu_devices, &key); +} + +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn, +HostIOMMUDevice *hiod,Error **errp) +{ +VirtIOIOMMU *viommu = opaque; +VirtioHostIOMMUDevice *vhiod; +struct hiod_key *new_key; + +assert (hiod); + + vhiod =get_host_iommu_device (viommu, bus, devfn); +if (vhiod) { +error_setg (errp,"VirtioHostIOMMUDevice already exists" ); +return false; + } + + vhiod =g_malloc0 (sizeof (VirtioHostIOMMUDevice)); + vhiod->bus = bus; + vhiod->devfn = (uint8_t )devfn; + vhiod->viommu = viommu; + vhiod->dev = hiod; + + new_key =g_malloc (sizeof (*new_key)); + new_key->bus = bus; + new_key->devfn = devfn; + +object_ref (hiod); +g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); + +return true; +} + +static void +virtio_iommu_unset_iommu_device (PCIBus *bus,void *opaque,int devfn) +{ +VirtIOIOMMU *viommu = opaque; +VirtioHostIOMMUDevice *vhiod; +struct hiod_key key = { + .bus = bus, + .devfn = devfn, + }; + + vhiod =g_hash_table_lookup (viommu->host_iommu_devices, &key); +if (!vhiod) { +return ; + } + +g_hash_table_remove (viommu->host_iommu_devices, &key); +} +static const PCIIOMMUOps virtio_iommu_ops = { .get_address_space = virtio_iommu_find_add_as, + .set_iommu_device = virtio_iommu_set_iommu_device, + .unset_iommu_device = virtio_iommu_unset_iommu_device, };static int virtio_iommu_attach(VirtIOIOMMU *s,
hw/virtio/virtio-iommu.c +3
-0@@ -1357,6 +1444,9 @@ static voidvirtio_iommu_device_realize (DeviceState *dev, Error **errp) s->as_by_busptr = g_hash_table_new_full(NULL, NULL, NULL, g_free); + s->host_iommu_devices =g_hash_table_new_full (hiod_hash, hiod_equal, + g_free, hiod_destroy); +if (s->primary_bus) {pci_setup_iommu(s->primary_bus, &virtio_iommu_ops, s); } else {--
2.41.0
Re: [PATCH v3 2/7] virtio-iommu: Implement set|unset]_iommu_device() callbacks
In reply to Duan, ZhenzhongHi Zhenzhong,
Quoted 43 lines · Zhenzhong
Quoted 43 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 2/7] virtio-iommu: Implement
set|unset]_iommu_device() callbacksImplement PCIIOMMUOPs [set|unset]_iommu_device() callbacks. In
set(), a VirtioHostIOMMUDevice is allocated which holds a reference to the HostIOMMUDevice. This object is stored in a hash table indexed by PCI BDF. The handle to the Host IOMMU device will allow to retrieve information related to the physical IOMMU.Signed-off-by Eric Auger <[email protected]>---
v2 -> v3:
- includehost_iommu_device.hinvirtio-iommu.hheader
- introducehiod_destroy()and fix UAF invirtio_iommu_unset_iommu_device()
---include/hw/virtio/virtio-iommu.h| 10 ++++hw/virtio/virtio-iommu.c| 90
++++++++++++++++++++++++++++++++2 files changed, 100 insertions(+)include/hw/virtio/virtio- renamed from include/hw/virtio/virtio-iommu.h +0
-0
iommu.h
index83a52cc446..a5926de947100644include/hw/virtio/virtio-iommu.h +1
-0@@ -25,6 +25,7 @@#include "hw/pci/pci.h" #include "qom/object.h" #include "qapi/qapi-types-virtio.h" +#include "sysemu/host_iommu_device.h" #define TYPE_VIRTIO_IOMMU "virtio-iommu-device"#define TYPE_VIRTIO_IOMMU_PCI "virtio-iommu-pci"@@ -45,6 +46,14 @@ typedef struct IOMMUDevice {bool probe_done; } IOMMUDevice;
Quoted 8 lines
include/hw/virtio/virtio-iommu.h typedef struct IOMMUDevice { · line 55
Quoted 7 lines
include/hw/virtio/virtio-iommu.h +7
-0+typedef struct VirtioHostIOMMUDevice { +void *viommu; +PCIBus *bus; +uint8_t devfn; +HostIOMMUDevice *dev; +QLIST_ENTRY (VirtioHostIOMMUDevice) next; +}VirtioHostIOMMUDevice ;Do you have plan to use the elements in VirtioHostIOMMUDevice?
As of now I get rid of this derived object and do something similar as what you did on vtd.
Quoted 47 lines
Quoted 47 lines
include/hw/virtio/virtio-iommu.h +1
-0+ typedef struct IOMMUPciBus { PCIBus *bus; IOMMUDevice *pbdev[]; /* Parent array is sparse, so dynamically alloc*/
include/hw/virtio/virtio-iommu.h +1
-0@@ -57,6 +66,7 @@ struct VirtIOIOMMU {struct virtio_iommu_config config;uint64_t features;GHashTable *as_by_busptr; +GHashTable *host_iommu_devices;IOMMUPciBus *iommu_pcibus_by_bus_num[PCI_BUS_MAX ];PCIBus *primary_bus;ReservedRegion *prop_resv_regions;hw/virtio/virtio-iommu.c +17
-0@@ -69,6 +69,11 @@ typedef struct VirtIOIOMMUMapping {uint32_t flags; } VirtIOIOMMUMapping; +struct hiod_key { +PCIBus *bus; +uint8_t devfn; +}; +static inline uint16_t virtio_iommu_get_bdf (IOMMUDevice *dev) {return PCI_BUILD_BDF (pci_bus_num (dev->bus), dev->devfn);@@ -462,8 +467,90 @@ static AddressSpace *virtio_iommu_find_add_as (PCIBus *bus, void *opaque, return &sdev->as; } +static gboolean hiod_equal (gconstpointer v1,gconstpointer v2) +{ +const struct hiod_key *key1 = v1; +const struct hiod_key *key2 = v2; + +return (key1->bus == key2->bus) && (key1->devfn == key2->devfn); +} + +static guint hiod_hash (gconstpointer v) +{ +const struct hiod_key *key = v; +guint value = (guint )(uintptr_t )key->bus;
Quoted 8 lines
hw/virtio/virtio-iommu.c static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void *opaque, · line 488
Quoted 7 lines
hw/virtio/virtio-iommu.c +7
-0+ +return (guint)(value <<8 | key->devfn); +} + +static void hiod_destroy (gpointer v) +{ +object_unref (v);This is a bit different from
intel_iommu, here v represents VirtioHostIOMMUDevice *.
damned, you are totally right.
Fixed in next version.
Thank you very much!
Eric
Quoted 87 lines
Thanks
ZhenzhongQuoted 83 lines
hw/virtio/virtio-iommu.c +63
-0+} + +static VirtioHostIOMMUDevice * +get_host_iommu_device (VirtIOIOMMU *viommu,PCIBus *bus,int devfn) { +struct hiod_key key = { + .bus = bus, + .devfn = devfn, + }; + +return g_hash_table_lookup (viommu->host_iommu_devices, &key); +} + +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn, +HostIOMMUDevice *hiod,Error **errp) +{ +VirtIOIOMMU *viommu = opaque; +VirtioHostIOMMUDevice *vhiod; +struct hiod_key *new_key; + +assert (hiod); + + vhiod =get_host_iommu_device (viommu, bus, devfn); +if (vhiod) { +error_setg (errp,"VirtioHostIOMMUDevice already exists" ); +return false; + } + + vhiod =g_malloc0 (sizeof (VirtioHostIOMMUDevice)); + vhiod->bus = bus; + vhiod->devfn = (uint8_t )devfn; + vhiod->viommu = viommu; + vhiod->dev = hiod; + + new_key =g_malloc (sizeof (*new_key)); + new_key->bus = bus; + new_key->devfn = devfn; + +object_ref (hiod); +g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); + +return true; +} + +static void +virtio_iommu_unset_iommu_device (PCIBus *bus,void *opaque,int devfn) +{ +VirtIOIOMMU *viommu = opaque; +VirtioHostIOMMUDevice *vhiod; +struct hiod_key key = { + .bus = bus, + .devfn = devfn, + }; + + vhiod =g_hash_table_lookup (viommu->host_iommu_devices, &key); +if (!vhiod) { +return ; + } + +g_hash_table_remove (viommu->host_iommu_devices, &key); +} +static const PCIIOMMUOps virtio_iommu_ops = { .get_address_space = virtio_iommu_find_add_as, + .set_iommu_device = virtio_iommu_set_iommu_device, + .unset_iommu_device = virtio_iommu_unset_iommu_device, };static int virtio_iommu_attach(VirtIOIOMMU *s,
hw/virtio/virtio-iommu.c +3
-0@@ -1357,6 +1444,9 @@ static voidvirtio_iommu_device_realize (DeviceState *dev, Error **errp) s->as_by_busptr = g_hash_table_new_full(NULL, NULL, NULL, g_free); + s->host_iommu_devices =g_hash_table_new_full (hiod_hash, hiod_equal, + g_free, hiod_destroy); +if (s->primary_bus) {pci_setup_iommu(s->primary_bus, &virtio_iommu_ops, s); } else {--
2.41.0
PATCH v3 3/7 HostIOMMUDevice: Introduce get_iova_ranges callback
In reply to Eric AugerIntroduce a new HostIOMMUDevice callback that allows to retrieve the usable IOVA ranges.
Implement this callback in the legacy VFIO and IOMMUFD VFIO host iommu devices. This relies on the VFIODevice agent's base container iova_ranges resource.
---
v2 -> v3:
- add g_assert(vdev)
--- include/sysemu/host_iommu_device.h | 8 ++++++++ hw/vfio/container.c | 16 ++++++++++++++++ hw/vfio/iommufd.c | 16 ++++++++++++++++ 3 files changed, 40 insertions(+)
include/sysemu/host_iommu_device.h +8 -0
@@ -80,6 +80,14 @@ struct HostIOMMUDeviceClass {* i.e., HOST_IOMMU_DEVICE_CAP_AW_BITS. */ int (*get_cap)(HostIOMMUDevice *hiod,int cap,Error **errp); +/** +* @get_iova_ranges: Return the list of usable iova_ranges along with +* @hiod Host IOMMU device +* +* @hiod: handle to the host IOMMU device +* @errp: error handle +*/ + GList* (*get_iova_ranges)(HostIOMMUDevice *hiod, Error **errp); };/*
hw/vfio/container.c +16 -0
@@ -1164,12 +1164,28 @@ static int hiod_legacy_vfio_get_cap(HostIOMMUDevice *hiod, int cap, } } +static GList * +hiod_legacy_vfio_get_iova_ranges (HostIOMMUDevice *hiod,Error **errp) +{ +VFIODevice *vdev = hiod->agent; +GList *l =NULL ; + +g_assert (vdev); + +if (vdev->bcontainer) { + l =g_list_copy (vdev->bcontainer->iova_ranges); + } + +return l; +} +static void hiod_legacy_vfio_class_init (ObjectClass *oc,void *data) {HostIOMMUDeviceClass *hioc =HOST_IOMMU_DEVICE_CLASS (oc); hioc->realize = hiod_legacy_vfio_realize; hioc->get_cap = hiod_legacy_vfio_get_cap; + hioc->get_iova_ranges = hiod_legacy_vfio_get_iova_ranges; };static const TypeInfo types[] = {
hw/vfio/iommufd.c +16 -0
@@ -645,11 +645,27 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,return true; } +static GList * +hiod_iommufd_vfio_get_iova_ranges (HostIOMMUDevice *hiod,Error **errp) +{ +VFIODevice *vdev = hiod->agent; +GList *l =NULL ; + +g_assert (vdev); + +if (vdev->bcontainer) { + l =g_list_copy (vdev->bcontainer->iova_ranges); + } + +return l; +} +static void hiod_iommufd_vfio_class_init (ObjectClass *oc,void *data) {HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_CLASS (oc); hiodc->realize = hiod_iommufd_vfio_realize; + hiodc->get_iova_ranges = hiod_iommufd_vfio_get_iova_ranges; };static const TypeInfo types[] = {
Signature · 1 line
2.41.0
RE: [PATCH v3 3/7] HostIOMMUDevice: Introduce get_iova_ranges callback
Quoted 7 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 3/7] HostIOMMUDevice: Introduceget_iova_rangescallback
Quoted 8 lines
Introduce a new HostIOMMUDevice callback that allows to retrieve the usable IOVA ranges.
Implement this callback in the legacy VFIO and IOMMUFD VFIO host iommu devices. This relies on the VFIODevice agent's base container
iova_rangesresource.Signed-off-by Eric Auger <[email protected]>
Thanks
Zhenzhong
Quoted 101 lines
---
v2 -> v3:
- addg_assert(vdev)
---include/sysemu/host_iommu_device.h | 8 ++++++++ hw/vfio/container.c | 16 ++++++++++++++++ hw/vfio/iommufd.c | 16 ++++++++++++++++ 3 files changed, 40 insertions(+)include/sysemu/host_iommu_device.h +8
-0@@ -80,6 +80,14 @@ struct HostIOMMUDeviceClass {* i.e., HOST_IOMMU_DEVICE_CAP_AW_BITS. */ int (*get_cap)(HostIOMMUDevice *hiod,int cap,Error **errp); +/** +* @get_iova_ranges: Return the list of usable iova_ranges along with +* @hiod Host IOMMU device +* +* @hiod: handle to the host IOMMU device +* @errp: error handle +*/ + GList* (*get_iova_ranges)(HostIOMMUDevice *hiod, Error **errp); };/* hw/vfio/container.c +16
-0@@ -1164,12 +1164,28 @@ static int hiod_legacy_vfio_get_cap(HostIOMMUDevice *hiod, int cap, } } +static GList * +hiod_legacy_vfio_get_iova_ranges(HostIOMMUDevice *hiod, Error **errp) +{ +VFIODevice *vdev = hiod->agent; +GList *l =NULL ; + +g_assert (vdev); + +if (vdev->bcontainer) { + l =g_list_copy (vdev->bcontainer->iova_ranges); + } + +return l; +} + static void hiod_legacy_vfio_class_init(ObjectClass *oc, void *data) {HostIOMMUDeviceClass *hioc =HOST_IOMMU_DEVICE_CLASS (oc); hioc->realize = hiod_legacy_vfio_realize; hioc->get_cap = hiod_legacy_vfio_get_cap; + hioc->get_iova_ranges = hiod_legacy_vfio_get_iova_ranges; };static const TypeInfo types[] = {
hw/vfio/iommufd.c +16
-0@@ -645,11 +645,27 @@ static boolhiod_iommufd_vfio_realize (HostIOMMUDevice *hiod, void *opaque, return true; } +static GList * +hiod_iommufd_vfio_get_iova_ranges(HostIOMMUDevice *hiod, Error **errp) +{ +VFIODevice *vdev = hiod->agent; +GList *l =NULL ; + +g_assert (vdev); + +if (vdev->bcontainer) { + l =g_list_copy (vdev->bcontainer->iova_ranges); + } + +return l; +} + static void hiod_iommufd_vfio_class_init(ObjectClass *oc, void *data) {HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_CLASS (oc); hiodc->realize = hiod_iommufd_vfio_realize; + hiodc->get_iova_ranges = hiod_iommufd_vfio_get_iova_ranges; };static const TypeInfo types[] = {
--
2.41.0
PATCH v3 4/7 virtio-iommu: Compute host reserved regions
In reply to Eric AugerCompute the host reserved regions in virtio_iommu_set_iommu_device(). The usable IOVA regions are retrieved from the HOSTIOMMUDevice. The virtio_iommu_set_host_iova_ranges() helper turns usable regions into complementary reserved regions while testing the inclusion into existing ones. virtio_iommu_set_host_iova_ranges() reuse the implementation of virtio_iommu_set_iova_ranges() which will be removed in subsequent patches. rebuild_resv_regions() is just moved.
---
- added g_assert(!sdev->probe_done)
--- hw/virtio/virtio-iommu.c | 146 ++++++++++++++++++++++++++++++--------- 1 file changed, 112 insertions(+), 34 deletions(-)
hw/virtio/virtio-iommu.c +112 -34
@@ -498,12 +498,109 @@ get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {return g_hash_table_lookup (viommu->host_iommu_devices, &key); } +/** +* rebuild_resv_regions: rebuild resv regions with both the +* info of host resv ranges and property set resv ranges +*/ +static int rebuild_resv_regions (IOMMUDevice *sdev) +{ +GList *l; +int i =0 ; + +/* free the existing list and rebuild it from scratch */ +g_list_free_full (sdev->resv_regions, g_free); + sdev->resv_regions =NULL ; + +/* First add host reserved regions if any, all tagged as RESERVED */ +for (l = sdev->host_resv_ranges; l; l = l->next) { +ReservedRegion *reg =g_new0 (ReservedRegion,1 ); +Range *r = (Range *)l->data; + + reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; +range_set_bounds (®->range,range_lob (r),range_upb (r)); + sdev->resv_regions =resv_region_list_insert (sdev->resv_regions, reg); +trace_virtio_iommu_host_resv_regions (sdev->iommu_mr.parent_obj.name, i, +range_lob (®->range), +range_upb (®->range)); + i++; + } +/* +* then add higher priority reserved regions set by the machine +* through properties +*/ +add_prop_resv_regions (sdev); +return 0 ; +} + +static int virtio_iommu_set_host_iova_ranges (VirtIOIOMMU *s,PCIBus *bus, +int devfn,GList *iova_ranges, +Error **errp) +{ +IOMMUPciBus *sbus =g_hash_table_lookup (s->as_by_busptr, bus); +IOMMUDevice *sdev; +GList *current_ranges; +GList *l, *tmp, *new_ranges =NULL ; +int ret = -EINVAL ; + +if (!sbus) { +error_report ("%s no sbus" , __func__); + } + + sdev = sbus->pbdev[devfn]; + + current_ranges = sdev->host_resv_ranges; + +g_assert (!sdev->probe_done); + +/* check that each new resv region is included in an existing one */ +if (sdev->host_resv_ranges) { +range_inverse_array (iova_ranges, + &new_ranges, +0 ,UINT64_MAX ); + +for (tmp = new_ranges; tmp; tmp = tmp->next) { +Range *newr = (Range *)tmp->data; +bool included = false; + +for (l = current_ranges; l; l = l->next) { +Range * r = (Range *)l->data; + +if (range_contains_range (r, newr)) { + included = true; +break ; + } + } +if (!included) { + goto error; + } + } +/* all new reserved ranges are included in existing ones */ + ret =0 ; + goto out; + } + +range_inverse_array (iova_ranges, + &sdev->host_resv_ranges, +0 ,UINT64_MAX ); +rebuild_resv_regions (sdev); + +return 0 ; +error: +error_setg (errp,"%s Conflicting host reserved ranges set!" , + __func__); +out: +g_list_free_full (new_ranges, g_free); +return ret; +} +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn,HostIOMMUDevice *hiod,Error **errp) {VirtIOIOMMU *viommu = opaque;VirtioHostIOMMUDevice *vhiod; +HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_GET_CLASS (hiod);struct hiod_key *new_key; +GList *host_iova_ranges =NULL ;assert (hiod);
@@ -513,6 +610,20 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn,return false; } +if (hiodc->get_iova_ranges) { +int ret; + host_iova_ranges = hiodc->get_iova_ranges (hiod, errp); +if (!host_iova_ranges) { +return true;/* some old kernels may not support that capability */ + } + ret =virtio_iommu_set_host_iova_ranges (viommu, bus, devfn, + host_iova_ranges, errp); +if (ret) { +g_list_free_full (host_iova_ranges, g_free); +return false; + } + } + vhiod =g_malloc0 (sizeof (VirtioHostIOMMUDevice)); vhiod->bus = bus; vhiod->devfn = (uint8_t )devfn;
@@ -525,6 +636,7 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn,object_ref (hiod);g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); +g_list_free_full (host_iova_ranges, g_free);return true; }
@@ -1246,40 +1358,6 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr,return 0 ; } -/** -* rebuild_resv_regions: rebuild resv regions with both the -* info of host resv ranges and property set resv ranges -*/ -static int rebuild_resv_regions (IOMMUDevice *sdev) -{ -GList *l; -int i =0 ; - -/* free the existing list and rebuild it from scratch */ -g_list_free_full (sdev->resv_regions, g_free); - sdev->resv_regions =NULL ; - -/* First add host reserved regions if any, all tagged as RESERVED */ -for (l = sdev->host_resv_ranges; l; l = l->next) { -ReservedRegion *reg =g_new0 (ReservedRegion,1 ); -Range *r = (Range *)l->data; - - reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; -range_set_bounds (®->range,range_lob (r),range_upb (r)); - sdev->resv_regions =resv_region_list_insert (sdev->resv_regions, reg); -trace_virtio_iommu_host_resv_regions (sdev->iommu_mr.parent_obj.name, i, -range_lob (®->range), -range_upb (®->range)); - i++; - } -/* -* then add higher priority reserved regions set by the machine -* through properties -*/ -add_prop_resv_regions (sdev); -return 0 ; -} -/** * virtio_iommu_set_iova_ranges: Conveys the usable IOVA ranges *
Signature · 1 line
2.41.0
RE: [PATCH v3 4/7] virtio-iommu: Compute host reserved regions
Hi Eric,
Quoted 62 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsCompute the host reserved regions in
virtio_iommu_set_iommu_device(). The usable IOVA regions are retrieved from the HOSTIOMMUDevice. Thevirtio_iommu_set_host_iova_ranges()helper turns usable regions into complementary reserved regions while testing the inclusion into existing ones.virtio_iommu_set_host_iova_ranges()reuse the implementation ofvirtio_iommu_set_iova_ranges()which will be removed in subsequent patches.rebuild_resv_regions()is just moved.Signed-off-by Eric Auger <[email protected]>---
- added
g_assert(!sdev->probe_done)
---hw/virtio/virtio-iommu.c| 146 ++++++++++++++++++++++++++++++-------- -1 file changed, 112 insertions(+), 34 deletions(-)hw/virtio/virtio-iommu.c +30
-0@@ -498,12 +498,109 @@ get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {return g_hash_table_lookup (viommu->host_iommu_devices, &key); } +/** +* rebuild_resv_regions: rebuild resv regions with both the +* info of host resv ranges and property set resv ranges +*/ +static int rebuild_resv_regions (IOMMUDevice *sdev) +{ +GList *l; +int i =0 ; + +/* free the existing list and rebuild it from scratch */ +g_list_free_full (sdev->resv_regions, g_free); + sdev->resv_regions =NULL ; + +/* First add host reserved regions if any, all tagged as RESERVED */ +for (l = sdev->host_resv_ranges; l; l = l->next) { +ReservedRegion *reg =g_new0 (ReservedRegion,1 ); +Range *r = (Range *)l->data; + + reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; +range_set_bounds (®->range,range_lob (r),range_upb (r)); + sdev->resv_regions =resv_region_list_insert (sdev->resv_regions, reg); +trace_virtio_iommu_host_resv_regions (sdev- >iommu_mr.parent_obj.name, i, +range_lob (®->range), +range_upb (®->range)); + i++; + } +/* +* then add higher priority reserved regions set by the machine +* through properties +*/
hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU · line 542
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+add_prop_resv_regions (sdev); +return 0 ; +} + +static int virtio_iommu_set_host_iova_ranges (VirtIOIOMMU *s,PCIBus *bus, +int devfn,GList *iova_ranges, +Error **errp) +{ +IOMMUPciBus *sbus =g_hash_table_lookup (s->as_by_busptr, bus);
Here the bus/devfn parameters are real device BDF not aliased one, But used to index s->as_by_busptr which expect aliased bus/devfn.
Do we need a translation of bus/devfn?
Thanks
Zhenzhong
Quoted 145 lines
hw/virtio/virtio-iommu.c +57
-0+IOMMUDevice *sdev; +GList *current_ranges; +GList *l, *tmp, *new_ranges =NULL ; +int ret = -EINVAL ; + +if (!sbus) { +error_report ("%s no sbus" , __func__); + } + + sdev = sbus->pbdev[devfn]; + + current_ranges = sdev->host_resv_ranges; + +g_assert (!sdev->probe_done); + +/* check that each new resv region is included in an existing one */ +if (sdev->host_resv_ranges) { +range_inverse_array (iova_ranges, + &new_ranges, +0 ,UINT64_MAX ); + +for (tmp = new_ranges; tmp; tmp = tmp->next) { +Range *newr = (Range *)tmp->data; +bool included = false; + +for (l = current_ranges; l; l = l->next) { +Range * r = (Range *)l->data; + +if (range_contains_range (r, newr)) { + included = true; +break ; + } + } +if (!included) { + goto error; + } + } +/* all new reserved ranges are included in existing ones */ + ret =0 ; + goto out; + } + +range_inverse_array (iova_ranges, + &sdev->host_resv_ranges, +0 ,UINT64_MAX ); +rebuild_resv_regions (sdev); + +return 0 ; +error: +error_setg (errp,"%s Conflicting host reserved ranges set!" , + __func__); +out: +g_list_free_full (new_ranges, g_free); +return ret; +} +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn,HostIOMMUDevice *hiod,Error **errp) {VirtIOIOMMU *viommu = opaque;VirtioHostIOMMUDevice *vhiod; + HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_GET_CLASS(hiod); struct hiod_key *new_key;hw/virtio/virtio-iommu.c +15
-0+GList *host_iova_ranges =NULL ;assert (hiod);@@ -513,6 +610,20 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, return false; } + if (hiodc->get_iova_ranges) { +int ret; + host_iova_ranges = hiodc->get_iova_ranges (hiod, errp); +if (!host_iova_ranges) { +return true;/* some old kernels may not support that capability */ + } + ret =virtio_iommu_set_host_iova_ranges (viommu, bus, devfn, + host_iova_ranges, errp); +if (ret) { +g_list_free_full (host_iova_ranges, g_free); +return false; + } + } + vhiod = g_malloc0(sizeof(VirtioHostIOMMUDevice)); vhiod->bus = bus;vhiod->devfn = (uint8_t)devfn;hw/virtio/virtio-iommu.c +1
-0@@ -525,6 +636,7 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, object_ref(hiod);g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); +g_list_free_full (host_iova_ranges, g_free);return true;}
hw/virtio/virtio-iommu.c +0
-34@@ -1246,40 +1358,6 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr, return 0; } -/** -* rebuild_resv_regions: rebuild resv regions with both the -* info of host resv ranges and property set resv ranges -*/ -static int rebuild_resv_regions (IOMMUDevice *sdev) -{ -GList *l; -int i =0 ; - -/* free the existing list and rebuild it from scratch */ -g_list_free_full (sdev->resv_regions, g_free); - sdev->resv_regions =NULL ; - -/* First add host reserved regions if any, all tagged as RESERVED */ -for (l = sdev->host_resv_ranges; l; l = l->next) { -ReservedRegion *reg =g_new0 (ReservedRegion,1 ); -Range *r = (Range *)l->data; - - reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; -range_set_bounds (®->range,range_lob (r),range_upb (r)); - sdev->resv_regions =resv_region_list_insert (sdev->resv_regions, reg); -trace_virtio_iommu_host_resv_regions (sdev- >iommu_mr.parent_obj.name, i, -range_lob (®->range), -range_upb (®->range)); - i++; - } -/* -* then add higher priority reserved regions set by the machine -* through properties -*/ -add_prop_resv_regions (sdev); -return 0 ; -} -/** *
virtio_iommu_set_iova_ranges: Conveys the usable IOVA ranges *
--
2.41.0
Re: [PATCH v3 4/7] virtio-iommu: Compute host reserved regions
Quoted 77 lines · Zhenzhong
Hi Eric,
Quoted 62 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsCompute the host reserved regions in
virtio_iommu_set_iommu_device(). The usable IOVA regions are retrieved from the HOSTIOMMUDevice. Thevirtio_iommu_set_host_iova_ranges()helper turns usable regions into complementary reserved regions while testing the inclusion into existing ones.virtio_iommu_set_host_iova_ranges()reuse the implementation ofvirtio_iommu_set_iova_ranges()which will be removed in subsequent patches.rebuild_resv_regions()is just moved.Signed-off-by Eric Auger <[email protected]>---
- added
g_assert(!sdev->probe_done)
---hw/virtio/virtio-iommu.c| 146 ++++++++++++++++++++++++++++++-------- -1 file changed, 112 insertions(+), 34 deletions(-)hw/virtio/virtio-iommu.c +30
-0@@ -498,12 +498,109 @@ get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {return g_hash_table_lookup (viommu->host_iommu_devices, &key); } +/** +* rebuild_resv_regions: rebuild resv regions with both the +* info of host resv ranges and property set resv ranges +*/ +static int rebuild_resv_regions (IOMMUDevice *sdev) +{ +GList *l; +int i =0 ; + +/* free the existing list and rebuild it from scratch */ +g_list_free_full (sdev->resv_regions, g_free); + sdev->resv_regions =NULL ; + +/* First add host reserved regions if any, all tagged as RESERVED */ +for (l = sdev->host_resv_ranges; l; l = l->next) { +ReservedRegion *reg =g_new0 (ReservedRegion,1 ); +Range *r = (Range *)l->data; + + reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; +range_set_bounds (®->range,range_lob (r),range_upb (r)); + sdev->resv_regions =resv_region_list_insert (sdev->resv_regions, reg); +trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, +range_lob (®->range), +range_upb (®->range)); + i++; + } +/* +* then add higher priority reserved regions set by the machine +* through properties +*/ hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU · line 542
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+add_prop_resv_regions (sdev); +return 0 ; +} + +static int virtio_iommu_set_host_iova_ranges (VirtIOIOMMU *s,PCIBus *bus, +int devfn,GList *iova_ranges, +Error **errp) +{ +IOMMUPciBus *sbus =g_hash_table_lookup (s->as_by_busptr, bus);Here the bus/devfn parameters are real device BDF not aliased one, But used to index
s->as_by_busptrwhich expect aliased bus/devfn.
Quoted 1 line
Do we need a translation of bus/devfn?
Hum that's a good point actually. I need to further study that. that's not easy to translate, is it?
Now I am not totally sure why we don't use the alias as well for HostIOMMUDevices or at least store the aliased bdf.
Thanks
Eric
Quoted 149 lines
Thanks
ZhenzhongQuoted 145 lines
hw/virtio/virtio-iommu.c +57
-0+IOMMUDevice *sdev; +GList *current_ranges; +GList *l, *tmp, *new_ranges =NULL ; +int ret = -EINVAL ; + +if (!sbus) { +error_report ("%s no sbus" , __func__); + } + + sdev = sbus->pbdev[devfn]; + + current_ranges = sdev->host_resv_ranges; + +g_assert (!sdev->probe_done); + +/* check that each new resv region is included in an existing one */ +if (sdev->host_resv_ranges) { +range_inverse_array (iova_ranges, + &new_ranges, +0 ,UINT64_MAX ); + +for (tmp = new_ranges; tmp; tmp = tmp->next) { +Range *newr = (Range *)tmp->data; +bool included = false; + +for (l = current_ranges; l; l = l->next) { +Range * r = (Range *)l->data; + +if (range_contains_range (r, newr)) { + included = true; +break ; + } + } +if (!included) { + goto error; + } + } +/* all new reserved ranges are included in existing ones */ + ret =0 ; + goto out; + } + +range_inverse_array (iova_ranges, + &sdev->host_resv_ranges, +0 ,UINT64_MAX ); +rebuild_resv_regions (sdev); + +return 0 ; +error: +error_setg (errp,"%s Conflicting host reserved ranges set!" , + __func__); +out: +g_list_free_full (new_ranges, g_free); +return ret; +} +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn,HostIOMMUDevice *hiod,Error **errp) {VirtIOIOMMU *viommu = opaque;VirtioHostIOMMUDevice *vhiod; + HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_GET_CLASS(hiod); struct hiod_key *new_key;hw/virtio/virtio-iommu.c +15
-0+GList *host_iova_ranges =NULL ;assert (hiod);@@ -513,6 +610,20 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, return false; } + if (hiodc->get_iova_ranges) { +int ret; + host_iova_ranges = hiodc->get_iova_ranges (hiod, errp); +if (!host_iova_ranges) { +return true;/* some old kernels may not support that capability */ + } + ret =virtio_iommu_set_host_iova_ranges (viommu, bus, devfn, + host_iova_ranges, errp); +if (ret) { +g_list_free_full (host_iova_ranges, g_free); +return false; + } + } + vhiod = g_malloc0(sizeof(VirtioHostIOMMUDevice)); vhiod->bus = bus;vhiod->devfn = (uint8_t)devfn;hw/virtio/virtio-iommu.c +1
-0@@ -525,6 +636,7 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, object_ref(hiod);g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); +g_list_free_full (host_iova_ranges, g_free);return true;}
hw/virtio/virtio-iommu.c +0
-34@@ -1246,40 +1358,6 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr, return 0; } -/** -* rebuild_resv_regions: rebuild resv regions with both the -* info of host resv ranges and property set resv ranges -*/ -static int rebuild_resv_regions (IOMMUDevice *sdev) -{ -GList *l; -int i =0 ; - -/* free the existing list and rebuild it from scratch */ -g_list_free_full (sdev->resv_regions, g_free); - sdev->resv_regions =NULL ; - -/* First add host reserved regions if any, all tagged as RESERVED */ -for (l = sdev->host_resv_ranges; l; l = l->next) { -ReservedRegion *reg =g_new0 (ReservedRegion,1 ); -Range *r = (Range *)l->data; - - reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; -range_set_bounds (®->range,range_lob (r),range_upb (r)); - sdev->resv_regions =resv_region_list_insert (sdev->resv_regions, reg); -trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, -range_lob (®->range), -range_upb (®->range)); - i++; - } -/* -* then add higher priority reserved regions set by the machine -* through properties -*/ -add_prop_resv_regions (sdev); -return 0 ; -} -/** *
virtio_iommu_set_iova_ranges: Conveys the usable IOVA ranges *
--
2.41.0
RE: [PATCH v3 4/7] virtio-iommu: Compute host reserved regions
Quoted 88 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: Re: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsQuoted 21 lines · Zhenzhong
Hi Eric,
Quoted 19 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsCompute the host reserved regions in
virtio_iommu_set_iommu_device(). The usable IOVA regions are retrieved from the HOSTIOMMUDevice. Thevirtio_iommu_set_host_iova_ranges()helper turns usable regions into complementary reserved regions while testing the inclusion into existing ones.virtio_iommu_set_host_iova_ranges()reuse the implementation ofvirtio_iommu_set_iova_ranges()which will be removed in subsequent patches.rebuild_resv_regions()is just moved.Signed-off-by Eric Auger <[email protected]>---
- added
g_assert(!sdev->probe_done)
---hw/virtio/virtio-iommu.c| 146 ++++++++++++++++++++++++++++++--------
Quoted 25 lines
Quoted 25 lines
-
1 file changed, 112 insertions(+), 34 deletions(-)hw/virtio/virtio-iommu.c +13
-0@@ -498,12 +498,109 @@ get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {return g_hash_table_lookup (viommu->host_iommu_devices, &key); } +/** +* rebuild_resv_regions: rebuild resv regions with both the +* info of host resv ranges and property set resv ranges +*/ +static int rebuild_resv_regions (IOMMUDevice *sdev) +{ +GList *l; +int i =0 ; + +/* free the existing list and rebuild it from scratch */ +g_list_free_full (sdev->resv_regions, g_free); + sdev->resv_regions =NULL ; +Quoted 8 lines
hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {
Quoted 8 lines
hw/virtio/virtio-iommu.c +8
-0+/* First add host reserved regions if any, all tagged as RESERVED */ +for (l = sdev->host_resv_ranges; l; l = l->next) { +ReservedRegion *reg =g_new0 (ReservedRegion,1 ); +Range *r = (Range *)l->data; + + reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; +range_set_bounds (®->range,range_lob (r),range_upb (r)); + sdev->resv_regions = resv_region_list_insert(sdev->resv_regions,reg);
Quoted 23 lines
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, +range_lob (®->range), +range_upb (®->range)); + i++; + } +/* +* then add higher priority reserved regions set by the machine +* through properties +*/ hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) { · line 539
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+add_prop_resv_regions (sdev); +return 0 ; +} + +static int virtio_iommu_set_host_iova_ranges (VirtIOIOMMU *s,PCIBus *bus, +int devfn,GList *iova_ranges, +Error **errp) +{ +IOMMUPciBus *sbus =g_hash_table_lookup (s->as_by_busptr, bus);Here the bus/devfn parameters are real device BDF not aliased one, But used to index
s->as_by_busptrwhich expect aliased bus/devfn.
Quoted 4 lines
Quoted 1 line
Do we need a translation of bus/devfn?
Hum that's a good point actually. I need to further study that. that's not easy to translate, is it?
Yes, may need a path to call into pci_device_get_iommu_bus_devfn(). Maybe a new HostIOMMUDevice callback.
Quoted 3 lines
Now I am not totally sure why we don't use the alias as well for HostIOMMUDevices or at least store the aliased bdf.
Because we need to store HostIOMMUDevice in vIOMMU in real bdf granularity, Not aliased bdf granularity. Virtual vtd calls VFIO device uAPI on real device not the aliased device, i.e., VFIO_DEVICE_[ATTACH|DETACH]_IOMMUFD_PT.
I also have a question, could we define host_resv_ranges in VirtioHostIOMMUDevice to avoid translation?
Thanks
Zhenzhong
Quoted 155 lines
Thanks
Eric
Quoted 78 lines
Thanks
ZhenzhongQuoted 74 lines
hw/virtio/virtio-iommu.c +57
-0+IOMMUDevice *sdev; +GList *current_ranges; +GList *l, *tmp, *new_ranges =NULL ; +int ret = -EINVAL ; + +if (!sbus) { +error_report ("%s no sbus" , __func__); + } + + sdev = sbus->pbdev[devfn]; + + current_ranges = sdev->host_resv_ranges; + +g_assert (!sdev->probe_done); + +/* check that each new resv region is included in an existing one */ +if (sdev->host_resv_ranges) { +range_inverse_array (iova_ranges, + &new_ranges, +0 ,UINT64_MAX ); + +for (tmp = new_ranges; tmp; tmp = tmp->next) { +Range *newr = (Range *)tmp->data; +bool included = false; + +for (l = current_ranges; l; l = l->next) { +Range * r = (Range *)l->data; + +if (range_contains_range (r, newr)) { + included = true; +break ; + } + } +if (!included) { + goto error; + } + } +/* all new reserved ranges are included in existing ones */ + ret =0 ; + goto out; + } + +range_inverse_array (iova_ranges, + &sdev->host_resv_ranges, +0 ,UINT64_MAX ); +rebuild_resv_regions (sdev); + +return 0 ; +error: +error_setg (errp,"%s Conflicting host reserved ranges set!" , + __func__); +out: +g_list_free_full (new_ranges, g_free); +return ret; +} +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn,HostIOMMUDevice *hiod,Error **errp) {VirtIOIOMMU *viommu = opaque;VirtioHostIOMMUDevice *vhiod; + HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_GET_CLASS(hiod); struct hiod_key *new_key;hw/virtio/virtio-iommu.c +1
-0+GList *host_iova_ranges =NULL ;assert (hiod);@@ -513,6 +610,20 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, return false; }Quoted 5 lines
hw/virtio/virtio-iommu.c static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
Quoted 5 lines
hw/virtio/virtio-iommu.c +5
-0+if (hiodc->get_iova_ranges) { +int ret; + host_iova_ranges = hiodc->get_iova_ranges (hiod, errp); +if (!host_iova_ranges) { +return true;/* some old kernels may not support that capability */
Quoted 39 lines
Quoted 39 lines
hw/virtio/virtio-iommu.c +9
-0+ } + ret =virtio_iommu_set_host_iova_ranges (viommu, bus, devfn, + host_iova_ranges, errp); +if (ret) { +g_list_free_full (host_iova_ranges, g_free); +return false; + } + } + vhiod =g_malloc0 (sizeof (VirtioHostIOMMUDevice)); vhiod->bus = bus;vhiod->devfn = (uint8_t)devfn;hw/virtio/virtio-iommu.c +1
-0@@ -525,6 +636,7 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, object_ref(hiod);g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); +g_list_free_full (host_iova_ranges, g_free);return true;}
hw/virtio/virtio-iommu.c +0
-13@@ -1246,40 +1358,6 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr, return 0; } -/** -* rebuild_resv_regions: rebuild resv regions with both the -* info of host resv ranges and property set resv ranges -*/ -static int rebuild_resv_regions (IOMMUDevice *sdev) -{ -GList *l; -int i =0 ; - -/* free the existing list and rebuild it from scratch */ -g_list_free_full (sdev->resv_regions, g_free); - sdev->resv_regions =NULL ; -Quoted 8 lines
hw/virtio/virtio-iommu.c static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr,
Quoted 8 lines
hw/virtio/virtio-iommu.c +0
-8-/* First add host reserved regions if any, all tagged as RESERVED */ -for (l = sdev->host_resv_ranges; l; l = l->next) { -ReservedRegion *reg =g_new0 (ReservedRegion,1 ); -Range *r = (Range *)l->data; - - reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; -range_set_bounds (®->range,range_lob (r),range_upb (r)); - sdev->resv_regions = resv_region_list_insert(sdev->resv_regions,reg);
Quoted 19 lines
Quoted 19 lines
hw/virtio/virtio-iommu.c +0
-13-trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, -range_lob (®->range), -range_upb (®->range)); - i++; - } -/* -* then add higher priority reserved regions set by the machine -* through properties -*/ -add_prop_resv_regions (sdev); -return 0 ; -} -/** *
virtio_iommu_set_iova_ranges: Conveys the usable IOVA ranges *
--
2.41.0
Re: [PATCH v3 4/7] virtio-iommu: Compute host reserved regions
Quoted 93 lines · Zhenzhong
Quoted 87 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: Re: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsQuoted 21 lines · Zhenzhong
Hi Eric,
Quoted 19 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsCompute the host reserved regions in
virtio_iommu_set_iommu_device(). The usable IOVA regions are retrieved from the HOSTIOMMUDevice. Thevirtio_iommu_set_host_iova_ranges()helper turns usable regions into complementary reserved regions while testing the inclusion into existing ones.virtio_iommu_set_host_iova_ranges()reuse the implementation ofvirtio_iommu_set_iova_ranges()which will be removed in subsequent patches.rebuild_resv_regions()is just moved.Signed-off-by Eric Auger <[email protected]>---
- added
g_assert(!sdev->probe_done)
---hw/virtio/virtio-iommu.c| 146 ++++++++++++++++++++++++++++++--------
Quoted 25 lines
Quoted 25 lines
-
1 file changed, 112 insertions(+), 34 deletions(-)hw/virtio/virtio-iommu.c +13
-0@@ -498,12 +498,109 @@ get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {return g_hash_table_lookup (viommu->host_iommu_devices, &key); } +/** +* rebuild_resv_regions: rebuild resv regions with both the +* info of host resv ranges and property set resv ranges +*/ +static int rebuild_resv_regions (IOMMUDevice *sdev) +{ +GList *l; +int i =0 ; + +/* free the existing list and rebuild it from scratch */ +g_list_free_full (sdev->resv_regions, g_free); + sdev->resv_regions =NULL ; +Quoted 8 lines
hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {
Quoted 8 lines
hw/virtio/virtio-iommu.c +8
-0+/* First add host reserved regions if any, all tagged as RESERVED */ +for (l = sdev->host_resv_ranges; l; l = l->next) { +ReservedRegion *reg =g_new0 (ReservedRegion,1 ); +Range *r = (Range *)l->data; + + reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; +range_set_bounds (®->range,range_lob (r),range_upb (r)); + sdev->resv_regions = resv_region_list_insert(sdev->resv_regions,reg);
Quoted 23 lines
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, +range_lob (®->range), +range_upb (®->range)); + i++; + } +/* +* then add higher priority reserved regions set by the machine +* through properties +*/ hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) { · line 539
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+add_prop_resv_regions (sdev); +return 0 ; +} + +static int virtio_iommu_set_host_iova_ranges (VirtIOIOMMU *s,PCIBus *bus, +int devfn,GList *iova_ranges, +Error **errp) +{ +IOMMUPciBus *sbus =g_hash_table_lookup (s->as_by_busptr, bus);Here the bus/devfn parameters are real device BDF not aliased one, But used to index
s->as_by_busptrwhich expect aliased bus/devfn.Quoted 3 lines
Quoted 1 line
Do we need a translation of bus/devfn?
Hum that's a good point actually. I need to further study that. that's not easy to translate, is it?
Yes, may need a path to call into
pci_device_get_iommu_bus_devfn(). Maybe a new HostIOMMUDevice callback.
Quoted 8 lines
Quoted 2 lines
Now I am not totally sure why we don't use the alias as well for HostIOMMUDevices or at least store the aliased bdf.
Because we need to store HostIOMMUDevice in vIOMMU in real bdf granularity, Not aliased bdf granularity. Virtual vtd calls VFIO device uAPI on real device not the aliased device, i.e., VFIO_DEVICE_[ATTACH|DETACH]_IOMMUFD_PT.
I also have a question, could we define
host_resv_rangesin VirtioHostIOMMUDevice to avoid translation?
well reserved iova regions are rather associated to iommu groups (associated to the concept of alias bdfs), no? Also potentially emulated devices could expose some.
Eric
Quoted 157 lines
Thanks
ZhenzhongQuoted 153 lines
Thanks
Eric
Quoted 77 lines
Thanks
ZhenzhongQuoted 74 lines
hw/virtio/virtio-iommu.c +57
-0+IOMMUDevice *sdev; +GList *current_ranges; +GList *l, *tmp, *new_ranges =NULL ; +int ret = -EINVAL ; + +if (!sbus) { +error_report ("%s no sbus" , __func__); + } + + sdev = sbus->pbdev[devfn]; + + current_ranges = sdev->host_resv_ranges; + +g_assert (!sdev->probe_done); + +/* check that each new resv region is included in an existing one */ +if (sdev->host_resv_ranges) { +range_inverse_array (iova_ranges, + &new_ranges, +0 ,UINT64_MAX ); + +for (tmp = new_ranges; tmp; tmp = tmp->next) { +Range *newr = (Range *)tmp->data; +bool included = false; + +for (l = current_ranges; l; l = l->next) { +Range * r = (Range *)l->data; + +if (range_contains_range (r, newr)) { + included = true; +break ; + } + } +if (!included) { + goto error; + } + } +/* all new reserved ranges are included in existing ones */ + ret =0 ; + goto out; + } + +range_inverse_array (iova_ranges, + &sdev->host_resv_ranges, +0 ,UINT64_MAX ); +rebuild_resv_regions (sdev); + +return 0 ; +error: +error_setg (errp,"%s Conflicting host reserved ranges set!" , + __func__); +out: +g_list_free_full (new_ranges, g_free); +return ret; +} +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn,HostIOMMUDevice *hiod,Error **errp) {VirtIOIOMMU *viommu = opaque;VirtioHostIOMMUDevice *vhiod; + HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_GET_CLASS(hiod); struct hiod_key *new_key;hw/virtio/virtio-iommu.c +1
-0+GList *host_iova_ranges =NULL ;assert (hiod);@@ -513,6 +610,20 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, return false; }Quoted 5 lines
hw/virtio/virtio-iommu.c static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
Quoted 5 lines
hw/virtio/virtio-iommu.c +5
-0+if (hiodc->get_iova_ranges) { +int ret; + host_iova_ranges = hiodc->get_iova_ranges (hiod, errp); +if (!host_iova_ranges) { +return true;/* some old kernels may not support that capability */
Quoted 39 lines
Quoted 39 lines
hw/virtio/virtio-iommu.c +9
-0+ } + ret =virtio_iommu_set_host_iova_ranges (viommu, bus, devfn, + host_iova_ranges, errp); +if (ret) { +g_list_free_full (host_iova_ranges, g_free); +return false; + } + } + vhiod =g_malloc0 (sizeof (VirtioHostIOMMUDevice)); vhiod->bus = bus;vhiod->devfn = (uint8_t)devfn;hw/virtio/virtio-iommu.c +1
-0@@ -525,6 +636,7 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, object_ref(hiod);g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); +g_list_free_full (host_iova_ranges, g_free);return true;}
hw/virtio/virtio-iommu.c +0
-13@@ -1246,40 +1358,6 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr, return 0; } -/** -* rebuild_resv_regions: rebuild resv regions with both the -* info of host resv ranges and property set resv ranges -*/ -static int rebuild_resv_regions (IOMMUDevice *sdev) -{ -GList *l; -int i =0 ; - -/* free the existing list and rebuild it from scratch */ -g_list_free_full (sdev->resv_regions, g_free); - sdev->resv_regions =NULL ; -Quoted 8 lines
hw/virtio/virtio-iommu.c static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr,
Quoted 8 lines
hw/virtio/virtio-iommu.c +0
-8-/* First add host reserved regions if any, all tagged as RESERVED */ -for (l = sdev->host_resv_ranges; l; l = l->next) { -ReservedRegion *reg =g_new0 (ReservedRegion,1 ); -Range *r = (Range *)l->data; - - reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; -range_set_bounds (®->range,range_lob (r),range_upb (r)); - sdev->resv_regions = resv_region_list_insert(sdev->resv_regions,reg);
Quoted 19 lines
Quoted 19 lines
hw/virtio/virtio-iommu.c +0
-13-trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, -range_lob (®->range), -range_upb (®->range)); - i++; - } -/* -* then add higher priority reserved regions set by the machine -* through properties -*/ -add_prop_resv_regions (sdev); -return 0 ; -} -/** *
virtio_iommu_set_iova_ranges: Conveys the usable IOVA ranges *
--
2.41.0
Re: [PATCH v3 4/7] virtio-iommu: Compute host reserved regions
In reply to Duan, ZhenzhongHi Zhenzhong,
Quoted 93 lines · Zhenzhong
Quoted 87 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: Re: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsQuoted 21 lines · Zhenzhong
Hi Eric,
Quoted 19 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Subject: [PATCH v3 4/7] virtio-iommu: Compute host reserved regionsCompute the host reserved regions in
virtio_iommu_set_iommu_device(). The usable IOVA regions are retrieved from the HOSTIOMMUDevice. Thevirtio_iommu_set_host_iova_ranges()helper turns usable regions into complementary reserved regions while testing the inclusion into existing ones.virtio_iommu_set_host_iova_ranges()reuse the implementation ofvirtio_iommu_set_iova_ranges()which will be removed in subsequent patches.rebuild_resv_regions()is just moved.Signed-off-by Eric Auger <[email protected]>---
- added
g_assert(!sdev->probe_done)
---hw/virtio/virtio-iommu.c| 146 ++++++++++++++++++++++++++++++--------
Quoted 25 lines
Quoted 25 lines
-
1 file changed, 112 insertions(+), 34 deletions(-)hw/virtio/virtio-iommu.c +13
-0@@ -498,12 +498,109 @@ get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {return g_hash_table_lookup (viommu->host_iommu_devices, &key); } +/** +* rebuild_resv_regions: rebuild resv regions with both the +* info of host resv ranges and property set resv ranges +*/ +static int rebuild_resv_regions (IOMMUDevice *sdev) +{ +GList *l; +int i =0 ; + +/* free the existing list and rebuild it from scratch */ +g_list_free_full (sdev->resv_regions, g_free); + sdev->resv_regions =NULL ; +Quoted 8 lines
hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) {
Quoted 8 lines
hw/virtio/virtio-iommu.c +8
-0+/* First add host reserved regions if any, all tagged as RESERVED */ +for (l = sdev->host_resv_ranges; l; l = l->next) { +ReservedRegion *reg =g_new0 (ReservedRegion,1 ); +Range *r = (Range *)l->data; + + reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; +range_set_bounds (®->range,range_lob (r),range_upb (r)); + sdev->resv_regions = resv_region_list_insert(sdev->resv_regions,reg);
Quoted 23 lines
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, +range_lob (®->range), +range_upb (®->range)); + i++; + } +/* +* then add higher priority reserved regions set by the machine +* through properties +*/ hw/virtio/virtio-iommu.c get_host_iommu_device(VirtIOIOMMU *viommu, PCIBus *bus, int devfn) { · line 539
Quoted 10 lines
hw/virtio/virtio-iommu.c +9
-0+add_prop_resv_regions (sdev); +return 0 ; +} + +static int virtio_iommu_set_host_iova_ranges (VirtIOIOMMU *s,PCIBus *bus, +int devfn,GList *iova_ranges, +Error **errp) +{ +IOMMUPciBus *sbus =g_hash_table_lookup (s->as_by_busptr, bus);Here the bus/devfn parameters are real device BDF not aliased one, But used to index
s->as_by_busptrwhich expect aliased bus/devfn.Quoted 3 lines
Quoted 1 line
Do we need a translation of bus/devfn?
Hum that's a good point actually. I need to further study that. that's not easy to translate, is it?
Yes, may need a path to call into
pci_device_get_iommu_bus_devfn(). Maybe a new HostIOMMUDevice callback.
Quoted 8 lines
Quoted 2 lines
Now I am not totally sure why we don't use the alias as well for HostIOMMUDevices or at least store the aliased bdf.
Because we need to store HostIOMMUDevice in vIOMMU in real bdf granularity, Not aliased bdf granularity. Virtual vtd calls VFIO device uAPI on real device not the aliased device, i.e., VFIO_DEVICE_[ATTACH|DETACH]_IOMMUFD_PT.
I also have a question, could we define
host_resv_rangesin VirtioHostIOMMUDevice to avoid translation?
I would suggest to pass the aliased info through the set_iommu_device() and store them in the HostIOMMUDevice. I will submit a patch accordingly
Thanks
Eric
Quoted 157 lines
Thanks
ZhenzhongQuoted 153 lines
Thanks
Eric
Quoted 77 lines
Thanks
ZhenzhongQuoted 74 lines
hw/virtio/virtio-iommu.c +57
-0+IOMMUDevice *sdev; +GList *current_ranges; +GList *l, *tmp, *new_ranges =NULL ; +int ret = -EINVAL ; + +if (!sbus) { +error_report ("%s no sbus" , __func__); + } + + sdev = sbus->pbdev[devfn]; + + current_ranges = sdev->host_resv_ranges; + +g_assert (!sdev->probe_done); + +/* check that each new resv region is included in an existing one */ +if (sdev->host_resv_ranges) { +range_inverse_array (iova_ranges, + &new_ranges, +0 ,UINT64_MAX ); + +for (tmp = new_ranges; tmp; tmp = tmp->next) { +Range *newr = (Range *)tmp->data; +bool included = false; + +for (l = current_ranges; l; l = l->next) { +Range * r = (Range *)l->data; + +if (range_contains_range (r, newr)) { + included = true; +break ; + } + } +if (!included) { + goto error; + } + } +/* all new reserved ranges are included in existing ones */ + ret =0 ; + goto out; + } + +range_inverse_array (iova_ranges, + &sdev->host_resv_ranges, +0 ,UINT64_MAX ); +rebuild_resv_regions (sdev); + +return 0 ; +error: +error_setg (errp,"%s Conflicting host reserved ranges set!" , + __func__); +out: +g_list_free_full (new_ranges, g_free); +return ret; +} +static bool virtio_iommu_set_iommu_device (PCIBus *bus,void *opaque,int devfn,HostIOMMUDevice *hiod,Error **errp) {VirtIOIOMMU *viommu = opaque;VirtioHostIOMMUDevice *vhiod; + HostIOMMUDeviceClass *hiodc =HOST_IOMMU_DEVICE_GET_CLASS(hiod); struct hiod_key *new_key;hw/virtio/virtio-iommu.c +1
-0+GList *host_iova_ranges =NULL ;assert (hiod);@@ -513,6 +610,20 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, return false; }Quoted 5 lines
hw/virtio/virtio-iommu.c static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
Quoted 5 lines
hw/virtio/virtio-iommu.c +5
-0+if (hiodc->get_iova_ranges) { +int ret; + host_iova_ranges = hiodc->get_iova_ranges (hiod, errp); +if (!host_iova_ranges) { +return true;/* some old kernels may not support that capability */
Quoted 39 lines
Quoted 39 lines
hw/virtio/virtio-iommu.c +9
-0+ } + ret =virtio_iommu_set_host_iova_ranges (viommu, bus, devfn, + host_iova_ranges, errp); +if (ret) { +g_list_free_full (host_iova_ranges, g_free); +return false; + } + } + vhiod =g_malloc0 (sizeof (VirtioHostIOMMUDevice)); vhiod->bus = bus;vhiod->devfn = (uint8_t)devfn;hw/virtio/virtio-iommu.c +1
-0@@ -525,6 +636,7 @@ static bool virtio_iommu_set_iommu_device(PCIBus *bus, void *opaque, int devfn, object_ref(hiod);g_hash_table_insert (viommu->host_iommu_devices, new_key, vhiod); +g_list_free_full (host_iova_ranges, g_free);return true;}
hw/virtio/virtio-iommu.c +0
-13@@ -1246,40 +1358,6 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr, return 0; } -/** -* rebuild_resv_regions: rebuild resv regions with both the -* info of host resv ranges and property set resv ranges -*/ -static int rebuild_resv_regions (IOMMUDevice *sdev) -{ -GList *l; -int i =0 ; - -/* free the existing list and rebuild it from scratch */ -g_list_free_full (sdev->resv_regions, g_free); - sdev->resv_regions =NULL ; -Quoted 8 lines
hw/virtio/virtio-iommu.c static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr,
Quoted 8 lines
hw/virtio/virtio-iommu.c +0
-8-/* First add host reserved regions if any, all tagged as RESERVED */ -for (l = sdev->host_resv_ranges; l; l = l->next) { -ReservedRegion *reg =g_new0 (ReservedRegion,1 ); -Range *r = (Range *)l->data; - - reg->type =VIRTIO_IOMMU_RESV_MEM_T_RESERVED ; -range_set_bounds (®->range,range_lob (r),range_upb (r)); - sdev->resv_regions = resv_region_list_insert(sdev->resv_regions,reg);
Quoted 19 lines
Quoted 19 lines
hw/virtio/virtio-iommu.c +0
-13-trace_virtio_iommu_host_resv_regions (sdev- > iommu_mr.parent_obj.name, i, -range_lob (®->range), -range_upb (®->range)); - i++; - } -/* -* then add higher priority reserved regions set by the machine -* through properties -*/ -add_prop_resv_regions (sdev); -return 0 ; -} -/** *
virtio_iommu_set_iova_ranges: Conveys the usable IOVA ranges *
--
2.41.0
PATCH v3 5/7 virtio-iommu: Remove the implementation of iommu_set_iova_range
In reply to Eric AugerNow that we use PCIIOMMUOps to convey information about usable IOVA ranges we do not to implement the iommu_set_iova_ranges IOMMU MR callback.
--- hw/virtio/virtio-iommu.c | 67 ---------------------------------------- 1 file changed, 67 deletions(-)
hw/virtio/virtio-iommu.c +0 -67
@@ -1358,72 +1358,6 @@ static int virtio_iommu_set_page_size_mask(IOMMUMemoryRegion *mr,return 0 ; } -/** -* virtio_iommu_set_iova_ranges: Conveys the usable IOVA ranges -* -* The function turns those into reserved ranges. Once some -* reserved ranges have been set, new reserved regions cannot be -* added outside of the original ones. -* -* @mr: IOMMU MR -* @iova_ranges: list of usable IOVA ranges -* @errp: error handle -*/ -static int virtio_iommu_set_iova_ranges (IOMMUMemoryRegion *mr, -GList *iova_ranges, -Error **errp) -{ -IOMMUDevice *sdev =container_of (mr, IOMMUDevice, iommu_mr); -GList *current_ranges = sdev->host_resv_ranges; -GList *l, *tmp, *new_ranges =NULL ; -int ret = -EINVAL ; - -/* check that each new resv region is included in an existing one */ -if (sdev->host_resv_ranges) { -range_inverse_array (iova_ranges, - &new_ranges, -0 ,UINT64_MAX ); - -for (tmp = new_ranges; tmp; tmp = tmp->next) { -Range *newr = (Range *)tmp->data; -bool included = false; - -for (l = current_ranges; l; l = l->next) { -Range * r = (Range *)l->data; - -if (range_contains_range (r, newr)) { - included = true; -break ; - } - } -if (!included) { - goto error; - } - } -/* all new reserved ranges are included in existing ones */ - ret =0 ; - goto out; - } - -if (sdev->probe_done) { -warn_report ("%s: Notified about new host reserved regions after probe" , - mr->parent_obj.name); - } - -range_inverse_array (iova_ranges, - &sdev->host_resv_ranges, -0 ,UINT64_MAX ); -rebuild_resv_regions (sdev); - -return 0 ; -error: -error_setg (errp,"IOMMU mr=%s Conflicting host reserved ranges set!" , - mr->parent_obj.name); -out: -g_list_free_full (new_ranges, g_free); -return ret; -} -static void virtio_iommu_system_reset (void *opaque) {VirtIOIOMMU *s = opaque;
@@ -1749,7 +1683,6 @@ static void virtio_iommu_memory_region_class_init(ObjectClass *klass, imrc->replay = virtio_iommu_replay; imrc->notify_flag_changed = virtio_iommu_notify_flag_changed; imrc->iommu_set_page_size_mask = virtio_iommu_set_page_size_mask; - imrc->iommu_set_iova_ranges = virtio_iommu_set_iova_ranges; }static const TypeInfo virtio_iommu_info = {
Signature · 1 line
2.41.0
PATCH v3 6/7 hw/vfio: Remove memory_region_iommu_set_iova_ranges() call
In reply to Eric AugerAs we have just removed the only implementation of iommu_set_iova_ranges IOMMU MR callback in the virtio-iommu, let's remove the call to the memory wrapper. Usable IOVA ranges are now conveyed through the PCIIOMMUOps in VFIO-PCI.
--- hw/vfio/common.c | 10 ---------- 1 file changed, 10 deletions(-)
hw/vfio/common.c +0 -10
@@ -630,16 +630,6 @@ static void vfio_listener_region_add(MemoryListener *listener, goto fail; } -if (bcontainer->iova_ranges) { - ret =memory_region_iommu_set_iova_ranges (giommu->iommu_mr, - bcontainer->iova_ranges, - &err); -if (ret) { -g_free (giommu); - goto fail; - } - } - ret =memory_region_register_iommu_notifier (section->mr, &giommu->n, &err);if (ret) {
Signature · 1 line
2.41.0
PATCH v3 7/7 memory: Remove IOMMU MR iommu_set_iova_range API
In reply to Eric AugerSince the host IOVA ranges are now passed through the PCIIOMMUOps set_host_resv_regions and we have removed the only implementation of iommu_set_iova_range() in the virtio-iommu and the only call site in vfio/common, let's retire the IOMMU MR API and its memory wrapper.
--- include/exec/memory.h | 32 -------------------------------- system/memory.c | 13 ------------- 2 files changed, 45 deletions(-)
include/exec/memory.h +0 -32
@@ -530,26 +530,6 @@ struct IOMMUMemoryRegionClass {int (*iommu_set_page_size_mask)(IOMMUMemoryRegion *iommu,uint64_t page_size_mask,Error **errp); -/** -* @iommu_set_iova_ranges: -* -* Propagate information about the usable IOVA ranges for a given IOMMU -* memory region. Used for example to propagate host physical device -* reserved memory region constraints to the virtual IOMMU. -* -* Optional method: if this method is not provided, then the default IOVA -* aperture is used. -* -* @iommu: the IOMMUMemoryRegion -* -* @iova_ranges: list of ordered IOVA ranges (at least one range) -* -* Returns 0 on success, or a negative error. In case of failure, the error -* object must be created. -*/ -int (*iommu_set_iova_ranges)(IOMMUMemoryRegion *iommu, -GList *iova_ranges, -Error **errp); };typedef struct RamDiscardListener RamDiscardListener ;
@@ -1951,18 +1931,6 @@ int memory_region_iommu_set_page_size_mask(IOMMUMemoryRegion *iommu_mr,uint64_t page_size_mask, Error **errp); -/** -* memory_region_iommu_set_iova_ranges - Set the usable IOVA ranges -* for a given IOMMU MR region -* -* @iommu: IOMMU memory region -* @iova_ranges: list of ordered IOVA ranges (at least one range) -* @errp: pointer to Error*, to store an error if it happens. -*/ -int memory_region_iommu_set_iova_ranges (IOMMUMemoryRegion *iommu, -GList *iova_ranges, -Error **errp); -/** * memory_region_name: get a memory region's name *
system/memory.c +0 -13
@@ -1914,19 +1914,6 @@ int memory_region_iommu_set_page_size_mask(IOMMUMemoryRegion *iommu_mr,return ret; } -int memory_region_iommu_set_iova_ranges (IOMMUMemoryRegion *iommu_mr, -GList *iova_ranges, -Error **errp) -{ -IOMMUMemoryRegionClass *imrc =IOMMU_MEMORY_REGION_GET_CLASS (iommu_mr); -int ret =0 ; - -if (imrc->iommu_set_iova_ranges) { - ret = imrc->iommu_set_iova_ranges (iommu_mr, iova_ranges, errp); - } -return ret; -} -int memory_region_register_iommu_notifier (MemoryRegion *mr,IOMMUNotifier *n,Error **errp) {
Signature · 1 line
2.41.0