vfio: Adopt iommufd
Subsystems (2)
v2 cover vfio: Adopt iommufd
With the introduction of iommufd[1], the Linux kernel provides a generic interface for userspace drivers to propagate their DMA mappings to kernel for assigned devices. This series does the porting of the VFIO devices onto the /dev/iommu uapi and let it coexist with the legacy implementation. Other devices like vpda, vfio mdev and etc. are not considered yet.
At QEMU level, interactions with the /dev/iommu are abstracted by a new iommufd object (compiled in with the CONFIG_IOMMUFD option).
Any QEMU device (e.g. vfio device) wishing to use /dev/iommu must be linked with an iommufd object. In this series, the vfio-pci device is granted with such capability:
It gets a new optional parameter named iommufd which allows to pass an iommufd object:
-object iommufd,id=iommufd0
-device vfio-pci,host=0000:02:00.0,iommufd=iommufd0Note the /dev/iommu can be externally opened by a management layer. In such a case the fd is passed along with the iommufd object:
-object iommufd,id=iommufd0,fd=22
-device vfio-pci,host=0000:02:00.0,iommufd=iommufd0If the fd parameter is not passed, the fd (/dev/iommu) is opened by QEMU.
If no iommufd option is passed to the vfio-pci device, iommufd is not used and the end-user gets the behavior based on the legacy vfio iommu interfaces:
-device vfio-pci,host=0000:02:00.0
While the legacy kernel interface is group-centric, the new iommufd interface is device-centric, relying on device fd and iommufd.
To support both interfaces in the QEMU VFIO device we reworked the vfio container abstraction so that the generic VFIO code can use either backend.
The VFIOContainer object becomes a base object derived into
a) the legacy VFIO container and
b) the new iommufd based container.
The base object implements generic code such as code related to memory_listener and address space management whereas the derived objects implement callbacks specific to either BE, legacy and iommufd. Indeed each backend has its own way to setup secure context and dma management interface. The below diagram shows how it looks like with both BEs.
VFIO AddressSpace/Memory
+-------+ +----------+ +-----+ +-----+
| pci | | platform | | ap | | ccw |
+---+---+ +----+-----+ +--+--+ +--+--+ +----------------------+
| | | | | AddressSpace |
| | | | +------------+---------+
+---V-----------V-----------V--------V----+ /
| VFIOAddressSpace | <------------+
| | | MemoryListener
| VFIOContainer list |
+-------+----------------------------+----+
| |
| |
+-------V------+ +--------V----------+
| iommufd | | vfio legacy |
| container | | container |
+-------+------+ +--------+----------+
| |
| /dev/iommu | /dev/vfio/vfio
| /dev/vfio/devices/vfioX | /dev/vfio/$group_id
Userspace | |
===========+============================+================================
Kernel | device fd |
+---------------+ | group/container fd
| (BIND_IOMMUFD | | (SET_CONTAINER/SET_IOMMU)
| ATTACH_IOAS) | | device fd
| | |
| +-------V------------V-----------------+
iommufd | | vfio |
(map/unmap | +---------+--------------------+-------+
ioas_copy) | | | map/unmap
| | |
+------V------+ +-----V------+ +------V--------+
| iommfd core | | device | | vfio iommu |
+-------------+ +------------+ +---------------+[Secure Context setup]
- iommufd BE: uses device fd and iommufd to setup secure context (bind_iommufd, attach_ioas)
- vfio legacy BE: uses group fd and container fd to setup secure context (set_container, set_iommu)
[Device access]
- iommufd BE: device fd is opened through /dev/vfio/devices/vfioX
- vfio legacy BE: device fd is retrieved from group fd ioctl [DMA Mapping flow]
- VFIOAddressSpace receives MemoryRegion add/del via MemoryListener
- VFIO populates DMA map/unmap via the container BEs
*) iommufd BE: uses iommufd
*) vfio legacy BE: uses container fd
Test done:
- PCI and Platform device were tested
- ccw and ap were only compile-tested
- limited device hotplug test
- vIOMMU test run for both legacy and iommufd backends (limited tests)
This series was co-developed by Eric Auger and me based on the iommufd exploration kernel (https://github.com/luxis1999/iommufd/tree/iommufd-v5.17-rc6) which encompasses:
- Jason’s IOMMUFD Generic interface series
<https://lore.kernel.org/kvm/[email protected]/>
- Jason's vfio ccw life circle cleanup (Eric Farman has renewed it recently)
<https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/kvm/[email protected]/>
- Yi's move vfio_device allo/free into vfio core (not pushed to upstream yet)
- Yi's vfio device cdev (introduces device-centric interface, not pushed to upstream yet)
- Nicolin's vfio iommufd compat code (not pushed to upstream yet)
This QEMU series can be fount at https://github.com/luxis1999/qemu/tree/qemu-for-5.17-rc6-vm-rfcv2.
As iommufd kernel is in the early step (only iommufd generic interface is in mailing list), so this series hasn't made the iommufd backend fully on par with legacy backend w.r.t. features like:
- p2p mappings
- coherency tracking
- live migration
- vfio pci device hot reset
- and etc.
Only vfio-pci device supports iommufd BE, other types of devices still uses legacy BE.
TODOs:
- Add DMA alias check for iommufd BE (group level)
- Make pci.c to be BE agnostic. Needs kernel change as well to fix the VFIO_DEVICE_PCI_HOT_RESET gap
- Cleanup the VFIODevice fields as it's used in both backends
- Add device fd parameter to vfio-device in case the iommufd option is used
- Add locks
- Replace list with g_tree
- More tests
Change log:
v2:
- remove the first three patches of rfcv1
- add open cdev helper suggested by Jason
- remove the QOMification of the VFIOContainer and simply use standard ops (David)
- add "-object iommufd" suggested by Alex
v1: https://lore.kernel.org/qemu-devel/[email protected]/
Thanks,
Yi & Eric
Eric Auger (10):
scripts/update-linux-headers: Add iommufd.h
linux-headers: Import latest vfio.h and iommufd.h vfio/container: Introduce vfio_[attach/detach]_device vfio/platform: Use vfio_[attach/detach]_device vfio/ap: Use vfio_[attach/detach]_device
vfio/ccw: Use vfio_[attach/detach]_device vfio/container-base: Introduce [attach/detach]_device container
callbacks
vfio/container-base: Introduce VFIOContainer reset callback backends/iommufd: Introduce the iommufd object vfio/as: Allow the selection of a given iommu backend
Yi Liu (5):
vfio/common: Split common.c into common.c, container.c and as.c
vfio: Add base container
util/char_dev: Add open_cdev()
vfio/iommufd: Implement the iommufd backend vfio/iommufd: Add IOAS_COPY_DMA support
MAINTAINERS | 13 + backends/Kconfig | 5 + backends/iommufd.c | 265 +++ backends/meson.build | 1 + backends/trace-events | 12 + hw/vfio/ap.c | 62 +- hw/vfio/as.c | 1035 +++++++++++ hw/vfio/ccw.c | 118 +- hw/vfio/common.c | 2365 ++----------------------- hw/vfio/container-base.c | 172 ++ hw/vfio/container.c | 1298 ++++++++++++++ hw/vfio/iommufd.c | 546 ++++++ hw/vfio/meson.build | 6 + hw/vfio/migration.c | 5 +- hw/vfio/pci.c | 83 +- hw/vfio/platform.c | 42 +- hw/vfio/spapr.c | 22 +- hw/vfio/trace-events | 11 + include/hw/vfio/vfio-common.h | 101 +- include/hw/vfio/vfio-container-base.h | 148 ++ include/qemu/char_dev.h | 16 + include/sysemu/iommufd.h | 47 + linux-headers/linux/iommufd.h | 223 +++ linux-headers/linux/vfio.h | 84 + qapi/qom.json | 16 +- qemu-options.hx | 12 + scripts/update-linux-headers.sh | 2 +- util/chardev_open.c | 58 + util/meson.build | 1 + 29 files changed, 4291 insertions(+), 2478 deletions(-) create mode 100644 backends/iommufd.c create mode 100644 hw/vfio/as.c create mode 100644 hw/vfio/container-base.c create mode 100644 hw/vfio/container.c create mode 100644 hw/vfio/iommufd.c create mode 100644 include/hw/vfio/vfio-container-base.h create mode 100644 include/qemu/char_dev.h create mode 100644 include/sysemu/iommufd.h create mode 100644 linux-headers/linux/iommufd.h create mode 100644 util/chardev_open.c
Base commit: 9b1f58854959c5a9bdb347e3e04c252ab7fc9ef5
Signature · 1 line
2.27.0
RFC v2 1/15 scripts/update-linux-headers: Add iommufd.h
From: Eric Auger <[email protected]>
Update the script to import iommufd.h
--- scripts/update-linux-headers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/update-linux-headers.sh +1 -1
@@ -160,7 +160,7 @@ done rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" -for header in kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \ +for header in kvm.h vfio.h iommufd.h vfio_ccw.h vfio_zdev.h vhost.h \ psci.h psp-sev.h userfaultfd.h mman.h; do cp "$tmpdir/include/linux/$header" "$output/linux-headers/linux" done
Signature · 1 line
2.27.0
RFC v2 2/15 linux-headers: Import latest vfio.h and iommufd.h
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Imported from https://github.com/luxis1999/iommufd/tree/iommufd-v5.17-rc6
--- linux-headers/linux/iommufd.h | 223 ++++++++++++++++++++++++++++++++++ linux-headers/linux/vfio.h | 84 +++++++++++++ 2 files changed, 307 insertions(+) create mode 100644 linux-headers/linux/iommufd.h
linux-headers/linux/iommufd.h new file +223 -0
@@ -0,0 +1,223 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* Copyright (c) 2021-2022, NVIDIA CORPORATION & AFFILIATES. +*/ +#ifndef _IOMMUFD_H +#define _IOMMUFD_H + +#include <linux/types.h> +#include <linux/ioctl.h> + +#define IOMMUFD_TYPE (';') + +/** +* DOC: General ioctl format +* +* The ioctl mechanims follows a general format to allow for extensibility. Each +* ioctl is passed in a structure pointer as the argument providing the size of +* the structure in the first u32. The kernel checks that any structure space +* beyond what it understands is 0. This allows userspace to use the backward +* compatible portion while consistently using the newer, larger, structures. +* +* ioctls use a standard meaning for common errnos: +* +* - ENOTTY: The IOCTL number itself is not supported at all +* - E2BIG: The IOCTL number is supported, but the provided structure has +* non-zero in a part the kernel does not understand. +* - EOPNOTSUPP: The IOCTL number is supported, and the structure is +* understood, however a known field has a value the kernel does not +* understand or support. +* - EINVAL: Everything about the IOCTL was understood, but a field is not +* correct. +* - ENOENT: An ID or IOVA provided does not exist. +* - ENOMEM: Out of memory. +* - EOVERFLOW: Mathematics oveflowed. +* +* As well as additional errnos. within specific ioctls. +*/ +enum { +IOMMUFD_CMD_BASE =0x80 , +IOMMUFD_CMD_DESTROY =IOMMUFD_CMD_BASE , +IOMMUFD_CMD_IOAS_ALLOC , +IOMMUFD_CMD_IOAS_IOVA_RANGES , +IOMMUFD_CMD_IOAS_MAP , +IOMMUFD_CMD_IOAS_COPY , +IOMMUFD_CMD_IOAS_UNMAP , +IOMMUFD_CMD_VFIO_IOAS , +}; + +/** +* struct iommu_destroy - ioctl(IOMMU_DESTROY) +* @size: sizeof(struct iommu_destroy) +* @id: iommufd object ID to destroy. Can by any destroyable object type. +* +* Destroy any object held within iommufd. +*/ +struct iommu_destroy { +__u32 size; +__u32 id; +}; +#define IOMMU_DESTROY _IO(IOMMUFD_TYPE, IOMMUFD_CMD_DESTROY) + +/** +* struct iommu_ioas_alloc - ioctl(IOMMU_IOAS_ALLOC) +* @size: sizeof(struct iommu_ioas_alloc) +* @flags: Must be 0 +* @out_ioas_id: Output IOAS ID for the allocated object +* +* Allocate an IO Address Space (IOAS) which holds an IO Virtual Address (IOVA) +* to memory mapping. +*/ +struct iommu_ioas_alloc { +__u32 size; +__u32 flags; +__u32 out_ioas_id; +}; +#define IOMMU_IOAS_ALLOC _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_ALLOC) + +/** +* struct iommu_ioas_iova_ranges - ioctl(IOMMU_IOAS_IOVA_RANGES) +* @size: sizeof(struct iommu_ioas_iova_ranges) +* @ioas_id: IOAS ID to read ranges from +* @out_num_iovas: Output total number of ranges in the IOAS +* @__reserved: Must be 0 +* @out_valid_iovas: Array of valid IOVA ranges. The array length is the smaller +* of out_num_iovas or the length implied by size. +* @out_valid_iovas.start: First IOVA in the allowed range +* @out_valid_iovas.last: Inclusive last IOVA in the allowed range +* +* Query an IOAS for ranges of allowed IOVAs. Operation outside these ranges is +* not allowed. out_num_iovas will be set to the total number of iovas +* and the out_valid_iovas[] will be filled in as space permits. +* size should include the allocated flex array. +*/ +struct iommu_ioas_iova_ranges { +__u32 size; +__u32 ioas_id; +__u32 out_num_iovas; +__u32 __reserved; +struct iommu_valid_iovas { +__aligned_u64 start; +__aligned_u64 last; + } out_valid_iovas[]; +}; +#define IOMMU_IOAS_IOVA_RANGES _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_IOVA_RANGES) + +/** +* enum iommufd_ioas_map_flags - Flags for map and copy +* @IOMMU_IOAS_MAP_FIXED_IOVA: If clear the kernel will compute an appropriate +* IOVA to place the mapping at +* @IOMMU_IOAS_MAP_WRITEABLE: DMA is allowed to write to this mapping +* @IOMMU_IOAS_MAP_READABLE: DMA is allowed to read from this mapping +*/ +enum iommufd_ioas_map_flags { +IOMMU_IOAS_MAP_FIXED_IOVA =1 <<0 , +IOMMU_IOAS_MAP_WRITEABLE =1 <<1 , +IOMMU_IOAS_MAP_READABLE =1 <<2 , +}; + +/** +* struct iommu_ioas_map - ioctl(IOMMU_IOAS_MAP) +* @size: sizeof(struct iommu_ioas_map) +* @flags: Combination of enum iommufd_ioas_map_flags +* @ioas_id: IOAS ID to change the mapping of +* @__reserved: Must be 0 +* @user_va: Userspace pointer to start mapping from +* @length: Number of bytes to map +* @iova: IOVA the mapping was placed at. If IOMMU_IOAS_MAP_FIXED_IOVA is set +* then this must be provided as input. +* +* Set an IOVA mapping from a user pointer. If FIXED_IOVA is specified then the +* mapping will be established at iova, otherwise a suitable location will be +* automatically selected and returned in iova. +*/ +struct iommu_ioas_map { +__u32 size; +__u32 flags; +__u32 ioas_id; +__u32 __reserved; +__aligned_u64 user_va; +__aligned_u64 length; +__aligned_u64 iova; +}; +#define IOMMU_IOAS_MAP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP) + +/** +* struct iommu_ioas_copy - ioctl(IOMMU_IOAS_COPY) +* @size: sizeof(struct iommu_ioas_copy) +* @flags: Combination of enum iommufd_ioas_map_flags +* @dst_ioas_id: IOAS ID to change the mapping of +* @src_ioas_id: IOAS ID to copy from +* @length: Number of bytes to copy and map +* @dst_iova: IOVA the mapping was placed at. If IOMMU_IOAS_MAP_FIXED_IOVA is +* set then this must be provided as input. +* @src_iova: IOVA to start the copy +* +* Copy an already existing mapping from src_ioas_id and establish it in +* dst_ioas_id. The src iova/length must exactly match a range used with +* IOMMU_IOAS_MAP. +*/ +struct iommu_ioas_copy { +__u32 size; +__u32 flags; +__u32 dst_ioas_id; +__u32 src_ioas_id; +__aligned_u64 length; +__aligned_u64 dst_iova; +__aligned_u64 src_iova; +}; +#define IOMMU_IOAS_COPY _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_COPY) + +/** +* struct iommu_ioas_unmap - ioctl(IOMMU_IOAS_UNMAP) +* @size: sizeof(struct iommu_ioas_copy) +* @ioas_id: IOAS ID to change the mapping of +* @iova: IOVA to start the unmapping at +* @length: Number of bytes to unmap +* +* Unmap an IOVA range. The iova/length must exactly match a range +* used with IOMMU_IOAS_PAGETABLE_MAP, or be the values 0 & U64_MAX. +* In the latter case all IOVAs will be unmaped. +*/ +struct iommu_ioas_unmap { +__u32 size; +__u32 ioas_id; +__aligned_u64 iova; +__aligned_u64 length; +}; +#define IOMMU_IOAS_UNMAP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_UNMAP) + +/** +* enum iommufd_vfio_ioas_op +* @IOMMU_VFIO_IOAS_GET: Get the current compatibility IOAS +* @IOMMU_VFIO_IOAS_SET: Change the current compatibility IOAS +* @IOMMU_VFIO_IOAS_CLEAR: Disable VFIO compatibility +*/ +enum iommufd_vfio_ioas_op { +IOMMU_VFIO_IOAS_GET =0 , +IOMMU_VFIO_IOAS_SET =1 , +IOMMU_VFIO_IOAS_CLEAR =2 , +}; + +/** +* struct iommu_vfio_ioas - ioctl(IOMMU_VFIO_IOAS) +* @size: sizeof(struct iommu_ioas_copy) +* @ioas_id: For IOMMU_VFIO_IOAS_SET the input IOAS ID to set +* For IOMMU_VFIO_IOAS_GET will output the IOAS ID +* @op: One of enum iommufd_vfio_ioas_op +* @__reserved: Must be 0 +* +* The VFIO compatibility support uses a single ioas because VFIO APIs do not +* support the ID field. Set or Get the IOAS that VFIO compatibility will use. +* When VFIO_GROUP_SET_CONTAINER is used on an iommufd it will get the +* compatibility ioas, either by taking what is already set, or auto creating +* one. From then on VFIO will continue to use that ioas and is not effected by +* this ioctl. SET or CLEAR does not destroy any auto-created IOAS. +*/ +struct iommu_vfio_ioas { +__u32 size; +__u32 ioas_id; +__u16 op; +__u16 __reserved; +}; +#define IOMMU_VFIO_IOAS _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VFIO_IOAS) +#endif
linux-headers/linux/vfio.h +84 -0
90 lines not on this page · raw message
Signature · 1 line
2.27.0
RFC v2 3/15 vfio/common: Split common.c into common.c, container.c and as.c
In reply to Yi LiuBefore introducing the support for the new /dev/iommu backend in VFIO let's try to split common.c file into 3 parts:
- in common.c we keep backend agnostic code unrelated to dma mapping
- as.c is created and contains code related to VFIOAddressSpace and MemoryListeners. This code will be backend agnostic.
- container.c is created and will contain code related to the legacy VFIO backend (containers, groups, ...).
No functional change intended
--- hw/vfio/as.c | 893 +++++++++++++ hw/vfio/common.c | 2365 +++------------------------------ hw/vfio/container.c | 1193 +++++++++++++++++ hw/vfio/meson.build | 2 + include/hw/vfio/vfio-common.h | 28 + 5 files changed, 2303 insertions(+), 2178 deletions(-) create mode 100644 hw/vfio/as.c create mode 100644 hw/vfio/container.c
hw/vfio/as.c new file +893 -0
@@ -0,0 +1,893 @@ +/* +* generic functions used by VFIO devices +* +* Copyright Red Hat, Inc. 2012 +* +* Authors: +* Alex Williamson <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +* +* Based on qemu-kvm device-assignment: +* Adapted for KVM by Qumranet. +* Copyright (c) 2007, Neocleus, Alex Novik ([email protected]) +* Copyright (c) 2007, Neocleus, Guy Zana ([email protected]) +* Copyright (C) 2008, Qumranet, Amit Shah ([email protected]) +* Copyright (C) 2008, Red Hat, Amit Shah ([email protected]) +* Copyright (C) 2008, IBM, Muli Ben-Yehuda ([email protected]) +*/ + +#include "qemu/osdep.h" +#include <sys/ioctl.h> +#ifdef CONFIG_KVM +#include <linux/kvm.h> +#endif +#include <linux/vfio.h> + +#include "hw/vfio/vfio-common.h" +#include "hw/vfio/vfio.h" +#include "exec/address-spaces.h" +#include "exec/memory.h" +#include "exec/ram_addr.h" +#include "hw/hw.h" +#include "qemu/error-report.h" +#include "qemu/main-loop.h" +#include "qemu/range.h" +#include "sysemu/kvm.h" +#include "sysemu/reset.h" +#include "sysemu/runstate.h" +#include "trace.h" +#include "qapi/error.h" +#include "migration/migration.h" +#include "sysemu/tpm.h" + +static QLIST_HEAD(, VFIOAddressSpace) vfio_address_spaces = +QLIST_HEAD_INITIALIZER (vfio_address_spaces); + +void vfio_host_win_add (VFIOContainer *container, +hwaddr min_iova,hwaddr max_iova, +uint64_t iova_pgsizes) +{ +VFIOHostDMAWindow *hostwin; + + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { +if (ranges_overlap (hostwin->min_iova, + hostwin->max_iova - hostwin->min_iova +1 , + min_iova, + max_iova - min_iova +1 )) { +hw_error ("%s: Overlapped IOMMU are not enabled" , __func__); + } + } + + hostwin =g_malloc0 (sizeof (*hostwin)); + + hostwin->min_iova = min_iova; + hostwin->max_iova = max_iova; + hostwin->iova_pgsizes = iova_pgsizes; +QLIST_INSERT_HEAD (&container->hostwin_list, hostwin, hostwin_next); +} + +int vfio_host_win_del (VFIOContainer *container,hwaddr min_iova, +hwaddr max_iova) +{ +VFIOHostDMAWindow *hostwin; + + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { +if (hostwin->min_iova == min_iova && hostwin->max_iova == max_iova) { +QLIST_REMOVE (hostwin, hostwin_next); +g_free (hostwin); +return 0 ; + } + } + +return -1 ; +} + +static bool vfio_listener_skipped_section (MemoryRegionSection *section) +{ +return (!memory_region_is_ram (section->mr) && + !memory_region_is_iommu (section->mr)) || +memory_region_is_protected (section->mr) || +/* +* Sizing an enabled 64-bit BAR can cause spurious mappings to +* addresses in the upper part of the 64-bit address space. These +* are never accessed by the CPU and beyond the address width of +* some IOMMU hardware. TODO: VFIO should tell us the IOMMU width. +*/ + section->offset_within_address_space & (1ULL <<63 ); +} + +/* Called with rcu_read_lock held. */ +static bool vfio_get_xlat_addr (IOMMUTLBEntry *iotlb,void **vaddr, +ram_addr_t *ram_addr,bool *read_only) +{ +MemoryRegion *mr; +hwaddr xlat; +hwaddr len = iotlb->addr_mask +1 ; +bool writable = iotlb->perm &IOMMU_WO ; + +/* +* The IOMMU TLB entry we have just covers translation through +* this IOMMU to its immediate target. We need to translate +* it the rest of the way through to memory. +*/ + mr =address_space_translate (&address_space_memory, + iotlb->translated_addr, + &xlat, &len, writable, +MEMTXATTRS_UNSPECIFIED ); +if (!memory_region_is_ram (mr)) { +error_report ("iommu map to non memory area %" HWADDR_PRIx"" , + xlat); +return false; + }else if (memory_region_has_ram_discard_manager (mr)) { +RamDiscardManager *rdm =memory_region_get_ram_discard_manager (mr); +MemoryRegionSection tmp = { + .mr = mr, + .offset_within_region = xlat, + .size =int128_make64 (len), + }; + +/* +* Malicious VMs can map memory into the IOMMU, which is expected +* to remain discarded. vfio will pin all pages, populating memory. +* Disallow that. vmstate priorities make sure any RamDiscardManager +* were already restored before IOMMUs are restored. +*/ +if (!ram_discard_manager_is_populated (rdm, &tmp)) { +error_report ("iommu map to discarded memory (e.g., unplugged via" +" virtio-mem): %" HWADDR_PRIx"" , + iotlb->translated_addr); +return false; + } + +/* +* Malicious VMs might trigger discarding of IOMMU-mapped memory. The +* pages will remain pinned inside vfio until unmapped, resulting in a +* higher memory consumption than expected. If memory would get +* populated again later, there would be an inconsistency between pages +* pinned by vfio and pages seen by QEMU. This is the case until +* unmapped from the IOMMU (e.g., during device reset). +* +* With malicious guests, we really only care about pinning more memory +* than expected. RLIMIT_MEMLOCK set for the user/process can never be +* exceeded and can be used to mitigate this problem. +*/ +warn_report_once ("Using vfio with vIOMMUs and coordinated discarding of" +" RAM (e.g., virtio-mem) works, however, malicious" +" guests can trigger pinning of more memory than" +" intended via an IOMMU. It's possible to mitigate " +" by setting/adjusting RLIMIT_MEMLOCK." ); + } + +/* +* Translation truncates length to the IOMMU page size, +* check that it did not truncate too much. +*/ +if (len & iotlb->addr_mask) { +error_report ("iommu has granularity incompatible with target AS" ); +return false; + } + +if (vaddr) { + *vaddr =memory_region_get_ram_ptr (mr) + xlat; + } + +if (ram_addr) { + *ram_addr =memory_region_get_ram_addr (mr) + xlat; + } + +if (read_only) { + *read_only = !writable || mr->readonly; + } + +return true; +} + +static void vfio_iommu_map_notify (IOMMUNotifier *n,IOMMUTLBEntry *iotlb) +{ +VFIOGuestIOMMU *giommu =container_of (n, VFIOGuestIOMMU, n); +VFIOContainer *container = giommu->container; +hwaddr iova = iotlb->iova + giommu->iommu_offset; +void *vaddr; +int ret; + +trace_vfio_iommu_map_notify (iotlb->perm ==IOMMU_NONE ?"UNMAP" :"MAP" , + iova, iova + iotlb->addr_mask); + +if (iotlb->target_as != &address_space_memory) { +error_report ("Wrong target AS \"%s\", only system memory is allowed" , + iotlb->target_as->name ? iotlb->target_as->name :"none" ); +return ; + } + +rcu_read_lock (); + +if ((iotlb->perm &IOMMU_RW ) !=IOMMU_NONE ) { +bool read_only; + +if (!vfio_get_xlat_addr (iotlb, &vaddr,NULL , &read_only)) { + goto out; + } +/* +* vaddr is only valid until rcu_read_unlock(). But after +* vfio_dma_map has set up the mapping the pages will be +* pinned by the kernel. This makes sure that the RAM backend +* of vaddr will always be there, even if the memory object is +* destroyed and its backing memory munmap-ed. +*/ + ret =vfio_dma_map (container, iova, + iotlb->addr_mask +1 , vaddr, + read_only); +if (ret) { +error_report ("vfio_dma_map(%p, 0x%" HWADDR_PRIx", " +"0x%" HWADDR_PRIx", %p) = %d (%m)" , + container, iova, + iotlb->addr_mask +1 , vaddr, ret); + } + }else { + ret =vfio_dma_unmap (container, iova, iotlb->addr_mask +1 , iotlb); +if (ret) { +error_report ("vfio_dma_unmap(%p, 0x%" HWADDR_PRIx", " +"0x%" HWADDR_PRIx") = %d (%m)" , + container, iova, + iotlb->addr_mask +1 , ret); + } + } +out: +rcu_read_unlock (); +} + +static void vfio_ram_discard_notify_discard (RamDiscardListener *rdl, +MemoryRegionSection *section) +{ +VFIORamDiscardListener *vrdl =container_of (rdl, VFIORamDiscardListener, + listener); +const hwaddr size =int128_get64 (section->size); +const hwaddr iova = section->offset_within_address_space; +int ret; + +/* Unmap with a single call. */ + ret =vfio_dma_unmap (vrdl->container, iova, size ,NULL ); +if (ret) { +error_report ("%s: vfio_dma_unmap() failed: %s" , __func__, +strerror (-ret)); + } +} + +static int vfio_ram_discard_notify_populate (RamDiscardListener *rdl, +MemoryRegionSection *section) +{ +VFIORamDiscardListener *vrdl =container_of (rdl, VFIORamDiscardListener, + listener); +const hwaddr end = section->offset_within_region + +int128_get64 (section->size); +hwaddr start, next, iova; +void *vaddr; +int ret; + +/* +* Map in (aligned within memory region) minimum granularity, so we can +* unmap in minimum granularity later. +*/ +for (start = section->offset_within_region; start < end; start = next) { + next =ROUND_UP (start +1 , vrdl->granularity); + next =MIN (next, end); + + iova = start - section->offset_within_region + + section->offset_within_address_space; + vaddr =memory_region_get_ram_ptr (section->mr) + start; + + ret =vfio_dma_map (vrdl->container, iova, next - start, + vaddr, section->readonly); +if (ret) { +/* Rollback */ +vfio_ram_discard_notify_discard (rdl, section); +return ret; + } + } +return 0 ; +} + +static void vfio_register_ram_discard_listener (VFIOContainer *container, +MemoryRegionSection *section) +{ +RamDiscardManager *rdm =memory_region_get_ram_discard_manager (section->mr); +VFIORamDiscardListener *vrdl; + +/* Ignore some corner cases not relevant in practice. */ +g_assert (QEMU_IS_ALIGNED (section->offset_within_region,TARGET_PAGE_SIZE )); +g_assert (QEMU_IS_ALIGNED (section->offset_within_address_space, +TARGET_PAGE_SIZE )); +g_assert (QEMU_IS_ALIGNED (int128_get64 (section->size),TARGET_PAGE_SIZE )); + + vrdl =g_new0 (VFIORamDiscardListener,1 ); + vrdl->container = container; + vrdl->mr = section->mr; + vrdl->offset_within_address_space = section->offset_within_address_space; + vrdl->size =int128_get64 (section->size); + vrdl->granularity =ram_discard_manager_get_min_granularity (rdm, + section->mr); + +g_assert (vrdl->granularity &&is_power_of_2 (vrdl->granularity)); +g_assert (container->pgsizes && + vrdl->granularity >=1ULL <<ctz64 (container->pgsizes)); + +ram_discard_listener_init (&vrdl->listener, + vfio_ram_discard_notify_populate, + vfio_ram_discard_notify_discard, true); +ram_discard_manager_register_listener (rdm, &vrdl->listener, section); +QLIST_INSERT_HEAD (&container->vrdl_list, vrdl, next); + +/* +* Sanity-check if we have a theoretically problematic setup where we could +* exceed the maximum number of possible DMA mappings over time. We assume +* that each mapped section in the same address space as a RamDiscardManager +* section consumes exactly one DMA mapping, with the exception of +* RamDiscardManager sections; i.e., we don't expect to have gIOMMU sections +* in the same address space as RamDiscardManager sections. +* +* We assume that each section in the address space consumes one memslot. +* We take the number of KVM memory slots as a best guess for the maximum +* number of sections in the address space we could have over time, +* also consuming DMA mappings. +*/ +if (container->dma_max_mappings) { +unsigned int vrdl_count =0 , vrdl_mappings =0 , max_memslots =512 ; + +#ifdef CONFIG_KVM +if (kvm_enabled ()) { + max_memslots =kvm_get_max_memslots (); + } +#endif + + QLIST_FOREACH(vrdl, &container->vrdl_list, next) { +hwaddr start, end; + + start =QEMU_ALIGN_DOWN (vrdl->offset_within_address_space, + vrdl->granularity); + end =ROUND_UP (vrdl->offset_within_address_space + vrdl->size, + vrdl->granularity); + vrdl_mappings += (end - start) / vrdl->granularity; + vrdl_count++; + } + +if (vrdl_mappings + max_memslots - vrdl_count > + container->dma_max_mappings) { +warn_report ("%s: possibly running out of DMA mappings. E.g., try" +" increasing the 'block-size' of virtio-mem devies." +" Maximum possible DMA mappings: %d, Maximum possible" +" memslots: %d" , __func__, container->dma_max_mappings, + max_memslots); + } + } +} + +static void vfio_unregister_ram_discard_listener (VFIOContainer *container, +MemoryRegionSection *section) +{ +RamDiscardManager *rdm =memory_region_get_ram_discard_manager (section->mr); +VFIORamDiscardListener *vrdl =NULL ; + + QLIST_FOREACH(vrdl, &container->vrdl_list, next) { +if (vrdl->mr == section->mr && + vrdl->offset_within_address_space == + section->offset_within_address_space) { +break ; + } + } + +if (!vrdl) { +hw_error ("vfio: Trying to unregister missing RAM discard listener" ); + } + +ram_discard_manager_unregister_listener (rdm, &vrdl->listener); +QLIST_REMOVE (vrdl, next); +g_free (vrdl); +} + +static bool vfio_known_safe_misalignment (MemoryRegionSection *section) +{ +MemoryRegion *mr = section->mr; + +if (!TPM_IS_CRB (mr->owner)) { +return false; + } + +/* this is a known safe misaligned region, just trace for debug purpose */ +trace_vfio_known_safe_misalignment (memory_region_name (mr), + section->offset_within_address_space, + section->offset_within_region, +qemu_real_host_page_size ()); +return true; +} + +static void vfio_listener_region_add (MemoryListener *listener, +MemoryRegionSection *section) +{ +VFIOContainer *container =container_of (listener, VFIOContainer, listener); +hwaddr iova, end; +Int128 llend, llsize; +void *vaddr; +int ret; +VFIOHostDMAWindow *hostwin; +bool hostwin_found; +Error *err =NULL ; + +if (vfio_listener_skipped_section (section)) { +trace_vfio_listener_region_add_skip ( + section->offset_within_address_space, + section->offset_within_address_space + +int128_get64 (int128_sub (section->size,int128_one ()))); +return ; + } + +if (unlikely ((section->offset_within_address_space & + ~qemu_real_host_page_mask ()) != + (section->offset_within_region & ~qemu_real_host_page_mask ()))) { +if (!vfio_known_safe_misalignment (section)) { +error_report ("%s received unaligned region %s iova=0x%" PRIx64 +" offset_within_region=0x%" PRIx64 +" qemu_real_host_page_size=0x%" PRIxPTR, + __func__,memory_region_name (section->mr), + section->offset_within_address_space, + section->offset_within_region, +qemu_real_host_page_size ()); + } +return ; + } + + iova =REAL_HOST_PAGE_ALIGN (section->offset_within_address_space); + llend =int128_make64 (section->offset_within_address_space); + llend =int128_add (llend, section->size); + llend =int128_and (llend,int128_exts64 (qemu_real_host_page_mask ())); + +if (int128_ge (int128_make64 (iova), llend)) { +if (memory_region_is_ram_device (section->mr)) { +trace_vfio_listener_region_add_no_dma_map ( +memory_region_name (section->mr), + section->offset_within_address_space, +int128_getlo (section->size), +qemu_real_host_page_size ()); + } +return ; + } + end =int128_get64 (int128_sub (llend,int128_one ())); + +if (vfio_container_add_section_window (container, section, &err)) { + goto fail; + } + + hostwin_found = false; + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { +if (hostwin->min_iova <= iova && end <= hostwin->max_iova) { + hostwin_found = true; +break ; + } + } + +if (!hostwin_found) { +error_setg (&err,"Container %p can't map guest IOVA region" +" 0x%" HWADDR_PRIx"..0x%" HWADDR_PRIx, container, iova, end); + goto fail; + } + +memory_region_ref (section->mr); + +if (memory_region_is_iommu (section->mr)) { +VFIOGuestIOMMU *giommu; +IOMMUMemoryRegion *iommu_mr =IOMMU_MEMORY_REGION (section->mr); +int iommu_idx; + +trace_vfio_listener_region_add_iommu (iova, end); +/* +* FIXME: For VFIO iommu types which have KVM acceleration to +* avoid bouncing all map/unmaps through qemu this way, this +* would be the right place to wire that up (tell the KVM +* device emulation the VFIO iommu handles to use). +*/ + giommu =g_malloc0 (sizeof (*giommu)); + giommu->iommu_mr = iommu_mr; + giommu->iommu_offset = section->offset_within_address_space - + section->offset_within_region; + giommu->container = container; + llend =int128_add (int128_make64 (section->offset_within_region), + section->size); + llend =int128_sub (llend,int128_one ()); + iommu_idx =memory_region_iommu_attrs_to_index (iommu_mr, +MEMTXATTRS_UNSPECIFIED ); +iommu_notifier_init (&giommu->n, vfio_iommu_map_notify, +IOMMU_NOTIFIER_IOTLB_EVENTS , + section->offset_within_region, +int128_get64 (llend), + iommu_idx); + + ret =memory_region_iommu_set_page_size_mask (giommu->iommu_mr, + container->pgsizes, + &err); +if (ret) { +g_free (giommu); + goto fail; + } + + ret =memory_region_register_iommu_notifier (section->mr, &giommu->n, + &err); +if (ret) { +g_free (giommu); + goto fail; + } +QLIST_INSERT_HEAD (&container->giommu_list, giommu, giommu_next); +memory_region_iommu_replay (giommu->iommu_mr, &giommu->n); + +return ; + } + +/* Here we assume that memory_region_is_ram(section->mr)==true */ + +/* +* For RAM memory regions with a RamDiscardManager, we only want to map the +* actually populated parts - and update the mapping whenever we're notified +* about changes. +*/ +if (memory_region_has_ram_discard_manager (section->mr)) { +vfio_register_ram_discard_listener (container, section); +return ; + } + + vaddr =memory_region_get_ram_ptr (section->mr) + + section->offset_within_region + + (iova - section->offset_within_address_space); + +trace_vfio_listener_region_add_ram (iova, end, vaddr); + + llsize =int128_sub (llend,int128_make64 (iova)); + +if (memory_region_is_ram_device (section->mr)) { +hwaddr pgmask = (1ULL <<ctz64 (hostwin->iova_pgsizes)) -1 ; + +if ((iova & pgmask) || (int128_get64 (llsize) & pgmask)) { +trace_vfio_listener_region_add_no_dma_map ( +memory_region_name (section->mr), + section->offset_within_address_space, +int128_getlo (section->size), + pgmask +1 ); +return ; + } + } + + ret =vfio_dma_map (container, iova,int128_get64 (llsize), + vaddr, section->readonly); +if (ret) { +error_setg (&err,"vfio_dma_map(%p, 0x%" HWADDR_PRIx", " +"0x%" HWADDR_PRIx", %p) = %d (%m)" , + container, iova,int128_get64 (llsize), vaddr, ret); +if (memory_region_is_ram_device (section->mr)) { +/* Allow unexpected mappings not to be fatal for RAM devices */ +error_report_err (err); +return ; + } + goto fail; + } + +return ; + +fail: +if (memory_region_is_ram_device (section->mr)) { +error_report ("failed to vfio_dma_map. pci p2p may not work" ); +return ; + } +/* +* On the initfn path, store the first error in the container so we +* can gracefully fail. Runtime, there's not much we can do other +* than throw a hardware error. +*/ +if (!container->initialized) { +if (!container->error) { +error_propagate_prepend (&container->error, err, +"Region %s: " , +memory_region_name (section->mr)); + }else { +error_free (err); + } + }else { +error_report_err (err); +hw_error ("vfio: DMA mapping failed, unable to continue" ); + } +} + +static void vfio_listener_region_del (MemoryListener *listener, +MemoryRegionSection *section) +{ +VFIOContainer *container =container_of (listener, VFIOContainer, listener); +hwaddr iova, end; +Int128 llend, llsize; +int ret; +bool try_unmap = true; + +if (vfio_listener_skipped_section (section)) { +trace_vfio_listener_region_del_skip ( + section->offset_within_address_space, + section->offset_within_address_space + +int128_get64 (int128_sub (section->size,int128_one ()))); +return ; + } + +if (unlikely ((section->offset_within_address_space & + ~qemu_real_host_page_mask ()) != + (section->offset_within_region & ~qemu_real_host_page_mask ()))) { +error_report ("%s received unaligned region" , __func__); +return ; + } + +if (memory_region_is_iommu (section->mr)) { +VFIOGuestIOMMU *giommu; + + QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { +if (MEMORY_REGION (giommu->iommu_mr) == section->mr && + giommu->n.start == section->offset_within_region) { +memory_region_unregister_iommu_notifier (section->mr, + &giommu->n); +QLIST_REMOVE (giommu, giommu_next); +g_free (giommu); +break ; + } + } + +/* +* FIXME: We assume the one big unmap below is adequate to +* remove any individual page mappings in the IOMMU which +* might have been copied into VFIO. This works for a page table +* based IOMMU where a big unmap flattens a large range of IO-PTEs. +* That may not be true for all IOMMU types. +*/ + } + + iova =REAL_HOST_PAGE_ALIGN (section->offset_within_address_space); + llend =int128_make64 (section->offset_within_address_space); + llend =int128_add (llend, section->size); + llend =int128_and (llend,int128_exts64 (qemu_real_host_page_mask ())); + +if (int128_ge (int128_make64 (iova), llend)) { +return ; + } + end =int128_get64 (int128_sub (llend,int128_one ())); + + llsize =int128_sub (llend,int128_make64 (iova)); + +trace_vfio_listener_region_del (iova, end); + +if (memory_region_is_ram_device (section->mr)) { +hwaddr pgmask; +VFIOHostDMAWindow *hostwin; +bool hostwin_found = false; + + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { +if (hostwin->min_iova <= iova && end <= hostwin->max_iova) { + hostwin_found = true; +break ; + } + } +assert (hostwin_found);/* or region_add() would have failed */ + + pgmask = (1ULL <<ctz64 (hostwin->iova_pgsizes)) -1 ; + try_unmap = !((iova & pgmask) || (int128_get64 (llsize) & pgmask)); + }else if (memory_region_has_ram_discard_manager (section->mr)) { +vfio_unregister_ram_discard_listener (container, section); +/* Unregistering will trigger an unmap. */ + try_unmap = false; + } + +if (try_unmap) { +if (int128_eq (llsize,int128_2_64 ())) { +/* The unmap ioctl doesn't accept a full 64-bit span. */ + llsize =int128_rshift (llsize,1 ); + ret =vfio_dma_unmap (container, iova,int128_get64 (llsize),NULL ); +if (ret) { +error_report ("vfio_dma_unmap(%p, 0x%" HWADDR_PRIx", " +"0x%" HWADDR_PRIx") = %d (%m)" , + container, iova,int128_get64 (llsize), ret); + } + iova +=int128_get64 (llsize); + } + ret =vfio_dma_unmap (container, iova,int128_get64 (llsize),NULL ); +if (ret) { +error_report ("vfio_dma_unmap(%p, 0x%" HWADDR_PRIx", " +"0x%" HWADDR_PRIx") = %d (%m)" , + container, iova,int128_get64 (llsize), ret); + } + } + +memory_region_unref (section->mr); + +vfio_container_del_section_window (container, section); +} + +static void vfio_listener_log_global_start (MemoryListener *listener) +{ +VFIOContainer *container =container_of (listener, VFIOContainer, listener); + +vfio_set_dirty_page_tracking (container, true); +} + +static void vfio_listener_log_global_stop (MemoryListener *listener) +{ +VFIOContainer *container =container_of (listener, VFIOContainer, listener); + +vfio_set_dirty_page_tracking (container, false); +} + +typedef struct { +IOMMUNotifier n; +VFIOGuestIOMMU *giommu; +}vfio_giommu_dirty_notifier ; + +static void vfio_iommu_map_dirty_notify (IOMMUNotifier *n,IOMMUTLBEntry *iotlb) +{ +vfio_giommu_dirty_notifier *gdn =container_of (n, + vfio_giommu_dirty_notifier, n); +VFIOGuestIOMMU *giommu = gdn->giommu; +VFIOContainer *container = giommu->container; +hwaddr iova = iotlb->iova + giommu->iommu_offset; +ram_addr_t translated_addr; + +trace_vfio_iommu_map_dirty_notify (iova, iova + iotlb->addr_mask); + +if (iotlb->target_as != &address_space_memory) { +error_report ("Wrong target AS \"%s\", only system memory is allowed" , + iotlb->target_as->name ? iotlb->target_as->name :"none" ); +return ; + } + +rcu_read_lock (); +if (vfio_get_xlat_addr (iotlb,NULL , &translated_addr,NULL )) { +int ret; + + ret =vfio_get_dirty_bitmap (container, iova, iotlb->addr_mask +1 , + translated_addr); +if (ret) { +error_report ("vfio_iommu_map_dirty_notify(%p, 0x%" HWADDR_PRIx", " +"0x%" HWADDR_PRIx") = %d (%m)" , + container, iova, + iotlb->addr_mask +1 , ret); + } + } +rcu_read_unlock (); +} + +static int vfio_ram_discard_get_dirty_bitmap (MemoryRegionSection *section, +void *opaque) +{ +const hwaddr size =int128_get64 (section->size); +const hwaddr iova = section->offset_within_address_space; +const ram_addr_t ram_addr =memory_region_get_ram_addr (section->mr) + + section->offset_within_region; +VFIORamDiscardListener *vrdl = opaque; + +/* +* Sync the whole mapped region (spanning multiple individual mappings) +* in one go. +*/ +return vfio_get_dirty_bitmap (vrdl->container, iova, size, ram_addr); +} + +static int vfio_sync_ram_discard_listener_dirty_bitmap (VFIOContainer *container, +MemoryRegionSection *section) +{ +RamDiscardManager *rdm =memory_region_get_ram_discard_manager (section->mr); +VFIORamDiscardListener *vrdl =NULL ; + + QLIST_FOREACH(vrdl, &container->vrdl_list, next) { +if (vrdl->mr == section->mr && + vrdl->offset_within_address_space == + section->offset_within_address_space) { +break ; + } + } + +if (!vrdl) { +hw_error ("vfio: Trying to sync missing RAM discard listener" ); + } + +/* +* We only want/can synchronize the bitmap for actually mapped parts - +* which correspond to populated parts. Replay all populated parts. +*/ +return ram_discard_manager_replay_populated (rdm, section, + vfio_ram_discard_get_dirty_bitmap, + &vrdl); +} + +static int vfio_sync_dirty_bitmap (VFIOContainer *container, +MemoryRegionSection *section) +{ +ram_addr_t ram_addr; + +if (memory_region_is_iommu (section->mr)) { +VFIOGuestIOMMU *giommu; + + QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { +if (MEMORY_REGION (giommu->iommu_mr) == section->mr && + giommu->n.start == section->offset_within_region) { +Int128 llend; +vfio_giommu_dirty_notifier gdn = { .giommu = giommu }; +int idx =memory_region_iommu_attrs_to_index (giommu->iommu_mr, +MEMTXATTRS_UNSPECIFIED ); + + llend =int128_add (int128_make64 (section->offset_within_region), + section->size); + llend =int128_sub (llend,int128_one ()); + +iommu_notifier_init (&gdn.n, + vfio_iommu_map_dirty_notify, +IOMMU_NOTIFIER_MAP , + section->offset_within_region, +int128_get64 (llend), + idx); +memory_region_iommu_replay (giommu->iommu_mr, &gdn.n); +break ; + } + } +return 0 ; + }else if (memory_region_has_ram_discard_manager (section->mr)) { +return vfio_sync_ram_discard_listener_dirty_bitmap (container, section); + } + + ram_addr =memory_region_get_ram_addr (section->mr) + + section->offset_within_region; + +return vfio_get_dirty_bitmap (container, +REAL_HOST_PAGE_ALIGN (section->offset_within_address_space), +int128_get64 (section->size), ram_addr); +} + +static void vfio_listener_log_sync (MemoryListener *listener, +MemoryRegionSection *section) +{ +VFIOContainer *container =container_of (listener, VFIOContainer, listener); + +if (vfio_listener_skipped_section (section) || + !container->dirty_pages_supported) { +return ; + } + +if (vfio_devices_all_dirty_tracking (container)) { +vfio_sync_dirty_bitmap (container, section); + } +} + +const MemoryListener vfio_memory_listener = { + .name ="vfio" , + .region_add = vfio_listener_region_add, + .region_del = vfio_listener_region_del, + .log_global_start = vfio_listener_log_global_start, + .log_global_stop = vfio_listener_log_global_stop, + .log_sync = vfio_listener_log_sync, +}; + +VFIOAddressSpace *vfio_get_address_space (AddressSpace *as) +{ +VFIOAddressSpace *space; + + QLIST_FOREACH(space, &vfio_address_spaces, list) { +if (space->as == as) { +return space; + } + } + +/* No suitable VFIOAddressSpace, create a new one */ + space =g_malloc0 (sizeof (*space)); + space->as = as; +QLIST_INIT (&space->containers); + +QLIST_INSERT_HEAD (&vfio_address_spaces, space, list); + +return space; +} + +void vfio_put_address_space (VFIOAddressSpace *space) +{ +if (QLIST_EMPTY (&space->containers)) { +QLIST_REMOVE (space, list); +g_free (space); + } +}
hw/vfio/common.c +187 -2178
2,107 lines not on this page · raw message
hw/vfio/container.c new file +1193 -0
1,193 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.27.0
Part of the diff is not shown here. View the raw message.
RFC v2 4/15 vfio: Add base container
In reply to Yi LiuAbstract the VFIOContainer to be a base object. It is supposed to be embedded by legacy VFIO container and later on, into the new iommufd based container.
The base container implements generic code such as code related to memory_listener and address space management. The VFIOContainerOps implements callbacks that depend on the kernel user space being used.
'as.c' only manipulates the base container with wrapper functions that calls the functions defined in VFIOContainerOps. Existing 'container.c' code is converted to implement the legacy container ops functions.
Existing migration code only works with the legacy container. Also 'spapr.c' isn't BE agnostic.
Below is the base container. It's named as VFIOContainer, old VFIOContainer is replaced with VFIOLegacyContainer.
struct VFIOContainer {
VFIOContainerOps *ops;
VFIOAddressSpace *space;
MemoryListener listener;
Error *error;
bool initialized;
bool dirty_pages_supported;
uint64_t dirty_pgsizes;
uint64_t max_dirty_bitmap_size;
unsigned long pgsizes;
unsigned int dma_max_mappings;
QLIST_HEAD(, VFIOGuestIOMMU) giommu_list;
QLIST_HEAD(, VFIOHostDMAWindow) hostwin_list;
QLIST_HEAD(, VFIORamDiscardListener) vrdl_list;
QLIST_ENTRY(VFIOContainer) next;
};struct VFIOLegacyContainer {
VFIOContainer bcontainer;
int fd; /* /dev/vfio/vfio, empowered by the attached groups */
MemoryListener prereg_listener;
unsigned iommu_type;
QLIST_HEAD(, VFIOGroup) group_list;
};---
v1 -> v2:
- Remove QOM for VFIOContainer object, use callback instead per David's comment.
- Rename container-obj.c/.h to be container-base.c/.h
--- hw/vfio/as.c | 48 +++--- hw/vfio/container-base.c | 154 +++++++++++++++++++ hw/vfio/container.c | 211 +++++++++++++++----------- hw/vfio/meson.build | 1 + hw/vfio/migration.c | 5 +- hw/vfio/pci.c | 4 +- hw/vfio/spapr.c | 22 +-- include/hw/vfio/vfio-common.h | 79 ++-------- include/hw/vfio/vfio-container-base.h | 136 +++++++++++++++++ 9 files changed, 470 insertions(+), 190 deletions(-) create mode 100644 hw/vfio/container-base.c create mode 100644 include/hw/vfio/vfio-container-base.h
hw/vfio/as.c +28 -20
@@ -216,9 +216,9 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb)* of vaddr will always be there, even if the memory object is * destroyed and its backing memory munmap-ed. */ - ret =vfio_dma_map (container, iova, - iotlb->addr_mask +1 , vaddr, - read_only); + ret =vfio_container_dma_map (container, iova, + iotlb->addr_mask +1 , vaddr, + read_only);if (ret) { error_report("vfio_dma_map(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx", %p) = %d (%m)",
@@ -226,7 +226,8 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) iotlb->addr_mask + 1, vaddr, ret); } }else { - ret =vfio_dma_unmap (container, iova, iotlb->addr_mask +1 , iotlb); + ret =vfio_container_dma_unmap (container, iova, + iotlb->addr_mask +1 , iotlb);if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx") = %d (%m)",
@@ -243,12 +244,13 @@ static void vfio_ram_discard_notify_discard(RamDiscardListener *rdl, {VFIORamDiscardListener *vrdl =container_of (rdl, VFIORamDiscardListener, listener); +VFIOContainer *container = vrdl->container;const hwaddr size =int128_get64 (section->size);const hwaddr iova = section->offset_within_address_space;int ret;/* Unmap with a single call. */ - ret =vfio_dma_unmap (vrdl->container, iova, size ,NULL ); + ret =vfio_container_dma_unmap (container, iova, size ,NULL );if (ret) {error_report ("%s: vfio_dma_unmap() failed: %s" , __func__,strerror (-ret));
@@ -260,6 +262,7 @@ static int vfio_ram_discard_notify_populate(RamDiscardListener *rdl, {VFIORamDiscardListener *vrdl =container_of (rdl, VFIORamDiscardListener, listener); +VFIOContainer *container = vrdl->container;const hwaddr end = section->offset_within_region +int128_get64 (section->size);hwaddr start, next, iova;
@@ -278,8 +281,8 @@ static int vfio_ram_discard_notify_populate(RamDiscardListener *rdl, section->offset_within_address_space; vaddr =memory_region_get_ram_ptr (section->mr) + start; - ret =vfio_dma_map (vrdl->container, iova, next - start, - vaddr, section->readonly); + ret =vfio_container_dma_map (container, iova, next - start, + vaddr, section->readonly);if (ret) {/* Rollback */ vfio_ram_discard_notify_discard (rdl, section);
@@ -555,8 +558,8 @@ static void vfio_listener_region_add(MemoryListener *listener, } } - ret =vfio_dma_map (container, iova,int128_get64 (llsize), - vaddr, section->readonly); + ret =vfio_container_dma_map (container, iova,int128_get64 (llsize), + vaddr, section->readonly);if (ret) { error_setg(&err, "vfio_dma_map(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx", %p) = %d (%m)",
@@ -681,7 +684,8 @@ static void vfio_listener_region_del(MemoryListener *listener,if (int128_eq (llsize,int128_2_64 ())) {/* The unmap ioctl doesn't accept a full 64-bit span. */ llsize =int128_rshift (llsize,1 ); - ret =vfio_dma_unmap (container, iova,int128_get64 (llsize),NULL ); + ret =vfio_container_dma_unmap (container, iova, +int128_get64 (llsize),NULL );if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx") = %d (%m)",
@@ -689,7 +693,8 @@ static void vfio_listener_region_del(MemoryListener *listener, } iova +=int128_get64 (llsize); } - ret =vfio_dma_unmap (container, iova,int128_get64 (llsize),NULL ); + ret =vfio_container_dma_unmap (container, iova, +int128_get64 (llsize),NULL );if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx") = %d (%m)",
@@ -706,14 +711,14 @@ static void vfio_listener_log_global_start(MemoryListener *listener) {VFIOContainer *container =container_of (listener, VFIOContainer, listener); -vfio_set_dirty_page_tracking (container, true); +vfio_container_set_dirty_page_tracking (container, true); }static void vfio_listener_log_global_stop (MemoryListener *listener) {VFIOContainer *container =container_of (listener, VFIOContainer, listener); -vfio_set_dirty_page_tracking (container, false); +vfio_container_set_dirty_page_tracking (container, false); } typedef struct {
@@ -742,8 +747,9 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb)if (vfio_get_xlat_addr (iotlb,NULL , &translated_addr,NULL )) {int ret; - ret =vfio_get_dirty_bitmap (container, iova, iotlb->addr_mask +1 , - translated_addr); + ret =vfio_container_get_dirty_bitmap (container, iova, + iotlb->addr_mask +1 , + translated_addr);if (ret) { error_report("vfio_iommu_map_dirty_notify(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx") = %d (%m)",
@@ -767,11 +773,13 @@ static int vfio_ram_discard_get_dirty_bitmap(MemoryRegionSection *section,* Sync the whole mapped region (spanning multiple individual mappings) * in one go. */ -return vfio_get_dirty_bitmap (vrdl->container, iova, size, ram_addr); +return vfio_container_get_dirty_bitmap (vrdl->container, iova, + size, ram_addr); } -static int vfio_sync_ram_discard_listener_dirty_bitmap (VFIOContainer *container, -MemoryRegionSection *section) +static int +vfio_sync_ram_discard_listener_dirty_bitmap (VFIOContainer *container, +MemoryRegionSection *section) {RamDiscardManager *rdm =memory_region_get_ram_discard_manager (section->mr);VFIORamDiscardListener *vrdl =NULL ;
@@ -835,7 +843,7 @@ static int vfio_sync_dirty_bitmap(VFIOContainer *container, ram_addr =memory_region_get_ram_addr (section->mr) + section->offset_within_region; -return vfio_get_dirty_bitmap (container, +return vfio_container_get_dirty_bitmap (container,REAL_HOST_PAGE_ALIGN (section->offset_within_address_space),int128_get64 (section->size), ram_addr); }
@@ -850,7 +858,7 @@ static void vfio_listener_log_sync(MemoryListener *listener,return ; } -if (vfio_devices_all_dirty_tracking (container)) { +if (vfio_container_devices_all_dirty_tracking (container)) {vfio_sync_dirty_bitmap (container, section); } }
hw/vfio/container-base.c new file +154 -0
154 lines not on this page · raw message
hw/vfio/container.c +124 -87
495 lines not on this page · raw message
hw/vfio/meson.build +1 -0
@@ -2,6 +2,7 @@ vfio_ss = ss.source_set() vfio_ss.add(files( 'common.c', 'as.c', + 'container-base.c', 'container.c', 'spapr.c', 'migration.c',
hw/vfio/migration.c +3 -2
@@ -857,11 +857,12 @@ int64_t vfio_mig_bytes_transferred(void)int vfio_migration_probe (VFIODevice *vbasedev,Error **errp) { -VFIOContainer *container = vbasedev->group->container; +VFIOLegacyContainer *container = vbasedev->group->container;struct vfio_region_info *info =NULL ;int ret = -ENOTSUP ; -if (!vbasedev->enable_migration || !container->dirty_pages_supported) { +if (!vbasedev->enable_migration || + !container->bcontainer.dirty_pages_supported) { goto add_blocker; }
hw/vfio/pci.c +3 -1
@@ -3144,7 +3144,9 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) } } -if (!pdev->failover_pair_id) { +if (!pdev->failover_pair_id && +vfio_container_check_extension (&vbasedev->group->container->bcontainer, +VFIO_FEAT_LIVE_MIGRATION )) { ret =vfio_migration_probe (vbasedev, errp);if (ret) {error_report ("%s: Migration disabled" , vbasedev->name);
hw/vfio/spapr.c +11 -11
@@ -39,8 +39,8 @@ static void *vfio_prereg_gpa_to_vaddr(MemoryRegionSection *section, hwaddr gpa)static void vfio_prereg_listener_region_add (MemoryListener *listener,MemoryRegionSection *section) { -VFIOContainer *container =container_of (listener, VFIOContainer, - prereg_listener); +VFIOLegacyContainer *container =container_of (listener, VFIOLegacyContainer, + prereg_listener);const hwaddr gpa = section->offset_within_address_space;hwaddr end;int ret;
@@ -83,9 +83,9 @@ static void vfio_prereg_listener_region_add(MemoryListener *listener,* can gracefully fail. Runtime, there's not much we can do other * than throw a hardware error. */ -if (!container->initialized) { -if (!container->error) { -error_setg_errno (&container->error, -ret, +if (!container->bcontainer.initialized) { +if (!container->bcontainer.error) { +error_setg_errno (&container->bcontainer.error, -ret,"Memory registering failed" ); } }else {
@@ -97,8 +97,8 @@ static void vfio_prereg_listener_region_add(MemoryListener *listener,static void vfio_prereg_listener_region_del (MemoryListener *listener,MemoryRegionSection *section) { -VFIOContainer *container =container_of (listener, VFIOContainer, - prereg_listener); +VFIOLegacyContainer *container =container_of (listener, VFIOLegacyContainer, + prereg_listener);const hwaddr gpa = section->offset_within_address_space;hwaddr end;int ret;
@@ -141,7 +141,7 @@ const MemoryListener vfio_prereg_listener = { .region_del = vfio_prereg_listener_region_del, }; -int vfio_spapr_create_window (VFIOContainer *container, +int vfio_spapr_create_window (VFIOLegacyContainer *container,MemoryRegionSection *section,hwaddr *pgsize) {
@@ -159,13 +159,13 @@ int vfio_spapr_create_window(VFIOContainer *container,if (pagesize > rampagesize) { pagesize = rampagesize; } - pgmask = container->pgsizes & (pagesize | (pagesize -1 )); + pgmask = container->bcontainer.pgsizes & (pagesize | (pagesize -1 )); pagesize = pgmask ? (1ULL << (63 -clz64 (pgmask))) :0 ;if (!pagesize) {error_report ("Host doesn't support page size 0x%" PRIx64", the supported mask is 0x%lx" ,memory_region_iommu_get_min_page_size (iommu_mr), - container->pgsizes); + container->bcontainer.pgsizes);return -EINVAL ; }
@@ -233,7 +233,7 @@ int vfio_spapr_create_window(VFIOContainer *container,return 0 ; } -int vfio_spapr_remove_window (VFIOContainer *container, +int vfio_spapr_remove_window (VFIOLegacyContainer *container,hwaddr offset_within_address_space) {struct vfio_iommu_spapr_tce_remove remove = {
include/hw/vfio/vfio-common.h +10 -69
126 lines not on this page · raw message
include/hw/vfio/vfio-container-base.h new file +136 -0
136 lines not on this page · raw message
Signature · 1 line
2.27.0
RFC v2 5/15 vfio/container: Introduce vfio_[attach/detach]_device
In reply to Yi LiuFrom: Eric Auger <[email protected]>
We want the VFIO devices to be able to use two different IOMMU callbacks, the legacy VFIO one and the new iommufd one.
Introduce vfio_[attach/detach]_device which aim at hiding the underlying IOMMU backend (IOCTLs, datatypes, ...).
Once vfio_attach_device completes, the device is attached to a security context and its fd can be used. Conversely When vfio_detach_device completes, the device has been detached to the security context.
In this patch, only the vfio-pci device gets converted to use the new API. Subsequent patches will handle other devices.
--- hw/vfio/container.c | 65 +++++++++++++++++++++++++++++++++++ hw/vfio/pci.c | 50 +++------------------------ include/hw/vfio/vfio-common.h | 2 ++ 3 files changed, 72 insertions(+), 45 deletions(-)
hw/vfio/container.c +65 -0
@@ -1218,6 +1218,71 @@ int vfio_eeh_as_op(AddressSpace *as, uint32_t op)return vfio_eeh_container_op (container, op); } +static int vfio_device_groupid (VFIODevice *vbasedev,Error **errp) +{ +char *tmp, group_path[PATH_MAX ], *group_name; +int ret, groupid; +ssize_t len; + + tmp =g_strdup_printf ("%s/iommu_group" , vbasedev->sysfsdev); + len =readlink (tmp, group_path,sizeof (group_path)); +g_free (tmp); + +if (len <=0 || len >=sizeof (group_path)) { + ret = len <0 ? -errno : -ENAMETOOLONG ; +error_setg_errno (errp, -ret,"no iommu_group found" ); +return ret; + } + + group_path[len] =0 ; + + group_name =basename (group_path); +if (sscanf (group_name,"%d" , &groupid) !=1 ) { +error_setg_errno (errp, errno,"failed to read %s" , group_path); +return -errno; + } +return groupid; +} + +int vfio_attach_device (VFIODevice *vbasedev,AddressSpace *as,Error **errp) +{ +int groupid =vfio_device_groupid (vbasedev, errp); +VFIODevice *vbasedev_iter; +VFIOGroup *group; +int ret; + +if (groupid <0 ) { +return groupid; + } + +trace_vfio_realize (vbasedev->name, groupid); + group =vfio_get_group (groupid, as, errp); +if (!group) { +return -1 ; + } + + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { +if (strcmp (vbasedev_iter->name, vbasedev->name) ==0 ) { +error_setg (errp,"device is already attached" ); +vfio_put_group (group); +return -1 ; + } + } + ret =vfio_get_device (group, vbasedev->name, vbasedev, errp); +if (ret) { +vfio_put_group (group); +return -1 ; + } + +return 0 ; +} + +void vfio_detach_device (VFIODevice *vbasedev) +{ +vfio_put_base_device (vbasedev); +vfio_put_group (vbasedev->group); +} +const VFIOContainerOps legacy_container_ops = { .dma_map = vfio_dma_map, .dma_unmap = vfio_dma_unmap,
hw/vfio/pci.c +5 -45
99 lines not on this page · raw message
include/hw/vfio/vfio-common.h +2 -0
@@ -181,6 +181,8 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp);void vfio_put_group (VFIOGroup *group);int vfio_get_device (VFIOGroup *group,const char *name,VFIODevice *vbasedev,Error **errp); +int vfio_attach_device (VFIODevice *vbasedev,AddressSpace *as,Error **errp); +void vfio_detach_device (VFIODevice *vbasedev);extern const MemoryRegionOps vfio_region_ops; typedef QLIST_HEAD(VFIOGroupList, VFIOGroup) VFIOGroupList;
Signature · 1 line
2.27.0
RFC v2 6/15 vfio/platform: Use vfio_[attach/detach]_device
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Let the vfio-platform device use vfio_attach_device() and vfio_detach_device(), hence hiding the details of the used IOMMU backend.
--- hw/vfio/platform.c | 42 ++---------------------------------------- 1 file changed, 2 insertions(+), 40 deletions(-)
hw/vfio/platform.c +2 -40
@@ -529,12 +529,7 @@ static VFIODeviceOps vfio_platform_ops = {*/ static int vfio_base_device_init (VFIODevice *vbasedev,Error **errp) { -VFIOGroup *group; -VFIODevice *vbasedev_iter; -char *tmp, group_path[PATH_MAX ], *group_name; -ssize_t len;struct stat st; -int groupid;int ret;/* @sysfsdev takes precedence over @host */
@@ -557,47 +552,14 @@ static int vfio_base_device_init(VFIODevice *vbasedev, Error **errp)return -errno; } - tmp =g_strdup_printf ("%s/iommu_group" , vbasedev->sysfsdev); - len =readlink (tmp, group_path,sizeof (group_path)); -g_free (tmp); - -if (len <0 || len >=sizeof (group_path)) { - ret = len <0 ? -errno : -ENAMETOOLONG ; -error_setg_errno (errp, -ret,"no iommu_group found" ); -return ret; - } - - group_path[len] =0 ; - - group_name =basename (group_path); -if (sscanf (group_name,"%d" , &groupid) !=1 ) { -error_setg_errno (errp, errno,"failed to read %s" , group_path); -return -errno; - } - -trace_vfio_platform_base_device_init (vbasedev->name, groupid); - - group =vfio_get_group (groupid, &address_space_memory, errp); -if (!group) { -return -ENOENT ; - } - - QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { -if (strcmp (vbasedev_iter->name, vbasedev->name) ==0 ) { -error_setg (errp,"device is already attached" ); -vfio_put_group (group); -return -EBUSY ; - } - } - ret =vfio_get_device (group, vbasedev->name, vbasedev, errp); + ret =vfio_attach_device (vbasedev, &address_space_memory, errp);if (ret) { -vfio_put_group (group);return ret; } ret =vfio_populate_device (vbasedev, errp);if (ret) { -vfio_put_group (group); +vfio_detach_device (vbasedev); }return ret;
Signature · 1 line
2.27.0
RFC v2 7/15 vfio/ap: Use vfio_[attach/detach]_device
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Let the vfio-ap device use vfio_attach_device() and vfio_detach_device(), hence hiding the details of the used IOMMU backend.
--- hw/vfio/ap.c | 62 ++++++++-------------------------------------------- 1 file changed, 9 insertions(+), 53 deletions(-)
hw/vfio/ap.c +9 -53
@@ -50,58 +50,17 @@ struct VFIODeviceOps vfio_ap_ops = { .vfio_compute_needs_reset = vfio_ap_compute_needs_reset, }; -static void vfio_ap_put_device (VFIOAPDevice *vapdev) -{ -g_free (vapdev->vdev.name); -vfio_put_base_device (&vapdev->vdev); -} - -static VFIOGroup *vfio_ap_get_group (VFIOAPDevice *vapdev,Error **errp) -{ -GError *gerror =NULL ; -char *symlink, *group_path; -int groupid; - - symlink =g_strdup_printf ("%s/iommu_group" , vapdev->vdev.sysfsdev); - group_path =g_file_read_link (symlink, &gerror); -g_free (symlink); - -if (!group_path) { -error_setg (errp,"%s: no iommu_group found for %s: %s" , -TYPE_VFIO_AP_DEVICE , vapdev->vdev.sysfsdev, gerror->message); -g_error_free (gerror); -return NULL ; - } - -if (sscanf (basename (group_path),"%d" , &groupid) !=1 ) { -error_setg (errp,"vfio: failed to read %s" , group_path); -g_free (group_path); -return NULL ; - } - -g_free (group_path); - -return vfio_get_group (groupid, &address_space_memory, errp); -} -static void vfio_ap_realize (DeviceState *dev,Error **errp) { -int ret; -char *mdevid; -VFIOGroup *vfio_group;APDevice *apdev =AP_DEVICE (dev);VFIOAPDevice *vapdev =VFIO_AP_DEVICE (apdev); +VFIODevice *vbasedev = &vapdev->vdev; +int ret; - vfio_group =vfio_ap_get_group (vapdev, errp); -if (!vfio_group) { -return ; - } - - vapdev->vdev.ops = &vfio_ap_ops; - vapdev->vdev.type =VFIO_DEVICE_TYPE_AP ; - mdevid =basename (vapdev->vdev.sysfsdev); - vapdev->vdev.name =g_strdup_printf ("%s" , mdevid); - vapdev->vdev.dev = dev; + vbasedev->name =g_path_get_basename (vbasedev->sysfsdev); + vbasedev->ops = &vfio_ap_ops; + vbasedev->type =VFIO_DEVICE_TYPE_AP ; + vbasedev->dev = dev;/* * vfio-ap devices operate in a way compatible with discarding of
@@ -111,7 +70,7 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)*/ vapdev->vdev.ram_block_discard_allowed = true; - ret =vfio_get_device (vfio_group, mdevid, &vapdev->vdev, errp); + ret =vfio_attach_device (vbasedev, &address_space_memory, errp);if (ret) { goto out_get_dev_err; }
@@ -119,18 +78,15 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)return ; out_get_dev_err: -vfio_ap_put_device (vapdev); -vfio_put_group (vfio_group); +vfio_detach_device (vbasedev); }static void vfio_ap_unrealize (DeviceState *dev) {APDevice *apdev =AP_DEVICE (dev);VFIOAPDevice *vapdev =VFIO_AP_DEVICE (apdev); -VFIOGroup *group = vapdev->vdev.group; -vfio_ap_put_device (vapdev); -vfio_put_group (group); +vfio_detach_device (&vapdev->vdev); }static Property vfio_ap_properties[] = {
Signature · 1 line
2.27.0
RFC v2 8/15 vfio/ccw: Use vfio_[attach/detach]_device
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Let the vfio-ccw device use vfio_attach_device() and vfio_detach_device(), hence hiding the details of the used IOMMU backend.
Also now all the devices have been migrated to use the new vfio_attach_device/vfio_detach_device API, let's turn the legacy functions into static functions, local to container.c.
--- hw/vfio/ccw.c | 118 ++++++++-------------------------- hw/vfio/container.c | 8 +-- include/hw/vfio/vfio-common.h | 4 -- 3 files changed, 32 insertions(+), 98 deletions(-)
hw/vfio/ccw.c +28 -90
@@ -579,27 +579,32 @@ static void vfio_ccw_put_region(VFIOCCWDevice *vcdev)g_free (vcdev->io_region); } -static void vfio_ccw_put_device (VFIOCCWDevice *vcdev) -{ -g_free (vcdev->vdev.name); -vfio_put_base_device (&vcdev->vdev); -} - -static void vfio_ccw_get_device (VFIOGroup *group,VFIOCCWDevice *vcdev, -Error **errp) +static void vfio_ccw_realize (DeviceState *dev,Error **errp) { +CcwDevice *ccw_dev =DO_UPCAST (CcwDevice, parent_obj, dev); +S390CCWDevice *cdev =DO_UPCAST (S390CCWDevice, parent_obj, ccw_dev); +VFIOCCWDevice *vcdev =DO_UPCAST (VFIOCCWDevice, cdev, cdev); +S390CCWDeviceClass *cdc =S390_CCW_DEVICE_GET_CLASS (cdev); +VFIODevice *vbasedev = &vcdev->vdev; +Error *err =NULL ;char *name =g_strdup_printf ("%x.%x.%04x" , vcdev->cdev.hostid.cssid, vcdev->cdev.hostid.ssid, vcdev->cdev.hostid.devid); -VFIODevice *vbasedev; +int ret; - QLIST_FOREACH(vbasedev, &group->device_list, next) { -if (strcmp (vbasedev->name, name) ==0 ) { -error_setg (errp,"vfio: subchannel %s has already been attached" , - name); - goto out_err; +/* Call the class init function for subchannel. */ +if (cdc->realize) { + cdc->realize (cdev, vcdev->vdev.sysfsdev, &err); +if (err) { + goto out_err_propagate; } } + vbasedev->sysfsdev =g_strdup_printf ("/sys/bus/css/devices/%s/%s" , + name, cdev->mdevid); + vbasedev->ops = &vfio_ccw_ops; + vbasedev->type =VFIO_DEVICE_TYPE_CCW ; + vbasedev->name = name; + vbasedev->dev = &vcdev->cdev.parent_obj.parent_obj;/* * All vfio-ccw devices are believed to operate in a way compatible with
@@ -609,80 +614,18 @@ static void vfio_ccw_get_device(VFIOGroup *group, VFIOCCWDevice *vcdev,* needs to be set before vfio_get_device() for vfio common to handle * ram_block_discard_disable(). */ - vcdev->vdev.ram_block_discard_allowed = true; - -if (vfio_get_device (group, vcdev->cdev.mdevid, &vcdev->vdev, errp)) { - goto out_err; - } - - vcdev->vdev.ops = &vfio_ccw_ops; - vcdev->vdev.type =VFIO_DEVICE_TYPE_CCW ; - vcdev->vdev.name = name; - vcdev->vdev.dev = &vcdev->cdev.parent_obj.parent_obj; - -return ; - -out_err: -g_free (name); -} - -static VFIOGroup *vfio_ccw_get_group (S390CCWDevice *cdev,Error **errp) -{ -char *tmp, group_path[PATH_MAX ]; -ssize_t len; -int groupid; - tmp =g_strdup_printf ("/sys/bus/css/devices/%x.%x.%04x/%s/iommu_group" , - cdev->hostid.cssid, cdev->hostid.ssid, - cdev->hostid.devid, cdev->mdevid); - len =readlink (tmp, group_path,sizeof (group_path)); -g_free (tmp); + vbasedev->ram_block_discard_allowed = true; -if (len <=0 || len >=sizeof (group_path)) { -error_setg (errp,"vfio: no iommu_group found" ); -return NULL ; - } - - group_path[len] =0 ; - -if (sscanf (basename (group_path),"%d" , &groupid) !=1 ) { -error_setg (errp,"vfio: failed to read %s" , group_path); -return NULL ; - } - -return vfio_get_group (groupid, &address_space_memory, errp); -} - -static void vfio_ccw_realize (DeviceState *dev,Error **errp) -{ -VFIOGroup *group; -CcwDevice *ccw_dev =DO_UPCAST (CcwDevice, parent_obj, dev); -S390CCWDevice *cdev =DO_UPCAST (S390CCWDevice, parent_obj, ccw_dev); -VFIOCCWDevice *vcdev =DO_UPCAST (VFIOCCWDevice, cdev, cdev); -S390CCWDeviceClass *cdc =S390_CCW_DEVICE_GET_CLASS (cdev); -Error *err =NULL ; - -/* Call the class init function for subchannel. */ -if (cdc->realize) { - cdc->realize (cdev, vcdev->vdev.sysfsdev, &err); -if (err) { - goto out_err_propagate; - } - } - - group =vfio_ccw_get_group (cdev, &err); -if (!group) { - goto out_group_err; - } - -vfio_ccw_get_device (group, vcdev, &err); -if (err) { - goto out_device_err; + ret =vfio_attach_device (vbasedev, &address_space_memory, errp); +if (ret) { +g_free (vbasedev->name); +g_free (vbasedev->sysfsdev); }vfio_ccw_get_region (vcdev, &err);if (err) { - goto out_region_err; + goto out_get_dev_err; }vfio_ccw_register_irq_notifier (vcdev,VFIO_CCW_IO_IRQ_INDEX , &err);
@@ -714,11 +657,8 @@ out_irq_notifier_err:vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_IO_IRQ_INDEX ); out_io_notifier_err:vfio_ccw_put_region (vcdev); -out_region_err: -vfio_ccw_put_device (vcdev); -out_device_err: -vfio_put_group (group); -out_group_err: +out_get_dev_err: +vfio_detach_device (vbasedev);if (cdc->unrealize) { cdc->unrealize (cdev); }
@@ -732,14 +672,12 @@ static void vfio_ccw_unrealize(DeviceState *dev)S390CCWDevice *cdev =DO_UPCAST (S390CCWDevice, parent_obj, ccw_dev);VFIOCCWDevice *vcdev =DO_UPCAST (VFIOCCWDevice, cdev, cdev);S390CCWDeviceClass *cdc =S390_CCW_DEVICE_GET_CLASS (cdev); -VFIOGroup *group = vcdev->vdev.group;vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_REQ_IRQ_INDEX );vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_CRW_IRQ_INDEX );vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_IO_IRQ_INDEX );vfio_ccw_put_region (vcdev); -vfio_ccw_put_device (vcdev); -vfio_put_group (group); +vfio_detach_device (&vcdev->vdev);if (cdc->unrealize) { cdc->unrealize (cdev);
hw/vfio/container.c +4 -4
@@ -958,7 +958,7 @@ static void vfio_disconnect_container(VFIOGroup *group) } } -VFIOGroup *vfio_get_group (int groupid,AddressSpace *as,Error **errp) +static VFIOGroup *vfio_get_group (int groupid,AddressSpace *as,Error **errp) {VFIOGroup *group;VFIOContainer *bcontainer;
@@ -1027,7 +1027,7 @@ free_group_exit:return NULL ; } -void vfio_put_group (VFIOGroup *group) +static void vfio_put_group (VFIOGroup *group) {if (!group || !QLIST_EMPTY (&group->device_list)) {return ;
@@ -1048,8 +1048,8 @@ void vfio_put_group(VFIOGroup *group) } } -int vfio_get_device (VFIOGroup *group,const char *name, -VFIODevice *vbasedev,Error **errp) +static int vfio_get_device (VFIOGroup *group,const char *name, +VFIODevice *vbasedev,Error **errp) {struct vfio_device_info dev_info = { .argsz =sizeof (dev_info) };int ret, fd;
include/hw/vfio/vfio-common.h +0 -4
@@ -177,10 +177,6 @@ void vfio_region_unmap(VFIORegion *region);void vfio_region_exit (VFIORegion *region);void vfio_region_finalize (VFIORegion *region);void vfio_reset_handler (void *opaque); -VFIOGroup *vfio_get_group (int groupid,AddressSpace *as,Error **errp); -void vfio_put_group (VFIOGroup *group); -int vfio_get_device (VFIOGroup *group,const char *name, -VFIODevice *vbasedev,Error **errp);int vfio_attach_device (VFIODevice *vbasedev,AddressSpace *as,Error **errp);void vfio_detach_device (VFIODevice *vbasedev);
Signature · 1 line
2.27.0
RFC v2 9/15 vfio/container-base: Introduce [attach/detach]_device container callbacks
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Let's turn attach/detach_device as container callbacks. That way, their implementation can be easily customized for a given backend.
For the time being, only the legacy container is supported.
--- hw/vfio/as.c | 30 +++++++++++++++++++++++++++ hw/vfio/container.c | 9 ++++++-- hw/vfio/pci.c | 2 +- include/hw/vfio/vfio-common.h | 7 +++++++ include/hw/vfio/vfio-container-base.h | 4 ++++ 5 files changed, 49 insertions(+), 3 deletions(-)
hw/vfio/as.c +30 -0
@@ -899,3 +899,33 @@ void vfio_put_address_space(VFIOAddressSpace *space)g_free (space); } } + +static const VFIOContainerOps * +vfio_get_container_ops (VFIOIOMMUBackendType be) +{ +switch (be) { +case VFIO_IOMMU_BACKEND_TYPE_LEGACY : +return &legacy_container_ops; +default : +return NULL ; + } +} + +int vfio_attach_device (VFIODevice *vbasedev,AddressSpace *as,Error **errp) +{ +const VFIOContainerOps *ops; + + ops =vfio_get_container_ops (VFIO_IOMMU_BACKEND_TYPE_LEGACY ); +if (!ops) { +return -ENOENT ; + } +return ops->attach_device (vbasedev, as, errp); +} + +void vfio_detach_device (VFIODevice *vbasedev) +{ +if (!vbasedev->container) { +return ; + } + vbasedev->container->ops->detach_device (vbasedev); +}
hw/vfio/container.c +7 -2
@@ -1244,7 +1244,8 @@ static int vfio_device_groupid(VFIODevice *vbasedev, Error **errp)return groupid; } -int vfio_attach_device (VFIODevice *vbasedev,AddressSpace *as,Error **errp) +static int +legacy_attach_device (VFIODevice *vbasedev,AddressSpace *as,Error **errp) {int groupid =vfio_device_groupid (vbasedev, errp);VFIODevice *vbasedev_iter;
@@ -1273,14 +1274,16 @@ int vfio_attach_device(VFIODevice *vbasedev, AddressSpace *as, Error **errp)vfio_put_group (group);return -1 ; } + vbasedev->container = &group->container->bcontainer;return 0 ; } -void vfio_detach_device (VFIODevice *vbasedev) +static void legacy_detach_device (VFIODevice *vbasedev) {vfio_put_base_device (vbasedev);vfio_put_group (vbasedev->group); + vbasedev->container =NULL ; }const VFIOContainerOps legacy_container_ops = {
@@ -1292,4 +1295,6 @@ const VFIOContainerOps legacy_container_ops = { .add_window = vfio_legacy_container_add_section_window, .del_window = vfio_legacy_container_del_section_window, .check_extension = vfio_legacy_container_check_extension, + .attach_device = legacy_attach_device, + .detach_device = legacy_detach_device, };
hw/vfio/pci.c +1 -1
@@ -3106,7 +3106,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) }if (!pdev->failover_pair_id && -vfio_container_check_extension (&vbasedev->group->container->bcontainer, +vfio_container_check_extension (vbasedev->container,VFIO_FEAT_LIVE_MIGRATION )) { ret =vfio_migration_probe (vbasedev, errp);if (ret) {
include/hw/vfio/vfio-common.h +7 -0
@@ -84,9 +84,15 @@ typedef struct VFIOLegacyContainer {typedef struct VFIODeviceOps VFIODeviceOps ; +typedef enum VFIOIOMMUBackendType { +VFIO_IOMMU_BACKEND_TYPE_LEGACY =0 , +VFIO_IOMMU_BACKEND_TYPE_IOMMUFD =1 , +}VFIOIOMMUBackendType ; + typedef struct VFIODevice { QLIST_ENTRY(VFIODevice) next; struct VFIOGroup *group; + VFIOContainer *container; char *sysfsdev; char *name; DeviceState *dev;
@@ -98,6 +104,7 @@ typedef struct VFIODevice {bool ram_block_discard_allowed;bool enable_migration;VFIODeviceOps *ops; +VFIOIOMMUBackendType be;unsigned int num_irqs;unsigned int num_regions;unsigned int flags;
include/hw/vfio/vfio-container-base.h +4 -0
@@ -66,6 +66,8 @@ typedef struct VFIOHostDMAWindow {QLIST_ENTRY (VFIOHostDMAWindow) hostwin_next; } VFIOHostDMAWindow; +typedef struct VFIODevice VFIODevice ; + typedef struct VFIOContainerOps { /* required */ bool (*check_extension)(VFIOContainer *container,
@@ -88,6 +90,8 @@ typedef struct VFIOContainerOps { Error **errp);void (*del_window)(VFIOContainer *container,MemoryRegionSection *section); +int (*attach_device)(VFIODevice *vbasedev,AddressSpace *as,Error **errp); +void (*detach_device)(VFIODevice *vbasedev); } VFIOContainerOps;/*
Signature · 1 line
2.27.0
RFC v2 10/15 vfio/container-base: Introduce VFIOContainer reset callback
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Reset implementation depends on the container backend. Let's introduce a VFIOContainer class function and register a generic reset handler that will be able to call the right reset function depending on the container type. Also, let's move the registration/unregistration to a place that is not backend-specific (first vfio address space created instead of the first group).
--- hw/vfio/as.c | 18 ++++++++++++++++++ hw/vfio/container-base.c | 9 +++++++++ hw/vfio/container.c | 27 +++++++++++++++------------ include/hw/vfio/vfio-container-base.h | 2 ++ 4 files changed, 44 insertions(+), 12 deletions(-)
hw/vfio/as.c +18 -0
@@ -872,6 +872,18 @@ const MemoryListener vfio_memory_listener = { .log_sync = vfio_listener_log_sync, }; +void vfio_reset_handler (void *opaque) +{ +VFIOAddressSpace *space; +VFIOContainer *bcontainer; + + QLIST_FOREACH(space, &vfio_address_spaces, list) { + QLIST_FOREACH(bcontainer, &space->containers, next) { +vfio_container_reset (bcontainer); + } + } +} +VFIOAddressSpace *vfio_get_address_space (AddressSpace *as) {VFIOAddressSpace *space;
@@ -887,6 +899,9 @@ VFIOAddressSpace *vfio_get_address_space(AddressSpace *as) space->as = as;QLIST_INIT (&space->containers); +if (QLIST_EMPTY (&vfio_address_spaces)) { +qemu_register_reset (vfio_reset_handler,NULL ); + }QLIST_INSERT_HEAD (&vfio_address_spaces, space, list);return space;
@@ -898,6 +913,9 @@ void vfio_put_address_space(VFIOAddressSpace *space)QLIST_REMOVE (space, list);g_free (space); } +if (QLIST_EMPTY (&vfio_address_spaces)) { +qemu_unregister_reset (vfio_reset_handler,NULL ); + } } static const VFIOContainerOps *
hw/vfio/container-base.c +9 -0
@@ -58,6 +58,15 @@ int vfio_container_dma_unmap(VFIOContainer *container,return container->ops->dma_unmap (container, iova, size, iotlb); } +int vfio_container_reset (VFIOContainer *container) +{ +if (!container->ops->reset) { +return -ENOENT ; + } + +return container->ops->reset (container); +} +void vfio_container_set_dirty_page_tracking (VFIOContainer *container,bool start) {
hw/vfio/container.c +15 -12
@@ -465,12 +465,16 @@ vfio_legacy_container_del_section_window(VFIOContainer *bcontainer, } } -void vfio_reset_handler (void *opaque) +static int vfio_legacy_container_reset (VFIOContainer *bcontainer) { +VFIOLegacyContainer *container =container_of (bcontainer, + VFIOLegacyContainer, + bcontainer);VFIOGroup *group;VFIODevice *vbasedev; +int ret, final_ret =0 ; - QLIST_FOREACH(group, &vfio_group_list, next) { + QLIST_FOREACH(group, &container->group_list, container_next) { QLIST_FOREACH(vbasedev, &group->device_list, next) {if (vbasedev->dev->realized) { vbasedev->ops->vfio_compute_needs_reset (vbasedev);
@@ -478,13 +482,19 @@ void vfio_reset_handler(void *opaque) } } - QLIST_FOREACH(group, &vfio_group_list, next) { + QLIST_FOREACH(group, &container->group_list, next) { QLIST_FOREACH(vbasedev, &group->device_list, next) {if (vbasedev->dev->realized && vbasedev->needs_reset) { - vbasedev->ops->vfio_hot_reset_multi (vbasedev); + ret = vbasedev->ops->vfio_hot_reset_multi (vbasedev); +if (ret) { +error_report ("failed to reset %s (%d)" , + vbasedev->name, ret); + final_ret = ret; + } } } } +return final_ret; }static void vfio_kvm_device_add_group (VFIOGroup *group)
@@ -1010,10 +1020,6 @@ static VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp) goto close_fd_exit; } -if (QLIST_EMPTY (&vfio_group_list)) { -qemu_register_reset (vfio_reset_handler,NULL ); - } -QLIST_INSERT_HEAD (&vfio_group_list, group, next);return group;
@@ -1042,10 +1048,6 @@ static void vfio_put_group(VFIOGroup *group)trace_vfio_put_group (group->fd);close (group->fd);g_free (group); - -if (QLIST_EMPTY (&vfio_group_list)) { -qemu_unregister_reset (vfio_reset_handler,NULL ); - } } static int vfio_get_device(VFIOGroup *group, const char *name,
@@ -1295,6 +1297,7 @@ const VFIOContainerOps legacy_container_ops = { .add_window = vfio_legacy_container_add_section_window, .del_window = vfio_legacy_container_del_section_window, .check_extension = vfio_legacy_container_check_extension, + .reset = vfio_legacy_container_reset, .attach_device = legacy_attach_device, .detach_device = legacy_detach_device, };
include/hw/vfio/vfio-container-base.h +2 -0
@@ -78,6 +78,7 @@ typedef struct VFIOContainerOps {int (*dma_unmap)(VFIOContainer *container,hwaddr iova,ram_addr_t size,IOMMUTLBEntry *iotlb); +int (*reset)(VFIOContainer *container);/* migration feature */ bool (*devices_all_dirty_tracking)(VFIOContainer *container);void (*set_dirty_page_tracking)(VFIOContainer *container,bool start);
@@ -122,6 +123,7 @@ int vfio_container_dma_map(VFIOContainer *container,int vfio_container_dma_unmap (VFIOContainer *container,hwaddr iova,ram_addr_t size,IOMMUTLBEntry *iotlb); +int vfio_container_reset (VFIOContainer *container);bool vfio_container_devices_all_dirty_tracking (VFIOContainer *container);void vfio_container_set_dirty_page_tracking (VFIOContainer *container,bool start);
Signature · 1 line
2.27.0
RFC v2 11/15 backends/iommufd: Introduce the iommufd object
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Introduce an iommufd object which allows the interaction with the host /dev/iommu device.
The /dev/iommu can have been already pre-opened outside of qemu, in which case the fd can be passed directly along with the iommufd object:
This allows the iommufd object to be shared accross several subsystems (VFIO, VDPA, ...). For example, libvirt would open the /dev/iommu once.
If no fd is passed along with the iommufd object, the /dev/iommu is opened by the qemu code.
The CONFIG_IOMMUFD option must be set to compile this new object.
--- MAINTAINERS | 7 ++ backends/Kconfig | 5 + backends/iommufd.c | 265 +++++++++++++++++++++++++++++++++++++++ backends/meson.build | 1 + backends/trace-events | 12 ++ include/sysemu/iommufd.h | 47 +++++++ qapi/qom.json | 16 ++- qemu-options.hx | 12 ++ 8 files changed, 364 insertions(+), 1 deletion(-) create mode 100644 backends/iommufd.c create mode 100644 include/sysemu/iommufd.h
MAINTAINERS +7 -0
@@ -1984,6 +1984,13 @@ F: hw/vfio/ap.c F: docs/system/s390x/vfio-ap.rst L: [email protected] +iommufd +M: Yi Liu <[email protected]> +M: Eric Auger <[email protected]> +S: Supported +F: backends/iommufd.c +F: include/sysemu/iommufd.h + vhost M: Michael S. Tsirkin <[email protected]> S: Supported
backends/Kconfig +5 -0
@@ -1 +1,6 @@ source tpm/Kconfig + +config IOMMUFD + bool + default y + depends on LINUX
backends/iommufd.c new file +265 -0
265 lines not on this page · raw message
backends/meson.build +1 -0
@@ -15,6 +15,7 @@ softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('hostmem-memfd.c')) if have_vhost_user softmmu_ss.add(when: 'CONFIG_VIRTIO', if_true: files('vhost-user.c')) endif +specific_ss.add(when: 'CONFIG_IOMMUFD', if_true: files('iommufd.c')) softmmu_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('cryptodev-vhost.c')) if have_vhost_user_crypto softmmu_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('cryptodev-vhost-user.c'))
backends/trace-events +12 -0
@@ -5,3 +5,15 @@ dbus_vmstate_pre_save(void) dbus_vmstate_post_load(int version_id) "version_id: %d" dbus_vmstate_loading(const char *id) "id: %s" dbus_vmstate_saving(const char *id) "id: %s" + +# iommufd.c +iommufd_backend_connect(int fd, bool owned, uint32_t users, int ret) "fd=%d owned=%d users=%d (%d)" +iommufd_backend_disconnect(int fd, uint32_t users) "fd=%d users=%d" +iommu_backend_set_fd(int fd) "pre-opened /dev/iommu fd=%d" +iommufd_backend_get_ioas(int iommufd, uint32_t ioas, int ret) " iommufd=%d ioas=%d (%d)" +iommufd_backend_put_ioas(int iommufd, uint32_t ioas) " iommufd=%d ioas=%d" +iommufd_backend_unmap_dma(int iommufd, uint32_t ioas, uint64_t iova, uint64_t size, int ret) " iommufd=%d ioas=%d iova=0x%"PRIx64" size=0x%"PRIx64" (%d)" +iommufd_backend_map_dma(int iommufd, uint32_t ioas, uint64_t iova, uint64_t size, void *vaddr, bool readonly, int ret) " iommufd=%d ioas=%d iova=0x%"PRIx64" size=0x%"PRIx64" addr=%p readonly=%d (%d)" +iommufd_backend_copy_dma(int iommufd, uint32_t src_ioas, uint32_t dst_ioas, uint64_t iova, uint64_t size, bool readonly, int ret) " iommufd=%d src_ioas=%d dst_ioas=%d iova=0x%"PRIx64" size=0x%"PRIx64" readonly=%d (%d)" +iommufd_backend_alloc_ioas(int iommufd, uint32_t ioas, int ret) " iommufd=%d ioas=%d (%d)" +iommufd_backend_free_ioas(int iommufd, uint32_t ioas, int ret) " iommufd=%d ioas=%d (%d)"
include/sysemu/iommufd.h new file +47 -0
47 lines not on this page · raw message
qapi/qom.json +15 -1
@@ -734,6 +734,18 @@ { 'struct': 'RemoteObjectProperties', 'data': { 'fd': 'str', 'devid': 'str' } } +## +# @IOMMUFDProperties: +# +# Properties for IOMMUFDbackend objects. +# +# fd: file descriptor name +# +# Since: 7.2 +## +{ 'struct': 'IOMMUFDProperties', + 'data': { '*fd': 'str' } } + ## # @RngProperties: #
@@ -862,6 +874,7 @@ 'qtest', 'rng-builtin', 'rng-egd', + 'iommufd', { 'name': 'rng-random', 'if': 'CONFIG_POSIX' }, 'secret',
@@ -938,7 +951,8 @@ 'tls-creds-psk': 'TlsCredsPskProperties', 'tls-creds-x509': 'TlsCredsX509Properties', 'tls-cipher-suites': 'TlsCredsProperties', - 'x-remote-object': 'RemoteObjectProperties' + 'x-remote-object': 'RemoteObjectProperties', + 'iommufd': 'IOMMUFDProperties' } } ##
qemu-options.hx +12 -0
@@ -4860,6 +4860,18 @@ SRST The ``share`` boolean option is on by default with memfd. + ``-object iommufd,id=id[,fd=fd]`` + Creates an iommufd backend which allows control of DMA mapping + through the /dev/iommu device. + + The ``id`` parameter is a unique ID which frontends (such as + vfio-pci of vdpa) will use to connect withe the iommufd backend. + + The ``fd`` parameter is an optional pre-opened file descriptor + resulting from /dev/iommu opening. Usually the iommufd is shared + accross all subsystems, bringing the benefit of centralized + reference counting. + ``-object rng-builtin,id=id`` Creates a random number generator backend which obtains entropy from QEMU builtin functions. The ``id`` parameter is a unique ID
Signature · 1 line
2.27.0
RFC v2 12/15 util/char_dev: Add open_cdev()
In reply to Yi Liu/dev/vfio/devices/vfioX may not exist. In that case it is still possible to open /dev/char/$major:$minor instead. Add helper function to abstract the cdev open.
--- MAINTAINERS | 6 +++++ include/qemu/char_dev.h | 16 ++++++++++++ util/chardev_open.c | 58 +++++++++++++++++++++++++++++++++++++++++ util/meson.build | 1 + 4 files changed, 81 insertions(+) create mode 100644 include/qemu/char_dev.h create mode 100644 util/chardev_open.c
MAINTAINERS +6 -0
@@ -3171,6 +3171,12 @@ S: Maintained F: include/qemu/iova-tree.h F: util/iova-tree.c +cdev Open +M: Yi Liu <[email protected]> +S: Maintained +F: include/qemu/char_dev.h +F: util/chardev_open.c + elf2dmp M: Viktor Prutyanov <[email protected]> S: Maintained
include/qemu/char_dev.h new file +16 -0
@@ -0,0 +1,16 @@ +/* +* QEMU Chardev Helper +* +* Copyright (C) 2022 Intel Corporation. +* +* Authors: Yi Liu <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +*/ + +#ifndef QEMU_CHARDEV_HELPERS_H +#define QEMU_CHARDEV_HELPERS_H + +int open_cdev (const char *devpath,dev_t cdev); +#endif
util/chardev_open.c new file +58 -0
58 lines not on this page · raw message
util/meson.build +1 -0
@@ -95,4 +95,5 @@ if have_block util_ss.add(files('filemonitor-stub.c')) endif util_ss.add(when: 'CONFIG_LINUX', if_true: files('vfio-helpers.c')) + util_ss.add(when: 'CONFIG_LINUX', if_true: files('chardev_open.c')) endif
Signature · 1 line
2.27.0
RFC v2 13/15 vfio/iommufd: Implement the iommufd backend
In reply to Yi LiuAdd the iommufd backend. The IOMMUFD container class is implemented based on the new /dev/iommu user API. This backend obviously depends on CONFIG_IOMMUFD.
So far, the iommufd backend doesn't support live migration and cache coherency yet due to missing support in the host kernel meaning that only a subset of the container class callbacks is implemented.
---
v1 -> v2:
- arbitrarily set bcontainer->pgsizes to 4K to fix interoperability with virtio-iommu (Nicolin)
- add tear down in iommufd_attach_device error path
- use cdev open for /dev/vfio/devices/vfioX open
--- hw/vfio/as.c | 2 +- hw/vfio/iommufd.c | 517 ++++++++++++++++++++++++++++++++++ hw/vfio/meson.build | 3 + hw/vfio/pci.c | 10 + hw/vfio/trace-events | 11 + include/hw/vfio/vfio-common.h | 22 ++ 6 files changed, 564 insertions(+), 1 deletion(-) create mode 100644 hw/vfio/iommufd.c
hw/vfio/as.c +1 -1
@@ -42,7 +42,7 @@#include "migration/migration.h" #include "sysemu/tpm.h" -static QLIST_HEAD(, VFIOAddressSpace) vfio_address_spaces = +VFIOAddressSpaceList vfio_address_spaces =QLIST_HEAD_INITIALIZER (vfio_address_spaces); void vfio_host_win_add(VFIOContainer *container,
hw/vfio/iommufd.c new file +517 -0
517 lines not on this page · raw message
hw/vfio/meson.build +3 -0
@@ -7,6 +7,9 @@ vfio_ss.add(files( 'spapr.c', 'migration.c', )) +vfio_ss.add(when: 'CONFIG_IOMMUFD', if_true: files( + 'iommufd.c', +)) vfio_ss.add(when: 'CONFIG_VFIO_PCI', if_true: files( 'display.c', 'pci-quirks.c',
hw/vfio/pci.c +10 -0
@@ -3191,6 +3191,16 @@ static void vfio_pci_reset(DeviceState *dev) goto post_reset; } +/* +* This is a temporary check, long term iommufd should +* support hot reset as well +*/ +if (vdev->vbasedev.be ==VFIO_IOMMU_BACKEND_TYPE_IOMMUFD ) { +error_report ("Dangerous: iommufd BE doesn't support hot " +"reset, please stop the VM" ); + goto post_reset; + } +/* See if we can do our own bus reset */ if (!vfio_pci_hot_reset_one (vdev)) { goto post_reset;
hw/vfio/trace-events +11 -0
@@ -166,3 +166,14 @@ vfio_load_state_device_data(const char *name, uint64_t data_offset, uint64_t dat vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 + +#iommufd.c + +vfio_iommufd_get_devicefd(const char *dev, int devfd) " %s (fd=%d)" +vfio_iommufd_bind_device(int iommufd, const char *name, int devfd, int devid) " [iommufd=%d] Succesfully bound device %s (fd=%d): output devid=%d" +vfio_iommufd_attach_device(int iommufd, const char *name, int devfd, int ioasid, int hwptid) " [iommufd=%d] Succesfully attached device %s (%d) to ioasid=%d: output hwptd=%d" +vfio_iommufd_detach_device(int iommufd, const char *name, int ioasid) " [iommufd=%d] Detached %s from ioasid=%d" +vfio_iommufd_alloc_ioas(int iommufd, int ioas_id) " [iommufd=%d] new IOMMUFD container with ioasid=%d" +vfio_iommufd_device_info(char *name, int devfd, int num_irqs, int num_regions, int flags) " %s (%d) num_irqs=%d num_regions=%d flags=%d" +vfio_iommufd_fail_attach_existing_container(const char *msg) " %s" +vfio_iommufd_container_reset(char *name) " Successfully reset %s"
include/hw/vfio/vfio-common.h +22 -0
@@ -36,6 +36,7 @@extern const MemoryListener vfio_memory_listener;extern const VFIOContainerOps legacy_container_ops; +extern const VFIOContainerOps iommufd_container_ops;enum {VFIO_DEVICE_TYPE_PCI =0 ,
@@ -82,6 +83,24 @@ typedef struct VFIOLegacyContainer { QLIST_HEAD(, VFIOGroup) group_list; } VFIOLegacyContainer; +typedef struct VFIOIOASHwpt { +uint32_t hwpt_id; + QLIST_HEAD(, VFIODevice) device_list; +QLIST_ENTRY (VFIOIOASHwpt) next; +}VFIOIOASHwpt ; + +typedef struct IOMMUFDBackend IOMMUFDBackend ; + +typedef struct VFIOIOMMUFDContainer { +VFIOContainer bcontainer; +IOMMUFDBackend *be; +uint32_t ioas_id; + QLIST_HEAD(, VFIOIOASHwpt) hwpt_list; +}VFIOIOMMUFDContainer ; + +typedef QLIST_HEAD(VFIOAddressSpaceList, VFIOAddressSpace) VFIOAddressSpaceList; +extern VFIOAddressSpaceList vfio_address_spaces; +typedef struct VFIODeviceOps VFIODeviceOps ; typedef enum VFIOIOMMUBackendType {
@@ -91,6 +110,7 @@ typedef enum VFIOIOMMUBackendType { typedef struct VFIODevice { QLIST_ENTRY(VFIODevice) next; + QLIST_ENTRY(VFIODevice) hwpt_next; struct VFIOGroup *group; VFIOContainer *container; char *sysfsdev;
@@ -98,6 +118,7 @@ typedef struct VFIODevice {DeviceState *dev;int fd;int type; +int devid;bool reset_works;bool needs_reset;bool no_mmap;
@@ -111,6 +132,7 @@ typedef struct VFIODevice {VFIOMigration *migration;Error *migration_blocker;OnOffAuto pre_copy_dirty_page_tracking; +IOMMUFDBackend *iommufd; } VFIODevice;struct VFIODeviceOps {
Signature · 1 line
2.27.0
Re: [RFC v2 13/15] vfio/iommufd: Implement the iommufd backend
Yi Liu <[email protected]> writes:
[...]
Quoted 54 lines
hw/vfio/iommufd.c new file +54
-0+static int vfio_get_devicefd (const char *sysfs_path,Error **errp) +{ +long int ret = -ENOTTY ; +char *path, *vfio_dev_path =NULL , *vfio_path =NULL ; +DIR *dir; +struct dirent *dent; +gchar *contents; +struct stat st; +gsize length; +int major, minor; +dev_t vfio_devt; + + path =g_strdup_printf ("%s/vfio-device" , sysfs_path); +if (stat (path, &st) <0 ) { +error_setg_errno (errp, errno,"no such host device" ); + goto out_free_path; + } + + dir =opendir (path); +if (!dir) { +error_setg_errno (errp, errno,"couldn't open dirrectory %s" , path); + goto out_free_path; + } + +while ((dent =readdir (dir))) { +if (!strncmp (dent->d_name,"vfio" ,4 )) { + vfio_dev_path =g_strdup_printf ("%s/%s/dev" , path, dent->d_name); +break ; + } + } + +if (!vfio_dev_path) { +error_setg (errp,"failed to find vfio-device/vfioX/dev" ); + goto out_free_path; + } + +if (!g_file_get_contents (vfio_dev_path, &contents, &length,NULL )) { +error_setg (errp,"failed to load \"%s\"" , vfio_dev_path); + goto out_free_dev_path; + } + +if (sscanf (contents,"%d:%d" , &major, &minor) !=2 ) { +error_setg (errp,"failed to get major:mino for \"%s\"" , vfio_dev_path); + goto out_free_dev_path; + } +g_free (contents); + vfio_devt =makedev (major, minor); + + vfio_path =g_strdup_printf ("/dev/vfio/devices/%s" , dent->d_name); + ret =open_cdev (vfio_path, vfio_devt); +if (ret <0 ) { +error_setg (errp,"Failed to open %s" , vfio_path); + } +
hw/vfio/iommufd.c line 133
Quoted 10 lines
hw/vfio/iommufd.c new file +10
-0+trace_vfio_iommufd_get_devicefd (vfio_path, ret); +g_free (vfio_path); + +out_free_dev_path: +g_free (vfio_dev_path); +out_free_path: +g_free (path); + +if (*errp) { +error_prepend (errp,VFIO_MSG_PREFIX , path);
I ran into this while trying to get things running, so haven't reviewed the patch but noticed path is used after it's freed if !!*errp.
- Alistair
Quoted 3 lines
hw/vfio/iommufd.c new file +3
-0+ } +return ret; +}
Re: [RFC v2 13/15] vfio/iommufd: Implement the iommufd backend
Hi Alister,
Quoted 62 lines · Alistair Popple
Yi Liu <[email protected]> writes:
[...]
Quoted 58 lines
hw/vfio/iommufd.c new file +58
-0+static int vfio_get_devicefd (const char *sysfs_path,Error **errp) +{ +long int ret = -ENOTTY ; +char *path, *vfio_dev_path =NULL , *vfio_path =NULL ; +DIR *dir; +struct dirent *dent; +gchar *contents; +struct stat st; +gsize length; +int major, minor; +dev_t vfio_devt; + + path =g_strdup_printf ("%s/vfio-device" , sysfs_path); +if (stat (path, &st) <0 ) { +error_setg_errno (errp, errno,"no such host device" ); + goto out_free_path; + } + + dir =opendir (path); +if (!dir) { +error_setg_errno (errp, errno,"couldn't open dirrectory %s" , path); + goto out_free_path; + } + +while ((dent =readdir (dir))) { +if (!strncmp (dent->d_name,"vfio" ,4 )) { + vfio_dev_path =g_strdup_printf ("%s/%s/dev" , path, dent->d_name); +break ; + } + } + +if (!vfio_dev_path) { +error_setg (errp,"failed to find vfio-device/vfioX/dev" ); + goto out_free_path; + } + +if (!g_file_get_contents (vfio_dev_path, &contents, &length,NULL )) { +error_setg (errp,"failed to load \"%s\"" , vfio_dev_path); + goto out_free_dev_path; + } + +if (sscanf (contents,"%d:%d" , &major, &minor) !=2 ) { +error_setg (errp,"failed to get major:mino for \"%s\"" , vfio_dev_path); + goto out_free_dev_path; + } +g_free (contents); + vfio_devt =makedev (major, minor); + + vfio_path =g_strdup_printf ("/dev/vfio/devices/%s" , dent->d_name); + ret =open_cdev (vfio_path, vfio_devt); +if (ret <0 ) { +error_setg (errp,"Failed to open %s" , vfio_path); + } + +trace_vfio_iommufd_get_devicefd (vfio_path, ret); +g_free (vfio_path); + +out_free_dev_path:
Quoted 8 lines
hw/vfio/iommufd.c line 133
Quoted 6 lines
hw/vfio/iommufd.c new file +6
-0+g_free (vfio_dev_path); +out_free_path: +g_free (path); + +if (*errp) { +error_prepend (errp,VFIO_MSG_PREFIX , path);I ran into this while trying to get things running, so haven't reviewed the patch but noticed path is used after it's freed if !!*errp.
thank you for the bug report! We will fix that on the next iteration.
Eric
Quoted 6 lines
- Alistair
Quoted 3 lines
hw/vfio/iommufd.c new file +3
-0+ } +return ret; +}
RFC v2 14/15 vfio/iommufd: Add IOAS_COPY_DMA support
In reply to Yi LiuCompared with legacy vfio container BE, one of the benefits provided by iommufd is to reduce the redundant page pinning on kernel side through the usage of IOAS_COPY_DMA. For iommufd containers within the same address space, IOVA mappings can be copied from a source container to destination container.
To achieve this, move the vfio_memory_listener to be per address space. In the memory listener callbacks, all the containers within the address space will be looped. For the iommufd containers, QEMU uses IOAS_MAP_DMA on the first one, and then uses IOAS_COPY_DMA to copy the IOVA mappings from the first iommufd container to other iommufd containers within the address space. For legacy containers, IOVA mapping is done by VFIO_IOMMU_MAP_DMA.
--- hw/vfio/as.c | 117 ++++++++++++++++++++++---- hw/vfio/container-base.c | 13 ++- hw/vfio/container.c | 19 ++--- hw/vfio/iommufd.c | 47 +++++++++-- include/hw/vfio/vfio-common.h | 5 +- include/hw/vfio/vfio-container-base.h | 8 +- 6 files changed, 167 insertions(+), 42 deletions(-)
hw/vfio/as.c +100 -17
@@ -405,16 +405,16 @@ static bool vfio_known_safe_misalignment(MemoryRegionSection *section)return true; } -static void vfio_listener_region_add (MemoryListener *listener, -MemoryRegionSection *section) +static void vfio_container_region_add (VFIOContainer *container, +VFIOContainer **src_container, +MemoryRegionSection *section) { -VFIOContainer *container =container_of (listener, VFIOContainer, listener);hwaddr iova, end;Int128 llend, llsize;void *vaddr;int ret;VFIOHostDMAWindow *hostwin; -bool hostwin_found; +bool hostwin_found, copy_dma_supported = false;Error *err =NULL ;if (vfio_listener_skipped_section (section)) {
@@ -558,12 +558,25 @@ static void vfio_listener_region_add(MemoryListener *listener, } } + copy_dma_supported =vfio_container_check_extension (container, +VFIO_FEAT_DMA_COPY ); + +if (copy_dma_supported && *src_container) { +if (!vfio_container_dma_copy (*src_container, container, + iova,int128_get64 (llsize), + section->readonly)) { +return ; + }else { +info_report ("IOAS copy failed try map for container: %p" , container); + } + } + ret =vfio_container_dma_map (container, iova,int128_get64 (llsize), vaddr, section->readonly);if (ret) { -error_setg (&err,"vfio_dma_map(%p, 0x%" HWADDR_PRIx", " -"0x%" HWADDR_PRIx", %p) = %d (%m)" , - container, iova,int128_get64 (llsize), vaddr, ret); +error_setg (&err,"vfio_container_dma_map(%p, 0x%" HWADDR_PRIx", " +"0x%" HWADDR_PRIx", %p) = %d (%m)" , container, iova, +int128_get64 (llsize), vaddr, ret);if (memory_region_is_ram_device (section->mr)) {/* Allow unexpected mappings not to be fatal for RAM devices */ error_report_err (err);
@@ -572,6 +585,9 @@ static void vfio_listener_region_add(MemoryListener *listener, goto fail; } +if (copy_dma_supported) { + *src_container = container; + }return ; fail:
@@ -598,10 +614,22 @@ fail: } } -static void vfio_listener_region_del (MemoryListener *listener, +static void vfio_listener_region_add (MemoryListener *listener,MemoryRegionSection *section) { -VFIOContainer *container =container_of (listener, VFIOContainer, listener); +VFIOAddressSpace *space =container_of (listener, + VFIOAddressSpace, listener); +VFIOContainer *container, *src_container; + + src_container =NULL ; + QLIST_FOREACH(container, &space->containers, next) { +vfio_container_region_add (container, &src_container, section); + } +} + +static void vfio_container_region_del (VFIOContainer *container, +MemoryRegionSection *section) +{hwaddr iova, end;Int128 llend, llsize;int ret;
@@ -707,18 +735,38 @@ static void vfio_listener_region_del(MemoryListener *listener,vfio_container_del_section_window (container, section); } +static void vfio_listener_region_del (MemoryListener *listener, +MemoryRegionSection *section) +{ +VFIOAddressSpace *space =container_of (listener, + VFIOAddressSpace, listener); +VFIOContainer *container; + + QLIST_FOREACH(container, &space->containers, next) { +vfio_container_region_del (container, section); + } +} +static void vfio_listener_log_global_start (MemoryListener *listener) { -VFIOContainer *container =container_of (listener, VFIOContainer, listener); +VFIOAddressSpace *space =container_of (listener, + VFIOAddressSpace, listener); +VFIOContainer *container; -vfio_container_set_dirty_page_tracking (container, true); + QLIST_FOREACH(container, &space->containers, next) { +vfio_container_set_dirty_page_tracking (container, true); + } }static void vfio_listener_log_global_stop (MemoryListener *listener) { -VFIOContainer *container =container_of (listener, VFIOContainer, listener); +VFIOAddressSpace *space =container_of (listener, + VFIOAddressSpace, listener); +VFIOContainer *container; -vfio_container_set_dirty_page_tracking (container, false); + QLIST_FOREACH(container, &space->containers, next) { +vfio_container_set_dirty_page_tracking (container, false); + } } typedef struct {
@@ -848,11 +896,9 @@ static int vfio_sync_dirty_bitmap(VFIOContainer *container, int128_get64(section->size), ram_addr); } -static void vfio_listener_log_sync (MemoryListener *listener, -MemoryRegionSection *section) +static void vfio_container_log_sync (VFIOContainer *container, +MemoryRegionSection *section) { -VFIOContainer *container =container_of (listener, VFIOContainer, listener); -if (vfio_listener_skipped_section (section) || !container->dirty_pages_supported) {return ;
@@ -863,6 +909,18 @@ static void vfio_listener_log_sync(MemoryListener *listener, } } +static void vfio_listener_log_sync (MemoryListener *listener, +MemoryRegionSection *section) +{ +VFIOAddressSpace *space =container_of (listener, + VFIOAddressSpace, listener); +VFIOContainer *container; + + QLIST_FOREACH(container, &space->containers, next) { +vfio_container_log_sync (container, section); + } +} +const MemoryListener vfio_memory_listener = { .name ="vfio" , .region_add = vfio_listener_region_add,
@@ -907,6 +965,31 @@ VFIOAddressSpace *vfio_get_address_space(AddressSpace *as)return space; } +void vfio_as_add_container (VFIOAddressSpace *space, +VFIOContainer *container) +{ +if (space->listener_initialized) { +memory_listener_unregister (&space->listener); + } + +QLIST_INSERT_HEAD (&space->containers, container, next); + +/* Unregistration happen in vfio_as_del_container() */ + space->listener = vfio_memory_listener; +memory_listener_register (&space->listener, space->as); + space->listener_initialized = true; +} + +void vfio_as_del_container (VFIOAddressSpace *space, +VFIOContainer *container) +{ +QLIST_SAFE_REMOVE (container, next); + +if (QLIST_EMPTY (&space->containers)) { +memory_listener_unregister (&space->listener); + } +} +void vfio_put_address_space (VFIOAddressSpace *space) {if (QLIST_EMPTY (&space->containers)) {
hw/vfio/container-base.c +11 -2
@@ -47,6 +47,17 @@ int vfio_container_dma_map(VFIOContainer *container,return container->ops->dma_map (container, iova, size, vaddr, readonly); } +int vfio_container_dma_copy (VFIOContainer *src,VFIOContainer *dst, +hwaddr iova,ram_addr_t size,bool readonly) +{ +if (!src->ops->dma_copy || src->ops->dma_copy != dst->ops->dma_copy) { +error_report ("Incompatible container: unable to copy dma" ); +return -EINVAL ; + } + +return src->ops->dma_copy (src, dst, iova, size, readonly); +} +int vfio_container_dma_unmap (VFIOContainer *container,hwaddr iova,ram_addr_t size,IOMMUTLBEntry *iotlb)
@@ -137,8 +148,6 @@ void vfio_container_destroy(VFIOContainer *container)VFIOGuestIOMMU *giommu, *tmp;VFIOHostDMAWindow *hostwin, *next; -QLIST_SAFE_REMOVE (container, next); - QLIST_FOREACH_SAFE(vrdl, &container->vrdl_list, next, vrdl_tmp) {RamDiscardManager *rdm;
hw/vfio/container.c +7 -12
@@ -362,9 +362,6 @@ err_out:static void vfio_listener_release (VFIOLegacyContainer *container) { -VFIOContainer *bcontainer = &container->bcontainer; - -memory_listener_unregister (&bcontainer->listener);if (container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU) {memory_listener_unregister (&container->prereg_listener); }
@@ -894,14 +891,11 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as,vfio_kvm_device_add_group (group);QLIST_INIT (&container->group_list); -QLIST_INSERT_HEAD (&space->containers, bcontainer, next); group->container = container;QLIST_INSERT_HEAD (&container->group_list, group, container_next); - bcontainer->listener = vfio_memory_listener; - -memory_listener_register (&bcontainer->listener, bcontainer->space->as); +vfio_as_add_container (space, bcontainer);if (bcontainer->error) { ret =-1 ;
@@ -914,8 +908,8 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as,return 0 ; listener_release_exit: +vfio_as_del_container (space, bcontainer);QLIST_REMOVE (group, container_next); -QLIST_REMOVE (bcontainer, next);vfio_kvm_device_del_group (group);vfio_listener_release (container);
@@ -938,6 +932,7 @@ static void vfio_disconnect_container(VFIOGroup *group) {VFIOLegacyContainer *container = group->container;VFIOContainer *bcontainer = &container->bcontainer; +VFIOAddressSpace *space = bcontainer->space;QLIST_REMOVE (group, container_next); group->container =NULL ;
@@ -945,10 +940,12 @@ static void vfio_disconnect_container(VFIOGroup *group)/* * Explicitly release the listener first before unset container, * since unset may destroy the backend container if it's the last -* group. +* group. By removing container from the list, container is disconnected +* with address space memory listener. */ if (QLIST_EMPTY (&container->group_list)) {vfio_listener_release (container); +vfio_as_del_container (space, bcontainer); }if (ioctl (group->fd,VFIO_GROUP_UNSET_CONTAINER , &container->fd)) {
@@ -957,10 +954,8 @@ static void vfio_disconnect_container(VFIOGroup *group) }if (QLIST_EMPTY (&container->group_list)) { -VFIOAddressSpace *space = bcontainer->space; - -vfio_container_destroy (bcontainer);trace_vfio_disconnect_container (container->fd); +vfio_container_destroy (bcontainer);close (container->fd);g_free (container);
hw/vfio/iommufd.c +38 -9
97 lines not on this page · raw message
include/hw/vfio/vfio-common.h +4 -1
@@ -34,7 +34,6 @@#define VFIO_MSG_PREFIX "vfio %s: " -extern const MemoryListener vfio_memory_listener;extern const VFIOContainerOps legacy_container_ops;extern const VFIOContainerOps iommufd_container_ops;
@@ -186,6 +185,10 @@ void vfio_host_win_add(VFIOContainer *bcontainer,int vfio_host_win_del (VFIOContainer *bcontainer,hwaddr min_iova,hwaddr max_iova);VFIOAddressSpace *vfio_get_address_space (AddressSpace *as); +void vfio_as_add_container (VFIOAddressSpace *space, +VFIOContainer *bcontainer); +void vfio_as_del_container (VFIOAddressSpace *space, +VFIOContainer *container);void vfio_put_address_space (VFIOAddressSpace *space);void vfio_put_base_device (VFIODevice *vbasedev);
include/hw/vfio/vfio-container-base.h +7 -1
@@ -31,12 +31,15 @@typedef enum VFIOContainerFeature {VFIO_FEAT_LIVE_MIGRATION , +VFIO_FEAT_DMA_COPY , }VFIOContainerFeature ;typedef struct VFIOContainer VFIOContainer ;typedef struct VFIOAddressSpace {AddressSpace *as; +MemoryListener listener; +bool listener_initialized; QLIST_HEAD(, VFIOContainer) containers;QLIST_ENTRY (VFIOAddressSpace) list; }VFIOAddressSpace ;
@@ -75,6 +78,8 @@ typedef struct VFIOContainerOps {int (*dma_map)(VFIOContainer *container,hwaddr iova,ram_addr_t size,void *vaddr,bool readonly); +int (*dma_copy)(VFIOContainer *src,VFIOContainer *dst, +hwaddr iova,ram_addr_t size,bool readonly);int (*dma_unmap)(VFIOContainer *container,hwaddr iova,ram_addr_t size,IOMMUTLBEntry *iotlb);
@@ -101,7 +106,6 @@ typedef struct VFIOContainerOps {struct VFIOContainer {const VFIOContainerOps *ops;VFIOAddressSpace *space; -MemoryListener listener;Error *error;bool initialized;bool dirty_pages_supported;
@@ -120,6 +124,8 @@ bool vfio_container_check_extension(VFIOContainer *container,int vfio_container_dma_map (VFIOContainer *container,hwaddr iova,ram_addr_t size,void *vaddr,bool readonly); +int vfio_container_dma_copy (VFIOContainer *src,VFIOContainer *dst, +hwaddr iova,ram_addr_t size,bool readonly);int vfio_container_dma_unmap (VFIOContainer *container,hwaddr iova,ram_addr_t size,IOMMUTLBEntry *iotlb);
Signature · 1 line
2.27.0
RFC v2 15/15 vfio/as: Allow the selection of a given iommu backend
In reply to Yi LiuFrom: Eric Auger <[email protected]>
Now we support two types of iommu backends, let's add the capability to select one of them. This depends on whether an iommufd object has been linked with the vfio-pci device:
if the user wants to use the legacy backend, it shall not link the vfio-pci device with any iommufd object:
-device vfio-pci,host=0000:02:00.0
This is called the legacy mode/backend.
If the user wants to use the iommufd backend (/dev/iommu) it shall pass an iommufd object id in the vfio-pci device options:
-object iommufd,id=iommufd0
-device vfio-pci,host=0000:02:00.0,iommufd=iommufd0
Note the /dev/iommu device may have been pre-opened by a management tool such as libvirt. This mode is no more considered for the legacy backend. So let's remove the "TODO" comment.
--- hw/vfio/as.c | 9 ++++++--- hw/vfio/pci.c | 19 ++++++++++++++----- 2 files changed, 20 insertions(+), 8 deletions(-)
hw/vfio/as.c +6 -3
@@ -1007,6 +1007,8 @@ vfio_get_container_ops(VFIOIOMMUBackendType be)switch (be) {case VFIO_IOMMU_BACKEND_TYPE_LEGACY :return &legacy_container_ops; +case VFIO_IOMMU_BACKEND_TYPE_IOMMUFD : +return &iommufd_container_ops;default :return NULL ; }
@@ -1016,9 +1018,10 @@ int vfio_attach_device(VFIODevice *vbasedev, AddressSpace *as, Error **errp) {const VFIOContainerOps *ops; - ops =vfio_get_container_ops (VFIO_IOMMU_BACKEND_TYPE_LEGACY ); -if (!ops) { -return -ENOENT ; +if (vbasedev->iommufd) { + ops =vfio_get_container_ops (VFIO_IOMMU_BACKEND_TYPE_IOMMUFD ); + }else { + ops =vfio_get_container_ops (VFIO_IOMMU_BACKEND_TYPE_LEGACY ); }return ops->attach_device (vbasedev, as, errp); }
hw/vfio/pci.c +14 -5
@@ -42,6 +42,7 @@#include "qapi/error.h" #include "migration/blocker.h" #include "migration/qemu-file.h" +#include "sysemu/iommufd.h" #define TYPE_VFIO_PCI_NOHOTPLUG "vfio-pci-nohotplug"
@@ -2852,6 +2853,13 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)int i, ret;bool is_mdev; +if (vbasedev->iommufd) { +iommufd_backend_connect (vbasedev->iommufd, errp); +if (*errp) { +return ; + } + } +if (!vbasedev->sysfsdev) {if (!(~vdev->host.domain || ~vdev->host.bus || ~vdev->host.slot || ~vdev->host.function)) {
@@ -3134,6 +3142,7 @@ error:static void vfio_instance_finalize (Object *obj) {VFIOPCIDevice *vdev =VFIO_PCI (obj); +VFIODevice *vbasedev = &vdev->vbasedev;vfio_display_finalize (vdev);vfio_bars_finalize (vdev);
@@ -3146,6 +3155,9 @@ static void vfio_instance_finalize(Object *obj)* * g_free(vdev->igd_opregion); */ +if (vbasedev->iommufd) { +iommufd_backend_disconnect (vbasedev->iommufd); + }vfio_put_device (vdev); }
@@ -3281,11 +3293,8 @@ static Property vfio_pci_dev_properties[] = { qdev_prop_nv_gpudirect_clique, uint8_t),DEFINE_PROP_OFF_AUTO_PCIBAR ("x-msix-relocation" , VFIOPCIDevice, msix_relo,OFF_AUTOPCIBAR_OFF ), -/* -* TODO - support passed fds... is this necessary? -* DEFINE_PROP_STRING("vfiofd", VFIOPCIDevice, vfiofd_name), -* DEFINE_PROP_STRING("vfiogroupfd, VFIOPCIDevice, vfiogroupfd_name), -*/ +DEFINE_PROP_LINK ("iommufd" , VFIOPCIDevice, vbasedev.iommufd, + TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *), DEFINE_PROP_END_OF_LIST(), };
Signature · 1 line
2.27.0