vfio: expose virtual Shared Virtual Addressing to VMs
Subsystems (5)
v6 cover vfio: expose virtual Shared Virtual Addressing to VMs
Shared Virtual Addressing (SVA), a.k.a, Shared Virtual Memory (SVM) on Intel platforms allows address space sharing between device DMA and applications. SVA can reduce programming complexity and enhance security.
This VFIO series is intended to expose SVA usage to VMs. i.e. Sharing guest application address space with passthru devices. This is called vSVA in this series. The whole vSVA enabling requires QEMU/VFIO/IOMMU changes. For IOMMU and QEMU changes, they are in separate series (listed in the "Related series").
The high-level architecture for SVA virtualization is as below, the key design of vSVA support is to utilize the dual-stage IOMMU translation ( also known as IOMMU nesting translation) capability in host IOMMU.
.-------------. .---------------------------.
| vIOMMU | | Guest process CR3, FL only|
| | '---------------------------' .----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest
------| Shadow |--------------------------|--------
v v v
Host
.-------------. .----------------------.
| pIOMMU | | Bind FL for GVA-GPA |
| | '----------------------'
.----------------/ |
| PASID Entry | V (Nested xlate)
'----------------\.------------------------------.
| | |SL for GPA-HPA, default domain|
| | '------------------------------'
'-------------'Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables
Patch Overview:
1. a refactor to vfio_iommu_type1 ioctl (patch 0001)
2. reports IOMMU nesting info to userspace ( patch 0002, 0003, 0004 and 0015)
3. vfio support for PASID allocation and free for VMs (patch 0005, 0006, 0007)
4. vfio support for binding guest page table to host (patch 0008, 0009, 0010)
5. vfio support for IOMMU cache invalidation from VMs (patch 0011)
6. vfio support for vSVA usage on IOMMU-backed mdevs (patch 0012)
7. expose PASID capability to VM (patch 0013)
8. add doc for VFIO dual stage control (patch 0014)
The complete vSVA kernel upstream patches are divided into three phases:
1. Common APIs and PCI device direct assignment
2. IOMMU-backed Mediated Device assignment
3. Page Request Services (PRS) support
This patchset is aiming for the phase 1 and phase 2, and based on Jacob's below series.
*) [PATCH v6 0/6] IOMMU user API enhancement - wip
https://lore.kernel.org/linux-iommu/[email protected]/
*) [PATCH 00/10] IOASID extensions for guest SVA - wip
https://lore.kernel.org/linux-iommu/[email protected]/
The latest IOASID code added below new interface for itertate all PASIDs of an ioasid_set. The implementation is not sent out yet as Jacob needs some cleanup, it can be found in branch vsva-linux-5.8-rc6-v6 on github (mentioned below):
int ioasid_set_for_each_ioasid(int sid, void (*fn)(ioasid_t id, void *data), void *data);
Complete set for current vSVA can be found in below branch.
https://github.com/luxis1999/linux-vsva.git: vsva-linux-5.8-rc6-v6
The corresponding QEMU patch series is included in below branch:
https://github.com/luxis1999/qemu.git: vsva_5.8_rc6_qemu_rfcv9
Regards,
Yi Liu
Changelog:
- Patch v5 -> Patch v6:
a) Address comments against v5 from Eric.
b) rebase on Jacob's v6 IOMMU uapi enhancement
Patch v5: https://lore.kernel.org/kvm/[email protected]/
- Patch v4 -> Patch v5:
a) Address comments against v4
Patch v4: https://lore.kernel.org/kvm/[email protected]/
- Patch v3 -> Patch v4:
a) Address comments against v3
b) Add rb from Stefan on patch 14/15
Patch v3: https://lore.kernel.org/linux-iommu/[email protected]/
- Patch v2 -> Patch v3:
a) Rebase on top of Jacob's v3 iommu uapi patchset
b) Address comments from Kevin and Stefan Hajnoczi
c) Reuse DOMAIN_ATTR_NESTING to get iommu nesting info
d) Drop [PATCH v2 07/15] iommu/uapi: Add iommu_gpasid_unbind_data
Patch v2: https://lore.kernel.org/linux-iommu/[email protected]/#r
- Patch v1 -> Patch v2:
a) Refactor vfio_iommu_type1_ioctl() per suggestion from Christoph
Hellwig.
b) Re-sequence the patch series for better bisect support.
c) Report IOMMU nesting cap info in detail instead of a format in
v1.
d) Enforce one group per nesting type container for vfio iommu type1
driver.
e) Build the vfio_mm related code from vfio.c to be a separate
vfio_pasid.ko.
f) Add PASID ownership check in IOMMU driver.
g) Adopted to latest IOMMU UAPI design. Removed IOMMU UAPI version
check. Added iommu_gpasid_unbind_data for unbind requests from
userspace.
h) Define a single ioctl:VFIO_IOMMU_NESTING_OP for bind/unbind_gtbl
and cahce_invld.
i) Document dual stage control in vfio.rst.
Patch v1: https://lore.kernel.org/linux-iommu/[email protected]/
- RFC v3 -> Patch v1:
a) Address comments to the PASID request(alloc/free) path
b) Report PASID alloc/free availabitiy to user-space
c) Add a vfio_iommu_type1 parameter to support pasid quota tuning
d) Adjusted to latest ioasid code implementation. e.g. remove the
code for tracking the allocated PASIDs as latest ioasid code
will track it, VFIO could use ioasid_free_set() to free all
PASIDs.
RFC v3: https://lore.kernel.org/linux-iommu/[email protected]/
- RFC v2 -> v3:
a) Refine the whole patchset to fit the roughly parts in this series
b) Adds complete vfio PASID management framework. e.g. pasid alloc,
free, reclaim in VM crash/down and per-VM PASID quota to prevent
PASID abuse.
c) Adds IOMMU uAPI version check and page table format check to ensure
version compatibility and hardware compatibility.
d) Adds vSVA vfio support for IOMMU-backed mdevs.
RFC v2: https://lore.kernel.org/linux-iommu/[email protected]/
- RFC v1 -> v2:
Dropped vfio: VFIO_IOMMU_ATTACH/DETACH_PASID_TABLE.
RFC v1: https://lore.kernel.org/linux-iommu/[email protected]/
---
Eric Auger (1):
vfio: Document dual stage control
Liu Yi L (13):
vfio/type1: Refactor vfio_iommu_type1_ioctl()
iommu: Report domain nesting info
iommu/smmu: Report empty domain nesting info vfio/type1: Report iommu nesting info to userspace
vfio: Add PASID allocation/free support iommu/vt-d: Support setting ioasid set to domain vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free) iommu/vt-d: Check ownership for PASIDs from user-space vfio/type1: Support binding guest page tables to PASID vfio/type1: Allow invalidating first-level/stage IOMMU cache vfio/type1: Add vSVA support for IOMMU-backed mdevs vfio/pci: Expose PCIe PASID capability to guest iommu/vt-d: Support reporting nesting capability info
Yi Sun (1):
iommu: Pass domain to sva_unbind_gpasid()
Documentation/driver-api/vfio.rst | 75 ++++ drivers/iommu/arm-smmu-v3.c | 29 +- drivers/iommu/arm-smmu.c | 29 +- drivers/iommu/intel/iommu.c | 114 +++++- drivers/iommu/intel/svm.c | 10 +- drivers/iommu/iommu.c | 2 +- drivers/vfio/Kconfig | 6 + drivers/vfio/Makefile | 1 + drivers/vfio/pci/vfio_pci_config.c | 2 +- drivers/vfio/vfio_iommu_type1.c | 796 ++++++++++++++++++++++++++++--------- drivers/vfio/vfio_pasid.c | 284 +++++++++++++ include/linux/intel-iommu.h | 23 +- include/linux/iommu.h | 4 +- include/linux/vfio.h | 54 +++ include/uapi/linux/iommu.h | 74 ++++ include/uapi/linux/vfio.h | 90 +++++ 16 files changed, 1391 insertions(+), 202 deletions(-) create mode 100644 drivers/vfio/vfio_pasid.c
Signature · 1 line
2.7.4
PATCH v6 1/15 vfio/type1: Refactor vfio_iommu_type1_ioctl()
This patch refactors the vfio_iommu_type1_ioctl() to use switch instead of if-else, and each command got a helper function.
---
v4 -> v5:
*) address comments from Eric Auger, add r-b from Eric.
--- drivers/vfio/vfio_iommu_type1.c | 394 ++++++++++++++++++++++------------------ 1 file changed, 213 insertions(+), 181 deletions(-)
drivers/vfio/vfio_iommu_type1.c +213 -181
@@ -2453,6 +2453,23 @@ static int vfio_domains_have_iommu_cache(struct vfio_iommu *iommu)return ret; } +static int vfio_iommu_type1_check_extension (struct vfio_iommu *iommu, +unsigned long arg) +{ +switch (arg) { +case VFIO_TYPE1_IOMMU : +case VFIO_TYPE1v2_IOMMU: +case VFIO_TYPE1_NESTING_IOMMU : +return 1 ; +case VFIO_DMA_CC_IOMMU : +if (!iommu) +return 0 ; +return vfio_domains_have_iommu_cache (iommu); +default : +return 0 ; + } +} +static int vfio_iommu_iova_add_cap (struct vfio_info_cap *caps,struct vfio_iommu_type1_info_cap_iova_range *cap_iovas,size_t size)
@@ -2529,241 +2546,256 @@ static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig)); } -static long vfio_iommu_type1_ioctl (void *iommu_data, -unsigned int cmd,unsigned long arg) +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu, +unsigned long arg) { -struct vfio_iommu *iommu = iommu_data; +struct vfio_iommu_type1_info info;unsigned long minsz; +struct vfio_info_cap caps = { .buf =NULL , .size =0 }; +unsigned long capsz; +int ret; -if (cmd ==VFIO_CHECK_EXTENSION ) { -switch (arg) { -case VFIO_TYPE1_IOMMU : -case VFIO_TYPE1v2_IOMMU: -case VFIO_TYPE1_NESTING_IOMMU : -return 1 ; -case VFIO_DMA_CC_IOMMU : -if (!iommu) -return 0 ; -return vfio_domains_have_iommu_cache (iommu); -default : -return 0 ; - } - }else if (cmd ==VFIO_IOMMU_GET_INFO ) { -struct vfio_iommu_type1_info info; -struct vfio_info_cap caps = { .buf =NULL , .size =0 }; -unsigned long capsz; -int ret; - - minsz = offsetofend(struct vfio_iommu_type1_info, iova_pgsizes); + minsz = offsetofend(struct vfio_iommu_type1_info, iova_pgsizes); -/* For backward compatibility, cannot require this */ - capsz = offsetofend(struct vfio_iommu_type1_info, cap_offset); +/* For backward compatibility, cannot require this */ + capsz = offsetofend(struct vfio_iommu_type1_info, cap_offset); -if (copy_from_user (&info, (void __user *)arg, minsz)) -return -EFAULT ; +if (copy_from_user (&info, (void __user *)arg, minsz)) +return -EFAULT ; -if (info.argsz < minsz) -return -EINVAL ; +if (info.argsz < minsz) +return -EINVAL ; -if (info.argsz >= capsz) { - minsz = capsz; - info.cap_offset =0 ;/* output, no-recopy necessary */ - } +if (info.argsz >= capsz) { + minsz = capsz; + info.cap_offset =0 ;/* output, no-recopy necessary */ + } -mutex_lock (&iommu->lock); - info.flags =VFIO_IOMMU_INFO_PGSIZES ; +mutex_lock (&iommu->lock); + info.flags =VFIO_IOMMU_INFO_PGSIZES ; - info.iova_pgsizes = iommu->pgsize_bitmap; + info.iova_pgsizes = iommu->pgsize_bitmap; - ret =vfio_iommu_migration_build_caps (iommu, &caps); + ret =vfio_iommu_migration_build_caps (iommu, &caps); -if (!ret) - ret =vfio_iommu_iova_build_caps (iommu, &caps); +if (!ret) + ret =vfio_iommu_iova_build_caps (iommu, &caps); -mutex_unlock (&iommu->lock); +mutex_unlock (&iommu->lock); -if (ret) -return ret; +if (ret) +return ret; -if (caps.size) { - info.flags |=VFIO_IOMMU_INFO_CAPS ; +if (caps.size) { + info.flags |=VFIO_IOMMU_INFO_CAPS ; -if (info.argsz <sizeof (info) + caps.size) { - info.argsz =sizeof (info) + caps.size; - }else { -vfio_info_cap_shift (&caps,sizeof (info)); -if (copy_to_user ((void __user *)arg + -sizeof (info), caps.buf, - caps.size)) { -kfree (caps.buf); -return -EFAULT ; - } - info.cap_offset =sizeof (info); +if (info.argsz <sizeof (info) + caps.size) { + info.argsz =sizeof (info) + caps.size; + }else { +vfio_info_cap_shift (&caps,sizeof (info)); +if (copy_to_user ((void __user *)arg + +sizeof (info), caps.buf, + caps.size)) { +kfree (caps.buf); +return -EFAULT ; } - -kfree (caps.buf); + info.cap_offset =sizeof (info); } -return copy_to_user ((void __user *)arg, &info, minsz) ? - -EFAULT :0 ; +kfree (caps.buf); + } - }else if (cmd ==VFIO_IOMMU_MAP_DMA ) { -struct vfio_iommu_type1_dma_map map; -uint32_t mask =VFIO_DMA_MAP_FLAG_READ | -VFIO_DMA_MAP_FLAG_WRITE ; +return copy_to_user ((void __user *)arg, &info, minsz) ? + -EFAULT :0 ; +} - minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); +static int vfio_iommu_type1_map_dma (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dma_map map; +unsigned long minsz; +uint32_t mask =VFIO_DMA_MAP_FLAG_READ |VFIO_DMA_MAP_FLAG_WRITE ; -if (copy_from_user (&map, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); -if (map.argsz < minsz || map.flags & ~mask) -return -EINVAL ; +if (copy_from_user (&map, (void __user *)arg, minsz)) +return -EFAULT ; -return vfio_dma_do_map (iommu, &map); +if (map.argsz < minsz || map.flags & ~mask) +return -EINVAL ; - }else if (cmd ==VFIO_IOMMU_UNMAP_DMA ) { -struct vfio_iommu_type1_dma_unmap unmap; -struct vfio_bitmap bitmap = {0 }; -int ret; +return vfio_dma_do_map (iommu, &map); +} - minsz = offsetofend(struct vfio_iommu_type1_dma_unmap, size); +static int vfio_iommu_type1_unmap_dma (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dma_unmap unmap; +struct vfio_bitmap bitmap = {0 }; +unsigned long minsz; +int ret; -if (copy_from_user (&unmap, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dma_unmap, size); -if (unmap.argsz < minsz || - unmap.flags & ~VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) -return -EINVAL ; +if (copy_from_user (&unmap, (void __user *)arg, minsz)) +return -EFAULT ; -if (unmap.flags &VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) { -unsigned long pgshift; +if (unmap.argsz < minsz || + unmap.flags & ~VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) +return -EINVAL ; -if (unmap.argsz < (minsz +sizeof (bitmap))) -return -EINVAL ; +if (unmap.flags &VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) { +unsigned long pgshift; -if (copy_from_user (&bitmap, - (void __user *)(arg + minsz), -sizeof (bitmap))) -return -EFAULT ; +if (unmap.argsz < (minsz +sizeof (bitmap))) +return -EINVAL ; -if (!access_ok ((void __user *)bitmap.data, bitmap.size)) -return -EINVAL ; +if (copy_from_user (&bitmap, + (void __user *)(arg + minsz), +sizeof (bitmap))) +return -EFAULT ; - pgshift =__ffs (bitmap.pgsize); - ret =verify_bitmap_size (unmap.size >> pgshift, - bitmap.size); -if (ret) -return ret; - } +if (!access_ok ((void __user *)bitmap.data, bitmap.size)) +return -EINVAL ; - ret =vfio_dma_do_unmap (iommu, &unmap, &bitmap); + pgshift =__ffs (bitmap.pgsize); + ret =verify_bitmap_size (unmap.size >> pgshift, + bitmap.size);if (ret)return ret; + } + + ret =vfio_dma_do_unmap (iommu, &unmap, &bitmap); +if (ret) +return ret; -return copy_to_user ((void __user *)arg, &unmap, minsz) ? +return copy_to_user ((void __user *)arg, &unmap, minsz) ? -EFAULT :0 ; - }else if (cmd ==VFIO_IOMMU_DIRTY_PAGES ) { -struct vfio_iommu_type1_dirty_bitmap dirty; -uint32_t mask =VFIO_IOMMU_DIRTY_PAGES_FLAG_START | -VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP | -VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ; -int ret =0 ; +} -if (!iommu->v2) -return -EACCES ; +static int vfio_iommu_type1_dirty_pages (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dirty_bitmap dirty; +uint32_t mask =VFIO_IOMMU_DIRTY_PAGES_FLAG_START | +VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP | +VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ; +unsigned long minsz; +int ret =0 ; - minsz = offsetofend(struct vfio_iommu_type1_dirty_bitmap, - flags); +if (!iommu->v2) +return -EACCES ; -if (copy_from_user (&dirty, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dirty_bitmap, flags); -if (dirty.argsz < minsz || dirty.flags & ~mask) -return -EINVAL ; +if (copy_from_user (&dirty, (void __user *)arg, minsz)) +return -EFAULT ; -/* only one flag should be set at a time */ -if (__ffs (dirty.flags) !=__fls (dirty.flags)) -return -EINVAL ; +if (dirty.argsz < minsz || dirty.flags & ~mask) +return -EINVAL ; -if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_START ) { -size_t pgsize; +/* only one flag should be set at a time */ +if (__ffs (dirty.flags) !=__fls (dirty.flags)) +return -EINVAL ; -mutex_lock (&iommu->lock); - pgsize =1 <<__ffs (iommu->pgsize_bitmap); -if (!iommu->dirty_page_tracking) { - ret =vfio_dma_bitmap_alloc_all (iommu, pgsize); -if (!ret) - iommu->dirty_page_tracking = true; - } -mutex_unlock (&iommu->lock); -return ret; - }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP ) { -mutex_lock (&iommu->lock); -if (iommu->dirty_page_tracking) { - iommu->dirty_page_tracking = false; -vfio_dma_bitmap_free_all (iommu); - } -mutex_unlock (&iommu->lock); -return 0 ; - }else if (dirty.flags & -VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ) { -struct vfio_iommu_type1_dirty_bitmap_get range; -unsigned long pgshift; -size_t data_size = dirty.argsz - minsz; -size_t iommu_pgsize; - -if (!data_size || data_size <sizeof (range)) -return -EINVAL ; - -if (copy_from_user (&range, (void __user *)(arg + minsz), -sizeof (range))) -return -EFAULT ; +if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_START ) { +size_t pgsize; -if (range.iova + range.size < range.iova) -return -EINVAL ; -if (!access_ok ((void __user *)range.bitmap.data, - range.bitmap.size)) -return -EINVAL ; +mutex_lock (&iommu->lock); + pgsize =1 <<__ffs (iommu->pgsize_bitmap); +if (!iommu->dirty_page_tracking) { + ret =vfio_dma_bitmap_alloc_all (iommu, pgsize); +if (!ret) + iommu->dirty_page_tracking = true; + } +mutex_unlock (&iommu->lock); +return ret; + }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP ) { +mutex_lock (&iommu->lock); +if (iommu->dirty_page_tracking) { + iommu->dirty_page_tracking = false; +vfio_dma_bitmap_free_all (iommu); + } +mutex_unlock (&iommu->lock); +return 0 ; + }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ) { +struct vfio_iommu_type1_dirty_bitmap_get range; +unsigned long pgshift; +size_t data_size = dirty.argsz - minsz; +size_t iommu_pgsize; - pgshift =__ffs (range.bitmap.pgsize); - ret =verify_bitmap_size (range.size >> pgshift, - range.bitmap.size); -if (ret) -return ret; +if (!data_size || data_size <sizeof (range)) +return -EINVAL ; -mutex_lock (&iommu->lock); +if (copy_from_user (&range, (void __user *)(arg + minsz), +sizeof (range))) +return -EFAULT ; - iommu_pgsize = (size_t )1 <<__ffs (iommu->pgsize_bitmap); +if (range.iova + range.size < range.iova) +return -EINVAL ; +if (!access_ok ((void __user *)range.bitmap.data, + range.bitmap.size)) +return -EINVAL ; -/* allow only smallest supported pgsize */ -if (range.bitmap.pgsize != iommu_pgsize) { - ret = -EINVAL ; - goto out_unlock; - } -if (range.iova & (iommu_pgsize -1 )) { - ret = -EINVAL ; - goto out_unlock; - } -if (!range.size || range.size & (iommu_pgsize -1 )) { - ret = -EINVAL ; - goto out_unlock; - } + pgshift =__ffs (range.bitmap.pgsize); + ret =verify_bitmap_size (range.size >> pgshift, + range.bitmap.size); +if (ret) +return ret; -if (iommu->dirty_page_tracking) - ret =vfio_iova_dirty_bitmap (range.bitmap.data, - iommu, range.iova, range.size, - range.bitmap.pgsize); -else - ret = -EINVAL ; -out_unlock: -mutex_unlock (&iommu->lock); +mutex_lock (&iommu->lock); -return ret; + iommu_pgsize = (size_t )1 <<__ffs (iommu->pgsize_bitmap); + +/* allow only smallest supported pgsize */ +if (range.bitmap.pgsize != iommu_pgsize) { + ret = -EINVAL ; + goto out_unlock; + } +if (range.iova & (iommu_pgsize -1 )) { + ret = -EINVAL ; + goto out_unlock; + } +if (!range.size || range.size & (iommu_pgsize -1 )) { + ret = -EINVAL ; + goto out_unlock; } + +if (iommu->dirty_page_tracking) + ret =vfio_iova_dirty_bitmap (range.bitmap.data, + iommu, range.iova, + range.size, + range.bitmap.pgsize); +else + ret = -EINVAL ; +out_unlock: +mutex_unlock (&iommu->lock); + +return ret; } -return -ENOTTY ; +return -EINVAL ; +} + +static long vfio_iommu_type1_ioctl (void *iommu_data, +unsigned int cmd,unsigned long arg) +{ +struct vfio_iommu *iommu = iommu_data; + +switch (cmd) { +case VFIO_CHECK_EXTENSION : +return vfio_iommu_type1_check_extension (iommu, arg); +case VFIO_IOMMU_GET_INFO : +return vfio_iommu_type1_get_info (iommu, arg); +case VFIO_IOMMU_MAP_DMA : +return vfio_iommu_type1_map_dma (iommu, arg); +case VFIO_IOMMU_UNMAP_DMA : +return vfio_iommu_type1_unmap_dma (iommu, arg); +case VFIO_IOMMU_DIRTY_PAGES : +return vfio_iommu_type1_dirty_pages (iommu, arg); +default : +return -ENOTTY ; + } } static int vfio_iommu_type1_register_notifier(void *iommu_data,
Signature · 1 line
2.7.4
Re: [PATCH v6 01/15] vfio/type1: Refactor vfio_iommu_type1_ioctl()
Quoted 13 lines · On Mon, 27 Jul 2020 23:27:30 -0700 Liu Yi L <[email protected]> wrote:
This patch refactors the
vfio_iommu_type1_ioctl()to use switch instead of if-else, and each command got a helper function.Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Eric Auger <[email protected]>Suggested-by Christoph Hellwig <[email protected]>Signed-off-by Liu Yi L <[email protected]>
Quoted 1 line
---
FYI, this commit is already in my next branch and linux-next as of today, you can drop it from future series. Thanks,
Alex
Quoted 472 lines
v4 -> v5:
*) address comments from Eric Auger, add r-b from Eric.--- drivers/vfio/vfio_iommu_type1.c | 394 ++++++++++++++++++++++------------------ 1 file changed, 213 insertions(+), 181 deletions(-)drivers/vfio/vfio_iommu_type1.c +213
-181@@ -2453,6 +2453,23 @@ static int vfio_domains_have_iommu_cache(struct vfio_iommu *iommu)return ret; } +static int vfio_iommu_type1_check_extension (struct vfio_iommu *iommu, +unsigned long arg) +{ +switch (arg) { +case VFIO_TYPE1_IOMMU : +case VFIO_TYPE1v2_IOMMU: +case VFIO_TYPE1_NESTING_IOMMU : +return 1 ; +case VFIO_DMA_CC_IOMMU : +if (!iommu) +return 0 ; +return vfio_domains_have_iommu_cache (iommu); +default : +return 0 ; + } +} +static int vfio_iommu_iova_add_cap (struct vfio_info_cap *caps,struct vfio_iommu_type1_info_cap_iova_range *cap_iovas,size_t size)@@ -2529,241 +2546,256 @@ static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig)); } -static long vfio_iommu_type1_ioctl (void *iommu_data, -unsigned int cmd,unsigned long arg) +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu, +unsigned long arg) { -struct vfio_iommu *iommu = iommu_data; +struct vfio_iommu_type1_info info;unsigned long minsz; +struct vfio_info_cap caps = { .buf =NULL , .size =0 }; +unsigned long capsz; +int ret; -if (cmd ==VFIO_CHECK_EXTENSION ) { -switch (arg) { -case VFIO_TYPE1_IOMMU : -case VFIO_TYPE1v2_IOMMU: -case VFIO_TYPE1_NESTING_IOMMU : -return 1 ; -case VFIO_DMA_CC_IOMMU : -if (!iommu) -return 0 ; -return vfio_domains_have_iommu_cache (iommu); -default : -return 0 ; - } - }else if (cmd ==VFIO_IOMMU_GET_INFO ) { -struct vfio_iommu_type1_info info; -struct vfio_info_cap caps = { .buf =NULL , .size =0 }; -unsigned long capsz; -int ret; - - minsz = offsetofend(struct vfio_iommu_type1_info, iova_pgsizes); + minsz = offsetofend(struct vfio_iommu_type1_info, iova_pgsizes); -/* For backward compatibility, cannot require this */ - capsz = offsetofend(struct vfio_iommu_type1_info, cap_offset); +/* For backward compatibility, cannot require this */ + capsz = offsetofend(struct vfio_iommu_type1_info, cap_offset); -if (copy_from_user (&info, (void __user *)arg, minsz)) -return -EFAULT ; +if (copy_from_user (&info, (void __user *)arg, minsz)) +return -EFAULT ; -if (info.argsz < minsz) -return -EINVAL ; +if (info.argsz < minsz) +return -EINVAL ; -if (info.argsz >= capsz) { - minsz = capsz; - info.cap_offset =0 ;/* output, no-recopy necessary */ - } +if (info.argsz >= capsz) { + minsz = capsz; + info.cap_offset =0 ;/* output, no-recopy necessary */ + } -mutex_lock (&iommu->lock); - info.flags =VFIO_IOMMU_INFO_PGSIZES ; +mutex_lock (&iommu->lock); + info.flags =VFIO_IOMMU_INFO_PGSIZES ; - info.iova_pgsizes = iommu->pgsize_bitmap; + info.iova_pgsizes = iommu->pgsize_bitmap; - ret =vfio_iommu_migration_build_caps (iommu, &caps); + ret =vfio_iommu_migration_build_caps (iommu, &caps); -if (!ret) - ret =vfio_iommu_iova_build_caps (iommu, &caps); +if (!ret) + ret =vfio_iommu_iova_build_caps (iommu, &caps); -mutex_unlock (&iommu->lock); +mutex_unlock (&iommu->lock); -if (ret) -return ret; +if (ret) +return ret; -if (caps.size) { - info.flags |=VFIO_IOMMU_INFO_CAPS ; +if (caps.size) { + info.flags |=VFIO_IOMMU_INFO_CAPS ; -if (info.argsz <sizeof (info) + caps.size) { - info.argsz =sizeof (info) + caps.size; - }else { -vfio_info_cap_shift (&caps,sizeof (info)); -if (copy_to_user ((void __user *)arg + -sizeof (info), caps.buf, - caps.size)) { -kfree (caps.buf); -return -EFAULT ; - } - info.cap_offset =sizeof (info); +if (info.argsz <sizeof (info) + caps.size) { + info.argsz =sizeof (info) + caps.size; + }else { +vfio_info_cap_shift (&caps,sizeof (info)); +if (copy_to_user ((void __user *)arg + +sizeof (info), caps.buf, + caps.size)) { +kfree (caps.buf); +return -EFAULT ; } - -kfree (caps.buf); + info.cap_offset =sizeof (info); } -return copy_to_user ((void __user *)arg, &info, minsz) ? - -EFAULT :0 ; +kfree (caps.buf); + } - }else if (cmd ==VFIO_IOMMU_MAP_DMA ) { -struct vfio_iommu_type1_dma_map map; -uint32_t mask =VFIO_DMA_MAP_FLAG_READ | -VFIO_DMA_MAP_FLAG_WRITE ; +return copy_to_user ((void __user *)arg, &info, minsz) ? + -EFAULT :0 ; +} - minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); +static int vfio_iommu_type1_map_dma (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dma_map map; +unsigned long minsz; +uint32_t mask =VFIO_DMA_MAP_FLAG_READ |VFIO_DMA_MAP_FLAG_WRITE ; -if (copy_from_user (&map, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); -if (map.argsz < minsz || map.flags & ~mask) -return -EINVAL ; +if (copy_from_user (&map, (void __user *)arg, minsz)) +return -EFAULT ; -return vfio_dma_do_map (iommu, &map); +if (map.argsz < minsz || map.flags & ~mask) +return -EINVAL ; - }else if (cmd ==VFIO_IOMMU_UNMAP_DMA ) { -struct vfio_iommu_type1_dma_unmap unmap; -struct vfio_bitmap bitmap = {0 }; -int ret; +return vfio_dma_do_map (iommu, &map); +} - minsz = offsetofend(struct vfio_iommu_type1_dma_unmap, size); +static int vfio_iommu_type1_unmap_dma (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dma_unmap unmap; +struct vfio_bitmap bitmap = {0 }; +unsigned long minsz; +int ret; -if (copy_from_user (&unmap, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dma_unmap, size); -if (unmap.argsz < minsz || - unmap.flags & ~VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) -return -EINVAL ; +if (copy_from_user (&unmap, (void __user *)arg, minsz)) +return -EFAULT ; -if (unmap.flags &VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) { -unsigned long pgshift; +if (unmap.argsz < minsz || + unmap.flags & ~VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) +return -EINVAL ; -if (unmap.argsz < (minsz +sizeof (bitmap))) -return -EINVAL ; +if (unmap.flags &VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) { +unsigned long pgshift; -if (copy_from_user (&bitmap, - (void __user *)(arg + minsz), -sizeof (bitmap))) -return -EFAULT ; +if (unmap.argsz < (minsz +sizeof (bitmap))) +return -EINVAL ; -if (!access_ok ((void __user *)bitmap.data, bitmap.size)) -return -EINVAL ; +if (copy_from_user (&bitmap, + (void __user *)(arg + minsz), +sizeof (bitmap))) +return -EFAULT ; - pgshift =__ffs (bitmap.pgsize); - ret =verify_bitmap_size (unmap.size >> pgshift, - bitmap.size); -if (ret) -return ret; - } +if (!access_ok ((void __user *)bitmap.data, bitmap.size)) +return -EINVAL ; - ret =vfio_dma_do_unmap (iommu, &unmap, &bitmap); + pgshift =__ffs (bitmap.pgsize); + ret =verify_bitmap_size (unmap.size >> pgshift, + bitmap.size);if (ret)return ret; + } + + ret =vfio_dma_do_unmap (iommu, &unmap, &bitmap); +if (ret) +return ret; -return copy_to_user ((void __user *)arg, &unmap, minsz) ? +return copy_to_user ((void __user *)arg, &unmap, minsz) ? -EFAULT :0 ; - }else if (cmd ==VFIO_IOMMU_DIRTY_PAGES ) { -struct vfio_iommu_type1_dirty_bitmap dirty; -uint32_t mask =VFIO_IOMMU_DIRTY_PAGES_FLAG_START | -VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP | -VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ; -int ret =0 ; +} -if (!iommu->v2) -return -EACCES ; +static int vfio_iommu_type1_dirty_pages (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dirty_bitmap dirty; +uint32_t mask =VFIO_IOMMU_DIRTY_PAGES_FLAG_START | +VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP | +VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ; +unsigned long minsz; +int ret =0 ; - minsz = offsetofend(struct vfio_iommu_type1_dirty_bitmap, - flags); +if (!iommu->v2) +return -EACCES ; -if (copy_from_user (&dirty, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dirty_bitmap, flags); -if (dirty.argsz < minsz || dirty.flags & ~mask) -return -EINVAL ; +if (copy_from_user (&dirty, (void __user *)arg, minsz)) +return -EFAULT ; -/* only one flag should be set at a time */ -if (__ffs (dirty.flags) !=__fls (dirty.flags)) -return -EINVAL ; +if (dirty.argsz < minsz || dirty.flags & ~mask) +return -EINVAL ; -if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_START ) { -size_t pgsize; +/* only one flag should be set at a time */ +if (__ffs (dirty.flags) !=__fls (dirty.flags)) +return -EINVAL ; -mutex_lock (&iommu->lock); - pgsize =1 <<__ffs (iommu->pgsize_bitmap); -if (!iommu->dirty_page_tracking) { - ret =vfio_dma_bitmap_alloc_all (iommu, pgsize); -if (!ret) - iommu->dirty_page_tracking = true; - } -mutex_unlock (&iommu->lock); -return ret; - }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP ) { -mutex_lock (&iommu->lock); -if (iommu->dirty_page_tracking) { - iommu->dirty_page_tracking = false; -vfio_dma_bitmap_free_all (iommu); - } -mutex_unlock (&iommu->lock); -return 0 ; - }else if (dirty.flags & -VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ) { -struct vfio_iommu_type1_dirty_bitmap_get range; -unsigned long pgshift; -size_t data_size = dirty.argsz - minsz; -size_t iommu_pgsize; - -if (!data_size || data_size <sizeof (range)) -return -EINVAL ; - -if (copy_from_user (&range, (void __user *)(arg + minsz), -sizeof (range))) -return -EFAULT ; +if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_START ) { +size_t pgsize; -if (range.iova + range.size < range.iova) -return -EINVAL ; -if (!access_ok ((void __user *)range.bitmap.data, - range.bitmap.size)) -return -EINVAL ; +mutex_lock (&iommu->lock); + pgsize =1 <<__ffs (iommu->pgsize_bitmap); +if (!iommu->dirty_page_tracking) { + ret =vfio_dma_bitmap_alloc_all (iommu, pgsize); +if (!ret) + iommu->dirty_page_tracking = true; + } +mutex_unlock (&iommu->lock); +return ret; + }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP ) { +mutex_lock (&iommu->lock); +if (iommu->dirty_page_tracking) { + iommu->dirty_page_tracking = false; +vfio_dma_bitmap_free_all (iommu); + } +mutex_unlock (&iommu->lock); +return 0 ; + }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ) { +struct vfio_iommu_type1_dirty_bitmap_get range; +unsigned long pgshift; +size_t data_size = dirty.argsz - minsz; +size_t iommu_pgsize; - pgshift =__ffs (range.bitmap.pgsize); - ret =verify_bitmap_size (range.size >> pgshift, - range.bitmap.size); -if (ret) -return ret; +if (!data_size || data_size <sizeof (range)) +return -EINVAL ; -mutex_lock (&iommu->lock); +if (copy_from_user (&range, (void __user *)(arg + minsz), +sizeof (range))) +return -EFAULT ; - iommu_pgsize = (size_t )1 <<__ffs (iommu->pgsize_bitmap); +if (range.iova + range.size < range.iova) +return -EINVAL ; +if (!access_ok ((void __user *)range.bitmap.data, + range.bitmap.size)) +return -EINVAL ; -/* allow only smallest supported pgsize */ -if (range.bitmap.pgsize != iommu_pgsize) { - ret = -EINVAL ; - goto out_unlock; - } -if (range.iova & (iommu_pgsize -1 )) { - ret = -EINVAL ; - goto out_unlock; - } -if (!range.size || range.size & (iommu_pgsize -1 )) { - ret = -EINVAL ; - goto out_unlock; - } + pgshift =__ffs (range.bitmap.pgsize); + ret =verify_bitmap_size (range.size >> pgshift, + range.bitmap.size); +if (ret) +return ret; -if (iommu->dirty_page_tracking) - ret =vfio_iova_dirty_bitmap (range.bitmap.data, - iommu, range.iova, range.size, - range.bitmap.pgsize); -else - ret = -EINVAL ; -out_unlock: -mutex_unlock (&iommu->lock); +mutex_lock (&iommu->lock); -return ret; + iommu_pgsize = (size_t )1 <<__ffs (iommu->pgsize_bitmap); + +/* allow only smallest supported pgsize */ +if (range.bitmap.pgsize != iommu_pgsize) { + ret = -EINVAL ; + goto out_unlock; + } +if (range.iova & (iommu_pgsize -1 )) { + ret = -EINVAL ; + goto out_unlock; + } +if (!range.size || range.size & (iommu_pgsize -1 )) { + ret = -EINVAL ; + goto out_unlock; } + +if (iommu->dirty_page_tracking) + ret =vfio_iova_dirty_bitmap (range.bitmap.data, + iommu, range.iova, + range.size, + range.bitmap.pgsize); +else + ret = -EINVAL ; +out_unlock: +mutex_unlock (&iommu->lock); + +return ret; } -return -ENOTTY ; +return -EINVAL ; +} + +static long vfio_iommu_type1_ioctl (void *iommu_data, +unsigned int cmd,unsigned long arg) +{ +struct vfio_iommu *iommu = iommu_data; + +switch (cmd) { +case VFIO_CHECK_EXTENSION : +return vfio_iommu_type1_check_extension (iommu, arg); +case VFIO_IOMMU_GET_INFO : +return vfio_iommu_type1_get_info (iommu, arg); +case VFIO_IOMMU_MAP_DMA : +return vfio_iommu_type1_map_dma (iommu, arg); +case VFIO_IOMMU_UNMAP_DMA : +return vfio_iommu_type1_unmap_dma (iommu, arg); +case VFIO_IOMMU_DIRTY_PAGES : +return vfio_iommu_type1_dirty_pages (iommu, arg); +default : +return -ENOTTY ; + } } static int vfio_iommu_type1_register_notifier(void *iommu_data,
RE: [PATCH v6 01/15] vfio/type1: Refactor vfio_iommu_type1_ioctl()
Quoted 21 lines
From: Alex Williamson <[email protected]>
Sent: Tuesday, July 28, 2020 11:54 PMQuoted 13 lines · On Mon, 27 Jul 2020 23:27:30 -0700 Liu Yi L <[email protected]> wrote:
This patch refactors the
vfio_iommu_type1_ioctl()to use switch instead of if-else, and each command got a helper function.Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Eric Auger <[email protected]>Suggested-by Christoph Hellwig <[email protected]>Signed-off-by Liu Yi L <[email protected]>Quoted 1 line
---
Quoted 2 lines
FYI, this commit is already in my next branch and linux-next as of today, you can drop it from future series. Thanks,
got it. thanks. :-)
Regards,
Yi Liu
Quoted 476 lines
Alex
drivers/vfio/vfio_iommu_type1.c static int vfio_domains_have_iommu_cache(struct
Quoted 12 lines
v4 -> v5:
*) address comments from Eric Auger, add r-b from Eric.--- drivers/vfio/vfio_iommu_type1.c | 394 ++++++++++++++++++++++------------------ 1 file changed, 213 insertions(+), 181 deletions(-)(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex 5e556ac..3bd70ff 100644drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2453,6 +2453,23 @@ static int vfio_domains_have_iommu_cache(struct
vfio_iommu*iommu)drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct
Quoted 24 lines
drivers/vfio/vfio_iommu_type1.c +17
-0return ret; } +static int vfio_iommu_type1_check_extension (struct vfio_iommu *iommu, +unsigned long arg) +{ +switch (arg) { +case VFIO_TYPE1_IOMMU : +case VFIO_TYPE1v2_IOMMU: +case VFIO_TYPE1_NESTING_IOMMU : +return 1 ; +case VFIO_DMA_CC_IOMMU : +if (!iommu) +return 0 ; +return vfio_domains_have_iommu_cache (iommu); +default : +return 0 ; + } +} +static int vfio_iommu_iova_add_cap (struct vfio_info_cap *caps,struct vfio_iommu_type1_info_cap_iova_range *cap_iovas,size_t size)@@ -2529,241 +2546,256 @@ static int vfio_iommu_migration_build_caps(struct
vfio_iommu*iommu,Quoted 126 lines
drivers/vfio/vfio_iommu_type1.c +41
-63return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig)); } -static long vfio_iommu_type1_ioctl (void *iommu_data, -unsigned int cmd,unsigned long arg) +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu, +unsigned long arg) { -struct vfio_iommu *iommu = iommu_data; +struct vfio_iommu_type1_info info;unsigned long minsz; +struct vfio_info_cap caps = { .buf =NULL , .size =0 }; +unsigned long capsz; +int ret; -if (cmd ==VFIO_CHECK_EXTENSION ) { -switch (arg) { -case VFIO_TYPE1_IOMMU : -case VFIO_TYPE1v2_IOMMU: -case VFIO_TYPE1_NESTING_IOMMU : -return 1 ; -case VFIO_DMA_CC_IOMMU : -if (!iommu) -return 0 ; -return vfio_domains_have_iommu_cache (iommu); -default : -return 0 ; - } - }else if (cmd ==VFIO_IOMMU_GET_INFO ) { -struct vfio_iommu_type1_info info; -struct vfio_info_cap caps = { .buf =NULL , .size =0 }; -unsigned long capsz; -int ret; - - minsz = offsetofend(struct vfio_iommu_type1_info, iova_pgsizes); + minsz = offsetofend(struct vfio_iommu_type1_info, iova_pgsizes); -/* For backward compatibility, cannot require this */ - capsz = offsetofend(struct vfio_iommu_type1_info, cap_offset); +/* For backward compatibility, cannot require this */ + capsz = offsetofend(struct vfio_iommu_type1_info, cap_offset); -if (copy_from_user (&info, (void __user *)arg, minsz)) -return -EFAULT ; +if (copy_from_user (&info, (void __user *)arg, minsz)) +return -EFAULT ; -if (info.argsz < minsz) -return -EINVAL ; +if (info.argsz < minsz) +return -EINVAL ; -if (info.argsz >= capsz) { - minsz = capsz; - info.cap_offset =0 ;/* output, no-recopy necessary */ - } +if (info.argsz >= capsz) { + minsz = capsz; + info.cap_offset =0 ;/* output, no-recopy necessary */ + } -mutex_lock (&iommu->lock); - info.flags =VFIO_IOMMU_INFO_PGSIZES ; +mutex_lock (&iommu->lock); + info.flags =VFIO_IOMMU_INFO_PGSIZES ; - info.iova_pgsizes = iommu->pgsize_bitmap; + info.iova_pgsizes = iommu->pgsize_bitmap; - ret =vfio_iommu_migration_build_caps (iommu, &caps); + ret =vfio_iommu_migration_build_caps (iommu, &caps); -if (!ret) - ret =vfio_iommu_iova_build_caps (iommu, &caps); +if (!ret) + ret =vfio_iommu_iova_build_caps (iommu, &caps); -mutex_unlock (&iommu->lock); +mutex_unlock (&iommu->lock); -if (ret) -return ret; +if (ret) +return ret; -if (caps.size) { - info.flags |=VFIO_IOMMU_INFO_CAPS ; +if (caps.size) { + info.flags |=VFIO_IOMMU_INFO_CAPS ; -if (info.argsz <sizeof (info) + caps.size) { - info.argsz =sizeof (info) + caps.size; - }else { -vfio_info_cap_shift (&caps,sizeof (info)); -if (copy_to_user ((void __user *)arg + -sizeof (info), caps.buf, - caps.size)) { -kfree (caps.buf); -return -EFAULT ; - } - info.cap_offset =sizeof (info); +if (info.argsz <sizeof (info) + caps.size) { + info.argsz =sizeof (info) + caps.size; + }else { +vfio_info_cap_shift (&caps,sizeof (info)); +if (copy_to_user ((void __user *)arg + +sizeof (info), caps.buf, + caps.size)) { +kfree (caps.buf); +return -EFAULT ; } - -kfree (caps.buf); + info.cap_offset =sizeof (info); } -return copy_to_user ((void __user *)arg, &info, minsz) ? - -EFAULT :0 ; +kfree (caps.buf); + } - }else if (cmd ==VFIO_IOMMU_MAP_DMA ) { -struct vfio_iommu_type1_dma_map map; -uint32_t mask =VFIO_DMA_MAP_FLAG_READ | -VFIO_DMA_MAP_FLAG_WRITE ; + return copy_to_user((void __user *)arg, &info, minsz) ?drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-1+ -EFAULT : 0; +} - minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); +static int vfio_iommu_type1_map_dma (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dma_map map; +unsigned long minsz; + uint32_t mask = VFIO_DMA_MAP_FLAG_READ |
VFIO_DMA_MAP_FLAG_WRITE;Quoted 143 lines
drivers/vfio/vfio_iommu_type1.c +58
-61-if (copy_from_user (&map, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); -if (map.argsz < minsz || map.flags & ~mask) -return -EINVAL ; +if (copy_from_user (&map, (void __user *)arg, minsz)) +return -EFAULT ; -return vfio_dma_do_map (iommu, &map); +if (map.argsz < minsz || map.flags & ~mask) +return -EINVAL ; - }else if (cmd ==VFIO_IOMMU_UNMAP_DMA ) { -struct vfio_iommu_type1_dma_unmap unmap; -struct vfio_bitmap bitmap = {0 }; -int ret; +return vfio_dma_do_map (iommu, &map); } - minsz = offsetofend(struct vfio_iommu_type1_dma_unmap, size); +static int vfio_iommu_type1_unmap_dma (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dma_unmap unmap; +struct vfio_bitmap bitmap = {0 }; +unsigned long minsz; +int ret; -if (copy_from_user (&unmap, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dma_unmap, size); -if (unmap.argsz < minsz || - unmap.flags & ~VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) -return -EINVAL ; +if (copy_from_user (&unmap, (void __user *)arg, minsz)) +return -EFAULT ; -if (unmap.flags &VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) { -unsigned long pgshift; +if (unmap.argsz < minsz || + unmap.flags & ~VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) +return -EINVAL ; -if (unmap.argsz < (minsz +sizeof (bitmap))) -return -EINVAL ; +if (unmap.flags &VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP ) { +unsigned long pgshift; -if (copy_from_user (&bitmap, - (void __user *)(arg + minsz), -sizeof (bitmap))) -return -EFAULT ; +if (unmap.argsz < (minsz +sizeof (bitmap))) +return -EINVAL ; -if (!access_ok ((void __user *)bitmap.data, bitmap.size)) -return -EINVAL ; +if (copy_from_user (&bitmap, + (void __user *)(arg + minsz), +sizeof (bitmap))) +return -EFAULT ; - pgshift =__ffs (bitmap.pgsize); - ret =verify_bitmap_size (unmap.size >> pgshift, - bitmap.size); -if (ret) -return ret; - } +if (!access_ok ((void __user *)bitmap.data, bitmap.size)) +return -EINVAL ; - ret =vfio_dma_do_unmap (iommu, &unmap, &bitmap); + pgshift =__ffs (bitmap.pgsize); + ret =verify_bitmap_size (unmap.size >> pgshift, + bitmap.size);if (ret)return ret; + } + + ret =vfio_dma_do_unmap (iommu, &unmap, &bitmap); +if (ret) +return ret; -return copy_to_user ((void __user *)arg, &unmap, minsz) ? +return copy_to_user ((void __user *)arg, &unmap, minsz) ? -EFAULT :0 ; - }else if (cmd ==VFIO_IOMMU_DIRTY_PAGES ) { -struct vfio_iommu_type1_dirty_bitmap dirty; -uint32_t mask =VFIO_IOMMU_DIRTY_PAGES_FLAG_START | -VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP | -VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ; -int ret =0 ; +} -if (!iommu->v2) -return -EACCES ; +static int vfio_iommu_type1_dirty_pages (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_dirty_bitmap dirty; +uint32_t mask =VFIO_IOMMU_DIRTY_PAGES_FLAG_START | +VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP | +VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ; +unsigned long minsz; +int ret =0 ; - minsz = offsetofend(struct vfio_iommu_type1_dirty_bitmap, - flags); +if (!iommu->v2) +return -EACCES ; -if (copy_from_user (&dirty, (void __user *)arg, minsz)) -return -EFAULT ; + minsz = offsetofend(struct vfio_iommu_type1_dirty_bitmap, flags); -if (dirty.argsz < minsz || dirty.flags & ~mask) -return -EINVAL ; +if (copy_from_user (&dirty, (void __user *)arg, minsz)) +return -EFAULT ; -/* only one flag should be set at a time */ -if (__ffs (dirty.flags) !=__fls (dirty.flags)) -return -EINVAL ; +if (dirty.argsz < minsz || dirty.flags & ~mask) +return -EINVAL ; -if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_START ) { -size_t pgsize; +/* only one flag should be set at a time */ +if (__ffs (dirty.flags) !=__fls (dirty.flags)) +return -EINVAL ; -mutex_lock (&iommu->lock); - pgsize =1 <<__ffs (iommu->pgsize_bitmap); -if (!iommu->dirty_page_tracking) { - ret =vfio_dma_bitmap_alloc_all (iommu, pgsize); -if (!ret) - iommu->dirty_page_tracking = true; - } -mutex_unlock (&iommu->lock); -return ret;drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +0
-10- }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP ) { -mutex_lock (&iommu->lock); -if (iommu->dirty_page_tracking) { - iommu->dirty_page_tracking = false; -vfio_dma_bitmap_free_all (iommu); - } -mutex_unlock (&iommu->lock); -return 0 ; - }else if (dirty.flags & -VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ){
Quoted 145 lines
drivers/vfio/vfio_iommu_type1.c +87
-46-struct vfio_iommu_type1_dirty_bitmap_get range; -unsigned long pgshift; -size_t data_size = dirty.argsz - minsz; -size_t iommu_pgsize; - -if (!data_size || data_size <sizeof (range)) -return -EINVAL ; - -if (copy_from_user (&range, (void __user *)(arg + minsz), -sizeof (range))) -return -EFAULT ; +if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_START ) { +size_t pgsize; -if (range.iova + range.size < range.iova) -return -EINVAL ; -if (!access_ok ((void __user *)range.bitmap.data, - range.bitmap.size)) -return -EINVAL ; +mutex_lock (&iommu->lock); + pgsize =1 <<__ffs (iommu->pgsize_bitmap); +if (!iommu->dirty_page_tracking) { + ret =vfio_dma_bitmap_alloc_all (iommu, pgsize); +if (!ret) + iommu->dirty_page_tracking = true; + } +mutex_unlock (&iommu->lock); +return ret; + }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP ) { +mutex_lock (&iommu->lock); +if (iommu->dirty_page_tracking) { + iommu->dirty_page_tracking = false; +vfio_dma_bitmap_free_all (iommu); + } +mutex_unlock (&iommu->lock); +return 0 ; + }else if (dirty.flags &VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP ) { +struct vfio_iommu_type1_dirty_bitmap_get range; +unsigned long pgshift; +size_t data_size = dirty.argsz - minsz; +size_t iommu_pgsize; - pgshift =__ffs (range.bitmap.pgsize); - ret =verify_bitmap_size (range.size >> pgshift, - range.bitmap.size); -if (ret) -return ret; +if (!data_size || data_size <sizeof (range)) +return -EINVAL ; -mutex_lock (&iommu->lock); +if (copy_from_user (&range, (void __user *)(arg + minsz), +sizeof (range))) +return -EFAULT ; - iommu_pgsize = (size_t )1 <<__ffs (iommu->pgsize_bitmap); +if (range.iova + range.size < range.iova) +return -EINVAL ; +if (!access_ok ((void __user *)range.bitmap.data, + range.bitmap.size)) +return -EINVAL ; -/* allow only smallest supported pgsize */ -if (range.bitmap.pgsize != iommu_pgsize) { - ret = -EINVAL ; - goto out_unlock; - } -if (range.iova & (iommu_pgsize -1 )) { - ret = -EINVAL ; - goto out_unlock; - } -if (!range.size || range.size & (iommu_pgsize -1 )) { - ret = -EINVAL ; - goto out_unlock; - } + pgshift =__ffs (range.bitmap.pgsize); + ret =verify_bitmap_size (range.size >> pgshift, + range.bitmap.size); +if (ret) +return ret; -if (iommu->dirty_page_tracking) - ret =vfio_iova_dirty_bitmap (range.bitmap.data, - iommu, range.iova, range.size, - range.bitmap.pgsize); -else - ret = -EINVAL ; -out_unlock: -mutex_unlock (&iommu->lock); +mutex_lock (&iommu->lock); -return ret; + iommu_pgsize = (size_t )1 <<__ffs (iommu->pgsize_bitmap); + +/* allow only smallest supported pgsize */ +if (range.bitmap.pgsize != iommu_pgsize) { + ret = -EINVAL ; + goto out_unlock; + } +if (range.iova & (iommu_pgsize -1 )) { + ret = -EINVAL ; + goto out_unlock; + } +if (!range.size || range.size & (iommu_pgsize -1 )) { + ret = -EINVAL ; + goto out_unlock; } + +if (iommu->dirty_page_tracking) + ret =vfio_iova_dirty_bitmap (range.bitmap.data, + iommu, range.iova, + range.size, + range.bitmap.pgsize); +else + ret = -EINVAL ; +out_unlock: +mutex_unlock (&iommu->lock); + +return ret; } -return -ENOTTY ; +return -EINVAL ; +} + +static long vfio_iommu_type1_ioctl (void *iommu_data, +unsigned int cmd,unsigned long arg) { +struct vfio_iommu *iommu = iommu_data; + +switch (cmd) { +case VFIO_CHECK_EXTENSION : +return vfio_iommu_type1_check_extension (iommu, arg); +case VFIO_IOMMU_GET_INFO : +return vfio_iommu_type1_get_info (iommu, arg); +case VFIO_IOMMU_MAP_DMA : +return vfio_iommu_type1_map_dma (iommu, arg); +case VFIO_IOMMU_UNMAP_DMA : +return vfio_iommu_type1_unmap_dma (iommu, arg); +case VFIO_IOMMU_DIRTY_PAGES : +return vfio_iommu_type1_dirty_pages (iommu, arg); +default : +return -ENOTTY ; + } }
Quoted 1 line
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu, · line 2801
Quoted 1 line
drivers/vfio/vfio_iommu_type1.c +0
-0static int vfio_iommu_type1_register_notifier(void *iommu_data,
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 2/15 iommu: Report domain nesting info
In reply to Liu Yi LIOMMUs that support nesting translation needs report the capability info to userspace. It gives information about requirements the userspace needs to implement plus other features characterizing the physical implementation.
This patch reports nesting info by DOMAIN_ATTR_NESTING. Caller can get nesting info after setting DOMAIN_ATTR_NESTING. For VFIO, it is after selecting VFIO_TYPE1_NESTING_IOMMU.
---
v5 -> v6:
*) rephrase the feature notes per comments from Eric Auger. *) rename @size of struct iommu_nesting_info to @argsz.
v4 -> v5:
*) address comments from Eric Auger.
v3 -> v4:
*) split the SMMU driver changes to be a separate patch *) move the @addr_width and @pasid_bits from vendor specific
part to generic part.
*) tweak the description for the @features field of struct
iommu_nesting_info.
*) add description on the @data[] field of struct iommu_nesting_info
v2 -> v3:
*) remvoe cap/ecap_mask in iommu_nesting_info.
*) reuse DOMAIN_ATTR_NESTING to get nesting info.
*) return an empty iommu_nesting_info for SMMU drivers per Jean'
suggestion.
--- include/uapi/linux/iommu.h | 74 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+)
include/uapi/linux/iommu.h +74 -0
@@ -332,4 +332,78 @@ struct iommu_gpasid_bind_data { } vendor; }; +/* +* struct iommu_nesting_info - Information for nesting-capable IOMMU. +* userspace should check it before using +* nesting capability. +* +* @argsz: size of the whole structure. +* @flags: currently reserved for future extension. must set to 0. +* @format: PASID table entry format, the same definition as struct +* iommu_gpasid_bind_data @format. +* @features: supported nesting features. +* @addr_width: The output addr width of first level/stage translation +* @pasid_bits: Maximum supported PASID bits, 0 represents no PASID +* support. +* @data: vendor specific cap info. data[] structure type can be deduced +* from @format field. +* +* +===============+======================================================+ +* | feature | Notes | +* +===============+======================================================+ +* | SYSWIDE_PASID | IOMMU vendor driver sets it to mandate userspace | +* | | to allocate PASID from kernel. All PASID allocation | +* | | free must be mediated through the TBD API. | +* +---------------+------------------------------------------------------+ +* | BIND_PGTBL | IOMMU vendor driver sets it to mandate userspace | +* | | bind the first level/stage page table to associated | +* | | PASID (either the one specified in bind request or | +* | | the default PASID of iommu domain), through IOMMU | +* | | UAPI. | +* +---------------+------------------------------------------------------+ +* | CACHE_INVLD | IOMMU vendor driver sets it to mandate userspace | +* | | explicitly invalidate the IOMMU cache through IOMMU | +* | | UAPI according to vendor-specific requirement when | +* | | changing the 1st level/stage page table. | +* +---------------+------------------------------------------------------+ +* +* @data[] types defined for @format: +* +================================+=====================================+ +* | @format | @data[] | +* +================================+=====================================+ +* | IOMMU_PASID_FORMAT_INTEL_VTD | struct iommu_nesting_info_vtd | +* +--------------------------------+-------------------------------------+ +* +*/ +struct iommu_nesting_info { +__u32 argsz; +__u32 flags; +__u32 format; +#define IOMMU_NESTING_FEAT_SYSWIDE_PASID (1 << 0) +#define IOMMU_NESTING_FEAT_BIND_PGTBL (1 << 1) +#define IOMMU_NESTING_FEAT_CACHE_INVLD (1 << 2) +__u32 features; +__u16 addr_width; +__u16 pasid_bits; +__u32 padding; +__u8 data[]; +}; + +/* +* struct iommu_nesting_info_vtd - Intel VT-d specific nesting info. +* +* @flags: VT-d specific flags. Currently reserved for future +* extension. must be set to 0. +* @cap_reg: Describe basic capabilities as defined in VT-d capability +* register. +* @ecap_reg: Describe the extended capabilities as defined in VT-d +* extended capability register. +*/ +struct iommu_nesting_info_vtd { +__u32 flags; +__u32 padding; +__u64 cap_reg; +__u64 ecap_reg; +}; +#endif /* _UAPI_IOMMU_H */
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 02/15] iommu: Report domain nesting info
Yi,
Quoted 1 line · Liu Yi L
IOMMUs that support nesting translation needs report the capability info
s/needs/need to
Quoted 6 lines
to userspace. It gives information about requirements the userspace needs to implement plus other features characterizing the physical implementation.
This patch reports nesting info by
DOMAIN_ATTR_NESTING. Caller can get nesting info after settingDOMAIN_ATTR_NESTING. For VFIO, it is after selectingVFIO_TYPE1_NESTING_IOMMU.
This is not what this patch does ;-) It introduces a new IOMMU UAPI struct that gives information about the nesting capabilities and features. This struct is supposed to be returned by iommu_domain_get_attr() with DOMAIN_ATTR_NESTING attribute parameter, one a domain whose type has been set to DOMAIN_ATTR_NESTING.
Quoted 55 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) rephrase the feature notes per comments from Eric Auger. *) rename @size of structiommu_nesting_infoto @argsz.v4 -> v5:
*) address comments from Eric Auger.v3 -> v4:
*) split the SMMU driver changes to be a separate patch *) move the @addr_width and @pasid_bits from vendor specific
part to generic part.
*) tweak the description for the @features field of struct
iommu_nesting_info.
*) add description on the @data[] field of structiommu_nesting_infov2 -> v3:
*) remvoe cap/ecap_mask iniommu_nesting_info.
*) reuseDOMAIN_ATTR_NESTINGto get nesting info.
*) return an emptyiommu_nesting_infofor SMMU drivers per Jean'
suggestion.--- include/uapi/linux/iommu.h | 74 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+)include/uapi/linux/iommu.h +12
-0@@ -332,4 +332,78 @@ struct iommu_gpasid_bind_data { } vendor; }; +/* +* struct iommu_nesting_info - Information for nesting-capable IOMMU. +* userspace should check it before using +* nesting capability. +* +* @argsz: size of the whole structure. +* @flags: currently reserved for future extension. must set to 0. +* @format: PASID table entry format, the same definition as struct +* iommu_gpasid_bind_data @format. +* @features: supported nesting features. +* @addr_width: The output addr width of first level/stage translation +* @pasid_bits: Maximum supported PASID bits, 0 represents no PASID
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 356
Quoted 10 lines
include/uapi/linux/iommu.h +10
-0+* support. +* @data: vendor specific cap info. data[] structure type can be deduced +* from @format field. +* +* +===============+======================================================+ +* | feature | Notes | +* +===============+======================================================+ +* | SYSWIDE_PASID | IOMMU vendor driver sets it to mandate userspace | +* | | to allocate PASID from kernel. All PASID allocation | +* | | free must be mediated through the TBD API. |
s/TBD/IOMMU
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 359
Quoted 3 lines
include/uapi/linux/iommu.h +3
-0+* +---------------+------------------------------------------------------+ +* | BIND_PGTBL | IOMMU vendor driver sets it to mandate userspace | +* | | bind the first level/stage page table to associated |
s/bind/to bind
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 365
Quoted 7 lines
include/uapi/linux/iommu.h +6
-0+* | | PASID (either the one specified in bind request or | +* | | the default PASID of iommu domain), through IOMMU | +* | | UAPI. | +* +---------------+------------------------------------------------------+ +* | CACHE_INVLD | IOMMU vendor driver sets it to mandate userspace | +* | | explicitly invalidate the IOMMU cache through IOMMU |
to explicitly
Quoted 15 lines
include/uapi/linux/iommu.h +15
-0+* | | UAPI according to vendor-specific requirement when | +* | | changing the 1st level/stage page table. | +* +---------------+------------------------------------------------------+ +* +* @data[] types defined for @format: +* +================================+=====================================+ +* | @format | @data[] | +* +================================+=====================================+ +* | IOMMU_PASID_FORMAT_INTEL_VTD | struct iommu_nesting_info_vtd | +* +--------------------------------+-------------------------------------+ +* +*/ +struct iommu_nesting_info { +__u32 argsz; +__u32 flags;
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 390
Quoted 10 lines
include/uapi/linux/iommu.h +10
-0+__u32 format; +#define IOMMU_NESTING_FEAT_SYSWIDE_PASID (1 << 0) +#define IOMMU_NESTING_FEAT_BIND_PGTBL (1 << 1) +#define IOMMU_NESTING_FEAT_CACHE_INVLD (1 << 2) +__u32 features; +__u16 addr_width; +__u16 pasid_bits; +__u32 padding; +__u8 data[]; +};
As opposed to other IOMMU UAPI structs there is no union member at the end. Also this struct is not documented in [PATCH v7 1/7] docs: IOMMU user API. Shouldn't we align.
You may also consider to move this patch in Jacob's series for consistency, thoughts?
Quoted 18 lines
include/uapi/linux/iommu.h +18
-0+ +/* +* struct iommu_nesting_info_vtd - Intel VT-d specific nesting info. +* +* @flags: VT-d specific flags. Currently reserved for future +* extension. must be set to 0. +* @cap_reg: Describe basic capabilities as defined in VT-d capability +* register. +* @ecap_reg: Describe the extended capabilities as defined in VT-d +* extended capability register. +*/ +struct iommu_nesting_info_vtd { +__u32 flags; +__u32 padding; +__u64 cap_reg; +__u64 ecap_reg; +}; +
Quoted 2 lines
#endif /*
_UAPI_IOMMU_H*/
Thanks
Eric
RE: [PATCH v6 02/15] iommu: Report domain nesting info
Hi Eric,
Quoted 10 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 8:53 PMYi,
Quoted 1 line · Liu Yi L
IOMMUs that support nesting translation needs report the capability info
s/needs/need to
Quoted 3 lines
to userspace. It gives information about requirements the userspace needs to implement plus other features characterizing the physical implementation.
Quoted 8 lines
Quoted 3 lines
This patch reports nesting info by
DOMAIN_ATTR_NESTING. Caller can get nesting info after settingDOMAIN_ATTR_NESTING. For VFIO, it is after selectingVFIO_TYPE1_NESTING_IOMMU.This is not what this patch does ;-) It introduces a new IOMMU UAPI struct that gives information about the nesting capabilities and features. This struct is supposed to be returned by
iommu_domain_get_attr()withDOMAIN_ATTR_NESTINGattribute parameter, one a domain whose type has been set toDOMAIN_ATTR_NESTING.
got it. let me apply your suggestion. thanks. :-)
Quoted 63 lines
Quoted 26 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) rephrase the feature notes per comments from Eric Auger. *) rename @size of structiommu_nesting_infoto @argsz.v4 -> v5:
*) address comments from Eric Auger.v3 -> v4:
*) split the SMMU driver changes to be a separate patch *) move the @addr_width and @pasid_bits from vendor specific
part to generic part.
*) tweak the description for the @features field of struct
iommu_nesting_info.
*) add description on the @data[] field of structiommu_nesting_infoQuoted 7 lines
v2 -> v3:
*) remvoe cap/ecap_mask iniommu_nesting_info.
*) reuseDOMAIN_ATTR_NESTINGto get nesting info.
*) return an emptyiommu_nesting_infofor SMMU drivers per Jean'
suggestion.--- include/uapi/linux/iommu.h | 74++++++++++++++++++++++++++++++++++++++++++++++
Quoted 17 lines
1 file changed, 74 insertions(+)include/uapi/linux/iommu.h +7
-0@@ -332,4 +332,78 @@ struct iommu_gpasid_bind_data { } vendor; }; +/* +* struct iommu_nesting_info - Information for nesting-capable IOMMU. +* userspace should check it before using +* nesting capability. +* +* @argsz: size of the whole structure. +* @flags: currently reserved for future extension. must set to 0. include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 351
Quoted 10 lines
include/uapi/linux/iommu.h +10
-0+* @format: PASID table entry format, the same definition as struct +* iommu_gpasid_bind_data @format. +* @features: supported nesting features. +* @addr_width: The output addr width of first level/stage translation +* @pasid_bits: Maximum supported PASID bits, 0 represents no PASID +* support. +* @data: vendor specific cap info. data[] structure type can be deduced +* from @format field. +* +* +===============+===================================================
===+
Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 353
Quoted 2 lines
include/uapi/linux/iommu.h +2
-0+* | feature | Notes | +* +===============+===================================================
===+include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 356
Quoted 3 lines
include/uapi/linux/iommu.h +3
-0+* | SYSWIDE_PASID | IOMMU vendor driver sets it to mandate userspace | +* | | to allocate PASID from kernel. All PASID allocation | +* | | free must be mediated through the TBD API. | s/TBD/IOMMU
got it.
Quoted 4 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 359
Quoted 3 lines
include/uapi/linux/iommu.h +3
-0+* +---------------+------------------------------------------------------+ +* | BIND_PGTBL | IOMMU vendor driver sets it to mandate userspace | +* | | bind the first level/stage page table to associated | s/bind/to bind
got it.
Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 365
Quoted 7 lines
include/uapi/linux/iommu.h +6
-0+* | | PASID (either the one specified in bind request or | +* | | the default PASID of iommu domain), through IOMMU | +* | | UAPI. | +* +---------------+------------------------------------------------------+ +* | CACHE_INVLD | IOMMU vendor driver sets it to mandate userspace | +* | | explicitly invalidate the IOMMU cache through IOMMU | to explicitly
I see.
Quoted 24 lines
Quoted 7 lines
include/uapi/linux/iommu.h +1
-0+* | | U API according to vendor-specific requirement when |
+ * | | changing the 1st level/stage page table. |
+ * +---------------+------------------------------------------------------+
+ *
+ * @data[] types defined for @format:
+ *+================================+==================================
===+Quoted 2 lines
+ * | @format | @data[] |
+ *+================================+==================================
===+Quoted 11 lines
+ * |
IOMMU_PASID_FORMAT_INTEL_VTD| structiommu_nesting_info_vtd|
+ * +--------------------------------+-------------------------------------+
+ *
+ */include/uapi/linux/iommu.h +7
-0+struct iommu_nesting_info { +__u32 argsz; +__u32 flags; +__u32 format; +#define IOMMU_NESTING_FEAT_SYSWIDE_PASID (1 << 0) +#define IOMMU_NESTING_FEAT_BIND_PGTBL (1 << 1) +#define IOMMU_NESTING_FEAT_CACHE_INVLD (1 << 2)
Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 390
Quoted 6 lines
include/uapi/linux/iommu.h +6
-0+__u32 features; +__u16 addr_width; +__u16 pasid_bits; +__u32 padding; +__u8 data[]; +};As opposed to other IOMMU UAPI structs there is no union member at the end.
nice catch. do you think it would be better to adding a union and put the struct iommu_nesting_info_vtd in it?
Quoted 4 lines
Also this struct is not documented in [PATCH v7 1/7] docs: IOMMU user API. Shouldn't we align.
You may also consider to move this patch in Jacob's series for consistency, thoughts?
this was talked one time between Jacob and me. It was put in this series as the major user of nesting_info is in this series. e.g. vfio checks the SYSWIDE_PASID. but I'm open to merge it with Jacob's series if it would make the merge easier.
Thanks,
Yi Liu
Quoted 23 lines
Quoted 10 lines
include/uapi/linux/iommu.h +9
-0+ +/* +* struct iommu_nesting_info_vtd - Intel VT-d specific nesting info. +* +* @flags: VT-d specific flags. Currently reserved for future +* extension. must be set to 0. +* @cap_reg: Describe basic capabilities as defined in VT-d capability +* register. +* @ecap_reg: Describe the extended capabilities as defined in VT-d include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 409
Quoted 10 lines
include/uapi/linux/iommu.h +9
-0+* extended capability register. +*/ +struct iommu_nesting_info_vtd { +__u32 flags; +__u32 padding; +__u64 cap_reg; +__u64 ecap_reg; +}; +#endif /* _UAPI_IOMMU_H */ Thanks
Quoted 2 lines
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 02/15] iommu: Report domain nesting info
Hi Yi,
Quoted 148 lines · Yi L
Hi Eric,
Quoted 10 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 8:53 PMYi,
Quoted 1 line · Liu Yi L
IOMMUs that support nesting translation needs report the capability info
s/needs/need to
Quoted 3 lines
to userspace. It gives information about requirements the userspace needs to implement plus other features characterizing the physical implementation.
Quoted 8 lines
Quoted 3 lines
This patch reports nesting info by
DOMAIN_ATTR_NESTING. Caller can get nesting info after settingDOMAIN_ATTR_NESTING. For VFIO, it is after selectingVFIO_TYPE1_NESTING_IOMMU.This is not what this patch does ;-) It introduces a new IOMMU UAPI struct that gives information about the nesting capabilities and features. This struct is supposed to be returned by
iommu_domain_get_attr()withDOMAIN_ATTR_NESTINGattribute parameter, one a domain whose type has been set toDOMAIN_ATTR_NESTING.got it. let me apply your suggestion. thanks. :-)
Quoted 63 lines
Quoted 26 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) rephrase the feature notes per comments from Eric Auger. *) rename @size of structiommu_nesting_infoto @argsz.v4 -> v5:
*) address comments from Eric Auger.v3 -> v4:
*) split the SMMU driver changes to be a separate patch *) move the @addr_width and @pasid_bits from vendor specific
part to generic part.
*) tweak the description for the @features field of struct
iommu_nesting_info.
*) add description on the @data[] field of structiommu_nesting_infoQuoted 7 lines
v2 -> v3:
*) remvoe cap/ecap_mask iniommu_nesting_info.
*) reuseDOMAIN_ATTR_NESTINGto get nesting info.
*) return an emptyiommu_nesting_infofor SMMU drivers per Jean'
suggestion.--- include/uapi/linux/iommu.h | 74++++++++++++++++++++++++++++++++++++++++++++++
Quoted 17 lines
1 file changed, 74 insertions(+)include/uapi/linux/iommu.h +7
-0@@ -332,4 +332,78 @@ struct iommu_gpasid_bind_data { } vendor; }; +/* +* struct iommu_nesting_info - Information for nesting-capable IOMMU. +* userspace should check it before using +* nesting capability. +* +* @argsz: size of the whole structure. +* @flags: currently reserved for future extension. must set to 0. include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 351
Quoted 10 lines
include/uapi/linux/iommu.h +10
-0+* @format: PASID table entry format, the same definition as struct +* iommu_gpasid_bind_data @format. +* @features: supported nesting features. +* @addr_width: The output addr width of first level/stage translation +* @pasid_bits: Maximum supported PASID bits, 0 represents no PASID +* support. +* @data: vendor specific cap info. data[] structure type can be deduced +* from @format field. +* +* +===============+===================================================
===+Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 353
Quoted 2 lines
include/uapi/linux/iommu.h +2
-0+* | feature | Notes | +* +===============+===================================================
===+include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 356
Quoted 3 lines
include/uapi/linux/iommu.h +3
-0+* | SYSWIDE_PASID | IOMMU vendor driver sets it to mandate userspace | +* | | to allocate PASID from kernel. All PASID allocation | +* | | free must be mediated through the TBD API. | s/TBD/IOMMU
got it.
Quoted 4 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 359
Quoted 3 lines
include/uapi/linux/iommu.h +3
-0+* +---------------+------------------------------------------------------+ +* | BIND_PGTBL | IOMMU vendor driver sets it to mandate userspace | +* | | bind the first level/stage page table to associated | s/bind/to bind
got it.
Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 365
Quoted 7 lines
include/uapi/linux/iommu.h +6
-0+* | | PASID (either the one specified in bind request or | +* | | the default PASID of iommu domain), through IOMMU | +* | | UAPI. | +* +---------------+------------------------------------------------------+ +* | CACHE_INVLD | IOMMU vendor driver sets it to mandate userspace | +* | | explicitly invalidate the IOMMU cache through IOMMU | to explicitly
I see.
Quoted 24 lines
Quoted 7 lines
include/uapi/linux/iommu.h +1
-0+* | | U API according to vendor-specific requirement when |
+ * | | changing the 1st level/stage page table. |
+ * +---------------+------------------------------------------------------+
+ *
+ * @data[] types defined for @format:
+ *+================================+==================================
===+Quoted 2 lines
+ * | @format | @data[] |
+ *+================================+==================================
===+Quoted 11 lines
+ * |
IOMMU_PASID_FORMAT_INTEL_VTD| structiommu_nesting_info_vtd|
+ * +--------------------------------+-------------------------------------+
+ *
+ */include/uapi/linux/iommu.h +7
-0+struct iommu_nesting_info { +__u32 argsz; +__u32 flags; +__u32 format; +#define IOMMU_NESTING_FEAT_SYSWIDE_PASID (1 << 0) +#define IOMMU_NESTING_FEAT_BIND_PGTBL (1 << 1) +#define IOMMU_NESTING_FEAT_CACHE_INVLD (1 << 2)Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 390
Quoted 6 lines
include/uapi/linux/iommu.h +6
-0+__u32 features; +__u16 addr_width; +__u16 pasid_bits; +__u32 padding; +__u8 data[]; +};As opposed to other IOMMU UAPI structs there is no union member at the end.
Quoted 2 lines
nice catch. do you think it would be better to adding a union and put the struct
iommu_nesting_info_vtdin it?
Yes I think so. At least it would be consistent with the rest of the API and with the guidelines.
Quoted 10 lines
Quoted 5 lines
Also this struct is not documented in [PATCH v7 1/7] docs: IOMMU user API. Shouldn't we align.
You may also consider to move this patch in Jacob's series for consistency, thoughts?this was talked one time between Jacob and me. It was put in this series as the major user of
nesting_infois in this series. e.g. vfio checks theSYSWIDE_PASID. but I'm open to merge it with Jacob's series if it would make the merge easier.
Yep I think it would make sense to move in Jacob's series to have a general understanding of the uapi
Thanks
Eric
Quoted 30 lines
Thanks,
Yi LiuQuoted 25 lines
Quoted 10 lines
include/uapi/linux/iommu.h +9
-0+ +/* +* struct iommu_nesting_info_vtd - Intel VT-d specific nesting info. +* +* @flags: VT-d specific flags. Currently reserved for future +* extension. must be set to 0. +* @cap_reg: Describe basic capabilities as defined in VT-d capability +* register. +* @ecap_reg: Describe the extended capabilities as defined in VT-d include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 409
Quoted 10 lines
include/uapi/linux/iommu.h +9
-0+* extended capability register. +*/ +struct iommu_nesting_info_vtd { +__u32 flags; +__u32 padding; +__u64 cap_reg; +__u64 ecap_reg; +}; +#endif /* _UAPI_IOMMU_H */ Thanks
Eric
Re: [PATCH v6 02/15] iommu: Report domain nesting info
Quoted 190 lines · On Sun, 16 Aug 2020 14:40:57 +0200 Auger Eric <[email protected]> wrote:
Hi Yi,
Quoted 177 lines · Yi L
Hi Eric,
Quoted 15 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 8:53 PMYi,
Quoted 2 lines · Liu Yi L
IOMMUs that support nesting translation needs report the capability info
s/needs/need to
Quoted 7 lines
to userspace. It gives information about requirements the userspace needs to implement plus other features characterizing the physical implementation.
This patch reports nesting info by
DOMAIN_ATTR_NESTING. Caller can get nesting info after settingDOMAIN_ATTR_NESTING. For VFIO, it is after selectingVFIO_TYPE1_NESTING_IOMMU.Quoted 6 lines
This is not what this patch does ;-) It introduces a new IOMMU UAPI struct that gives information about the nesting capabilities and features. This struct is supposed to be returned by
iommu_domain_get_attr()withDOMAIN_ATTR_NESTINGattribute parameter, one a domain whose type has been set toDOMAIN_ATTR_NESTING.got it. let me apply your suggestion. thanks. :-)
Quoted 75 lines
Quoted 27 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) rephrase the feature notes per comments from Eric Auger. *) rename @size of structiommu_nesting_infoto @argsz.v4 -> v5:
*) address comments from Eric Auger.v3 -> v4:
*) split the SMMU driver changes to be a separate patch *) move the @addr_width and @pasid_bits from vendor specific
part to generic part.
*) tweak the description for the @features field of struct
iommu_nesting_info.
*) add description on the @data[] field of structiommu_nesting_infoQuoted 7 lines
v2 -> v3:
*) remvoe cap/ecap_mask iniommu_nesting_info.
*) reuseDOMAIN_ATTR_NESTINGto get nesting info.
*) return an emptyiommu_nesting_infofor SMMU drivers per Jean'
suggestion.--- include/uapi/linux/iommu.h | 74++++++++++++++++++++++++++++++++++++++++++++++
Quoted 26 lines
1 file changed, 74 insertions(+)(quoted diff) +0
-0
b/include/uapi/linux/iommu.hindex 7c8e075..5e4745a 100644include/uapi/linux/iommu.h +2
-0@@ -332,4 +332,78 @@ struct iommu_gpasid_bind_data { } vendor; }; +/* +* struct iommu_nesting_info - Information for nesting-capable IOMMU.
+ * userspace should check it
before using
+ * nesting capability.
+ *
+ * @argsz: size of the whole structure.
+ * @flags: currently reserved for future extension. must
set to 0.
+ * @format: PASID table entry format, the same definition
as struct
+ *iommu_gpasid_bind_data@format.
+ * @features: supported nesting features.
+ * @addr_width: The output addr width of first
level/stage translationQuoted 8 lines
+ * @pasid_bits: Maximum supported PASID bits, 0
represents no PASID
+ * support.
+ * @data: vendor specific cap info. data[] structure type
can be deduced
+ * from @format field.
+ *
+ *+===============+=================================================== ===+ > + * | feature | > Notes | > + * +===============+===================================================Quoted 8 lines
===+ > + * | SYSWIDE_PASID | IOMMU vendor driver sets it to mandate > userspace | > + * | | to allocate PASID from kernel. All PASID > allocation | > + * | | free must be mediated through the TBD > API. | s/TBD/IOMMUgot it.
Quoted 7 lines
Quoted 6 lines
+ *
+---------------+------------------------------------------------------+ + * | BIND_PGTBL | IOMMU vendor driver sets it to mandate userspace | + * | | bind the first level/stage page table to associated |s/bind/to bind
got it.
Quoted 11 lines
Quoted 10 lines
+ * | | PASID (either the one specified in bind
request or |
+ * | | the default PASID of iommu domain),
through IOMMU |
+ * | |
UAPI. |
+ *+---------------+------------------------------------------------------+ + * | CACHE_INVLD | IOMMU vendor driver sets it to mandate userspace |Quoted 3 lines
Quoted 2 lines
+ * | | explicitly invalidate the IOMMU cache
through IOMMU |to explicitly
I see.
Quoted 29 lines
Quoted 9 lines
+ * | | U
API according to vendor-specific requirement when |
+ * | | changing the 1st level/stage page
table. |
+ *
+---------------+------------------------------------------------------+
+ *
+ * @data[] types defined for @format:
+ *+================================+================================== ===+ > + * | @format | > @data[] | > + * +================================+================================== ===+ > + * | IOMMU_PASID_FORMAT_INTEL_VTD | struct > iommu_nesting_info_vtd |Quoted 11 lines
+ *
+--------------------------------+-------------------------------------+
+ *
+ */include/uapi/linux/iommu.h +7
-0+struct iommu_nesting_info { +__u32 argsz; +__u32 flags; +__u32 format; +#define IOMMU_NESTING_FEAT_SYSWIDE_PASID (1 << 0) +#define IOMMU_NESTING_FEAT_BIND_PGTBL (1 << 1) +#define IOMMU_NESTING_FEAT_CACHE_INVLD (1 << 2)Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 390
Quoted 6 lines
include/uapi/linux/iommu.h +6
-0+__u32 features; +__u16 addr_width; +__u16 pasid_bits; +__u32 padding; +__u8 data[]; +};As opposed to other IOMMU UAPI structs there is no union member at the end.
Quoted 2 lines
nice catch. do you think it would be better to adding a union and put the struct
iommu_nesting_info_vtdin it?Yes I think so. At least it would be consistent with the rest of the API and with the guidelines.
Quoted 6 lines
Quoted 5 lines
Also this struct is not documented in [PATCH v7 1/7] docs: IOMMU user API. Shouldn't we align.
You may also consider to move this patch in Jacob's series for consistency, thoughts?
Quoted 7 lines
Quoted 4 lines
this was talked one time between Jacob and me. It was put in this series as the major user of
nesting_infois in this series. e.g. vfio checks theSYSWIDE_PASID. but I'm open to merge it with Jacob's series if it would make the merge easier.Yep I think it would make sense to move in Jacob's series to have a general understanding of the uapi
I a little reluctant to include this in my UAPI set, the reason is that there are two dimensions IOMMU UAPI are extended:
1. Define the protocols in interaction with VFIO, sanity checking, and backward compatibility.
2. Adding more UAPI data structures that are parallel to the existing ones.
My patchset is to address #1, this patch is for #2. My thinking is that once we have reached consensus on #1, new UAPI structures such as this patch can just follow the suit.
If that is OK with you, I would like to keep them separate to avoid diverging conversations.
Thanks,
Jacob
Quoted 38 lines
Thanks
Eric
Quoted 34 lines
Thanks,
Yi LiuQuoted 29 lines
Quoted 16 lines
include/uapi/linux/iommu.h +3
-0+ +/* +* struct iommu_nesting_info_vtd - Intel VT-d specific nesting info.
+ *
+ * @flags: VT-d specific flags. Currently reserved for
future
+ * extension. must be set to 0.
+ * @cap_reg: Describe basic capabilities as defined in
VT-d capability
+ * register.
+ * @ecap_reg: Describe the extended capabilities as
defined in VT-d
+ * extended capability register.
+ */include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 409
Quoted 8 lines
include/uapi/linux/iommu.h +7
-0+struct iommu_nesting_info_vtd { +__u32 flags; +__u32 padding; +__u64 cap_reg; +__u64 ecap_reg; +}; +#endif /* _UAPI_IOMMU_H */ Thanks
Eric
[Jacob Pan]
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 02/15] iommu: Report domain nesting info
Hi Jacob,
Quoted 207 lines · Jacob Pan
Quoted 190 lines · On Sun, 16 Aug 2020 14:40:57 +0200 Auger Eric <[email protected]> wrote:
Hi Yi,
Quoted 177 lines · Yi L
Hi Eric,
Quoted 15 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 8:53 PMYi,
Quoted 2 lines · Liu Yi L
IOMMUs that support nesting translation needs report the capability info
s/needs/need to
Quoted 7 lines
to userspace. It gives information about requirements the userspace needs to implement plus other features characterizing the physical implementation.
This patch reports nesting info by
DOMAIN_ATTR_NESTING. Caller can get nesting info after settingDOMAIN_ATTR_NESTING. For VFIO, it is after selectingVFIO_TYPE1_NESTING_IOMMU.Quoted 6 lines
This is not what this patch does ;-) It introduces a new IOMMU UAPI struct that gives information about the nesting capabilities and features. This struct is supposed to be returned by
iommu_domain_get_attr()withDOMAIN_ATTR_NESTINGattribute parameter, one a domain whose type has been set toDOMAIN_ATTR_NESTING.got it. let me apply your suggestion. thanks. :-)
Quoted 75 lines
Quoted 27 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) rephrase the feature notes per comments from Eric Auger. *) rename @size of structiommu_nesting_infoto @argsz.v4 -> v5:
*) address comments from Eric Auger.v3 -> v4:
*) split the SMMU driver changes to be a separate patch *) move the @addr_width and @pasid_bits from vendor specific
part to generic part.
*) tweak the description for the @features field of struct
iommu_nesting_info.
*) add description on the @data[] field of structiommu_nesting_infoQuoted 7 lines
v2 -> v3:
*) remvoe cap/ecap_mask iniommu_nesting_info.
*) reuseDOMAIN_ATTR_NESTINGto get nesting info.
*) return an emptyiommu_nesting_infofor SMMU drivers per Jean'
suggestion.--- include/uapi/linux/iommu.h | 74++++++++++++++++++++++++++++++++++++++++++++++
Quoted 26 lines
1 file changed, 74 insertions(+)(quoted diff) +0
-0
b/include/uapi/linux/iommu.hindex 7c8e075..5e4745a 100644include/uapi/linux/iommu.h +2
-0@@ -332,4 +332,78 @@ struct iommu_gpasid_bind_data { } vendor; }; +/* +* struct iommu_nesting_info - Information for nesting-capable IOMMU.
+ * userspace should check it
before using
+ * nesting capability.
+ *
+ * @argsz: size of the whole structure.
+ * @flags: currently reserved for future extension. must
set to 0.
+ * @format: PASID table entry format, the same definition
as struct
+ *iommu_gpasid_bind_data@format.
+ * @features: supported nesting features.
+ * @addr_width: The output addr width of first
level/stage translationQuoted 8 lines
+ * @pasid_bits: Maximum supported PASID bits, 0
represents no PASID
+ * support.
+ * @data: vendor specific cap info. data[] structure type
can be deduced
+ * from @format field.
+ *
+ *+===============+=================================================== ===+ > + * | feature | > Notes | > + * +===============+===================================================Quoted 8 lines
===+ > + * | SYSWIDE_PASID | IOMMU vendor driver sets it to mandate > userspace | > + * | | to allocate PASID from kernel. All PASID > allocation | > + * | | free must be mediated through the TBD > API. | s/TBD/IOMMUgot it.
Quoted 7 lines
Quoted 6 lines
+ *
+---------------+------------------------------------------------------+ + * | BIND_PGTBL | IOMMU vendor driver sets it to mandate userspace | + * | | bind the first level/stage page table to associated |s/bind/to bind
got it.
Quoted 11 lines
Quoted 10 lines
+ * | | PASID (either the one specified in bind
request or |
+ * | | the default PASID of iommu domain),
through IOMMU |
+ * | |
UAPI. |
+ *+---------------+------------------------------------------------------+ + * | CACHE_INVLD | IOMMU vendor driver sets it to mandate userspace |Quoted 3 lines
Quoted 2 lines
+ * | | explicitly invalidate the IOMMU cache
through IOMMU |to explicitly
I see.
Quoted 29 lines
Quoted 9 lines
+ * | | U
API according to vendor-specific requirement when |
+ * | | changing the 1st level/stage page
table. |
+ *
+---------------+------------------------------------------------------+
+ *
+ * @data[] types defined for @format:
+ *+================================+================================== ===+ > + * | @format | > @data[] | > + * +================================+================================== ===+ > + * | IOMMU_PASID_FORMAT_INTEL_VTD | struct > iommu_nesting_info_vtd |Quoted 11 lines
+ *
+--------------------------------+-------------------------------------+
+ *
+ */include/uapi/linux/iommu.h +7
-0+struct iommu_nesting_info { +__u32 argsz; +__u32 flags; +__u32 format; +#define IOMMU_NESTING_FEAT_SYSWIDE_PASID (1 << 0) +#define IOMMU_NESTING_FEAT_BIND_PGTBL (1 << 1) +#define IOMMU_NESTING_FEAT_CACHE_INVLD (1 << 2)Quoted 8 lines
include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 390
Quoted 6 lines
include/uapi/linux/iommu.h +6
-0+__u32 features; +__u16 addr_width; +__u16 pasid_bits; +__u32 padding; +__u8 data[]; +};As opposed to other IOMMU UAPI structs there is no union member at the end.
Quoted 2 lines
nice catch. do you think it would be better to adding a union and put the struct
iommu_nesting_info_vtdin it?Yes I think so. At least it would be consistent with the rest of the API and with the guidelines.
Quoted 6 lines
Quoted 5 lines
Also this struct is not documented in [PATCH v7 1/7] docs: IOMMU user API. Shouldn't we align.
You may also consider to move this patch in Jacob's series for consistency, thoughts?Quoted 7 lines
Quoted 4 lines
this was talked one time between Jacob and me. It was put in this series as the major user of
nesting_infois in this series. e.g. vfio checks theSYSWIDE_PASID. but I'm open to merge it with Jacob's series if it would make the merge easier.Yep I think it would make sense to move in Jacob's series to have a general understanding of the uapi
I a little reluctant to include this in my UAPI set, the reason is that there are two dimensions IOMMU UAPI are extended:
1. Define the protocols in interaction with VFIO, sanity checking, and backward compatibility.
2. Adding more UAPI data structures that are parallel to the existing ones.
Quoted 6 lines
My patchset is to address #1, this patch is for #2. My thinking is that once we have reached consensus on #1, new UAPI structures such as this patch can just follow the suit.
If that is OK with you, I would like to keep them separate to avoid diverging conversations.
OK no problem for me, as long as the new APIs follow the rules & guidelines introduced in your series.
Thanks
Eric
Quoted 46 lines
Thanks,
Jacob
Quoted 38 lines
Thanks
Eric
Quoted 34 lines
Thanks,
Yi LiuQuoted 29 lines
Quoted 16 lines
include/uapi/linux/iommu.h +3
-0+ +/* +* struct iommu_nesting_info_vtd - Intel VT-d specific nesting info.
+ *
+ * @flags: VT-d specific flags. Currently reserved for
future
+ * extension. must be set to 0.
+ * @cap_reg: Describe basic capabilities as defined in
VT-d capability
+ * register.
+ * @ecap_reg: Describe the extended capabilities as
defined in VT-d
+ * extended capability register.
+ */include/uapi/linux/iommu.h struct iommu_gpasid_bind_data { · line 409
Quoted 8 lines
include/uapi/linux/iommu.h +7
-0+struct iommu_nesting_info_vtd { +__u32 flags; +__u32 padding; +__u64 cap_reg; +__u64 ecap_reg; +}; +#endif /* _UAPI_IOMMU_H */ Thanks
Eric
[Jacob Pan]
PATCH v6 3/15 iommu/smmu: Report empty domain nesting info
In reply to Liu Yi LThis patch is added as instead of returning a boolean for DOMAIN_ATTR_NESTING, iommu_domain_get_attr() should return an iommu_nesting_info handle. For now, return an empty nesting info struct for now as true nesting is not yet supported by the SMMUs.
---
v5 -> v6:
*) add review-by from Eric Auger.
v4 -> v5:
*) address comments from Eric Auger.
--- drivers/iommu/arm-smmu-v3.c | 29 +++++++++++++++++++++++++++-- drivers/iommu/arm-smmu.c | 29 +++++++++++++++++++++++++++-- 2 files changed, 54 insertions(+), 4 deletions(-)
drivers/iommu/arm-smmu-v3.c +27 -2
@@ -3019,6 +3019,32 @@ static struct iommu_group *arm_smmu_device_group(struct device *dev)return group; } +static int arm_smmu_domain_nesting_info (struct arm_smmu_domain *smmu_domain, +void *data) +{ +struct iommu_nesting_info *info = (struct iommu_nesting_info *)data; +unsigned int size; + +if (!info || smmu_domain->stage !=ARM_SMMU_DOMAIN_NESTED ) +return -ENODEV ; + + size =sizeof (struct iommu_nesting_info ); + +/* +* if provided buffer size is smaller than expected, should +* return 0 and also the expected buffer size to caller. +*/ +if (info->argsz < size) { + info->argsz = size; +return 0 ; + } + +/* report an empty iommu_nesting_info for now */ +memset (info,0x0 , size); + info->argsz = size; +return 0 ; +} +static int arm_smmu_domain_get_attr (struct iommu_domain *domain,enum iommu_attr attr,void *data) {
@@ -3028,8 +3054,7 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain,case IOMMU_DOMAIN_UNMANAGED :switch (attr) {case DOMAIN_ATTR_NESTING : - *(int *)data = (smmu_domain->stage ==ARM_SMMU_DOMAIN_NESTED ); -return 0 ; +return arm_smmu_domain_nesting_info (smmu_domain, data);default :return -ENODEV ; }
drivers/iommu/arm-smmu.c +27 -2
@@ -1506,6 +1506,32 @@ static struct iommu_group *arm_smmu_device_group(struct device *dev)return group; } +static int arm_smmu_domain_nesting_info (struct arm_smmu_domain *smmu_domain, +void *data) +{ +struct iommu_nesting_info *info = (struct iommu_nesting_info *)data; +unsigned int size; + +if (!info || smmu_domain->stage !=ARM_SMMU_DOMAIN_NESTED ) +return -ENODEV ; + + size =sizeof (struct iommu_nesting_info ); + +/* +* if provided buffer size is smaller than expected, should +* return 0 and also the expected buffer size to caller. +*/ +if (info->argsz < size) { + info->argsz = size; +return 0 ; + } + +/* report an empty iommu_nesting_info for now */ +memset (info,0x0 , size); + info->argsz = size; +return 0 ; +} +static int arm_smmu_domain_get_attr (struct iommu_domain *domain,enum iommu_attr attr,void *data) {
@@ -1515,8 +1541,7 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain,case IOMMU_DOMAIN_UNMANAGED :switch (attr) {case DOMAIN_ATTR_NESTING : - *(int *)data = (smmu_domain->stage ==ARM_SMMU_DOMAIN_NESTED ); -return 0 ; +return arm_smmu_domain_nesting_info (smmu_domain, data);default :return -ENODEV ; }
Signature · 1 line
2.7.4
PATCH v6 4/15 vfio/type1: Report iommu nesting info to userspace
In reply to Liu Yi LThis patch exports iommu nesting capability info to user space through VFIO. Userspace is expected to check this info for supported uAPIs (e.g. PASID alloc/free, bind page table, and cache invalidation) and the vendor specific format information for first level/stage page table that will be bound to.
The nesting info is available only after container set to be NESTED type. Current implementation imposes one limitation - one nesting container should include at most one iommu group. The philosophy of vfio container is having all groups/devices within the container share the same IOMMU context. When vSVA is enabled, one IOMMU context could include one 2nd- level address space and multiple 1st-level address spaces. While the 2nd-level address space is reasonably sharable by multiple groups, blindly sharing 1st-level address spaces across all groups within the container might instead break the guest expectation. In the future sub/super container concept might be introduced to allow partial address space sharing within an IOMMU context. But for now let's go with this restriction by requiring singleton container for using nesting iommu features. Below link has the related discussion about this decision.
https://lore.kernel.org/kvm/[email protected]/
This patch also changes the NESTING type container behaviour. Something that would have succeeded before will now fail: Before this series, if user asked for a VFIO_IOMMU_TYPE1_NESTING, it would have succeeded even if the SMMU didn't support stage-2, as the driver would have silently fallen back on stage-1 mappings (which work exactly the same as stage-2 only since there was no nesting supported). After the series, we do check for DOMAIN_ATTR_NESTING so if user asks for VFIO_IOMMU_TYPE1_NESTING and the SMMU doesn't support stage-2, the ioctl fails. But it should be a good fix and completely harmless. Detail can be found in below link as well.
https://lore.kernel.org/kvm/20200717090900.GC4850@myrica/
---
v5 -> v6:
*) address comments against v5 from Eric Auger.
*) don't report nesting cap to userspace if the nesting_info->format is
invalid.
v4 -> v5:
*) address comments from Eric Auger.
*) return struct iommu_nesting_info for VFIO_IOMMU_TYPE1_INFO_CAP_NESTING as
cap is much "cheap", if needs extension in future, just define another cap.
https://lore.kernel.org/kvm/[email protected]/
v3 -> v4:
*) address comments against v3.
v1 -> v2:
*) added in v2
--- drivers/vfio/vfio_iommu_type1.c | 106 +++++++++++++++++++++++++++++++++++----- include/uapi/linux/vfio.h | 19 +++++++ 2 files changed, 113 insertions(+), 12 deletions(-)
drivers/vfio/vfio_iommu_type1.c +94 -12
@@ -62,18 +62,20 @@ MODULE_PARM_DESC(dma_entry_limit, "Maximum number of user DMA mappings per container (65535).");struct vfio_iommu { -struct list_head domain_list; -struct list_head iova_list; -struct vfio_domain *external_domain;/* domain for external user */ -struct mutex lock; -struct rb_root dma_list; -struct blocking_notifier_head notifier; -unsigned int dma_avail; -uint64_t pgsize_bitmap; -bool v2; -bool nesting; -bool dirty_page_tracking; -bool pinned_page_dirty_scope; +struct list_head domain_list; +struct list_head iova_list; +/* domain for external user */ +struct vfio_domain *external_domain; +struct mutex lock; +struct rb_root dma_list; +struct blocking_notifier_head notifier; +unsigned int dma_avail; +uint64_t pgsize_bitmap; +bool v2; +bool nesting; +bool dirty_page_tracking; +bool pinned_page_dirty_scope; +struct iommu_nesting_info *nesting_info; };struct vfio_domain {
@@ -130,6 +132,9 @@ struct vfio_regions {#define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER (iommu) \ (!list_empty(&iommu->domain_list)) +#define CONTAINER_HAS_DOMAIN (iommu) (((iommu)->external_domain) || \ + (!list_empty(&(iommu)->domain_list))) +#define DIRTY_BITMAP_BYTES (n) (ALIGN(n, BITS_PER_TYPE(u64)) / BITS_PER_BYTE)/*
@@ -1929,6 +1934,13 @@ static void vfio_iommu_iova_insert_copy(struct vfio_iommu *iommu,list_splice_tail (iova_copy, iova); } + +static void vfio_iommu_release_nesting_info (struct vfio_iommu *iommu) +{ +kfree (iommu->nesting_info); + iommu->nesting_info =NULL ; +} +static int vfio_iommu_type1_attach_group (void *iommu_data,struct iommu_group *iommu_group) {
@@ -1959,6 +1971,12 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, } } +/* Nesting type container can include only one group */ +if (iommu->nesting &&CONTAINER_HAS_DOMAIN (iommu)) { +mutex_unlock (&iommu->lock); +return -EINVAL ; + } + group =kzalloc (sizeof (*group),GFP_KERNEL ); domain =kzalloc (sizeof (*domain),GFP_KERNEL );if (!group || !domain) {
@@ -2029,6 +2047,32 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,if (ret) goto out_domain; +/* Nesting cap info is available only after attaching */ +if (iommu->nesting) { +struct iommu_nesting_info tmp = { .argsz =0 , }; + +/* First get the size of vendor specific nesting info */ + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + &tmp); +if (ret) + goto out_detach; + + iommu->nesting_info =kzalloc (tmp.argsz,GFP_KERNEL ); +if (!iommu->nesting_info) { + ret = -ENOMEM ; + goto out_detach; + } + +/* Now get the nesting info */ + iommu->nesting_info->argsz = tmp.argsz; + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + iommu->nesting_info); +if (ret) + goto out_detach; + } +/* Get aperture info */ iommu_domain_get_attr (domain->domain,DOMAIN_ATTR_GEOMETRY , &geo);
@@ -2138,6 +2182,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,return 0 ; out_detach: +vfio_iommu_release_nesting_info (iommu);vfio_iommu_detach_group (domain, group); out_domain:iommu_domain_free (domain->domain);
@@ -2338,6 +2383,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,vfio_iommu_unmap_unpin_all (iommu); elsevfio_iommu_unmap_unpin_reaccount (iommu ); + +vfio_iommu_release_nesting_info (iommu); }iommu_domain_free (domain->domain);list_del (&domain->next);
@@ -2546,6 +2593,39 @@ static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig)); } +static int vfio_iommu_add_nesting_cap (struct vfio_iommu *iommu, +struct vfio_info_cap *caps) +{ +struct vfio_info_cap_header *header; +struct vfio_iommu_type1_info_cap_nesting *nesting_cap; +size_t size; + +/* when nesting_info is null, no need go further */ +if (!iommu->nesting_info) +return 0 ; + +/* when @format of nesting_info is 0, fail the call */ +if (iommu->nesting_info->format ==0 ) +return -ENOENT ; + + size = offsetof(struct vfio_iommu_type1_info_cap_nesting , info) + + iommu->nesting_info->argsz; + + header =vfio_info_cap_add (caps, size, +VFIO_IOMMU_TYPE1_INFO_CAP_NESTING ,1 ); +if (IS_ERR (header)) +return PTR_ERR (header); + + nesting_cap =container_of (header, + struct vfio_iommu_type1_info_cap_nesting, + header); + +memcpy (&nesting_cap->info, iommu->nesting_info, + iommu->nesting_info->argsz); + +return 0 ; +} +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu,unsigned long arg) {
@@ -2581,6 +2661,8 @@ static int vfio_iommu_type1_get_info(struct vfio_iommu *iommu,if (!ret) ret =vfio_iommu_iova_build_caps (iommu, &caps); + ret =vfio_iommu_add_nesting_cap (iommu, &caps); +mutex_unlock (&iommu->lock);if (ret)
include/uapi/linux/vfio.h +19 -0
@@ -14,6 +14,7 @@#include <linux/types.h> #include <linux/ioctl.h> +#include <linux/iommu.h> #define VFIO_API_VERSION 0
@@ -1039,6 +1040,24 @@ struct vfio_iommu_type1_info_cap_migration {__u64 max_dirty_bitmap_size;/* in bytes */ }; +/* +* The nesting capability allows to report the related capability +* and info for nesting iommu type. +* +* The structures below define version 1 of this capability. +* +* Userspace selected VFIO_TYPE1_NESTING_IOMMU type should check +* this capability to get supported features. +* +* @info: the nesting info provided by IOMMU driver. +*/ +#define VFIO_IOMMU_TYPE1_INFO_CAP_NESTING 3 + +struct vfio_iommu_type1_info_cap_nesting { +struct vfio_info_cap_header header; +struct iommu_nesting_info info; +}; +#define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)/**
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 04/15] vfio/type1: Report iommu nesting info to userspace
Hi Yi,
Quoted 190 lines · Liu Yi L
This patch exports iommu nesting capability info to user space through VFIO. Userspace is expected to check this info for supported uAPIs (e.g. PASID alloc/free, bind page table, and cache invalidation) and the vendor specific format information for first level/stage page table that will be bound to.
The nesting info is available only after container set to be NESTED type. Current implementation imposes one limitation - one nesting container should include at most one iommu group. The philosophy of vfio container is having all groups/devices within the container share the same IOMMU context. When vSVA is enabled, one IOMMU context could include one 2nd- level address space and multiple 1st-level address spaces. While the 2nd-level address space is reasonably sharable by multiple groups, blindly sharing 1st-level address spaces across all groups within the container might instead break the guest expectation. In the future sub/super container concept might be introduced to allow partial address space sharing within an IOMMU context. But for now let's go with this restriction by requiring singleton container for using nesting iommu features. Below link has the related discussion about this decision.
https://lore.kernel.org/kvm/[email protected]/
This patch also changes the NESTING type container behaviour. Something that would have succeeded before will now fail: Before this series, if user asked for a
VFIO_IOMMU_TYPE1_NESTING, it would have succeeded even if the SMMU didn't support stage-2, as the driver would have silently fallen back on stage-1 mappings (which work exactly the same as stage-2 only since there was no nesting supported). After the series, we do check forDOMAIN_ATTR_NESTINGso if user asks forVFIO_IOMMU_TYPE1_NESTINGand the SMMU doesn't support stage-2, the ioctl fails. But it should be a good fix and completely harmless. Detail can be found in below link as well.https://lore.kernel.org/kvm/20200717090900.GC4850@myrica/
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) address comments against v5 from Eric Auger.
*) don't report nesting cap to userspace if thenesting_info->formatis
invalid.v4 -> v5:
*) address comments from Eric Auger.
*) return structiommu_nesting_infoforVFIO_IOMMU_TYPE1_INFO_CAP_NESTINGas
cap is much "cheap", if needs extension in future, just define another cap.
https://lore.kernel.org/kvm/[email protected]/v3 -> v4:
*) address comments against v3.v1 -> v2:
*) added in v2--- drivers/vfio/vfio_iommu_type1.c | 106 +++++++++++++++++++++++++++++++++++----- include/uapi/linux/vfio.h | 19 +++++++ 2 files changed, 113 insertions(+), 12 deletions(-)drivers/vfio/vfio_iommu_type1.c +59
-12@@ -62,18 +62,20 @@ MODULE_PARM_DESC(dma_entry_limit, "Maximum number of user DMA mappings per container (65535).");struct vfio_iommu { -struct list_head domain_list; -struct list_head iova_list; -struct vfio_domain *external_domain;/* domain for external user */ -struct mutex lock; -struct rb_root dma_list; -struct blocking_notifier_head notifier; -unsigned int dma_avail; -uint64_t pgsize_bitmap; -bool v2; -bool nesting; -bool dirty_page_tracking; -bool pinned_page_dirty_scope; +struct list_head domain_list; +struct list_head iova_list; +/* domain for external user */ +struct vfio_domain *external_domain; +struct mutex lock; +struct rb_root dma_list; +struct blocking_notifier_head notifier; +unsigned int dma_avail; +uint64_t pgsize_bitmap; +bool v2; +bool nesting; +bool dirty_page_tracking; +bool pinned_page_dirty_scope; +struct iommu_nesting_info *nesting_info; };struct vfio_domain {@@ -130,6 +132,9 @@ struct vfio_regions {#define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER (iommu) \ (!list_empty(&iommu->domain_list)) +#define CONTAINER_HAS_DOMAIN (iommu) (((iommu)->external_domain) || \ + (!list_empty(&(iommu)->domain_list))) +#define DIRTY_BITMAP_BYTES (n) (ALIGN(n, BITS_PER_TYPE(u64)) / BITS_PER_BYTE)/* @@ -1929,6 +1934,13 @@ static void vfio_iommu_iova_insert_copy(struct vfio_iommu *iommu,list_splice_tail (iova_copy, iova); } + +static void vfio_iommu_release_nesting_info (struct vfio_iommu *iommu) +{ +kfree (iommu->nesting_info); + iommu->nesting_info =NULL ; +} +static int vfio_iommu_type1_attach_group (void *iommu_data,struct iommu_group *iommu_group) {@@ -1959,6 +1971,12 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, } } +/* Nesting type container can include only one group */ +if (iommu->nesting &&CONTAINER_HAS_DOMAIN (iommu)) { +mutex_unlock (&iommu->lock); +return -EINVAL ; + } + group =kzalloc (sizeof (*group),GFP_KERNEL ); domain =kzalloc (sizeof (*domain),GFP_KERNEL );if (!group || !domain) {@@ -2029,6 +2047,32 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,if (ret) goto out_domain; +/* Nesting cap info is available only after attaching */ +if (iommu->nesting) { +struct iommu_nesting_info tmp = { .argsz =0 , }; + +/* First get the size of vendor specific nesting info */ + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + &tmp); +if (ret) + goto out_detach; + + iommu->nesting_info =kzalloc (tmp.argsz,GFP_KERNEL ); +if (!iommu->nesting_info) { + ret = -ENOMEM ; + goto out_detach; + } + +/* Now get the nesting info */ + iommu->nesting_info->argsz = tmp.argsz; + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + iommu->nesting_info); +if (ret) + goto out_detach; + } +/* Get aperture info */ iommu_domain_get_attr (domain->domain,DOMAIN_ATTR_GEOMETRY , &geo);@@ -2138,6 +2182,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,return 0 ; out_detach: +vfio_iommu_release_nesting_info (iommu);vfio_iommu_detach_group (domain, group); out_domain:iommu_domain_free (domain->domain);@@ -2338,6 +2383,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,vfio_iommu_unmap_unpin_all (iommu); elsevfio_iommu_unmap_unpin_reaccount (iommu ); + +vfio_iommu_release_nesting_info (iommu); }iommu_domain_free (domain->domain);list_del (&domain->next);@@ -2546,6 +2593,39 @@ static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig));
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu, · line 2603
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-0} +static int vfio_iommu_add_nesting_cap (struct vfio_iommu *iommu, +struct vfio_info_cap *caps) +{ +struct vfio_info_cap_header *header; +struct vfio_iommu_type1_info_cap_nesting *nesting_cap; +size_t size; + +/* when nesting_info is null, no need go further */
no need to go
Quoted 11 lines
drivers/vfio/vfio_iommu_type1.c +11
-0+if (!iommu->nesting_info) +return 0 ; + +/* when @format of nesting_info is 0, fail the call */ +if (iommu->nesting_info->format ==0 ) +return -ENOENT ; + + size = offsetof(struct vfio_iommu_type1_info_cap_nesting , info) + + iommu->nesting_info->argsz; + + header = vfio_info_cap_add(caps, size,
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu, · line 2624
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +10
-0+ VFIO_IOMMU_TYPE1_INFO_CAP_NESTING, 1); +if (IS_ERR (header)) +return PTR_ERR (header); + + nesting_cap =container_of (header, + struct vfio_iommu_type1_info_cap_nesting, + header); + +memcpy (&nesting_cap->info, iommu->nesting_info, + iommu->nesting_info->argsz);
can't you use vfio_info_add_capability() directly?
Quoted 30 lines
drivers/vfio/vfio_iommu_type1.c +6
-0+ +return 0 ; +} +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu,unsigned long arg) {@@ -2581,6 +2661,8 @@ static int vfio_iommu_type1_get_info(struct vfio_iommu *iommu,if (!ret) ret =vfio_iommu_iova_build_caps (iommu, &caps); + ret =vfio_iommu_add_nesting_cap (iommu, &caps); +mutex_unlock (&iommu->lock);if (ret)include/uapi/linux/vfio.h +1
-0@@ -14,6 +14,7 @@#include <linux/types.h> #include <linux/ioctl.h> +#include <linux/iommu.h> #define VFIO_API_VERSION 0@@ -1039,6 +1040,24 @@ struct vfio_iommu_type1_info_cap_migration {__u64 max_dirty_bitmap_size;/* in bytes */
include/uapi/linux/vfio.h struct vfio_iommu_type1_info_cap_migration { · line 1050
Quoted 10 lines
include/uapi/linux/vfio.h +8
-0}; +/* +* The nesting capability allows to report the related capability +* and info for nesting iommu type. +* +* The structures below define version 1 of this capability. +* +* Userspace selected VFIO_TYPE1_NESTING_IOMMU type should check +* this capability to get supported features.
nested capabilities should be checked by the userspace after setting VFIO_TYPE1_NESTING_IOMMU?
Quoted 4 lines
include/uapi/linux/vfio.h +3
-0+* +* @info: the nesting info provided by IOMMU driver. +*/
include/uapi/linux/vfio.h struct vfio_iommu_type1_info_cap_migration { · line 1063
Quoted 10 lines
include/uapi/linux/vfio.h +7
-0+#define VFIO_IOMMU_TYPE1_INFO_CAP_NESTING 3 + +struct vfio_iommu_type1_info_cap_nesting { +struct vfio_info_cap_header header; +struct iommu_nesting_info info; +}; +#define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)/**
Thanks
Eric
RE: [PATCH v6 04/15] vfio/type1: Report iommu nesting info to userspace
Hi Eric,
Quoted 209 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 9:20 PMHi Yi,
Quoted 6 lines · Liu Yi L
This patch exports iommu nesting capability info to user space through VFIO. Userspace is expected to check this info for supported uAPIs (e.g. PASID alloc/free, bind page table, and cache invalidation) and the vendor specific format information for first level/stage page table that will be bound to.
Quoted 13 lines
The nesting info is available only after container set to be NESTED type. Current implementation imposes one limitation - one nesting container should include at most one iommu group. The philosophy of vfio container is having all groups/devices within the container share the same IOMMU context. When vSVA is enabled, one IOMMU context could include one 2nd- level address space and multiple 1st-level address spaces. While the 2nd-level address space is reasonably sharable by multiple groups, blindly sharing 1st-level address spaces across all groups within the container might instead break the guest expectation. In the future sub/super container concept might be introduced to allow partial address space sharing within an IOMMU context. But for now let's go with this restriction by requiring singleton container for using nesting iommu features. Below link has the related discussion about this
decision.
Quoted 12 lines
https://lore.kernel.org/kvm/[email protected]/
This patch also changes the NESTING type container behaviour.
Something that would have succeeded before will now fail: Before this series, if user asked for aVFIO_IOMMU_TYPE1_NESTING, it would have succeeded even if the SMMU didn't support stage-2, as the driver would have silently fallen back on stage-1 mappings (which work exactly the same as stage-2 only since there was no nesting supported). After the series, we do check forDOMAIN_ATTR_NESTINGso if user asks forVFIO_IOMMU_TYPE1_NESTINGand the SMMU doesn't support stage-2, the ioctl fails. But it should be a good fix and completely harmless. Detail can be foundin below link as well.
Quoted 17 lines
https://lore.kernel.org/kvm/20200717090900.GC4850@myrica/
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) address comments against v5 from Eric Auger.
*) don't report nesting cap to userspace if thenesting_info->formatis
invalid.Quoted 3 lines
v4 -> v5:
*) address comments from Eric Auger.
*) return structiommu_nesting_infofor
VFIO_IOMMU_TYPE1_INFO_CAP_NESTINGasQuoted 10 lines
cap is much "cheap", if needs extension in future, just define another cap.
https://lore.kernel.org/kvm/[email protected]/v3 -> v4:
*) address comments against v3.v1 -> v2:
*) added in v2--- drivers/vfio/vfio_iommu_type1.c | 106+++++++++++++++++++++++++++++++++++-----
Quoted 41 lines
include/uapi/linux/vfio.h | 19 +++++++ 2 files changed, 113 insertions(+), 12 deletions(-)(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex 3bd70ff..18ff0c3 100644drivers/vfio/vfio_iommu_type1.c +14
-12@@ -62,18 +62,20 @@ MODULE_PARM_DESC(dma_entry_limit, "Maximum number of user DMA mappings per container (65535).");struct vfio_iommu { -struct list_head domain_list; -struct list_head iova_list; -struct vfio_domain *external_domain;/* domain for external user */ -struct mutex lock; -struct rb_root dma_list; -struct blocking_notifier_head notifier; -unsigned int dma_avail; -uint64_t pgsize_bitmap; -bool v2; -bool nesting; -bool dirty_page_tracking; -bool pinned_page_dirty_scope; +struct list_head domain_list; +struct list_head iova_list; +/* domain for external user */ +struct vfio_domain *external_domain; +struct mutex lock; +struct rb_root dma_list; +struct blocking_notifier_head notifier; +unsigned int dma_avail; +uint64_t pgsize_bitmap; +bool v2; +bool nesting; +bool dirty_page_tracking; +bool pinned_page_dirty_scope; +struct iommu_nesting_info *nesting_info; };struct vfio_domain {drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 138
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +3
-0@@ -130,6 +132,9 @@ struct vfio_regions {#define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER (iommu) \ (!list_empty(&iommu->domain_list)) +#define CONTAINER_HAS_DOMAIN (iommu) (((iommu)->external_domain) || \ + (!list_empty(&(iommu)->domain_list))) +#define DIRTY_BITMAP_BYTES (n) (ALIGN(n, BITS_PER_TYPE(u64)) /
BITS_PER_BYTE)Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +0
-0/* @@ -1929,6 +1934,13 @@ static void vfio_iommu_iova_insert_copy(structvfio_iommu *iommu,list_splice_tail (iova_copy ,iova );drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_iova_insert_copy(struct · line 1946
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +7
-0} + +static void vfio_iommu_release_nesting_info (struct vfio_iommu *iommu) +{ +kfree (iommu->nesting_info); + iommu->nesting_info =NULL ; +} +static int vfio_iommu_type1_attach_group (void *iommu_data,struct iommu_group *iommu_group){ @@ -1959,6 +1971,12 @@
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 14 lines
static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c +6
-0} } +/* Nesting type container can include only one group */ +if (iommu->nesting &&CONTAINER_HAS_DOMAIN (iommu)) { +mutex_unlock (&iommu->lock); +return -EINVAL ; + } + group =kzalloc (sizeof (*group),GFP_KERNEL ); domain =kzalloc (sizeof (*domain),GFP_KERNEL );if (!group || !domain) {@@ -2029,6 +2047,32 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,Quoted 21 lines
drivers/vfio/vfio_iommu_type1.c +18
-0if (ret) goto out_domain; +/* Nesting cap info is available only after attaching */ +if (iommu->nesting) { +struct iommu_nesting_info tmp = { .argsz =0 , }; + +/* First get the size of vendor specific nesting info */ + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + &tmp); +if (ret) + goto out_detach; + + iommu->nesting_info =kzalloc (tmp.argsz,GFP_KERNEL ); +if (!iommu->nesting_info) { + ret = -ENOMEM ; + goto out_detach; + } + +/* Now get the nesting info */ drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+ iommu->nesting_info->argsz = tmp.argsz; + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + iommu->nesting_info); +if (ret) + goto out_detach; + } +/* Get aperture info */ iommu_domain_get_attr(domain->domain, DOMAIN_ATTR_GEOMETRY,&geo);
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 2 lines
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2138,6 +2182,7 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_type1_detach_group(void
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c +1
-0return 0 ; out_detach: +vfio_iommu_release_nesting_info (iommu);vfio_iommu_detach_group (domain, group); out_domain:iommu_domain_free (domain->domain);@@ -2338,6 +2383,8 @@ static void vfio_iommu_type1_detach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_type1_detach_group(void *iommu_data,
Quoted 3 lines
drivers/vfio/vfio_iommu_type1.c +0
-0vfio_iommu_unmap_unpin_all (iommu); elsevfio_iommu_unmap_unpin_reaccount(iommu);drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +2
-0+ +vfio_iommu_release_nesting_info (iommu); }iommu_domain_free (domain->domain);list_del (&domain->next);@@ -2546,6 +2593,39 @@ static int vfio_iommu_migration_build_caps(struct
vfio_iommu*iommu,Quoted 3 lines
drivers/vfio/vfio_iommu_type1.c +0
-0return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig)); }
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu, · line 2603
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +7
-0+static int vfio_iommu_add_nesting_cap (struct vfio_iommu *iommu, +struct vfio_info_cap *caps) { +struct vfio_info_cap_header *header; +struct vfio_iommu_type1_info_cap_nesting *nesting_cap; +size_t size; + +/* when nesting_info is null, no need go further */ no need to go
nice catch.
Quoted 14 lines
Quoted 14 lines
drivers/vfio/vfio_iommu_type1.c +14
-0+if (!iommu->nesting_info) +return 0 ; + +/* when @format of nesting_info is 0, fail the call */ +if (iommu->nesting_info->format ==0 ) +return -ENOENT ; + + size = offsetof(struct vfio_iommu_type1_info_cap_nesting , info) + + iommu->nesting_info->argsz; + + header =vfio_info_cap_add (caps, size, +VFIO_IOMMU_TYPE1_INFO_CAP_NESTING ,1 ); +if (IS_ERR (header)) +return PTR_ERR (header);
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu, · line 2624
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +7
-0+ + nesting_cap =container_of (header, + struct vfio_iommu_type1_info_cap_nesting, + header); + +memcpy (&nesting_cap->info, iommu->nesting_info, + iommu->nesting_info->argsz);can't you use
vfio_info_add_capability()directly?
yes, the below lines will be covered by vfio_info_add_capability().
+ header = vfio_info_cap_add(caps, size,
+ VFIO_IOMMU_TYPE1_INFO_CAP_NESTING, 1);
+ if (IS_ERR(header))
+ return PTR_ERR(header);
+
+ nesting_cap = container_of(header,
+ struct vfio_iommu_type1_info_cap_nesting,
+ header);
Quoted 33 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_get_info(struct
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c +4
-0+ +return 0 ; +} +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu,unsigned long arg) {@@ -2581,6 +2661,8 @@ static int vfio_iommu_type1_get_info(struct
vfio_iommu*iommu,Quoted 24 lines
drivers/vfio/vfio_iommu_type1.c +2
-0if (!ret) ret =vfio_iommu_iova_build_caps (iommu, &caps); + ret =vfio_iommu_add_nesting_cap (iommu, &caps); +mutex_unlock (&iommu->lock);if (ret)include/uapi/linux/vfio.h +1
-0@@ -14,6 +14,7 @@#include <linux/types.h> #include <linux/ioctl.h> +#include <linux/iommu.h> #define VFIO_API_VERSION 0@@ -1039,6 +1040,24 @@ struct vfio_iommu_type1_info_cap_migration {__u64 max_dirty_bitmap_size;/* in bytes */ };
Quoted 10 lines
include/uapi/linux/vfio.h struct vfio_iommu_type1_info_cap_migration { · line 1050
Quoted 8 lines
include/uapi/linux/vfio.h +8
-0+/* +* The nesting capability allows to report the related capability +* and info for nesting iommu type. +* +* The structures below define version 1 of this capability. +* +* Userspace selected VFIO_TYPE1_NESTING_IOMMU type should check +* this capability to get supported features. nested capabilities should be checked by the userspace after setting
VFIO_TYPE1_NESTING_IOMMU?
yes, will modify it.
Regards,
Yi Liu
Quoted 12 lines
Quoted 12 lines
include/uapi/linux/vfio.h +10
-0+* +* @info: the nesting info provided by IOMMU driver. +*/ +#define VFIO_IOMMU_TYPE1_INFO_CAP_NESTING 3 + +struct vfio_iommu_type1_info_cap_nesting { +struct vfio_info_cap_header header; +struct iommu_nesting_info info; +}; +#define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
Quoted 5 lines
include/uapi/linux/vfio.h struct vfio_iommu_type1_info_cap_migration { · line 1063
Quoted 2 lines
include/uapi/linux/vfio.h +0
-0/** Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 04/15] vfio/type1: Report iommu nesting info to userspace
In reply to Liu Yi LQuoted 139 lines · On Mon, 27 Jul 2020 23:27:33 -0700 Liu Yi L <[email protected]> wrote:
This patch exports iommu nesting capability info to user space through VFIO. Userspace is expected to check this info for supported uAPIs (e.g. PASID alloc/free, bind page table, and cache invalidation) and the vendor specific format information for first level/stage page table that will be bound to.
The nesting info is available only after container set to be NESTED type. Current implementation imposes one limitation - one nesting container should include at most one iommu group. The philosophy of vfio container is having all groups/devices within the container share the same IOMMU context. When vSVA is enabled, one IOMMU context could include one 2nd- level address space and multiple 1st-level address spaces. While the 2nd-level address space is reasonably sharable by multiple groups, blindly sharing 1st-level address spaces across all groups within the container might instead break the guest expectation. In the future sub/super container concept might be introduced to allow partial address space sharing within an IOMMU context. But for now let's go with this restriction by requiring singleton container for using nesting iommu features. Below link has the related discussion about this decision.
https://lore.kernel.org/kvm/[email protected]/
This patch also changes the NESTING type container behaviour. Something that would have succeeded before will now fail: Before this series, if user asked for a
VFIO_IOMMU_TYPE1_NESTING, it would have succeeded even if the SMMU didn't support stage-2, as the driver would have silently fallen back on stage-1 mappings (which work exactly the same as stage-2 only since there was no nesting supported). After the series, we do check forDOMAIN_ATTR_NESTINGso if user asks forVFIO_IOMMU_TYPE1_NESTINGand the SMMU doesn't support stage-2, the ioctl fails. But it should be a good fix and completely harmless. Detail can be found in below link as well.https://lore.kernel.org/kvm/20200717090900.GC4850@myrica/
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) address comments against v5 from Eric Auger.
*) don't report nesting cap to userspace if thenesting_info->formatis
invalid.v4 -> v5:
*) address comments from Eric Auger.
*) return structiommu_nesting_infoforVFIO_IOMMU_TYPE1_INFO_CAP_NESTINGas
cap is much "cheap", if needs extension in future, just define another cap.
https://lore.kernel.org/kvm/[email protected]/v3 -> v4:
*) address comments against v3.v1 -> v2:
*) added in v2--- drivers/vfio/vfio_iommu_type1.c | 106 +++++++++++++++++++++++++++++++++++----- include/uapi/linux/vfio.h | 19 +++++++ 2 files changed, 113 insertions(+), 12 deletions(-)drivers/vfio/vfio_iommu_type1.c +30
-12@@ -62,18 +62,20 @@ MODULE_PARM_DESC(dma_entry_limit, "Maximum number of user DMA mappings per container (65535).");struct vfio_iommu { -struct list_head domain_list; -struct list_head iova_list; -struct vfio_domain *external_domain;/* domain for external user */ -struct mutex lock; -struct rb_root dma_list; -struct blocking_notifier_head notifier; -unsigned int dma_avail; -uint64_t pgsize_bitmap; -bool v2; -bool nesting; -bool dirty_page_tracking; -bool pinned_page_dirty_scope; +struct list_head domain_list; +struct list_head iova_list; +/* domain for external user */ +struct vfio_domain *external_domain; +struct mutex lock; +struct rb_root dma_list; +struct blocking_notifier_head notifier; +unsigned int dma_avail; +uint64_t pgsize_bitmap; +bool v2; +bool nesting; +bool dirty_page_tracking; +bool pinned_page_dirty_scope; +struct iommu_nesting_info *nesting_info; };struct vfio_domain {@@ -130,6 +132,9 @@ struct vfio_regions {#define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER (iommu) \ (!list_empty(&iommu->domain_list)) +#define CONTAINER_HAS_DOMAIN (iommu) (((iommu)->external_domain) || \ + (!list_empty(&(iommu)->domain_list))) +#define DIRTY_BITMAP_BYTES (n) (ALIGN(n, BITS_PER_TYPE(u64)) / BITS_PER_BYTE)/* @@ -1929,6 +1934,13 @@ static void vfio_iommu_iova_insert_copy(struct vfio_iommu *iommu,list_splice_tail (iova_copy, iova); } + +static void vfio_iommu_release_nesting_info (struct vfio_iommu *iommu) +{ +kfree (iommu->nesting_info); + iommu->nesting_info =NULL ; +} +static int vfio_iommu_type1_attach_group (void *iommu_data,struct iommu_group *iommu_group) {@@ -1959,6 +1971,12 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, } } +/* Nesting type container can include only one group */ +if (iommu->nesting &&CONTAINER_HAS_DOMAIN (iommu)) { +mutex_unlock (&iommu->lock); +return -EINVAL ; + } + group =kzalloc (sizeof (*group),GFP_KERNEL ); domain =kzalloc (sizeof (*domain),GFP_KERNEL );if (!group || !domain) {
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data, · line 2048
Quoted 2 lines
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2029,6 +2047,32 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,if (ret) goto out_domain;
I think that currently a user can configure a VFIO_TYPE1_NESTING_IOMMU IOMMU type with a non-IOMMU backed mdev. Does that make any sort of sense or is this a bug? Thanks,
Alex
Quoted 133 lines
drivers/vfio/vfio_iommu_type1.c +64
-0+/* Nesting cap info is available only after attaching */ +if (iommu->nesting) { +struct iommu_nesting_info tmp = { .argsz =0 , }; + +/* First get the size of vendor specific nesting info */ + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + &tmp); +if (ret) + goto out_detach; + + iommu->nesting_info =kzalloc (tmp.argsz,GFP_KERNEL ); +if (!iommu->nesting_info) { + ret = -ENOMEM ; + goto out_detach; + } + +/* Now get the nesting info */ + iommu->nesting_info->argsz = tmp.argsz; + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + iommu->nesting_info); +if (ret) + goto out_detach; + } +/* Get aperture info */ iommu_domain_get_attr (domain->domain,DOMAIN_ATTR_GEOMETRY , &geo);@@ -2138,6 +2182,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,return 0 ; out_detach: +vfio_iommu_release_nesting_info (iommu);vfio_iommu_detach_group (domain, group); out_domain:iommu_domain_free (domain->domain);@@ -2338,6 +2383,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,vfio_iommu_unmap_unpin_all (iommu); elsevfio_iommu_unmap_unpin_reaccount (iommu ); + +vfio_iommu_release_nesting_info (iommu); }iommu_domain_free (domain->domain);list_del (&domain->next);@@ -2546,6 +2593,39 @@ static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig)); } +static int vfio_iommu_add_nesting_cap (struct vfio_iommu *iommu, +struct vfio_info_cap *caps) +{ +struct vfio_info_cap_header *header; +struct vfio_iommu_type1_info_cap_nesting *nesting_cap; +size_t size; + +/* when nesting_info is null, no need go further */ +if (!iommu->nesting_info) +return 0 ; + +/* when @format of nesting_info is 0, fail the call */ +if (iommu->nesting_info->format ==0 ) +return -ENOENT ; + + size = offsetof(struct vfio_iommu_type1_info_cap_nesting , info) + + iommu->nesting_info->argsz; + + header =vfio_info_cap_add (caps, size, +VFIO_IOMMU_TYPE1_INFO_CAP_NESTING ,1 ); +if (IS_ERR (header)) +return PTR_ERR (header); + + nesting_cap =container_of (header, + struct vfio_iommu_type1_info_cap_nesting, + header); + +memcpy (&nesting_cap->info, iommu->nesting_info, + iommu->nesting_info->argsz); + +return 0 ; +} +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu,unsigned long arg) {@@ -2581,6 +2661,8 @@ static int vfio_iommu_type1_get_info(struct vfio_iommu *iommu,if (!ret) ret =vfio_iommu_iova_build_caps (iommu, &caps); + ret =vfio_iommu_add_nesting_cap (iommu, &caps); +mutex_unlock (&iommu->lock);if (ret)include/uapi/linux/vfio.h +19
-0@@ -14,6 +14,7 @@#include <linux/types.h> #include <linux/ioctl.h> +#include <linux/iommu.h> #define VFIO_API_VERSION 0@@ -1039,6 +1040,24 @@ struct vfio_iommu_type1_info_cap_migration {__u64 max_dirty_bitmap_size;/* in bytes */ }; +/* +* The nesting capability allows to report the related capability +* and info for nesting iommu type. +* +* The structures below define version 1 of this capability. +* +* Userspace selected VFIO_TYPE1_NESTING_IOMMU type should check +* this capability to get supported features. +* +* @info: the nesting info provided by IOMMU driver. +*/ +#define VFIO_IOMMU_TYPE1_INFO_CAP_NESTING 3 + +struct vfio_iommu_type1_info_cap_nesting { +struct vfio_info_cap_header header; +struct iommu_nesting_info info; +}; +#define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)/**
RE: [PATCH v6 04/15] vfio/type1: Report iommu nesting info to userspace
Hi Alex,
Quoted 152 lines
From: Alex Williamson <[email protected]>
Sent: Friday, August 21, 2020 3:52 AMQuoted 22 lines · On Mon, 27 Jul 2020 23:27:33 -0700 Liu Yi L <[email protected]> wrote:
This patch exports iommu nesting capability info to user space through VFIO. Userspace is expected to check this info for supported uAPIs (e.g. PASID alloc/free, bind page table, and cache invalidation) and the vendor specific format information for first level/stage page table that will be bound to.
The nesting info is available only after container set to be NESTED type. Current implementation imposes one limitation - one nesting container should include at most one iommu group. The philosophy of vfio container is having all groups/devices within the container share the same IOMMU context. When vSVA is enabled, one IOMMU context could include one 2nd- level address space and multiple 1st-level address spaces. While the 2nd-level address space is reasonably sharable by multiple groups, blindly sharing 1st-level address spaces across all groups within the container might instead break the guest expectation. In the future sub/super container concept might be introduced to allow partial address space sharing within an IOMMU context. But for now let's go with this restriction by requiring singleton container for using nesting iommu features. Below link has the related discussion about this decision.
Quoted 7 lines
This patch also changes the NESTING type container behaviour. Something that would have succeeded before will now fail: Before this series, if user asked for a
VFIO_IOMMU_TYPE1_NESTING, it would have succeeded even if the SMMU didn't support stage-2, as the driver would have silently fallen back on stage-1 mappings (which work exactly the same as stage-2 only since there was no nesting supported). After the series, we do check forDOMAIN_ATTR_NESTINGso if user asks forVFIO_IOMMU_TYPE1_NESTINGand
Quoted 19 lines
the SMMU doesn't support stage-2, the ioctl fails. But it should be a good fix and completely harmless. Detail can be found in below link as well.
https://lore.kernel.org/kvm/20200717090900.GC4850@myrica/
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) address comments against v5 from Eric Auger.
*) don't report nesting cap to userspace if thenesting_info->formatis
invalid.Quoted 3 lines
v4 -> v5:
*) address comments from Eric Auger.
*) return structiommu_nesting_infofor
VFIO_IOMMU_TYPE1_INFO_CAP_NESTINGasQuoted 10 lines
cap is much "cheap", if needs extension in future, just define another cap.
https://lore.kernel.org/kvm/[email protected]/v3 -> v4:
*) address comments against v3.v1 -> v2:
*) added in v2--- drivers/vfio/vfio_iommu_type1.c | 106+++++++++++++++++++++++++++++++++++-----
Quoted 41 lines
include/uapi/linux/vfio.h | 19 +++++++ 2 files changed, 113 insertions(+), 12 deletions(-)drivers/vfio/vfio_iommu_type1.c +14
-12@@ -62,18 +62,20 @@ MODULE_PARM_DESC(dma_entry_limit, "Maximum number of user DMA mappings per container (65535).");struct vfio_iommu { -struct list_head domain_list; -struct list_head iova_list; -struct vfio_domain *external_domain;/* domain for external user */ -struct mutex lock; -struct rb_root dma_list; -struct blocking_notifier_head notifier; -unsigned int dma_avail; -uint64_t pgsize_bitmap; -bool v2; -bool nesting; -bool dirty_page_tracking; -bool pinned_page_dirty_scope; +struct list_head domain_list; +struct list_head iova_list; +/* domain for external user */ +struct vfio_domain *external_domain; +struct mutex lock; +struct rb_root dma_list; +struct blocking_notifier_head notifier; +unsigned int dma_avail; +uint64_t pgsize_bitmap; +bool v2; +bool nesting; +bool dirty_page_tracking; +bool pinned_page_dirty_scope; +struct iommu_nesting_info *nesting_info; };struct vfio_domain {drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 138
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +3
-0@@ -130,6 +132,9 @@ struct vfio_regions {#define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER (iommu) \ (!list_empty(&iommu->domain_list)) +#define CONTAINER_HAS_DOMAIN (iommu) (((iommu)->external_domain) || \ + (!list_empty(&(iommu)->domain_list))) +#define DIRTY_BITMAP_BYTES (n) (ALIGN(n, BITS_PER_TYPE(u64)) /
BITS_PER_BYTE)drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_iova_insert_copy(struct
Quoted 3 lines
drivers/vfio/vfio_iommu_type1.c +0
-0/* @@ -1929,6 +1934,13 @@ static void vfio_iommu_iova_insert_copy(struct
vfio_iommu*iommu,drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 14 lines
drivers/vfio/vfio_iommu_type1.c +7
-0list_splice_tail (iova_copy, iova); } + +static void vfio_iommu_release_nesting_info (struct vfio_iommu *iommu) +{ +kfree (iommu->nesting_info); + iommu->nesting_info =NULL ; +} +static int vfio_iommu_type1_attach_group (void *iommu_data,struct iommu_group *iommu_group) {@@ -1959,6 +1971,12 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 13 lines
drivers/vfio/vfio_iommu_type1.c +6
-0} } +/* Nesting type container can include only one group */ +if (iommu->nesting &&CONTAINER_HAS_DOMAIN (iommu)) { +mutex_unlock (&iommu->lock); +return -EINVAL ; + } + group =kzalloc (sizeof (*group),GFP_KERNEL ); domain =kzalloc (sizeof (*domain),GFP_KERNEL );if (!group || !domain) {@@ -2029,6 +2047,32 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data, · line 2048
Quoted 2 lines
drivers/vfio/vfio_iommu_type1.c +0
-0if (ret) goto out_domain;I think that currently a user can configure a
VFIO_TYPE1_NESTING_IOMMUIOMMU type with a non-IOMMU backed mdev. Does that make any sort of sense or is this a bug? Thanks,
I guess we don't support such case as non-IOMMU mdev is attached to the external domain. It won't come to the part where allocates domain and sets nesting for the domain.
Regards,
Yi Liu
Quoted 140 lines
Alex
Quoted 19 lines
drivers/vfio/vfio_iommu_type1.c +18
-0+/* Nesting cap info is available only after attaching */ +if (iommu->nesting) { +struct iommu_nesting_info tmp = { .argsz =0 , }; + +/* First get the size of vendor specific nesting info */ + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + &tmp); +if (ret) + goto out_detach; + + iommu->nesting_info =kzalloc (tmp.argsz,GFP_KERNEL ); +if (!iommu->nesting_info) { + ret = -ENOMEM ; + goto out_detach; + } + +/* Now get the nesting info */ drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+ iommu->nesting_info->argsz = tmp.argsz; + ret =iommu_domain_get_attr (domain->domain, +DOMAIN_ATTR_NESTING , + iommu->nesting_info); +if (ret) + goto out_detach; + } +/* Get aperture info */ iommu_domain_get_attr(domain->domain, DOMAIN_ATTR_GEOMETRY,&geo);
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 2 lines
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2138,6 +2182,7 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_type1_detach_group(void
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c +1
-0return 0 ; out_detach: +vfio_iommu_release_nesting_info (iommu);vfio_iommu_detach_group (domain, group); out_domain:iommu_domain_free (domain->domain);@@ -2338,6 +2383,8 @@ static void vfio_iommu_type1_detach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_type1_detach_group(void *iommu_data,
Quoted 3 lines
drivers/vfio/vfio_iommu_type1.c +0
-0vfio_iommu_unmap_unpin_all (iommu); elsevfio_iommu_unmap_unpin_reaccount(iommu);drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_migration_build_caps(struct
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +2
-0+ +vfio_iommu_release_nesting_info (iommu); }iommu_domain_free (domain->domain);list_del (&domain->next);@@ -2546,6 +2593,39 @@ static int vfio_iommu_migration_build_caps(struct
vfio_iommu*iommu,drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_get_info(struct
Quoted 40 lines
drivers/vfio/vfio_iommu_type1.c +33
-0return vfio_info_add_capability (caps, &cap_mig.header,sizeof (cap_mig)); } +static int vfio_iommu_add_nesting_cap (struct vfio_iommu *iommu, +struct vfio_info_cap *caps) +{ +struct vfio_info_cap_header *header; +struct vfio_iommu_type1_info_cap_nesting *nesting_cap; +size_t size; + +/* when nesting_info is null, no need go further */ +if (!iommu->nesting_info) +return 0 ; + +/* when @format of nesting_info is 0, fail the call */ +if (iommu->nesting_info->format ==0 ) +return -ENOENT ; + + size = offsetof(struct vfio_iommu_type1_info_cap_nesting , info) + + iommu->nesting_info->argsz; + + header =vfio_info_cap_add (caps, size, +VFIO_IOMMU_TYPE1_INFO_CAP_NESTING ,1 ); +if (IS_ERR (header)) +return PTR_ERR (header); + + nesting_cap =container_of (header, + struct vfio_iommu_type1_info_cap_nesting, + header); + +memcpy (&nesting_cap->info, iommu->nesting_info, + iommu->nesting_info->argsz); + +return 0 ; +} +static int vfio_iommu_type1_get_info (struct vfio_iommu *iommu,unsigned long arg) {@@ -2581,6 +2661,8 @@ static int vfio_iommu_type1_get_info(struct
vfio_iommu*iommu,Quoted 44 lines
drivers/vfio/vfio_iommu_type1.c +2
-0if (!ret) ret =vfio_iommu_iova_build_caps (iommu, &caps); + ret =vfio_iommu_add_nesting_cap (iommu, &caps); +mutex_unlock (&iommu->lock);if (ret)include/uapi/linux/vfio.h +19
-0@@ -14,6 +14,7 @@#include <linux/types.h> #include <linux/ioctl.h> +#include <linux/iommu.h> #define VFIO_API_VERSION 0@@ -1039,6 +1040,24 @@ struct vfio_iommu_type1_info_cap_migration {__u64 max_dirty_bitmap_size;/* in bytes */ }; +/* +* The nesting capability allows to report the related capability +* and info for nesting iommu type. +* +* The structures below define version 1 of this capability. +* +* Userspace selected VFIO_TYPE1_NESTING_IOMMU type should check +* this capability to get supported features. +* +* @info: the nesting info provided by IOMMU driver. +*/ +#define VFIO_IOMMU_TYPE1_INFO_CAP_NESTING 3 + +struct vfio_iommu_type1_info_cap_nesting { +struct vfio_info_cap_header header; +struct iommu_nesting_info info; +}; +#define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
Quoted 1 line
include/uapi/linux/vfio.h struct vfio_iommu_type1_info_cap_migration { · line 1063
Quoted 1 line
include/uapi/linux/vfio.h +0
-0/**
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 5/15 vfio: Add PASID allocation/free support
In reply to Liu Yi LShared Virtual Addressing (a.k.a Shared Virtual Memory) allows sharing multiple process virtual address spaces with the device for simplified programming model. PASID is used to tag an virtual address space in DMA requests and to identify the related translation structure in IOMMU. When a PASID-capable device is assigned to a VM, we want the same capability of using PASID to tag guest process virtual address spaces to achieve virtual SVA (vSVA).
PASID management for guest is vendor specific. Some vendors (e.g. Intel VT-d) requires system-wide managed PASIDs across all devices, regardless of whether a device is used by host or assigned to guest. Other vendors (e.g. ARM SMMU) may allow PASIDs managed per-device thus could be fully delegated to the guest for assigned devices.
For system-wide managed PASIDs, this patch introduces a vfio module to handle explicit PASID alloc/free requests from guest. Allocated PASIDs are associated to a process (or, mm_struct) in IOASID core. A vfio_mm object is introduced to track mm_struct. Multiple VFIO containers within a process share the same vfio_mm object.
A quota mechanism is provided to prevent malicious user from exhausting available PASIDs. Currently the quota is a global parameter applied to all VFIO devices. In the future per-device quota might be supported too.
---
v5 -> v6:
*) address comments from Eric. Add vfio_unlink_pasid() to be consistent
with vfio_unlink_dma(). Add a comment in vfio_pasid_exit().
v4 -> v5:
*) address comments from Eric Auger.
*) address the comments from Alex on the pasid free range support. Added
per vfio_mm pasid r-b tree.
https://lore.kernel.org/kvm/[email protected]/
v3 -> v4:
*) fix lock leam in vfio_mm_get_from_task()
*) drop pasid_quota field in struct vfio_mm
*) vfio_mm_get_from_task() returns ERR_PTR(-ENOTTY) when !CONFIG_VFIO_PASID
v1 -> v2:
*) added in v2, split from the pasid alloc/free support of v1
--- drivers/vfio/Kconfig | 5 + drivers/vfio/Makefile | 1 + drivers/vfio/vfio_pasid.c | 248 ++++++++++++++++++++++++++++++++++++++++++++++ include/linux/vfio.h | 28 ++++++ 4 files changed, 282 insertions(+) create mode 100644 drivers/vfio/vfio_pasid.c
drivers/vfio/Kconfig +5 -0
@@ -19,6 +19,11 @@ config VFIO_VIRQFD depends on VFIO && EVENTFD default n +config VFIO_PASID + tristate + depends on IOASID && VFIO + default n + menuconfig VFIO tristate "VFIO Non-Privileged userspace driver framework" depends on IOMMU_API
drivers/vfio/Makefile +1 -0
@@ -3,6 +3,7 @@ vfio_virqfd-y := virqfd.o obj-$(CONFIG_VFIO) += vfio.o obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o +obj-$(CONFIG_VFIO_PASID) += vfio_pasid.o obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o obj-$(CONFIG_VFIO_SPAPR_EEH) += vfio_spapr_eeh.o
drivers/vfio/vfio_pasid.c new file +248 -0
248 lines not on this page · raw message
include/linux/vfio.h +28 -0
@@ -97,6 +97,34 @@ extern int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops);extern void vfio_unregister_iommu_driver (const struct vfio_iommu_driver_ops *ops); +struct vfio_mm ; +#if IS_ENABLED (CONFIG_VFIO_PASID ) +extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task); +extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max); +extern void vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max); +#else +static inline struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task) +{ +return ERR_PTR (-ENOTTY ); +} + +static inline void vfio_mm_put (struct vfio_mm *vmm) +{ +} + +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) +{ +return -ENOTTY ; +} + +static inline void vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max) +{ +} +#endif /* CONFIG_VFIO_PASID */ +/* * External user API */
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 05/15] vfio: Add PASID allocation/free support
Yi,
Quoted 92 lines · Liu Yi L
Shared Virtual Addressing (a.k.a Shared Virtual Memory) allows sharing multiple process virtual address spaces with the device for simplified programming model. PASID is used to tag an virtual address space in DMA requests and to identify the related translation structure in IOMMU. When a PASID-capable device is assigned to a VM, we want the same capability of using PASID to tag guest process virtual address spaces to achieve virtual SVA (vSVA).
PASID management for guest is vendor specific. Some vendors (e.g. Intel VT-d) requires system-wide managed PASIDs across all devices, regardless of whether a device is used by host or assigned to guest. Other vendors (e.g. ARM SMMU) may allow PASIDs managed per-device thus could be fully delegated to the guest for assigned devices.
For system-wide managed PASIDs, this patch introduces a vfio module to handle explicit PASID alloc/free requests from guest. Allocated PASIDs are associated to a process (or,
mm_struct) in IOASID core. Avfio_mmobject is introduced to trackmm_struct. Multiple VFIO containers within a process share the samevfio_mmobject.A quota mechanism is provided to prevent malicious user from exhausting available PASIDs. Currently the quota is a global parameter applied to all VFIO devices. In the future per-device quota might be supported too.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Suggested-by Alex Williamson <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) address comments from Eric. Addvfio_unlink_pasid()to be consistent
withvfio_unlink_dma(). Add a comment invfio_pasid_exit().v4 -> v5:
*) address comments from Eric Auger.
*) address the comments from Alex on the pasid free range support. Added
pervfio_mmpasid r-b tree.
https://lore.kernel.org/kvm/[email protected]/v3 -> v4:
*) fix lock leam invfio_mm_get_from_task()
*) droppasid_quotafield in structvfio_mm
*)vfio_mm_get_from_task()returnsERR_PTR(-ENOTTY)when !CONFIG_VFIO_PASIDv1 -> v2:
*) added in v2, split from the pasid alloc/free support of v1--- drivers/vfio/Kconfig | 5 + drivers/vfio/Makefile | 1 + drivers/vfio/vfio_pasid.c | 248 ++++++++++++++++++++++++++++++++++++++++++++++ include/linux/vfio.h | 28 ++++++ 4 files changed, 282 insertions(+) create mode 100644 drivers/vfio/vfio_pasid.cdrivers/vfio/Kconfig +5
-0@@ -19,6 +19,11 @@ config VFIO_VIRQFD depends on VFIO && EVENTFD default n +config VFIO_PASID + tristate + depends on IOASID && VFIO + default n + menuconfig VFIO tristate "VFIO Non-Privileged userspace driver framework" depends on IOMMU_APIdrivers/vfio/Makefile +1
-0@@ -3,6 +3,7 @@ vfio_virqfd-y := virqfd.o obj-$(CONFIG_VFIO) += vfio.o obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o +obj-$(CONFIG_VFIO_PASID) += vfio_pasid.o obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o obj-$(CONFIG_VFIO_SPAPR_EEH) += vfio_spapr_eeh.odrivers/vfio/vfio_pasid.c new file +0
-0
drivers/vfio/vfio_pasid.c line 9
Quoted 9 lines
drivers/vfio/vfio_pasid.c new file +9
-0@@ -0,0 +1,248 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2020 Intel Corporation. +* Author: Liu Yi L <[email protected]> +* +*/ + +#include <linux/vfio.h> +#include <linux/eventfd.h>
not needed
Quoted 268 lines
drivers/vfio/vfio_pasid.c new file +238
-0+#include <linux/file.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/sched/mm.h> + +#define DRIVER_VERSION "0.1" +#define DRIVER_AUTHOR "Liu Yi L <[email protected]>" +#define DRIVER_DESC "PASID management for VFIO bus drivers" + +#define VFIO_DEFAULT_PASID_QUOTA 1000 +static int pasid_quota =VFIO_DEFAULT_PASID_QUOTA ; +module_param_named (pasid_quota, pasid_quota, uint,0444 ); +MODULE_PARM_DESC (pasid_quota, +"Set the quota for max number of PASIDs that an application is allowed to request (default 1000)" ); + +struct vfio_mm_token { +unsigned long long val; +}; + +struct vfio_mm { +struct kref kref; +int ioasid_sid; +struct mutex pasid_lock; +struct rb_root pasid_list; +struct list_head next; +struct vfio_mm_token token; +}; + +static struct mutex vfio_mm_lock; +static struct list_head vfio_mm_list; + +struct vfio_pasid { +struct rb_node node; +ioasid_t pasid; +}; + +static void vfio_remove_all_pasids (struct vfio_mm *vmm); + +/* called with vfio.vfio_mm_lock held */ +static void vfio_mm_release (struct kref *kref) +{ + struct vfio_mm *vmm = container_of(kref, struct vfio_mm, kref); + +list_del (&vmm->next); +mutex_unlock (&vfio_mm_lock); +vfio_remove_all_pasids (vmm); +ioasid_free_set (vmm->ioasid_sid, true); +kfree (vmm); +} + +void vfio_mm_put (struct vfio_mm *vmm) +{ +kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); +} + +static void vfio_mm_get (struct vfio_mm *vmm) +{ +kref_get (&vmm->kref); +} + +struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task) +{ +struct mm_struct *mm =get_task_mm (task); +struct vfio_mm *vmm; +unsigned long long val = (unsigned long long )mm; +int ret; + +mutex_lock (&vfio_mm_lock); +/* Search existing vfio_mm with current mm pointer */ +list_for_each_entry (vmm, &vfio_mm_list, next) { +if (vmm->token.val == val) { +vfio_mm_get (vmm); + goto out; + } + } + + vmm =kzalloc (sizeof (*vmm),GFP_KERNEL ); +if (!vmm) { + vmm =ERR_PTR (-ENOMEM ); + goto out; + } + +/* +* IOASID core provides a 'IOASID set' concept to track all +* PASIDs associated with a token. Here we use mm_struct as +* the token and create a IOASID set per mm_struct. All the +* containers of the process share the same IOASID set. +*/ + ret =ioasid_alloc_set ((struct ioasid_set *)mm, pasid_quota, + &vmm->ioasid_sid); +if (ret) { +kfree (vmm); + vmm =ERR_PTR (ret); + goto out; + } + +kref_init (&vmm->kref); + vmm->token.val = val; +mutex_init (&vmm->pasid_lock); + vmm->pasid_list =RB_ROOT ; + + list_add(&vmm->next, &vfio_mm_list);> +out: +mutex_unlock (&vfio_mm_lock); +mmput (mm); +return vmm; +} + +/* +* Find PASID within @min and @max +*/ +static struct vfio_pasid *vfio_find_pasid (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max) +{ +struct rb_node *node = vmm->pasid_list.rb_node; + +while (node) { +struct vfio_pasid *vid =rb_entry (node, + struct vfio_pasid, node); + +if (max < vid->pasid) + node = node->rb_left; +else if (min > vid->pasid) + node = node->rb_right; +else +return vid; + } + +return NULL ; +} + +static void vfio_link_pasid (struct vfio_mm *vmm,struct vfio_pasid *new) +{ +struct rb_node **link = &vmm->pasid_list.rb_node, *parent =NULL ; +struct vfio_pasid *vid; + +while (*link) { + parent = *link; + vid = rb_entry(parent, struct vfio_pasid, node); + +if (new->pasid <= vid->pasid) + link = &(*link)->rb_left; +else + link = &(*link)->rb_right; + } + +rb_link_node (&new->node, parent, link); +rb_insert_color (&new->node, &vmm->pasid_list); +} + +static void vfio_unlink_pasid (struct vfio_mm *vmm,struct vfio_pasid *old) +{ +rb_erase (&old->node, &vmm->pasid_list); +} + +static void vfio_remove_pasid (struct vfio_mm *vmm,struct vfio_pasid *vid) +{ +vfio_unlink_pasid (vmm, vid); +ioasid_free (vid->pasid); +kfree (vid); +} + +static void vfio_remove_all_pasids (struct vfio_mm *vmm) +{ +struct rb_node *node; + +mutex_lock (&vmm->pasid_lock); +while ((node =rb_first (&vmm->pasid_list))) + vfio_remove_pasid(vmm, rb_entry(node, struct vfio_pasid, node)); +mutex_unlock (&vmm->pasid_lock); +} + +int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) +{ +ioasid_t pasid; +struct vfio_pasid *vid; + + pasid =ioasid_alloc (vmm->ioasid_sid, min, max,NULL ); +if (pasid ==INVALID_IOASID ) +return -ENOSPC ; + + vid =kzalloc (sizeof (*vid),GFP_KERNEL ); +if (!vid) { +ioasid_free (pasid); +return -ENOMEM ; + } + + vid->pasid = pasid; + +mutex_lock (&vmm->pasid_lock); +vfio_link_pasid (vmm, vid); +mutex_unlock (&vmm->pasid_lock); + +return pasid; +} + +void vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max) +{ +struct vfio_pasid *vid =NULL ; + +/* +* IOASID core will notify PASID users (e.g. IOMMU driver) to +* teardown necessary structures depending on the to-be-freed +* PASID. +*/ +mutex_lock (&vmm->pasid_lock); +while ((vid =vfio_find_pasid (vmm, min, max)) !=NULL ) +vfio_remove_pasid (vmm, vid); +mutex_unlock (&vmm->pasid_lock); +} + +static int __init vfio_pasid_init (void ) +{ +mutex_init (&vfio_mm_lock); +INIT_LIST_HEAD (&vfio_mm_list); +return 0 ; +} + +static void __exit vfio_pasid_exit(void) +{ +/* +* VFIO_PASID is supposed to be referenced by VFIO_IOMMU_TYPE1 +* and may be other module. once vfio_pasid_exit() is triggered, +* that means its user (e.g. VFIO_IOMMU_TYPE1) has been removed. +* All the vfio_mm instances should have been released. If not, +* means there is vfio_mm leak, should be a bug of user module. +* So just warn here. +*/ +WARN_ON (!list_empty (&vfio_mm_list)); +} + +module_init (vfio_pasid_init); +module_exit (vfio_pasid_exit); + +MODULE_VERSION (DRIVER_VERSION ); +MODULE_LICENSE ("GPL v2" ); +MODULE_AUTHOR (DRIVER_AUTHOR ); +MODULE_DESCRIPTION (DRIVER_DESC );include/linux/vfio.h +21
-0@@ -97,6 +97,34 @@ extern int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops);extern void vfio_unregister_iommu_driver (const struct vfio_iommu_driver_ops *ops); +struct vfio_mm ; +#if IS_ENABLED (CONFIG_VFIO_PASID ) +extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task); +extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max); +extern void vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max); +#else +static inline struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task) +{ +return ERR_PTR (-ENOTTY ); +} + +static inline void vfio_mm_put (struct vfio_mm *vmm) +{ +} + +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) +{ +return -ENOTTY ; +}
include/linux/vfio.h extern int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops); · line 130
Quoted 10 lines
include/linux/vfio.h +7
-0+ +static inline void vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max) +{ +} +#endif /* CONFIG_VFIO_PASID */ +/* * External user API */
Besides looks good to me
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 05/15] vfio: Add PASID allocation/free support
Hi Eric,
Quoted 102 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 11:07 PMYi,
Quoted 50 lines · Liu Yi L
Shared Virtual Addressing (a.k.a Shared Virtual Memory) allows sharing multiple process virtual address spaces with the device for simplified programming model. PASID is used to tag an virtual address space in DMA requests and to identify the related translation structure in IOMMU. When a PASID-capable device is assigned to a VM, we want the same capability of using PASID to tag guest process virtual address spaces to achieve virtual SVA (vSVA).
PASID management for guest is vendor specific. Some vendors (e.g. Intel
VT-d) requires system-wide managed PASIDs across all devices, regardless of whether a device is used by host or assigned to guest. Other vendors (e.g. ARM SMMU) may allow PASIDs managed per-device thus could be fully delegated to the guest for assigned devices.For system-wide managed PASIDs, this patch introduces a vfio module to handle explicit PASID alloc/free requests from guest. Allocated PASIDs are associated to a process (or,
mm_struct) in IOASID core. Avfio_mmobject is introduced to trackmm_struct. Multiple VFIO containers within a process share the samevfio_mmobject.A quota mechanism is provided to prevent malicious user from exhausting available PASIDs. Currently the quota is a global parameter applied to all VFIO devices. In the future per-device quota might be supported too.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Suggested-by Alex Williamson <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) address comments from Eric. Addvfio_unlink_pasid()to be consistent
withvfio_unlink_dma(). Add a comment invfio_pasid_exit().v4 -> v5:
*) address comments from Eric Auger.
*) address the comments from Alex on the pasid free range support. Added
pervfio_mmpasid r-b tree.
https://lore.kernel.org/kvm/[email protected]/v3 -> v4:
*) fix lock leam invfio_mm_get_from_task()
*) droppasid_quotafield in structvfio_mm
*)vfio_mm_get_from_task()returnsERR_PTR(-ENOTTY)when !CONFIG_VFIO_PASIDQuoted 6 lines
v1 -> v2:
*) added in v2, split from the pasid alloc/free support of v1--- drivers/vfio/Kconfig | 5 + drivers/vfio/Makefile | 1 + drivers/vfio/vfio_pasid.c | 248++++++++++++++++++++++++++++++++++++++++++++++
Quoted 39 lines
include/linux/vfio.h | 28 ++++++ 4 files changed, 282 insertions(+) create mode 100644 drivers/vfio/vfio_pasid.cdrivers/vfio/Kconfig index renamed from drivers/vfio/Kconfig +0
-0fd17db9..3d8a108 100644
drivers/vfio/Kconfig +5
-0@@ -19,6 +19,11 @@ config VFIO_VIRQFD depends on VFIO && EVENTFD default n +config VFIO_PASID + tristate + depends on IOASID && VFIO + default n + menuconfig VFIO tristate "VFIO Non-Privileged userspace driver framework" depends on IOMMU_APIdrivers/vfio/Makefile index renamed from drivers/vfio/Makefile +0
-0de67c47..bb836a3 100644
drivers/vfio/Makefile +1
-0@@ -3,6 +3,7 @@ vfio_virqfd-y := virqfd.o obj-$(CONFIG_VFIO) += vfio.o obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o +obj-$(CONFIG_VFIO_PASID) += vfio_pasid.o obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o obj-$(CONFIG_VFIO_SPAPR_EEH) += vfio_spapr_eeh.o diff --git
a/drivers/vfio/vfio_pasid.cb/drivers/vfio/vfio_pasid.cnew file mode 100644 index 0000000..befcf29drivers/vfio/vfio_pasid.c new file +2
-0@@ -0,0 +1,248 @@ +// SPDX-License-Identifier: GPL-2.0-only +/*
Quoted 8 lines
drivers/vfio/vfio_pasid.c line 9
Quoted 7 lines
drivers/vfio/vfio_pasid.c new file +7
-0+* Copyright (C) 2020 Intel Corporation. +* Author: Liu Yi L <[email protected]> +* +*/ + +#include <linux/vfio.h> +#include <linux/eventfd.h> not needed
oh, yes.
Quoted 267 lines
Quoted 256 lines
drivers/vfio/vfio_pasid.c new file +228
-0+#include <linux/file.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/sched/mm.h> + +#define DRIVER_VERSION "0.1" +#define DRIVER_AUTHOR "Liu Yi L <[email protected]>" +#define DRIVER_DESC "PASID management for VFIO bus drivers" + +#define VFIO_DEFAULT_PASID_QUOTA 1000 +static int pasid_quota =VFIO_DEFAULT_PASID_QUOTA ; +module_param_named (pasid_quota, pasid_quota, uint,0444 ); +MODULE_PARM_DESC (pasid_quota, + "Set the quota for max number of PASIDs that an application is +allowed to request (default 1000)"); + +struct vfio_mm_token { +unsigned long long val; +}; + +struct vfio_mm { +struct kref kref; +int ioasid_sid; +struct mutex pasid_lock; +struct rb_root pasid_list; +struct list_head next; +struct vfio_mm_token token; +}; + +static struct mutex vfio_mm_lock; +static struct list_head vfio_mm_list; + +struct vfio_pasid { +struct rb_node node; +ioasid_t pasid; +}; + +static void vfio_remove_all_pasids (struct vfio_mm *vmm); + +/* called with vfio.vfio_mm_lock held */ static void +vfio_mm_release (struct kref *kref) { + struct vfio_mm *vmm = container_of(kref, struct vfio_mm, kref); + +list_del (&vmm->next); +mutex_unlock (&vfio_mm_lock); +vfio_remove_all_pasids (vmm); +ioasid_free_set (vmm->ioasid_sid, true); +kfree (vmm); +} + +void vfio_mm_put (struct vfio_mm *vmm) { +kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } + +static void vfio_mm_get (struct vfio_mm *vmm) { +kref_get (&vmm->kref); +} + +struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task) { +struct mm_struct *mm =get_task_mm (task); +struct vfio_mm *vmm; +unsigned long long val = (unsigned long long )mm; +int ret; + +mutex_lock (&vfio_mm_lock); +/* Search existing vfio_mm with current mm pointer */ +list_for_each_entry (vmm, &vfio_mm_list, next) { +if (vmm->token.val == val) { +vfio_mm_get (vmm); + goto out; + } + } + + vmm =kzalloc (sizeof (*vmm),GFP_KERNEL ); +if (!vmm) { + vmm =ERR_PTR (-ENOMEM ); + goto out; + } + +/* +* IOASID core provides a 'IOASID set' concept to track all +* PASIDs associated with a token. Here we use mm_struct as +* the token and create a IOASID set per mm_struct. All the +* containers of the process share the same IOASID set. +*/ + ret =ioasid_alloc_set ((struct ioasid_set *)mm, pasid_quota, + &vmm->ioasid_sid); +if (ret) { +kfree (vmm); + vmm =ERR_PTR (ret); + goto out; + } + +kref_init (&vmm->kref); + vmm->token.val = val; +mutex_init (&vmm->pasid_lock); + vmm->pasid_list =RB_ROOT ; + + list_add(&vmm->next, &vfio_mm_list);> +out: +mutex_unlock (&vfio_mm_lock); +mmput (mm); +return vmm; +} + +/* +* Find PASID within @min and @max +*/ +static struct vfio_pasid *vfio_find_pasid (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max) +{ +struct rb_node *node = vmm->pasid_list.rb_node; + +while (node) { +struct vfio_pasid *vid =rb_entry (node, + struct vfio_pasid, node); + +if (max < vid->pasid) + node = node->rb_left; +else if (min > vid->pasid) + node = node->rb_right; +else +return vid; + } + +return NULL ; +} + +static void vfio_link_pasid (struct vfio_mm *vmm,struct vfio_pasid +*new) { +struct rb_node **link = &vmm->pasid_list.rb_node, *parent =NULL ; +struct vfio_pasid *vid; + +while (*link) { + parent = *link; + vid = rb_entry(parent, struct vfio_pasid, node); + +if (new->pasid <= vid->pasid) + link = &(*link)->rb_left; +else + link = &(*link)->rb_right; + } + +rb_link_node (&new->node, parent, link); +rb_insert_color (&new->node, &vmm->pasid_list); } + +static void vfio_unlink_pasid (struct vfio_mm *vmm,struct vfio_pasid +*old) { +rb_erase (&old->node, &vmm->pasid_list); } + +static void vfio_remove_pasid (struct vfio_mm *vmm,struct vfio_pasid +*vid) { +vfio_unlink_pasid (vmm, vid); +ioasid_free (vid->pasid); +kfree (vid); +} + +static void vfio_remove_all_pasids (struct vfio_mm *vmm) { +struct rb_node *node; + +mutex_lock (&vmm->pasid_lock); +while ((node =rb_first (&vmm->pasid_list))) + vfio_remove_pasid(vmm, rb_entry(node, struct vfio_pasid, node)); +mutex_unlock (&vmm->pasid_lock); +} + +int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) { +ioasid_t pasid; +struct vfio_pasid *vid; + + pasid =ioasid_alloc (vmm->ioasid_sid, min, max,NULL ); +if (pasid ==INVALID_IOASID ) +return -ENOSPC ; + + vid =kzalloc (sizeof (*vid),GFP_KERNEL ); +if (!vid) { +ioasid_free (pasid); +return -ENOMEM ; + } + + vid->pasid = pasid; + +mutex_lock (&vmm->pasid_lock); +vfio_link_pasid (vmm, vid); +mutex_unlock (&vmm->pasid_lock); + +return pasid; +} + +void vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max) +{ +struct vfio_pasid *vid =NULL ; + +/* +* IOASID core will notify PASID users (e.g. IOMMU driver) to +* teardown necessary structures depending on the to-be-freed +* PASID. +*/ +mutex_lock (&vmm->pasid_lock); +while ((vid =vfio_find_pasid (vmm, min, max)) !=NULL ) +vfio_remove_pasid (vmm, vid); +mutex_unlock (&vmm->pasid_lock); +} + +static int __init vfio_pasid_init (void ) { +mutex_init (&vfio_mm_lock); +INIT_LIST_HEAD (&vfio_mm_list); +return 0 ; +} + +static void __exit vfio_pasid_exit(void) { +/* +* VFIO_PASID is supposed to be referenced by VFIO_IOMMU_TYPE1 +* and may be other module. once vfio_pasid_exit() is triggered, +* that means its user (e.g. VFIO_IOMMU_TYPE1) has been removed. +* All the vfio_mm instances should have been released. If not, +* means there is vfio_mm leak, should be a bug of user module. +* So just warn here. +*/ +WARN_ON (!list_empty (&vfio_mm_list)); +} + +module_init (vfio_pasid_init); +module_exit (vfio_pasid_exit); + +MODULE_VERSION (DRIVER_VERSION ); +MODULE_LICENSE ("GPL v2" ); +MODULE_AUTHOR (DRIVER_AUTHOR ); +MODULE_DESCRIPTION (DRIVER_DESC );include/linux/vfio.h index renamed from include/linux/vfio.h +0
-038d3c6a..31472a9 100644
include/linux/vfio.h +0
-0@@ -97,6 +97,34 @@ extern int vfio_register_iommu_driver(const struct
vfio_iommu_driver_ops*ops); extern void vfio_unregister_iommu_driver(include/linux/vfio.h +19
-0const struct vfio_iommu_driver_ops *ops); +struct vfio_mm ; +#if IS_ENABLED (CONFIG_VFIO_PASID ) +extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct +*task);extern void vfio_mm_put (struct vfio_mm *vmm);extern int +vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void +vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max); +#else +static inline struct vfio_mm *vfio_mm_get_from_task (struct +task_struct *task) { +return ERR_PTR (-ENOTTY ); +} + +static inline void vfio_mm_put (struct vfio_mm *vmm) { } + +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int +max) { +return -ENOTTY ; +}include/linux/vfio.h extern int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops); · line 130
Quoted 10 lines
include/linux/vfio.h +7
-0+ +static inline void vfio_pasid_free_range (struct vfio_mm *vmm, +ioasid_t min,ioasid_t max) +{ +} +#endif /* CONFIG_VFIO_PASID */ +/* * External user API */ Besides looks good to me
Quoted 1 line
Reviewed-by Eric Auger <[email protected]>
thanks. :-)
Regards,
Yi Liu
Quoted 3 lines
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 6/15 iommu/vt-d: Support setting ioasid set to domain
In reply to Liu Yi LFrom IOMMU p.o.v., PASIDs allocated and managed by external components (e.g. VFIO) will be passed in for gpasid_bind/unbind operation. IOMMU needs some knowledge to check the PASID ownership, hence add an interface for those components to tell the PASID owner.
In latest kernel design, PASID ownership is managed by IOASID set where the PASID is allocated from. This patch adds support for setting ioasid set ID to the domains used for nesting/vSVA. Subsequent SVA operations will check the PASID against its IOASID set for proper ownership.
---
v5 -> v6:
*) address comments against v5 from Eric Auger.
v4 -> v5:
*) address comments from Eric Auger.
--- drivers/iommu/intel/iommu.c | 23 +++++++++++++++++++++++ include/linux/intel-iommu.h | 4 ++++ include/linux/iommu.h | 1 + 3 files changed, 28 insertions(+)
drivers/iommu/intel/iommu.c +23 -0
@@ -1793,6 +1793,7 @@ static struct dmar_domain *alloc_domain(int flags)if (first_level_by_default ()) domain->flags |=DOMAIN_FLAG_USE_FIRST_LEVEL ; domain->has_iotlb_device = false; + domain->ioasid_sid =INVALID_IOASID_SET ;INIT_LIST_HEAD (&domain->devices);return domain;
@@ -6040,6 +6041,28 @@ intel_iommu_domain_set_attr(struct iommu_domain *domain, }spin_unlock_irqrestore (&device_domain_lock, flags);break ; +case DOMAIN_ATTR_IOASID_SID : + { +int sid = *(int *)data; + +spin_lock_irqsave (&device_domain_lock, flags); +if (!(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) { + ret = -ENODEV ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } +if (dmar_domain->ioasid_sid !=INVALID_IOASID_SET && + dmar_domain->ioasid_sid != sid) { +pr_warn_ratelimited ("multi ioasid_set (%d:%d) setting" , + dmar_domain->ioasid_sid, sid); + ret = -EBUSY ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } + dmar_domain->ioasid_sid = sid; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + }default : ret = -EINVAL ;break ;
include/linux/intel-iommu.h +4 -0
@@ -549,6 +549,10 @@ struct dmar_domain {2 == 1GiB, 3 == 512GiB, 4 == 1TiB */ u64 max_addr;/* maximum mapped address */ +int ioasid_sid;/* +* the ioasid set which tracks all +* PASIDs used by the domain. +*/ int default_pasid;/* * The default pasid used for non-SVM * traffic on mediated devices.
include/linux/iommu.h +1 -0
@@ -124,6 +124,7 @@ enum iommu_attr {DOMAIN_ATTR_FSL_PAMUV1 ,DOMAIN_ATTR_NESTING ,/* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE , +DOMAIN_ATTR_IOASID_SID , DOMAIN_ATTR_MAX, };
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 06/15] iommu/vt-d: Support setting ioasid set to domain
Hi Yi,
Quoted 59 lines · Liu Yi L
From IOMMU p.o.v., PASIDs allocated and managed by external components (e.g. VFIO) will be passed in for gpasid_bind/unbind operation. IOMMU needs some knowledge to check the PASID ownership, hence add an interface for those components to tell the PASID owner.
In latest kernel design, PASID ownership is managed by IOASID set where the PASID is allocated from. This patch adds support for setting ioasid set ID to the domains used for nesting/vSVA. Subsequent SVA operations will check the PASID against its IOASID set for proper ownership.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments against v5 from Eric Auger.v4 -> v5:
*) address comments from Eric Auger.--- drivers/iommu/intel/iommu.c | 23 +++++++++++++++++++++++ include/linux/intel-iommu.h | 4 ++++ include/linux/iommu.h | 1 + 3 files changed, 28 insertions(+)drivers/iommu/intel/iommu.c +12
-0@@ -1793,6 +1793,7 @@ static struct dmar_domain *alloc_domain(int flags)if (first_level_by_default ()) domain->flags |=DOMAIN_FLAG_USE_FIRST_LEVEL ; domain->has_iotlb_device = false; + domain->ioasid_sid =INVALID_IOASID_SET ;INIT_LIST_HEAD (&domain->devices);return domain;@@ -6040,6 +6041,28 @@ intel_iommu_domain_set_attr(struct iommu_domain *domain, }spin_unlock_irqrestore (&device_domain_lock, flags);break ; +case DOMAIN_ATTR_IOASID_SID : + { +int sid = *(int *)data; + +spin_lock_irqsave (&device_domain_lock, flags); +if (!(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) { + ret = -ENODEV ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } + if (dmar_domain->ioasid_sid != INVALID_IOASID_SET &&
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6065
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+ dmar_domain->ioasid_sid != sid) { +pr_warn_ratelimited ("multi ioasid_set (%d:%d) setting" , + dmar_domain->ioasid_sid, sid); + ret = -EBUSY ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } + dmar_domain->ioasid_sid = sid; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ;
nit: Adding a small helperint__set_ioasid_sid(struct dmar_domain *dmar_domain, int sid_id)
may simplify the lock handling
Quoted 9 lines
drivers/iommu/intel/iommu.c +1
-0+ }default : ret = -EINVAL ;drivers/iommu/intel/iommu.c +0
-0break ;include/linux/intel-iommu.h +0
-0
include/linux/intel-iommu.h struct dmar_domain { · line 553
Quoted 5 lines
include/linux/intel-iommu.h +2
-0@@ -549,6 +549,10 @@ struct dmar_domain {2 == 1GiB, 3 == 512GiB, 4 == 1TiB */ u64 max_addr;/* maximum mapped address */ +int ioasid_sid;/* +* the ioasid set which tracks all
id of the ioasid set?
Quoted 11 lines
include/linux/intel-iommu.h +2
-0+* PASIDs used by the domain. +*/ int default_pasid;/* * The default pasid used for non-SVM * traffic on mediated devices. include/linux/iommu.h +0
-0
include/linux/iommu.h enum iommu_attr { · line 130
Quoted 7 lines
include/linux/iommu.h +1
-0@@ -124,6 +124,7 @@ enum iommu_attr {DOMAIN_ATTR_FSL_PAMUV1 ,DOMAIN_ATTR_NESTING ,/* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE , +DOMAIN_ATTR_IOASID_SID , DOMAIN_ATTR_MAX, };
Besides
Eric
RE: [PATCH v6 06/15] iommu/vt-d: Support setting ioasid set to domain
Hi Eric,
Quoted 79 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 11:07 PMHi Yi,
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain
Quoted 44 lines · Liu Yi L
From IOMMU p.o.v., PASIDs allocated and managed by external components (e.g. VFIO) will be passed in for gpasid_bind/unbind operation. IOMMU needs some knowledge to check the PASID ownership, hence add an interface for those components to tell the PASID owner.
In latest kernel design, PASID ownership is managed by IOASID set where the PASID is allocated from. This patch adds support for setting ioasid set ID to the domains used for nesting/vSVA. Subsequent SVA operations will check the PASID against its IOASID set for proper ownership.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments against v5 from Eric Auger.v4 -> v5:
*) address comments from Eric Auger.--- drivers/iommu/intel/iommu.c | 23 +++++++++++++++++++++++ include/linux/intel-iommu.h | 4 ++++ include/linux/iommu.h | 1 + 3 files changed, 28 insertions(+)drivers/iommu/intel/iommu.c +1
-0@@ -1793,6 +1793,7 @@ static struct dmar_domain *alloc_domain(int flags)if (first_level_by_default ()) domain->flags |=DOMAIN_FLAG_USE_FIRST_LEVEL ; domain->has_iotlb_device = false; + domain->ioasid_sid =INVALID_IOASID_SET ;INIT_LIST_HEAD (&domain->devices);return domain;@@ -6040,6 +6041,28 @@ intel_iommu_domain_set_attr(struct iommu_domain*domain,
Quoted 15 lines
drivers/iommu/intel/iommu.c +11
-0}spin_unlock_irqrestore (&device_domain_lock, flags);break ; +case DOMAIN_ATTR_IOASID_SID : + { +int sid = *(int *)data; + +spin_lock_irqsave (&device_domain_lock, flags); +if (!(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) { + ret = -ENODEV ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } + if (dmar_domain->ioasid_sid != INVALID_IOASID_SET &&drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6064
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+ dmar_domain->ioasid_sid != sid) { +pr_warn_ratelimited ("multi ioasid_set (%d:%d) setting" , + dmar_domain->ioasid_sid, sid); + ret = -EBUSY ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } + dmar_domain->ioasid_sid = sid; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ;nit: Adding a small helper
int__set_ioasid_sid(struct dmar_domain *dmar_domain, int sid_id)
Quoted 1 line
may simplify the lock handling
ok. will do.
Quoted 13 lines
Quoted 13 lines
drivers/iommu/intel/iommu.c +1
-0+ }default : ret = -EINVAL ;drivers/iommu/intel/iommu.c +0
-0break ;include/linux/intel-iommu.h +0
-0@@ -549,6 +549,10 @@ struct dmar_domain {2 == 1GiB, 3 == 512GiB, 4 == 1TiB */ u64 max_addr;/* maximum mapped address */
Quoted 3 lines
include/linux/intel-iommu.h struct dmar_domain { · line 553
Quoted 2 lines
include/linux/intel-iommu.h +2
-0+int ioasid_sid;/* +* the ioasid set which tracks all id of the ioasid set?
should be ioasid_set. however, ioasid_alloc_set() returns sid in Jacob's series. but, I heard from Jacob, he will remove ioasid_sid, and return ioasid_set. so I will modify it once his patch is sent out.
https://lore.kernel.org/linux-iommu/[email protected]/
Quoted 19 lines
Quoted 11 lines
include/linux/intel-iommu.h +2
-0+* PASIDs used by the domain. +*/ int default_pasid;/* * The default pasid used for non-SVM * traffic on mediated devices. include/linux/iommu.h index renamed from include/linux/iommu.h +0
-04a02c9e..b1ff702 100644
include/linux/iommu.h enum iommu_attr { · line 130
Quoted 7 lines
include/linux/iommu.h +1
-0@@ -124,6 +124,7 @@ enum iommu_attr {DOMAIN_ATTR_FSL_PAMUV1 ,DOMAIN_ATTR_NESTING ,/* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE , +DOMAIN_ATTR_IOASID_SID , DOMAIN_ATTR_MAX, };Besides
Quoted 1 line
Reviewed-by Eric Auger <[email protected]>
thanks :-)
Regards,
Yi Liu
Quoted 2 lines
Eric
Re: [PATCH v6 06/15] iommu/vt-d: Support setting ioasid set to domain
Quoted 102 lines · Yi L
Hi Eric,
Quoted 79 lines
From: Auger Eric <[email protected]>
Sent: Thursday, August 13, 2020 11:07 PMHi Yi,
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain
Quoted 44 lines · Liu Yi L
From IOMMU p.o.v., PASIDs allocated and managed by external components (e.g. VFIO) will be passed in for gpasid_bind/unbind operation. IOMMU needs some knowledge to check the PASID ownership, hence add an interface for those components to tell the PASID owner.
In latest kernel design, PASID ownership is managed by IOASID set where the PASID is allocated from. This patch adds support for setting ioasid set ID to the domains used for nesting/vSVA. Subsequent SVA operations will check the PASID against its IOASID set for proper ownership.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments against v5 from Eric Auger.v4 -> v5:
*) address comments from Eric Auger.--- drivers/iommu/intel/iommu.c | 23 +++++++++++++++++++++++ include/linux/intel-iommu.h | 4 ++++ include/linux/iommu.h | 1 + 3 files changed, 28 insertions(+)drivers/iommu/intel/iommu.c +1
-0@@ -1793,6 +1793,7 @@ static struct dmar_domain *alloc_domain(int flags)if (first_level_by_default ()) domain->flags |=DOMAIN_FLAG_USE_FIRST_LEVEL ; domain->has_iotlb_device = false; + domain->ioasid_sid =INVALID_IOASID_SET ;INIT_LIST_HEAD (&domain->devices);return domain;@@ -6040,6 +6041,28 @@ intel_iommu_domain_set_attr(struct iommu_domain*domain,
Quoted 15 lines
drivers/iommu/intel/iommu.c +11
-0}spin_unlock_irqrestore (&device_domain_lock, flags);break ; +case DOMAIN_ATTR_IOASID_SID : + { +int sid = *(int *)data; + +spin_lock_irqsave (&device_domain_lock, flags); +if (!(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) { + ret = -ENODEV ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } + if (dmar_domain->ioasid_sid != INVALID_IOASID_SET &&drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6064
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+ dmar_domain->ioasid_sid != sid) { +pr_warn_ratelimited ("multi ioasid_set (%d:%d) setting" , + dmar_domain->ioasid_sid, sid); + ret = -EBUSY ; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ; + } + dmar_domain->ioasid_sid = sid; +spin_unlock_irqrestore (&device_domain_lock, flags); +break ;nit: Adding a small helper
int__set_ioasid_sid(struct dmar_domain *dmar_domain, int sid_id)Quoted 1 line
may simplify the lock handling
ok. will do.
Quoted 13 lines
Quoted 13 lines
drivers/iommu/intel/iommu.c +1
-0+ }default : ret = -EINVAL ;drivers/iommu/intel/iommu.c +0
-0break ;include/linux/intel-iommu.h +0
-0@@ -549,6 +549,10 @@ struct dmar_domain {2 == 1GiB, 3 == 512GiB, 4 == 1TiB */ u64 max_addr;/* maximum mapped address */ Quoted 3 lines
include/linux/intel-iommu.h struct dmar_domain { · line 553
Quoted 2 lines
include/linux/intel-iommu.h +2
-0+int ioasid_sid;/* +* the ioasid set which tracks all id of the ioasid set?
Quoted 5 lines
should be
ioasid_set. however,ioasid_alloc_set()returns sid in Jacob's series. but, I heard from Jacob, he will removeioasid_sid, and returnioasid_set. so I will modify it once his patch is sent out.
OK
Thanks
Eric
Quoted 30 lines
Quoted 20 lines
Quoted 12 lines
include/linux/intel-iommu.h +2
-0+* PASIDs used by the domain. +*/ int default_pasid;/* * The default pasid used for non-SVM * traffic on mediated devices. include/linux/iommu.h index renamed from include/linux/iommu.h +0
-04a02c9e..b1ff702 100644
include/linux/iommu.h enum iommu_attr { · line 130
Quoted 7 lines
include/linux/iommu.h +1
-0@@ -124,6 +124,7 @@ enum iommu_attr {DOMAIN_ATTR_FSL_PAMUV1 ,DOMAIN_ATTR_NESTING ,/* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE , +DOMAIN_ATTR_IOASID_SID , DOMAIN_ATTR_MAX, };Besides
Quoted 1 line
Reviewed-by Eric Auger <[email protected]>thanks :-)
Regards,
Yi LiuQuoted 2 lines
Eric
PATCH v6 7/15 vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free)
In reply to Liu Yi LThis patch allows userspace to request PASID allocation/free, e.g. when serving the request from the guest.
PASIDs that are not freed by userspace are automatically freed when the IOASID set is destroyed when process exits.
---
v5 -> v6:
*) address comments from Eric against v5. remove the alloc/free helper.
v4 -> v5:
*) address comments from Eric Auger.
*) the comments for the PASID_FREE request is addressed in patch 5/15 of
this series.
v3 -> v4:
*) address comments from v3, except the below comment against the range
of PASID_FREE request. needs more help on it.
"> +if (req.range.min > req.range.max)
Is it exploitable that a user can spin the kernel for a long time in the case of a free by calling this with [0, MAX_UINT] regardless of their actual allocations?"
https://lore.kernel.org/linux-iommu/[email protected]/
v1 -> v2:
*) move the vfio_mm related code to be a seprate module
*) use a single structure for alloc/free, could support a range of PASIDs *) fetch vfio_mm at group_attach time instead of at iommu driver open time
--- drivers/vfio/Kconfig | 1 + drivers/vfio/vfio_iommu_type1.c | 69 +++++++++++++++++++++++++++++++++++++++++ drivers/vfio/vfio_pasid.c | 10 ++++++ include/linux/vfio.h | 6 ++++ include/uapi/linux/vfio.h | 37 ++++++++++++++++++++++ 5 files changed, 123 insertions(+)
drivers/vfio/Kconfig +1 -0
@@ -2,6 +2,7 @@ config VFIO_IOMMU_TYPE1 tristate depends on VFIO + select VFIO_PASID if (X86) default n config VFIO_IOMMU_SPAPR_TCE
drivers/vfio/vfio_iommu_type1.c +69 -0
99 lines not on this page · raw message
drivers/vfio/vfio_pasid.c +10 -0
@@ -61,6 +61,7 @@ void vfio_mm_put(struct vfio_mm *vmm) {kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } +EXPORT_SYMBOL_GPL (vfio_mm_put);static void vfio_mm_get (struct vfio_mm *vmm) {
@@ -114,6 +115,13 @@ struct vfio_mm *vfio_mm_get_from_task(struct task_struct *task)mmput (mm);return vmm; } +EXPORT_SYMBOL_GPL (vfio_mm_get_from_task); + +int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return vmm->ioasid_sid; +} +EXPORT_SYMBOL_GPL (vfio_mm_ioasid_sid);/* * Find PASID within @min and @max
@@ -202,6 +210,7 @@ int vfio_pasid_alloc(struct vfio_mm *vmm, int min, int max)return pasid; } +EXPORT_SYMBOL_GPL (vfio_pasid_alloc);void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max)
@@ -218,6 +227,7 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,vfio_remove_pasid (vmm, vid);mutex_unlock (&vmm->pasid_lock); } +EXPORT_SYMBOL_GPL (vfio_pasid_free_range);static int __init vfio_pasid_init (void ) {
include/linux/vfio.h +6 -0
@@ -101,6 +101,7 @@ struct vfio_mm;#if IS_ENABLED (CONFIG_VFIO_PASID )extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task);extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_mm_ioasid_sid (struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max);
@@ -114,6 +115,11 @@ static inline void vfio_mm_put(struct vfio_mm *vmm) { } +static inline int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return -ENOTTY ; +} +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) {return -ENOTTY ;
include/uapi/linux/vfio.h +37 -0
@@ -1172,6 +1172,43 @@ struct vfio_iommu_type1_dirty_bitmap_get {#define VFIO_IOMMU_DIRTY_PAGES _IO(VFIO_TYPE, VFIO_BASE + 17) +/** +* VFIO_IOMMU_PASID_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 18, +* struct vfio_iommu_type1_pasid_request) +* +* PASID (Processor Address Space ID) is a PCIe concept for tagging +* address spaces in DMA requests. When system-wide PASID allocation +* is required by the underlying iommu driver (e.g. Intel VT-d), this +* provides an interface for userspace to request pasid alloc/free +* for its assigned devices. Userspace should check the availability +* of this API by checking VFIO_IOMMU_TYPE1_INFO_CAP_NESTING through +* VFIO_IOMMU_GET_INFO. +* +* @flags=VFIO_IOMMU_FLAG_ALLOC_PASID, allocate a single PASID within @range. +* @flags=VFIO_IOMMU_FLAG_FREE_PASID, free the PASIDs within @range. +* @range is [min, max], which means both @min and @max are inclusive. +* ALLOC_PASID and FREE_PASID are mutually exclusive. +* +* returns: allocated PASID value on success, -errno on failure for +* ALLOC_PASID; +* 0 for FREE_PASID operation; +*/ +struct vfio_iommu_type1_pasid_request { +__u32 argsz; +#define VFIO_IOMMU_FLAG_ALLOC_PASID (1 << 0) +#define VFIO_IOMMU_FLAG_FREE_PASID (1 << 1) +__u32 flags; +struct { +__u32 min; +__u32 max; + } range; +}; + +#define VFIO_PASID_REQUEST_MASK (VFIO_IOMMU_FLAG_ALLOC_PASID | \ + VFIO_IOMMU_FLAG_FREE_PASID) + +#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /*
Signature · 1 line
2.7.4
Re: [PATCH v6 07/15] vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free)
Yi,
Quoted 272 lines · Liu Yi L
This patch allows userspace to request PASID allocation/free, e.g. when serving the request from the guest.
PASIDs that are not freed by userspace are automatically freed when the IOASID set is destroyed when process exits.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments from Eric against v5. remove the alloc/free helper.v4 -> v5:
*) address comments from Eric Auger.
*) the comments for thePASID_FREErequest is addressed in patch 5/15 of
this series.v3 -> v4:
*) address comments from v3, except the below comment against the range
ofPASID_FREErequest. needs more help on it.
"> +if (req.range.min > req.range.max)Is it exploitable that a user can spin the kernel for a long time in the case of a free by calling this with [0,
MAX_UINT] regardless of their actual allocations?"
https://lore.kernel.org/linux-iommu/[email protected]/v1 -> v2:
*) move thevfio_mmrelated code to be a seprate module
*) use a single structure for alloc/free, could support a range of PASIDs *) fetchvfio_mmatgroup_attachtime instead of at iommu driver open time--- drivers/vfio/Kconfig | 1 + drivers/vfio/vfio_iommu_type1.c | 69 +++++++++++++++++++++++++++++++++++++++++ drivers/vfio/vfio_pasid.c | 10 ++++++ include/linux/vfio.h | 6 ++++ include/uapi/linux/vfio.h | 37 ++++++++++++++++++++++ 5 files changed, 123 insertions(+)drivers/vfio/Kconfig +1
-0@@ -2,6 +2,7 @@ config VFIO_IOMMU_TYPE1 tristate depends on VFIO + select VFIO_PASID if (X86) default n config VFIO_IOMMU_SPAPR_TCEdrivers/vfio/vfio_iommu_type1.c +69
-0@@ -76,6 +76,7 @@ struct vfio_iommu {bool dirty_page_tracking;bool pinned_page_dirty_scope;struct iommu_nesting_info *nesting_info; +struct vfio_mm *vmm; };struct vfio_domain {@@ -1937,6 +1938,11 @@ static void vfio_iommu_iova_insert_copy(struct vfio_iommu *iommu,static void vfio_iommu_release_nesting_info (struct vfio_iommu *iommu) { +if (iommu->vmm) { +vfio_mm_put (iommu->vmm); + iommu->vmm =NULL ; + } +kfree (iommu->nesting_info); iommu->nesting_info =NULL ; }@@ -2071,6 +2077,26 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, iommu->nesting_info);if (ret) goto out_detach; + +if (iommu->nesting_info->features & +IOMMU_NESTING_FEAT_SYSWIDE_PASID ) { +struct vfio_mm *vmm; +int sid; + + vmm =vfio_mm_get_from_task (current); +if (IS_ERR (vmm)) { + ret =PTR_ERR (vmm); + goto out_detach; + } + iommu->vmm = vmm; + + sid =vfio_mm_ioasid_sid (vmm); + ret =iommu_domain_set_attr (domain->domain, +DOMAIN_ATTR_IOASID_SID , + &sid); +if (ret) + goto out_detach; + } }/* Get aperture info */ @@ -2859,6 +2885,47 @@ static int vfio_iommu_type1_dirty_pages(struct vfio_iommu *iommu,return -EINVAL ; } +static int vfio_iommu_type1_pasid_request (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_pasid_request req; +unsigned long minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_pasid_request, range); + +if (copy_from_user (&req, (void __user *)arg, minsz)) +return -EFAULT ; + +if (req.argsz < minsz || (req.flags & ~VFIO_PASID_REQUEST_MASK )) +return -EINVAL ; + +if (req.range.min > req.range.max) +return -EINVAL ; + +mutex_lock (&iommu->lock); +if (!iommu->vmm) { +mutex_unlock (&iommu->lock); +return -EOPNOTSUPP ; + } + +switch (req.flags &VFIO_PASID_REQUEST_MASK ) { +case VFIO_IOMMU_FLAG_ALLOC_PASID : + ret =vfio_pasid_alloc (iommu->vmm, req.range.min, + req.range.max); +break ; +case VFIO_IOMMU_FLAG_FREE_PASID : +vfio_pasid_free_range (iommu->vmm, req.range.min, + req.range.max); + ret =0 ; +break ; +default : + ret = -EINVAL ; + } +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data,unsigned int cmd,unsigned long arg) {@@ -2875,6 +2942,8 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,return vfio_iommu_type1_unmap_dma (iommu, arg);case VFIO_IOMMU_DIRTY_PAGES :return vfio_iommu_type1_dirty_pages (iommu, arg); +case VFIO_IOMMU_PASID_REQUEST : +return vfio_iommu_type1_pasid_request (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +10
-0@@ -61,6 +61,7 @@ void vfio_mm_put(struct vfio_mm *vmm) {kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } +EXPORT_SYMBOL_GPL (vfio_mm_put);static void vfio_mm_get (struct vfio_mm *vmm) {@@ -114,6 +115,13 @@ struct vfio_mm *vfio_mm_get_from_task(struct task_struct *task)mmput (mm);return vmm; } +EXPORT_SYMBOL_GPL (vfio_mm_get_from_task); + +int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return vmm->ioasid_sid; +} +EXPORT_SYMBOL_GPL (vfio_mm_ioasid_sid);/* * Find PASID within @min and @max @@ -202,6 +210,7 @@ int vfio_pasid_alloc(struct vfio_mm *vmm, int min, int max)return pasid; } +EXPORT_SYMBOL_GPL (vfio_pasid_alloc);void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max)@@ -218,6 +227,7 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,vfio_remove_pasid (vmm, vid);mutex_unlock (&vmm->pasid_lock); } +EXPORT_SYMBOL_GPL (vfio_pasid_free_range);static int __init vfio_pasid_init (void ) {include/linux/vfio.h +6
-0@@ -101,6 +101,7 @@ struct vfio_mm;#if IS_ENABLED (CONFIG_VFIO_PASID )extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task);extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_mm_ioasid_sid (struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max);@@ -114,6 +115,11 @@ static inline void vfio_mm_put(struct vfio_mm *vmm) { } +static inline int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return -ENOTTY ; +} +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) {return -ENOTTY ;include/uapi/linux/vfio.h +30
-0@@ -1172,6 +1172,43 @@ struct vfio_iommu_type1_dirty_bitmap_get {#define VFIO_IOMMU_DIRTY_PAGES _IO(VFIO_TYPE, VFIO_BASE + 17) +/** +* VFIO_IOMMU_PASID_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 18, +* struct vfio_iommu_type1_pasid_request) +* +* PASID (Processor Address Space ID) is a PCIe concept for tagging +* address spaces in DMA requests. When system-wide PASID allocation +* is required by the underlying iommu driver (e.g. Intel VT-d), this +* provides an interface for userspace to request pasid alloc/free +* for its assigned devices. Userspace should check the availability +* of this API by checking VFIO_IOMMU_TYPE1_INFO_CAP_NESTING through +* VFIO_IOMMU_GET_INFO. +* +* @flags=VFIO_IOMMU_FLAG_ALLOC_PASID, allocate a single PASID within @range. +* @flags=VFIO_IOMMU_FLAG_FREE_PASID, free the PASIDs within @range. +* @range is [min, max], which means both @min and @max are inclusive. +* ALLOC_PASID and FREE_PASID are mutually exclusive. +* +* returns: allocated PASID value on success, -errno on failure for +* ALLOC_PASID; +* 0 for FREE_PASID operation; +*/ +struct vfio_iommu_type1_pasid_request { +__u32 argsz; +#define VFIO_IOMMU_FLAG_ALLOC_PASID (1 << 0) +#define VFIO_IOMMU_FLAG_FREE_PASID (1 << 1) +__u32 flags; +struct { +__u32 min; +__u32 max; + } range;
include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1214
Quoted 10 lines
include/uapi/linux/vfio.h +7
-0+}; + +#define VFIO_PASID_REQUEST_MASK (VFIO_IOMMU_FLAG_ALLOC_PASID | \ + VFIO_IOMMU_FLAG_FREE_PASID) + +#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /*
Looks OK to me
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 07/15] vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free)
Thanks, Eric.
Regards,
Yi Liu
Quoted 297 lines
From: Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 12:30 AMYi,
Quoted 42 lines · Liu Yi L
This patch allows userspace to request PASID allocation/free, e.g. when serving the request from the guest.
PASIDs that are not freed by userspace are automatically freed when the IOASID set is destroyed when process exits.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments from Eric against v5. remove the alloc/free helper.v4 -> v5:
*) address comments from Eric Auger.
*) the comments for thePASID_FREErequest is addressed in patch 5/15 of
this series.v3 -> v4:
*) address comments from v3, except the below comment against the range
ofPASID_FREErequest. needs more help on it.
"> +if (req.range.min > req.range.max)Is it exploitable that a user can spin the kernel for a long time in the case of a free by calling this with [0,
MAX_UINT] regardless of their actual allocations?"https://lore.kernel.org/linux-iommu/[email protected]/
v1 -> v2:
*) move thevfio_mmrelated code to be a seprate module
*) use a single structure for alloc/free, could support a range of PASIDs
*) fetchvfio_mmatgroup_attachtime instead of at iommu driver open timeQuoted 3 lines
--- drivers/vfio/Kconfig | 1 + drivers/vfio/vfio_iommu_type1.c | 69+++++++++++++++++++++++++++++++++++++++++
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 43 lines
drivers/vfio/vfio_pasid.c | 10 ++++++ include/linux/vfio.h | 6 ++++ include/uapi/linux/vfio.h | 37 ++++++++++++++++++++++ 5 files changed, 123 insertions(+)drivers/vfio/Kconfig index renamed from drivers/vfio/Kconfig +0
-03d8a108..95d90c6 100644
drivers/vfio/Kconfig +1
-0@@ -2,6 +2,7 @@ config VFIO_IOMMU_TYPE1 tristate depends on VFIO + select VFIO_PASID if (X86) default n config VFIO_IOMMU_SPAPR_TCE(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex 18ff0c3..ea89c7c 100644drivers/vfio/vfio_iommu_type1.c +6
-0@@ -76,6 +76,7 @@ struct vfio_iommu {bool dirty_page_tracking;bool pinned_page_dirty_scope;struct iommu_nesting_info *nesting_info; +struct vfio_mm *vmm; };struct vfio_domain {@@ -1937,6 +1938,11 @@ static void vfio_iommu_iova_insert_copy(structvfio_iommu *iommu, static void vfio_iommu_release_nesting_info(struct vfio_iommu *iommu) { + if (iommu->vmm) { + vfio_mm_put(iommu->vmm); + iommu->vmm =NULL ; + } +kfree (iommu->nesting_info); iommu->nesting_info =NULL ;}
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2071,6 +2077,26 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data,
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +3
-0iommu->nesting_info);if (ret) goto out_detach; + +if (iommu->nesting_info->features & +IOMMU_NESTING_FEAT_SYSWIDE_PASID ){
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct
Quoted 21 lines
drivers/vfio/vfio_iommu_type1.c +17
-0+struct vfio_mm *vmm; +int sid; + + vmm =vfio_mm_get_from_task (current); +if (IS_ERR (vmm)) { + ret =PTR_ERR (vmm); + goto out_detach; + } + iommu->vmm = vmm; + + sid =vfio_mm_ioasid_sid (vmm); + ret =iommu_domain_set_attr (domain->domain, +DOMAIN_ATTR_IOASID_SID , + &sid); +if (ret) + goto out_detach; + } }/* Get aperture info */ @@ -2859,6 +2885,47 @@ static int vfio_iommu_type1_dirty_pages(struct
vfio_iommu*iommu,Quoted 36 lines
drivers/vfio/vfio_iommu_type1.c +33
-0return -EINVAL ; } +static int vfio_iommu_type1_pasid_request (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_pasid_request req; +unsigned long minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_pasid_request, range); + +if (copy_from_user (&req, (void __user *)arg, minsz)) +return -EFAULT ; + +if (req.argsz < minsz || (req.flags & ~VFIO_PASID_REQUEST_MASK )) +return -EINVAL ; + +if (req.range.min > req.range.max) +return -EINVAL ; + +mutex_lock (&iommu->lock); +if (!iommu->vmm) { +mutex_unlock (&iommu->lock); +return -EOPNOTSUPP ; + } + +switch (req.flags &VFIO_PASID_REQUEST_MASK ) { +case VFIO_IOMMU_FLAG_ALLOC_PASID : + ret =vfio_pasid_alloc (iommu->vmm, req.range.min, + req.range.max); +break ; +case VFIO_IOMMU_FLAG_FREE_PASID : +vfio_pasid_free_range (iommu->vmm, req.range.min, + req.range.max); + ret =0 ;drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct vfio_iommu *iommu,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+break ; +default : + ret = -EINVAL ; + } +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data, unsigned int cmd, unsigned long arg) { @@ -2875,6 +2942,8 @@
drivers/vfio/vfio_pasid.c struct vfio_mm *vfio_mm_get_from_task(struct
Quoted 20 lines
drivers/vfio/vfio_iommu_type1.c +0
-0static long vfio_iommu_type1_ioctl(void *iommu_data,drivers/vfio/vfio_iommu_type1.c +2
-0return vfio_iommu_type1_unmap_dma (iommu, arg);case VFIO_IOMMU_DIRTY_PAGES :return vfio_iommu_type1_dirty_pages (iommu, arg); +case VFIO_IOMMU_PASID_REQUEST : +return vfio_iommu_type1_pasid_request (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +1
-0@@ -61,6 +61,7 @@ void vfio_mm_put(struct vfio_mm *vmm) {kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } +EXPORT_SYMBOL_GPL (vfio_mm_put);static void vfio_mm_get (struct vfio_mm *vmm) {@@ -114,6 +115,13 @@ struct vfio_mm *vfio_mm_get_from_task(struct
task_struct*task)Quoted 64 lines
drivers/vfio/vfio_pasid.c +9
-0mmput (mm);return vmm; } +EXPORT_SYMBOL_GPL (vfio_mm_get_from_task); + +int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return vmm->ioasid_sid; +} +EXPORT_SYMBOL_GPL (vfio_mm_ioasid_sid);/* * Find PASID within @min and @max @@ -202,6 +210,7 @@ int vfio_pasid_alloc(struct vfio_mm *vmm, int min, int max)return pasid; } +EXPORT_SYMBOL_GPL (vfio_pasid_alloc);void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max)@@ -218,6 +227,7 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,vfio_remove_pasid (vmm, vid);mutex_unlock (&vmm->pasid_lock); } +EXPORT_SYMBOL_GPL (vfio_pasid_free_range);static int __init vfio_pasid_init (void ) {include/linux/vfio.h +6
-0@@ -101,6 +101,7 @@ struct vfio_mm;#if IS_ENABLED (CONFIG_VFIO_PASID )extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task);extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_mm_ioasid_sid (struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max);@@ -114,6 +115,11 @@ static inline void vfio_mm_put(struct vfio_mm *vmm) { } +static inline int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return -ENOTTY ; +} +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) {return -ENOTTY ;include/uapi/linux/vfio.h +3
-0@@ -1172,6 +1172,43 @@ struct vfio_iommu_type1_dirty_bitmap_get {#define VFIO_IOMMU_DIRTY_PAGES _IO(VFIO_TYPE, VFIO_BASE + 17) +/** +* VFIO_IOMMU_PASID_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 18, +* struct vfio_iommu_type1_pasid_request) include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1187
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+* +* PASID (Processor Address Space ID) is a PCIe concept for tagging +* address spaces in DMA requests. When system-wide PASID allocation +* is required by the underlying iommu driver (e.g. Intel VT-d), this +* provides an interface for userspace to request pasid alloc/free +* for its assigned devices. Userspace should check the availability +* of this API by checking VFIO_IOMMU_TYPE1_INFO_CAP_NESTING through +* VFIO_IOMMU_GET_INFO. +* +* @flags=VFIO_IOMMU_FLAG_ALLOC_PASID, allocate a single PASID within @range.
Quoted 18 lines
include/uapi/linux/vfio.h +17
-0+* @flags=VFIO_IOMMU_FLAG_FREE_PASID, free the PASIDs within @range. +* @range is [min, max], which means both @min and @max are inclusive. +* ALLOC_PASID and FREE_PASID are mutually exclusive. +* +* returns: allocated PASID value on success, -errno on failure for +* ALLOC_PASID; +* 0 for FREE_PASID operation; +*/ +struct vfio_iommu_type1_pasid_request { +__u32 argsz; +#define VFIO_IOMMU_FLAG_ALLOC_PASID (1 << 0) +#define VFIO_IOMMU_FLAG_FREE_PASID (1 << 1) +__u32 flags; +struct { +__u32 min; +__u32 max; + } range;include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1214
Quoted 10 lines
include/uapi/linux/vfio.h +7
-0+}; + +#define VFIO_PASID_REQUEST_MASK (VFIO_IOMMU_FLAG_ALLOC_PASID | \ + VFIO_IOMMU_FLAG_FREE_PASID) + +#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /* Looks OK to me
Quoted 6 lines
Reviewed-by Eric Auger <[email protected]>Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 07/15] vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free)
In reply to Liu Yi LQuoted 262 lines · On Mon, 27 Jul 2020 23:27:36 -0700 Liu Yi L <[email protected]> wrote:
This patch allows userspace to request PASID allocation/free, e.g. when serving the request from the guest.
PASIDs that are not freed by userspace are automatically freed when the IOASID set is destroyed when process exits.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments from Eric against v5. remove the alloc/free helper.v4 -> v5:
*) address comments from Eric Auger.
*) the comments for thePASID_FREErequest is addressed in patch 5/15 of
this series.v3 -> v4:
*) address comments from v3, except the below comment against the range
ofPASID_FREErequest. needs more help on it.
"> +if (req.range.min > req.range.max)Is it exploitable that a user can spin the kernel for a long time in the case of a free by calling this with [0,
MAX_UINT] regardless of their actual allocations?"
https://lore.kernel.org/linux-iommu/[email protected]/v1 -> v2:
*) move thevfio_mmrelated code to be a seprate module
*) use a single structure for alloc/free, could support a range of PASIDs *) fetchvfio_mmatgroup_attachtime instead of at iommu driver open time--- drivers/vfio/Kconfig | 1 + drivers/vfio/vfio_iommu_type1.c | 69 +++++++++++++++++++++++++++++++++++++++++ drivers/vfio/vfio_pasid.c | 10 ++++++ include/linux/vfio.h | 6 ++++ include/uapi/linux/vfio.h | 37 ++++++++++++++++++++++ 5 files changed, 123 insertions(+)drivers/vfio/Kconfig +1
-0@@ -2,6 +2,7 @@ config VFIO_IOMMU_TYPE1 tristate depends on VFIO + select VFIO_PASID if (X86) default n config VFIO_IOMMU_SPAPR_TCEdrivers/vfio/vfio_iommu_type1.c +69
-0@@ -76,6 +76,7 @@ struct vfio_iommu {bool dirty_page_tracking;bool pinned_page_dirty_scope;struct iommu_nesting_info *nesting_info; +struct vfio_mm *vmm; };struct vfio_domain {@@ -1937,6 +1938,11 @@ static void vfio_iommu_iova_insert_copy(struct vfio_iommu *iommu,static void vfio_iommu_release_nesting_info (struct vfio_iommu *iommu) { +if (iommu->vmm) { +vfio_mm_put (iommu->vmm); + iommu->vmm =NULL ; + } +kfree (iommu->nesting_info); iommu->nesting_info =NULL ; }@@ -2071,6 +2077,26 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, iommu->nesting_info);if (ret) goto out_detach; + +if (iommu->nesting_info->features & +IOMMU_NESTING_FEAT_SYSWIDE_PASID ) { +struct vfio_mm *vmm; +int sid; + + vmm =vfio_mm_get_from_task (current); +if (IS_ERR (vmm)) { + ret =PTR_ERR (vmm); + goto out_detach; + } + iommu->vmm = vmm; + + sid =vfio_mm_ioasid_sid (vmm); + ret =iommu_domain_set_attr (domain->domain, +DOMAIN_ATTR_IOASID_SID , + &sid); +if (ret) + goto out_detach; + } }/* Get aperture info */ @@ -2859,6 +2885,47 @@ static int vfio_iommu_type1_dirty_pages(struct vfio_iommu *iommu,return -EINVAL ; } +static int vfio_iommu_type1_pasid_request (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_pasid_request req; +unsigned long minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_pasid_request, range); + +if (copy_from_user (&req, (void __user *)arg, minsz)) +return -EFAULT ; + +if (req.argsz < minsz || (req.flags & ~VFIO_PASID_REQUEST_MASK )) +return -EINVAL ; + +if (req.range.min > req.range.max) +return -EINVAL ; + +mutex_lock (&iommu->lock); +if (!iommu->vmm) { +mutex_unlock (&iommu->lock); +return -EOPNOTSUPP ; + } + +switch (req.flags &VFIO_PASID_REQUEST_MASK ) { +case VFIO_IOMMU_FLAG_ALLOC_PASID : + ret =vfio_pasid_alloc (iommu->vmm, req.range.min, + req.range.max); +break ; +case VFIO_IOMMU_FLAG_FREE_PASID : +vfio_pasid_free_range (iommu->vmm, req.range.min, + req.range.max); + ret =0 ; +break ; +default : + ret = -EINVAL ; + } +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data,unsigned int cmd,unsigned long arg) {@@ -2875,6 +2942,8 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,return vfio_iommu_type1_unmap_dma (iommu, arg);case VFIO_IOMMU_DIRTY_PAGES :return vfio_iommu_type1_dirty_pages (iommu, arg); +case VFIO_IOMMU_PASID_REQUEST : +return vfio_iommu_type1_pasid_request (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +10
-0@@ -61,6 +61,7 @@ void vfio_mm_put(struct vfio_mm *vmm) {kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } +EXPORT_SYMBOL_GPL (vfio_mm_put);static void vfio_mm_get (struct vfio_mm *vmm) {@@ -114,6 +115,13 @@ struct vfio_mm *vfio_mm_get_from_task(struct task_struct *task)mmput (mm);return vmm; } +EXPORT_SYMBOL_GPL (vfio_mm_get_from_task); + +int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return vmm->ioasid_sid; +} +EXPORT_SYMBOL_GPL (vfio_mm_ioasid_sid);/* * Find PASID within @min and @max @@ -202,6 +210,7 @@ int vfio_pasid_alloc(struct vfio_mm *vmm, int min, int max)return pasid; } +EXPORT_SYMBOL_GPL (vfio_pasid_alloc);void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max)@@ -218,6 +227,7 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,vfio_remove_pasid (vmm, vid);mutex_unlock (&vmm->pasid_lock); } +EXPORT_SYMBOL_GPL (vfio_pasid_free_range);static int __init vfio_pasid_init (void ) {include/linux/vfio.h +6
-0@@ -101,6 +101,7 @@ struct vfio_mm;#if IS_ENABLED (CONFIG_VFIO_PASID )extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task);extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_mm_ioasid_sid (struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max);@@ -114,6 +115,11 @@ static inline void vfio_mm_put(struct vfio_mm *vmm) { } +static inline int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return -ENOTTY ; +} +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) {return -ENOTTY ;include/uapi/linux/vfio.h +21
-0@@ -1172,6 +1172,43 @@ struct vfio_iommu_type1_dirty_bitmap_get {#define VFIO_IOMMU_DIRTY_PAGES _IO(VFIO_TYPE, VFIO_BASE + 17) +/** +* VFIO_IOMMU_PASID_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 18, +* struct vfio_iommu_type1_pasid_request) +* +* PASID (Processor Address Space ID) is a PCIe concept for tagging +* address spaces in DMA requests. When system-wide PASID allocation +* is required by the underlying iommu driver (e.g. Intel VT-d), this +* provides an interface for userspace to request pasid alloc/free +* for its assigned devices. Userspace should check the availability +* of this API by checking VFIO_IOMMU_TYPE1_INFO_CAP_NESTING through +* VFIO_IOMMU_GET_INFO. +* +* @flags=VFIO_IOMMU_FLAG_ALLOC_PASID, allocate a single PASID within @range. +* @flags=VFIO_IOMMU_FLAG_FREE_PASID, free the PASIDs within @range. +* @range is [min, max], which means both @min and @max are inclusive. +* ALLOC_PASID and FREE_PASID are mutually exclusive. +* +* returns: allocated PASID value on success, -errno on failure for +* ALLOC_PASID; +* 0 for FREE_PASID operation; +*/
include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1205
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+struct vfio_iommu_type1_pasid_request { +__u32 argsz; +#define VFIO_IOMMU_FLAG_ALLOC_PASID (1 << 0) +#define VFIO_IOMMU_FLAG_FREE_PASID (1 << 1) +__u32 flags; +struct { +__u32 min; +__u32 max; + } range; +};
IOCTL(2) Linux Programmer's Manual IOCTL(2)
NAME
ioctl - control device
SYNOPSIS
#include <sys/ioctl.h>
int ioctl(int fd, unsigned long request, ...);ioctl(2) returns a signed int, how can it support returning a __u32 pasid and -errno? Thanks,
Alex
include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1214
Quoted 9 lines
include/uapi/linux/vfio.h +6
-0+ +#define VFIO_PASID_REQUEST_MASK (VFIO_IOMMU_FLAG_ALLOC_PASID | \ + VFIO_IOMMU_FLAG_FREE_PASID) + +#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /*
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 07/15] vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free)
Hi Alex,
Quoted 293 lines
From: Alex Williamson <[email protected]>
Sent: Friday, August 21, 2020 4:51 AMQuoted 42 lines · On Mon, 27 Jul 2020 23:27:36 -0700 Liu Yi L <[email protected]> wrote:
This patch allows userspace to request PASID allocation/free, e.g. when serving the request from the guest.
PASIDs that are not freed by userspace are automatically freed when the IOASID set is destroyed when process exits.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments from Eric against v5. remove the alloc/free helper.v4 -> v5:
*) address comments from Eric Auger.
*) the comments for thePASID_FREErequest is addressed in patch 5/15 of
this series.v3 -> v4:
*) address comments from v3, except the below comment against the range
ofPASID_FREErequest. needs more help on it.
"> +if (req.range.min > req.range.max)Is it exploitable that a user can spin the kernel for a long time in the case of a free by calling this with [0,
MAX_UINT] regardless of their actual allocations?"https://lore.kernel.org/linux-iommu/[email protected]/
v1 -> v2:
*) move thevfio_mmrelated code to be a seprate module
*) use a single structure for alloc/free, could support a range of PASIDs
*) fetchvfio_mmatgroup_attachtime instead of at iommu driver open timeQuoted 3 lines
--- drivers/vfio/Kconfig | 1 + drivers/vfio/vfio_iommu_type1.c | 69+++++++++++++++++++++++++++++++++++++++++
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 43 lines
drivers/vfio/vfio_pasid.c | 10 ++++++ include/linux/vfio.h | 6 ++++ include/uapi/linux/vfio.h | 37 ++++++++++++++++++++++ 5 files changed, 123 insertions(+)drivers/vfio/Kconfig index renamed from drivers/vfio/Kconfig +0
-03d8a108..95d90c6 100644
drivers/vfio/Kconfig +1
-0@@ -2,6 +2,7 @@ config VFIO_IOMMU_TYPE1 tristate depends on VFIO + select VFIO_PASID if (X86) default n config VFIO_IOMMU_SPAPR_TCE(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex 18ff0c3..ea89c7c 100644drivers/vfio/vfio_iommu_type1.c +6
-0@@ -76,6 +76,7 @@ struct vfio_iommu {bool dirty_page_tracking;bool pinned_page_dirty_scope;struct iommu_nesting_info *nesting_info; +struct vfio_mm *vmm; };struct vfio_domain {@@ -1937,6 +1938,11 @@ static void vfio_iommu_iova_insert_copy(structvfio_iommu *iommu, static void vfio_iommu_release_nesting_info(struct vfio_iommu *iommu) { + if (iommu->vmm) { + vfio_mm_put(iommu->vmm); + iommu->vmm =NULL ; + } +kfree (iommu->nesting_info); iommu->nesting_info =NULL ;}
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2071,6 +2077,26 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data,
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +3
-0iommu->nesting_info);if (ret) goto out_detach; + +if (iommu->nesting_info->features & +IOMMU_NESTING_FEAT_SYSWIDE_PASID ){
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct
Quoted 21 lines
drivers/vfio/vfio_iommu_type1.c +17
-0+struct vfio_mm *vmm; +int sid; + + vmm =vfio_mm_get_from_task (current); +if (IS_ERR (vmm)) { + ret =PTR_ERR (vmm); + goto out_detach; + } + iommu->vmm = vmm; + + sid =vfio_mm_ioasid_sid (vmm); + ret =iommu_domain_set_attr (domain->domain, +DOMAIN_ATTR_IOASID_SID , + &sid); +if (ret) + goto out_detach; + } }/* Get aperture info */ @@ -2859,6 +2885,47 @@ static int vfio_iommu_type1_dirty_pages(struct
vfio_iommu*iommu,Quoted 36 lines
drivers/vfio/vfio_iommu_type1.c +33
-0return -EINVAL ; } +static int vfio_iommu_type1_pasid_request (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_pasid_request req; +unsigned long minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_pasid_request, range); + +if (copy_from_user (&req, (void __user *)arg, minsz)) +return -EFAULT ; + +if (req.argsz < minsz || (req.flags & ~VFIO_PASID_REQUEST_MASK )) +return -EINVAL ; + +if (req.range.min > req.range.max) +return -EINVAL ; + +mutex_lock (&iommu->lock); +if (!iommu->vmm) { +mutex_unlock (&iommu->lock); +return -EOPNOTSUPP ; + } + +switch (req.flags &VFIO_PASID_REQUEST_MASK ) { +case VFIO_IOMMU_FLAG_ALLOC_PASID : + ret =vfio_pasid_alloc (iommu->vmm, req.range.min, + req.range.max); +break ; +case VFIO_IOMMU_FLAG_FREE_PASID : +vfio_pasid_free_range (iommu->vmm, req.range.min, + req.range.max); + ret =0 ;drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct vfio_iommu *iommu,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+break ; +default : + ret = -EINVAL ; + } +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data, unsigned int cmd, unsigned long arg) { @@ -2875,6 +2942,8 @@
drivers/vfio/vfio_pasid.c struct vfio_mm *vfio_mm_get_from_task(struct
Quoted 20 lines
drivers/vfio/vfio_iommu_type1.c +0
-0static long vfio_iommu_type1_ioctl(void *iommu_data,drivers/vfio/vfio_iommu_type1.c +2
-0return vfio_iommu_type1_unmap_dma (iommu, arg);case VFIO_IOMMU_DIRTY_PAGES :return vfio_iommu_type1_dirty_pages (iommu, arg); +case VFIO_IOMMU_PASID_REQUEST : +return vfio_iommu_type1_pasid_request (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +1
-0@@ -61,6 +61,7 @@ void vfio_mm_put(struct vfio_mm *vmm) {kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } +EXPORT_SYMBOL_GPL (vfio_mm_put);static void vfio_mm_get (struct vfio_mm *vmm) {@@ -114,6 +115,13 @@ struct vfio_mm *vfio_mm_get_from_task(struct
task_struct*task)Quoted 64 lines
drivers/vfio/vfio_pasid.c +9
-0mmput (mm);return vmm; } +EXPORT_SYMBOL_GPL (vfio_mm_get_from_task); + +int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return vmm->ioasid_sid; +} +EXPORT_SYMBOL_GPL (vfio_mm_ioasid_sid);/* * Find PASID within @min and @max @@ -202,6 +210,7 @@ int vfio_pasid_alloc(struct vfio_mm *vmm, int min, int max)return pasid; } +EXPORT_SYMBOL_GPL (vfio_pasid_alloc);void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max)@@ -218,6 +227,7 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,vfio_remove_pasid (vmm, vid);mutex_unlock (&vmm->pasid_lock); } +EXPORT_SYMBOL_GPL (vfio_pasid_free_range);static int __init vfio_pasid_init (void ) {include/linux/vfio.h +6
-0@@ -101,6 +101,7 @@ struct vfio_mm;#if IS_ENABLED (CONFIG_VFIO_PASID )extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task);extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_mm_ioasid_sid (struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max);@@ -114,6 +115,11 @@ static inline void vfio_mm_put(struct vfio_mm *vmm) { } +static inline int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return -ENOTTY ; +} +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) {return -ENOTTY ;include/uapi/linux/vfio.h +3
-0@@ -1172,6 +1172,43 @@ struct vfio_iommu_type1_dirty_bitmap_get {#define VFIO_IOMMU_DIRTY_PAGES _IO(VFIO_TYPE, VFIO_BASE + 17) +/** +* VFIO_IOMMU_PASID_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 18, +* struct vfio_iommu_type1_pasid_request) include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1187
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+* +* PASID (Processor Address Space ID) is a PCIe concept for tagging +* address spaces in DMA requests. When system-wide PASID allocation +* is required by the underlying iommu driver (e.g. Intel VT-d), this +* provides an interface for userspace to request pasid alloc/free +* for its assigned devices. Userspace should check the availability +* of this API by checking VFIO_IOMMU_TYPE1_INFO_CAP_NESTING through +* VFIO_IOMMU_GET_INFO. +* +* @flags=VFIO_IOMMU_FLAG_ALLOC_PASID, allocate a single PASID within @range.
Quoted 8 lines
include/uapi/linux/vfio.h +8
-0+* @flags=VFIO_IOMMU_FLAG_FREE_PASID, free the PASIDs within @range. +* @range is [min, max], which means both @min and @max are inclusive. +* ALLOC_PASID and FREE_PASID are mutually exclusive. +* +* returns: allocated PASID value on success, -errno on failure for +* ALLOC_PASID; +* 0 for FREE_PASID operation; +*/ include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1205
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+struct vfio_iommu_type1_pasid_request { +__u32 argsz; +#define VFIO_IOMMU_FLAG_ALLOC_PASID (1 << 0) +#define VFIO_IOMMU_FLAG_FREE_PASID (1 << 1) +__u32 flags; +struct { +__u32 min; +__u32 max; + } range; +};IOCTL(2) Linux Programmer's Manual IOCTL(2)
NAME
ioctl - control device
Quoted 8 lines
SYNOPSIS
#include <sys/ioctl.h> int ioctl(int fd, unsigned long request, ...);ioctl(2) returns a signed int, how can it support returning a
__u32pasid and -errno? Thanks,
yeah, pasid is 20 bits today per PCI spec, so a valid pasid won't use full 32 bits, so I used it. perhaps I need to add a field in the data struct to return allocated pasid. that may be safer. do you think it works?
Regards,
Yi Liu
Quoted 11 lines
Alex
Quoted 9 lines
include/uapi/linux/vfio.h +6
-0+ +#define VFIO_PASID_REQUEST_MASK (VFIO_IOMMU_FLAG_ALLOC_PASID | \ + VFIO_IOMMU_FLAG_FREE_PASID) + +#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /*
Re: [PATCH v6 07/15] vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free)
Quoted 301 lines · Yi L
Hi Alex,
Quoted 299 lines
From: Alex Williamson <[email protected]>
Sent: Friday, August 21, 2020 4:51 AMQuoted 42 lines · On Mon, 27 Jul 2020 23:27:36 -0700 Liu Yi L <[email protected]> wrote:
This patch allows userspace to request PASID allocation/free, e.g. when serving the request from the guest.
PASIDs that are not freed by userspace are automatically freed when the IOASID set is destroyed when process exits.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments from Eric against v5. remove the alloc/free helper.v4 -> v5:
*) address comments from Eric Auger.
*) the comments for thePASID_FREErequest is addressed in patch 5/15 of
this series.v3 -> v4:
*) address comments from v3, except the below comment against the range
ofPASID_FREErequest. needs more help on it.
"> +if (req.range.min > req.range.max)Is it exploitable that a user can spin the kernel for a long time in the case of a free by calling this with [0,
MAX_UINT] regardless of their actual allocations?"https://lore.kernel.org/linux-iommu/[email protected]/
v1 -> v2:
*) move thevfio_mmrelated code to be a seprate module
*) use a single structure for alloc/free, could support a range of PASIDs
*) fetchvfio_mmatgroup_attachtime instead of at iommu driver open timeQuoted 3 lines
--- drivers/vfio/Kconfig | 1 + drivers/vfio/vfio_iommu_type1.c | 69+++++++++++++++++++++++++++++++++++++++++
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 43 lines
drivers/vfio/vfio_pasid.c | 10 ++++++ include/linux/vfio.h | 6 ++++ include/uapi/linux/vfio.h | 37 ++++++++++++++++++++++ 5 files changed, 123 insertions(+)drivers/vfio/Kconfig index renamed from drivers/vfio/Kconfig +0
-03d8a108..95d90c6 100644
drivers/vfio/Kconfig +1
-0@@ -2,6 +2,7 @@ config VFIO_IOMMU_TYPE1 tristate depends on VFIO + select VFIO_PASID if (X86) default n config VFIO_IOMMU_SPAPR_TCE(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex 18ff0c3..ea89c7c 100644drivers/vfio/vfio_iommu_type1.c +6
-0@@ -76,6 +76,7 @@ struct vfio_iommu {bool dirty_page_tracking;bool pinned_page_dirty_scope;struct iommu_nesting_info *nesting_info; +struct vfio_mm *vmm; };struct vfio_domain {@@ -1937,6 +1938,11 @@ static void vfio_iommu_iova_insert_copy(structvfio_iommu *iommu, static void vfio_iommu_release_nesting_info(struct vfio_iommu *iommu) { + if (iommu->vmm) { + vfio_mm_put(iommu->vmm); + iommu->vmm =NULL ; + } +kfree (iommu->nesting_info); iommu->nesting_info =NULL ;}
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2071,6 +2077,26 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data,
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +3
-0iommu->nesting_info);if (ret) goto out_detach; + +if (iommu->nesting_info->features & +IOMMU_NESTING_FEAT_SYSWIDE_PASID ){
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct
Quoted 21 lines
drivers/vfio/vfio_iommu_type1.c +17
-0+struct vfio_mm *vmm; +int sid; + + vmm =vfio_mm_get_from_task (current); +if (IS_ERR (vmm)) { + ret =PTR_ERR (vmm); + goto out_detach; + } + iommu->vmm = vmm; + + sid =vfio_mm_ioasid_sid (vmm); + ret =iommu_domain_set_attr (domain->domain, +DOMAIN_ATTR_IOASID_SID , + &sid); +if (ret) + goto out_detach; + } }/* Get aperture info */ @@ -2859,6 +2885,47 @@ static int vfio_iommu_type1_dirty_pages(struct
vfio_iommu*iommu,Quoted 36 lines
drivers/vfio/vfio_iommu_type1.c +33
-0return -EINVAL ; } +static int vfio_iommu_type1_pasid_request (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_pasid_request req; +unsigned long minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_pasid_request, range); + +if (copy_from_user (&req, (void __user *)arg, minsz)) +return -EFAULT ; + +if (req.argsz < minsz || (req.flags & ~VFIO_PASID_REQUEST_MASK )) +return -EINVAL ; + +if (req.range.min > req.range.max) +return -EINVAL ; + +mutex_lock (&iommu->lock); +if (!iommu->vmm) { +mutex_unlock (&iommu->lock); +return -EOPNOTSUPP ; + } + +switch (req.flags &VFIO_PASID_REQUEST_MASK ) { +case VFIO_IOMMU_FLAG_ALLOC_PASID : + ret =vfio_pasid_alloc (iommu->vmm, req.range.min, + req.range.max); +break ; +case VFIO_IOMMU_FLAG_FREE_PASID : +vfio_pasid_free_range (iommu->vmm, req.range.min, + req.range.max); + ret =0 ;drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct vfio_iommu *iommu,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+break ; +default : + ret = -EINVAL ; + } +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data, unsigned int cmd, unsigned long arg) { @@ -2875,6 +2942,8 @@
drivers/vfio/vfio_pasid.c struct vfio_mm *vfio_mm_get_from_task(struct
Quoted 20 lines
drivers/vfio/vfio_iommu_type1.c +0
-0static long vfio_iommu_type1_ioctl(void *iommu_data,drivers/vfio/vfio_iommu_type1.c +2
-0return vfio_iommu_type1_unmap_dma (iommu, arg);case VFIO_IOMMU_DIRTY_PAGES :return vfio_iommu_type1_dirty_pages (iommu, arg); +case VFIO_IOMMU_PASID_REQUEST : +return vfio_iommu_type1_pasid_request (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +1
-0@@ -61,6 +61,7 @@ void vfio_mm_put(struct vfio_mm *vmm) {kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } +EXPORT_SYMBOL_GPL (vfio_mm_put);static void vfio_mm_get (struct vfio_mm *vmm) {@@ -114,6 +115,13 @@ struct vfio_mm *vfio_mm_get_from_task(struct
task_struct*task)Quoted 64 lines
drivers/vfio/vfio_pasid.c +9
-0mmput (mm);return vmm; } +EXPORT_SYMBOL_GPL (vfio_mm_get_from_task); + +int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return vmm->ioasid_sid; +} +EXPORT_SYMBOL_GPL (vfio_mm_ioasid_sid);/* * Find PASID within @min and @max @@ -202,6 +210,7 @@ int vfio_pasid_alloc(struct vfio_mm *vmm, int min, int max)return pasid; } +EXPORT_SYMBOL_GPL (vfio_pasid_alloc);void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max)@@ -218,6 +227,7 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,vfio_remove_pasid (vmm, vid);mutex_unlock (&vmm->pasid_lock); } +EXPORT_SYMBOL_GPL (vfio_pasid_free_range);static int __init vfio_pasid_init (void ) {include/linux/vfio.h +6
-0@@ -101,6 +101,7 @@ struct vfio_mm;#if IS_ENABLED (CONFIG_VFIO_PASID )extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task);extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_mm_ioasid_sid (struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max);@@ -114,6 +115,11 @@ static inline void vfio_mm_put(struct vfio_mm *vmm) { } +static inline int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return -ENOTTY ; +} +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) {return -ENOTTY ;include/uapi/linux/vfio.h +3
-0@@ -1172,6 +1172,43 @@ struct vfio_iommu_type1_dirty_bitmap_get {#define VFIO_IOMMU_DIRTY_PAGES _IO(VFIO_TYPE, VFIO_BASE + 17) +/** +* VFIO_IOMMU_PASID_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 18, +* struct vfio_iommu_type1_pasid_request) include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1187
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+* +* PASID (Processor Address Space ID) is a PCIe concept for tagging +* address spaces in DMA requests. When system-wide PASID allocation +* is required by the underlying iommu driver (e.g. Intel VT-d), this +* provides an interface for userspace to request pasid alloc/free +* for its assigned devices. Userspace should check the availability +* of this API by checking VFIO_IOMMU_TYPE1_INFO_CAP_NESTING through +* VFIO_IOMMU_GET_INFO. +* +* @flags=VFIO_IOMMU_FLAG_ALLOC_PASID, allocate a single PASID within @range.
Quoted 8 lines
include/uapi/linux/vfio.h +8
-0+* @flags=VFIO_IOMMU_FLAG_FREE_PASID, free the PASIDs within @range. +* @range is [min, max], which means both @min and @max are inclusive. +* ALLOC_PASID and FREE_PASID are mutually exclusive. +* +* returns: allocated PASID value on success, -errno on failure for +* ALLOC_PASID; +* 0 for FREE_PASID operation; +*/ include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1205
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+struct vfio_iommu_type1_pasid_request { +__u32 argsz; +#define VFIO_IOMMU_FLAG_ALLOC_PASID (1 << 0) +#define VFIO_IOMMU_FLAG_FREE_PASID (1 << 1) +__u32 flags; +struct { +__u32 min; +__u32 max; + } range; +};IOCTL(2) Linux Programmer's Manual IOCTL(2)
NAME
ioctl - control deviceSYNOPSIS
#include <sys/ioctl.h> int ioctl(int fd, unsigned long request, ...);
Quoted 7 lines
Quoted 2 lines
ioctl(2) returns a signed int, how can it support returning a
__u32pasid and -errno? Thanks,yeah, pasid is 20 bits today per PCI spec, so a valid pasid won't use full 32 bits, so I used it. perhaps I need to add a field in the data struct to return allocated pasid. that may be safer. do you think it works?
AIUI, the pasid is defined as 20 bits, this isn't simply an implementation issue, right? So perhaps it's ok as is and just needs some supplemental comments and checking to verify that the desired pasid range is valid. Thanks,
Alex
RE: [PATCH v6 07/15] vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free)
Hi Alex,
Quoted 316 lines
From: Alex Williamson <[email protected]>
Sent: Friday, August 21, 2020 9:49 AMQuoted 206 lines · Yi L
Hi Alex,
Quoted 204 lines
From: Alex Williamson <[email protected]>
Sent: Friday, August 21, 2020 4:51 AMQuoted 42 lines · On Mon, 27 Jul 2020 23:27:36 -0700 Liu Yi L <[email protected]> wrote:
This patch allows userspace to request PASID allocation/free, e.g. when serving the request from the guest.
PASIDs that are not freed by userspace are automatically freed when the IOASID set is destroyed when process exits.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) address comments from Eric against v5. remove the alloc/free helper.v4 -> v5:
*) address comments from Eric Auger.
*) the comments for thePASID_FREErequest is addressed in patch 5/15 of
this series.v3 -> v4:
*) address comments from v3, except the below comment against the range
ofPASID_FREErequest. needs more help on it.
"> +if (req.range.min > req.range.max)Is it exploitable that a user can spin the kernel for a long time in the case of a free by calling this with [0,
MAX_UINT] regardless of their actual allocations?"https://lore.kernel.org/linux-iommu/[email protected]/
v1 -> v2:
*) move thevfio_mmrelated code to be a seprate module
*) use a single structure for alloc/free, could support a range of PASIDs
*) fetchvfio_mmatgroup_attachtime instead of at iommu driver open timeQuoted 3 lines
--- drivers/vfio/Kconfig | 1 + drivers/vfio/vfio_iommu_type1.c | 69+++++++++++++++++++++++++++++++++++++++++
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void
Quoted 43 lines
drivers/vfio/vfio_pasid.c | 10 ++++++ include/linux/vfio.h | 6 ++++ include/uapi/linux/vfio.h | 37 ++++++++++++++++++++++ 5 files changed, 123 insertions(+)drivers/vfio/Kconfig index renamed from drivers/vfio/Kconfig +0
-03d8a108..95d90c6 100644
drivers/vfio/Kconfig +1
-0@@ -2,6 +2,7 @@ config VFIO_IOMMU_TYPE1 tristate depends on VFIO + select VFIO_PASID if (X86) default n config VFIO_IOMMU_SPAPR_TCE(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex 18ff0c3..ea89c7c 100644drivers/vfio/vfio_iommu_type1.c +6
-0@@ -76,6 +76,7 @@ struct vfio_iommu {bool dirty_page_tracking;bool pinned_page_dirty_scope;struct iommu_nesting_info *nesting_info; +struct vfio_mm *vmm; };struct vfio_domain {@@ -1937,6 +1938,11 @@ static void vfio_iommu_iova_insert_copy(structvfio_iommu *iommu, static void vfio_iommu_release_nesting_info(struct vfio_iommu *iommu) { + if (iommu->vmm) { + vfio_mm_put(iommu->vmm); + iommu->vmm =NULL ; + } +kfree (iommu->nesting_info); iommu->nesting_info =NULL ;}
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2071,6 +2077,26 @@ static int vfio_iommu_type1_attach_group(void
*iommu_data,drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_attach_group(void *iommu_data,
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +3
-0iommu->nesting_info);if (ret) goto out_detach; + +if (iommu->nesting_info->features & +IOMMU_NESTING_FEAT_SYSWIDE_PASID ){
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct
Quoted 21 lines
drivers/vfio/vfio_iommu_type1.c +17
-0+struct vfio_mm *vmm; +int sid; + + vmm =vfio_mm_get_from_task (current); +if (IS_ERR (vmm)) { + ret =PTR_ERR (vmm); + goto out_detach; + } + iommu->vmm = vmm; + + sid =vfio_mm_ioasid_sid (vmm); + ret =iommu_domain_set_attr (domain->domain, +DOMAIN_ATTR_IOASID_SID , + &sid); +if (ret) + goto out_detach; + } }/* Get aperture info */ @@ -2859,6 +2885,47 @@ static int vfio_iommu_type1_dirty_pages(struct
vfio_iommu*iommu,Quoted 36 lines
drivers/vfio/vfio_iommu_type1.c +33
-0return -EINVAL ; } +static int vfio_iommu_type1_pasid_request (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_pasid_request req; +unsigned long minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_pasid_request, range); + +if (copy_from_user (&req, (void __user *)arg, minsz)) +return -EFAULT ; + +if (req.argsz < minsz || (req.flags & ~VFIO_PASID_REQUEST_MASK )) +return -EINVAL ; + +if (req.range.min > req.range.max) +return -EINVAL ; + +mutex_lock (&iommu->lock); +if (!iommu->vmm) { +mutex_unlock (&iommu->lock); +return -EOPNOTSUPP ; + } + +switch (req.flags &VFIO_PASID_REQUEST_MASK ) { +case VFIO_IOMMU_FLAG_ALLOC_PASID : + ret =vfio_pasid_alloc (iommu->vmm, req.range.min, + req.range.max); +break ; +case VFIO_IOMMU_FLAG_FREE_PASID : +vfio_pasid_free_range (iommu->vmm, req.range.min, + req.range.max); + ret =0 ;drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_dirty_pages(struct vfio_iommu *iommu,
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+break ; +default : + ret = -EINVAL ; + } +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data, unsigned int cmd, unsigned long arg) { @@ -2875,6 +2942,8 @@
drivers/vfio/vfio_pasid.c struct vfio_mm *vfio_mm_get_from_task(struct
Quoted 20 lines
drivers/vfio/vfio_iommu_type1.c +0
-0static long vfio_iommu_type1_ioctl(void *iommu_data,drivers/vfio/vfio_iommu_type1.c +2
-0return vfio_iommu_type1_unmap_dma (iommu, arg);case VFIO_IOMMU_DIRTY_PAGES :return vfio_iommu_type1_dirty_pages (iommu, arg); +case VFIO_IOMMU_PASID_REQUEST : +return vfio_iommu_type1_pasid_request (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +1
-0@@ -61,6 +61,7 @@ void vfio_mm_put(struct vfio_mm *vmm) {kref_put_mutex (&vmm->kref, vfio_mm_release, &vfio_mm_lock); } +EXPORT_SYMBOL_GPL (vfio_mm_put);static void vfio_mm_get (struct vfio_mm *vmm) {@@ -114,6 +115,13 @@ struct vfio_mm *vfio_mm_get_from_task(struct
task_struct*task)Quoted 11 lines
drivers/vfio/vfio_pasid.c +7
-0mmput (mm);return vmm; } +EXPORT_SYMBOL_GPL (vfio_mm_get_from_task); + +int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return vmm->ioasid_sid; +} +EXPORT_SYMBOL_GPL (vfio_mm_ioasid_sid);Quoted 3 lines
Quoted 3 lines
drivers/vfio/vfio_pasid.c int vfio_pasid_alloc(struct vfio_mm *vmm, int min, int
Quoted 3 lines
drivers/vfio/vfio_pasid.c +0
-0/* * Find PASID within @min and @max @@ -202,6 +210,7 @@ int vfio_pasid_alloc(struct vfio_mm *vmm, int min, intmax)
Quoted 92 lines
Quoted 92 lines
Quoted 50 lines
drivers/vfio/vfio_pasid.c +2
-0return pasid; } +EXPORT_SYMBOL_GPL (vfio_pasid_alloc);void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max)@@ -218,6 +227,7 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,vfio_remove_pasid (vmm, vid);mutex_unlock (&vmm->pasid_lock); } +EXPORT_SYMBOL_GPL (vfio_pasid_free_range);static int __init vfio_pasid_init (void ) {include/linux/vfio.h +6
-0@@ -101,6 +101,7 @@ struct vfio_mm;#if IS_ENABLED (CONFIG_VFIO_PASID )extern struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task);extern void vfio_mm_put (struct vfio_mm *vmm); +extern int vfio_mm_ioasid_sid (struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max);@@ -114,6 +115,11 @@ static inline void vfio_mm_put(struct vfio_mm *vmm) { } +static inline int vfio_mm_ioasid_sid (struct vfio_mm *vmm) +{ +return -ENOTTY ; +} +static inline int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max) {return -ENOTTY ;include/uapi/linux/vfio.h +3
-0@@ -1172,6 +1172,43 @@ struct vfio_iommu_type1_dirty_bitmap_get {#define VFIO_IOMMU_DIRTY_PAGES _IO(VFIO_TYPE, VFIO_BASE + 17) +/** +* VFIO_IOMMU_PASID_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 18, +* struct vfio_iommu_type1_pasid_request) include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1187
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+* +* PASID (Processor Address Space ID) is a PCIe concept for tagging +* address spaces in DMA requests. When system-wide PASID allocation +* is required by the underlying iommu driver (e.g. Intel VT-d), this +* provides an interface for userspace to request pasid alloc/free +* for its assigned devices. Userspace should check the availability +* of this API by checking VFIO_IOMMU_TYPE1_INFO_CAP_NESTING through +* VFIO_IOMMU_GET_INFO. +* +* @flags=VFIO_IOMMU_FLAG_ALLOC_PASID, allocate a single PASID within @range.
Quoted 8 lines
include/uapi/linux/vfio.h +8
-0+* @flags=VFIO_IOMMU_FLAG_FREE_PASID, free the PASIDs within @range. +* @range is [min, max], which means both @min and @max are inclusive. +* ALLOC_PASID and FREE_PASID are mutually exclusive. +* +* returns: allocated PASID value on success, -errno on failure for +* ALLOC_PASID; +* 0 for FREE_PASID operation; +*/ include/uapi/linux/vfio.h struct vfio_iommu_type1_dirty_bitmap_get { · line 1205
Quoted 10 lines
include/uapi/linux/vfio.h +10
-0+struct vfio_iommu_type1_pasid_request { +__u32 argsz; +#define VFIO_IOMMU_FLAG_ALLOC_PASID (1 << 0) +#define VFIO_IOMMU_FLAG_FREE_PASID (1 << 1) +__u32 flags; +struct { +__u32 min; +__u32 max; + } range; +};IOCTL(2) Linux Programmer's Manual IOCTL(2)
NAME
ioctl - control deviceSYNOPSIS
#include <sys/ioctl.h> int ioctl(int fd, unsigned long request, ...);Quoted 7 lines
Quoted 2 lines
ioctl(2) returns a signed int, how can it support returning a
__u32pasid and -errno? Thanks,yeah, pasid is 20 bits today per PCI spec, so a valid pasid won't use full 32 bits, so I used it. perhaps I need to add a field in the data struct to return allocated pasid. that may be safer. do you think it works?
Quoted 4 lines
AIUI, the pasid is defined as 20 bits, this isn't simply an implementation issue, right? So perhaps it's ok as is and just needs some supplemental comments and checking to verify that the desired pasid range is valid. Thanks,
yep. may add some comments. if VFIO ensures the desired pasid range is valid, then the allocated pasid should be valid all the same. :-)
Regards,
Yi Liu
Quoted 1 line
Alex
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 8/15 iommu: Pass domain to sva_unbind_gpasid()
In reply to Liu Yi LFrom: Yi Sun <[email protected]>
Current interface is good enough for SVA virtualization on an assigned physical PCI device, but when it comes to mediated devices, a physical device may attached with multiple aux-domains. Also, for guest unbind, the PASID to be unbind should be allocated to the VM. This check requires to know the ioasid_set which is associated with the domain.
So this interface needs to pass in domain info. Then the iommu driver is able to know which domain will be used for the 2nd stage translation of the nesting mode and also be able to do PASID ownership check. This patch passes @domain per the above reason. Also, the prototype of &pasid is changed frnt" to "u32" as the below link.
https://lore.kernel.org/kvm/[email protected]/
---
v5 -> v6:
*) use "u32" prototype for @pasid.
*) add review-by from Eric Auger.
v2 -> v3:
*) pass in domain info only
*) use u32 for pasid instead of int type
v1 -> v2:
*) added in v2.
--- drivers/iommu/intel/svm.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/intel-iommu.h | 3 ++- include/linux/iommu.h | 3 ++- 4 files changed, 7 insertions(+), 4 deletions(-)
drivers/iommu/intel/svm.c +2 -1
@@ -436,7 +436,8 @@ int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev,return ret; } -int intel_svm_unbind_gpasid (struct device *dev,int pasid) +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev);struct intel_svm_dev *sdev;
drivers/iommu/iommu.c +1 -1
@@ -2145,7 +2145,7 @@ int iommu_sva_unbind_gpasid(struct iommu_domain *domain, struct device *dev,if (unlikely (!domain->ops->sva_unbind_gpasid))return -ENODEV ; -return domain->ops->sva_unbind_gpasid (dev, data->hpasid); +return domain->ops->sva_unbind_gpasid (domain, dev, data->hpasid); }EXPORT_SYMBOL_GPL (iommu_sva_unbind_gpasid);
include/linux/intel-iommu.h +2 -1
@@ -738,7 +738,8 @@ extern int intel_svm_enable_prq(struct intel_iommu *iommu);extern int intel_svm_finish_prq (struct intel_iommu *iommu);int intel_svm_bind_gpasid (struct iommu_domain *domain,struct device *dev,struct iommu_gpasid_bind_data *data); -int intel_svm_unbind_gpasid (struct device *dev,int pasid); +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid);struct iommu_sva *intel_svm_bind (struct device *dev,struct mm_struct *mm,void *drvdata);void intel_svm_unbind (struct iommu_sva *handle);
include/linux/iommu.h +2 -1
@@ -303,7 +303,8 @@ struct iommu_ops {int (*sva_bind_gpasid)(struct iommu_domain *domain,struct device *dev,struct iommu_gpasid_bind_data *data); -int (*sva_unbind_gpasid)(struct device *dev,int pasid); +int (*sva_unbind_gpasid)(struct iommu_domain *domain, +struct device *dev,u32 pasid);int (*def_domain_type)(struct device *dev);
Signature · 1 line
2.7.4
Re: [PATCH v6 08/15] iommu: Pass domain to sva_unbind_gpasid()
Quoted 5 lines · On Mon, 27 Jul 2020 23:27:37 -0700 Liu Yi L <[email protected]> wrote:
From: Yi Sun <[email protected]>
Current interface is good enough for SVA virtualization on an assigned physical PCI device, but when it comes to mediated devices, a physical device may attached with multiple aux-domains. Also, for guest unbind,
s/may/may be/
Quoted 8 lines
the PASID to be unbind should be allocated to the VM. This check requires to know the
ioasid_setwhich is associated with the domain.So this interface needs to pass in domain info. Then the iommu driver is able to know which domain will be used for the 2nd stage translation of the nesting mode and also be able to do PASID ownership check. This patch passes @domain per the above reason. Also, the prototype of &pasid is changed frnt" to "u32" as the below link.
s/frnt"/from an "int"/
Quoted 1 line
This is really confusing, the link is to Eric's comment asking that the conversion from (at the time) int to ioasid_t be included in the commit log. The text here implies that it's pointing to some sort of justification for the change, which it isn't. It just notes that it happened, not why it happened, with a mostly irrelevant link.
Quoted 10 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Liu Yi L <[email protected]>
Quoted 4 lines
---
v5 -> v6:
*) use "u32" prototype for @pasid.
*) add review-by from Eric Auger.
I'd probably hold off on adding Eric's R-b given the additional change in this version FWIW. Thanks,
Alex
Quoted 59 lines
v2 -> v3:
*) pass in domain info only
*) use u32 for pasid instead of int typev1 -> v2:
*) added in v2.--- drivers/iommu/intel/svm.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/intel-iommu.h | 3 ++- include/linux/iommu.h | 3 ++- 4 files changed, 7 insertions(+), 4 deletions(-)drivers/iommu/intel/svm.c +2
-1@@ -436,7 +436,8 @@ int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev,return ret; } -int intel_svm_unbind_gpasid (struct device *dev,int pasid) +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev);struct intel_svm_dev *sdev;drivers/iommu/iommu.c +1
-1@@ -2145,7 +2145,7 @@ int iommu_sva_unbind_gpasid(struct iommu_domain *domain, struct device *dev,if (unlikely (!domain->ops->sva_unbind_gpasid))return -ENODEV ; -return domain->ops->sva_unbind_gpasid (dev, data->hpasid); +return domain->ops->sva_unbind_gpasid (domain, dev, data->hpasid); }EXPORT_SYMBOL_GPL (iommu_sva_unbind_gpasid);include/linux/intel-iommu.h +2
-1@@ -738,7 +738,8 @@ extern int intel_svm_enable_prq(struct intel_iommu *iommu);extern int intel_svm_finish_prq (struct intel_iommu *iommu);int intel_svm_bind_gpasid (struct iommu_domain *domain,struct device *dev,struct iommu_gpasid_bind_data *data); -int intel_svm_unbind_gpasid (struct device *dev,int pasid); +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid);struct iommu_sva *intel_svm_bind (struct device *dev,struct mm_struct *mm,void *drvdata);void intel_svm_unbind (struct iommu_sva *handle);include/linux/iommu.h +0
-0
include/linux/iommu.h struct iommu_ops { · line 310
Quoted 9 lines
include/linux/iommu.h +2
-1@@ -303,7 +303,8 @@ struct iommu_ops {int (*sva_bind_gpasid)(struct iommu_domain *domain,struct device *dev,struct iommu_gpasid_bind_data *data); -int (*sva_unbind_gpasid)(struct device *dev,int pasid); +int (*sva_unbind_gpasid)(struct iommu_domain *domain, +struct device *dev,u32 pasid);int (*def_domain_type)(struct device *dev);
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 08/15] iommu: Pass domain to sva_unbind_gpasid()
Hi Alex,
Quoted 6 lines
From: Alex Williamson <[email protected]>
Sent: Friday, August 21, 2020 5:06 AMOn Mon, 27 Jul 2020 23:27:37 -0700
Liu Yi L <[email protected]> wrote:
Quoted 7 lines
Quoted 5 lines
From: Yi Sun <[email protected]>
Current interface is good enough for SVA virtualization on an assigned physical PCI device, but when it comes to mediated devices, a physical device may attached with multiple aux-domains. Also, for guest unbind,
s/may/may be/
got it.
Quoted 11 lines
Quoted 9 lines
the PASID to be unbind should be allocated to the VM. This check requires to know the
ioasid_setwhich is associated with the domain.So this interface needs to pass in domain info. Then the iommu driver is able to know which domain will be used for the 2nd stage translation of the nesting mode and also be able to do PASID ownership check. This patch passes @domain per the above reason. Also, the prototype of &pasid is changed frnt" to "u32" as the below link.
s/frnt"/from an "int"/
got it.
Quoted 7 lines
Quoted 2 lines
https://lore.kernel.org/kvm/27ac7880-bdd3-2891-139e-b4a7cd18420b@redha
t.com/This is really confusing, the link is to Eric's comment asking that the conversion from (at the time) int to
ioasid_tbe included in the commit log. The text here implies that it's pointing to some sort of justification for the change, which it isn't. It just notes that it happened, not why it happened, with a mostly irrelevant link.
really sorry, a mistake from me. it should be the below link.
[PATCH v6 01/12] iommu: Change type of pasid to u32
https://lore.kernel.org/linux-iommu/[email protected]/
Quoted 15 lines
Quoted 10 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Liu Yi L <[email protected]>Quoted 4 lines
---
v5 -> v6:
*) use "u32" prototype for @pasid.
*) add review-by from Eric Auger.
Quoted 2 lines
I'd probably hold off on adding Eric's R-b given the additional change in this version FWIW. Thanks,
ok, will hold on it. :-)
Regards,
Yi Liu
Quoted 66 lines
Alex
drivers/iommu/intel/svm.c int intel_svm_bind_gpasid(struct iommu_domain *domain,
Quoted 18 lines
v2 -> v3:
*) pass in domain info only
*) use u32 for pasid instead of int typev1 -> v2:
*) added in v2.--- drivers/iommu/intel/svm.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/intel-iommu.h | 3 ++- include/linux/iommu.h | 3 ++- 4 files changed, 7 insertions(+), 4 deletions(-)drivers/iommu/intel/svm.c +0
-0@@ -436,7 +436,8 @@ int intel_svm_bind_gpasid(struct iommu_domain *domain,struct device *dev,
drivers/iommu/iommu.c int iommu_sva_unbind_gpasid(struct iommu_domain
Quoted 14 lines
drivers/iommu/intel/svm.c +2
-1return ret; } -int intel_svm_unbind_gpasid (struct device *dev,int pasid) +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev);struct intel_svm_dev *sdev;drivers/iommu/iommu.c index renamed from drivers/iommu/iommu.c +0
-01ce2a61..bee79d7 100644
drivers/iommu/iommu.c +0
-0@@ -2145,7 +2145,7 @@ int iommu_sva_unbind_gpasid(struct iommu_domain*domain, struct device *dev,
Quoted 30 lines
drivers/iommu/iommu.c +1
-1if (unlikely (!domain->ops->sva_unbind_gpasid))return -ENODEV ; -return domain->ops->sva_unbind_gpasid (dev, data->hpasid); +return domain->ops->sva_unbind_gpasid (domain, dev, data->hpasid); }EXPORT_SYMBOL_GPL (iommu_sva_unbind_gpasid);include/linux/intel-iommu.h +2
-0@@ -738,7 +738,8 @@ extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu);int intel_svm_bind_gpasid (struct iommu_domain *domain,struct device *dev, struct iommu_gpasid_bind_data *data); -int intel_svm_unbind_gpasid(struct device *dev, int pasid); +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid);struct iommu_sva *intel_svm_bind (struct device *dev,struct mm_struct *mm,void *drvdata); void intel_svm_unbind(struct iommu_sva *handle); diff --git
a/include/linux/iommu.hb/include/linux/iommu.hindex b1ff702..80467fc 100644include/linux/iommu.h +0
-0@@ -303,7 +303,8 @@ struct iommu_ops {int (*sva_bind_gpasid)(struct iommu_domain *domain,struct device *dev,struct iommu_gpasid_bind_data *data);
Quoted 6 lines
include/linux/iommu.h struct iommu_ops { · line 310
Quoted 6 lines
include/linux/iommu.h +2
-1-int (*sva_unbind_gpasid)(struct device *dev,int pasid); +int (*sva_unbind_gpasid)(struct iommu_domain *domain, +struct device *dev,u32 pasid);int (*def_domain_type)(struct device *dev);
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 08/15] iommu: Pass domain to sva_unbind_gpasid()
In reply to Alex WilliamsonHi,
Quoted 45 lines · Alex Williamson
Quoted 5 lines · On Mon, 27 Jul 2020 23:27:37 -0700 Liu Yi L <[email protected]> wrote:
From: Yi Sun <[email protected]>
Current interface is good enough for SVA virtualization on an assigned physical PCI device, but when it comes to mediated devices, a physical device may attached with multiple aux-domains. Also, for guest unbind,
s/may/may be/
Quoted 8 lines
the PASID to be unbind should be allocated to the VM. This check requires to know the
ioasid_setwhich is associated with the domain.So this interface needs to pass in domain info. Then the iommu driver is able to know which domain will be used for the 2nd stage translation of the nesting mode and also be able to do PASID ownership check. This patch passes @domain per the above reason. Also, the prototype of &pasid is changed frnt" to "u32" as the below link.
s/frnt"/from an "int"/
Quoted 1 line
This is really confusing, the link is to Eric's comment asking that the conversion from (at the time) int to
ioasid_tbe included in the commit log. The text here implies that it's pointing to some sort of justification for the change, which it isn't. It just notes that it happened, not why it happened, with a mostly irrelevant link.Quoted 10 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Yi Sun <[email protected]>Signed-off-by Liu Yi L <[email protected]>Quoted 4 lines
---
v5 -> v6:
*) use "u32" prototype for @pasid.
*) add review-by from Eric Auger.
Quoted 2 lines
I'd probably hold off on adding Eric's R-b given the additional change in this version FWIW. Thanks,
Yep I did not notice that change given the R-b was applied ;-)
Thanks
Eric
Quoted 72 lines
Alex
Quoted 68 lines
v2 -> v3:
*) pass in domain info only
*) use u32 for pasid instead of int typev1 -> v2:
*) added in v2.--- drivers/iommu/intel/svm.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/intel-iommu.h | 3 ++- include/linux/iommu.h | 3 ++- 4 files changed, 7 insertions(+), 4 deletions(-)drivers/iommu/intel/svm.c +2
-1@@ -436,7 +436,8 @@ int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev,return ret; } -int intel_svm_unbind_gpasid (struct device *dev,int pasid) +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev);struct intel_svm_dev *sdev;drivers/iommu/iommu.c +1
-1@@ -2145,7 +2145,7 @@ int iommu_sva_unbind_gpasid(struct iommu_domain *domain, struct device *dev,if (unlikely (!domain->ops->sva_unbind_gpasid))return -ENODEV ; -return domain->ops->sva_unbind_gpasid (dev, data->hpasid); +return domain->ops->sva_unbind_gpasid (domain, dev, data->hpasid); }EXPORT_SYMBOL_GPL (iommu_sva_unbind_gpasid);include/linux/intel-iommu.h +2
-1@@ -738,7 +738,8 @@ extern int intel_svm_enable_prq(struct intel_iommu *iommu);extern int intel_svm_finish_prq (struct intel_iommu *iommu);int intel_svm_bind_gpasid (struct iommu_domain *domain,struct device *dev,struct iommu_gpasid_bind_data *data); -int intel_svm_unbind_gpasid (struct device *dev,int pasid); +int intel_svm_unbind_gpasid (struct iommu_domain *domain, +struct device *dev,u32 pasid);struct iommu_sva *intel_svm_bind (struct device *dev,struct mm_struct *mm,void *drvdata);void intel_svm_unbind (struct iommu_sva *handle);include/linux/iommu.h +2
-1@@ -303,7 +303,8 @@ struct iommu_ops {int (*sva_bind_gpasid)(struct iommu_domain *domain,struct device *dev,struct iommu_gpasid_bind_data *data); -int (*sva_unbind_gpasid)(struct device *dev,int pasid); +int (*sva_unbind_gpasid)(struct iommu_domain *domain, +struct device *dev,u32 pasid);int (*def_domain_type)(struct device *dev);
PATCH v6 9/15 iommu/vt-d: Check ownership for PASIDs from user-space
In reply to Liu Yi LWhen an IOMMU domain with nesting attribute is used for guest SVA, a system-wide PASID is allocated for binding with the device and the domain. For security reason, we need to check the PASID passed from user-space. e.g. page table bind/unbind and PASID related cache invalidation.
--- drivers/iommu/intel/iommu.c | 10 ++++++++++ drivers/iommu/intel/svm.c | 7 +++++-- 2 files changed, 15 insertions(+), 2 deletions(-)
drivers/iommu/intel/iommu.c +10 -0
@@ -5436,6 +5436,7 @@ intel_iommu_sva_invalidate(struct iommu_domain *domain, struct device *dev,int granu =0 ;u64 pasid =0 ;u64 addr =0 ; +void *pdata; granu =to_vtd_granularity (cache_type, inv_info->granularity);if (granu == -EINVAL ) {
@@ -5456,6 +5457,15 @@ intel_iommu_sva_invalidate(struct iommu_domain *domain, struct device *dev, (inv_info->granu.addr_info.flags & IOMMU_INV_ADDR_FLAGS_PASID)) pasid = inv_info->granu.addr_info.pasid; + pdata =ioasid_find (dmar_domain->ioasid_sid, pasid,NULL ); +if (!pdata) { + ret = -EINVAL ; + goto out_unlock; + }else if (IS_ERR (pdata)) { + ret =PTR_ERR (pdata); + goto out_unlock; + } +switch (BIT (cache_type)) {case IOMMU_CACHE_INV_TYPE_IOTLB :/* HW will ignore LSB bits based on address mask */
drivers/iommu/intel/svm.c +5 -2
@@ -323,7 +323,7 @@ int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev, dmar_domain =to_dmar_domain (domain);mutex_lock (&pasid_mutex); - svm =ioasid_find (INVALID_IOASID_SET , data->hpasid,NULL ); + svm =ioasid_find (dmar_domain->ioasid_sid, data->hpasid,NULL );if (IS_ERR (svm)) { ret =PTR_ERR (svm); goto out;
@@ -440,6 +440,7 @@ int intel_svm_unbind_gpasid(struct iommu_domain *domain, struct device *dev, u32 pasid) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev); +struct dmar_domain *dmar_domain;struct intel_svm_dev *sdev;struct intel_svm *svm;int ret = -EINVAL ;
@@ -447,8 +448,10 @@ int intel_svm_unbind_gpasid(struct iommu_domain *domain,if (WARN_ON (!iommu))return -EINVAL ; + dmar_domain =to_dmar_domain (domain); +mutex_lock (&pasid_mutex); - svm =ioasid_find (INVALID_IOASID_SET , pasid,NULL ); + svm =ioasid_find (dmar_domain->ioasid_sid, pasid,NULL );if (!svm) { ret = -EINVAL ; goto out;
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 09/15] iommu/vt-d: Check ownership for PASIDs from user-space
Hi Yi,
Quoted 52 lines · Liu Yi L
When an IOMMU domain with nesting attribute is used for guest SVA, a system-wide PASID is allocated for binding with the device and the domain. For security reason, we need to check the PASID passed from user-space. e.g. page table bind/unbind and PASID related cache invalidation.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>--- drivers/iommu/intel/iommu.c | 10 ++++++++++ drivers/iommu/intel/svm.c | 7 +++++-- 2 files changed, 15 insertions(+), 2 deletions(-)drivers/iommu/intel/iommu.c +10
-0@@ -5436,6 +5436,7 @@ intel_iommu_sva_invalidate(struct iommu_domain *domain, struct device *dev,int granu =0 ;u64 pasid =0 ;u64 addr =0 ; +void *pdata; granu =to_vtd_granularity (cache_type, inv_info->granularity);if (granu == -EINVAL ) {@@ -5456,6 +5457,15 @@ intel_iommu_sva_invalidate(struct iommu_domain *domain, struct device *dev, (inv_info->granu.addr_info.flags & IOMMU_INV_ADDR_FLAGS_PASID)) pasid = inv_info->granu.addr_info.pasid; + pdata =ioasid_find (dmar_domain->ioasid_sid, pasid,NULL ); +if (!pdata) { + ret = -EINVAL ; + goto out_unlock; + }else if (IS_ERR (pdata)) { + ret =PTR_ERR (pdata); + goto out_unlock; + } +switch (BIT (cache_type)) {case IOMMU_CACHE_INV_TYPE_IOTLB :/* HW will ignore LSB bits based on address mask */ drivers/iommu/intel/svm.c +0
-0
drivers/iommu/intel/svm.c int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev, · line 326
Quoted 5 lines
drivers/iommu/intel/svm.c +1
-1@@ -323,7 +323,7 @@ int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev, dmar_domain =to_dmar_domain (domain);mutex_lock (&pasid_mutex); - svm =ioasid_find (INVALID_IOASID_SET , data->hpasid,NULL ); + svm =ioasid_find (dmar_domain->ioasid_sid, data->hpasid,NULL );
A question about the locking strategy. We don't take the device_domain_lock here. Could you clarify whether it is safe?
Quoted 12 lines
drivers/iommu/intel/svm.c +1
-0if (IS_ERR (svm)) { ret =PTR_ERR (svm); goto out;@@ -440,6 +440,7 @@ int intel_svm_unbind_gpasid(struct iommu_domain *domain, struct device *dev, u32 pasid) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev); +struct dmar_domain *dmar_domain;struct intel_svm_dev *sdev;struct intel_svm *svm;int ret = -EINVAL ;
drivers/iommu/intel/svm.c int intel_svm_unbind_gpasid(struct iommu_domain *domain, · line 454
Quoted 8 lines
drivers/iommu/intel/svm.c +3
-1@@ -447,8 +448,10 @@ int intel_svm_unbind_gpasid(struct iommu_domain *domain,if (WARN_ON (!iommu))return -EINVAL ; + dmar_domain =to_dmar_domain (domain); +mutex_lock (&pasid_mutex); - svm =ioasid_find (INVALID_IOASID_SET , pasid,NULL ); + svm =ioasid_find (dmar_domain->ioasid_sid, pasid,NULL );
same here.
drivers/iommu/intel/svm.c int intel_svm_unbind_gpasid(struct iommu_domain *domain, · line 457
Quoted 4 lines
drivers/iommu/intel/svm.c +0
-0if (!svm) { ret = -EINVAL ; goto out;
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 09/15] iommu/vt-d: Check ownership for PASIDs from user-space
Hi Eric,
Quoted 62 lines
From: Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 12:30 AMHi Yi,
drivers/iommu/intel/iommu.c intel_iommu_sva_invalidate(struct iommu_domain
Quoted 24 lines · Liu Yi L
When an IOMMU domain with nesting attribute is used for guest SVA, a system-wide PASID is allocated for binding with the device and the domain. For security reason, we need to check the PASID passed from user-space. e.g. page table bind/unbind and PASID related cache invalidation.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>--- drivers/iommu/intel/iommu.c | 10 ++++++++++ drivers/iommu/intel/svm.c | 7 +++++-- 2 files changed, 15 insertions(+), 2 deletions(-)drivers/iommu/intel/iommu.c +0
-0@@ -5436,6 +5436,7 @@ intel_iommu_sva_invalidate(struct iommu_domain*domain, struct device *dev,
drivers/iommu/intel/iommu.c intel_iommu_sva_invalidate(struct iommu_domain
Quoted 8 lines
drivers/iommu/intel/iommu.c +1
-0int granu =0 ;u64 pasid =0 ;u64 addr =0 ; +void *pdata; granu =to_vtd_granularity (cache_type, inv_info->granularity);if (granu == -EINVAL ) {@@ -5456,6 +5457,15 @@ intel_iommu_sva_invalidate(struct iommu_domain*domain, struct device *dev,
drivers/iommu/intel/iommu.c intel_iommu_sva_invalidate(struct iommu_domain · line 5457
Quoted 1 line
drivers/iommu/intel/iommu.c +0
-0(inv_info->granu.addr_info.flags &
IOMMU_INV_ADDR_FLAGS_PASID))drivers/iommu/intel/svm.c int intel_svm_bind_gpasid(struct iommu_domain *domain,
Quoted 19 lines
drivers/iommu/intel/iommu.c +9
-0pasid = inv_info->granu.addr_info.pasid; + pdata =ioasid_find (dmar_domain->ioasid_sid, pasid,NULL ); +if (!pdata) { + ret = -EINVAL ; + goto out_unlock; + }else if (IS_ERR (pdata)) { + ret =PTR_ERR (pdata); + goto out_unlock; + } +switch (BIT (cache_type)) {case IOMMU_CACHE_INV_TYPE_IOTLB :/* HW will ignore LSB bits based on address mask */ drivers/iommu/intel/svm.c +0
-0@@ -323,7 +323,7 @@ int intel_svm_bind_gpasid(struct iommu_domain *domain,struct device *dev,
Quoted 7 lines
drivers/iommu/intel/svm.c int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev, · line 326
Quoted 5 lines
drivers/iommu/intel/svm.c +1
-1dmar_domain =to_dmar_domain (domain);mutex_lock (&pasid_mutex); - svm =ioasid_find (INVALID_IOASID_SET , data->hpasid,NULL ); + svm =ioasid_find (dmar_domain->ioasid_sid, data->hpasid,NULL );A question about the locking strategy. We don't take the
device_domain_lockhere. Could you clarify whether it is safe?
I guess it is better to take the same lock as what iommu_domain_set_attr() takes. thanks for catching it. :-)
Quoted 18 lines
drivers/iommu/intel/svm.c int intel_svm_unbind_gpasid(struct iommu_domain
Quoted 5 lines
drivers/iommu/intel/svm.c +0
-0if (IS_ERR (svm)) { ret =PTR_ERR (svm); goto out;@@ -440,6 +440,7 @@ int intel_svm_unbind_gpasid(struct iommu_domain*domain,
drivers/iommu/intel/svm.c int intel_svm_unbind_gpasid(struct iommu_domain
Quoted 8 lines
drivers/iommu/intel/svm.c +1
-0struct device *dev, u32 pasid) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev); +struct dmar_domain *dmar_domain;struct intel_svm_dev *sdev;struct intel_svm *svm;int ret = -EINVAL ;@@ -447,8 +448,10 @@ int intel_svm_unbind_gpasid(struct iommu_domain*domain,
Quoted 3 lines
drivers/iommu/intel/svm.c +0
-0if (WARN_ON (!iommu))return -EINVAL ;
Quoted 6 lines
drivers/iommu/intel/svm.c int intel_svm_unbind_gpasid(struct iommu_domain *domain, · line 454
Quoted 5 lines
drivers/iommu/intel/svm.c +3
-1+ dmar_domain =to_dmar_domain (domain); +mutex_lock (&pasid_mutex); - svm =ioasid_find (INVALID_IOASID_SET , pasid,NULL ); + svm =ioasid_find (dmar_domain->ioasid_sid, pasid,NULL );same here.
same.
Regards,
Yi Liu
Quoted 7 lines
drivers/iommu/intel/svm.c int intel_svm_unbind_gpasid(struct iommu_domain *domain, · line 457
Quoted 4 lines
drivers/iommu/intel/svm.c +0
-0if (!svm) { ret = -EINVAL ; goto out;Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 10/15 vfio/type1: Support binding guest page tables to PASID
In reply to Liu Yi LNesting translation allows two-levels/stages page tables, with 1st level for guest translations (e.g. GVA->GPA), 2nd level for host translations (e.g. GPA->HPA). This patch adds interface for binding guest page tables to a PASID. This PASID must have been allocated by the userspace before the binding request.
---
v5 -> v6:
*) dropped vfio_find_nesting_group() and add vfio_get_nesting_domain_capsule().
per comment from Eric.
*) use iommu_uapi_sva_bind/unbind_gpasid() and iommu_sva_unbind_gpasid() in
linux/iommu.h for userspace operation and in-kernel operation.
v3 -> v4:
*) address comments from Alex on v3
v2 -> v3:
*) use __iommu_sva_unbind_gpasid() for unbind call issued by VFIO
https://lore.kernel.org/linux-iommu/[email protected]/
v1 -> v2:
*) rename subject from "vfio/type1: Bind guest page tables to host" *) remove VFIO_IOMMU_BIND, introduce VFIO_IOMMU_NESTING_OP to support bind/
unbind guet page table
*) replaced vfio_iommu_for_each_dev() with a group level loop since this
series enforces one group per container w/ nesting type as start.
*) rename vfio_bind/unbind_gpasid_fn() to vfio_dev_bind/unbind_gpasid_fn() *) vfio_dev_unbind_gpasid() always successful
*) use vfio_mm->pasid_lock to avoid race between PASID free and page table
bind/unbind
--- drivers/vfio/vfio_iommu_type1.c | 161 ++++++++++++++++++++++++++++++++++++++++ drivers/vfio/vfio_pasid.c | 26 +++++++ include/linux/vfio.h | 20 +++++ include/uapi/linux/vfio.h | 31 ++++++++ 4 files changed, 238 insertions(+)
drivers/vfio/vfio_iommu_type1.c +161 -0
@@ -149,6 +149,36 @@ struct vfio_regions {#define DIRTY_BITMAP_PAGES_MAX ((u64)INT_MAX)#define DIRTY_BITMAP_SIZE_MAX DIRTY_BITMAP_BYTES(DIRTY_BITMAP_PAGES_MAX) +struct domain_capsule { +struct vfio_group *group; +struct iommu_domain *domain; +void *data; +}; + +/* iommu->lock must be held */ +static int vfio_get_nesting_domain_capsule (struct vfio_iommu *iommu, +struct domain_capsule *dc) +{ +struct vfio_domain *domain =NULL ; +struct vfio_group *group =NULL ; + +if (!iommu->nesting_info) +return -EINVAL ; + +/* +* Only support singleton container with nesting type. +* If nesting_info is non-NULL, the conatiner should +* be non-empty. Also domain should be non-empty. +*/ + domain =list_first_entry (&iommu->domain_list, + struct vfio_domain, next); + group =list_first_entry (&domain->group_list, + struct vfio_group, next); + dc->group = group; + dc->domain = domain->domain; +return 0 ; +} +static int put_pfn (unsigned long pfn,int prot); static struct vfio_group *vfio_iommu_find_iommu_group(struct vfio_iommu *iommu,
@@ -2349,6 +2379,48 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu,return ret; } +static int vfio_dev_bind_gpasid_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +return iommu_uapi_sva_bind_gpasid (dc->domain, dev, (void __user *)arg); +} + +static int vfio_dev_unbind_gpasid_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +iommu_uapi_sva_unbind_gpasid (dc->domain, dev, (void __user *)arg); +return 0 ; +} + +static int __vfio_dev_unbind_gpasid_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +struct iommu_gpasid_bind_data *unbind_data = + (struct iommu_gpasid_bind_data *)dc->data; + +iommu_sva_unbind_gpasid (dc->domain, dev, unbind_data); +return 0 ; +} + +static void vfio_group_unbind_gpasid_fn (ioasid_t pasid,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +struct iommu_gpasid_bind_data unbind_data; + + unbind_data.argsz = offsetof(struct iommu_gpasid_bind_data , vendor); + unbind_data.flags =0 ; + unbind_data.hpasid = pasid; + + dc->data = &unbind_data; + +iommu_group_for_each_dev (dc->group->iommu_group, + dc, __vfio_dev_unbind_gpasid_fn); +} +static void vfio_iommu_type1_detach_group (void *iommu_data,struct iommu_group *iommu_group) {
@@ -2392,6 +2464,21 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,if (!group)continue ; +if (iommu->nesting_info && iommu->vmm && + (iommu->nesting_info->features & +IOMMU_NESTING_FEAT_BIND_PGTBL )) { +struct domain_capsule dc = { .group = group, + .domain = domain->domain, + .data =NULL }; + +/* +* Unbind page tables bound with system wide PASIDs +* which are allocated to userspace. +*/ +vfio_mm_for_each_pasid (iommu->vmm, &dc, + vfio_group_unbind_gpasid_fn); + } +vfio_iommu_detach_group (domain, group); update_dirty_scope = !group->pinned_page_dirty_scope;list_del (&group->next);
@@ -2926,6 +3013,78 @@ static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu,return ret; } +static long vfio_iommu_handle_pgtbl_op (struct vfio_iommu *iommu, +bool is_bind,unsigned long arg) +{ +struct domain_capsule dc = { .data = &arg }; +struct iommu_nesting_info *info; +int ret; + +mutex_lock (&iommu->lock); + + info = iommu->nesting_info; +if (!info || !(info->features &IOMMU_NESTING_FEAT_BIND_PGTBL )) { + ret = -EOPNOTSUPP ; + goto out_unlock; + } + +if (!iommu->vmm) { + ret = -EINVAL ; + goto out_unlock; + } + + ret =vfio_get_nesting_domain_capsule (iommu, &dc); +if (ret) + goto out_unlock; + +/* Avoid race with other containers within the same process */ +vfio_mm_pasid_lock (iommu->vmm); + + ret =0 ; +if (is_bind) + ret =iommu_group_for_each_dev (dc.group->iommu_group, &dc, + vfio_dev_bind_gpasid_fn); +if (ret || !is_bind) { +iommu_group_for_each_dev (dc.group->iommu_group, + &dc, vfio_dev_unbind_gpasid_fn); + ret =0 ; + } + +vfio_mm_pasid_unlock (iommu->vmm); +out_unlock: +mutex_unlock (&iommu->lock); +return ret; +} + +static long vfio_iommu_type1_nesting_op (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_nesting_op hdr; +unsigned int minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_nesting_op, flags); + +if (copy_from_user (&hdr, (void __user *)arg, minsz)) +return -EFAULT ; + +if (hdr.argsz < minsz || hdr.flags & ~VFIO_NESTING_OP_MASK ) +return -EINVAL ; + +switch (hdr.flags &VFIO_NESTING_OP_MASK ) { +case VFIO_IOMMU_NESTING_OP_BIND_PGTBL : + ret =vfio_iommu_handle_pgtbl_op (iommu, true, arg + minsz); +break ; +case VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL : + ret =vfio_iommu_handle_pgtbl_op (iommu, false, arg + minsz); +break ; +default : + ret = -EINVAL ; + } + +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data,unsigned int cmd,unsigned long arg) {
@@ -2944,6 +3103,8 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,return vfio_iommu_type1_dirty_pages (iommu, arg);case VFIO_IOMMU_PASID_REQUEST :return vfio_iommu_type1_pasid_request (iommu, arg); +case VFIO_IOMMU_NESTING_OP : +return vfio_iommu_type1_nesting_op (iommu, arg);default :return -ENOTTY ; }
drivers/vfio/vfio_pasid.c +26 -0
@@ -221,6 +221,8 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,* IOASID core will notify PASID users (e.g. IOMMU driver) to * teardown necessary structures depending on the to-be-freed * PASID. +* Hold pasid_lock also avoids race with PASID usages like bind/ +* unbind page tables to requested PASID. */ mutex_lock (&vmm->pasid_lock);while ((vid =vfio_find_pasid (vmm, min, max)) !=NULL )
@@ -229,6 +231,30 @@ void vfio_pasid_free_range(struct vfio_mm *vmm, }EXPORT_SYMBOL_GPL (vfio_pasid_free_range); +int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)) +{ +int ret; + +mutex_lock (&vmm->pasid_lock); + ret =ioasid_set_for_each_ioasid (vmm->ioasid_sid, fn, data); +mutex_unlock (&vmm->pasid_lock); +return ret; +} +EXPORT_SYMBOL_GPL (vfio_mm_for_each_pasid); + +void vfio_mm_pasid_lock (struct vfio_mm *vmm) +{ +mutex_lock (&vmm->pasid_lock); +} +EXPORT_SYMBOL_GPL (vfio_mm_pasid_lock); + +void vfio_mm_pasid_unlock (struct vfio_mm *vmm) +{ +mutex_unlock (&vmm->pasid_lock); +} +EXPORT_SYMBOL_GPL (vfio_mm_pasid_unlock); +static int __init vfio_pasid_init (void ) {mutex_init (&vfio_mm_lock);
include/linux/vfio.h +20 -0
@@ -105,6 +105,11 @@ extern int vfio_mm_ioasid_sid(struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max); +extern int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)); +extern void vfio_mm_pasid_lock (struct vfio_mm *vmm); +extern void vfio_mm_pasid_unlock (struct vfio_mm *vmm); +#else static inline struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task) {
@@ -129,6 +134,21 @@ static inline void vfio_pasid_free_range(struct vfio_mm *vmm, ioasid_t min, ioasid_t max) { } + +static inline int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)) +{ +return -ENOTTY ; +} + +static inline void vfio_mm_pasid_lock (struct vfio_mm *vmm) +{ +} + +static inline void vfio_mm_pasid_unlock (struct vfio_mm *vmm) +{ +} +#endif /* CONFIG_VFIO_PASID */ /*
include/uapi/linux/vfio.h +31 -0
@@ -1209,6 +1209,37 @@ struct vfio_iommu_type1_pasid_request {#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18) +/** +* VFIO_IOMMU_NESTING_OP - _IOW(VFIO_TYPE, VFIO_BASE + 19, +* struct vfio_iommu_type1_nesting_op) +* +* This interface allows userspace to utilize the nesting IOMMU +* capabilities as reported in VFIO_IOMMU_TYPE1_INFO_CAP_NESTING +* cap through VFIO_IOMMU_GET_INFO. +* +* @data[] types defined for each op: +* +=================+===============================================+ +* | NESTING OP | @data[] | +* +=================+===============================================+ +* | BIND_PGTBL | struct iommu_gpasid_bind_data | +* +-----------------+-----------------------------------------------+ +* | UNBIND_PGTBL | struct iommu_gpasid_bind_data | +* +-----------------+-----------------------------------------------+ +* +* returns: 0 on success, -errno on failure. +*/ +struct vfio_iommu_type1_nesting_op { +__u32 argsz; +__u32 flags; +#define VFIO_NESTING_OP_MASK (0xffff)/* lower 16-bits for op */ +__u8 data[]; +}; + +#define VFIO_IOMMU_NESTING_OP_BIND_PGTBL (0) +#define VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL (1) + +#define VFIO_IOMMU_NESTING_OP _IO(VFIO_TYPE, VFIO_BASE + 19) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /*
Signature · 1 line
2.7.4
Re: [PATCH v6 10/15] vfio/type1: Support binding guest page tables to PASID
Hi Yi,
Quoted 52 lines · Liu Yi L
Nesting translation allows two-levels/stages page tables, with 1st level for guest translations (e.g.
GVA->GPA), 2nd level for host translations (e.g.GPA->HPA). This patch adds interface for binding guest page tables to a PASID. This PASID must have been allocated by the userspace before the binding request.Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Jean-Philippe Brucker <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) droppedvfio_find_nesting_group()and addvfio_get_nesting_domain_capsule().
per comment from Eric.
*) use iommu_uapi_sva_bind/unbind_gpasid() andiommu_sva_unbind_gpasid()in
linux/iommu.hfor userspace operation and in-kernel operation.v3 -> v4:
*) address comments from Alex on v3v2 -> v3:
*) use__iommu_sva_unbind_gpasid()for unbind call issued by VFIO
https://lore.kernel.org/linux-iommu/[email protected]/v1 -> v2:
*) rename subject from "vfio/type1: Bind guest page tables to host" *) removeVFIO_IOMMU_BIND, introduceVFIO_IOMMU_NESTING_OPto support bind/
unbind guet page table
*) replacedvfio_iommu_for_each_dev()with a group level loop since this
series enforces one group per container w/ nesting type as start.
*) rename vfio_bind/unbind_gpasid_fn() to vfio_dev_bind/unbind_gpasid_fn() *)vfio_dev_unbind_gpasid()always successful
*) usevfio_mm->pasid_lockto avoid race between PASID free and page table
bind/unbind--- drivers/vfio/vfio_iommu_type1.c | 161 ++++++++++++++++++++++++++++++++++++++++ drivers/vfio/vfio_pasid.c | 26 +++++++ include/linux/vfio.h | 20 +++++ include/uapi/linux/vfio.h | 31 ++++++++ 4 files changed, 238 insertions(+)drivers/vfio/vfio_iommu_type1.c +0
-0
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 155
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +4
-0@@ -149,6 +149,36 @@ struct vfio_regions {#define DIRTY_BITMAP_PAGES_MAX ((u64)INT_MAX)#define DIRTY_BITMAP_SIZE_MAX DIRTY_BITMAP_BYTES(DIRTY_BITMAP_PAGES_MAX) +struct domain_capsule { +struct vfio_group *group; +struct iommu_domain *domain; +void *data;
You may add a bool indicating whether the data is a user pointer or the direct IOMMU UAPI struct.
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 160
Quoted 5 lines
drivers/vfio/vfio_iommu_type1.c +5
-0+}; + +/* iommu->lock must be held */ +static int vfio_get_nesting_domain_capsule (struct vfio_iommu *iommu, +struct domain_capsule *dc)
I would rename the function into vfio_prepare_nesting_domain_capsule
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 170
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +10
-0+{ +struct vfio_domain *domain =NULL ; +struct vfio_group *group =NULL ; + +if (!iommu->nesting_info) +return -EINVAL ; + +/* +* Only support singleton container with nesting type. +* If nesting_info is non-NULL, the conatiner should
s/should/is here and below
s/conatiner/container
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 178
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+* be non-empty. Also domain should be non-empty. +*/ + domain =list_first_entry (&iommu->domain_list, + struct vfio_domain, next); + group =list_first_entry (&domain->group_list, + struct vfio_group, next); + dc->group = group; + dc->domain = domain->domain;
dc->user = true;?
Quoted 18 lines
drivers/vfio/vfio_iommu_type1.c +11
-0+return 0 ; +} +static int put_pfn (unsigned long pfn,int prot); static struct vfio_group *vfio_iommu_find_iommu_group(struct vfio_iommu *iommu,@@ -2349,6 +2379,48 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu,return ret; } +static int vfio_dev_bind_gpasid_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +return iommu_uapi_sva_bind_gpasid (dc->domain, dev, (void __user *)arg); +} +
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu, · line 2399
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +10
-0+static int vfio_dev_unbind_gpasid_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +iommu_uapi_sva_unbind_gpasid (dc->domain, dev, (void __user *)arg); +return 0 ; +} + +static int __vfio_dev_unbind_gpasid_fn (struct device *dev,void *data)
can be removed if dc->user flag gets added
Quoted 5 lines
drivers/vfio/vfio_iommu_type1.c +5
-0+{ +struct domain_capsule *dc = (struct domain_capsule *)data; +struct iommu_gpasid_bind_data *unbind_data = + (struct iommu_gpasid_bind_data *)dc->data; +
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu, · line 2414
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +10
-0+iommu_sva_unbind_gpasid (dc->domain, dev, unbind_data); +return 0 ; +} + +static void vfio_group_unbind_gpasid_fn (ioasid_t pasid,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +struct iommu_gpasid_bind_data unbind_data; + + unbind_data.argsz = offsetof(struct iommu_gpasid_bind_data , vendor);
I do not remember to have seen this documented anywhere, ie. on unbind, the vendor data are not used.
Quoted 13 lines
drivers/vfio/vfio_iommu_type1.c +9
-0+ unbind_data.flags =0 ; + unbind_data.hpasid = pasid; + + dc->data = &unbind_data; + +iommu_group_for_each_dev (dc->group->iommu_group, + dc, __vfio_dev_unbind_gpasid_fn); +} +static void vfio_iommu_type1_detach_group (void *iommu_data,struct iommu_group *iommu_group) {
drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_type1_detach_group(void *iommu_data, · line 2467
Quoted 4 lines
drivers/vfio/vfio_iommu_type1.c +1
-0@@ -2392,6 +2464,21 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,if (!group)continue ; + if (iommu->nesting_info && iommu->vmm &&
if vmm is set, doesn't it mean nesting_info is set?
Quoted 27 lines
drivers/vfio/vfio_iommu_type1.c +20
-0+ (iommu->nesting_info->features & + IOMMU_NESTING_FEAT_BIND_PGTBL)) { +struct domain_capsule dc = { .group = group, + .domain = domain->domain, + .data =NULL }; + +/* +* Unbind page tables bound with system wide PASIDs +* which are allocated to userspace. +*/ +vfio_mm_for_each_pasid (iommu->vmm, &dc, + vfio_group_unbind_gpasid_fn); + } +vfio_iommu_detach_group (domain, group); update_dirty_scope = !group->pinned_page_dirty_scope;list_del (&group->next);@@ -2926,6 +3013,78 @@ static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu,return ret; } +static long vfio_iommu_handle_pgtbl_op (struct vfio_iommu *iommu, +bool is_bind,unsigned long arg) +{ +struct domain_capsule dc = { .data = &arg }; +struct iommu_nesting_info *info; +int ret;
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3031
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +10
-0+ +mutex_lock (&iommu->lock); + + info = iommu->nesting_info; +if (!info || !(info->features &IOMMU_NESTING_FEAT_BIND_PGTBL )) { + ret = -EOPNOTSUPP ; + goto out_unlock; + } + +if (!iommu->vmm) {
Is it documented anywhere, besides in this commit message that IOMMU_NESTING_FEAT_BIND_PGTBL requires IOMMU_NESTING_FEAT_SYSWIDE_PASID (in uapi doc)?
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3043
Quoted 12 lines
drivers/vfio/vfio_iommu_type1.c +12
-0+ ret = -EINVAL ; + goto out_unlock; + } + + ret =vfio_get_nesting_domain_capsule (iommu, &dc); +if (ret) + goto out_unlock; + +/* Avoid race with other containers within the same process */ +vfio_mm_pasid_lock (iommu->vmm); + + ret =0 ;
non needed
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3050
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +7
-0+if (is_bind) + ret =iommu_group_for_each_dev (dc.group->iommu_group, &dc, + vfio_dev_bind_gpasid_fn); +if (ret || !is_bind) { +iommu_group_for_each_dev (dc.group->iommu_group, + &dc, vfio_dev_unbind_gpasid_fn); + ret =0 ;
ret is not correct in case bind previously failed. You should return the original ret in that case.
Quoted 11 lines
drivers/vfio/vfio_iommu_type1.c +11
-0+ } + +vfio_mm_pasid_unlock (iommu->vmm); +out_unlock: +mutex_unlock (&iommu->lock); +return ret; +} + +static long vfio_iommu_type1_nesting_op (struct vfio_iommu *iommu, +unsigned long arg) +{
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3071
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +10
-0+struct vfio_iommu_type1_nesting_op hdr; +unsigned int minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_nesting_op, flags); + +if (copy_from_user (&hdr, (void __user *)arg, minsz)) +return -EFAULT ; + +if (hdr.argsz < minsz || hdr.flags & ~VFIO_NESTING_OP_MASK )
Checking against VFIO_NESTING_OP_MASK is not enough precise enough (I mean OP_MASK should only encompasses the actual supported ops). If a a userspace tries to use a new OP in the future and if this runs on this kernel version, you will fait to recognize the lack of support.
Quoted 115 lines
drivers/vfio/vfio_iommu_type1.c +18
-0+return -EINVAL ; + +switch (hdr.flags &VFIO_NESTING_OP_MASK ) { +case VFIO_IOMMU_NESTING_OP_BIND_PGTBL : + ret =vfio_iommu_handle_pgtbl_op (iommu, true, arg + minsz); +break ; +case VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL : + ret =vfio_iommu_handle_pgtbl_op (iommu, false, arg + minsz); +break ; +default : + ret = -EINVAL ; + } + +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data,unsigned int cmd,unsigned long arg) {@@ -2944,6 +3103,8 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,return vfio_iommu_type1_dirty_pages (iommu, arg);case VFIO_IOMMU_PASID_REQUEST :return vfio_iommu_type1_pasid_request (iommu, arg); +case VFIO_IOMMU_NESTING_OP : +return vfio_iommu_type1_nesting_op (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +26
-0@@ -221,6 +221,8 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,* IOASID core will notify PASID users (e.g. IOMMU driver) to * teardown necessary structures depending on the to-be-freed * PASID. +* Hold pasid_lock also avoids race with PASID usages like bind/ +* unbind page tables to requested PASID. */ mutex_lock (&vmm->pasid_lock);while ((vid =vfio_find_pasid (vmm, min, max)) !=NULL )@@ -229,6 +231,30 @@ void vfio_pasid_free_range(struct vfio_mm *vmm, }EXPORT_SYMBOL_GPL (vfio_pasid_free_range); +int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)) +{ +int ret; + +mutex_lock (&vmm->pasid_lock); + ret =ioasid_set_for_each_ioasid (vmm->ioasid_sid, fn, data); +mutex_unlock (&vmm->pasid_lock); +return ret; +} +EXPORT_SYMBOL_GPL (vfio_mm_for_each_pasid); + +void vfio_mm_pasid_lock (struct vfio_mm *vmm) +{ +mutex_lock (&vmm->pasid_lock); +} +EXPORT_SYMBOL_GPL (vfio_mm_pasid_lock); + +void vfio_mm_pasid_unlock (struct vfio_mm *vmm) +{ +mutex_unlock (&vmm->pasid_lock); +} +EXPORT_SYMBOL_GPL (vfio_mm_pasid_unlock); +static int __init vfio_pasid_init (void ) {mutex_init (&vfio_mm_lock);include/linux/vfio.h +20
-0@@ -105,6 +105,11 @@ extern int vfio_mm_ioasid_sid(struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max); +extern int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)); +extern void vfio_mm_pasid_lock (struct vfio_mm *vmm); +extern void vfio_mm_pasid_unlock (struct vfio_mm *vmm); +#else static inline struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task) {@@ -129,6 +134,21 @@ static inline void vfio_pasid_free_range(struct vfio_mm *vmm, ioasid_t min, ioasid_t max) { } + +static inline int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)) +{ +return -ENOTTY ; +} + +static inline void vfio_mm_pasid_lock (struct vfio_mm *vmm) +{ +} + +static inline void vfio_mm_pasid_unlock (struct vfio_mm *vmm) +{ +} +#endif /* CONFIG_VFIO_PASID */ /* include/uapi/linux/vfio.h +0
-0
include/uapi/linux/vfio.h struct vfio_iommu_type1_pasid_request { · line 1218
Quoted 10 lines
include/uapi/linux/vfio.h +7
-0@@ -1209,6 +1209,37 @@ struct vfio_iommu_type1_pasid_request {#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18) +/** +* VFIO_IOMMU_NESTING_OP - _IOW(VFIO_TYPE, VFIO_BASE + 19, +* struct vfio_iommu_type1_nesting_op) +* +* This interface allows userspace to utilize the nesting IOMMU +* capabilities as reported in VFIO_IOMMU_TYPE1_INFO_CAP_NESTING +* cap through VFIO_IOMMU_GET_INFO.
PASID alloc/free are handled through another IOCTL so you should be more precise I think.
Quoted 18 lines
include/uapi/linux/vfio.h +17
-0+* +* @data[] types defined for each op: +* +=================+===============================================+ +* | NESTING OP | @data[] | +* +=================+===============================================+ +* | BIND_PGTBL | struct iommu_gpasid_bind_data | +* +-----------------+-----------------------------------------------+ +* | UNBIND_PGTBL | struct iommu_gpasid_bind_data | +* +-----------------+-----------------------------------------------+ +* +* returns: 0 on success, -errno on failure. +*/ +struct vfio_iommu_type1_nesting_op { +__u32 argsz; +__u32 flags; +#define VFIO_NESTING_OP_MASK (0xffff)/* lower 16-bits for op */ +__u8 data[];
include/uapi/linux/vfio.h struct vfio_iommu_type1_pasid_request { · line 1245
Quoted 10 lines
include/uapi/linux/vfio.h +7
-0+}; + +#define VFIO_IOMMU_NESTING_OP_BIND_PGTBL (0) +#define VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL (1) + +#define VFIO_IOMMU_NESTING_OP _IO(VFIO_TYPE, VFIO_BASE + 19) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /*
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 10/15] vfio/type1: Support binding guest page tables to PASID
Hi Eric,
Quoted 65 lines
From: Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 7:29 PMHi Yi,
Quoted 23 lines · Liu Yi L
Nesting translation allows two-levels/stages page tables, with 1st level for guest translations (e.g.
GVA->GPA), 2nd level for host translations (e.g.GPA->HPA). This patch adds interface for binding guest page tables to a PASID. This PASID must have been allocated by the userspace before the binding request.Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Jean-Philippe Brucker <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5 -> v6:
*) droppedvfio_find_nesting_group()and addvfio_get_nesting_domain_capsule().
per comment from Eric.
*) use iommu_uapi_sva_bind/unbind_gpasid() andiommu_sva_unbind_gpasid()in
linux/iommu.hfor userspace operation and in-kernel operation.Quoted 6 lines
v3 -> v4:
*) address comments from Alex on v3v2 -> v3:
*) use__iommu_sva_unbind_gpasid()for unbind call issued by VFIO
https://lore.kernel.org/linux-iommu/1592931837-58223-6-git-send-email-Quoted 4 lines
v1 -> v2:
*) rename subject from "vfio/type1: Bind guest page tables to host" *) removeVFIO_IOMMU_BIND, introduceVFIO_IOMMU_NESTING_OPto supportbind/
Quoted 9 lines
unbind guet page table
*) replacedvfio_iommu_for_each_dev()with a group level loop since this
series enforces one group per container w/ nesting type as start.
*) rename vfio_bind/unbind_gpasid_fn() to vfio_dev_bind/unbind_gpasid_fn() *)vfio_dev_unbind_gpasid()always successful
*) usevfio_mm->pasid_lockto avoid race between PASID free and page table
bind/unbind--- drivers/vfio/vfio_iommu_type1.c | 161++++++++++++++++++++++++++++++++++++++++
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 150
Quoted 12 lines
drivers/vfio/vfio_pasid.c | 26 +++++++ include/linux/vfio.h | 20 +++++ include/uapi/linux/vfio.h | 31 ++++++++ 4 files changed, 238 insertions(+)drivers/vfio/vfio_iommu_type1.c +0
-0@@ -149,6 +149,36 @@ struct vfio_regions {#define DIRTY_BITMAP_PAGES_MAX ((u64)INT_MAX)#define DIRTY_BITMAP_SIZE_MAX
DIRTY_BITMAP_BYTES(DIRTY_BITMAP_PAGES_MAX)Quoted 1 line
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 155
Quoted 4 lines
drivers/vfio/vfio_iommu_type1.c +4
-0+struct domain_capsule { +struct vfio_group *group; +struct iommu_domain *domain; +void *data;You may add a bool indicating whether the data is a user pointer or the direct IOMMU UAPI struct.
yes, it is helpful.
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 160
Quoted 5 lines
drivers/vfio/vfio_iommu_type1.c +5
-0+}; + +/* iommu->lock must be held */ +static int vfio_get_nesting_domain_capsule (struct vfio_iommu *iommu, +struct domain_capsule *dc)I would rename the function into
vfio_prepare_nesting_domain_capsule
got it. :-)
Quoted 4 lines
Quoted 4 lines
drivers/vfio/vfio_iommu_type1.c +4
-0+{ +struct vfio_domain *domain =NULL ; +struct vfio_group *group =NULL ; +
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 170
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +6
-0+if (!iommu->nesting_info) +return -EINVAL ; + +/* +* Only support singleton container with nesting type. +* If nesting_info is non-NULL, the conatiner should s/should/is here and below
s/conatiner/container
got it. thanks.
Quoted 9 lines
drivers/vfio/vfio_iommu_type1.c struct vfio_regions { · line 178
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c +8
-0+* be non-empty. Also domain should be non-empty. +*/ + domain =list_first_entry (&iommu->domain_list, + struct vfio_domain, next); + group =list_first_entry (&domain->group_list, + struct vfio_group, next); + dc->group = group; + dc->domain = domain->domain;
dc->user= true;?
yep.
Quoted 23 lines
drivers/vfio/vfio_iommu_type1.c struct vfio_regions {
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +3
-0+return 0 ; +} +static int put_pfn (unsigned long pfn,int prot); static struct vfio_group *vfio_iommu_find_iommu_group(struct vfio_iommu*iommu,
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_resv_refresh(struct vfio_iommu
Quoted 1 line
drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2349,6 +2379,48 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu*iommu,
Quoted 14 lines
drivers/vfio/vfio_iommu_type1.c +11
-0return ret; } +static int vfio_dev_bind_gpasid_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +return iommu_uapi_sva_bind_gpasid (dc->domain, dev, (void __user *)arg); +} + +static int vfio_dev_unbind_gpasid_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data;
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu, · line 2399
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +7
-0+unsigned long arg = *(unsigned long *)dc->data; + +iommu_uapi_sva_unbind_gpasid (dc->domain, dev, (void __user *)arg); +return 0 ; +} + +static int __vfio_dev_unbind_gpasid_fn (struct device *dev,void *data)can be removed if
dc->userflag gets added
yes.
Quoted 9 lines
Quoted 9 lines
drivers/vfio/vfio_iommu_type1.c +9
-0+{ +struct domain_capsule *dc = (struct domain_capsule *)data; +struct iommu_gpasid_bind_data *unbind_data = + (struct iommu_gpasid_bind_data *)dc->data; + +iommu_sva_unbind_gpasid (dc->domain, dev, unbind_data); +return 0 ; +} +
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu, · line 2414
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +6
-0+static void vfio_group_unbind_gpasid_fn (ioasid_t pasid,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +struct iommu_gpasid_bind_data unbind_data; + + unbind_data.argsz = offsetof(struct iommu_gpasid_bind_data , vendor);I do not remember to have seen this documented anywhere, ie. on unbind, the vendor data are not used.
right. perhaps needs to doc it in Jacob's patch. @Pan, Jacob jun
Quoted 14 lines
drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_type1_detach_group(void
Quoted 13 lines
drivers/vfio/vfio_iommu_type1.c +9
-0+ unbind_data.flags =0 ; + unbind_data.hpasid = pasid; + + dc->data = &unbind_data; + +iommu_group_for_each_dev (dc->group->iommu_group, + dc, __vfio_dev_unbind_gpasid_fn); +} +static void vfio_iommu_type1_detach_group (void *iommu_data,struct iommu_group *iommu_group) {@@ -2392,6 +2464,21 @@ static void vfio_iommu_type1_detach_group(void
*iommu_data,
Quoted 5 lines
drivers/vfio/vfio_iommu_type1.c static void vfio_iommu_type1_detach_group(void *iommu_data, · line 2467
Quoted 4 lines
drivers/vfio/vfio_iommu_type1.c +1
-0if (!group)continue ; + if (iommu->nesting_info && iommu->vmm &&if vmm is set, doesn't it mean
nesting_infois set?
yes, will remove the check against iommu->nesting_info.
Quoted 34 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct
Quoted 18 lines
drivers/vfio/vfio_iommu_type1.c +14
-0+ (iommu->nesting_info->features & + IOMMU_NESTING_FEAT_BIND_PGTBL)) { +struct domain_capsule dc = { .group = group, + .domain = domain->domain, + .data =NULL }; + +/* +* Unbind page tables bound with system wide PASIDs +* which are allocated to userspace. +*/ +vfio_mm_for_each_pasid (iommu->vmm, &dc, + vfio_group_unbind_gpasid_fn); + } +vfio_iommu_detach_group (domain, group); update_dirty_scope = !group->pinned_page_dirty_scope;list_del (&group->next);@@ -2926,6 +3013,78 @@ static int vfio_iommu_type1_pasid_request(struct
vfio_iommu*iommu,Quoted 15 lines
drivers/vfio/vfio_iommu_type1.c +12
-0return ret; } +static long vfio_iommu_handle_pgtbl_op (struct vfio_iommu *iommu, +bool is_bind,unsigned long arg) +{ +struct domain_capsule dc = { .data = &arg }; +struct iommu_nesting_info *info; +int ret; + +mutex_lock (&iommu->lock); + + info = iommu->nesting_info; +if (!info || !(info->features &IOMMU_NESTING_FEAT_BIND_PGTBL )) { + ret = -EOPNOTSUPP ;
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3031
Quoted 4 lines
drivers/vfio/vfio_iommu_type1.c +4
-0+ goto out_unlock; + } + +if (!iommu->vmm) {Is it documented anywhere, besides in this commit message that
IOMMU_NESTING_FEAT_BIND_PGTBLrequiresIOMMU_NESTING_FEAT_SYSWIDE_PASID
(in uapi doc)?
nice catch. it is not documented anywhere. VFIO_IOMMU_NESTING_OP_BIND_PGTBL should work for the bind with pasid or without pasid. perhaps, I can add a flag in vfio to differentiate the two bind cases. is it workable per your opinion?
Quoted 5 lines
Quoted 5 lines
drivers/vfio/vfio_iommu_type1.c +5
-0+ ret = -EINVAL ; + goto out_unlock; + } + + ret =vfio_get_nesting_domain_capsule (iommu, &dc);
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3043
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +7
-0+if (ret) + goto out_unlock; + +/* Avoid race with other containers within the same process */ +vfio_mm_pasid_lock (iommu->vmm); + + ret =0 ;non needed
oh yes.
Quoted 9 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3050
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +7
-0+if (is_bind) + ret =iommu_group_for_each_dev (dc.group->iommu_group, &dc, + vfio_dev_bind_gpasid_fn); +if (ret || !is_bind) { +iommu_group_for_each_dev (dc.group->iommu_group, + &dc, vfio_dev_unbind_gpasid_fn); + ret =0 ;ret is not correct in case bind previously failed. You should return the original ret in that case.
right. will remove ret = 0.
Quoted 17 lines
Quoted 17 lines
drivers/vfio/vfio_iommu_type1.c +17
-0+ } + +vfio_mm_pasid_unlock (iommu->vmm); +out_unlock: +mutex_unlock (&iommu->lock); +return ret; +} + +static long vfio_iommu_type1_nesting_op (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct vfio_iommu_type1_nesting_op hdr; +unsigned int minsz; +int ret; + + minsz = offsetofend(struct vfio_iommu_type1_nesting_op, flags); +
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_type1_pasid_request(struct vfio_iommu *iommu, · line 3071
Quoted 4 lines
drivers/vfio/vfio_iommu_type1.c +4
-0+if (copy_from_user (&hdr, (void __user *)arg, minsz)) +return -EFAULT ; + +if (hdr.argsz < minsz || hdr.flags & ~VFIO_NESTING_OP_MASK )Checking against
VFIO_NESTING_OP_MASKis not enough precise enough (I meanOP_MASKshould only encompasses the actual supported ops). If a a userspace tries to use a new OP in the future and if this runs on this kernel version, you will fait to recognize the lack of support.
yes, this should be fixed. this code only ensures the high 16 bits are valid, but failed to check the lower 16 bits. the op counts from 0 to 2^16-1. so I may add a MAX_OP macro to serve the purpose of failing unsupported op usage.
Quoted 119 lines
include/linux/vfio.h static inline void vfio_pasid_free_range(struct vfio_mm
Quoted 89 lines
drivers/vfio/vfio_iommu_type1.c +18
-0+return -EINVAL ; + +switch (hdr.flags &VFIO_NESTING_OP_MASK ) { +case VFIO_IOMMU_NESTING_OP_BIND_PGTBL : + ret =vfio_iommu_handle_pgtbl_op (iommu, true, arg + minsz); +break ; +case VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL : + ret =vfio_iommu_handle_pgtbl_op (iommu, false, arg + minsz); +break ; +default : + ret = -EINVAL ; + } + +return ret; +} +static long vfio_iommu_type1_ioctl (void *iommu_data,unsigned int cmd,unsigned long arg) {@@ -2944,6 +3103,8 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,return vfio_iommu_type1_dirty_pages (iommu, arg);case VFIO_IOMMU_PASID_REQUEST :return vfio_iommu_type1_pasid_request (iommu, arg); +case VFIO_IOMMU_NESTING_OP : +return vfio_iommu_type1_nesting_op (iommu, arg);default :return -ENOTTY ; }drivers/vfio/vfio_pasid.c +26
-0@@ -221,6 +221,8 @@ void vfio_pasid_free_range(struct vfio_mm *vmm,* IOASID core will notify PASID users (e.g. IOMMU driver) to * teardown necessary structures depending on the to-be-freed * PASID. +* Hold pasid_lock also avoids race with PASID usages like bind/ +* unbind page tables to requested PASID. */ mutex_lock (&vmm->pasid_lock);while ((vid =vfio_find_pasid (vmm, min, max)) !=NULL )@@ -229,6 +231,30 @@ void vfio_pasid_free_range(struct vfio_mm *vmm, }EXPORT_SYMBOL_GPL (vfio_pasid_free_range); +int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)) +{ +int ret; + +mutex_lock (&vmm->pasid_lock); + ret =ioasid_set_for_each_ioasid (vmm->ioasid_sid, fn, data); +mutex_unlock (&vmm->pasid_lock); +return ret; +} +EXPORT_SYMBOL_GPL (vfio_mm_for_each_pasid); + +void vfio_mm_pasid_lock (struct vfio_mm *vmm) +{ +mutex_lock (&vmm->pasid_lock); +} +EXPORT_SYMBOL_GPL (vfio_mm_pasid_lock); + +void vfio_mm_pasid_unlock (struct vfio_mm *vmm) +{ +mutex_unlock (&vmm->pasid_lock); +} +EXPORT_SYMBOL_GPL (vfio_mm_pasid_unlock); +static int __init vfio_pasid_init (void ) {mutex_init (&vfio_mm_lock);include/linux/vfio.h +5
-0@@ -105,6 +105,11 @@ extern int vfio_mm_ioasid_sid(struct vfio_mm *vmm);extern int vfio_pasid_alloc (struct vfio_mm *vmm,int min,int max);extern void vfio_pasid_free_range (struct vfio_mm *vmm,ioasid_t min,ioasid_t max); +extern int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)); +extern void vfio_mm_pasid_lock (struct vfio_mm *vmm); +extern void vfio_mm_pasid_unlock (struct vfio_mm *vmm); +#else static inline struct vfio_mm *vfio_mm_get_from_task (struct task_struct *task) {@@ -129,6 +134,21 @@ static inline void vfio_pasid_free_range(struct vfio_mm*vmm,
Quoted 29 lines
include/linux/vfio.h +15
-0ioasid_t min, ioasid_t max) { } + +static inline int vfio_mm_for_each_pasid (struct vfio_mm *vmm,void *data, +void (*fn)(ioasid_t id,void *data)) +{ +return -ENOTTY ; +} + +static inline void vfio_mm_pasid_lock (struct vfio_mm *vmm) +{ +} + +static inline void vfio_mm_pasid_unlock (struct vfio_mm *vmm) +{ +} +#endif /* CONFIG_VFIO_PASID */ /* include/uapi/linux/vfio.h +0
-0@@ -1209,6 +1209,37 @@ struct vfio_iommu_type1_pasid_request {#define VFIO_IOMMU_PASID_REQUEST _IO(VFIO_TYPE, VFIO_BASE + 18)
Quoted 9 lines
include/uapi/linux/vfio.h struct vfio_iommu_type1_pasid_request { · line 1218
Quoted 7 lines
include/uapi/linux/vfio.h +7
-0+/** +* VFIO_IOMMU_NESTING_OP - _IOW(VFIO_TYPE, VFIO_BASE + 19, +* struct vfio_iommu_type1_nesting_op) +* +* This interface allows userspace to utilize the nesting IOMMU +* capabilities as reported in VFIO_IOMMU_TYPE1_INFO_CAP_NESTING +* cap through VFIO_IOMMU_GET_INFO. PASID alloc/free are handled through another IOCTL so you should be more precise I think.
yep. will add it.
Regards,
Yi Liu
Quoted 28 lines
include/uapi/linux/vfio.h struct vfio_iommu_type1_pasid_request { · line 1228
Quoted 3 lines
include/uapi/linux/vfio.h +3
-0+* +* @data[] types defined for each op: +* +=================+===============================================+ > + * | NESTING OP | @data[] |include/uapi/linux/vfio.h struct vfio_iommu_type1_pasid_request { · line 1222
Quoted 1 line
include/uapi/linux/vfio.h +1
-0+* +=================+===============================================+ > + * | BIND_PGTBL | struct iommu_gpasid_bind_data |Quoted 20 lines
include/uapi/linux/vfio.h +18
-0+* +-----------------+-----------------------------------------------+ +* | UNBIND_PGTBL | struct iommu_gpasid_bind_data | +* +-----------------+-----------------------------------------------+ +* +* returns: 0 on success, -errno on failure. +*/ +struct vfio_iommu_type1_nesting_op { +__u32 argsz; +__u32 flags; +#define VFIO_NESTING_OP_MASK (0xffff)/* lower 16-bits for op */ +__u8 data[]; +}; + +#define VFIO_IOMMU_NESTING_OP_BIND_PGTBL (0) +#define VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL (1) + +#define VFIO_IOMMU_NESTING_OP _IO(VFIO_TYPE, VFIO_BASE + 19) +/* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */
Quoted 5 lines
include/uapi/linux/vfio.h struct vfio_iommu_type1_pasid_request { · line 1243
Quoted 2 lines
include/uapi/linux/vfio.h +0
-0/* Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 11/15 vfio/type1: Allow invalidating first-level/stage IOMMU cache
In reply to Liu Yi LThis patch provides an interface allowing the userspace to invalidate IOMMU cache for first-level page table. It is required when the first level IOMMU page table is not managed by the host kernel in the nested translation setup.
---
v1 -> v2:
*) rename from "vfio/type1: Flush stage-1 IOMMU cache for nesting type" *) rename vfio_cache_inv_fn() to vfio_dev_cache_invalidate_fn() *) vfio_dev_cache_inv_fn() always successful
*) remove VFIO_IOMMU_CACHE_INVALIDATE, and reuse VFIO_IOMMU_NESTING_OP
--- drivers/vfio/vfio_iommu_type1.c | 42 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 3 +++ 2 files changed, 45 insertions(+)
drivers/vfio/vfio_iommu_type1.c +42 -0
@@ -3056,6 +3056,45 @@ static long vfio_iommu_handle_pgtbl_op(struct vfio_iommu *iommu,return ret; } +static int vfio_dev_cache_invalidate_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +iommu_uapi_cache_invalidate (dc->domain, dev, (void __user *)arg); +return 0 ; +} + +static long vfio_iommu_invalidate_cache (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct domain_capsule dc = { .data = &arg }; +struct iommu_nesting_info *info; +int ret; + +mutex_lock (&iommu->lock); +/* +* Cache invalidation is required for any nesting IOMMU, +* so no need to check system-wide PASID support. +*/ + info = iommu->nesting_info; +if (!info || !(info->features &IOMMU_NESTING_FEAT_CACHE_INVLD )) { + ret = -EOPNOTSUPP ; + goto out_unlock; + } + + ret =vfio_get_nesting_domain_capsule (iommu, &dc); +if (ret) + goto out_unlock; + +iommu_group_for_each_dev (dc.group->iommu_group, &dc, + vfio_dev_cache_invalidate_fn); + +out_unlock: +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_nesting_op (struct vfio_iommu *iommu,unsigned long arg) {
@@ -3078,6 +3117,9 @@ static long vfio_iommu_type1_nesting_op(struct vfio_iommu *iommu,case VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL : ret =vfio_iommu_handle_pgtbl_op (iommu, false, arg + minsz);break ; +case VFIO_IOMMU_NESTING_OP_CACHE_INVLD : + ret =vfio_iommu_invalidate_cache (iommu, arg + minsz); +break ;default : ret = -EINVAL ; }
include/uapi/linux/vfio.h +3 -0
@@ -1225,6 +1225,8 @@ struct vfio_iommu_type1_pasid_request {* +-----------------+-----------------------------------------------+ * | UNBIND_PGTBL | struct iommu_gpasid_bind_data | * +-----------------+-----------------------------------------------+ +* | CACHE_INVLD | struct iommu_cache_invalidate_info | +* +-----------------+-----------------------------------------------+ * * returns: 0 on success, -errno on failure. */
@@ -1237,6 +1239,7 @@ struct vfio_iommu_type1_nesting_op {#define VFIO_IOMMU_NESTING_OP_BIND_PGTBL (0)#define VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL (1) +#define VFIO_IOMMU_NESTING_OP_CACHE_INVLD (2)#define VFIO_IOMMU_NESTING_OP _IO(VFIO_TYPE, VFIO_BASE + 19)
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 11/15] vfio/type1: Allow invalidating first-level/stage IOMMU cache
Hi Yi,
Quoted 43 lines · Liu Yi L
This patch provides an interface allowing the userspace to invalidate IOMMU cache for first-level page table. It is required when the first level IOMMU page table is not managed by the host kernel in the nested translation setup.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Eric Auger <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v1 -> v2:
*) rename from "vfio/type1: Flush stage-1 IOMMU cache for nesting type" *) renamevfio_cache_inv_fn()tovfio_dev_cache_invalidate_fn()*)vfio_dev_cache_inv_fn()always successful
*) removeVFIO_IOMMU_CACHE_INVALIDATE, and reuseVFIO_IOMMU_NESTING_OP--- drivers/vfio/vfio_iommu_type1.c | 42 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 3 +++ 2 files changed, 45 insertions(+)drivers/vfio/vfio_iommu_type1.c +9
-0@@ -3056,6 +3056,45 @@ static long vfio_iommu_handle_pgtbl_op(struct vfio_iommu *iommu,return ret; } +static int vfio_dev_cache_invalidate_fn (struct device *dev,void *data) +{ +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +iommu_uapi_cache_invalidate (dc->domain, dev, (void __user *)arg); +return 0 ; +} +
drivers/vfio/vfio_iommu_type1.c static long vfio_iommu_handle_pgtbl_op(struct vfio_iommu *iommu, · line 3077
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +10
-0+static long vfio_iommu_invalidate_cache (struct vfio_iommu *iommu, +unsigned long arg) +{ +struct domain_capsule dc = { .data = &arg }; +struct iommu_nesting_info *info; +int ret; + +mutex_lock (&iommu->lock); +/* +* Cache invalidation is required for any nesting IOMMU,
So why do we expose the IOMMU_NESTING_FEAT_CACHE_INVLD capability? :-)
Quoted 47 lines
drivers/vfio/vfio_iommu_type1.c +23
-0+* so no need to check system-wide PASID support. +*/ + info = iommu->nesting_info; +if (!info || !(info->features &IOMMU_NESTING_FEAT_CACHE_INVLD )) { + ret = -EOPNOTSUPP ; + goto out_unlock; + } + + ret =vfio_get_nesting_domain_capsule (iommu, &dc); +if (ret) + goto out_unlock; + +iommu_group_for_each_dev (dc.group->iommu_group, &dc, + vfio_dev_cache_invalidate_fn); + +out_unlock: +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_nesting_op (struct vfio_iommu *iommu,unsigned long arg) {@@ -3078,6 +3117,9 @@ static long vfio_iommu_type1_nesting_op(struct vfio_iommu *iommu,case VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL : ret =vfio_iommu_handle_pgtbl_op (iommu, false, arg + minsz);break ; +case VFIO_IOMMU_NESTING_OP_CACHE_INVLD : + ret =vfio_iommu_invalidate_cache (iommu, arg + minsz); +break ;default : ret = -EINVAL ; }include/uapi/linux/vfio.h +2
-0@@ -1225,6 +1225,8 @@ struct vfio_iommu_type1_pasid_request {* +-----------------+-----------------------------------------------+ * | UNBIND_PGTBL | struct iommu_gpasid_bind_data | * +-----------------+-----------------------------------------------+ +* | CACHE_INVLD | struct iommu_cache_invalidate_info | +* +-----------------+-----------------------------------------------+ * * returns: 0 on success, -errno on failure. */
include/uapi/linux/vfio.h struct vfio_iommu_type1_nesting_op { · line 1242
Quoted 4 lines
include/uapi/linux/vfio.h +1
-0@@ -1237,6 +1239,7 @@ struct vfio_iommu_type1_nesting_op {#define VFIO_IOMMU_NESTING_OP_BIND_PGTBL (0)#define VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL (1) +#define VFIO_IOMMU_NESTING_OP_CACHE_INVLD (2)
According to my previous comment, you may refine VFIO_NESTING_OP_MASK too
Thanks
Eric
include/uapi/linux/vfio.h struct vfio_iommu_type1_nesting_op {
Quoted 4 lines
include/uapi/linux/vfio.h +0
-0#define VFIO_IOMMU_NESTING_OP _IO(VFIO_TYPE, VFIO_BASE + 19)
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 11/15] vfio/type1: Allow invalidating first-level/stage IOMMU cache
Hi Eric,
Quoted 55 lines
From: Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 7:35 PMHi Yi,
Quoted 15 lines · Liu Yi L
This patch provides an interface allowing the userspace to invalidate IOMMU cache for first-level page table. It is required when the first level IOMMU page table is not managed by the host kernel in the nested translation setup.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Eric Auger <[email protected]>Signed-off-by Jacob Pan <[email protected]>Quoted 6 lines
---
v1 -> v2:
*) rename from "vfio/type1: Flush stage-1 IOMMU cache for nesting type" *) renamevfio_cache_inv_fn()tovfio_dev_cache_invalidate_fn()*)vfio_dev_cache_inv_fn()always successful
*) removeVFIO_IOMMU_CACHE_INVALIDATE, and reuse
VFIO_IOMMU_NESTING_OPQuoted 2 lines
--- drivers/vfio/vfio_iommu_type1.c | 42+++++++++++++++++++++++++++++++++++++++++
drivers/vfio/vfio_iommu_type1.c static long vfio_iommu_handle_pgtbl_op(struct
Quoted 8 lines
include/uapi/linux/vfio.h | 3 +++ 2 files changed, 45 insertions(+)(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex 245436e..bf95a0f 100644drivers/vfio/vfio_iommu_type1.c +0
-0@@ -3056,6 +3056,45 @@ static long vfio_iommu_handle_pgtbl_op(struct
vfio_iommu*iommu,Quoted 15 lines
drivers/vfio/vfio_iommu_type1.c +12
-0return ret; } +static int vfio_dev_cache_invalidate_fn (struct device *dev,void +*data) { +struct domain_capsule *dc = (struct domain_capsule *)data; +unsigned long arg = *(unsigned long *)dc->data; + +iommu_uapi_cache_invalidate (dc->domain, dev, (void __user *)arg); +return 0 ; +} + +static long vfio_iommu_invalidate_cache (struct vfio_iommu *iommu, +unsigned long arg) +{
Quoted 8 lines
drivers/vfio/vfio_iommu_type1.c static long vfio_iommu_handle_pgtbl_op(struct vfio_iommu *iommu, · line 3077
Quoted 7 lines
drivers/vfio/vfio_iommu_type1.c +7
-0+struct domain_capsule dc = { .data = &arg }; +struct iommu_nesting_info *info; +int ret; + +mutex_lock (&iommu->lock); +/* +* Cache invalidation is required for any nesting IOMMU, So why do we expose the
IOMMU_NESTING_FEAT_CACHE_INVLDcapability? :-)
it's a stale comment. should be removed. :-)
Quoted 51 lines
drivers/vfio/vfio_iommu_type1.c static long vfio_iommu_type1_nesting_op(struct
Quoted 24 lines
drivers/vfio/vfio_iommu_type1.c +20
-0+* so no need to check system-wide PASID support. +*/ + info = iommu->nesting_info; +if (!info || !(info->features &IOMMU_NESTING_FEAT_CACHE_INVLD )) { + ret = -EOPNOTSUPP ; + goto out_unlock; + } + + ret =vfio_get_nesting_domain_capsule (iommu, &dc); +if (ret) + goto out_unlock; + +iommu_group_for_each_dev (dc.group->iommu_group, &dc, + vfio_dev_cache_invalidate_fn); + +out_unlock: +mutex_unlock (&iommu->lock); +return ret; +} +static long vfio_iommu_type1_nesting_op (struct vfio_iommu *iommu,unsigned long arg) {@@ -3078,6 +3117,9 @@ static long vfio_iommu_type1_nesting_op(struct
vfio_iommu*iommu,Quoted 26 lines
drivers/vfio/vfio_iommu_type1.c +3
-0case VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL : ret =vfio_iommu_handle_pgtbl_op (iommu, false, arg + minsz);break ; +case VFIO_IOMMU_NESTING_OP_CACHE_INVLD : + ret =vfio_iommu_invalidate_cache (iommu, arg + minsz); +break ;default : ret = -EINVAL ; }include/uapi/linux/vfio.h +4
-0@@ -1225,6 +1225,8 @@ struct vfio_iommu_type1_pasid_request {* +-----------------+-----------------------------------------------+ * | UNBIND_PGTBL | struct iommu_gpasid_bind_data | * +-----------------+-----------------------------------------------+ +* | CACHE_INVLD | struct iommu_cache_invalidate_info | +* ++-----------------+-----------------------------------------------+ * * returns: 0 on success, -errno on failure. */ @@ -1237,6 +1239,7 @@ struct vfio_iommu_type1_nesting_op {
Quoted 4 lines
include/uapi/linux/vfio.h struct vfio_iommu_type1_nesting_op { · line 1242
Quoted 3 lines
include/uapi/linux/vfio.h +1
-0#define VFIO_IOMMU_NESTING_OP_BIND_PGTBL (0)#define VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL (1) +#define VFIO_IOMMU_NESTING_OP_CACHE_INVLD (2)According to my previous comment, you may refine
VFIO_NESTING_OP_MASKtoo
yes, I've noticed it. also replied in patch 10/15.
Regards,
Yi Liu
Quoted 7 lines
Thanks
Eric
Quoted 4 lines
include/uapi/linux/vfio.h +0
-0#define VFIO_IOMMU_NESTING_OP _IO(VFIO_TYPE, VFIO_BASE + 19)
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 12/15 vfio/type1: Add vSVA support for IOMMU-backed mdevs
In reply to Liu Yi LRecent years, mediated device pass-through framework (e.g. vfio-mdev) is used to achieve flexible device sharing across domains (e.g. VMs). Also there are hardware assisted mediated pass-through solutions from platform vendors. e.g. Intel VT-d scalable mode which supports Intel Scalable I/O Virtualization technology. Such mdevs are called IOMMU- backed mdevs as there are IOMMU enforced DMA isolation for such mdevs. In kernel, IOMMU-backed mdevs are exposed to IOMMU layer by aux-domain concept, which means mdevs are protected by an iommu domain which is auxiliary to the domain that the kernel driver primarily uses for DMA API. Details can be found in the KVM presentation as below:
https://events19.linuxfoundation.org/wp-content/uploads/2017/12/\
Hardware-Assisted-Mediated-Pass-Through-with-VFIO-Kevin-Tian-Intel.pdf
This patch extends NESTING_IOMMU ops to IOMMU-backed mdev devices. The main requirement is to use the auxiliary domain associated with mdev.
---
v5 -> v6:
*) add review-by from Eric Auger.
v1 -> v2:
*) check the iommu_device to ensure the handling mdev is IOMMU-backed
--- drivers/vfio/vfio_iommu_type1.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-)
drivers/vfio/vfio_iommu_type1.c +36 -4
@@ -2379,20 +2379,41 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu,return ret; } +static struct device *vfio_get_iommu_device (struct vfio_group *group, +struct device *dev) +{ +if (group->mdev_group) +return vfio_mdev_get_iommu_device (dev); +else +return dev; +} +static int vfio_dev_bind_gpasid_fn (struct device *dev,void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -return iommu_uapi_sva_bind_gpasid (dc->domain, dev, (void __user *)arg); +return iommu_uapi_sva_bind_gpasid (dc->domain, iommu_device, + (void __user *)arg); }static int vfio_dev_unbind_gpasid_fn (struct device *dev,void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; -iommu_uapi_sva_unbind_gpasid (dc->domain, dev, (void __user *)arg); + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; + +iommu_uapi_sva_unbind_gpasid (dc->domain, iommu_device, + (void __user *)arg);return 0 ; }
@@ -2401,8 +2422,13 @@ static int __vfio_dev_unbind_gpasid_fn(struct device *dev, void *data)struct domain_capsule *dc = (struct domain_capsule *)data;struct iommu_gpasid_bind_data *unbind_data = (struct iommu_gpasid_bind_data *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -iommu_sva_unbind_gpasid (dc->domain, dev, unbind_data); +iommu_sva_unbind_gpasid (dc->domain, iommu_device, unbind_data);return 0 ; }
@@ -3060,8 +3086,14 @@ static int vfio_dev_cache_invalidate_fn(struct device *dev, void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -iommu_uapi_cache_invalidate (dc->domain, dev, (void __user *)arg); +iommu_uapi_cache_invalidate (dc->domain, iommu_device, + (void __user *)arg);return 0 ; }
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 12/15] vfio/type1: Add vSVA support for IOMMU-backed mdevs
Quoted 7 lines · On Mon, 27 Jul 2020 23:27:41 -0700 Liu Yi L <[email protected]> wrote:
Recent years, mediated device pass-through framework (e.g. vfio-mdev) is used to achieve flexible device sharing across domains (e.g. VMs). Also there are hardware assisted mediated pass-through solutions from platform vendors. e.g. Intel VT-d scalable mode which supports Intel Scalable I/O Virtualization technology. Such mdevs are called IOMMU- backed mdevs as there are IOMMU enforced DMA isolation for such mdevs. In kernel, IOMMU-backed mdevs are exposed to IOMMU layer by aux-domain
Or a physical IOMMU backing device.
Quoted 6 lines
concept, which means mdevs are protected by an iommu domain which is auxiliary to the domain that the kernel driver primarily uses for DMA API. Details can be found in the KVM presentation as below:
https://events19.linuxfoundation.org/wp-content/uploads/2017/12/\
Hardware-Assisted-Mediated-Pass-Through-with-VFIO-Kevin-Tian-Intel.pdf
I think letting the line exceed 80 columns is preferable so that it's clickable. Thanks,
Alex
Quoted 92 lines
This patch extends
NESTING_IOMMUops to IOMMU-backed mdev devices. The main requirement is to use the auxiliary domain associated with mdev.Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>CC Jun Tian <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) add review-by from Eric Auger.v1 -> v2:
*) check theiommu_deviceto ensure the handling mdev is IOMMU-backed--- drivers/vfio/vfio_iommu_type1.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-)drivers/vfio/vfio_iommu_type1.c +31
-3@@ -2379,20 +2379,41 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu,return ret; } +static struct device *vfio_get_iommu_device (struct vfio_group *group, +struct device *dev) +{ +if (group->mdev_group) +return vfio_mdev_get_iommu_device (dev); +else +return dev; +} +static int vfio_dev_bind_gpasid_fn (struct device *dev,void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -return iommu_uapi_sva_bind_gpasid (dc->domain, dev, (void __user *)arg); +return iommu_uapi_sva_bind_gpasid (dc->domain, iommu_device, + (void __user *)arg); }static int vfio_dev_unbind_gpasid_fn (struct device *dev,void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; -iommu_uapi_sva_unbind_gpasid (dc->domain, dev, (void __user *)arg); + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; + +iommu_uapi_sva_unbind_gpasid (dc->domain, iommu_device, + (void __user *)arg);return 0 ; }@@ -2401,8 +2422,13 @@ static int __vfio_dev_unbind_gpasid_fn(struct device *dev, void *data)struct domain_capsule *dc = (struct domain_capsule *)data;struct iommu_gpasid_bind_data *unbind_data = (struct iommu_gpasid_bind_data *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -iommu_sva_unbind_gpasid (dc->domain, dev, unbind_data); +iommu_sva_unbind_gpasid (dc->domain, iommu_device, unbind_data);return 0 ; }@@ -3060,8 +3086,14 @@ static int vfio_dev_cache_invalidate_fn(struct device *dev, void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; +
drivers/vfio/vfio_iommu_type1.c static int vfio_dev_cache_invalidate_fn(struct device *dev, void *data) · line 3099
Quoted 10 lines
drivers/vfio/vfio_iommu_type1.c +5
-1+ iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -iommu_uapi_cache_invalidate (dc->domain, dev, (void __user *)arg); +iommu_uapi_cache_invalidate (dc->domain, iommu_device, + (void __user *)arg);return 0 ; }
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 12/15] vfio/type1: Add vSVA support for IOMMU-backed mdevs
Hi Alex,
Quoted 14 lines
From: Alex Williamson <[email protected]>
Sent: Friday, August 21, 2020 5:49 AMQuoted 7 lines · On Mon, 27 Jul 2020 23:27:41 -0700 Liu Yi L <[email protected]> wrote:
Recent years, mediated device pass-through framework (e.g. vfio-mdev) is used to achieve flexible device sharing across domains (e.g. VMs). Also there are hardware assisted mediated pass-through solutions from platform vendors. e.g. Intel VT-d scalable mode which supports Intel Scalable I/O Virtualization technology. Such mdevs are called IOMMU- backed mdevs as there are IOMMU enforced DMA isolation for such mdevs. In kernel, IOMMU-backed mdevs are exposed to IOMMU layer by aux-domain
Quoted 1 line
Or a physical IOMMU backing device.
got it. :-)
Quoted 8 lines
Quoted 6 lines
concept, which means mdevs are protected by an iommu domain which is auxiliary to the domain that the kernel driver primarily uses for DMA API. Details can be found in the KVM presentation as below:
https://events19.linuxfoundation.org/wp-content/uploads/2017/12/\
Hardware-Assisted-Mediated-Pass-Through-with-VFIO-Kevin-Tian-Intel.pdfI think letting the line exceed 80 columns is preferable so that it's clickable. Thanks,
yeah, it's clickable now. will do it. :-)
Thanks,
Yi Liu
Quoted 100 lines
Alex
drivers/vfio/vfio_iommu_type1.c static int vfio_iommu_resv_refresh(struct
Quoted 29 lines
This patch extends
NESTING_IOMMUops to IOMMU-backed mdev devices. The main requirement is to use the auxiliary domain associated with mdev.Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>CC Jun Tian <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) add review-by from Eric Auger.v1 -> v2:
*) check theiommu_deviceto ensure the handling mdev is IOMMU-backed--- drivers/vfio/vfio_iommu_type1.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-)(quoted diff) +0
-0
b/drivers/vfio/vfio_iommu_type1.cindex bf95a0f..9d8f252 100644drivers/vfio/vfio_iommu_type1.c +0
-0@@ -2379,20 +2379,41 @@ static int vfio_iommu_resv_refresh(struct
vfio_iommu*iommu,drivers/vfio/vfio_iommu_type1.c static int __vfio_dev_unbind_gpasid_fn(struct device
Quoted 43 lines
drivers/vfio/vfio_iommu_type1.c +23
-2return ret; } +static struct device *vfio_get_iommu_device (struct vfio_group *group, +struct device *dev) +{ +if (group->mdev_group) +return vfio_mdev_get_iommu_device (dev); +else +return dev; +} +static int vfio_dev_bind_gpasid_fn (struct device *dev,void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -return iommu_uapi_sva_bind_gpasid (dc->domain, dev, (void __user *)arg); +return iommu_uapi_sva_bind_gpasid (dc->domain, iommu_device, + (void __user *)arg); }static int vfio_dev_unbind_gpasid_fn (struct device *dev,void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; -iommu_uapi_sva_unbind_gpasid (dc->domain, dev, (void __user *)arg); + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; + +iommu_uapi_sva_unbind_gpasid (dc->domain, iommu_device, + (void __user *)arg);return 0 ; }@@ -2401,8 +2422,13 @@ static int __vfio_dev_unbind_gpasid_fn(struct device*dev, void *data)
Quoted 24 lines
drivers/vfio/vfio_iommu_type1.c +11
-1struct domain_capsule *dc = (struct domain_capsule *)data;struct iommu_gpasid_bind_data *unbind_data = (struct iommu_gpasid_bind_data *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ; -iommu_sva_unbind_gpasid (dc->domain, dev, unbind_data); +iommu_sva_unbind_gpasid (dc->domain, iommu_device, unbind_data);return 0 ; }@@ -3060,8 +3086,14 @@ static int vfio_dev_cache_invalidate_fn(struct device *dev, void *data) {struct domain_capsule *dc = (struct domain_capsule *)data;unsigned long arg = *(unsigned long *)dc->data; +struct device *iommu_device; + + iommu_device =vfio_get_iommu_device (dc->group, dev); +if (!iommu_device) +return -EINVAL ;
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c static int vfio_dev_cache_invalidate_fn(struct device *dev, void *data) · line 3099
Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +2
-1-iommu_uapi_cache_invalidate (dc->domain, dev, (void __user *)arg); +iommu_uapi_cache_invalidate (dc->domain, iommu_device, + (void __user *)arg);return 0 ; }
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 13/15 vfio/pci: Expose PCIe PASID capability to guest
In reply to Liu Yi LThis patch exposes PCIe PASID capability to guest for assigned devices. Existing vfio_pci driver hides it from guest by setting the capability length as 0 in pci_ext_cap_length[].
And this patch only exposes PASID capability for devices which has PCIe PASID extended struture in its configuration space. VFs will not expose the PASID capability as they do not implement the PASID extended structure in their config space. It is a TODO in future. Related discussion can be found in below link:
https://lore.kernel.org/kvm/[email protected]/
---
v5 -> v6:
*) add review-by from Eric Auger.
v1 -> v2:
*) added in v2, but it was sent in a separate patchseries before
--- drivers/vfio/pci/vfio_pci_config.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/pci/vfio_pci_config.c +1 -1
@@ -95,7 +95,7 @@ static const u16 pci_ext_cap_length[PCI_EXT_CAP_ID_MAX + 1] = { [PCI_EXT_CAP_ID_LTR] = PCI_EXT_CAP_LTR_SIZEOF, [PCI_EXT_CAP_ID_SECPCI] = 0, /* not yet */ [PCI_EXT_CAP_ID_PMUX] = 0, /* not yet */ - [PCI_EXT_CAP_ID_PASID] = 0, /* not yet */ + [PCI_EXT_CAP_ID_PASID] = PCI_EXT_CAP_PASID_SIZEOF, };/*
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 14/15 vfio: Document dual stage control
In reply to Liu Yi LFrom: Eric Auger <[email protected]>
The VFIO API was enhanced to support nested stage control: a bunch of new ioctls and usage guideline.
Let's document the process to follow to set up nested mode.
---
v5 -> v6:
*) tweak per Eric's comments.
v3 -> v4:
*) add review-by from Stefan Hajnoczi
v2 -> v3:
*) address comments from Stefan Hajnoczi
v1 -> v2:
*) new in v2, compared with Eric's original version, pasid table bind
and fault reporting is removed as this series doesn't cover them. Original version from Eric.
https://lkml.org/lkml/2020/3/20/700
--- Documentation/driver-api/vfio.rst | 75 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+)
Documentation/driver-api/vfio.rst +75 -0
@@ -239,6 +239,81 @@ group and can access them as follows:: /* Gratuitous device reset and go... */ ioctl(device, VFIO_DEVICE_RESET); +IOMMU Dual Stage Control +------------------------ + +Some IOMMUs support 2 stages/levels of translation. Stage corresponds +to the ARM terminology while level corresponds to Intel's terminology. +In the following text we use either without distinction. + +This is useful when the guest is exposed with a virtual IOMMU and some +devices are assigned to the guest through VFIO. Then the guest OS can +use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor uses stage +2 for VM isolation (GPA -> HPA). + +Under dual stage translation, the guest gets ownership of the stage-1 page +tables and also owns stage-1 configuration structures. The hypervisor owns +the root configuration structure (for security reason), including stage-2 +configuration. This works as long as configuration structures and page table +formats are compatible between the virtual IOMMU and the physical IOMMU. + +Assuming the HW supports it, this nested mode is selected by choosing the +VFIO_TYPE1_NESTING_IOMMU type through: + + ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); + +This forces the hypervisor to use the stage-2, leaving stage-1 available +for guest usage. The stage-1 format and binding method are vendor specific +and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) through +VFIO_IOMMU_GET_INFO: + + ioctl(container->fd, VFIO_IOMMU_GET_INFO, &nesting_info); + +The nesting cap info is available only after NESTING_IOMMU is selected. +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails and +userspace should try other IOMMU types. Details of the nesting cap info +can be found in Documentation/userspace-api/iommu.rst. + +The stage-1 page table can be bound to the IOMMU in two methods: directly +or indirectly. Direct binding requires userspace to notify VFIO of every +guest stage-1 page table binding, while indirect binding allows userspace +to bind once with an intermediate structure (e.g. PASID table) which +indirectly links to guest stage-1 page tables. The actual binding method +depends on IOMMU vendor. Currently only the direct binding capability ( +IOMMU_NESTING_FEAT_BIND_PGTBL) is supported: + + nesting_op->flags = VFIO_IOMMU_NESTING_OP_BIND_PGTBL; + memcpy(&nesting_op->data, &bind_data, sizeof(bind_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +When multiple stage-1 page tables are supported on a device, each page +table is associated with a PASID (Process Address Space ID) to differentiate +with each other. In such case, userspace should include PASID in the +bind_data when issuing direct binding request. + +PASID could be managed per-device or system-wide which, again, depends on +IOMMU vendor and is reported in nesting cap info. When system-wide policy +is reported (IOMMU_NESTING_FEAT_SYSWIDE_PASID), e.g. as by Intel platforms, +userspace *must* allocate PASID from VFIO before attempting binding of +stage-1 page table: + + req.flags = VFIO_IOMMU_ALLOC_PASID; + ioctl(container, VFIO_IOMMU_PASID_REQUEST, &req); + +Once the stage-1 page table is bound to the IOMMU, the guest is allowed to +fully manage its mapping at its disposal. The IOMMU walks nested stage-1 +and stage-2 page tables when serving DMA requests from assigned device, and +may cache the stage-1 mapping in the IOTLB. When required (IOMMU_NESTING_ +FEAT_CACHE_INVLD), userspace *must* forward guest stage-1 invalidation to +the host, so the IOTLB is invalidated: + + nesting_op->flags = VFIO_IOMMU_NESTING_OP_CACHE_INVLD; + memcpy(&nesting_op->data, &cache_inv_data, sizeof(cache_inv_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +Forwarded invalidations can happen at various granularity levels (page +level, context level, etc.) + VFIO User API -------------------------------------------------------------------------------
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 14/15] vfio: Document dual stage control
Hi Yi,
Quoted 49 lines · Liu Yi L
From: Eric Auger <[email protected]>
The VFIO API was enhanced to support nested stage control: a bunch of> new ioctls and usage guideline.
Let's document the process to follow to set up nested mode.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>Signed-off-by Eric Auger <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) tweak per Eric's comments.v3 -> v4:
*) add review-by from Stefan Hajnocziv2 -> v3:
*) address comments from Stefan Hajnocziv1 -> v2:
*) new in v2, compared with Eric's original version, pasid table bind
and fault reporting is removed as this series doesn't cover them. Original version from Eric.
https://lkml.org/lkml/2020/3/20/700--- Documentation/driver-api/vfio.rst | 75 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+)Documentation/driver-api/vfio.rst +6
-0@@ -239,6 +239,81 @@ group and can access them as follows:: /* Gratuitous device reset and go... */ ioctl(device, VFIO_DEVICE_RESET); +IOMMU Dual Stage Control +------------------------ + +Some IOMMUs support 2 stages/levels of translation. Stage corresponds +to the ARM terminology while level corresponds to Intel's terminology. +In the following text we use either without distinction.
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 257
Quoted 10 lines
Documentation/driver-api/vfio.rst +10
-0+ +This is useful when the guest is exposed with a virtual IOMMU and some +devices are assigned to the guest through VFIO. Then the guest OS can +use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor uses stage +2 for VM isolation (GPA -> HPA). + +Under dual stage translation, the guest gets ownership of the stage-1 page +tables and also owns stage-1 configuration structures. The hypervisor owns +the root configuration structure (for security reason), including stage-2 +configuration.
This is only true for vtd. On ARM the stage2 cfg is the Context Descriptor table (aka PASID table). root cfg only store the GPA of the CD table.
This works as long as configuration structures and page table
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 266
Quoted 9 lines
Documentation/driver-api/vfio.rst +9
-0+formats are compatible between the virtual IOMMU and the physical IOMMU. + +Assuming the HW supports it, this nested mode is selected by choosing the +VFIO_TYPE1_NESTING_IOMMU type through: + + ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); + +This forces the hypervisor to use the stage-2, leaving stage-1 available +for guest usage. The stage-1 format and binding method are vendor specific
. There are reported in the nesting capability ...
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 273
Quoted 7 lines
Documentation/driver-api/vfio.rst +7
-0+and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) through +VFIO_IOMMU_GET_INFO: + + ioctl(container->fd, VFIO_IOMMU_GET_INFO, &nesting_info); + +The nesting cap info is available only after NESTING_IOMMU is selected. +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails and
If the underlying
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 4 lines
Documentation/driver-api/vfio.rst +4
-0+userspace should try other IOMMU types. Details of the nesting cap info +can be found in Documentation/userspace-api/iommu.rst. + +The stage-1 page table can be bound to the IOMMU in two methods: directly> +or indirectly. Direct binding requires userspace to notify VFIO of every
Not sure we shall use this direct/indirect terminology. I don't think this is part of either ARM or Intel SPEC.
Suggestion: On Intel, the stage1 page table info are mediated by the userspace for each PASID. On ARM, the userspace directly passes the GPA of the whole PASID table. Currently only Intel's binding is supported.
Quoted 32 lines
Documentation/driver-api/vfio.rst +31
-0+guest stage-1 page table binding, while indirect binding allows userspace +to bind once with an intermediate structure (e.g. PASID table) which +indirectly links to guest stage-1 page tables. The actual binding method +depends on IOMMU vendor. Currently only the direct binding capability ( +IOMMU_NESTING_FEAT_BIND_PGTBL) is supported: + + nesting_op->flags = VFIO_IOMMU_NESTING_OP_BIND_PGTBL; + memcpy(&nesting_op->data, &bind_data, sizeof(bind_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +When multiple stage-1 page tables are supported on a device, each page +table is associated with a PASID (Process Address Space ID) to differentiate +with each other. In such case, userspace should include PASID in the +bind_data when issuing direct binding request. + +PASID could be managed per-device or system-wide which, again, depends on +IOMMU vendor and is reported in nesting cap info. When system-wide policy +is reported (IOMMU_NESTING_FEAT_SYSWIDE_PASID), e.g. as by Intel platforms, +userspace *must* allocate PASID from VFIO before attempting binding of +stage-1 page table: + + req.flags = VFIO_IOMMU_ALLOC_PASID; + ioctl(container, VFIO_IOMMU_PASID_REQUEST, &req); + +Once the stage-1 page table is bound to the IOMMU, the guest is allowed to +fully manage its mapping at its disposal. The IOMMU walks nested stage-1 +and stage-2 page tables when serving DMA requests from assigned device, and +may cache the stage-1 mapping in the IOTLB. When required (IOMMU_NESTING_ +FEAT_CACHE_INVLD), userspace *must* forward guest stage-1 invalidation to +the host, so the IOTLB is invalidated: +
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 319
Quoted 10 lines
Documentation/driver-api/vfio.rst +7
-0+ nesting_op->flags = VFIO_IOMMU_NESTING_OP_CACHE_INVLD; + memcpy(&nesting_op->data, &cache_inv_data, sizeof(cache_inv_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +Forwarded invalidations can happen at various granularity levels (page +level, context level, etc.) + VFIO User API -------------------------------------------------------------------------------
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 14/15] vfio: Document dual stage control
Hi Eric,
Quoted 62 lines
From: Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 7:52 PMHi Yi,
Quoted 3 lines · Liu Yi L
From: Eric Auger <[email protected]>
The VFIO API was enhanced to support nested stage control: a bunch of> new
ioctls and usage guideline.
Quoted 23 lines
Let's document the process to follow to set up nested mode.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>Signed-off-by Eric Auger <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) tweak per Eric's comments.v3 -> v4:
*) add review-by from Stefan Hajnocziv2 -> v3:
*) address comments from Stefan HajnocziQuoted 7 lines
v1 -> v2:
*) new in v2, compared with Eric's original version, pasid table bind
and fault reporting is removed as this series doesn't cover them. Original version from Eric.
https://lkml.org/lkml/2020/3/20/700--- Documentation/driver-api/vfio.rst | 75+++++++++++++++++++++++++++++++++++++++
Quoted 21 lines
1 file changed, 75 insertions(+)Documentation/driver-api/vfio.rst +11
-0@@ -239,6 +239,81 @@ group and can access them as follows:: /* Gratuitous device reset and go... */ ioctl(device, VFIO_DEVICE_RESET); +IOMMU Dual Stage Control +------------------------ + +Some IOMMUs support 2 stages/levels of translation. Stage corresponds +to the ARM terminology while level corresponds to Intel's terminology. +In the following text we use either without distinction. + +This is useful when the guest is exposed with a virtual IOMMU and some +devices are assigned to the guest through VFIO. Then the guest OS can +use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor uses stage +2 for VM isolation (GPA -> HPA).
Quoted 8 lines
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 5 lines
Documentation/driver-api/vfio.rst +5
-0+ +Under dual stage translation, the guest gets ownership of the stage-1 page +tables and also owns stage-1 configuration structures. The hypervisor owns +the root configuration structure (for security reason), including stage-2 +configuration.This is only true for vtd. On ARM the stage2 cfg is the Context Descriptor table (aka PASID table). root cfg only store the GPA of the CD table.
I've a check with you on the meaning of "configuration structures". For Vt-d, does it mean the root table/context table/pasid table? if I'm correct, then how about below description?
"Under dual stage translation, the guest gets ownership of the stage-1 configuration structures or page tables. This depends on vendor. The hypervisor owns the root configuration structure (for security reason), including stage-2 configuration."
Quoted 11 lines
This works as long as configuration structures and page table
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 266
Quoted 9 lines
Documentation/driver-api/vfio.rst +9
-0+formats are compatible between the virtual IOMMU and the physical IOMMU. + +Assuming the HW supports it, this nested mode is selected by choosing the +VFIO_TYPE1_NESTING_IOMMU type through: + + ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); + +This forces the hypervisor to use the stage-2, leaving stage-1 available +for guest usage. The stage-1 format and binding method are vendor specific. There are reported in the nesting capability ...
got it.
"The stage-1 format and binding method are reported in nesting capability."
Quoted 8 lines
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 273
Quoted 7 lines
Documentation/driver-api/vfio.rst +7
-0+and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) through +VFIO_IOMMU_GET_INFO: + + ioctl(container->fd, VFIO_IOMMU_GET_INFO, &nesting_info); + +The nesting cap info is available only after NESTING_IOMMU is selected. +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails andIf the underlying
got it.
Quoted 11 lines
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 4 lines
Documentation/driver-api/vfio.rst +4
-0+userspace should try other IOMMU types. Details of the nesting cap info +can be found in Documentation/userspace-api/iommu.rst. + +The stage-1 page table can be bound to the IOMMU in two methods: directly>+or indirectly. Direct binding requires userspace to notify VFIO of every Not sure we shall use this direct/indirect terminology. I don't think this is part of either ARM or Intel SPEC.
Suggestion: On Intel, the stage1 page table info are mediated by the userspace for each PASID. On ARM, the userspace directly passes the GPA of the whole PASID table. Currently only Intel's binding is supported.
got it. this is what we want to say by ditect/indirect terminology.
Regards,
Yi Liu
Quoted 44 lines
Quoted 32 lines
Documentation/driver-api/vfio.rst +31
-0+guest stage-1 page table binding, while indirect binding allows userspace +to bind once with an intermediate structure (e.g. PASID table) which +indirectly links to guest stage-1 page tables. The actual binding method +depends on IOMMU vendor. Currently only the direct binding capability ( +IOMMU_NESTING_FEAT_BIND_PGTBL) is supported: + + nesting_op->flags = VFIO_IOMMU_NESTING_OP_BIND_PGTBL; + memcpy(&nesting_op->data, &bind_data, sizeof(bind_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +When multiple stage-1 page tables are supported on a device, each page +table is associated with a PASID (Process Address Space ID) to differentiate +with each other. In such case, userspace should include PASID in the +bind_data when issuing direct binding request. + +PASID could be managed per-device or system-wide which, again, depends on +IOMMU vendor and is reported in nesting cap info. When system-wide policy +is reported (IOMMU_NESTING_FEAT_SYSWIDE_PASID), e.g. as by Intel platforms, +userspace *must* allocate PASID from VFIO before attempting binding of +stage-1 page table: + + req.flags = VFIO_IOMMU_ALLOC_PASID; + ioctl(container, VFIO_IOMMU_PASID_REQUEST, &req); + +Once the stage-1 page table is bound to the IOMMU, the guest is allowed to +fully manage its mapping at its disposal. The IOMMU walks nested stage-1 +and stage-2 page tables when serving DMA requests from assigned device, and +may cache the stage-1 mapping in the IOTLB. When required (IOMMU_NESTING_ +FEAT_CACHE_INVLD), userspace *must* forward guest stage-1 invalidation to +the host, so the IOTLB is invalidated: +Documentation/driver-api/vfio.rst group and can access them as follows:: · line 319
Quoted 10 lines
Documentation/driver-api/vfio.rst +7
-0+ nesting_op->flags = VFIO_IOMMU_NESTING_OP_CACHE_INVLD; + memcpy(&nesting_op->data, &cache_inv_data, sizeof(cache_inv_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +Forwarded invalidations can happen at various granularity levels (page +level, context level, etc.) + VFIO User API -------------------------------------------------------------------------------Thanks
Quoted 1 line
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 14/15] vfio: Document dual stage control
Hi Yi,
Quoted 73 lines · Yi L
Hi Eric,
Quoted 62 lines
From: Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 7:52 PMHi Yi,
Quoted 3 lines · Liu Yi L
From: Eric Auger <[email protected]>
The VFIO API was enhanced to support nested stage control: a bunch of> new
ioctls and usage guideline.
Quoted 23 lines
Let's document the process to follow to set up nested mode.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>Signed-off-by Eric Auger <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) tweak per Eric's comments.v3 -> v4:
*) add review-by from Stefan Hajnocziv2 -> v3:
*) address comments from Stefan HajnocziQuoted 7 lines
v1 -> v2:
*) new in v2, compared with Eric's original version, pasid table bind
and fault reporting is removed as this series doesn't cover them. Original version from Eric.
https://lkml.org/lkml/2020/3/20/700--- Documentation/driver-api/vfio.rst | 75+++++++++++++++++++++++++++++++++++++++
Quoted 21 lines
1 file changed, 75 insertions(+)Documentation/driver-api/vfio.rst +11
-0@@ -239,6 +239,81 @@ group and can access them as follows:: /* Gratuitous device reset and go... */ ioctl(device, VFIO_DEVICE_RESET); +IOMMU Dual Stage Control +------------------------ + +Some IOMMUs support 2 stages/levels of translation. Stage corresponds +to the ARM terminology while level corresponds to Intel's terminology. +In the following text we use either without distinction. + +This is useful when the guest is exposed with a virtual IOMMU and some +devices are assigned to the guest through VFIO. Then the guest OS can +use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor uses stage +2 for VM isolation (GPA -> HPA).Quoted 8 lines
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 5 lines
Documentation/driver-api/vfio.rst +5
-0+ +Under dual stage translation, the guest gets ownership of the stage-1 page +tables and also owns stage-1 configuration structures. The hypervisor owns +the root configuration structure (for security reason), including stage-2 +configuration.This is only true for vtd. On ARM the stage2 cfg is the Context Descriptor table (aka PASID table). root cfg only store the GPA of the CD table.
Quoted 3 lines
I've a check with you on the meaning of "configuration structures". For Vt-d, does it mean the root table/context table/pasid table? if I'm correct, then how about below description?
Yes I agree
Quoted 3 lines
"Under dual stage translation, the guest gets ownership of the stage-1 configuration structures or page tables.
Actually on ARM the guest both owns the S1 configuration (CD table) and S1 page tables ;-)
on Intel I understand the guest only owns the S1 page tables.
If confirmed, you may use such kind of explicit statement.
Thanks
Eric
This depends on vendor. The
Quoted 93 lines
hypervisor owns the root configuration structure (for security reason), including stage-2 configuration."
Quoted 11 lines
This works as long as configuration structures and page table
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 266
Quoted 9 lines
Documentation/driver-api/vfio.rst +9
-0+formats are compatible between the virtual IOMMU and the physical IOMMU. + +Assuming the HW supports it, this nested mode is selected by choosing the +VFIO_TYPE1_NESTING_IOMMU type through: + + ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); + +This forces the hypervisor to use the stage-2, leaving stage-1 available +for guest usage. The stage-1 format and binding method are vendor specific. There are reported in the nesting capability ...
got it.
"The stage-1 format and binding method are reported in nesting capability."
Quoted 8 lines
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 273
Quoted 7 lines
Documentation/driver-api/vfio.rst +7
-0+and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) through +VFIO_IOMMU_GET_INFO: + + ioctl(container->fd, VFIO_IOMMU_GET_INFO, &nesting_info); + +The nesting cap info is available only after NESTING_IOMMU is selected. +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails andIf the underlying
got it.
Quoted 11 lines
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 4 lines
Documentation/driver-api/vfio.rst +4
-0+userspace should try other IOMMU types. Details of the nesting cap info +can be found in Documentation/userspace-api/iommu.rst. + +The stage-1 page table can be bound to the IOMMU in two methods: directly>+or indirectly. Direct binding requires userspace to notify VFIO of every Not sure we shall use this direct/indirect terminology. I don't think this is part of either ARM or Intel SPEC.
Suggestion: On Intel, the stage1 page table info are mediated by the userspace for each PASID. On ARM, the userspace directly passes the GPA of the whole PASID table. Currently only Intel's binding is supported.
got it. this is what we want to say by ditect/indirect terminology.
Regards,
Yi LiuQuoted 45 lines
Quoted 32 lines
Documentation/driver-api/vfio.rst +31
-0+guest stage-1 page table binding, while indirect binding allows userspace +to bind once with an intermediate structure (e.g. PASID table) which +indirectly links to guest stage-1 page tables. The actual binding method +depends on IOMMU vendor. Currently only the direct binding capability ( +IOMMU_NESTING_FEAT_BIND_PGTBL) is supported: + + nesting_op->flags = VFIO_IOMMU_NESTING_OP_BIND_PGTBL; + memcpy(&nesting_op->data, &bind_data, sizeof(bind_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +When multiple stage-1 page tables are supported on a device, each page +table is associated with a PASID (Process Address Space ID) to differentiate +with each other. In such case, userspace should include PASID in the +bind_data when issuing direct binding request. + +PASID could be managed per-device or system-wide which, again, depends on +IOMMU vendor and is reported in nesting cap info. When system-wide policy +is reported (IOMMU_NESTING_FEAT_SYSWIDE_PASID), e.g. as by Intel platforms, +userspace *must* allocate PASID from VFIO before attempting binding of +stage-1 page table: + + req.flags = VFIO_IOMMU_ALLOC_PASID; + ioctl(container, VFIO_IOMMU_PASID_REQUEST, &req); + +Once the stage-1 page table is bound to the IOMMU, the guest is allowed to +fully manage its mapping at its disposal. The IOMMU walks nested stage-1 +and stage-2 page tables when serving DMA requests from assigned device, and +may cache the stage-1 mapping in the IOTLB. When required (IOMMU_NESTING_ +FEAT_CACHE_INVLD), userspace *must* forward guest stage-1 invalidation to +the host, so the IOTLB is invalidated: +Documentation/driver-api/vfio.rst group and can access them as follows:: · line 319
Quoted 10 lines
Documentation/driver-api/vfio.rst +7
-0+ nesting_op->flags = VFIO_IOMMU_NESTING_OP_CACHE_INVLD; + memcpy(&nesting_op->data, &cache_inv_data, sizeof(cache_inv_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +Forwarded invalidations can happen at various granularity levels (page +level, context level, etc.) + VFIO User API -------------------------------------------------------------------------------Thanks
Eric
RE: [PATCH v6 14/15] vfio: Document dual stage control
Hi Eric,
Quoted 83 lines
From: Eric Auger <[email protected]>
Sent: Monday, August 17, 2020 3:41 PMHi Yi,
Quoted 55 lines · Yi L
Hi Eric,
Quoted 53 lines
From: Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 7:52 PMHi Yi,
Quoted 4 lines · Liu Yi L
From: Eric Auger <[email protected]>
The VFIO API was enhanced to support nested stage control: a bunch of> new
ioctls and usage guideline.
Quoted 23 lines
Let's document the process to follow to set up nested mode.
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>Signed-off-by Eric Auger <[email protected]>Signed-off-by Liu Yi L <[email protected]>---
v5 -> v6:
*) tweak per Eric's comments.v3 -> v4:
*) add review-by from Stefan Hajnocziv2 -> v3:
*) address comments from Stefan HajnocziQuoted 7 lines
v1 -> v2:
*) new in v2, compared with Eric's original version, pasid table bind
and fault reporting is removed as this series doesn't cover them. Original version from Eric.
https://lkml.org/lkml/2020/3/20/700--- Documentation/driver-api/vfio.rst | 75+++++++++++++++++++++++++++++++++++++++
Quoted 11 lines
1 file changed, 75 insertions(+)Documentation/driver-api/vfio.rst +0
-0@@ -239,6 +239,81 @@ group and can access them as follows:: /* Gratuitous device reset and go... */ ioctl(device, VFIO_DEVICE_RESET);Quoted 5 lines
Quoted 5 lines
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 5 lines
Documentation/driver-api/vfio.rst +5
-0+IOMMU Dual Stage Control +------------------------ + +Some IOMMUs support 2 stages/levels of translation. Stage +corresponds to the ARM terminology while level corresponds to Intel'sterminology.
Quoted 16 lines
Quoted 15 lines
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 258
Quoted 12 lines
Documentation/driver-api/vfio.rst +12
-0+In the following text we use either without distinction. + +This is useful when the guest is exposed with a virtual IOMMU and +some devices are assigned to the guest through VFIO. Then the guest +OS can use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor +uses stage +2 for VM isolation (GPA -> HPA). + +Under dual stage translation, the guest gets ownership of the +stage-1 page tables and also owns stage-1 configuration structures. +The hypervisor owns the root configuration structure (for security +reason), including stage-2 configuration.This is only true for vtd. On ARM the stage2 cfg is the Context Descriptor table (aka PASID table). root cfg only store the GPA of the CD table.
Quoted 4 lines
Quoted 3 lines
I've a check with you on the meaning of "configuration structures". For Vt-d, does it mean the root table/context table/pasid table? if I'm correct, then how about below description?
Yes I agree
thanks.
Quoted 5 lines
Quoted 3 lines
"Under dual stage translation, the guest gets ownership of the stage-1 configuration structures or page tables.
Actually on ARM the guest both owns the S1 configuration (CD table) and S1 page tables ;-)
I see. so on ARM platform, guest owns both configuration and page table.
Quoted 1 line
on Intel I understand the guest only owns the S1 page tables.
yes, on Intel, guest only owns the S1 page tables.
Quoted 1 line
If confirmed, you may use such kind of explicit statement.
will do.
Regards,
Yi Liu
Quoted 82 lines
Thanks
Eric
This depends on vendor. The
Quoted 77 lines
hypervisor owns the root configuration structure (for security reason), including stage-2 configuration."
Quoted 12 lines
This works as long as configuration structures and page table
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 10 lines
Documentation/driver-api/vfio.rst +10
-0+formats are compatible between the virtual IOMMU and the physical IOMMU. + +Assuming the HW supports it, this nested mode is selected by +choosing the VFIO_TYPE1_NESTING_IOMMU type through: + + ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); + +This forces the hypervisor to use the stage-2, leaving stage-1 +available for guest usage. The stage-1 format and binding method +are vendor specific. There are reported in the nesting capability ...
got it.
"The stage-1 format and binding method are reported in nesting capability."
Quoted 10 lines
Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 9 lines
Documentation/driver-api/vfio.rst +9
-0+and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) +through +VFIO_IOMMU_GET_INFO: + + ioctl(container->fd, VFIO_IOMMU_GET_INFO, &nesting_info); + +The nesting cap info is available only after NESTING_IOMMU is selected. +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails +andIf the underlying
got it.
Quoted 10 lines
Quoted 5 lines
Documentation/driver-api/vfio.rst +5
-0+userspace should try other IOMMU types. Details of the nesting cap +info can be found in Documentation/userspace-api/iommu.rst. + +The stage-1 page table can be bound to the IOMMU in two methods: +directly>(quoted diff) +2
-0+or indirectly. Direct binding requires userspace to notify VFIO of +everyNot sure we shall use this direct/indirect terminology. I don't think this is part of either ARM or Intel SPEC.
Quoted 3 lines
Suggestion: On Intel, the stage1 page table info are mediated by the userspace for each PASID. On ARM, the userspace directly passes the GPA of the whole PASID table. Currently only Intel's binding is supported.
got it. this is what we want to say by ditect/indirect terminology.
Regards,
Yi LiuQuoted 25 lines
Quoted 25 lines
Documentation/driver-api/vfio.rst +25
-0+guest stage-1 page table binding, while indirect binding allows +userspace to bind once with an intermediate structure (e.g. PASID +table) which indirectly links to guest stage-1 page tables. The +actual binding method depends on IOMMU vendor. Currently only the +direct binding capability ( +IOMMU_NESTING_FEAT_BIND_PGTBL) is supported: + + nesting_op->flags = VFIO_IOMMU_NESTING_OP_BIND_PGTBL; + memcpy(&nesting_op->data, &bind_data, sizeof(bind_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +When multiple stage-1 page tables are supported on a device, each +page table is associated with a PASID (Process Address Space ID) to +differentiate with each other. In such case, userspace should +include PASID in the bind_data when issuing direct binding request. + +PASID could be managed per-device or system-wide which, again, +depends on IOMMU vendor and is reported in nesting cap info. When +system-wide policy is reported (IOMMU_NESTING_FEAT_SYSWIDE_PASID), +e.g. as by Intel platforms, userspace *must* allocate PASID from +VFIO before attempting binding of +stage-1 page table: + + req.flags = VFIO_IOMMU_ALLOC_PASID; + ioctl(container, VFIO_IOMMU_PASID_REQUEST, &req);
Quoted 26 lines
Quoted 7 lines
Quoted 7 lines
Documentation/driver-api/vfio.rst group and can access them as follows:: · line 314
Quoted 7 lines
Documentation/driver-api/vfio.rst +7
-0+ +Once the stage-1 page table is bound to the IOMMU, the guest is +allowed to fully manage its mapping at its disposal. The IOMMU +walks nested stage-1 and stage-2 page tables when serving DMA +requests from assigned device, and may cache the stage-1 mapping in +the IOTLB. When required (IOMMU_NESTING_ FEAT_CACHE_INVLD), +userspace *must* forward guest stage-1 invalidation to the host, so the IOTLBis invalidated:
Quoted 18 lines
Quoted 17 lines
Quoted 4 lines
Documentation/driver-api/vfio.rst +2
-0+ + nesting_op->flags = VFIO_IOMMU_NESTING_OP_CACHE_INVLD;Documentation/driver-api/vfio.rst group and can access them as follows::
Quoted 10 lines
Documentation/driver-api/vfio.rst +6
-2+ memcpy(&nesting_op->data, &cache_inv_data, sizeof(cache_inv_data)); + ioctl(container->fd, VFIO_IOMMU_NESTING_OP, nesting_op); + +Forwarded invalidations can happen at various granularity levels +(page level, context level, etc.) + VFIO User API -------------------------------------------------------------------- -----------Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v6 15/15 iommu/vt-d: Support reporting nesting capability info
In reply to Liu Yi LThis patch reports nesting info, and only supports the case where all the physical iomms have the same CAP/ECAP MASKS.
---
v2 -> v3:
*) remove cap/ecap_mask in iommu_nesting_info.
--- drivers/iommu/intel/iommu.c | 81 +++++++++++++++++++++++++++++++++++++++++++-- include/linux/intel-iommu.h | 16 +++++++++ 2 files changed, 95 insertions(+), 2 deletions(-)
drivers/iommu/intel/iommu.c +79 -2
@@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)static inline bool nested_mode_support (void ) {struct dmar_drhd_unit *drhd; -struct intel_iommu *iommu; +struct intel_iommu *iommu, *prev =NULL ;bool ret = true;rcu_read_lock ();for_each_active_iommu (iommu, drhd) { -if (!sm_supported (iommu) || !ecap_nest (iommu->ecap)) { +if (!prev) + prev = iommu; +if (!sm_supported (iommu) || !ecap_nest (iommu->ecap) || + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) || + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) { ret = false;break ; }
@@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct iommu_domain *domain,return ret; } +static int intel_iommu_get_nesting_info (struct iommu_domain *domain, +struct iommu_nesting_info *info) +{ +struct dmar_domain *dmar_domain =to_dmar_domain (domain); +u64 cap =VTD_CAP_MASK , ecap =VTD_ECAP_MASK ; +struct device_domain_info *domain_info; +struct iommu_nesting_info_vtd vtd; +unsigned long flags; +unsigned int size; + +if (domain->type !=IOMMU_DOMAIN_UNMANAGED || + !(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) +return -ENODEV ; + +if (!info) +return -EINVAL ; + + size =sizeof (struct iommu_nesting_info ) + +sizeof (struct iommu_nesting_info_vtd ); +/* +* if provided buffer size is smaller than expected, should +* return 0 and also the expected buffer size to caller. +*/ +if (info->argsz < size) { + info->argsz = size; +return 0 ; + } + +spin_lock_irqsave (&device_domain_lock, flags); +/* +* arbitrary select the first domain_info as all nesting +* related capabilities should be consistent across iommu +* units. +*/ + domain_info =list_first_entry (&dmar_domain->devices, + struct device_domain_info, link); + cap &= domain_info->iommu->cap; + ecap &= domain_info->iommu->ecap; +spin_unlock_irqrestore (&device_domain_lock, flags); + + info->format =IOMMU_PASID_FORMAT_INTEL_VTD ; + info->features =IOMMU_NESTING_FEAT_SYSWIDE_PASID | +IOMMU_NESTING_FEAT_BIND_PGTBL | +IOMMU_NESTING_FEAT_CACHE_INVLD ; + info->addr_width = dmar_domain->gaw; + info->pasid_bits =ilog2 (intel_pasid_max_id); + info->padding =0 ; + vtd.flags =0 ; + vtd.padding =0 ; + vtd.cap_reg = cap; + vtd.ecap_reg = ecap; + +memcpy (info->data, &vtd,sizeof (vtd)); +return 0 ; +} + +static int intel_iommu_domain_get_attr (struct iommu_domain *domain, +enum iommu_attr attr,void *data) +{ +switch (attr) { +case DOMAIN_ATTR_NESTING : + { +struct iommu_nesting_info *info = + (struct iommu_nesting_info *)data; + +return intel_iommu_get_nesting_info (domain, info); + } +default : +return -ENOENT ; + } +} +/* * Check that the device does not live on an external facing PCI port that is * marked as untrusted. Such devices should not be able to apply quirks and
@@ -6103,6 +6179,7 @@ const struct iommu_ops intel_iommu_ops = { .domain_alloc = intel_iommu_domain_alloc, .domain_free = intel_iommu_domain_free, .domain_set_attr = intel_iommu_domain_set_attr, + .domain_get_attr = intel_iommu_domain_get_attr, .attach_dev = intel_iommu_attach_device, .detach_dev = intel_iommu_detach_device, .aux_attach_dev = intel_iommu_aux_attach_device,
include/linux/intel-iommu.h +16 -0
@@ -197,6 +197,22 @@#define ecap_max_handle_mask (e) ((e >> 20) & 0xf)#define ecap_sc_support (e) ((e >> 7) & 0x1)/* Snooping Control */ +/* Nesting Support Capability Alignment */ +#define VTD_CAP_FL1GP BIT_ULL(56) +#define VTD_CAP_FL5LP BIT_ULL(60) +#define VTD_ECAP_PRS BIT_ULL(29) +#define VTD_ECAP_ERS BIT_ULL(30) +#define VTD_ECAP_SRS BIT_ULL(31) +#define VTD_ECAP_EAFS BIT_ULL(34) +#define VTD_ECAP_PASID BIT_ULL(40) + +/* Only capabilities marked in below MASKs are reported */ +#define VTD_CAP_MASK (VTD_CAP_FL1GP | VTD_CAP_FL5LP) + +#define VTD_ECAP_MASK (VTD_ECAP_PRS | VTD_ECAP_ERS | \ + VTD_ECAP_SRS | VTD_ECAP_EAFS | \ + VTD_ECAP_PASID) +/* Virtual command interface capability */ #define vccap_pasid (v) (((v) & DMA_VCS_PAS))/* PASID allocation */
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 15/15] iommu/vt-d: Support reporting nesting capability info
Hi Yi,
Quoted 2 lines · Liu Yi L
This patch reports nesting info, and only supports the case where all the physical iomms have the same CAP/ECAP MASKS.
s/iomms/iommus
Quoted 30 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v2 -> v3:
*) remove cap/ecap_mask iniommu_nesting_info.--- drivers/iommu/intel/iommu.c | 81 +++++++++++++++++++++++++++++++++++++++++++-- include/linux/intel-iommu.h | 16 +++++++++ 2 files changed, 95 insertions(+), 2 deletions(-)drivers/iommu/intel/iommu.c +1
-1@@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)static inline bool nested_mode_support (void ) {struct dmar_drhd_unit *drhd; -struct intel_iommu *iommu; +struct intel_iommu *iommu, *prev =NULL ;bool ret = true;
drivers/iommu/intel/iommu.c static inline bool iommu_pasid_support(void) · line 5674
Quoted 10 lines
drivers/iommu/intel/iommu.c +5
-1rcu_read_lock ();for_each_active_iommu (iommu, drhd) { -if (!sm_supported (iommu) || !ecap_nest (iommu->ecap)) { +if (!prev) + prev = iommu; +if (!sm_supported (iommu) || !ecap_nest (iommu->ecap) || + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) || + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) { ret = false;break ;
So this changes the behavior of DOMAIN_ATTR_NESTING. Shouldn't it have a Fixes tag as well? And maybe add the capability getter in a separate patch?
Quoted 59 lines
drivers/iommu/intel/iommu.c +54
-0}@@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct iommu_domain *domain,return ret; } +static int intel_iommu_get_nesting_info (struct iommu_domain *domain, +struct iommu_nesting_info *info) +{ +struct dmar_domain *dmar_domain =to_dmar_domain (domain); +u64 cap =VTD_CAP_MASK , ecap =VTD_ECAP_MASK ; +struct device_domain_info *domain_info; +struct iommu_nesting_info_vtd vtd; +unsigned long flags; +unsigned int size; + +if (domain->type !=IOMMU_DOMAIN_UNMANAGED || + !(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) +return -ENODEV ; + +if (!info) +return -EINVAL ; + + size =sizeof (struct iommu_nesting_info ) + +sizeof (struct iommu_nesting_info_vtd ); +/* +* if provided buffer size is smaller than expected, should +* return 0 and also the expected buffer size to caller. +*/ +if (info->argsz < size) { + info->argsz = size; +return 0 ; + } + +spin_lock_irqsave (&device_domain_lock, flags); +/* +* arbitrary select the first domain_info as all nesting +* related capabilities should be consistent across iommu +* units. +*/ + domain_info =list_first_entry (&dmar_domain->devices, + struct device_domain_info, link); + cap &= domain_info->iommu->cap; + ecap &= domain_info->iommu->ecap; +spin_unlock_irqrestore (&device_domain_lock, flags); + + info->format =IOMMU_PASID_FORMAT_INTEL_VTD ; + info->features =IOMMU_NESTING_FEAT_SYSWIDE_PASID | +IOMMU_NESTING_FEAT_BIND_PGTBL | +IOMMU_NESTING_FEAT_CACHE_INVLD ; + info->addr_width = dmar_domain->gaw; + info->pasid_bits =ilog2 (intel_pasid_max_id); + info->padding =0 ; + vtd.flags =0 ; + vtd.padding =0 ; + vtd.cap_reg = cap; + vtd.ecap_reg = ecap; + +memcpy (info->data, &vtd,sizeof (vtd)); +return 0 ;
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6151
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+} + +static int intel_iommu_domain_get_attr (struct iommu_domain *domain, +enum iommu_attr attr,void *data) +{ +switch (attr) { +case DOMAIN_ATTR_NESTING : + { +struct iommu_nesting_info *info = + (struct iommu_nesting_info *)data;
don't you need to hold a device_domain_lock earlier to make sure domain attributes can't change behind your back (unmanaged type and nested mode)?
Quoted 37 lines
drivers/iommu/intel/iommu.c +9
-0+ +return intel_iommu_get_nesting_info (domain, info); + } +default : +return -ENOENT ; + } +} +/* * Check that the device does not live on an external facing PCI port that is * marked as untrusted. Such devices should not be able to apply quirks and @@ -6103,6 +6179,7 @@ const struct iommu_ops intel_iommu_ops = { .domain_alloc = intel_iommu_domain_alloc, .domain_free = intel_iommu_domain_free, .domain_set_attr = intel_iommu_domain_set_attr, + .domain_get_attr = intel_iommu_domain_get_attr, .attach_dev = intel_iommu_attach_device, .detach_dev = intel_iommu_detach_device, .aux_attach_dev = intel_iommu_aux_attach_device,include/linux/intel-iommu.h +9
-0@@ -197,6 +197,22 @@#define ecap_max_handle_mask (e) ((e >> 20) & 0xf)#define ecap_sc_support (e) ((e >> 7) & 0x1)/* Snooping Control */ +/* Nesting Support Capability Alignment */ +#define VTD_CAP_FL1GP BIT_ULL(56) +#define VTD_CAP_FL5LP BIT_ULL(60) +#define VTD_ECAP_PRS BIT_ULL(29) +#define VTD_ECAP_ERS BIT_ULL(30) +#define VTD_ECAP_SRS BIT_ULL(31) +#define VTD_ECAP_EAFS BIT_ULL(34) +#define VTD_ECAP_PASID BIT_ULL(40) +
include/linux/intel-iommu.h line 218
Quoted 10 lines
include/linux/intel-iommu.h +7
-0+/* Only capabilities marked in below MASKs are reported */ +#define VTD_CAP_MASK (VTD_CAP_FL1GP | VTD_CAP_FL5LP) + +#define VTD_ECAP_MASK (VTD_ECAP_PRS | VTD_ECAP_ERS | \ + VTD_ECAP_SRS | VTD_ECAP_EAFS | \ + VTD_ECAP_PASID) +/* Virtual command interface capability */ #define vccap_pasid (v) (((v) & DMA_VCS_PAS))/* PASID allocation */
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 15/15] iommu/vt-d: Support reporting nesting capability info
Hi Eric,
Quoted 9 lines
Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 8:01 PMHi Yi,
Quoted 2 lines · Liu Yi L
This patch reports nesting info, and only supports the case where all the physical iomms have the same CAP/ECAP MASKS.
s/iomms/iommus
yep.
Quoted 31 lines
Quoted 10 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>Quoted 5 lines
---
v2 -> v3:
*) remove cap/ecap_mask iniommu_nesting_info.--- drivers/iommu/intel/iommu.c | 81+++++++++++++++++++++++++++++++++++++++++++--
Quoted 15 lines
include/linux/intel-iommu.h | 16 +++++++++ 2 files changed, 95 insertions(+), 2 deletions(-)drivers/iommu/intel/iommu.c +1
-1@@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)static inline bool nested_mode_support (void ) {struct dmar_drhd_unit *drhd; -struct intel_iommu *iommu; +struct intel_iommu *iommu, *prev =NULL ;bool ret = true;
Quoted 12 lines
drivers/iommu/intel/iommu.c static inline bool iommu_pasid_support(void) · line 5674
Quoted 10 lines
drivers/iommu/intel/iommu.c +5
-1rcu_read_lock ();for_each_active_iommu (iommu, drhd) { -if (!sm_supported (iommu) || !ecap_nest (iommu->ecap)) { +if (!prev) + prev = iommu; +if (!sm_supported (iommu) || !ecap_nest (iommu->ecap) || + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) || + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) { ret = false;break ;So this changes the behavior of
DOMAIN_ATTR_NESTING. Shouldn't it have a Fixes tag as well? And maybe add the capability getter in a separate patch?
yes, this changed the behavior. so it would be better to be a separate patch and upstream along? how about your idea? @Lu, Baolu :-)
Quoted 6 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain
Quoted 2 lines
drivers/iommu/intel/iommu.c +0
-0}@@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct iommu_domain*domain,
Quoted 3 lines
drivers/iommu/intel/iommu.c +0
-0return ret; }
Quoted 10 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6097
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+static int intel_iommu_get_nesting_info (struct iommu_domain *domain, +struct iommu_nesting_info *info) +{ +struct dmar_domain *dmar_domain =to_dmar_domain (domain); +u64 cap =VTD_CAP_MASK , ecap =VTD_ECAP_MASK ; +struct device_domain_info *domain_info; +struct iommu_nesting_info_vtd vtd; +unsigned long flags; +unsigned int size; +
perhaps better to acquire the lock here. [1]
Quoted 49 lines
Quoted 49 lines
drivers/iommu/intel/iommu.c +49
-0+if (domain->type !=IOMMU_DOMAIN_UNMANAGED || + !(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) +return -ENODEV ; + +if (!info) +return -EINVAL ; + + size =sizeof (struct iommu_nesting_info ) + +sizeof (struct iommu_nesting_info_vtd ); +/* +* if provided buffer size is smaller than expected, should +* return 0 and also the expected buffer size to caller. +*/ +if (info->argsz < size) { + info->argsz = size; +return 0 ; + } + +spin_lock_irqsave (&device_domain_lock, flags); +/* +* arbitrary select the first domain_info as all nesting +* related capabilities should be consistent across iommu +* units. +*/ + domain_info =list_first_entry (&dmar_domain->devices, + struct device_domain_info, link); + cap &= domain_info->iommu->cap; + ecap &= domain_info->iommu->ecap; +spin_unlock_irqrestore (&device_domain_lock, flags); + + info->format =IOMMU_PASID_FORMAT_INTEL_VTD ; + info->features =IOMMU_NESTING_FEAT_SYSWIDE_PASID | +IOMMU_NESTING_FEAT_BIND_PGTBL | +IOMMU_NESTING_FEAT_CACHE_INVLD ; + info->addr_width = dmar_domain->gaw; + info->pasid_bits =ilog2 (intel_pasid_max_id); + info->padding =0 ; + vtd.flags =0 ; + vtd.padding =0 ; + vtd.cap_reg = cap; + vtd.ecap_reg = ecap; + +memcpy (info->data, &vtd,sizeof (vtd)); +return 0 ; +} + +static int intel_iommu_domain_get_attr (struct iommu_domain *domain, +enum iommu_attr attr,void *data) +{
Quoted 8 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6151
Quoted 5 lines
drivers/iommu/intel/iommu.c +5
-0+switch (attr) { +case DOMAIN_ATTR_NESTING : + { +struct iommu_nesting_info *info = + (struct iommu_nesting_info *)data;don't you need to hold a
device_domain_lockearlier to make sure domain attributes can't change behind your back (unmanaged type and nested mode)?
do you mean acquire lock at [1]?
Regards,
Yi Liu
Quoted 48 lines
Quoted 35 lines
drivers/iommu/intel/iommu.c +9
-0+ +return intel_iommu_get_nesting_info (domain, info); + } +default : +return -ENOENT ; + } +} +/* * Check that the device does not live on an external facing PCI port that is * marked as untrusted. Such devices should not be able to apply quirks and @@ -6103,6 +6179,7 @@ const struct iommu_ops intel_iommu_ops = { .domain_alloc = intel_iommu_domain_alloc, .domain_free = intel_iommu_domain_free, .domain_set_attr = intel_iommu_domain_set_attr, + .domain_get_attr = intel_iommu_domain_get_attr, .attach_dev = intel_iommu_attach_device, .detach_dev = intel_iommu_detach_device, .aux_attach_dev = intel_iommu_aux_attach_device,include/linux/intel-iommu.h +8
-0@@ -197,6 +197,22 @@#define ecap_max_handle_mask (e) ((e >> 20) & 0xf)#define ecap_sc_support (e) ((e >> 7) & 0x1)/* Snooping Control */ +/* Nesting Support Capability Alignment */ +#define VTD_CAP_FL1GP BIT_ULL(56) +#define VTD_CAP_FL5LP BIT_ULL(60) +#define VTD_ECAP_PRS BIT_ULL(29) +#define VTD_ECAP_ERS BIT_ULL(30) +#define VTD_ECAP_SRS BIT_ULL(31) +#define VTD_ECAP_EAFS BIT_ULL(34) +#define VTD_ECAP_PASID BIT_ULL(40)include/linux/intel-iommu.h line 217
Quoted 10 lines
include/linux/intel-iommu.h +8
-0+ +/* Only capabilities marked in below MASKs are reported */ +#define VTD_CAP_MASK (VTD_CAP_FL1GP | VTD_CAP_FL5LP) + +#define VTD_ECAP_MASK (VTD_ECAP_PRS | VTD_ECAP_ERS | \ + VTD_ECAP_SRS | VTD_ECAP_EAFS | \ + VTD_ECAP_PASID) +/* Virtual command interface capability */ #define vccap_pasid (v) (((v) & DMA_VCS_PAS))/* PASID allocation */
Quoted 2 lines
Quoted 3 lines
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 15/15] iommu/vt-d: Support reporting nesting capability info
Quoted 138 lines · Yi L
Hi Eric,
Quoted 9 lines
Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 8:01 PMHi Yi,
Quoted 2 lines · Liu Yi L
This patch reports nesting info, and only supports the case where all the physical iomms have the same CAP/ECAP MASKS.
s/iomms/iommus
yep.
Quoted 31 lines
Quoted 10 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>Quoted 5 lines
---
v2 -> v3:
*) remove cap/ecap_mask iniommu_nesting_info.--- drivers/iommu/intel/iommu.c | 81+++++++++++++++++++++++++++++++++++++++++++--
Quoted 15 lines
include/linux/intel-iommu.h | 16 +++++++++ 2 files changed, 95 insertions(+), 2 deletions(-)drivers/iommu/intel/iommu.c +1
-1@@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)static inline bool nested_mode_support (void ) {struct dmar_drhd_unit *drhd; -struct intel_iommu *iommu; +struct intel_iommu *iommu, *prev =NULL ;bool ret = true;Quoted 12 lines
drivers/iommu/intel/iommu.c static inline bool iommu_pasid_support(void) · line 5674
Quoted 10 lines
drivers/iommu/intel/iommu.c +5
-1rcu_read_lock ();for_each_active_iommu (iommu, drhd) { -if (!sm_supported (iommu) || !ecap_nest (iommu->ecap)) { +if (!prev) + prev = iommu; +if (!sm_supported (iommu) || !ecap_nest (iommu->ecap) || + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) || + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) { ret = false;break ;So this changes the behavior of
DOMAIN_ATTR_NESTING. Shouldn't it have a Fixes tag as well? And maybe add the capability getter in a separate patch?yes, this changed the behavior. so it would be better to be a separate patch and upstream along? how about your idea? @Lu, Baolu :-)
Quoted 6 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain
Quoted 2 lines
drivers/iommu/intel/iommu.c +0
-0}@@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct iommu_domain*domain,
Quoted 3 lines
drivers/iommu/intel/iommu.c +0
-0return ret; }Quoted 10 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6097
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+static int intel_iommu_get_nesting_info (struct iommu_domain *domain, +struct iommu_nesting_info *info) +{ +struct dmar_domain *dmar_domain =to_dmar_domain (domain); +u64 cap =VTD_CAP_MASK , ecap =VTD_ECAP_MASK ; +struct device_domain_info *domain_info; +struct iommu_nesting_info_vtd vtd; +unsigned long flags; +unsigned int size; +perhaps better to acquire the lock here. [1]
Quoted 49 lines
Quoted 49 lines
drivers/iommu/intel/iommu.c +49
-0+if (domain->type !=IOMMU_DOMAIN_UNMANAGED || + !(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) +return -ENODEV ; + +if (!info) +return -EINVAL ; + + size =sizeof (struct iommu_nesting_info ) + +sizeof (struct iommu_nesting_info_vtd ); +/* +* if provided buffer size is smaller than expected, should +* return 0 and also the expected buffer size to caller. +*/ +if (info->argsz < size) { + info->argsz = size; +return 0 ; + } + +spin_lock_irqsave (&device_domain_lock, flags); +/* +* arbitrary select the first domain_info as all nesting +* related capabilities should be consistent across iommu +* units. +*/ + domain_info =list_first_entry (&dmar_domain->devices, + struct device_domain_info, link); + cap &= domain_info->iommu->cap; + ecap &= domain_info->iommu->ecap; +spin_unlock_irqrestore (&device_domain_lock, flags); + + info->format =IOMMU_PASID_FORMAT_INTEL_VTD ; + info->features =IOMMU_NESTING_FEAT_SYSWIDE_PASID | +IOMMU_NESTING_FEAT_BIND_PGTBL | +IOMMU_NESTING_FEAT_CACHE_INVLD ; + info->addr_width = dmar_domain->gaw; + info->pasid_bits =ilog2 (intel_pasid_max_id); + info->padding =0 ; + vtd.flags =0 ; + vtd.padding =0 ; + vtd.cap_reg = cap; + vtd.ecap_reg = ecap; + +memcpy (info->data, &vtd,sizeof (vtd)); +return 0 ; +} + +static int intel_iommu_domain_get_attr (struct iommu_domain *domain, +enum iommu_attr attr,void *data) +{Quoted 8 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6151
Quoted 5 lines
drivers/iommu/intel/iommu.c +5
-0+switch (attr) { +case DOMAIN_ATTR_NESTING : + { +struct iommu_nesting_info *info = + (struct iommu_nesting_info *)data;don't you need to hold a
device_domain_lockearlier to make sure domain attributes can't change behind your back (unmanaged type and nested mode)?
Quoted 1 line
do you mean acquire lock at [1]?
yep either at [1] or before calling intel_iommu_get_nesting_info and adding a comment saying intel_iommu_get_nesting_info() shall be called with the lock held
Thanks
Eric
Quoted 56 lines
Regards,
Yi LiuQuoted 51 lines
Quoted 35 lines
drivers/iommu/intel/iommu.c +9
-0+ +return intel_iommu_get_nesting_info (domain, info); + } +default : +return -ENOENT ; + } +} +/* * Check that the device does not live on an external facing PCI port that is * marked as untrusted. Such devices should not be able to apply quirks and @@ -6103,6 +6179,7 @@ const struct iommu_ops intel_iommu_ops = { .domain_alloc = intel_iommu_domain_alloc, .domain_free = intel_iommu_domain_free, .domain_set_attr = intel_iommu_domain_set_attr, + .domain_get_attr = intel_iommu_domain_get_attr, .attach_dev = intel_iommu_attach_device, .detach_dev = intel_iommu_detach_device, .aux_attach_dev = intel_iommu_aux_attach_device,include/linux/intel-iommu.h +8
-0@@ -197,6 +197,22 @@#define ecap_max_handle_mask (e) ((e >> 20) & 0xf)#define ecap_sc_support (e) ((e >> 7) & 0x1)/* Snooping Control */ +/* Nesting Support Capability Alignment */ +#define VTD_CAP_FL1GP BIT_ULL(56) +#define VTD_CAP_FL5LP BIT_ULL(60) +#define VTD_ECAP_PRS BIT_ULL(29) +#define VTD_ECAP_ERS BIT_ULL(30) +#define VTD_ECAP_SRS BIT_ULL(31) +#define VTD_ECAP_EAFS BIT_ULL(34) +#define VTD_ECAP_PASID BIT_ULL(40)include/linux/intel-iommu.h line 217
Quoted 10 lines
include/linux/intel-iommu.h +8
-0+ +/* Only capabilities marked in below MASKs are reported */ +#define VTD_CAP_MASK (VTD_CAP_FL1GP | VTD_CAP_FL5LP) + +#define VTD_ECAP_MASK (VTD_ECAP_PRS | VTD_ECAP_ERS | \ + VTD_ECAP_SRS | VTD_ECAP_EAFS | \ + VTD_ECAP_PASID) +/* Virtual command interface capability */ #define vccap_pasid (v) (((v) & DMA_VCS_PAS))/* PASID allocation */
Quoted 2 lines
Thanks
Eric
RE: [PATCH v6 15/15] iommu/vt-d: Support reporting nesting capability info
Eric,
Quoted 143 lines
From: Auger Eric <[email protected]>
Sent: Monday, August 17, 2020 3:43 PMQuoted 58 lines · Yi L
Hi Eric,
Quoted 9 lines
Auger Eric <[email protected]>
Sent: Sunday, August 16, 2020 8:01 PMHi Yi,
Quoted 2 lines · Liu Yi L
This patch reports nesting info, and only supports the case where all the physical iomms have the same CAP/ECAP MASKS.
s/iomms/iommus
yep.
Quoted 31 lines
Quoted 10 lines
Cc Kevin Tian <[email protected]>CC Jacob Pan <[email protected]>Cc Alex Williamson <[email protected]>Cc Eric Auger <[email protected]>Cc Jean-Philippe Brucker <[email protected]>Cc Joerg Roedel <[email protected]>Cc Lu Baolu <[email protected]>Signed-off-by Liu Yi L <[email protected]>Signed-off-by Jacob Pan <[email protected]>Quoted 5 lines
---
v2 -> v3:
*) remove cap/ecap_mask iniommu_nesting_info.--- drivers/iommu/intel/iommu.c | 81+++++++++++++++++++++++++++++++++++++++++++--
Quoted 15 lines
include/linux/intel-iommu.h | 16 +++++++++ 2 files changed, 95 insertions(+), 2 deletions(-)drivers/iommu/intel/iommu.c +1
-1@@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)static inline bool nested_mode_support (void ) {struct dmar_drhd_unit *drhd; -struct intel_iommu *iommu; +struct intel_iommu *iommu, *prev =NULL ;bool ret = true;Quoted 12 lines
drivers/iommu/intel/iommu.c static inline bool iommu_pasid_support(void) · line 5674
Quoted 10 lines
drivers/iommu/intel/iommu.c +5
-1rcu_read_lock ();for_each_active_iommu (iommu, drhd) { -if (!sm_supported (iommu) || !ecap_nest (iommu->ecap)) { +if (!prev) + prev = iommu; +if (!sm_supported (iommu) || !ecap_nest (iommu->ecap) || + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) || + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) { ret = false;break ;So this changes the behavior of
DOMAIN_ATTR_NESTING. Shouldn't it have a Fixes tag as well? And maybe add the capability getter in a separate patch?Quoted 5 lines
yes, this changed the behavior. so it would be better to be a separate patch and upstream along? how about your idea? @Lu, Baolu :-)
Quoted 2 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct
Quoted 2 lines
drivers/iommu/intel/iommu.c +0
-0}@@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct
iommu_domainQuoted 75 lines
Quoted 4 lines
*domain,
Quoted 3 lines
drivers/iommu/intel/iommu.c +0
-0return ret; }Quoted 10 lines
drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6097
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+static int intel_iommu_get_nesting_info (struct iommu_domain *domain, +struct iommu_nesting_info *info) +{ +struct dmar_domain *dmar_domain =to_dmar_domain (domain); +u64 cap =VTD_CAP_MASK , ecap =VTD_ECAP_MASK ; +struct device_domain_info *domain_info; +struct iommu_nesting_info_vtd vtd; +unsigned long flags; +unsigned int size; +perhaps better to acquire the lock here. [1]
Quoted 57 lines
Quoted 44 lines
drivers/iommu/intel/iommu.c +44
-0+if (domain->type !=IOMMU_DOMAIN_UNMANAGED || + !(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) +return -ENODEV ; + +if (!info) +return -EINVAL ; + + size =sizeof (struct iommu_nesting_info ) + +sizeof (struct iommu_nesting_info_vtd ); +/* +* if provided buffer size is smaller than expected, should +* return 0 and also the expected buffer size to caller. +*/ +if (info->argsz < size) { + info->argsz = size; +return 0 ; + } + +spin_lock_irqsave (&device_domain_lock, flags); +/* +* arbitrary select the first domain_info as all nesting +* related capabilities should be consistent across iommu +* units. +*/ + domain_info =list_first_entry (&dmar_domain->devices, + struct device_domain_info, link); + cap &= domain_info->iommu->cap; + ecap &= domain_info->iommu->ecap; +spin_unlock_irqrestore (&device_domain_lock, flags); + + info->format =IOMMU_PASID_FORMAT_INTEL_VTD ; + info->features =IOMMU_NESTING_FEAT_SYSWIDE_PASID | +IOMMU_NESTING_FEAT_BIND_PGTBL | +IOMMU_NESTING_FEAT_CACHE_INVLD ; + info->addr_width = dmar_domain->gaw; + info->pasid_bits =ilog2 (intel_pasid_max_id); + info->padding =0 ; + vtd.flags =0 ; + vtd.padding =0 ; + vtd.cap_reg = cap; + vtd.ecap_reg = ecap; + +memcpy (info->data, &vtd,sizeof (vtd)); +return 0 ;drivers/iommu/intel/iommu.c intel_iommu_domain_set_attr(struct iommu_domain *domain, · line 6151
Quoted 10 lines
drivers/iommu/intel/iommu.c +10
-0+} + +static int intel_iommu_domain_get_attr (struct iommu_domain *domain, +enum iommu_attr attr,void *data) +{ +switch (attr) { +case DOMAIN_ATTR_NESTING : + { +struct iommu_nesting_info *info = + (struct iommu_nesting_info *)data;don't you need to hold a
device_domain_lockearlier to make sure domain attributes can't change behind your back (unmanaged type and nested mode)?
Quoted 4 lines
Quoted 1 line
do you mean acquire lock at [1]?
yep either at [1] or before calling
intel_iommu_get_nesting_infoand adding a comment sayingintel_iommu_get_nesting_info()shall be called with the lock held
will do. :-)
Regards,
Yi Liu
Quoted 59 lines
Thanks
Eric
Quoted 56 lines
Regards,
Yi LiuQuoted 51 lines
Quoted 35 lines
drivers/iommu/intel/iommu.c +9
-0+ +return intel_iommu_get_nesting_info (domain, info); + } +default : +return -ENOENT ; + } +} +/* * Check that the device does not live on an external facing PCI port that is * marked as untrusted. Such devices should not be able to apply quirks and @@ -6103,6 +6179,7 @@ const struct iommu_ops intel_iommu_ops = { .domain_alloc = intel_iommu_domain_alloc, .domain_free = intel_iommu_domain_free, .domain_set_attr = intel_iommu_domain_set_attr, + .domain_get_attr = intel_iommu_domain_get_attr, .attach_dev = intel_iommu_attach_device, .detach_dev = intel_iommu_detach_device, .aux_attach_dev = intel_iommu_aux_attach_device,include/linux/intel-iommu.h +8
-0@@ -197,6 +197,22 @@#define ecap_max_handle_mask (e) ((e >> 20) & 0xf)#define ecap_sc_support (e) ((e >> 7) & 0x1)/* Snooping Control */ +/* Nesting Support Capability Alignment */ +#define VTD_CAP_FL1GP BIT_ULL(56) +#define VTD_CAP_FL5LP BIT_ULL(60) +#define VTD_ECAP_PRS BIT_ULL(29) +#define VTD_ECAP_ERS BIT_ULL(30) +#define VTD_ECAP_SRS BIT_ULL(31) +#define VTD_ECAP_EAFS BIT_ULL(34) +#define VTD_ECAP_PASID BIT_ULL(40)include/linux/intel-iommu.h line 217
Quoted 10 lines
include/linux/intel-iommu.h +8
-0+ +/* Only capabilities marked in below MASKs are reported */ +#define VTD_CAP_MASK (VTD_CAP_FL1GP | VTD_CAP_FL5LP) + +#define VTD_ECAP_MASK (VTD_ECAP_PRS | VTD_ECAP_ERS | \ + VTD_ECAP_SRS | VTD_ECAP_EAFS | \ + VTD_ECAP_PASID) +/* Virtual command interface capability */ #define vccap_pasid (v) (((v) & DMA_VCS_PAS))/* PASID allocation */
Quoted 2 lines
Thanks
Eric
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu