PCI passthru on Hyper-V
Subsystems (6)
v5 cover PCI passthru on Hyper-V
Implement passthru of PCI devices to unprivileged virtual machines (VMs) when Linux is running as a privileged VM on Microsoft Hyper-V hypervisor. This support is made to fit within the workings of VFIO framework. This supports both full device passthru and SR-IOV based VFs.
At a high level, the hypervisor supports traditional paging iommu domains that use explicit map and unmap hypercalls for mapping and unmapping guest RAM into the iommu subsystem. Hyper-V also has a concept of direct attach devices (DDA) whereby the iommu subsystem simply uses the guest HW page table (ept/npt/..). This series adds support for the former.
This series rebased to: a4ffc59238be (origin/hyperv-next)
Testing:
o Most testing done on hyperv-next:a4ffc59238be using Cloud Hypervisor (51).
o All tests involved passthru of devices using MSIx.
o Tested on Baremetal dom0/root:
test 1: Mellanox ConnectX-6 Lx passthru
test 2: NVIDIA Tesla Tesla T4 GPU.
test 3: Both of above simultaneous passthru
o VF: Mellanox ConnectX-6 Lx passthru
This series strives to establish a base line. Some pending work items:
o arm64 : some delta to make this work on arm64 (in progress).
o Qemu and OpenVMM support (in progress).
o device sleep/wakeup.
o More stress testing with high end GPUs
Changes in V5:
o enforce no multiple vms per vmm instance
o make struct mshv_vfio_device_ops const
o add WARN to iommu call. delete mappings in domain free
o mmio_mfn++ in large page check in hv_map_mmio_pages()
o no need to pass gfn to hv_map_mmio_pages()
o use region access to make mmio mappings ro or rw
Changes in V4:
o Completely remove direct device attach (DDA) support
o Remove first two patches that were submitted separately and are already merged.
o Change hv_iommu_unmap_pages() to unmap in batches of HV_REP_COUNT_MAX, otherwise if a physical range happens to be larger than that, there would be errors from the hypercall.
o Use large page (2M) flag for mmio mappings as internal testing found that to speed things up greatly.
o rename hv_no_movbl_pgs to hv_memreg in last patch.
Changes in V3:
o patch #8: fix compiler issues incase of !CONFIG_HYPERV. Also, do forward declaration of struct pci_dev instead of including pci.h.
o patch #9: minor changes to comments. Pass hv_domain instead of iommu_domain to hv_iommu_detach_dev() since that's what it needs. Set device private to null if attach fails. Clam down number of PFNs passed to hv_iommu_map_pgs().
Changes in V2:
o rebase to 5170a82e8921
o minor fixes for arm64 build
o drop patch 03: "x86/hyperv: add insufficient memory support in irqdomain.c" as it that path is no longer used
o drop patch 08: "PCI: hv: rename hv_compose_msi_msg .. " and do it separately outside this series.
o minor updates to commit messages
Changes in V1:
o patch 1: Don't tie hyperv-irq.c to CONFIG_HYPERV_IOMMU.
o patch 4: Redesigned to address security vulnerability found by copilot with passing tgid as a parameter. Also, do tgid setting right after setting pt_id.
o patch 5: Remove unused type parameter from mshv_device_ops.device_create
o patch 7: mshv_partition_ioctl_create_device cleanup on copy_to_user.
o patch 10: Add export of hv_build_devid_type_pci here to get rid of
patch 11.
o patch 12: Move functions to build device ids from patch 11 here for the benefit of arm64. Rename file to: hyperv-iommu-root.c.
o patch 13: removed to be made part of interrupt part II of this support.
o patch 14: get rid of fast path to reduce review noise.
o New (last) patch to pin ram regions if device passthru to a VM.
Thanks,
-Mukesh
Mukesh R (9):
mshv: Provide a way to get partition ID if running in a VMM process
mshv: Add declarations and definitions for VFIO-MSHV bridge device
mshv: Introduce basic mshv bridge device for VFIO to build upon
mshv: Add ioctl support for MSHV-VFIO bridge device
mshv: Import data structs around device passthru from hyperv headers
PCI: hv: Export hv_build_devid_type_pci() and change return type x86/hyperv: Implement Hyper-V virtual IOMMU
mshv: Populate mmio mappings for PCI passthru
mshv: Disable movable regions upfront if device passthru
arch/x86/hyperv/irqdomain.c | 9 +- arch/x86/include/asm/mshyperv.h | 6 + arch/x86/kernel/pci-dma.c | 2 + drivers/hv/Makefile | 3 +- drivers/hv/mshv_root.h | 24 +- drivers/hv/mshv_root_hv_call.c | 104 +++- drivers/hv/mshv_root_main.c | 287 ++++++++++- drivers/hv/mshv_vfio.c | 211 ++++++++ drivers/iommu/Kconfig | 1 + drivers/iommu/hyperv/Kconfig | 15 + drivers/iommu/hyperv/Makefile | 1 + drivers/iommu/hyperv/hv-iommu-root.c | 706 +++++++++++++++++++++++++++ include/asm-generic/mshyperv.h | 8 + include/hyperv/hvgdk_mini.h | 8 + include/hyperv/hvhdk_mini.h | 67 +++ include/linux/hyperv.h | 6 + include/uapi/linux/mshv.h | 31 ++ 17 files changed, 1440 insertions(+), 49 deletions(-) create mode 100644 drivers/hv/mshv_vfio.c create mode 100644 drivers/iommu/hyperv/Kconfig create mode 100644 drivers/iommu/hyperv/hv-iommu-root.c
Signature · 1 line
2.51.2.vfs.0.1
PATCH v5 1/9 mshv: Provide a way to get partition ID if running in a VMM process
Many PCI passthru related hypercalls require partition ID of the target guest. Guests are actually managed by MSHV driver and the partition ID is only maintained there. Add a field in the partition struct in MSHV driver to save the tgid of the VMM process creating the partition, and add a function there to retrieve partition ID if the current process is a VMM process. Furthermore, enforce one partition per VMM instance.
--- drivers/hv/mshv_root.h | 1 + drivers/hv/mshv_root_main.c | 33 ++++++++++++++++++++++++++++++++- include/asm-generic/mshyperv.h | 5 +++++ 3 files changed, 38 insertions(+), 1 deletion(-)
drivers/hv/mshv_root.h +1 -0
@@ -138,6 +138,7 @@ struct mshv_partition {struct mshv_girq_routing_table __rcu *pt_girq_tbl;u64 isolation_type; +pid_t pt_vmm_tgid;bool import_completed;bool pt_initialized;#if IS_ENABLED (CONFIG_DEBUG_FS )
drivers/hv/mshv_root_main.c +32 -1
@@ -1908,17 +1908,48 @@ mshv_partition_release(struct inode *inode, struct file *filp)return 0 ; } +/* Given a process tgid, return partition id if it is a VMM process */ +u64 mshv_current_partid (void ) +{ +struct mshv_partition *pt; +int i; +u64 ret_ptid =HV_PARTITION_ID_INVALID ; + +rcu_read_lock (); + +hash_for_each_rcu (mshv_root.pt_htable, i, pt, pt_hnode) { +if (pt->pt_vmm_tgid == current->tgid) { + ret_ptid = pt->pt_id; +break ; + } + } + +rcu_read_unlock (); +return ret_ptid; +} +EXPORT_SYMBOL_GPL (mshv_current_partid); + +/* At present, we only allow one partition per VMM instance */ static int add_partition (struct mshv_partition *partition) { +int rc =0 ; +spin_lock (&mshv_root.pt_ht_lock); +if (mshv_current_partid () !=HV_PARTITION_ID_INVALID ) { + rc = -EEXIST ; + goto out; + } + + partition->pt_vmm_tgid = current->tgid;hash_add_rcu (mshv_root.pt_htable, &partition->pt_hnode, partition->pt_id); +out:spin_unlock (&mshv_root.pt_ht_lock); -return 0 ; +return rc; } static_assert(MSHV_NUM_CPU_FEATURES_BANKS ==
include/asm-generic/mshyperv.h +5 -0
@@ -350,6 +350,7 @@ int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);int hv_call_notify_all_processors_started (void );bool hv_lp_exists (u32 lp_index);int hv_call_create_vp (int node,u64 partition_id,u32 vp_index,u32 flags); +u64 mshv_current_partid (void );#else /* CONFIG_MSHV_ROOT */ static inline bool hv_root_partition (void ) {return false; }
@@ -380,6 +381,10 @@ static inline int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u3 {return -EOPNOTSUPP ; } +static inline u64 mshv_current_partid (void ) +{ +return HV_PARTITION_ID_INVALID ; +}#endif /* CONFIG_MSHV_ROOT */ static inline int hv_deposit_memory (u64 partition_id,u64 status)
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 1/9] mshv: Provide a way to get partition ID if running in a VMM process
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] Using current->tgid for implicit VM partition binding restricts multi-process architectures, introduces PID reuse vulnerabilities (Privilege Escalation and DoS), and causes spurious creation failures.
Pre-existing issues:
- [Critical] Unchecked virtual processor (VP) access in mshv_try_assert_irq_fast can lead to a kernel panic. --
--- Patch [1]: [PATCH v5 1/9] mshv: Provide a way to get partition ID if running in a VMM process --- commit c09a2034eaac3e66996c48aebdd226e834f0611e
Author: Mukesh R <[email protected]>
mshv: Provide a way to get partition ID if running in a VMM process
This commit adds a field to the MSHV partition structure to save the tgid of the VMM process that created it. It also introduces a helper to retrieve the partition ID by matching the current tgid and restricts creation to one partition per VMM instance.
Quoted 3 lines
drivers/hv/mshv_root_main.c +0
-0
[ ... ]
Quoted 24 lines
drivers/hv/mshv_root_main.c +21
-0+u64 mshv_current_partid (void ) +{ +struct mshv_partition *pt; +int i; +u64 ret_ptid =HV_PARTITION_ID_INVALID ; + +rcu_read_lock (); + +hash_for_each_rcu (mshv_root.pt_htable, i, pt, pt_hnode) { +if (pt->pt_vmm_tgid == current->tgid) { + ret_ptid = pt->pt_id; +break ; + } + } + +rcu_read_unlock (); +return ret_ptid; +} +EXPORT_SYMBOL_GPL (mshv_current_partid); + +/* At present, we only allow one partition per VMM instance */ static int add_partition (struct mshv_partition *partition) {
drivers/hv/mshv_root_main.c mshv_partition_release(struct inode *inode, struct file *filp) · line 1945
Quoted 10 lines
drivers/hv/mshv_root_main.c +8
-0+int rc =0 ; +spin_lock (&mshv_root.pt_ht_lock); +if (mshv_current_partid () !=HV_PARTITION_ID_INVALID ) { + rc = -EEXIST ; + goto out; + } + + partition->pt_vmm_tgid = current->tgid;
[Severity: High]
Does using current->tgid for implicit VM partition binding introduce risks with process lifecycle and PID reuse?
If a VMM process creates a partition, passes the file descriptor to another process (e.g., via a UNIX socket) and exits, the original tgid could be recycled for a new, unrelated process.
If that new process performs operations that invoke mshv_current_partid(), could it incorrectly inherit the partition context of the previous process?
Alternatively, if the new process attempts to create its own partition in add_partition(), would it incorrectly fail with -EEXIST?
[Severity: Critical]
This is a pre-existing issue, but I noticed a potential unchecked access during the review of this subsystem.
In drivers/hv/mshv_eventfd.c:mshv_try_assert_irq_fast():
vp = partition->pt_vp_array[irq->lapic_apic_id];
if (!vp->vp_register_page) {
...If userspace sets up an irqfd routing entry with an APIC ID that has no corresponding virtual processor created yet, will vp be NULL when fetched from pt_vp_array?
If so, would the immediate dereference of vp->vp_register_page result in a host kernel panic when the eventfd is signaled?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH v5 2/9 mshv: Add declarations and definitions for VFIO-MSHV bridge device
In reply to Mukesh RAdd data structs needed by the subsequent patch that introduces a new module to implement VFIO-MSHV pseudo device.
--- drivers/hv/mshv_root.h | 19 +++++++++++++++++++ include/uapi/linux/mshv.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+)
drivers/hv/mshv_root.h +19 -0
@@ -227,6 +227,25 @@ struct port_table_info { }; }; +struct mshv_device { +const struct mshv_device_ops *device_ops; +struct mshv_partition *device_pt; +void *device_private; +struct hlist_node device_ptnode; +}; + +struct mshv_device_ops { +const char *device_name; +long (*device_create)(struct mshv_device *dev); +void (*device_release)(struct mshv_device *dev); +long (*device_set_attr)(struct mshv_device *dev, +struct mshv_device_attr *attr); +long (*device_has_attr)(struct mshv_device *dev, +struct mshv_device_attr *attr); +}; + +extern const struct mshv_device_ops mshv_vfio_device_ops; +int mshv_update_routing_table (struct mshv_partition *partition,const struct mshv_user_irq_entry *entries,unsigned int numents);
include/uapi/linux/mshv.h +30 -0
@@ -404,4 +404,34 @@ struct mshv_sint_mask {/* hv_hvcall device */ #define MSHV_HVCALL_SETUP _IOW(MSHV_IOCTL, 0x1E, struct mshv_vtl_hvcall_setup)#define MSHV_HVCALL _IOWR(MSHV_IOCTL, 0x1F, struct mshv_vtl_hvcall) + +/* Device passhthru */ +#define MSHV_CREATE_DEVICE_TEST 1 + +enum { +MSHV_DEV_TYPE_VFIO , +MSHV_DEV_TYPE_MAX , +}; + +struct mshv_create_device { +__u32 type;/* in: MSHV_DEV_TYPE_xxx */ +__u32 fd;/* out: device handle */ +__u32 flags;/* in: MSHV_CREATE_DEVICE_xxx */ +}; + +#define MSHV_DEV_VFIO_FILE 1 +#define MSHV_DEV_VFIO_FILE_ADD 1 +#define MSHV_DEV_VFIO_FILE_DEL 2 + +struct mshv_device_attr { +__u32 flags;/* no flags currently defined */ +__u32 group;/* device-defined */ +__u64 attr;/* group-defined */ +__u64 addr;/* userspace address of attr data */ +}; + +/* Device fds created with MSHV_CREATE_DEVICE */ +#define MSHV_SET_DEVICE_ATTR _IOW(MSHV_IOCTL, 0x00, struct mshv_device_attr) +#define MSHV_HAS_DEVICE_ATTR _IOW(MSHV_IOCTL, 0x01, struct mshv_device_attr) +#endif
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 2/9] mshv: Add declarations and definitions for VFIO-MSHV bridge device
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The `MSHV_CREATE_DEVICE` ioctl command definition is completely missing from the UAPI header. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The MSHV_CREATE_DEVICE ioctl command definition is completely missing from the UAPI header. --
--- Patch [2]: [PATCH v5 2/9] mshv: Add declarations and definitions for VFIO-MSHV bridge device --- commit e1e53f07045678558c5d600cce2f941f492fd69b
Author: Mukesh R <[email protected]>
mshv: Add declarations and definitions for VFIO-MSHV bridge device
Add data structs needed by the subsequent patch that introduces a new module to implement VFIO-MSHV pseudo device.
Quoted 5 lines
include/uapi/linux/mshv.h +0
-0@@ -404,4 +404,34 @@ struct mshv_sint_mask {
[ ... ]
Quoted 18 lines
include/uapi/linux/mshv.h +18
-0+/* Device passhthru */ +#define MSHV_CREATE_DEVICE_TEST 1 + +enum { +MSHV_DEV_TYPE_VFIO , +MSHV_DEV_TYPE_MAX , +}; + +struct mshv_create_device { +__u32 type;/* in: MSHV_DEV_TYPE_xxx */ +__u32 fd;/* out: device handle */ +__u32 flags;/* in: MSHV_CREATE_DEVICE_xxx */ +}; + +#define MSHV_DEV_VFIO_FILE 1 +#define MSHV_DEV_VFIO_FILE_ADD 1 +#define MSHV_DEV_VFIO_FILE_DEL 2 +
include/uapi/linux/mshv.h struct mshv_sint_mask { · line 435
Quoted 10 lines
include/uapi/linux/mshv.h +10
-0+struct mshv_device_attr { +__u32 flags;/* no flags currently defined */ +__u32 group;/* device-defined */ +__u64 attr;/* group-defined */ +__u64 addr;/* userspace address of attr data */ +}; + +/* Device fds created with MSHV_CREATE_DEVICE */ +#define MSHV_SET_DEVICE_ATTR _IOW(MSHV_IOCTL, 0x00, struct mshv_device_attr) +#define MSHV_HAS_DEVICE_ATTR _IOW(MSHV_IOCTL, 0x01, struct mshv_device_attr)
[Severity: Medium]
Is the definition for the MSHV_CREATE_DEVICE ioctl missing here?
The comment right above mentions device fds created with MSHV_CREATE_DEVICE, and the patch adds struct mshv_create_device, but the actual ioctl macro is never defined. Would this prevent userspace from compiling against the header to use the device passthrough feature?
Quoted 2 lines
+
#endif
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
PATCH v5 3/9 mshv: Introduce basic mshv bridge device for VFIO to build upon
In reply to Mukesh RAdd a new file to implement basic VFIO-MSHV bridge pseudo device. These functions are called in the VFIO framework, and credits to kvm/vfio.c as this file was adapted from it. This is a basic version to build upon.
--- drivers/hv/Makefile | 3 +- drivers/hv/mshv_vfio.c | 211 ++++++++++++++++++++++++++++++++++++++ include/uapi/linux/mshv.h | 1 + 3 files changed, 214 insertions(+), 1 deletion(-) create mode 100644 drivers/hv/mshv_vfio.c
drivers/hv/Makefile +2 -1
@@ -14,7 +14,8 @@ hv_vmbus-y := vmbus_drv.o \ hv_vmbus-$(CONFIG_HYPERV_TESTING) += hv_debugfs.o hv_utils-y := hv_util.o hv_kvp.o hv_snapshot.o hv_utils_transport.o mshv_root-y := mshv_root_main.o mshv_synic.o mshv_eventfd.o mshv_irq.o \ - mshv_root_hv_call.o mshv_portid_table.o mshv_regions.o + mshv_root_hv_call.o mshv_portid_table.o mshv_regions.o \ + mshv_vfio.o mshv_root-$(CONFIG_DEBUG_FS) += mshv_debugfs.o mshv_root-$(CONFIG_TRACEPOINTS) += mshv_trace.o mshv_vtl-y := mshv_vtl_main.o
drivers/hv/mshv_vfio.c new file +211 -0
211 lines not on this page · raw message
include/uapi/linux/mshv.h +1 -0
@@ -254,6 +254,7 @@ struct mshv_root_hvcall {#define MSHV_GET_GPAP_ACCESS_BITMAP _IOWR(MSHV_IOCTL, 0x06, struct mshv_gpap_access_bitmap)/* Generic hypercall */ #define MSHV_ROOT_HVCALL _IOWR(MSHV_IOCTL, 0x07, struct mshv_root_hvcall) +#define MSHV_CREATE_DEVICE _IOWR(MSHV_IOCTL, 0x08, struct mshv_create_device)/* ********************************
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The `mshv_vfio` structure is allocated without `__GFP_ACCOUNT`, allowing a container to allocate…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The mshv_vfio structure is allocated without __GFP_ACCOUNT, allowing a container to allocate kernel memory without being charged to its memory cgroup limit. --
--- Patch [3]: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon --- commit c0b4902f22a48f131a914dbd4da4237e2be4826b
Author: Mukesh R <[email protected]>
mshv: Introduce basic mshv bridge device for VFIO to build upon
This commit adds a new file to implement a basic VFIO-MSHV bridge pseudo device. It provides the foundation for VFIO integration in the MSHV framework, adapting concepts from the KVM-VFIO bridge.
Quoted 5 lines
drivers/hv/mshv_vfio.c new file +0
-0
[ ... ]
drivers/hv/mshv_vfio.c line 177
Quoted 12 lines
drivers/hv/mshv_vfio.c new file +12
-0+static long mshv_vfio_create_device (struct mshv_device *mshvdev) +{ +struct mshv_device *tmp; +struct mshv_vfio *mshv_vfio; + +/* Only one VFIO "device" per VM */ +hlist_for_each_entry (tmp, &mshvdev->device_pt->pt_devices, + device_ptnode) +if (tmp->device_ops == &mshv_vfio_device_ops) +return -EBUSY ; + + mshv_vfio =kzalloc_obj (*mshv_vfio);
[Severity: Medium]
Should this allocation include __GFP_ACCOUNT?
When a user space process creates a new MSHV VFIO device, it triggers mshv_vfio_create_device(). The kzalloc_obj() macro defaults to GFP_KERNEL, which omits the necessary accounting flags.
This allows a container to allocate kernel memory without being charged to its memory cgroup limit.
Quoted 10 lines
drivers/hv/mshv_vfio.c new file +10
-0+if (mshv_vfio ==NULL ) +return -ENOMEM ; + +INIT_LIST_HEAD (&mshv_vfio->file_list); +mutex_init (&mshv_vfio->lock); + + mshvdev->device_private = mshv_vfio; + +return 0 ; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Re: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
In reply to Mukesh RHi Mukesh,
Quoted 106 lines · On Fri, 31 Jul 2026 15:34:21 -0700 Mukesh R <[email protected]> wrote:
Add a new file to implement basic VFIO-MSHV bridge pseudo device. These functions are called in the VFIO framework, and credits to
kvm/vfio.cas this file was adapted from it. This is a basic version to build upon.Co-developed-by Wei Liu <[email protected]>Signed-off-by Wei Liu <[email protected]>Signed-off-by Mukesh R <[email protected]>--- drivers/hv/Makefile | 3 +- drivers/hv/mshv_vfio.c | 211 ++++++++++++++++++++++++++++++++++++++ include/uapi/linux/mshv.h | 1 + 3 files changed, 214 insertions(+), 1 deletion(-) create mode 100644 drivers/hv/mshv_vfio.cdrivers/hv/Makefile +2
-1@@ -14,7 +14,8 @@ hv_vmbus-y := vmbus_drv.o \ hv_vmbus-$(CONFIG_HYPERV_TESTING) += hv_debugfs.o hv_utils-y := hv_util.o hv_kvp.o hv_snapshot.o hv_utils_transport.o mshv_root-y := mshv_root_main.o mshv_synic.o mshv_eventfd.o mshv_irq.o \ - mshv_root_hv_call.o mshv_portid_table.o mshv_regions.o + mshv_root_hv_call.o mshv_portid_table.o mshv_regions.o \ + mshv_vfio.o mshv_root-$(CONFIG_DEBUG_FS) += mshv_debugfs.omshv_root-$(CONFIG_TRACEPOINTS) += mshv_trace.o
mshv_vtl-y := mshv_vtl_main.odrivers/hv/mshv_vfio.c new file +45
-0@@ -0,0 +1,211 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VFIO-MSHV bridge pseudo device +* +* Heavily inspired by the VFIO-KVM bridge pseudo device. +*/ +#include <linux/errno.h> +#include <linux/file.h> +#include <linux/list.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/slab.h> +#include <linux/vfio.h> +#include <asm/mshyperv.h> + +#include "mshv.h" +#include "mshv_root.h" + +struct mshv_vfio_file { +struct list_head node; +struct file *file;/* list of struct mshv_vfio_file */ +}; + +struct mshv_vfio { +struct list_head file_list; +struct mutex lock; +}; + +static bool mshv_vfio_file_is_valid (struct file *file) +{ +bool (*fn)(struct file *file); +bool ret; + + fn =symbol_get (vfio_file_is_valid); +if (!fn) +return false; + + ret =fn (file); + +symbol_put (vfio_file_is_valid); + +return ret; +} + +static long mshv_vfio_file_add(struct mshv_device *mshvdev, unsignedint fd) +{
drivers/hv/mshv_vfio.c new file +23
-0+struct mshv_vfio *mshv_vfio = mshvdev->device_private; +struct mshv_vfio_file *mvf; +struct file *filp; +long ret =0 ; + + filp =fget (fd); +if (!filp) +return -EBADF ; + +/* Ensure the FD is a vfio FD. */ +if (!mshv_vfio_file_is_valid (filp)) { + ret = -EINVAL ; + goto out_fput; + } + +mutex_lock (&mshv_vfio->lock); + +list_for_each_entry (mvf, &mshv_vfio->file_list, node) { +if (mvf->file == filp) { + ret = -EEXIST ; + goto out_unlock; + } + }
drivers/hv/mshv_vfio.c line 79
Quoted 10 lines
drivers/hv/mshv_vfio.c new file +10
-0+ + mvf =kzalloc (sizeof (*mvf),GFP_KERNEL_ACCOUNT ); +if (!mvf) { + ret = -ENOMEM ; + goto out_unlock; + } + + mvf->file =get_file (filp); +list_add_tail (&mvf->node, &mshv_vfio->file_list); +
Why there is no vfio_device_file_set_kvm equivalent for mshv? since VFIO device open/bind path will do vfio_device/group_get_kvm_safe() to ensure lifetime alignment. Otherwise, vfio group/device can outlive mshv partition.
If we do want to support this kvm-vfio bridge semantics beyond kvm, maybe this should be abstracted as a generic VFIO "hypervisor partition" association, with hypervisor-specific get/put callbacks, rather than adding an MSHV-only copy of the KVM hook. +Alex
Quoted 146 lines
drivers/hv/mshv_vfio.c new file +8
-0+out_unlock: +mutex_unlock (&mshv_vfio->lock); +out_fput: +fput (filp); +return ret; +} + +static long mshv_vfio_file_del(struct mshv_device *mshvdev, unsignedint fd) +{
drivers/hv/mshv_vfio.c new file +28
-0+struct mshv_vfio *mshv_vfio = mshvdev->device_private; +struct mshv_vfio_file *mvf; +long ret; + +CLASS (fd, f)(fd); + +if (fd_empty (f)) +return -EBADF ; + + ret = -ENOENT ; +mutex_lock (&mshv_vfio->lock); + +list_for_each_entry (mvf, &mshv_vfio->file_list, node) { +if (mvf->file !=fd_file (f)) +continue ; + +list_del (&mvf->node); +fput (mvf->file); +kfree (mvf); + ret =0 ; +break ; + } + +mutex_unlock (&mshv_vfio->lock); +return ret; +} + +static long mshv_vfio_set_file(struct mshv_device *mshvdev, longattr,
drivers/hv/mshv_vfio.c new file +27
-0+ void __user *arg) +{ +int32_t __user *argp = arg; +int32_t fd; + +switch (attr) { +case MSHV_DEV_VFIO_FILE_ADD : +if (get_user (fd, argp)) +return -EFAULT ; +return mshv_vfio_file_add (mshvdev, fd); + +case MSHV_DEV_VFIO_FILE_DEL : +if (get_user (fd, argp)) +return -EFAULT ; +return mshv_vfio_file_del (mshvdev, fd); + } + +return -ENXIO ; +} + +static long mshv_vfio_set_attr (struct mshv_device *mshvdev, +struct mshv_device_attr *attr) +{ +switch (attr->group) { +case MSHV_DEV_VFIO_FILE : + return mshv_vfio_set_file(mshvdev, attr->attr, +
u64_to_user_ptr(attr->addr));drivers/hv/mshv_vfio.c new file +68
-0+ } + +return -ENXIO ; +} + +static long mshv_vfio_has_attr (struct mshv_device *mshvdev, +struct mshv_device_attr *attr) +{ +switch (attr->group) { +case MSHV_DEV_VFIO_FILE : +switch (attr->attr) { +case MSHV_DEV_VFIO_FILE_ADD : +case MSHV_DEV_VFIO_FILE_DEL : +return 0 ; + } + +break ; + } + +return -ENXIO ; +} + +static long mshv_vfio_create_device (struct mshv_device *mshvdev) +{ +struct mshv_device *tmp; +struct mshv_vfio *mshv_vfio; + +/* Only one VFIO "device" per VM */ +hlist_for_each_entry (tmp, &mshvdev->device_pt->pt_devices, + device_ptnode) +if (tmp->device_ops == &mshv_vfio_device_ops) +return -EBUSY ; + + mshv_vfio =kzalloc_obj (*mshv_vfio); +if (mshv_vfio ==NULL ) +return -ENOMEM ; + +INIT_LIST_HEAD (&mshv_vfio->file_list); +mutex_init (&mshv_vfio->lock); + + mshvdev->device_private = mshv_vfio; + +return 0 ; +} + +/* This is called from mshv_device_fop_release() */ +static void mshv_vfio_release_device (struct mshv_device *mshvdev) +{ +struct mshv_vfio *mv = mshvdev->device_private; +struct mshv_vfio_file *mvf, *tmp; + +list_for_each_entry_safe (mvf, tmp, &mv->file_list, node) { +fput (mvf->file); +list_del (&mvf->node); +kfree (mvf); + } + +kfree (mv); +kfree (mshvdev); +} + +const struct mshv_device_ops mshv_vfio_device_ops = { + .device_name ="mshv-vfio" , + .device_create = mshv_vfio_create_device, + .device_release = mshv_vfio_release_device, + .device_set_attr = mshv_vfio_set_attr, + .device_has_attr = mshv_vfio_has_attr, +};include/uapi/linux/mshv.h +0
-0@@ -254,6 +254,7 @@ struct mshv_root_hvcall {#define MSHV_GET_GPAP_ACCESS_BITMAP _IOWR(MSHV_IOCTL, 0x06,struct
mshv_gpap_access_bitmap) /* Generic hypercall */
#defineMSHV_ROOT_HVCALL_IOWR(MSHV_IOCTL,0x07,
structmshv_root_hvcall) +#defineMSHV_CREATE_DEVICE_IOWR(MSHV_IOCTL, 0x08, struct mshv_create_device)
/*
********************************
Re: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
Quoted 5 lines · Jacob Pan
If we do want to support this kvm-vfio bridge semantics beyond kvm, maybe this should be abstracted as a generic VFIO "hypervisor partition" association, with hypervisor-specific get/put callbacks, rather than adding an MSHV-only copy of the KVM hook. +Alex
I don't like this patch at all.
I think we should try to harmonize all these sort of "external" cases as best we can. mshv, confidential compute, xen, etc all have a very similar shape.
I am discussing here with Aneesh an approach to use the viommu almost exclusively for this.
https://lore.kernel.org/all/[email protected]/
Can you imagine a way to do that for your stuff?
In that case I imagine the iommufd vdevice create is what attaches the vfio device to the partition, not a seperate ioctl over in some other driver. When the viommu is created it knows what partition it is affiliated with because it took in a FD that describes that partition.
Does mshv use KVM at all? Can a kvm fd describe the partition?
Jason
Wei Liu
maintainerRe: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
Quoted 19 lines · Jason Gunthorpe
Quoted 5 lines · Jacob Pan
If we do want to support this kvm-vfio bridge semantics beyond kvm, maybe this should be abstracted as a generic VFIO "hypervisor partition" association, with hypervisor-specific get/put callbacks, rather than adding an MSHV-only copy of the KVM hook. +Alex
I don't like this patch at all.
I think we should try to harmonize all these sort of "external" cases as best we can. mshv, confidential compute, xen, etc all have a very similar shape.
I am discussing here with Aneesh an approach to use the viommu almost exclusively for this.
Quoted 8 lines
Can you imagine a way to do that for your stuff?
In that case I imagine the iommufd vdevice create is what attaches the vfio device to the partition, not a seperate ioctl over in some other driver. When the viommu is created it knows what partition it is affiliated with because it took in a FD that describes that partition.
Does mshv use KVM at all? Can a kvm fd describe the partition?
No. MSHV doesn't use KVM. We don't have a kvm fd. We have an mshv fd to describe the partition. They are different virtualization solutions.
In the ideal world, the fd should be hypervisor agnostic, not tied to kvm.
Wei
Quoted 2 lines
Jason
Re: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
Quoted 7 lines · Wei Liu
Quoted 1 line
Does mshv use KVM at all? Can a kvm fd describe the partition?
No. MSHV doesn't use KVM. We don't have a kvm fd. We have an mshv fd to describe the partition. They are different virtualization solutions.
In the ideal world, the fd should be hypervisor agnostic, not tied to kvm.
There is a patch generalizing vfio to work on FDs instead of KVM structures:
https://patch.msgid.link/[email protected]
That's probably a good step long that line.
Jason
Re: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
Hi Jason,
Quoted 10 lines · On Thu, 3 Sep 2026 14:40:14 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 8 lines · Wei Liu
Quoted 1 line
Does mshv use KVM at all? Can a kvm fd describe the partition?
No. MSHV doesn't use KVM. We don't have a kvm fd. We have an mshv fd to describe the partition. They are different virtualization solutions.
In the ideal world, the fd should be hypervisor agnostic, not tied to kvm.
Quoted 6 lines
There is a patch generalizing vfio to work on FDs instead of KVM structures:
https://patch.msgid.link/[email protected]
That's probably a good step long that line.
Yeah, ideally, it would be removing/rename device/group->kvm to something like group/device->vm_file?
Re: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
Quoted 15 lines · Jacob Pan
Hi Jason,
Quoted 10 lines · On Thu, 3 Sep 2026 14:40:14 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 8 lines · Wei Liu
Quoted 1 line
Does mshv use KVM at all? Can a kvm fd describe the partition?
No. MSHV doesn't use KVM. We don't have a kvm fd. We have an mshv fd to describe the partition. They are different virtualization solutions.
In the ideal world, the fd should be hypervisor agnostic, not tied to kvm.
Quoted 8 lines
Quoted 6 lines
There is a patch generalizing vfio to work on FDs instead of KVM structures:
https://patch.msgid.link/[email protected]
That's probably a good step long that line.
Yeah, ideally, it would be removing/rename device/group->kvm to something like group/device->vm_file?
Maybe it could? Do you need the file in VFIO land or is what you showed in the other message OK?
I think it should be broadly fine to pass in the file that way and hold a reference on it in the viommu object..
Jason
Re: [PATCH v5 3/9] mshv: Introduce basic mshv bridge device for VFIO to build upon
In reply to Jason GunthorpeHi Jason,
Quoted 17 lines · On Thu, 3 Sep 2026 11:42:36 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 5 lines · Jacob Pan
If we do want to support this kvm-vfio bridge semantics beyond kvm, maybe this should be abstracted as a generic VFIO "hypervisor partition" association, with hypervisor-specific get/put callbacks, rather than adding an MSHV-only copy of the KVM hook. +Alex
I don't like this patch at all.
I think we should try to harmonize all these sort of "external" cases as best we can. mshv, confidential compute, xen, etc all have a very similar shape.
I am discussing here with Aneesh an approach to use the viommu almost exclusively for this.
Quoted 8 lines
https://lore.kernel.org/all/[email protected]/
Can you imagine a way to do that for your stuff?
In that case I imagine the iommufd vdevice create is what attaches the vfio device to the partition, not a seperate ioctl over in some other driver. When the viommu is created it knows what partition it is affiliated with because it took in a FD that describes that partition.
Yes, viommu/vdevice model maps well in my POC/RFC(WIP) without new ioctls.
vdevice is attached to the vm partition via viommu (holds vm_fd). i.e.
struct iommu_vdevice_alloc alloc = {
.size = sizeof(alloc),
.viommu_id = viommu_id,
.dev_id = dev_id,
.virt_id = vm_device_id, //vmm assigned
};
int rc;
rc = do_ioctl(iommufd, IOMMU_VDEVICE_ALLOC, &alloc,
"IOMMU_VDEVICE_ALLOC");In our case, we don't need a vdevice_init op since all we need is the virt_id, hyperv iommu driver will retrieved the vmm assigned logical device ID (virt_id) by existing API:
static int hyperv_iommu_external_attach_dev(struct iommu_domain *domain,
struct device *dev, struct iommu_domain *old)
{
ret = iommufd_viommu_get_vdev_id(external_domain->viommu, dev,
&vdev_id);Then use vdev_id for the hypercall to attach external domain.
I currently have a new IOMMU_VIOMMU_TYPE_MSHV with:
/**
* struct iommu_viommu_mshv - Microsoft Hypervisor virtual IOMMU
* (IOMMU_VIOMMU_TYPE_MSHV)
* @vm_fd: MSHV partition file descriptor
* @flags: Must be 0
* @__reserved: Must be 0
*/
struct iommu_viommu_mshv {
__s32 vm_fd;
__u32 flags;
__aligned_u64 __reserved;
};That is MSHV-specific only because the fd validation and partition-ID lookup are MSHV-specific today. I think this could be generalized later into a common "external/hypervisor partition" vIOMMU type with driver-specific fd validation/get/put semantics.
Thanks,
Jacob
PATCH v5 4/9 mshv: Add ioctl support for MSHV-VFIO bridge device
In reply to Mukesh RAdd ioctl support for MSHV devices for a partition. At present only VFIO device types are supported, but more could be added. At a high level, a partition ioctl to create device verifies it is of type VFIO and does some setup for bridge code in mshv_vfio.c. Adapted from KVM device ioctls.
--- drivers/hv/mshv_root_main.c | 119 ++++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+)
drivers/hv/mshv_root_main.c +119 -0
@@ -1625,6 +1625,122 @@ mshv_partition_ioctl_initialize(struct mshv_partition *partition)return ret; } +static long mshv_device_attr_ioctl (struct mshv_device *mshv_dev,int cmd, +ulong uarg) +{ +struct mshv_device_attr attr; +const struct mshv_device_ops *devops = mshv_dev->device_ops; + +if (copy_from_user (&attr, (void __user *)uarg,sizeof (attr))) +return -EFAULT ; + +switch (cmd) { +case MSHV_SET_DEVICE_ATTR : +if (devops->device_set_attr) +return devops->device_set_attr (mshv_dev, &attr); +break ; +case MSHV_HAS_DEVICE_ATTR : +if (devops->device_has_attr) +return devops->device_has_attr (mshv_dev, &attr); +break ; + } + +return -EPERM ; +} + +static long mshv_device_fop_ioctl (struct file *filp,unsigned int cmd, +ulong uarg) +{ +struct mshv_device *mshv_dev = filp->private_data; + +switch (cmd) { +case MSHV_SET_DEVICE_ATTR : +case MSHV_HAS_DEVICE_ATTR : +return mshv_device_attr_ioctl (mshv_dev, cmd, uarg); + } + +return -ENOTTY ; +} + +static int mshv_device_fop_release (struct inode *inode,struct file *filp) +{ +struct mshv_device *mshv_dev = filp->private_data; +struct mshv_partition *partition = mshv_dev->device_pt; + +if (mshv_dev->device_ops->device_release) { +mutex_lock (&partition->pt_mutex); +hlist_del (&mshv_dev->device_ptnode); + mshv_dev->device_ops->device_release (mshv_dev); +mutex_unlock (&partition->pt_mutex); + } + +mshv_partition_put (partition); +return 0 ; +} + +static const struct file_operations mshv_device_fops = { + .owner =THIS_MODULE , + .unlocked_ioctl = mshv_device_fop_ioctl, + .release = mshv_device_fop_release, +}; + +static long mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *uarg) +{ +long rc; +struct mshv_create_device devargk; +struct mshv_device *mshv_dev; +const struct mshv_device_ops *vfio_ops; + +if (copy_from_user (&devargk, uarg,sizeof (devargk))) +return -EFAULT ; + +/* At present, only VFIO is supported */ +if (devargk.type !=MSHV_DEV_TYPE_VFIO ) +return -ENODEV ; + +if (devargk.flags &MSHV_CREATE_DEVICE_TEST ) +return 0 ; + +if (devargk.flags)/* currently, only MSHV_CREATE_DEVICE_TEST */ +return -EINVAL ; + +/* This is freed later by mshv_vfio_release_device() */ + mshv_dev =kzalloc_obj (*mshv_dev); +if (mshv_dev ==NULL ) +return -ENOMEM ; + + vfio_ops = &mshv_vfio_device_ops; + mshv_dev->device_ops = vfio_ops; + mshv_dev->device_pt = partition; + + rc = vfio_ops->device_create (mshv_dev); +if (rc <0 ) { +kfree (mshv_dev); +return rc; + } + +hlist_add_head (&mshv_dev->device_ptnode, &partition->pt_devices); + +mshv_partition_get (partition); + rc =anon_inode_getfd (vfio_ops->device_name, &mshv_device_fops, + mshv_dev,O_RDWR |O_CLOEXEC ); +if (rc <0 ) + goto undo_out; + + devargk.fd = rc; +if (copy_to_user (uarg, &devargk,sizeof (devargk))) +return -EFAULT ;/* cleanup in mshv_device_fop_release() */ + +return 0 ; + +undo_out: +hlist_del (&mshv_dev->device_ptnode); + vfio_ops->device_release (mshv_dev);/* will kfree(mshv_dev) */ +mshv_partition_put (partition); +return rc; +} +static long mshv_partition_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg) {
@@ -1661,6 +1777,9 @@ mshv_partition_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)case MSHV_ROOT_HVCALL : ret =mshv_ioctl_passthru_hvcall (partition, true, uarg);break ; +case MSHV_CREATE_DEVICE : + ret =mshv_partition_ioctl_create_device (partition, uarg); +break ;default : ret = -ENOTTY ; }
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 4/9] mshv: Add ioctl support for MSHV-VFIO bridge device
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Failing copy_to_user after anon_inode_getfd causes a file descriptor leak and leaves partition…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Failing copy_to_user after anon_inode_getfd causes a file descriptor leak and leaves partition state inconsistent.
- [Medium] The MSHV_CREATE_DEVICE_TEST flag ignores all other flags, rendering capability checks broken. --
--- Patch [4]: [PATCH v5 4/9] mshv: Add ioctl support for MSHV-VFIO bridge device --- commit a96aa99c275e6a51436ce19dce86ac7d1a20b10d
Author: Mukesh R <[email protected]>
mshv: Add ioctl support for MSHV-VFIO bridge device
This commit adds ioctl support for creating MSHV VFIO bridge devices in a partition. It introduces a partition ioctl to verify the VFIO type and sets up the bridge code.
Quoted 4 lines
drivers/hv/mshv_root_main.c +0
-0
[ ... ]
Quoted 1 line
drivers/hv/mshv_root_main.c +0
-0@@ -1625,6 +1625,122 @@ mshv_partition_ioctl_initialize(struct mshv_partition *partition)
[ ... ]
Quoted 10 lines
drivers/hv/mshv_root_main.c +10
-0+static long mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *uarg) +{ +long rc; +struct mshv_create_device devargk; +struct mshv_device *mshv_dev; +const struct mshv_device_ops *vfio_ops; + +if (copy_from_user (&devargk, uarg,sizeof (devargk))) +return -EFAULT ;
drivers/hv/mshv_root_main.c mshv_partition_ioctl_initialize(struct mshv_partition *partition) · line 1706
Quoted 10 lines
drivers/hv/mshv_root_main.c +10
-0+ +/* At present, only VFIO is supported */ +if (devargk.type !=MSHV_DEV_TYPE_VFIO ) +return -ENODEV ; + +if (devargk.flags &MSHV_CREATE_DEVICE_TEST ) +return 0 ; + +if (devargk.flags)/* currently, only MSHV_CREATE_DEVICE_TEST */ +return -EINVAL ;
[Severity: Medium]
If userspace calls this with MSHV_CREATE_DEVICE_TEST combined with an unknown flag, won't this unconditionally return 0 and bypass the check for unsupported flags?
Could this mislead userspace into thinking a new flag is supported when probing capabilities on older kernels?
[ ... ]
drivers/hv/mshv_root_main.c mshv_partition_ioctl_initialize(struct mshv_partition *partition) · line 1733
Quoted 9 lines
drivers/hv/mshv_root_main.c +9
-0+mshv_partition_get (partition); + rc =anon_inode_getfd (vfio_ops->device_name, &mshv_device_fops, + mshv_dev,O_RDWR |O_CLOEXEC ); +if (rc <0 ) + goto undo_out; + + devargk.fd = rc; +if (copy_to_user (uarg, &devargk,sizeof (devargk))) +return -EFAULT ;/* cleanup in mshv_device_fop_release() */
[Severity: High]
Does this leak the file descriptor and the associated mshv_dev kernel structures if copy_to_user() fails?
Since anon_inode_getfd() immediately installs the new file descriptor into the task's fdtable, returning -EFAULT here leaves the fd permanently open until the process exits.
Also, because the fd is installed before copy_to_user() is called, could another thread guess the fd and start using it while the calling thread receives -EFAULT and assumes device creation failed?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
PATCH v5 5/9 mshv: Import data structs around device passthru from hyperv headers
In reply to Mukesh RCopy/import from Hyper-V public headers, definitions and declarations that are related to creating domains in the hypervisor, attaching devices to them, etc.
--- include/hyperv/hvgdk_mini.h | 8 +++++ include/hyperv/hvhdk_mini.h | 67 +++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)
include/hyperv/hvgdk_mini.h +8 -0
@@ -326,6 +326,8 @@ union hv_hypervisor_version_info {/* stimer Direct Mode is available */ #define HV_STIMER_DIRECT_MODE_AVAILABLE BIT(19) +#define HV_DEVICE_DOMAIN_AVAILABLE BIT(24) +/* * Implementation recommendations. Indicates which behaviors the hypervisor * recommends the OS implement for optimal performance.
@@ -486,9 +488,15 @@ union hv_vp_assist_msr_contents { /* HV_REGISTER_VP_ASSIST_PAGE */#define HVCALL_GET_VP_INDEX_FROM_APIC_ID 0x009a#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0 +#define HVCALL_CREATE_DEVICE_DOMAIN 0x00b1 +#define HVCALL_ATTACH_DEVICE_DOMAIN 0x00b2 +#define HVCALL_MAP_DEVICE_GPA_PAGES 0x00b3 +#define HVCALL_UNMAP_DEVICE_GPA_PAGES 0x00b4#define HVCALL_SIGNAL_EVENT_DIRECT 0x00c0#define HVCALL_POST_MESSAGE_DIRECT 0x00c1#define HVCALL_DISPATCH_VP 0x00c2 +#define HVCALL_DETACH_DEVICE_DOMAIN 0x00c4 +#define HVCALL_DELETE_DEVICE_DOMAIN 0x00c5#define HVCALL_GET_GPA_PAGES_ACCESS_STATES 0x00c9#define HVCALL_ACQUIRE_SPARSE_SPA_PAGE_HOST_ACCESS 0x00d7#define HVCALL_RELEASE_SPARSE_SPA_PAGE_HOST_ACCESS 0x00d8
include/hyperv/hvhdk_mini.h +67 -0
71 lines not on this page · raw message
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 5/9] mshv: Import data structs around device passthru from hyperv headers
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] The fields in `union hv_device_domain_id` are ordered incorrectly, resulting in a flawed bitfield…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The fields in union hv_device_domain_id are ordered incorrectly, resulting in a flawed bitfield layout that contradicts the domain ID constants.
- [High] struct hv_input_create_device_domain is exactly 28 bytes in size, violating the MS C implicit padding expectations for 8-byte aligned structures. --
--- Patch [5]: [PATCH v5 5/9] mshv: Import data structs around device passthru from hyperv headers --- commit 9932d93213b193c633772c7ddbee866c39282647
Author: Mukesh R <[email protected]>
mshv: Import data structs around device passthru from hyperv headers
Copy/import from Hyper-V public headers, definitions and declarations that are related to creating domains in the hypervisor, attaching devices to them, etc.
Quoted 4 lines
include/hyperv/hvhdk_mini.h +0
-0
[ ... ]
include/hyperv/hvhdk_mini.h union hv_device_id { /* HV_DEVICE_ID */ · line 566
Quoted 12 lines
include/hyperv/hvhdk_mini.h +12
-0+/* ID for stage 2 default domain and NULL domain */ +#define HV_DEVICE_DOMAIN_ID_S2_DEFAULT 0 +#define HV_DEVICE_DOMAIN_ID_S2_NULL 0xFFFFFFFFULL + +union hv_device_domain_id { +u64 as_uint64; +struct { +u32 type :4 ; +u32 reserved :28 ; +u32 id; + }; +} __packed;
[Severity: High]
Are the fields in this struct ordered correctly for the ABI?
In little-endian C bitfields, the first member occupies the lowest bits. By placing type and reserved before id, they will occupy the lower 32 bits while id occupies the upper 32 bits.
This seems to contradict the constant HV_DEVICE_DOMAIN_ID_S2_NULL defined just above this union, which expects the id (0xFFFFFFFF) in the lower 32 bits and type (0) in the upper 32 bits.
Will this cause the hypervisor to receive swapped values and fail domain attachment?
[ ... ]
include/hyperv/hvhdk_mini.h union hv_device_id { /* HV_DEVICE_ID */ · line 587
Quoted 4 lines
include/hyperv/hvhdk_mini.h +4
-0+struct hv_input_create_device_domain {/* HV_INPUT_CREATE_DEVICE_DOMAIN */ +struct hv_input_device_domain device_domain; +union hv_create_device_domain_flags create_device_domain_flags; +} __packed;
[Severity: High]
Does this struct need explicit padding at the end to match the hypervisor's 32-byte size expectation?
Because of the __packed attribute, the compiler won't add the 4-byte implicit trailing padding required to bring the struct to its natural 32-byte size (due to the 8-byte alignment of device_domain).
If standard hypercall initialization clears the input using:memset(in, 0, sizeof(*in));
the sizeof will be 28, leaving the last 4 bytes of the 32-byte block uninitialized on the shared hypercall page.
If the hypervisor validates these padding bytes to be zero, could this fail with HV_STATUS_INVALID_PARAMETER if there is leftover garbage from a previous hypercall?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
PATCH v5 6/9 PCI: hv: Export hv_build_devid_type_pci() and change return type
In reply to Mukesh ROn Hyper-V, most hypercalls related to PCI passthru to map/unmap regions, interrupts, etc need a device ID as a parameter. This device ID refers to that specific device during the lifetime of passthru. So, make hv_build_devid_type_pci() public and change return type to u64 to enforce it's size.
--- arch/x86/hyperv/irqdomain.c | 9 +++++---- arch/x86/include/asm/mshyperv.h | 6 ++++++ 2 files changed, 11 insertions(+), 4 deletions(-)
arch/x86/hyperv/irqdomain.c +5 -4
@@ -112,7 +112,7 @@ static int get_rid_cb(struct pci_dev *pdev, u16 alias, void *data)return 0 ; } -static union hv_device_id hv_build_devid_type_pci (struct pci_dev *pdev) +u64 hv_build_devid_type_pci (struct pci_dev *pdev) {int pos;union hv_device_id hv_devid;
@@ -172,8 +172,9 @@ static union hv_device_id hv_build_devid_type_pci(struct pci_dev *pdev) } out: -return hv_devid; +return hv_devid.as_uint64; } +EXPORT_SYMBOL_GPL (hv_build_devid_type_pci);/* * hv_map_msi_interrupt() - Map the MSI IRQ in the hypervisor.
@@ -196,7 +197,7 @@ int hv_map_msi_interrupt(struct irq_data *data, msidesc =irq_data_get_msi_desc (data); pdev =msi_desc_to_pci_dev (msidesc); - hv_devid =hv_build_devid_type_pci (pdev); + hv_devid.as_uint64 =hv_build_devid_type_pci (pdev); cpu =cpumask_first (irq_data_get_effective_affinity_mask (data)); return hv_map_interrupt(hv_devid, false, cpu, cfg->vector,
@@ -271,7 +272,7 @@ static int hv_unmap_msi_interrupt(struct pci_dev *pdev, {union hv_device_id hv_devid; - hv_devid =hv_build_devid_type_pci (pdev); + hv_devid.as_uint64 =hv_build_devid_type_pci (pdev);return hv_unmap_interrupt (hv_devid.as_uint64, irq_entry); }
arch/x86/include/asm/mshyperv.h +6 -0
@@ -248,6 +248,12 @@ void hv_crash_asm_end(void);static inline void hv_root_crash_init (void ) {}#endif /* CONFIG_MSHV_ROOT && CONFIG_CRASH_DUMP */ +#ifdef CONFIG_PCI_MSI +u64 hv_build_devid_type_pci (struct pci_dev *pdev); +#else +static inline u64 hv_build_devid_type_pci (struct pci_dev *pdev) {return 0 ; } +#endif +#else /* CONFIG_HYPERV */ static inline void hyperv_init (void ) {}static inline void hyperv_setup_mmu_ops (void ) {}
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 6/9] PCI: hv: Export hv_build_devid_type_pci() and change return type
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [Medium] Incomplete API export for cross-architecture code
Pre-existing issues:
- [High] Resource leak of multi-vector MSI IRQs during teardown --
--- Patch [6]: [PATCH v5 6/9] PCI: hv: Export hv_build_devid_type_pci() and change return type --- commit 2b1777d94c87d493032cb98e1c21a4774a30a00a
Author: Mukesh R <[email protected]>
PCI: hv: Export hv_build_devid_type_pci() and change return type
On Hyper-V, most hypercalls related to PCI passthru to map/unmap regions, interrupts, etc need a device ID as a parameter. This device ID refers to that specific device during the lifetime of passthru. So, make hv_build_devid_type_pci() public and change return type to u64 to enforce it's size.
Quoted 4 lines
arch/x86/hyperv/irqdomain.c +0
-0
[ ... ]
arch/x86/hyperv/irqdomain.c static union hv_device_id hv_build_devid_type_pci(struct pci_dev *pdev) · line 177
Quoted 8 lines
arch/x86/hyperv/irqdomain.c +2
-1@@ -172,8 +172,9 @@ static union hv_device_id hv_build_devid_type_pci(struct pci_dev *pdev) } out: -return hv_devid; +return hv_devid.as_uint64; } +EXPORT_SYMBOL_GPL (hv_build_devid_type_pci);
[Severity: Medium]
Will exporting and implementing this API solely in an x86-specific file cause build failures for ARM64?
The commit message mentions making this public for PCI passthrough hypercalls, but the mshv driver supports both x86 and ARM64. It appears cross-architecture builds will lack this implementation when they try to utilize the newly exported API.
[ ... ]
Quoted 5 lines
arch/x86/include/asm/mshyperv.h +0
-0
arch/x86/include/asm/mshyperv.h void hv_crash_asm_end(void); · line 256
Quoted 9 lines
arch/x86/include/asm/mshyperv.h +6
-0@@ -248,6 +248,12 @@ void hv_crash_asm_end(void);static inline void hv_root_crash_init (void ) {}#endif /* CONFIG_MSHV_ROOT && CONFIG_CRASH_DUMP */ +#ifdef CONFIG_PCI_MSI +u64 hv_build_devid_type_pci (struct pci_dev *pdev); +#else +static inline u64 hv_build_devid_type_pci (struct pci_dev *pdev) {return 0 ; } +#endif +
[Severity: Medium]
Similarly, should this declaration be placed in a shared or cross-architecture header so that ARM64 drivers can safely include it without breaking?
While reviewing this file, I also noticed an issue in hv_msi_domain_free().
[Severity: High]
This is a pre-existing issue, but does this loop fail to tear down the remaining vectors for multi-vector MSI IRQs?
arch/x86/hyperv/irqdomain.c:hv_msi_domain_free() {
...
for (int i = 0; i < nr_irqs; ++i) {
struct irq_data *irqd = irq_domain_get_irq_data(d, virq);
...
}Because it looks up the base virq on every iteration instead of virq + i, does this continuously leak hypervisor-level interrupt resources and chip_data memory allocations during a device unbind?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Re: [PATCH v5 6/9] PCI: hv: Export hv_build_devid_type_pci() and change return type
In reply to Mukesh RQuoted 5 lines · Mukesh R
On Hyper-V, most hypercalls related to PCI passthru to map/unmap regions, interrupts, etc need a device ID as a parameter. This device ID refers to that specific device during the lifetime of passthru. So, make
hv_build_devid_type_pci()public and change return type to u64 to enforce it's size.
s/it's/its/
This patch doesn't have anything to do with the PCI core specifically, so the "PCI: hv:" subject line looks a little funny and doesn't look anything like previous history of these two files.
Quoted 69 lines
Signed-off-by Mukesh R <[email protected]>Reviewed-by Souradeep Chakrabarti <[email protected]>--- arch/x86/hyperv/irqdomain.c | 9 +++++---- arch/x86/include/asm/mshyperv.h | 6 ++++++ 2 files changed, 11 insertions(+), 4 deletions(-)arch/x86/hyperv/irqdomain.c +5
-4@@ -112,7 +112,7 @@ static int get_rid_cb(struct pci_dev *pdev, u16 alias, void *data)return 0 ; } -static union hv_device_id hv_build_devid_type_pci (struct pci_dev *pdev) +u64 hv_build_devid_type_pci (struct pci_dev *pdev) {int pos;union hv_device_id hv_devid;@@ -172,8 +172,9 @@ static union hv_device_id hv_build_devid_type_pci(struct pci_dev *pdev) } out: -return hv_devid; +return hv_devid.as_uint64; } +EXPORT_SYMBOL_GPL (hv_build_devid_type_pci);/* * hv_map_msi_interrupt() - Map the MSI IRQ in the hypervisor. @@ -196,7 +197,7 @@ int hv_map_msi_interrupt(struct irq_data *data, msidesc =irq_data_get_msi_desc (data); pdev =msi_desc_to_pci_dev (msidesc); - hv_devid =hv_build_devid_type_pci (pdev); + hv_devid.as_uint64 =hv_build_devid_type_pci (pdev); cpu =cpumask_first (irq_data_get_effective_affinity_mask (data)); return hv_map_interrupt(hv_devid, false, cpu, cfg->vector,@@ -271,7 +272,7 @@ static int hv_unmap_msi_interrupt(struct pci_dev *pdev, {union hv_device_id hv_devid; - hv_devid =hv_build_devid_type_pci (pdev); + hv_devid.as_uint64 =hv_build_devid_type_pci (pdev);return hv_unmap_interrupt (hv_devid.as_uint64, irq_entry); }arch/x86/include/asm/mshyperv.h +6
-0@@ -248,6 +248,12 @@ void hv_crash_asm_end(void);static inline void hv_root_crash_init (void ) {}#endif /* CONFIG_MSHV_ROOT && CONFIG_CRASH_DUMP */ +#ifdef CONFIG_PCI_MSI +u64 hv_build_devid_type_pci (struct pci_dev *pdev); +#else +static inline u64 hv_build_devid_type_pci (struct pci_dev *pdev) {return 0 ; } +#endif +#else /* CONFIG_HYPERV */ static inline void hyperv_init (void ) {}static inline void hyperv_setup_mmu_ops (void ) {}Signature · 1 line
2.51.2.vfs.0.1
PATCH v5 7/9 x86/hyperv: Implement Hyper-V virtual IOMMU
In reply to Mukesh RAdd a new file to implement a virtual IOMMU that works with Microsoft Hyper-V hypervisor. The hypervisor claims the IOMMU upon boot, and this driver communicates with it for creating and deleting paging domains, mapping and unmapping of pages, etc. The default domain type is IDENTITY. Lastly, this is only supported on baremetal or nested root/dom0, and not L1VH root/dom0.
--- arch/x86/kernel/pci-dma.c | 2 + drivers/iommu/Kconfig | 1 + drivers/iommu/hyperv/Kconfig | 15 + drivers/iommu/hyperv/Makefile | 1 + drivers/iommu/hyperv/hv-iommu-root.c | 706 +++++++++++++++++++++++++++ include/asm-generic/mshyperv.h | 3 + include/linux/hyperv.h | 6 + 7 files changed, 734 insertions(+) create mode 100644 drivers/iommu/hyperv/Kconfig create mode 100644 drivers/iommu/hyperv/hv-iommu-root.c
arch/x86/kernel/pci-dma.c +2 -0
@@ -8,6 +8,7 @@#include <linux/gfp.h> #include <linux/pci.h> #include <linux/amd-iommu.h> +#include <linux/hyperv.h> #include <asm/proto.h> #include <asm/dma.h>
@@ -103,6 +104,7 @@ void __init pci_iommu_alloc(void) }pci_swiotlb_detect ();gart_iommu_hole_init (); +hv_iommu_detect ();amd_iommu_detect ();detect_intel_iommu ();swiotlb_init (x86_swiotlb_enable, x86_swiotlb_flags);
drivers/iommu/Kconfig +1 -0
@@ -197,6 +197,7 @@ source "drivers/iommu/arm/Kconfig" source "drivers/iommu/intel/Kconfig" source "drivers/iommu/iommufd/Kconfig" source "drivers/iommu/riscv/Kconfig" +source "drivers/iommu/hyperv/Kconfig" config IRQ_REMAP bool "Support for Interrupt Remapping"
drivers/iommu/hyperv/Kconfig new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Hyper-V IOMMU support + +config HYPERV_IOMMU + bool "Hyper-V IOMMU Device" + depends on HYPERV && X86 + select IOMMU_API + default HYPERV + help + This enables Hyper-V pseudo IOMMU device. When running as privileged + VM aka root on Microsoft Hyper-V hypervisor, this must be enabled + for doing any PCI passthru of devices to guest VMs. This applies to + both PFs and VFs. When enabling this, it is best to disable amd/intel + iommus via: intel_iommu=off amd_iommu=off as the hypervisor really + owns the iommu.
drivers/iommu/hyperv/Makefile +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_IRQ_REMAP) += hv-irq-remap-x86.o +obj-$(CONFIG_HYPERV_IOMMU) += hv-iommu-root.o
drivers/iommu/hyperv/hv-iommu-root.c new file +706 -0
706 lines not on this page · raw message
include/asm-generic/mshyperv.h +3 -0
@@ -28,6 +28,9 @@#define VTPM_BASE_ADDRESS 0xfed40000 +#define HV_REP_COUNT_MAX \ + (HV_HYPERCALL_REP_COMP_MASK >> HV_HYPERCALL_REP_COMP_OFFSET) +enum hv_partition_type {HV_PARTITION_TYPE_GUEST ,HV_PARTITION_TYPE_ROOT ,
include/linux/hyperv.h +6 -0
@@ -1769,4 +1769,10 @@ static inline unsigned long virt_to_hvpfn(void *addr)#define HVPFN_DOWN (x) ((x) >> HV_HYP_PAGE_SHIFT)#define page_to_hvpfn (page) (page_to_pfn(page) * NR_HV_HYP_PAGES_IN_PAGE) +#ifdef CONFIG_HYPERV_IOMMU +void __init hv_iommu_detect (void ); +#else +static inline void hv_iommu_detect (void ) { } +#endif /* CONFIG_HYPERV_IOMMU */ +#endif /* _HYPERV_H */
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider: - [High] Broken partial unmapping logic incorrectly removes entire mapping ranges, leaks mappings, and…
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] Broken partial unmapping logic incorrectly removes entire mapping ranges, leaks mappings, and causes catastrophic error rollback failures.
- [High] Calling dev_iommu_priv_set() in attach_dev callback without holding iommu_probe_device_lock causes a lockdep violation.
- [High] hv_iommu_map_pages ignores the caller's gfp_t argument and calls a potentially sleeping function.
- [Medium] Atomic counter wrap-around in domain ID allocation risks colliding with reserved hypervisor domain IDs.
- [Low] The __setup callback hv_iommu_setup_skip incorrectly returns 0, causing the kernel to treat the command-line argument as an unknown parameter. --
--- Patch [7]: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU --- commit cdb89379b7cc5351b8dae8a1be7ede22f440c196
Author: Mukesh R <[email protected]>
x86/hyperv: Implement Hyper-V virtual IOMMU
Add a new file to implement a virtual IOMMU that works with Microsoft Hyper-V hypervisor. The driver communicates with the hypervisor for creating and deleting paging domains and mapping pages. The default domain type is IDENTITY.
Quoted 6 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +0
-0@@ -0,0 +1,706 @@
[ ... ]
drivers/iommu/hyperv/hv-iommu-root.c line 21
Quoted 7 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +7
-0+static char *pci_devs_to_skip; +static int __init hv_iommu_setup_skip (char *str) +{ + pci_devs_to_skip = str; + +return 0 ; +}
[Severity: Low]
Should this __setup callback return 1? Returning 0 causes the kernel to treat the command-line argument as an unknown parameter and pass it to the init process as an environment variable.
[ ... ]
Quoted 9 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +9
-0+static size_t hv_iommu_del_tree_mappings (struct hv_domain *hvdom, +unsigned long iova,size_t size) +{ +unsigned long flags; +size_t unmapped =0 ; +unsigned long last = iova + size -1 ; +struct hv_iommu_mapping *mapping =NULL ; +struct interval_tree_node *node, *next; +
drivers/iommu/hyperv/hv-iommu-root.c line 147
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+spin_lock_irqsave (&hvdom->mappings_lock, flags); + next =interval_tree_iter_first (&hvdom->mappings_tree, iova, last); +while (next) { + node = next; + mapping = container_of(node, struct hv_iommu_mapping, iova); + next =interval_tree_iter_next (node, iova, last); + +/* Splitting of a mapping is not supported at present */ +if (mapping->iova.start < iova) +break ;
[Severity: High]
How does this code handle partial unmaps? If an unmap request starts in the middle of a mapping, doesn't the break statement here leave the memory mapped? Furthermore, if the request starts at the beginning but is smaller than the full mapping, this removes the entire node and returns its full size, prompting the caller to erroneously unmap the full region from hardware.
[ ... ]
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+static struct iommu_domain *hv_iommu_domain_alloc_paging (struct device *dev) +{ +struct hv_domain *hvdom; +int rc; +u32 unique_id; +u64 ptid =hv_get_current_partid (); + +if (ptid ==HV_PARTITION_ID_INVALID ) +return NULL ; +
drivers/iommu/hyperv/hv-iommu-root.c line 208
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+ hvdom = kzalloc_obj(struct hv_domain); +if (hvdom ==NULL ) +return NULL ; + +spin_lock_init (&hvdom->mappings_lock); + hvdom->mappings_tree =RB_ROOT_CACHED ; + + unique_id = (u32 )atomic_inc_return (&hv_unique_id); +if (unique_id ==HV_DEVICE_DOMAIN_ID_S2_NULL )/* ie, UINTMAX */ + goto out_err;
[Severity: Medium]
What happens when this 32-bit atomic counter wraps around? Since it will wrap to 0 (which is HV_DEVICE_DOMAIN_ID_S2_DEFAULT), could this new domain improperly map to the hypervisor's default domain and cause hypercall failures? Should an IDA allocator be used instead?
[ ... ]
Quoted 7 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +7
-0+static int hv_iommu_attach_dev (struct iommu_domain *immdom,struct device *dev, +struct iommu_domain *old) +{ +struct pci_dev *pdev; +int rc; +struct hv_domain *hvdom_new =to_hv_domain (immdom); +struct hv_domain *hvdom_prev =to_hv_domain (old);
[ ... ]
drivers/iommu/hyperv/hv-iommu-root.c line 348
Quoted 4 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +4
-0+ rc =hv_iommu_att_dev2dom (hvdom_new, pdev); + +if (rc ==0 ) +dev_iommu_priv_set (dev, hvdom_new);/* sets "private" field */
[Severity: High]
Can this trigger a lockdep splat? The core function dev_iommu_priv_set() requires iommu_probe_device_lock to be held, but this attach_dev callback executes with group->mutex held. Is it safe to use this private field for dynamic domain state?
Quoted 8 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +8
-0+else { +if (hvdom_prev && !hv_special_domain (hvdom_prev)) +WARN (1 ,"Failed to attach dev:%s\n" ,pci_name (pdev)); +dev_iommu_priv_set (dev,NULL ); + } + +return rc; +}
[ ... ]
Quoted 4 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +4
-0+static int hv_iommu_map_pages (struct iommu_domain *immdom,ulong iova, +phys_addr_t paddr,size_t pgsize,size_t pgcount, +int prot,gfp_t gfp,size_t *mapped) +{
[ ... ]
drivers/iommu/hyperv/hv-iommu-root.c line 490
Quoted 4 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +4
-0+if (hv_result (status) ==HV_STATUS_INSUFFICIENT_MEMORY ) { + ret =hv_call_deposit_pages (NUMA_NO_NODE , + hv_current_partition_id, +256 );
[Severity: High]
Does this sequence sleep in atomic context? The IOMMU map_pages callback can be invoked in atomic contexts (such as inside interrupt handlers) and is passed a gfp_t argument. However, this ignores the gfp argument and calls hv_call_deposit_pages(), which allocates pages using GFP_KERNEL.
[ ... ]
drivers/iommu/hyperv/hv-iommu-root.c line 509
Quoted 11 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +11
-0+if (!hv_result_success (status)) { +size_t done_size = done <<HV_HYP_PAGE_SHIFT ; + +hv_status_err (status,"pgs:%lx/%lx iova:%lx\n" , + done, npages, iova); +/* +* lookup tree has all mappings [0 - size-1]. Below unmap will +* only remove from [0 - done], we need to remove second chunk +* [done+1 - size-1]. +*/ +hv_iommu_del_tree_mappings (hvdom, iova, size - done_size);
[Severity: High]
Is the error cleanup path unmapping memory that was never mapped? If a multi-page map fails halfway, it tries to unmap the unmapped portion, which hits the break condition in hv_iommu_del_tree_mappings() discussed earlier. This fails to remove the node and then erroneously issues a hypercall to unmap the entire size.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
In reply to Mukesh Rdrivers/iommu/hyperv/hv-iommu-root.c line 70
Quoted 5 lines · Mukesh R
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+struct iommu_domain_geometry default_geometry = (struct iommu_domain_geometry ) { + .aperture_start =0 , + .aperture_end =-1UL , + .force_aperture = true, +};
This should not exist
drivers/iommu/hyperv/hv-iommu-root.c line 81
Quoted 6 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +6
-0+/* +* If the current thread is a VMM thread, return the partition id of the VM it +* is managing, else return HV_PARTITION_ID_INVALID. +*/ +static u64 hv_get_current_partid (void ) +{
No, you cannot transparently detect VMMs and link them like this. The VMM makes it self visible to the iommu driver via the viommu interface and you get a kvm FD to fish your partid out of. This is hackery not OK.
You should come with VMM support as a followup once you get a basic kernel-only iommu driver working.
Quoted 15 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +15
-0+static struct iommu_domain *hv_iommu_domain_alloc_paging (struct device *dev) +{ +struct hv_domain *hvdom; +int rc; +u32 unique_id; +u64 ptid =hv_get_current_partid (); + +if (ptid ==HV_PARTITION_ID_INVALID ) +return NULL ; + + hvdom = kzalloc_obj(struct hv_domain); +if (hvdom ==NULL ) +return NULL ; + +spin_lock_init (&hvdom->mappings_lock);
drivers/iommu/hyperv/hv-iommu-root.c line 213
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+ hvdom->mappings_tree =RB_ROOT_CACHED ; + + unique_id = (u32 )atomic_inc_return (&hv_unique_id); +if (unique_id ==HV_DEVICE_DOMAIN_ID_S2_NULL )/* ie, UINTMAX */ + goto out_err; + + hvdom->domid_num = unique_id; + hvdom->partid = ptid; + hvdom->iommu_dom.geometry = default_geometry; + hvdom->iommu_dom.pgsize_bitmap =HV_IOMMU_PGSIZES ;
This is the only place that needs it, and I somehow doubt -1 is the right end value since that isn't supported by most HW.
Quoted 11 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +11
-0+static int hv_iommu_attach_dev (struct iommu_domain *immdom,struct device *dev, +struct iommu_domain *old) +{ +struct pci_dev *pdev; +int rc; +struct hv_domain *hvdom_new =to_hv_domain (immdom); +struct hv_domain *hvdom_prev =to_hv_domain (old); + +/* Only allow PCI devices for now */ +if (!dev_is_pci (dev)) +return -EINVAL ;
drivers/iommu/hyperv/hv-iommu-root.c line 343
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+ + pdev =to_pci_dev (dev); + +/* There are no explicit detach calls, hence check if we need to detach +* first. Also, in case of guest shutdown, it's the VMM thread that +* attaches it back to the hv_def_identity_dom, and hvdom_prev will not +* be null then. It is null during boot. +*/ +if (hvdom_prev && !hv_special_domain (hvdom_prev)) +hv_iommu_detach_dev (hvdom_prev, dev);
What translation does this set? If it is anything other than blocking it is security broken for VFIO.
If it is blocking then why does this:
drivers/iommu/hyperv/hv-iommu-root.c line 345
Quoted 1 line
drivers/iommu/hyperv/hv-iommu-root.c new file +1
-0+ rc =hv_iommu_att_dev2dom (hvdom_new, pdev);
Attach HV_DEVICE_DOMAIN_ID_S2_NULL ?
drivers/iommu/hyperv/hv-iommu-root.c line 348
Quoted 2 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +2
-0+if (rc ==0 ) +dev_iommu_priv_set (dev, hvdom_new);/* sets "private" field */
The only thing the priv is used for is release_device ?
It would be better to have a 'detach domain' as the release_domain so you don't need this.
drivers/iommu/hyperv/hv-iommu-root.c line 593
Quoted 9 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +9
-0+static void hv_iommu_probe_finalize (struct device *dev) +{ +struct iommu_domain *immdom =iommu_get_domain_for_dev (dev); + +if (immdom && immdom->type ==IOMMU_DOMAIN_DMA ) +iommu_setup_dma_ops (dev, immdom); +else +set_dma_ops (dev,NULL ); +}
I've forgotten now, but I thought we had reached the point of getting rid of this from most drivers? amd and vtd do not implement this, why does this need it?
drivers/iommu/hyperv/hv-iommu-root.c line 601
Quoted 7 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +7
-0+static void hv_iommu_release_device (struct device *dev) +{ +struct hv_domain *hvdom =dev_iommu_priv_get (dev); + +/* Need to detach device from device domain if necessary. */ +if (hvdom) +hv_iommu_detach_dev (hvdom, dev);
What does "detach" actually do? What translation will be in effect for the device?
Ideally you should set the release_domain to blocking or identity and arrange things so that is enough to destroy the iommu attachment. But I see both blocking and identity do new attaches so IDK what this trying to do..
drivers/iommu/hyperv/hv-iommu-root.c line 619
Quoted 5 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+static int hv_iommu_def_domain_type (struct device *dev) +{ +/* The hypervisor always creates this by default during boot */ +return IOMMU_DOMAIN_IDENTITY ; +}
That isn't what this does, it overrides the policy set by Linux. Fully functional HW should not implement this function, please remove it.
Quoted 5 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+static struct iommu_ops hv_iommu_ops = { + .capable = hv_iommu_capable, + .domain_alloc_paging = hv_iommu_domain_alloc_paging, + .probe_device = hv_iommu_probe_device, + .probe_finalize = hv_iommu_probe_finalize,
drivers/iommu/hyperv/hv-iommu-root.c line 635
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+ .release_device = hv_iommu_release_device, + .def_domain_type = hv_iommu_def_domain_type, + .device_group = hv_iommu_device_group, + .default_domain_ops = &(const struct iommu_domain_ops ) { + .attach_dev = hv_iommu_attach_dev, + .map_pages = hv_iommu_map_pages, + .unmap_pages = hv_iommu_unmap_pages, + .iova_to_phys = hv_iommu_iova_to_phys, + .free = hv_iommu_domain_free, + },
Please don't use default_domain_ops, this should a new struct hv_paging_domain_ops
drivers/iommu/hyperv/hv-iommu-root.c line 638
Quoted 3 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +3
-0+ .owner = THIS_MODULE, + .identity_domain = &hv_def_identity_dom.iommu_dom, + .blocked_domain = &hv_null_dom.iommu_dom,
Can we call null dom blocked dom please?
drivers/iommu/hyperv/hv-iommu-root.c line 657
Quoted 13 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +13
-0+static void __init hv_initialize_special_domains (void ) +{ + hv_def_identity_dom.iommu_dom.type =IOMMU_DOMAIN_IDENTITY ; + hv_def_identity_dom.iommu_dom.ops = &hv_special_domain_ops; + hv_def_identity_dom.iommu_dom.owner = &hv_iommu_ops; + hv_def_identity_dom.iommu_dom.geometry = default_geometry; + hv_def_identity_dom.domid_num =HV_DEVICE_DOMAIN_ID_S2_DEFAULT ;/* 0 */ + + hv_null_dom.iommu_dom.type =IOMMU_DOMAIN_BLOCKED ; + hv_null_dom.iommu_dom.ops = &hv_special_domain_ops; + hv_null_dom.iommu_dom.owner = &hv_iommu_ops; + hv_null_dom.iommu_dom.geometry = default_geometry; + hv_null_dom.domid_num =HV_DEVICE_DOMAIN_ID_S2_NULL ;/* INTMAX */
These ones don't use geometry. Didn't I say this once before?
Jason
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Hi Jason,
Quoted 13 lines · On Wed, 5 Aug 2026 09:48:38 -0300 Jason Gunthorpe <[email protected]> wrote:
Review
Quoted 7 lines
+/*
+ * If the current thread is a VMM thread, return the partition id of the VM it
+ * is managing, else returnHV_PARTITION_ID_INVALID.
+ */(quoted diff) +2
-0+static u64 hv_get_current_partid(void) +{No, you cannot transparently detect VMMs and link them like this. The VMM makes it self visible to the iommu driver via the viommu interface and you get a kvm FD to fish your partid out of. This is hackery not OK.
Quoted 2 lines
You should come with VMM support as a followup once you get a basic kernel-only iommu driver working.
Agreed with the plan, we should drop the current-thread based VMM detection from this series and keep this driver focused on the kernel-only IOMMU path, use part_id self for the time being.
I have been working on a prototype that is based on viommu/vdevice/hwpt for "direct attach" model, aka "hypercalls to move the device into the selected guest" in the discussions of the previous version of this patch.
It is based on our previous discussions with Xen as well.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/
The viommu object for direct attach should also cover the part_id retrieval from the partition created by mshv or xen.
The rough flow looks like:
- vmm creates an iommufd ctx
- open cdev FD for the assigned device
- bind vfio cdev FD with iommufd
- vmm calls /dev/mshv or xen to create a partition, stores hv allocated part_id in its vm FD's file private data
- vmm creates a per vm viommu ojbect of direct attach unique type
- in this hv-iommu-root driver's .viommu_init, fish out the part_id from the vm file's private data then stores in hv-iommu's viommu struct.
The rest is for direct domain attach, roughly
- allocate a vdevice under the direct type viommu
- allocate a direct type hwpt (new)
- attach the direct hwpt to the cdev, where this driver can intercept in its direct domain callback (instead of abusing the paging domain attach).
With this flow, the VM identity is explicit through the vIOMMU UAPI rather than inferred from current tgid.
One open naming question is whether "direct" is the right term for this object type since it hints 1:1 mapping which it is not. It really is an externally managed S2 domain/iopt, so "external" may describe the semantics better, as Teddy called it in [1].
I plan to send out RFC in the next few weeks before LPC. Any feedback highly appreciated.
+Teddy
Thanks,
Jacob
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Quoted 17 lines · Jacob Pan
The rough flow looks like:
- vmm creates an iommufd ctx
- open cdev FD for the assigned device
- bind vfio cdev FD with iommufd
- vmm calls/dev/mshvor xen to create a partition, stores hv allocatedpart_idin its vm FD's file private data
- vmm creates a per vm viommu ojbect of direct attach unique type
- in this hv-iommu-root driver's .viommu_init, fish out thepart_idfrom the vm file's private data then stores in hv-iommu's viommu struct.
The rest is for direct domain attach, roughly
- allocate a vdevice under the direct type viommu
- allocate a direct type hwpt (new)
- attach the direct hwpt to the cdev, where this driver can intercept in its direct domain callback (instead of abusing the paging domain attach).
Quoted 2 lines
With this flow, the VM identity is explicit through the vIOMMU UAPI rather than inferred from current tgid.
Yes, this seems broadly OK
Quoted 4 lines
One open naming question is whether "direct" is the right term for this object type since it hints 1:1 mapping which it is not. It really is an externally managed S2 domain/iopt, so "external" may describe the semantics better, as Teddy called it in [1].
I also think direct is not a good name. External is not bad with something like 'external hypervisor controlled translation' as perhaps the whole name?
Jason
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
In reply to Jason GunthorpeQuoted 8 lines · Jason Gunthorpe
drivers/iommu/hyperv/hv-iommu-root.c line 70
Quoted 5 lines · Mukesh R
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+struct iommu_domain_geometry default_geometry = (struct iommu_domain_geometry ) { + .aperture_start =0 , + .aperture_end =-1UL , + .force_aperture = true, +};This should not exist
Yup, we are working with hyp team on getting the hypercall to query iommu capabilities for S2 added for root/dom0.
Quoted 11 lines
drivers/iommu/hyperv/hv-iommu-root.c line 81
Quoted 6 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +6
-0+/* +* If the current thread is a VMM thread, return the partition id of the VM it +* is managing, else return HV_PARTITION_ID_INVALID. +*/ +static u64 hv_get_current_partid (void ) +{No, you cannot transparently detect VMMs and link them like this. The VMM makes it self visible to the iommu driver via the viommu interface and you get a kvm FD to fish your partid out of. This is hackery not OK.
Quoted 2 lines
You should come with VMM support as a followup once you get a basic kernel-only iommu driver working.
Ok, will do that. FWIW, this was originally done internally in the 5.x kernel by folks not here anymore, and has been getting ported forward.
Will work with Jacob to exploit the viommu object when his patch is done.
Quoted 45 lines
Quoted 15 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +15
-0+static struct iommu_domain *hv_iommu_domain_alloc_paging (struct device *dev) +{ +struct hv_domain *hvdom; +int rc; +u32 unique_id; +u64 ptid =hv_get_current_partid (); + +if (ptid ==HV_PARTITION_ID_INVALID ) +return NULL ; + + hvdom = kzalloc_obj(struct hv_domain); +if (hvdom ==NULL ) +return NULL ; + +spin_lock_init (&hvdom->mappings_lock);drivers/iommu/hyperv/hv-iommu-root.c line 213
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+ hvdom->mappings_tree =RB_ROOT_CACHED ; + + unique_id = (u32 )atomic_inc_return (&hv_unique_id); +if (unique_id ==HV_DEVICE_DOMAIN_ID_S2_NULL )/* ie, UINTMAX */ + goto out_err; + + hvdom->domid_num = unique_id; + hvdom->partid = ptid; + hvdom->iommu_dom.geometry = default_geometry; + hvdom->iommu_dom.pgsize_bitmap =HV_IOMMU_PGSIZES ;This is the only place that needs it, and I somehow doubt -1 is the right end value since that isn't supported by most HW.
Quoted 16 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +16
-0+static int hv_iommu_attach_dev (struct iommu_domain *immdom,struct device *dev, +struct iommu_domain *old) +{ +struct pci_dev *pdev; +int rc; +struct hv_domain *hvdom_new =to_hv_domain (immdom); +struct hv_domain *hvdom_prev =to_hv_domain (old); + +/* Only allow PCI devices for now */ +if (!dev_is_pci (dev)) +return -EINVAL ; + + pdev =to_pci_dev (dev); + +/* There are no explicit detach calls, hence check if we need to detach +* first. Also, in case of guest shutdown, it's the VMM thread that
Quoted 8 lines
drivers/iommu/hyperv/hv-iommu-root.c line 343
Quoted 5 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+* attaches it back to the hv_def_identity_dom, and hvdom_prev will not +* be null then. It is null during boot. +*/ +if (hvdom_prev && !hv_special_domain (hvdom_prev)) +hv_iommu_detach_dev (hvdom_prev, dev);What translation does this set? If it is anything other than blocking it is security broken for VFIO.
It uses whatever default was during boot. In case of baremetal dom0/root, looks like it would be identity domain. So, not blocked.
Quoted 6 lines
If it is blocking then why does this:
drivers/iommu/hyperv/hv-iommu-root.c line 345
Quoted 1 line
drivers/iommu/hyperv/hv-iommu-root.c new file +1
-0+ rc =hv_iommu_att_dev2dom (hvdom_new, pdev);Attach
HV_DEVICE_DOMAIN_ID_S2_NULL?
Quoted 7 lines
drivers/iommu/hyperv/hv-iommu-root.c line 348
Quoted 2 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +2
-0+if (rc ==0 ) +dev_iommu_priv_set (dev, hvdom_new);/* sets "private" field */ The only thing the priv is used for is
release_device?It would be better to have a 'detach domain' as the
release_domainso you don't need this.
Ok, sounds good.
Quoted 5 lines
Quoted 5 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+static void hv_iommu_probe_finalize (struct device *dev) +{ +struct iommu_domain *immdom =iommu_get_domain_for_dev (dev); + +if (immdom && immdom->type ==IOMMU_DOMAIN_DMA )
Quoted 8 lines
drivers/iommu/hyperv/hv-iommu-root.c line 593
Quoted 4 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +4
-0+iommu_setup_dma_ops (dev, immdom); + else +set_dma_ops (dev ,NULL ); +}I've forgotten now, but I thought we had reached the point of getting rid of this from most drivers? amd and vtd do not implement this, why does this need it?
Yeah, this is a much older driver. Will address it.
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c line 601
Quoted 7 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +7
-0+static void hv_iommu_release_device (struct device *dev) +{ +struct hv_domain *hvdom =dev_iommu_priv_get (dev); + +/* Need to detach device from device domain if necessary. */ +if (hvdom) +hv_iommu_detach_dev (hvdom, dev);What does "detach" actually do? What translation will be in effect for the device?
It puts the device in the default boot domain. In baremetal-dom0 case it would be identity. This is required when device moves from a device domain to the direct attachment, hyp requires it first be detached. But, this will go way in the kernel only iommu driver support, we can then do the add ons for vmm support and direct attaches.
Quoted 5 lines
Ideally you should set the
release_domainto blocking or identity and arrange things so that is enough to destroy the iommu attachment. But I see both blocking and identity do new attaches so IDK what this trying to do..
Quoted 8 lines
drivers/iommu/hyperv/hv-iommu-root.c line 619
Quoted 5 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+static int hv_iommu_def_domain_type (struct device *dev) +{ +/* The hypervisor always creates this by default during boot */ +return IOMMU_DOMAIN_IDENTITY ; +}That isn't what this does, it overrides the policy set by Linux. Fully functional HW should not implement this function, please remove it.
Ok, will do.
Quoted 10 lines
Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +10
-0+static struct iommu_ops hv_iommu_ops = { + .capable = hv_iommu_capable, + .domain_alloc_paging = hv_iommu_domain_alloc_paging, + .probe_device = hv_iommu_probe_device, + .probe_finalize = hv_iommu_probe_finalize, + .release_device = hv_iommu_release_device, + .def_domain_type = hv_iommu_def_domain_type, + .device_group = hv_iommu_device_group, + .default_domain_ops = &(const struct iommu_domain_ops ) { + .attach_dev = hv_iommu_attach_dev,
Quoted 8 lines
drivers/iommu/hyperv/hv-iommu-root.c line 635
Quoted 5 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +5
-0+ .map_pages = hv_iommu_map_pages, + .unmap_pages = hv_iommu_unmap_pages, + .iova_to_phys = hv_iommu_iova_to_phys, + .free = hv_iommu_domain_free, + },Please don't use
default_domain_ops, this should a new structhv_paging_domain_ops
Ok, this is an old driver from pre 5.x days, so lacks that. Jacob and I looked at it, and it appeared most drivers except intel were still using default ops, so we thought that was acceptable. But I will change it in the next version.
Quoted 5 lines
drivers/iommu/hyperv/hv-iommu-root.c line 638
Quoted 3 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +3
-0+ .owner = THIS_MODULE, + .identity_domain = &hv_def_identity_dom.iommu_dom, + .blocked_domain = &hv_null_dom.iommu_dom,Can we call null dom blocked dom please?
Sure.
Quoted 16 lines
drivers/iommu/hyperv/hv-iommu-root.c line 657
Quoted 13 lines
drivers/iommu/hyperv/hv-iommu-root.c new file +13
-0+static void __init hv_initialize_special_domains (void ) +{ + hv_def_identity_dom.iommu_dom.type =IOMMU_DOMAIN_IDENTITY ; + hv_def_identity_dom.iommu_dom.ops = &hv_special_domain_ops; + hv_def_identity_dom.iommu_dom.owner = &hv_iommu_ops; + hv_def_identity_dom.iommu_dom.geometry = default_geometry; + hv_def_identity_dom.domid_num =HV_DEVICE_DOMAIN_ID_S2_DEFAULT ;/* 0 */ + + hv_null_dom.iommu_dom.type =IOMMU_DOMAIN_BLOCKED ; + hv_null_dom.iommu_dom.ops = &hv_special_domain_ops; + hv_null_dom.iommu_dom.owner = &hv_iommu_ops; + hv_null_dom.iommu_dom.geometry = default_geometry; + hv_null_dom.domid_num =HV_DEVICE_DOMAIN_ID_S2_NULL ;/* INTMAX */ These ones don't use geometry. Didn't I say this once before?
Quoted 1 line
Jason
Thanks for the review. I know there is another set of patches for pvIOMMU ongoing (both came from the same source) and you are probably repeating things, and I appreciate your patience. I'll also try to look for your comments in that patch series going forward.
Thanks,
-Mukesh
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Quoted 4 lines · Mukesh R
Thanks for the review. I know there is another set of patches for pvIOMMU ongoing (both came from the same source) and you are probably repeating things, and I appreciate your patience. I'll also try to look for your comments in that patch series going forward.
Those patches look pretty good so you should try to copy their stuff :)
They ended up never calling detach, I hope you can do that too. Just always have a blocking or identity domain and just use it everywhere.
Since you can't predict what translation detach gives it should not be set on a device. release could be the only exception and only if you reall have to.
Jason
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Quoted 11 lines · Jason Gunthorpe
Quoted 4 lines · Mukesh R
Thanks for the review. I know there is another set of patches for pvIOMMU ongoing (both came from the same source) and you are probably repeating things, and I appreciate your patience. I'll also try to look for your comments in that patch series going forward.
Those patches look pretty good so you should try to copy their stuff :)
They ended up never calling detach, I hope you can do that too. Just
Unfortunately, my case is a bit different due to the "direct attach" feature for guest VMs. When a device moves from say a paging domain to be directly attached to a VM, today, it must first be detached. But since that just moves to identity, I am discussing with hyp team if detach is really required, and if we can get rid of it via just attaching to identity/blocked first before doing the direct attach. Hopefully, it can be done even if changes are needed in the hypervisor.
Thanks again,
-Mukesh
Quoted 7 lines
always have a blocking or identity domain and just use it everywhere.
Since you can't predict what translation detach gives it should not be set on a device. release could be the only exception and only if you reall have to.
Jason
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Quoted 13 lines · Mukesh R
Quoted 11 lines · Jason Gunthorpe
Quoted 4 lines · Mukesh R
Thanks for the review. I know there is another set of patches for pvIOMMU ongoing (both came from the same source) and you are probably repeating things, and I appreciate your patience. I'll also try to look for your comments in that patch series going forward.
Those patches look pretty good so you should try to copy their stuff :)
They ended up never calling detach, I hope you can do that too. Just
Quoted 7 lines
Unfortunately, my case is a bit different due to the "direct attach" feature for guest VMs. When a device moves from say a paging domain to be directly attached to a VM, today, it must first be detached. But since that just moves to identity, I am discussing with hyp team if detach is really required, and if we can get rid of it via just attaching to identity/blocked first before doing the direct attach. Hopefully, it can be done even if changes are needed in the hypervisor.
You definately cannot have it blip to identity, that is security unacceptable for VFIO.
Once it is attached to VFIO it must only be blocked or under a translation controlled by VFIO. Never something else.
Jason
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Hi Jason,
Quoted 25 lines · On Wed, 19 Aug 2026 09:46:52 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 15 lines · Mukesh R
Quoted 8 lines · Jason Gunthorpe
Quoted 5 lines · Mukesh R
Thanks for the review. I know there is another set of patches for pvIOMMU ongoing (both came from the same source) and you are probably repeating things, and I appreciate your patience. I'll also try to look for your comments in that patch series going forward.
Quoted 5 lines
Those patches look pretty good so you should try to copy their stuff :)
They ended up never calling detach, I hope you can do that too. Just
Quoted 8 lines
Unfortunately, my case is a bit different due to the "direct attach" feature for guest VMs. When a device moves from say a paging domain to be directly attached to a VM, today, it must first be detached. But since that just moves to identity, I am discussing with hyp team if detach is really required, and if we can get rid of it via just attaching to identity/blocked first before doing the direct attach. Hopefully, it can be done even if changes are needed in the hypervisor.
Quoted 5 lines
You definately cannot have it blip to identity, that is security unacceptable for VFIO.
Once it is attached to VFIO it must only be blocked or under a translation controlled by VFIO. Never something else.
Based on this requirement, I don't see there is a difference between this root driver and the guest driver in terms of VFIO DMA ownership transition in the layer above the immediate hypercall site. i.e.
old domain -> blocking domain -> new target domain
Regardless of source or target domain type (e.g. direct attach as the target domain), the blocking domain must be attached.
If the hypervisor detach operation internally implies transition through identity, then it needs to be hidden behind the blocking-domain attach transaction.
Re: [PATCH v5 7/9] x86/hyperv: Implement Hyper-V virtual IOMMU
Quoted 39 lines · Jacob Pan
Hi Jason,
Quoted 25 lines · On Wed, 19 Aug 2026 09:46:52 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 15 lines · Mukesh R
Quoted 8 lines · Jason Gunthorpe
Quoted 5 lines · Mukesh R
Thanks for the review. I know there is another set of patches for pvIOMMU ongoing (both came from the same source) and you are probably repeating things, and I appreciate your patience. I'll also try to look for your comments in that patch series going forward.
Quoted 5 lines
Those patches look pretty good so you should try to copy their stuff :)
They ended up never calling detach, I hope you can do that too. Just
Quoted 8 lines
Unfortunately, my case is a bit different due to the "direct attach" feature for guest VMs. When a device moves from say a paging domain to be directly attached to a VM, today, it must first be detached. But since that just moves to identity, I am discussing with hyp team if detach is really required, and if we can get rid of it via just attaching to identity/blocked first before doing the direct attach. Hopefully, it can be done even if changes are needed in the hypervisor.
Quoted 5 lines
You definately cannot have it blip to identity, that is security unacceptable for VFIO.
Once it is attached to VFIO it must only be blocked or under a translation controlled by VFIO. Never something else.
Based on this requirement, I don't see there is a difference between this root driver and the guest driver in terms of VFIO DMA ownership transition in the layer above the immediate hypercall site. i.e.
Quoted 8 lines
old domain -> blocking domain -> new target domain
Regardless of source or target domain type (e.g. direct attach as the target domain), the blocking domain must be attached.
If the hypervisor detach operation internally implies transition through identity, then it needs to be hidden behind the blocking-domain attach transaction.
The best thing is to do as the other driver did and just always and only do attach. Hypervisor just changes the attachment instead of demanding a pointless detach.
Jason
PATCH v5 8/9 mshv: Populate mmio mappings for PCI passthru
In reply to Mukesh RUpon guest accesses, in case of missing mmio mappings, the hypervisor generates unmapped gpa intercepts. In this path, lookup the PCI resource pfn for the guest gpa, and ask the hypervisor to map it via hypercall. The PCI resource pfn is maintained by the VFIO driver, and obtained via fixup_user_fault() call (similar to KVM). Try to use 2M page size as much as possible for significant performance gains.
Also, remove existing code that is using vma->vm_pgoff to map mmio space, it is broken and will cause panics.
--- drivers/hv/mshv_root.h | 3 +- drivers/hv/mshv_root_hv_call.c | 104 +++++++++++++++++++++------- drivers/hv/mshv_root_main.c | 120 ++++++++++++++++++++++++++++----- 3 files changed, 185 insertions(+), 42 deletions(-)
drivers/hv/mshv_root.h +2 -1
@@ -301,7 +301,8 @@ int hv_call_create_partition(u64 flags,int hv_call_initialize_partition (u64 partition_id);int hv_call_finalize_partition (u64 partition_id);int hv_call_delete_partition (u64 partition_id); -int hv_call_map_mmio_pages (u64 partition_id,u64 gfn,u64 mmio_spa,u64 numpgs); +int hv_map_mmio_pages (u64 partition_id,struct mshv_mem_region *reg, +u64 mmio_mfn);int hv_call_map_gpa_pages (u64 partition_id,u64 gpa_target,u64 page_count,u32 flags,struct page **pages); int hv_call_unmap_gpa_pages(u64 partition_id, u64 gpa_target, u64 page_count,
drivers/hv/mshv_root_hv_call.c +80 -24
149 lines not on this page · raw message
drivers/hv/mshv_root_main.c +103 -17
161 lines not on this page · raw message
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 8/9] mshv: Populate mmio mappings for PCI passthru
Thank you for your contribution! Sashiko AI review found 8 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 8 potential issue(s) to consider:
New issues:
- [Critical] Insecure MMIO region bounds check and PFN extrapolation allows mapping arbitrary host RAM into the guest.
- [Critical] Conflation of Host PFNs with Hyper-V 4KB PFNs causes arbitrary memory mapping on architectures with >4KB pages (ARM64).
- [Critical] Missing MMU interval notifier allows stale physical mappings (TOCTOU and Use-After-Unmap).
- [High] Use-After-Free of VMA pointer after dropping mmap_read_lock.
- [High] Alignment check logic error causes hypercall protocol violation for large pages.
- [High] Overloading mmio_mfn with 0 bypasses page_is_ram() checks for PFN 0.
Pre-existing issues:
- [High] Incorrect unit passed to unmap function on error path causes silent resource leak.
- [High] Unbounded O(N) page_is_ram() checks in process context cause CPU soft lockups. --
--- Patch [8]: [PATCH v5 8/9] mshv: Populate mmio mappings for PCI passthru --- commit ef68862e98379b2ba894e7946e9748f29e46d61f
Author: Mukesh R <[email protected]>
Subject: mshv: Populate mmio mappings for PCI passthru
This commit handles hypervisor unmapped GPA intercepts by resolving the PCI resource PFN and mapping it via hypercall. It attempts to map 2M pages where possible to improve performance and removes broken VMA offset code.
Quoted 4 lines
drivers/hv/mshv_root_hv_call.c +0
-0
[ ... ]
Quoted 5 lines
drivers/hv/mshv_root_hv_call.c +2
-2-static int hv_do_map_gpa_hcall (u64 partition_id,u64 gfn,u64 page_struct_count, -u32 flags,struct page **pages,u64 mmio_spa) +static int hv_do_map_gpa_hcall (u64 partition_id,u64 gfn,u64 page_count, +u32 flags,struct page **pages,u64 mmio_mfn) {
[ ... ]
drivers/hv/mshv_root_hv_call.c int hv_call_delete_partition(u64 partition_id) · line 207
Quoted 9 lines
drivers/hv/mshv_root_hv_call.c +6
-2-if (page_count ==0 || (pages && mmio_spa)) -return -EINVAL ; +if (mmio_mfn) { +if (pages)return -EINVAL ; +for (i =0 ; i < page_count; i++) +if (page_is_ram (mmio_mfn + i)) +return -EINVAL ; + }
[Severity: High]
Does this check evaluate to false when a legitimate MMIO region points to physical PFN 0?
If mmio_mfn is 0, it appears to bypass the page_is_ram() loop entirely. Could this allow a VMM process to map the host's physical RAM page 0 into the guest as MMIO?
[Severity: High]
This is a pre-existing issue, but does this loop iterate over a potentially massive number of pages without yielding?
Since page_count is derived from the user-provided region size, a maliciously large region might force the host CPU into a soft lockup. Should we add a cond_resched() here?
[ ... ]
drivers/hv/mshv_root_hv_call.c static int hv_do_map_gpa_hcall(u64 partition_id, u64 gfn, u64 page_struct_count, · line 265
Quoted 7 lines
drivers/hv/mshv_root_hv_call.c +3
-0}else if (!hv_result_success (status)) { +pr_err ("%s: failed to map pages at gfn %#llx: completed %u/%llu, flags=%#x, status=%#llx (%s)\n" , + __func__, gfn, done, page_count, flags, status, +hv_result_to_string (hv_result (status))); ret =hv_result_to_errno (status);break ; }
[Severity: High]
This is a pre-existing issue, but does the error rollback path near the end of this function pass the wrong units to hv_call_unmap_gpa_pages()?
The variable done tracks successfully mapped large pages, but hv_call_unmap_gpa_pages() expects its page_count argument in 4KB units. If we pass the large page count instead, it seems like it will fail the alignment check and silently leak dangling large page mappings in the hypervisor's stage 2 tables.
[ ... ]
Quoted 3 lines
drivers/hv/mshv_root_hv_call.c +2
-0+int hv_map_mmio_pages (u64 partition_id,struct mshv_mem_region *reg, +u64 mmio_mfn) {
[ ... ]
drivers/hv/mshv_root_hv_call.c int hv_call_map_gpa_pages(u64 partition_id, u64 gpa_target, u64 page_count, · line 319
Quoted 7 lines
drivers/hv/mshv_root_hv_call.c +7
-0+while (!HV_PAGE_COUNT_2M_ALIGNED (gfn) && + !HV_PAGE_COUNT_2M_ALIGNED (mmio_mfn) && + numpgs < reg->nr_pages) { + numpgs++; + gfn++; + mmio_mfn++; + }
[Severity: High]
Does the logical AND operator here cause the alignment loop to exit prematurely?
If the offsets of gfn and mmio_mfn relative to a 2MB boundary differ, the loop will break as soon as only one of them becomes aligned, leaving the other unaligned.
Will this cause us to pass unaligned addresses to a large-page hypercall later in this function, violating the Hyper-V protocol?
Quoted 4 lines
drivers/hv/mshv_root_main.c +0
-0
[ ... ]
Quoted 2 lines
drivers/hv/mshv_root_main.c +2
-0+static bool mshv_handle_unmapped_gpa (struct mshv_vp *vp) +{
[ ... ]
drivers/hv/mshv_root_main.c mshv_partition_region_by_gfn_get(struct mshv_partition *p, u64 gfn) · line 720
Quoted 1 line
drivers/hv/mshv_root_main.c +1
-0+ mmio_mfn = mmio_mfn - (gfn - rg->start_gfn);/* start of the range */
[Severity: Critical]
Could this backwards extrapolation yield arbitrary physical addresses if a region spans outside its intended VMA? By extrapolating the base PFN for the entire region backwards, it looks like it could map adjacent, unauthorized host physical memory into the guest.
Also, on architectures with page sizes larger than 4KB (like ARM64), does this mix incompatible units?
The function mshv_chk_get_mmio_start_pfn() returns a host PFN (e.g. 64KB scaled), but this subtracts a 4KB-based Hyper-V GFN offset. Passing this corrupted value to the hypervisor could cause incorrect physical addresses to be mapped.
drivers/hv/mshv_root_main.c mshv_partition_region_by_gfn_get(struct mshv_partition *p, u64 gfn) · line 727
Quoted 7 lines
drivers/hv/mshv_root_main.c +7
-0+ +/* Map the entire mmio region now. mmap lock not needed because we +* are just mapping GPA to static device addresses, and the above +* mshv_partition_region_by_gfn_get() gets a hold on the region and +* hence its GPA. Unmap will occur when region ref count goes to 0. +*/ + rc =hv_map_mmio_pages (pt->pt_id, rg, mmio_mfn);
[Severity: Critical]
Are these physical memory mappings protected against stale access?
Because MMIO regions do not register an mmu_interval_notifier, there doesn't appear to be a mechanism to invalidate the hypervisor's translation tables if the device is unbound or the VMA is unmapped.
Furthermore, dropping the mmap lock earlier seems to introduce a TOCTOU race where a concurrent munmap could invalidate the PFN before the hypercall is made. Will this allow stale mappings to persist?
Quoted 1 line
drivers/hv/mshv_root_main.c +0
-0@@ -1340,7 +1424,6 @@ mshv_map_user_memory(struct mshv_partition *partition,
[ ... ]
drivers/hv/mshv_root_main.c mshv_map_user_memory(struct mshv_partition *partition, · line 1442
Quoted 11 lines
drivers/hv/mshv_root_main.c +2
-1mmap_read_lock (current->mm); vma =vma_lookup (current->mm, mem->userspace_addr); is_mmio = vma ? !!(vma->vm_flags & (VM_IO |VM_PFNMAP )) :0 ; - mmio_pfn = is_mmio ? vma->vm_pgoff :0 ;mmap_read_unlock (current->mm);if (!vma)return -EINVAL ; +if (is_mmio) { +size_t vma_sz = vma->vm_end - vma->vm_start;
[Severity: High]
Can this result in a use-after-free?
The vma pointer is unconditionally dereferenced here, but the lock is dropped just above it with mmap_read_unlock(current->mm).
Could a concurrent thread in the VMM process calling munmap() unlink and free the VMA before this calculation occurs, leading to memory corruption?
drivers/hv/mshv_root_main.c mshv_map_user_memory(struct mshv_partition *partition, · line 1447
Quoted 5 lines
drivers/hv/mshv_root_main.c +5
-0+ +/* Upon mmio intercept, entire region is mapped */ +if (mem->size > vma_sz) +return -EPERM ; + }
[Severity: Critical]
Does this bounds check properly validate the starting offset?
It only compares the total size against vma_sz, neglecting to check if the region's start offset fits within the VMA. For example, it doesn't verify that userspace_addr + size <= vm_end.
Would this allow a malicious userspace VMM to create a region that spans outside its intended VMA?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
PATCH v5 9/9 mshv: Disable movable regions upfront if device passthru
In reply to Mukesh RIf a VM is started with device attached, the mem regions must be marked non-movable as the device attach hypercall right away allows the use of SLAT for IOMMU. Marking them non-movable forces mapping of the entire guest RAM in the SLAT at the time of region creation along with the region pinned. Also, because a device could be dynamically attached much later in a VM, create a module parameter to allow a user to pin them upfront if they anticipate such an action. It is understood that VFIO pins the regions also, so there will be double pinning. This will be enhanced in the near future with a third region type that is not movable nor pinned.
--- drivers/hv/mshv_root.h | 1 + drivers/hv/mshv_root_main.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
drivers/hv/mshv_root.h +1 -0
@@ -141,6 +141,7 @@ struct mshv_partition {pid_t pt_vmm_tgid;bool import_completed;bool pt_initialized; +bool pt_regions_pinned;#if IS_ENABLED (CONFIG_DEBUG_FS )struct dentry *pt_stats_dentry;struct dentry *pt_vp_dentry;
drivers/hv/mshv_root_main.c +14 -1
@@ -46,6 +46,10 @@ MODULE_DESCRIPTION("Microsoft Hyper-V root partition VMM interface /dev/mshv");#define HV_VP_COUNTER_ROOT_DISPATCH_THREAD_BLOCKED 95#endif +static uint hv_memreg;/* mshv_mem_region allocations type */ +module_param (hv_memreg, uint,0644 ); +MODULE_PARM_DESC (hv_memreg,"0: movable (default), 1: pinned" ); +struct mshv_root mshv_root;enum hv_scheduler_type hv_scheduler_type;
@@ -1302,6 +1306,12 @@ static void mshv_async_hvcall_handler(void *data, u64 *status) *status = partition->async_hypercall_status; } +static bool mshv_do_pt_regions_pinned (struct mshv_partition *pt) +{ +return pt->pt_regions_pinned ||mshv_partition_encrypted (pt) || + hv_memreg ==1 ; +} +/* * NB: caller checks and makes sure mem->size is page aligned * Returns: 0 with regionpp updated on success, or -errno
@@ -1332,7 +1342,7 @@ static int mshv_partition_create_region(struct mshv_partition *partition,if (is_mmio) rg->mreg_type =MSHV_REGION_TYPE_MMIO ; -else if (mshv_partition_encrypted (partition) || +else if (mshv_do_pt_regions_pinned (partition) || !mshv_region_movable_init (rg)) rg->mreg_type =MSHV_REGION_TYPE_MEM_PINNED ;else
@@ -1818,6 +1828,9 @@ static long mshv_partition_ioctl_create_device(struct mshv_partition *partition,if (copy_to_user (uarg, &devargk,sizeof (devargk)))return -EFAULT ;/* cleanup in mshv_device_fop_release() */ +/* For now, just mark regions pinned if there is device passthru. */ + partition->pt_regions_pinned = true; +return 0 ; undo_out:
Signature · 1 line
2.51.2.vfs.0.1
[email protected]
AI reviewRe: [PATCH v5 9/9] mshv: Disable movable regions upfront if device passthru
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] Bypass of memory pinning enforcement on device creation via copy_to_user failure.
Pre-existing issues:
- [High] Error paths in mshv_map_user_memory and mshv_prepare_pinned_region cause a NULL pointer dereference in unpin_user_pages(). --
--- Patch [9]: [PATCH v5 9/9] mshv: Disable movable regions upfront if device passthru --- commit 43489ca84b127042602836c2214d379f535126bf
Author: Mukesh R <[email protected]>
mshv: Disable movable regions upfront if device passthru
If a VM is started with a device attached, the memory regions must be marked non-movable to allow the use of SLAT for IOMMU. This commit introduces a module parameter to pin regions upfront and updates the device creation ioctl to mark regions as pinned when a device is passed through.
Quoted 3 lines
drivers/hv/mshv_root_main.c +0
-0
[ ... ]
drivers/hv/mshv_root_main.c static int mshv_partition_create_region(struct mshv_partition *partition, · line 1347
Quoted 8 lines
drivers/hv/mshv_root_main.c +1
-1@@ -1332,7 +1342,7 @@ static int mshv_partition_create_region(struct mshv_partition *partition,if (is_mmio) rg->mreg_type =MSHV_REGION_TYPE_MMIO ; -else if (mshv_partition_encrypted (partition) || +else if (mshv_do_pt_regions_pinned (partition) || !mshv_region_movable_init (rg)) rg->mreg_type =MSHV_REGION_TYPE_MEM_PINNED ;
[Severity: High]
This is a pre-existing issue, but I noticed an error handling problem with pinned regions that might be triggered more frequently now.
If pinning fails in mshv_map_user_memory(), we drop the reference:
mshv_map_user_memory() {
...
errout:
mshv_region_put(region);
return ret;
}This triggers mshv_region_destroy() which unconditionally invalidates the region:
mshv_region_destroy() {
...
mshv_region_invalidate(region);
vfree(region);
}If the region pages were already partially unpinned and NULLed out during the pinning failure, mshv_region_invalidate_pages() will pass those NULL pointers directly to unpin_user_pages():
mshv_region_invalidate_pages() {
...
if (region->mreg_type == MSHV_REGION_TYPE_MEM_PINNED)
unpin_user_pages(region->mreg_pages + page_offset, page_count);
}Since unpin_user_pages() does not check for NULL pointers, does this sequence result in a NULL pointer dereference when cleaning up after a pinning failure?
[ ... ]
drivers/hv/mshv_root_main.c static long mshv_partition_ioctl_create_device(struct mshv_partition *partition, · line 1834
Quoted 8 lines
drivers/hv/mshv_root_main.c +3
-0@@ -1818,6 +1828,9 @@ static long mshv_partition_ioctl_create_device(struct mshv_partition *partition,if (copy_to_user (uarg, &devargk,sizeof (devargk)))return -EFAULT ;/* cleanup in mshv_device_fop_release() */ +/* For now, just mark regions pinned if there is device passthru. */ + partition->pt_regions_pinned = true; +return 0 ;
[Severity: High]
If copy_to_user() fails here, we return -EFAULT, but the file descriptor has already been installed into the process's file descriptor table via anon_inode_getfd() and remains active.
Could this allow userspace to bypass setting pt_regions_pinned to true while successfully obtaining and using the device file descriptor? If a process intentionally passes an invalid pointer, it seems it might be able to map movable memory while still attaching a device.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9