MSHV: add PV-IOMMU driver
Subsystems (8)
Wei Liu
maintainercover MSHV: add PV-IOMMU driver
Hi all
Device passthrough is a critial feature for a virtualization stack. When designing this feature for MSHV support on Linux, one important considration is to not deviate from Linux's default VFIO stack. VFIO relies on an IOMMU or IOMMUs in the system to manipulate DMA mappings.
In this series an IOMMU driver is implemented using a set of hypercall interfaces provided by the Microsoft Hypervisor. At this stage only DMA remapping is implemented. Interrupt remapping will come later.
With this series I'm able to passthrough an NVMe drive to a guest with VFIO on a modified version of Cloud Hypervisor. From users' point of view, nothing needs changing. Cloud Hypervisor and Rust-VMM changes, which depend on the new kernel UAPIs from this series, will be upstreamed too.
This series is built on top of Nuno and Vineeth's patches [0][1].
The meat is in the patch named "mshv: add paravirtualized IOMMU support".
The in-kernel device framework and the VFIO bridge device are heavily inspired by KVM's code. I pondered whether it would be worth refactoring the code in KVM but decided against that route for two reasons: 1. it allowed faster prototyping and 2. I was not sure if that's something KVM community would agree to.
For the VT-D changes, what we're after is to build the RMRR regions list so that reserved regions are respected. Instead of doing a bad job myself, I decided to piggy-back on Intel's own code. AMD support is to be added until we have an AMD system.
Comments are welcome.
Thanks,
Wei.
[0] https://lore.kernel.org/linux-hyperv/1622241819-21155-1-git-send-email-nunodasneves@linux.microsoft.com/
[1] https://lore.kernel.org/linux-hyperv/[email protected]/
Wei Liu (8):
x86/hyperv: export hv_build_pci_dev_id asm-generic/hyperv: add device domain definitions intel/vt-d: make DMAR table parsing code more flexible intel/vt-d: export intel_iommu_get_resv_regions
mshv: add paravirtualized IOMMU support
mshv: command line option to skip devices in PV-IOMMU
mshv: implement in-kernel device framework
mshv: add vfio bridge device
Documentation/virt/mshv/api.rst | 12 + arch/x86/hyperv/irqdomain.c | 3 +- arch/x86/include/asm/mshyperv.h | 1 + drivers/hv/Kconfig | 4 + drivers/hv/Makefile | 2 +- drivers/hv/mshv_main.c | 186 ++++++++ drivers/hv/vfio.c | 244 ++++++++++ drivers/hv/vfio.h | 18 + drivers/iommu/Kconfig | 14 + drivers/iommu/hyperv-iommu.c | 673 ++++++++++++++++++++++++++++ drivers/iommu/intel/dmar.c | 38 +- drivers/iommu/intel/iommu.c | 7 +- drivers/iommu/intel/irq_remapping.c | 2 +- include/asm-generic/hyperv-tlfs.h | 144 ++++++ include/linux/dmar.h | 2 +- include/linux/intel-iommu.h | 4 + include/linux/mshv.h | 57 +++ include/uapi/linux/mshv.h | 36 ++ 18 files changed, 1429 insertions(+), 18 deletions(-) create mode 100644 drivers/hv/vfio.c create mode 100644 drivers/hv/vfio.h
Signature · 1 line
2.30.2
Wei Liu
maintainerRFC 1/8 x86/hyperv: export hv_build_pci_dev_id
--- arch/x86/hyperv/irqdomain.c | 3 ++- arch/x86/include/asm/mshyperv.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/hyperv/irqdomain.c +2 -1
@@ -110,7 +110,7 @@ static int get_rid_cb(struct pci_dev *pdev, u16 alias, void *data)return 0 ; } -static union hv_device_id hv_build_pci_dev_id (struct pci_dev *dev) +union hv_device_id hv_build_pci_dev_id (struct pci_dev *dev) {union hv_device_id dev_id;struct rid_data data = {
@@ -168,6 +168,7 @@ static union hv_device_id hv_build_pci_dev_id(struct pci_dev *dev)return dev_id; } +EXPORT_SYMBOL_GPL (hv_build_pci_dev_id);static int hv_map_msi_interrupt (struct pci_dev *dev,int cpu,int vector,struct hv_interrupt_entry *entry)
arch/x86/include/asm/mshyperv.h +1 -0
@@ -196,6 +196,7 @@ struct irq_domain *hv_create_pci_msi_domain(void);int hv_map_ioapic_interrupt (int ioapic_id,bool level,int vcpu,int vector,struct hv_interrupt_entry *entry);int hv_unmap_ioapic_interrupt (int ioapic_id,struct hv_interrupt_entry *entry); +union hv_device_id hv_build_pci_dev_id (struct pci_dev *dev);#else /* CONFIG_HYPERV */ static inline void hyperv_init (void ) {}
Signature · 1 line
2.30.2
Wei Liu
maintainerRFC 2/8 asm-generic/hyperv: add device domain definitions
In reply to Wei LiuThese definitions are for the IOMMU device domain interfaces exposed by Microsoft Hyperivsor. We will use them to implement DMA remapping.
--- include/asm-generic/hyperv-tlfs.h | 144 ++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+)
include/asm-generic/hyperv-tlfs.h +144 -0
@@ -93,6 +93,13 @@#define HV_ENABLE_EXTENDED_HYPERCALLS BIT(20)#define HV_ISOLATION BIT(22) +/* +* Group D features +*/ +#define HV_DEVICE_DOMAIN_AVAILABLE BIT(24) +#define HV_S1_DEVICE_DOMAIN_AVAILABLE BIT(25) + +/* * TSC page layout. */
@@ -179,6 +186,17 @@ struct ms_hyperv_tsc_page {#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0#define HVCALL_GET_GPA_PAGES_ACCESS_STATES 0x00c9 +#define HVCALL_CREATE_DEVICE_DOMAIN 0x00b1 +#define HVCALL_ATTACH_DEVICE_DOMAIN 0x00b2 +#define HVCALL_MAP_DEVICE_GPA_PAGES 0x00b3 +#define HVCALL_UNMAP_DEVICE_GPA_PAGES 0x00b4 +#define HVCALL_DETACH_DEVICE_DOMAIN 0x00c4 +#define HVCALL_DELETE_DEVICE_DOMAIN 0x00c5 +#define HVCALL_QUERY_DEVICE_DOMAIN 0x00c6 +#define HVCALL_MAP_SPARSE_DEVICE_GPA_PAGES 0x00c7 +#define HVCALL_UNMAP_SPARSE_DEVICE_GPA_PAGES 0x00c8 +#define HVCALL_CONFIGURE_DEVICE_DOMAIN 0x00ce +#define HVCALL_FLUSH_DEVICE_DOMAIN 0x00d0#define HVCALL_MAP_VP_STATE_PAGE 0x00e1#define HVCALL_GET_VP_STATE 0x00e3#define HVCALL_SET_VP_STATE 0x00e4
@@ -1069,4 +1087,130 @@ union hv_disconnect_port { u32 reserved: 31; }; } __packed; + +/* Device domain types */ +#define HV_DEVICE_DOMAIN_TYPE_S2 0/* Stage 2 domain */ +#define HV_DEVICE_DOMAIN_TYPE_S1 1/* Stage 1 domain */ +#define HV_DEVICE_DOMAIN_TYPE_SOC 2/* SOC domain */ + +/* ID for stage 2 default domain and NULL domain */ +#define HV_DEVICE_DOMAIN_ID_S2_DEFAULT 0 +#define HV_DEVICE_DOMAIN_ID_S2_NULL 0xFFFFFFFFULL + +union hv_device_domain_id { +u64 as_uint64; +struct { +u32 type:4 ; +u32 reserved:28 ; +u32 id; + }; +} __packed; + +union hv_input_vtl { +u8 as_uint8; +struct { +u8 target_vtl:4 ; +u8 use_target_vtl:1 ; +u8 reserved_z:3 ; + }; +} __packed; + +struct hv_input_device_domain { +u64 partition_id; +union hv_input_vtl owner_vtl; +u8 padding[7 ]; +union hv_device_domain_id domain_id; +} __packed; + +union hv_create_device_domain_flags { +u32 as_uint32; +struct { +u32 forward_progress_required:1 ; +u32 inherit_owning_vtl:1 ; +u32 reserved:30 ; + }; +} __packed; + +struct hv_input_create_device_domain { +struct hv_input_device_domain device_domain; +union hv_create_device_domain_flags create_device_domain_flags; +} __packed; + +struct hv_input_delete_device_domain { +struct hv_input_device_domain device_domain; +} __packed; + +struct hv_input_attach_device_domain { +struct hv_input_device_domain device_domain; +union hv_device_id device_id; +} __packed; + +struct hv_input_detach_device_domain { +u64 partition_id; +union hv_device_id device_id; +} __packed; + +struct hv_input_map_device_gpa_pages { +struct hv_input_device_domain device_domain; +union hv_input_vtl target_vtl; +u8 padding[3 ]; +u32 map_flags; +u64 target_device_va_base; +u64 gpa_page_list[]; +} __packed; + +struct hv_input_unmap_device_gpa_pages { +struct hv_input_device_domain device_domain; +u64 target_device_va_base; +} __packed; + +struct hv_input_query_device_domain { +struct hv_input_device_domain device_domain; +u64 target_device_va_list[]; +} __packed; + +struct hv_device_va_mapping { +u64 target_device_va; +u64 gpa_page_number; +} __packed; + +struct hv_input_map_sparse_device_gpa_pages { +struct hv_input_device_domain device_domain; +union hv_input_vtl target_vtl; +u32 map_flags; +struct hv_device_va_mapping page_list[]; +} __packed; + +struct hv_input_unmap_sparse_device_gpa_pages { +struct hv_input_device_domain device_domain; +u64 target_device_va_list[]; +} __packed; + +struct hv_device_domain_settings_x64 { +struct { +/* +* Enable translations. If not enabled, all transaction bypass +* S1 translations. +*/ +u64 translation_enabled:1 ; +u64 reserved:63 ; + } flags; +/* Page attribute table */ +u64 pat; +/* Address of translation table */ +u64 page_table_root; +} __packed; + +struct hv_device_domain_settings { +union { +struct hv_device_domain_settings_x64 x64; +/* ARM TBD */ + }; +} __packed; + +struct hv_input_configure_device_domain { +struct hv_input_device_domain device_domain; +struct hv_device_domain_settings settings; +} __packed; +#endif
Signature · 1 line
2.30.2
Wei Liu
maintainerRFC 3/8 intel/vt-d: make DMAR table parsing code more flexible
In reply to Wei LiuMicrosoft Hypervisor provides a set of hypercalls to manage device domains. The root kernel should parse the DMAR so that it can program the IOMMU (with hypercalls) correctly.
The DMAR code was designed to work with Intel IOMMU only. Add two more parameters to make it useful to Microsoft Hypervisor. Microsoft Hypervisor does not need the DMAR parsing code to allocate an Intel IOMMU structure; it also wishes to always reparse the DMAR table even after it has been parsed before.
Adjust Intel IOMMU code to use the new dmar_table_init. There should be no functional change to Intel IOMMU code.
---
We may be able to combine alloc and force_parse?
--- drivers/iommu/intel/dmar.c | 38 ++++++++++++++++++++--------- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/irq_remapping.c | 2 +- include/linux/dmar.h | 2 +- 4 files changed, 30 insertions(+), 14 deletions(-)
drivers/iommu/intel/dmar.c +27 -11
@@ -408,7 +408,8 @@ dmar_find_dmaru(struct acpi_dmar_hardware_unit *drhd)* structure which uniquely represent one DMA remapping hardware unit * present in the platform */ -static int dmar_parse_one_drhd (struct acpi_dmar_header *header,void *arg) +static int dmar_parse_one_drhd_internal (struct acpi_dmar_header *header, +void *arg,bool alloc) {struct acpi_dmar_hardware_unit *drhd;struct dmar_drhd_unit *dmaru;
@@ -440,12 +441,14 @@ static int dmar_parse_one_drhd(struct acpi_dmar_header *header, void *arg)return -ENOMEM ; } - ret =alloc_iommu (dmaru); -if (ret) { -dmar_free_dev_scope (&dmaru->devices, - &dmaru->devices_cnt); -kfree (dmaru); -return ret; +if (alloc) { + ret =alloc_iommu (dmaru); +if (ret) { +dmar_free_dev_scope (&dmaru->devices, + &dmaru->devices_cnt); +kfree (dmaru); +return ret; + } }dmar_register_drhd_unit (dmaru);
@@ -456,6 +459,16 @@ static int dmar_parse_one_drhd(struct acpi_dmar_header *header, void *arg)return 0 ; } +static int dmar_parse_one_drhd (struct acpi_dmar_header *header,void *arg) +{ +return dmar_parse_one_drhd_internal (header, arg, true); +} + +int dmar_parse_one_drhd_noalloc (struct acpi_dmar_header *header,void *arg) +{ +return dmar_parse_one_drhd_internal (header, arg, false); +} +static void dmar_free_drhd (struct dmar_drhd_unit *dmaru) {if (dmaru->devices && dmaru->devices_cnt)
@@ -633,7 +646,7 @@ static inline int dmar_walk_dmar_table(struct acpi_table_dmar *dmar,* parse_dmar_table - parses the DMA reporting table */ static int __init -parse_dmar_table (void ) +parse_dmar_table (bool alloc) {struct acpi_table_dmar *dmar;int drhd_count =0 ;
@@ -650,6 +663,9 @@ parse_dmar_table(void) .cb[ACPI_DMAR_TYPE_SATC] = &dmar_parse_one_satc, }; +if (!alloc) + cb.cb[ACPI_DMAR_TYPE_HARDWARE_UNIT ] = &dmar_parse_one_drhd_noalloc; +/* * Do it again, earlier dmar_tbl mapping could be mapped with * fixed map.
@@ -840,13 +856,13 @@ void __init dmar_register_bus_notifier(void) } -int __init dmar_table_init (void ) +int __init dmar_table_init (bool alloc,bool force_parse) {static int dmar_table_initialized;int ret; -if (dmar_table_initialized ==0 ) { - ret =parse_dmar_table (); +if (dmar_table_initialized ==0 || force_parse) { + ret =parse_dmar_table (alloc);if (ret <0 ) {if (ret != -ENODEV )pr_info ("Parse DMAR table failure.\n" );
drivers/iommu/intel/iommu.c +1 -1
@@ -4310,7 +4310,7 @@ int __init intel_iommu_init(void) }down_write (&dmar_global_lock); -if (dmar_table_init ()) { +if (dmar_table_init (true, false)) {if (force_on)panic ("tboot: Failed to initialize DMAR table\n" ); goto out_free_dmar;
drivers/iommu/intel/irq_remapping.c +1 -1
@@ -732,7 +732,7 @@ static int __init intel_prepare_irq_remapping(void)return -ENODEV ; } -if (dmar_table_init () <0 ) +if (dmar_table_init (true, false) <0 )return -ENODEV ;if (intel_cap_audit (CAP_AUDIT_STATIC_IRQR ,NULL ))
include/linux/dmar.h +1 -1
@@ -106,7 +106,7 @@ static inline bool dmar_rcu_check(void) for_each_dev_scope((devs), (cnt), (i), (tmp)) \if (!(tmp)) {continue ; }else -extern int dmar_table_init(void); +extern int dmar_table_init(bool alloc, bool force_parse);extern int dmar_dev_scope_init (void );extern void dmar_register_bus_notifier (void ); extern int dmar_parse_dev_scope(void *start, void *end, int *cnt,
Signature · 1 line
2.30.2
Robin Murphy
reviewerRe: [RFC v1 3/8] intel/vt-d: make DMAR table parsing code more flexible
Quoted 9 lines · Wei Liu
Microsoft Hypervisor provides a set of hypercalls to manage device domains. The root kernel should parse the DMAR so that it can program the IOMMU (with hypercalls) correctly.
The DMAR code was designed to work with Intel IOMMU only. Add two more parameters to make it useful to Microsoft Hypervisor. Microsoft Hypervisor does not need the DMAR parsing code to allocate an Intel IOMMU structure; it also wishes to always reparse the DMAR table even after it has been parsed before.
We've recently defined the VIOT table for describing paravirtualised IOMMUs - would it make more sense to extend that to support the Microsoft implementation than to abuse a hardware-specific table? Am I right in assuming said hypervisor isn't intended to only ever run on Intel hardware?
Robin.
Quoted 141 lines
Adjust Intel IOMMU code to use the new
dmar_table_init. There should be no functional change to Intel IOMMU code.Signed-off-by Wei Liu <[email protected]>---
We may be able to combine alloc andforce_parse?--- drivers/iommu/intel/dmar.c | 38 ++++++++++++++++++++--------- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/irq_remapping.c | 2 +- include/linux/dmar.h | 2 +- 4 files changed, 30 insertions(+), 14 deletions(-)drivers/iommu/intel/dmar.c +27
-11@@ -408,7 +408,8 @@ dmar_find_dmaru(struct acpi_dmar_hardware_unit *drhd)* structure which uniquely represent one DMA remapping hardware unit * present in the platform */ -static int dmar_parse_one_drhd (struct acpi_dmar_header *header,void *arg) +static int dmar_parse_one_drhd_internal (struct acpi_dmar_header *header, +void *arg,bool alloc) {struct acpi_dmar_hardware_unit *drhd;struct dmar_drhd_unit *dmaru;@@ -440,12 +441,14 @@ static int dmar_parse_one_drhd(struct acpi_dmar_header *header, void *arg)return -ENOMEM ; } - ret =alloc_iommu (dmaru); -if (ret) { -dmar_free_dev_scope (&dmaru->devices, - &dmaru->devices_cnt); -kfree (dmaru); -return ret; +if (alloc) { + ret =alloc_iommu (dmaru); +if (ret) { +dmar_free_dev_scope (&dmaru->devices, + &dmaru->devices_cnt); +kfree (dmaru); +return ret; + } }dmar_register_drhd_unit (dmaru);@@ -456,6 +459,16 @@ static int dmar_parse_one_drhd(struct acpi_dmar_header *header, void *arg)return 0 ; } +static int dmar_parse_one_drhd (struct acpi_dmar_header *header,void *arg) +{ +return dmar_parse_one_drhd_internal (header, arg, true); +} + +int dmar_parse_one_drhd_noalloc (struct acpi_dmar_header *header,void *arg) +{ +return dmar_parse_one_drhd_internal (header, arg, false); +} +static void dmar_free_drhd (struct dmar_drhd_unit *dmaru) {if (dmaru->devices && dmaru->devices_cnt)@@ -633,7 +646,7 @@ static inline int dmar_walk_dmar_table(struct acpi_table_dmar *dmar,* parse_dmar_table - parses the DMA reporting table */ static int __init -parse_dmar_table (void ) +parse_dmar_table (bool alloc) {struct acpi_table_dmar *dmar;int drhd_count =0 ;@@ -650,6 +663,9 @@ parse_dmar_table(void) .cb[ACPI_DMAR_TYPE_SATC] = &dmar_parse_one_satc, }; +if (!alloc) + cb.cb[ACPI_DMAR_TYPE_HARDWARE_UNIT ] = &dmar_parse_one_drhd_noalloc; +/* * Do it again, earlier dmar_tbl mapping could be mapped with * fixed map. @@ -840,13 +856,13 @@ void __init dmar_register_bus_notifier(void) } -int __init dmar_table_init (void ) +int __init dmar_table_init (bool alloc,bool force_parse) {static int dmar_table_initialized;int ret; -if (dmar_table_initialized ==0 ) { - ret =parse_dmar_table (); +if (dmar_table_initialized ==0 || force_parse) { + ret =parse_dmar_table (alloc);if (ret <0 ) {if (ret != -ENODEV )pr_info ("Parse DMAR table failure.\n" );drivers/iommu/intel/iommu.c +1
-1@@ -4310,7 +4310,7 @@ int __init intel_iommu_init(void) }down_write (&dmar_global_lock); -if (dmar_table_init ()) { +if (dmar_table_init (true, false)) {if (force_on)panic ("tboot: Failed to initialize DMAR table\n" ); goto out_free_dmar;drivers/iommu/intel/irq_remapping.c +1
-1@@ -732,7 +732,7 @@ static int __init intel_prepare_irq_remapping(void)return -ENODEV ; } -if (dmar_table_init () <0 ) +if (dmar_table_init (true, false) <0 )return -ENODEV ;if (intel_cap_audit (CAP_AUDIT_STATIC_IRQR ,NULL ))include/linux/dmar.h +1
-1@@ -106,7 +106,7 @@ static inline bool dmar_rcu_check(void) for_each_dev_scope((devs), (cnt), (i), (tmp)) \if (!(tmp)) {continue ; }else -extern int dmar_table_init(void); +extern int dmar_table_init(bool alloc, bool force_parse);extern int dmar_dev_scope_init (void );extern void dmar_register_bus_notifier (void ); extern int dmar_parse_dev_scope(void *start, void *end, int *cnt,
Wei Liu
maintainerRe: [RFC v1 3/8] intel/vt-d: make DMAR table parsing code more flexible
Quoted 11 lines · Robin Murphy
Quoted 9 lines · Wei Liu
Microsoft Hypervisor provides a set of hypercalls to manage device domains. The root kernel should parse the DMAR so that it can program the IOMMU (with hypercalls) correctly.
The DMAR code was designed to work with Intel IOMMU only. Add two more parameters to make it useful to Microsoft Hypervisor. Microsoft Hypervisor does not need the DMAR parsing code to allocate an Intel IOMMU structure; it also wishes to always reparse the DMAR table even after it has been parsed before.
Quoted 3 lines
We've recently defined the VIOT table for describing paravirtualised IOMMUs
- would it make more sense to extend that to support the Microsoft implementation than to abuse a hardware-specific table? Am I right in
I searched for VIOT and believed I found the correct link
https://lwn.net/Articles/859291/. My understanding is based on the reading of that series.
VIOT is useful. I think it solves the problem for guests.
It does not solve the problem we have though. The DMAR tables are not conjured up by some backend software running on the host side. They are the real tables provided by the firmware. The kernel here is part of the host setup, dealing with physical hardware.
No matter how much I wish all vendors unified their tables, I don't see how that's going to happen for readily available servers. :-(
Quoted 1 line
assuming said hypervisor isn't intended to only ever run on Intel hardware?
Yes, that's correct. We also plan to add support AMD and ARM64.
Wei.
Wei Liu
maintainerRFC 4/8 intel/vt-d: export intel_iommu_get_resv_regions
In reply to Wei LiuWhen Microsoft Hypervisor runs on Intel platforms it needs to know the reserved regions to program devices correctly. There is no reason to duplicate intel_iommu_get_resv_regions. Export it.
--- drivers/iommu/intel/iommu.c | 5 +++-- include/linux/intel-iommu.h | 4 ++++ 2 files changed, 7 insertions(+), 2 deletions(-)
drivers/iommu/intel/iommu.c +3 -2
@@ -5176,8 +5176,8 @@ static void intel_iommu_probe_finalize(struct device *dev)set_dma_ops (dev,NULL ); } -static void intel_iommu_get_resv_regions (struct device *device, -struct list_head *head) +void intel_iommu_get_resv_regions (struct device *device, +struct list_head *head) {int prot =DMA_PTE_READ |DMA_PTE_WRITE ;struct iommu_resv_region *reg;
@@ -5232,6 +5232,7 @@ static void intel_iommu_get_resv_regions(struct device *device,return ;list_add_tail (®->list, head); } +EXPORT_SYMBOL_GPL (intel_iommu_get_resv_regions);int intel_iommu_enable_pasid (struct intel_iommu *iommu,struct device *dev) {
include/linux/intel-iommu.h +4 -0
@@ -814,6 +814,8 @@ extern int iommu_calculate_max_sagaw(struct intel_iommu *iommu);extern int dmar_disabled;extern int intel_iommu_enabled;extern int intel_iommu_gfx_mapped; +extern void intel_iommu_get_resv_regions (struct device *device, +struct list_head *head);#else static inline int iommu_calculate_agaw (struct intel_iommu *iommu) {
@@ -825,6 +827,8 @@ static inline int iommu_calculate_max_sagaw(struct intel_iommu *iommu) }#define dmar_disabled (1)#define intel_iommu_enabled (0) +static inline void intel_iommu_get_resv_regions (struct device *device, +struct list_head *head) {}#endif #endif
Signature · 1 line
2.30.2
Lu Baolu
maintainerRe: [RFC v1 4/8] intel/vt-d: export intel_iommu_get_resv_regions
Quoted 3 lines · Wei Liu
When Microsoft Hypervisor runs on Intel platforms it needs to know the reserved regions to program devices correctly. There is no reason to duplicate
intel_iommu_get_resv_regions. Export it.
Why not using iommu_get_resv_regions()?
Best regards,
baolu
Quoted 53 lines
Signed-off-by Wei Liu <[email protected]>--- drivers/iommu/intel/iommu.c | 5 +++-- include/linux/intel-iommu.h | 4 ++++ 2 files changed, 7 insertions(+), 2 deletions(-)drivers/iommu/intel/iommu.c +3
-2@@ -5176,8 +5176,8 @@ static void intel_iommu_probe_finalize(struct device *dev)set_dma_ops (dev,NULL ); } -static void intel_iommu_get_resv_regions (struct device *device, -struct list_head *head) +void intel_iommu_get_resv_regions (struct device *device, +struct list_head *head) {int prot =DMA_PTE_READ |DMA_PTE_WRITE ;struct iommu_resv_region *reg;@@ -5232,6 +5232,7 @@ static void intel_iommu_get_resv_regions(struct device *device,return ;list_add_tail (®->list, head); } +EXPORT_SYMBOL_GPL (intel_iommu_get_resv_regions);int intel_iommu_enable_pasid (struct intel_iommu *iommu,struct device *dev) {include/linux/intel-iommu.h +4
-0@@ -814,6 +814,8 @@ extern int iommu_calculate_max_sagaw(struct intel_iommu *iommu);extern int dmar_disabled;extern int intel_iommu_enabled;extern int intel_iommu_gfx_mapped; +extern void intel_iommu_get_resv_regions (struct device *device, +struct list_head *head);#else static inline int iommu_calculate_agaw (struct intel_iommu *iommu) {@@ -825,6 +827,8 @@ static inline int iommu_calculate_max_sagaw(struct intel_iommu *iommu) }#define dmar_disabled (1)#define intel_iommu_enabled (0) +static inline void intel_iommu_get_resv_regions (struct device *device, +struct list_head *head) {}#endif #endif
Wei Liu
maintainerRe: [RFC v1 4/8] intel/vt-d: export intel_iommu_get_resv_regions
Quoted 6 lines · Lu Baolu
Quoted 3 lines · Wei Liu
When Microsoft Hypervisor runs on Intel platforms it needs to know the reserved regions to program devices correctly. There is no reason to duplicate
intel_iommu_get_resv_regions. Export it.Why not using
iommu_get_resv_regions()?
That calls into ops->get_resv_regions.
In this patch series, get_resv_regions is hv_iommu_resv_regions, which wants to use intel_iommu_get_resv_regions when it detects the underlying hardware platform is from Intel.
Wei.
Wei Liu
maintainerRFC 5/8 mshv: add paravirtualized IOMMU support
In reply to Wei LiuMicrosoft Hypervisor provides a set of hypercalls to manage device domains. Implement a type-1 IOMMU using those hypercalls.
Implement DMA remapping as the first step for this driver. Interrupt remapping will come in a later stage.
--- drivers/iommu/Kconfig | 14 + drivers/iommu/hyperv-iommu.c | 628 +++++++++++++++++++++++++++++++++++ 2 files changed, 642 insertions(+)
drivers/iommu/Kconfig +14 -0
@@ -397,6 +397,20 @@ config HYPERV_IOMMU Stub IOMMU driver to handle IRQs as to allow Hyper-V Linux guests to run with x2APIC mode enabled. +config HYPERV_ROOT_PVIOMMU + bool "Microsoft Hypervisor paravirtualized IOMMU support" + depends on HYPERV && X86 + select IOMMU_API + select IOMMU_DMA + select DMA_OPS + select IOMMU_IOVA + select INTEL_IOMMU + select DMAR_TABLE + default HYPERV + help + A paravirtualized IOMMU for Microsoft Hypervisor. It supports DMA + remapping. + config VIRTIO_IOMMU tristate "Virtio IOMMU driver" depends on VIRTIO
drivers/iommu/hyperv-iommu.c +628 -0
644 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [RFC v1 5/8] mshv: add paravirtualized IOMMU support
Quoted 12 lines · Wei Liu
drivers/iommu/hyperv-iommu.c +12
-0+static void hv_iommu_domain_free (struct iommu_domain *d) +{ +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +unsigned long flags; +u64 status; +struct hv_input_delete_device_domain *input; + +if (is_identity_domain (domain) ||is_null_domain (domain)) +return ; + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg);
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 536
Quoted 10 lines
drivers/iommu/hyperv-iommu.c +10
-0+memset (input,0 ,sizeof (*input)); + + input->device_domain= domain->device_domain; + + status =hv_do_hypercall (HVCALL_DELETE_DEVICE_DOMAIN , input,NULL ); + +local_irq_restore (flags); + +if (!hv_result_success (status)) +pr_err ("%s: hypercall failed, status %lld\n" , __func__, status);
Is it OK to deallocate the resources, if hypercall has failed ?
Do we have any specific error code EBUSY (kind of) which we need to wait upon ?
Quoted 28 lines
drivers/iommu/hyperv-iommu.c +28
-0+ +ida_free (&domain->hv_iommu->domain_ids, domain->device_domain.domain_id.id); + +iommu_put_dma_cookie (d); + +kfree (domain); +} + +static int hv_iommu_attach_dev (struct iommu_domain *d,struct device *dev) +{ +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +u64 status; +unsigned long flags; +struct hv_input_attach_device_domain *input; +struct pci_dev *pdev; +struct hv_iommu_endpoint *vdev =dev_iommu_priv_get (dev); + +/* Only allow PCI devices for now */ +if (!dev_is_pci (dev)) +return -EINVAL ; + + pdev =to_pci_dev (dev); + +dev_dbg (dev,"Attaching (%strusted) to %d\n" , pdev->untrusted ?"un" :"" , + domain->device_domain.domain_id.id); + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg);
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 574
Quoted 10 lines
drivers/iommu/hyperv-iommu.c +10
-0+memset (input,0 ,sizeof (*input)); + + input->device_domain = domain->device_domain; + input->device_id =hv_build_pci_dev_id (pdev); + + status =hv_do_hypercall (HVCALL_ATTACH_DEVICE_DOMAIN , input,NULL ); +local_irq_restore (flags); + +if (!hv_result_success (status)) +pr_err ("%s: hypercall failed, status %lld\n" , __func__, status);
Does it make sense to vdev->domain = NULL ?
Quoted 143 lines
drivers/iommu/hyperv-iommu.c +143
-0+else + vdev->domain = domain; + +return hv_status_to_errno (status); +} + +static void hv_iommu_detach_dev (struct iommu_domain *d,struct device *dev) +{ +u64 status; +unsigned long flags; +struct hv_input_detach_device_domain *input; +struct pci_dev *pdev; +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +struct hv_iommu_endpoint *vdev =dev_iommu_priv_get (dev); + +/* See the attach function, only PCI devices for now */ +if (!dev_is_pci (dev)) +return ; + + pdev =to_pci_dev (dev); + +dev_dbg (dev,"Detaching from %d\n" , domain->device_domain.domain_id.id); + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg); +memset (input,0 ,sizeof (*input)); + + input->partition_id =HV_PARTITION_ID_SELF ; + input->device_id =hv_build_pci_dev_id (pdev); + + status =hv_do_hypercall (HVCALL_DETACH_DEVICE_DOMAIN , input,NULL ); +local_irq_restore (flags); + +if (!hv_result_success (status)) +pr_err ("%s: hypercall failed, status %lld\n" , __func__, status); + + vdev->domain =NULL ; +} + +static int hv_iommu_add_mapping (struct hv_iommu_domain *domain,unsigned long iova, +phys_addr_t paddr,size_t size,u32 flags) +{ +unsigned long irqflags; +struct hv_iommu_mapping *mapping; + + mapping =kzalloc (sizeof (*mapping),GFP_ATOMIC ); +if (!mapping) +return -ENOMEM ; + + mapping->paddr = paddr; + mapping->iova.start = iova; + mapping->iova.last = iova + size -1 ; + mapping->flags = flags; + +spin_lock_irqsave (&domain->mappings_lock, irqflags); +interval_tree_insert (&mapping->iova, &domain->mappings); +spin_unlock_irqrestore (&domain->mappings_lock, irqflags); + +return 0 ; +} + +static size_t hv_iommu_del_mappings (struct hv_iommu_domain *domain, +unsigned long iova,size_t size) +{ +unsigned long flags; +size_t unmapped =0 ; +unsigned long last = iova + size -1 ; +struct hv_iommu_mapping *mapping =NULL ; +struct interval_tree_node *node, *next; + +spin_lock_irqsave (&domain->mappings_lock, flags); + next =interval_tree_iter_first (&domain->mappings, iova, last); +while (next) { + node = next; + mapping = container_of(node, struct hv_iommu_mapping, iova); + next =interval_tree_iter_next (node, iova, last); + +/* Trying to split a mapping? Not supported for now. */ +if (mapping->iova.start < iova) +break ; + + unmapped += mapping->iova.last - mapping->iova.start +1 ; + +interval_tree_remove (node, &domain->mappings); +kfree (mapping); + } +spin_unlock_irqrestore (&domain->mappings_lock, flags); + +return unmapped; +} + +static int hv_iommu_map (struct iommu_domain *d,unsigned long iova, +phys_addr_t paddr,size_t size,int prot,gfp_t gfp) +{ +u32 map_flags; +unsigned long flags, pfn, npages; +int ret, i; +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +struct hv_input_map_device_gpa_pages *input; +u64 status; + +/* Reject size that's not a whole page */ +if (size & ~HV_HYP_PAGE_MASK ) +return -EINVAL ; + + map_flags =HV_MAP_GPA_READABLE ;/* Always required */ + map_flags |= prot &IOMMU_WRITE ?HV_MAP_GPA_WRITABLE :0 ; + + ret =hv_iommu_add_mapping (domain, iova, paddr, size, flags); +if (ret) +return ret; + + npages = size >>HV_HYP_PAGE_SHIFT ; + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg); +memset (input,0 ,sizeof (*input)); + + input->device_domain = domain->device_domain; + input->map_flags = map_flags; + input->target_device_va_base = iova; + + pfn = paddr >>HV_HYP_PAGE_SHIFT ; +for (i =0 ; i < npages; i++) { + input->gpa_page_list[i] = pfn; + pfn +=1 ; + } + + status =hv_do_rep_hypercall (HVCALL_MAP_DEVICE_GPA_PAGES , npages,0 , + input,NULL ); + +local_irq_restore (flags); + +if (!hv_result_success (status)) { +pr_err ("%s: hypercall failed, status %lld\n" , __func__, status); +hv_iommu_del_mappings (domain, iova, size); + } + +return hv_status_to_errno (status); +} + +static size_t hv_iommu_unmap (struct iommu_domain *d,unsigned long iova, +size_t size,struct iommu_iotlb_gather *gather)
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 727
Quoted 10 lines
drivers/iommu/hyperv-iommu.c +10
-0+{ +size_t unmapped; +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +unsigned long flags, npages; +struct hv_input_unmap_device_gpa_pages *input; +u64 status; + + unmapped =hv_iommu_del_mappings (domain, iova, size); +if (unmapped < size) +return 0 ;
Is there a case where unmapped > 0 && unmapped < size ?
Quoted 12 lines
drivers/iommu/hyperv-iommu.c +12
-0+ + npages = size >>HV_HYP_PAGE_SHIFT ; + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg); +memset (input,0 ,sizeof (*input)); + + input->device_domain = domain->device_domain; + input->target_device_va_base = iova; + +/* Unmap `npages` pages starting from VA base */ + status = hv_do_rep_hypercall(HVCALL_UNMAP_DEVICE_GPA_PAGES, npages,
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 749
Quoted 10 lines
drivers/iommu/hyperv-iommu.c +10
-0+ 0, input, NULL); + +local_irq_restore (flags); + +if (!hv_result_success (status)) +pr_err ("%s: hypercall failed, status %lld\n" , __func__, status); + +return hv_result_success (status) ? unmapped :0 ; +} +
Regards,
~Praveen.
Wei Liu
maintainerRe: [RFC v1 5/8] mshv: add paravirtualized IOMMU support
Quoted 17 lines · Praveen Kumar
Quoted 16 lines · Wei Liu
drivers/iommu/hyperv-iommu.c +16
-0+static void hv_iommu_domain_free (struct iommu_domain *d) +{ +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +unsigned long flags; +u64 status; +struct hv_input_delete_device_domain *input; + +if (is_identity_domain (domain) ||is_null_domain (domain)) +return ; + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg); +memset (input,0 ,sizeof (*input)); + + input->device_domain= domain->device_domain; +
Quoted 8 lines
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 536
Quoted 6 lines
drivers/iommu/hyperv-iommu.c +6
-0+ status =hv_do_hypercall (HVCALL_DELETE_DEVICE_DOMAIN , input,NULL ); + +local_irq_restore (flags); + +if (!hv_result_success (status)) +pr_err ("%s: hypercall failed, status %lld\n" , __func__, status);Is it OK to deallocate the resources, if hypercall has failed ?
It should be fine. We leak some resources in the hypervisor, but Linux is in a rather wedged state anyway. Refusing to free up resources in Linux does not much good.
Quoted 2 lines
Do we have any specific error code EBUSY (kind of) which we need to wait upon ?
I have not found a circumstance that can happen.
Quoted 32 lines
Quoted 32 lines
drivers/iommu/hyperv-iommu.c +32
-0+ +ida_free (&domain->hv_iommu->domain_ids, domain->device_domain.domain_id.id); + +iommu_put_dma_cookie (d); + +kfree (domain); +} + +static int hv_iommu_attach_dev (struct iommu_domain *d,struct device *dev) +{ +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +u64 status; +unsigned long flags; +struct hv_input_attach_device_domain *input; +struct pci_dev *pdev; +struct hv_iommu_endpoint *vdev =dev_iommu_priv_get (dev); + +/* Only allow PCI devices for now */ +if (!dev_is_pci (dev)) +return -EINVAL ; + + pdev =to_pci_dev (dev); + +dev_dbg (dev,"Attaching (%strusted) to %d\n" , pdev->untrusted ?"un" :"" , + domain->device_domain.domain_id.id); + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg); +memset (input,0 ,sizeof (*input)); + + input->device_domain = domain->device_domain; + input->device_id =hv_build_pci_dev_id (pdev);
Quoted 9 lines
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 574
Quoted 6 lines
drivers/iommu/hyperv-iommu.c +6
-0+ + status =hv_do_hypercall (HVCALL_ATTACH_DEVICE_DOMAIN , input,NULL ); +local_irq_restore (flags); + +if (!hv_result_success (status)) +pr_err ("%s: hypercall failed, status %lld\n" , __func__, status);Does it make sense to
vdev->domain= NULL ?
It is already NULL -- there is no other code path that sets it and the detach path always sets the field to NULL.
Quoted 6 lines
Quoted 6 lines
drivers/iommu/hyperv-iommu.c +6
-0+else + vdev->domain = domain; + +return hv_status_to_errno (status); +} +
[...]
Quoted 6 lines
Quoted 6 lines
drivers/iommu/hyperv-iommu.c +6
-0+static size_t hv_iommu_unmap (struct iommu_domain *d,unsigned long iova, +size_t size,struct iommu_iotlb_gather *gather) +{ +size_t unmapped; +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +unsigned long flags, npages;
Quoted 9 lines
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 727
Quoted 6 lines
drivers/iommu/hyperv-iommu.c +6
-0+struct hv_input_unmap_device_gpa_pages *input; +u64 status; + + unmapped =hv_iommu_del_mappings (domain, iova, size); +if (unmapped < size) +return 0 ;Is there a case where unmapped > 0 && unmapped < size ?
There could be such a case -- hv_iommu_del_mappings' return value is >= 0. Is there a problem with this predicate?
Wei.
Re: [RFC v1 5/8] mshv: add paravirtualized IOMMU support
Quoted 14 lines · Wei Liu
Quoted 5 lines
Quoted 5 lines
drivers/iommu/hyperv-iommu.c +5
-0+static size_t hv_iommu_unmap (struct iommu_domain *d,unsigned long iova, +size_t size,struct iommu_iotlb_gather *gather) +{ +size_t unmapped; +struct hv_iommu_domain *domain =to_hv_iommu_domain (d);Quoted 9 lines
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 727
Quoted 7 lines
drivers/iommu/hyperv-iommu.c +7
-0+unsigned long flags, npages; +struct hv_input_unmap_device_gpa_pages *input; +u64 status; + + unmapped =hv_iommu_del_mappings (domain, iova, size); +if (unmapped < size) +return 0 ;Is there a case where unmapped > 0 && unmapped < size ?
Quoted 2 lines
There could be such a case --
hv_iommu_del_mappings' return value is >= 0. Is there a problem with this predicate?
What I understand, if we are unmapping and return 0, means nothing was unmapped, and will that not cause any corruption or illegal access of unmapped memory later? From __iommu_unmap
...
13 while (unmapped < size) {
12 size_t pgsize = iommu_pgsize(domain, iova, size - unmapped);
11
10 unmapped_page = ops->unmap(domain, iova, pgsize, iotlb_gather);
9 if (!unmapped_page)
8 break; <<< we just break here, thinking there is nothing unmapped, but actually hv_iommu_del_mappings has removed some pages.
7
6 pr_debug("unmapped: iova 0x%lx size 0x%zx\n",
5 ¦iova, unmapped_page);
4
3 iova += unmapped_page;
2 unmapped += unmapped_page;
1 }...
Am I missing something ?
Regards,
~Praveen.
Wei Liu
maintainerRe: [RFC v1 5/8] mshv: add paravirtualized IOMMU support
Quoted 15 lines · Praveen Kumar
Quoted 14 lines · Wei Liu
Quoted 14 lines
drivers/iommu/hyperv-iommu.c static const struct irq_domain_ops hyperv_root_ir_domain_ops = { · line 727
Quoted 12 lines
drivers/iommu/hyperv-iommu.c +12
-0+static size_t hv_iommu_unmap (struct iommu_domain *d,unsigned long iova, +size_t size,struct iommu_iotlb_gather *gather) +{ +size_t unmapped; +struct hv_iommu_domain *domain =to_hv_iommu_domain (d); +unsigned long flags, npages; +struct hv_input_unmap_device_gpa_pages *input; +u64 status; + + unmapped =hv_iommu_del_mappings (domain, iova, size); +if (unmapped < size) +return 0 ;Is there a case where unmapped > 0 && unmapped < size ?
Quoted 6 lines
Quoted 2 lines
There could be such a case --
hv_iommu_del_mappings' return value is >= 0. Is there a problem with this predicate?What I understand, if we are unmapping and return 0, means nothing was unmapped, and will that not cause any corruption or illegal access of unmapped memory later? From
__iommu_unmap
Those pages are not really unmapped. The hypercall is skipped.
Quoted 21 lines
...
13 while (unmapped < size) { 12 size_t pgsize = iommu_pgsize(domain, iova, size - unmapped); 11 10 unmapped_page = ops->unmap(domain, iova, pgsize, iotlb_gather); 9 if (!unmapped_page) 8 break; <<< we just break here, thinking there is nothing unmapped, but actually hv_iommu_del_mappings has removed some pages. 7 6 pr_debug("unmapped: iova 0x%lx size 0x%zx\n", 5 ¦iova, unmapped_page); 4 3 iova += unmapped_page; 2 unmapped += unmapped_page; 1 }...
Am I missing something ?
Regards,
~Praveen.
Wei Liu
maintainerRFC 6/8 mshv: command line option to skip devices in PV-IOMMU
In reply to Wei LiuSome devices may have been claimed by the hypervisor already. One such example is a user can assign a NIC for debugging purpose.
Ideally Linux should be able to tell retrieve that information, but there is no way to do that yet. And designing that new mechanism is going to take time.
Provide a command line option for skipping devices. This is a stopgap solution, so it is intentionally undocumented. Hopefully we can retire it in the future.
--- drivers/iommu/hyperv-iommu.c | 45 ++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
drivers/iommu/hyperv-iommu.c +45 -0
@@ -349,6 +349,16 @@ static const struct irq_domain_ops hyperv_root_ir_domain_ops = { #ifdef CONFIG_HYPERV_ROOT_PVIOMMU +/* The IOMMU will not claim these PCI devices. */ +static char *pci_devs_to_skip; +static int __init mshv_iommu_setup_skip(char *str) { + pci_devs_to_skip = str; + + return 0; +} +/* mshv_iommu_skip=(SSSS:BB:DD.F)(SSSS:BB:DD.F) */ +__setup("mshv_iommu_skip=", mshv_iommu_setup_skip); + /* DMA remapping support */ struct hv_iommu_domain { struct iommu_domain domain;
@@ -774,6 +784,41 @@ static struct iommu_device *hv_iommu_probe_device(struct device *dev)if (!dev_is_pci (dev))return ERR_PTR (-ENODEV ); +/* +* Skip the PCI device specified in `pci_devs_to_skip`. This is a +* temporary solution until we figure out a way to extract information +* from the hypervisor what devices it is already using. +*/ +if (pci_devs_to_skip && *pci_devs_to_skip) { +int pos =0 ; +int parsed; +int segment, bus, slot, func; +struct pci_dev *pdev =to_pci_dev (dev); + +do { + parsed =0 ; + +sscanf (pci_devs_to_skip + pos, +" (%x:%x:%x.%x) %n" , + &segment, &bus, &slot, &func, &parsed); + +if (parsed <=0 ) +break ; + +if (pci_domain_nr (pdev->bus) == segment && + pdev->bus->number == bus && +PCI_SLOT (pdev->devfn) == slot && +PCI_FUNC (pdev->devfn) == func) + { +dev_info (dev,"skipped by MSHV IOMMU\n" ); +return ERR_PTR (-ENODEV ); + } + + pos += parsed; + + }while (pci_devs_to_skip[pos]); + } + vdev =kzalloc (sizeof (*vdev),GFP_KERNEL );if (!vdev)return ERR_PTR (-ENOMEM );
Signature · 1 line
2.30.2
Robin Murphy
reviewerRe: [RFC v1 6/8] mshv: command line option to skip devices in PV-IOMMU
Quoted 10 lines · Wei Liu
Some devices may have been claimed by the hypervisor already. One such example is a user can assign a NIC for debugging purpose.
Ideally Linux should be able to tell retrieve that information, but there is no way to do that yet. And designing that new mechanism is going to take time.
Provide a command line option for skipping devices. This is a stopgap solution, so it is intentionally undocumented. Hopefully we can retire it in the future.
Huh? If the host is using a device, why the heck is it exposing any knowledge of that device to the guest at all, let alone allowing the guest to do anything that could affect its operation!?
Robin.
Quoted 69 lines
Signed-off-by Wei Liu <[email protected]>--- drivers/iommu/hyperv-iommu.c | 45 ++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)drivers/iommu/hyperv-iommu.c +45
-0@@ -349,6 +349,16 @@ static const struct irq_domain_ops hyperv_root_ir_domain_ops = { #ifdef CONFIG_HYPERV_ROOT_PVIOMMU +/* The IOMMU will not claim these PCI devices. */ +static char *pci_devs_to_skip; +static int __init mshv_iommu_setup_skip(char *str) { + pci_devs_to_skip = str; + + return 0; +} +/* mshv_iommu_skip=(SSSS:BB:DD.F)(SSSS:BB:DD.F) */ +__setup("mshv_iommu_skip=", mshv_iommu_setup_skip); + /* DMA remapping support */ struct hv_iommu_domain { struct iommu_domain domain;@@ -774,6 +784,41 @@ static struct iommu_device *hv_iommu_probe_device(struct device *dev)if (!dev_is_pci (dev))return ERR_PTR (-ENODEV ); +/* +* Skip the PCI device specified in `pci_devs_to_skip`. This is a +* temporary solution until we figure out a way to extract information +* from the hypervisor what devices it is already using. +*/ +if (pci_devs_to_skip && *pci_devs_to_skip) { +int pos =0 ; +int parsed; +int segment, bus, slot, func; +struct pci_dev *pdev =to_pci_dev (dev); + +do { + parsed =0 ; + +sscanf (pci_devs_to_skip + pos, +" (%x:%x:%x.%x) %n" , + &segment, &bus, &slot, &func, &parsed); + +if (parsed <=0 ) +break ; + +if (pci_domain_nr (pdev->bus) == segment && + pdev->bus->number == bus && +PCI_SLOT (pdev->devfn) == slot && +PCI_FUNC (pdev->devfn) == func) + { +dev_info (dev,"skipped by MSHV IOMMU\n" ); +return ERR_PTR (-ENODEV ); + } + + pos += parsed; + + }while (pci_devs_to_skip[pos]); + } + vdev =kzalloc (sizeof (*vdev),GFP_KERNEL );if (!vdev)return ERR_PTR (-ENOMEM );
Wei Liu
maintainerRe: [RFC v1 6/8] mshv: command line option to skip devices in PV-IOMMU
Quoted 8 lines · Robin Murphy
Quoted 7 lines · Wei Liu
Some devices may have been claimed by the hypervisor already. One such example is a user can assign a NIC for debugging purpose.
Ideally Linux should be able to tell retrieve that information, but there is no way to do that yet. And designing that new mechanism is going to take time.
Quoted 7 lines
Quoted 3 lines
Provide a command line option for skipping devices. This is a stopgap solution, so it is intentionally undocumented. Hopefully we can retire it in the future.
Huh? If the host is using a device, why the heck is it exposing any knowledge of that device to the guest at all, let alone allowing the guest to do anything that could affect its operation!?
The host in this setup consists of the hypervisor, the root kernel and a bunch of user space programs.
Root is not an ordinary guest. It does need to know all the hardware to manage the platform. Hypervisor does not claim more devices than it needs to, nor does it try to hide hardware details from the root.
The hypervisor can protect itself just fine. Any attempt to use the already claimed devices will be blocked or rejected, so are the attempts to attach them to device domains.
That, however, leads to some interesting interactions between the hypervisor and Linux kernel. When kernel initializes IOMMU during boot, it will try to attach all devices in one go. Any failure there will cause kernel to detach the already attached devices. That's not fatal to kernel, and is only a minor annoyance to our current use case, because the default domain is a passthrough domain anyway. It will become problematic once we switch the default domain to a DMA domain to further tighten security during Linux boot.
Wei.
Quoted 3 lines
Robin.
Re: [RFC v1 6/8] mshv: command line option to skip devices in PV-IOMMU
In reply to Wei LiuQuoted 64 lines · Wei Liu
Some devices may have been claimed by the hypervisor already. One such example is a user can assign a NIC for debugging purpose.
Ideally Linux should be able to tell retrieve that information, but there is no way to do that yet. And designing that new mechanism is going to take time.
Provide a command line option for skipping devices. This is a stopgap solution, so it is intentionally undocumented. Hopefully we can retire it in the future.
Signed-off-by Wei Liu <[email protected]>--- drivers/iommu/hyperv-iommu.c | 45 ++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)drivers/iommu/hyperv-iommu.c +33
-0@@ -349,6 +349,16 @@ static const struct irq_domain_ops hyperv_root_ir_domain_ops = { #ifdef CONFIG_HYPERV_ROOT_PVIOMMU +/* The IOMMU will not claim these PCI devices. */ +static char *pci_devs_to_skip; +static int __init mshv_iommu_setup_skip(char *str) { + pci_devs_to_skip = str; + + return 0; +} +/* mshv_iommu_skip=(SSSS:BB:DD.F)(SSSS:BB:DD.F) */ +__setup("mshv_iommu_skip=", mshv_iommu_setup_skip); + /* DMA remapping support */ struct hv_iommu_domain { struct iommu_domain domain;@@ -774,6 +784,41 @@ static struct iommu_device *hv_iommu_probe_device(struct device *dev)if (!dev_is_pci (dev))return ERR_PTR (-ENODEV ); +/* +* Skip the PCI device specified in `pci_devs_to_skip`. This is a +* temporary solution until we figure out a way to extract information +* from the hypervisor what devices it is already using. +*/ + if (pci_devs_to_skip && *pci_devs_to_skip) { + int pos = 0; + int parsed; + int segment, bus, slot, func; + struct pci_dev *pdev = to_pci_dev(dev); + + do { + parsed = 0; + + sscanf(pci_devs_to_skip + pos, + " (%x:%x:%x.%x) %n", + &segment, &bus, &slot, &func, &parsed); + + if (parsed <= 0) + break; + + if (pci_domain_nr(pdev->bus) == segment && + pdev->bus->number == bus &&
drivers/iommu/hyperv-iommu.c static struct iommu_device *hv_iommu_probe_device(struct device *dev) · line 819
Quoted 10 lines
drivers/iommu/hyperv-iommu.c +10
-0+ PCI_SLOT(pdev->devfn) == slot && + PCI_FUNC(pdev->devfn) == func) + { +dev_info (dev,"skipped by MSHV IOMMU\n" ); +return ERR_PTR (-ENODEV ); + } + + pos += parsed; + + }while (pci_devs_to_skip[pos]);
Is there a possibility of pci_devs_to_skip + pos > sizeof(pci_devs_to_skip) and also a valid memory ?
I would recommend to have a check of size as well before accessing the array content, just to be safer accessing any memory.
drivers/iommu/hyperv-iommu.c static struct iommu_device *hv_iommu_probe_device(struct device *dev) · line 824
Quoted 6 lines
drivers/iommu/hyperv-iommu.c +2
-0+ } + vdev =kzalloc (sizeof (*vdev),GFP_KERNEL );if (!vdev)return ERR_PTR (-ENOMEM );
Regards,
~Praveen.
Wei Liu
maintainerRe: [RFC v1 6/8] mshv: command line option to skip devices in PV-IOMMU
Quoted 70 lines · Praveen Kumar
Quoted 69 lines · Wei Liu
Some devices may have been claimed by the hypervisor already. One such example is a user can assign a NIC for debugging purpose.
Ideally Linux should be able to tell retrieve that information, but there is no way to do that yet. And designing that new mechanism is going to take time.
Provide a command line option for skipping devices. This is a stopgap solution, so it is intentionally undocumented. Hopefully we can retire it in the future.
Signed-off-by Wei Liu <[email protected]>--- drivers/iommu/hyperv-iommu.c | 45 ++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)drivers/iommu/hyperv-iommu.c +38
-0@@ -349,6 +349,16 @@ static const struct irq_domain_ops hyperv_root_ir_domain_ops = { #ifdef CONFIG_HYPERV_ROOT_PVIOMMU +/* The IOMMU will not claim these PCI devices. */ +static char *pci_devs_to_skip; +static int __init mshv_iommu_setup_skip(char *str) { + pci_devs_to_skip = str; + + return 0; +} +/* mshv_iommu_skip=(SSSS:BB:DD.F)(SSSS:BB:DD.F) */ +__setup("mshv_iommu_skip=", mshv_iommu_setup_skip); + /* DMA remapping support */ struct hv_iommu_domain { struct iommu_domain domain;@@ -774,6 +784,41 @@ static struct iommu_device *hv_iommu_probe_device(struct device *dev)if (!dev_is_pci (dev))return ERR_PTR (-ENODEV ); +/* +* Skip the PCI device specified in `pci_devs_to_skip`. This is a +* temporary solution until we figure out a way to extract information +* from the hypervisor what devices it is already using. +*/ +if (pci_devs_to_skip && *pci_devs_to_skip) { +int pos =0 ; +int parsed; +int segment, bus, slot, func; +struct pci_dev *pdev =to_pci_dev (dev); + + do { + parsed = 0; + + sscanf(pci_devs_to_skip + pos, + " (%x:%x:%x.%x) %n", + &segment, &bus, &slot, &func, &parsed); + + if (parsed <= 0) + break; + + if (pci_domain_nr(pdev->bus) == segment && + pdev->bus->number == bus && + PCI_SLOT(pdev->devfn) == slot && + PCI_FUNC(pdev->devfn) == func) + { + dev_info(dev, "skipped by MSHV IOMMU\n"); + return ERR_PTR(-ENODEV);
Quoted 8 lines
drivers/iommu/hyperv-iommu.c static struct iommu_device *hv_iommu_probe_device(struct device *dev) · line 819
Quoted 5 lines
drivers/iommu/hyperv-iommu.c +5
-0+ } + + pos += parsed; + + }while (pci_devs_to_skip[pos]);Is there a possibility of
pci_devs_to_skip+ pos >sizeof(pci_devs_to_skip)and also a valid memory ?
pos should point to the last parsed position. If parsing fails pos does not get updated and the code breaks out of the loop. If parsing is success pos should point to either the start of next element of '\0' (end of string). To me this is good enough.
Quoted 3 lines
I would recommend to have a check of size as well before accessing the array content, just to be safer accessing any memory.
What check do you have in mind?
Wei.
Quoted 10 lines
drivers/iommu/hyperv-iommu.c static struct iommu_device *hv_iommu_probe_device(struct device *dev) · line 824
Quoted 6 lines
drivers/iommu/hyperv-iommu.c +2
-0+ } + vdev =kzalloc (sizeof (*vdev),GFP_KERNEL );if (!vdev)return ERR_PTR (-ENOMEM );Regards,
~Praveen.
Re: [RFC v1 6/8] mshv: command line option to skip devices in PV-IOMMU
Quoted 42 lines · Wei Liu
Quoted 33 lines
Quoted 33 lines
drivers/iommu/hyperv-iommu.c +28
-0struct iommu_domain domain;@@ -774,6 +784,41 @@ static struct iommu_device *hv_iommu_probe_device(struct device *dev)if (!dev_is_pci (dev))return ERR_PTR (-ENODEV ); +/* +* Skip the PCI device specified in `pci_devs_to_skip`. This is a +* temporary solution until we figure out a way to extract information +* from the hypervisor what devices it is already using. +*/ +if (pci_devs_to_skip && *pci_devs_to_skip) { +int pos =0 ; +int parsed; +int segment, bus, slot, func; +struct pci_dev *pdev =to_pci_dev (dev); + + do { + parsed = 0; + + sscanf(pci_devs_to_skip + pos, + " (%x:%x:%x.%x) %n", + &segment, &bus, &slot, &func, &parsed); + + if (parsed <= 0) + break; + + if (pci_domain_nr(pdev->bus) == segment && + pdev->bus->number == bus && + PCI_SLOT(pdev->devfn) == slot && + PCI_FUNC(pdev->devfn) == func) + { + dev_info(dev, "skipped by MSHV IOMMU\n"); + return ERR_PTR(-ENODEV);Quoted 8 lines
drivers/iommu/hyperv-iommu.c static struct iommu_device *hv_iommu_probe_device(struct device *dev) · line 819
Quoted 5 lines
drivers/iommu/hyperv-iommu.c +5
-0+ } + + pos += parsed; + + }while (pci_devs_to_skip[pos]);Is there a possibility of
pci_devs_to_skip+ pos >sizeof(pci_devs_to_skip)and also a valid memory ?
Quoted 4 lines
pos should point to the last parsed position. If parsing fails pos does not get updated and the code breaks out of the loop. If parsing is success pos should point to either the start of next element of '\0' (end of string). To me this is good enough.
The point is, hypothetically the address to pci_devs_to_skip + pos can be valid address (later to '\0'), and thus there is a possibility, that parsing may not fail. Another, there is also a possibility of sscanf faulting accessing the illegal address, if pci_devs_to_skip[pos] turns out to be not NULL or valid address.
Quoted 6 lines
Quoted 4 lines
I would recommend to have a check of size as well before accessing the array content, just to be safer accessing any memory.
What check do you have in mind?
Something like,size_t len = strlen(pci_devs_to_skip);
do {
len -= parsed; } while (len);
OR
do {
...
pos += parsed; } while (pos < len);
Further, I'm also fine with the existing code, if you think this won't break and already been taken care. Thanks.
Regards,
~Praveen.
Wei Liu
maintainerRe: [RFC v1 6/8] mshv: command line option to skip devices in PV-IOMMU
Quoted 43 lines · Praveen Kumar
Quoted 42 lines · Wei Liu
Quoted 41 lines
Quoted 28 lines
drivers/iommu/hyperv-iommu.c +23
-0struct iommu_domain domain;@@ -774,6 +784,41 @@ static struct iommu_device *hv_iommu_probe_device(struct device *dev)if (!dev_is_pci (dev))return ERR_PTR (-ENODEV ); +/* +* Skip the PCI device specified in `pci_devs_to_skip`. This is a +* temporary solution until we figure out a way to extract information +* from the hypervisor what devices it is already using. +*/ + if (pci_devs_to_skip && *pci_devs_to_skip) { + int pos = 0; + int parsed; + int segment, bus, slot, func; + struct pci_dev *pdev = to_pci_dev(dev); + + do { + parsed = 0; + + sscanf(pci_devs_to_skip + pos, + " (%x:%x:%x.%x) %n", + &segment, &bus, &slot, &func, &parsed); + + if (parsed <= 0) + break; + + if (pci_domain_nr(pdev->bus) == segment && + pdev->bus->number == bus &&drivers/iommu/hyperv-iommu.c static struct iommu_device *hv_iommu_probe_device(struct device *dev) · line 819
Quoted 10 lines
drivers/iommu/hyperv-iommu.c +10
-0+ PCI_SLOT(pdev->devfn) == slot && + PCI_FUNC(pdev->devfn) == func) + { +dev_info (dev,"skipped by MSHV IOMMU\n" ); +return ERR_PTR (-ENODEV ); + } + + pos += parsed; + + }while (pci_devs_to_skip[pos]);Is there a possibility of
pci_devs_to_skip+ pos >sizeof(pci_devs_to_skip)and also a valid memory ?
Quoted 8 lines
Quoted 4 lines
pos should point to the last parsed position. If parsing fails pos does not get updated and the code breaks out of the loop. If parsing is success pos should point to either the start of next element of '\0' (end of string). To me this is good enough.
The point is, hypothetically the address to
pci_devs_to_skip+ pos can be valid address (later to '\0'), and thus there is a possibility, that parsing may not fail.
Have you found an example how at any given point in time pci_devs_to_skip + pos can point outside of user provided string?
Quoted 3 lines
Another, there is also a possibility of sscanf faulting accessing the illegal address, if pci_devs_to_skip[pos] turns out to be not NULL or valid address.
That depends on pci_devs_to_skip + pos can point to an invalid address in the first place, so that goes back to the question above.
Quoted 17 lines
Quoted 7 lines
Quoted 5 lines
I would recommend to have a check of size as well before accessing the array content, just to be safer accessing any memory.
What check do you have in mind?
Something like,
size_tlen =strlen(pci_devs_to_skip);
do {len -= parsed; } while (len);OR
Quoted 7 lines
do {
...pos += parsed; } while (pos < len);Further, I'm also fine with the existing code, if you think this won't break and already been taken care. Thanks.
But in the loop somewhere you will still need to parse pci_devs_to_skip
+ some_offset. The new code structure does not remove that, right?
Given this is for debugging and is supposed to be temporary, I think the code is good enough. But I want to make sure if there is anything I missed.
Wei.
Quoted 4 lines
Regards,
~Praveen.
Wei Liu
maintainerRFC 7/8 mshv: implement in-kernel device framework
In reply to Wei LiuThis is basically the same code adopted from KVM. The main user case is the future MSHV-VFIO bridge device. We don't have any plan to support in-kernel device emulation yet, but it wouldn't hurt to make the code more flexible.
--- Documentation/virt/mshv/api.rst | 12 +++ drivers/hv/mshv_main.c | 181 ++++++++++++++++++++++++++++++++ include/linux/mshv.h | 57 ++++++++++ include/uapi/linux/mshv.h | 36 +++++++ 4 files changed, 286 insertions(+)
Documentation/virt/mshv/api.rst +12 -0
@@ -170,4 +170,16 @@ Can be used to get/set various properties of a partition. Some properties can only be set at partition creation. For these, there are parameters in MSHV_CREATE_PARTITION. +3.12 MSHV_CREATE_DEVICE +----------------------- +:Type: partition ioctl +:Parameters: struct mshv_create_device +:Returns: 0 on success + +Can be used to create an in-kernel device. + +If the MSHV_CREATE_DEVICE_TEST flag is set, only test whether the +device type is supported (not necessarily whether it can be created +in the current vm). +Currently only supports VFIO type device.
drivers/hv/mshv_main.c +181 -0
223 lines not on this page · raw message
include/linux/mshv.h +57 -0
67 lines not on this page · raw message
include/uapi/linux/mshv.h +36 -0
@@ -158,6 +158,14 @@ struct mshv_msi_routing {#define MSHV_SET_VP_STATE _IOWR(MSHV_IOCTL, 0x0B, struct mshv_vp_state)#define MSHV_TRANSLATE_GVA _IOWR(MSHV_IOCTL, 0x0E, struct mshv_translate_gva) +/* ioctl for device fd */ +#define MSHV_CREATE_DEVICE _IOWR(MSHV_IOCTL, 0x13, struct mshv_create_device) + +/* ioctls for fds returned by MSHV_CREATE_DEVICE */ +#define MSHV_SET_DEVICE_ATTR _IOW(MSHV_IOCTL, 0x14, struct mshv_device_attr) +#define MSHV_GET_DEVICE_ATTR _IOW(MSHV_IOCTL, 0x15, struct mshv_device_attr) +#define MSHV_HAS_DEVICE_ATTR _IOW(MSHV_IOCTL, 0x16, struct mshv_device_attr) +/* register page mapping example: * struct hv_vp_register_page *regs = mmap(NULL, * 4096,
@@ -184,4 +192,32 @@ union mshv_partition_property_page_access_tracking_config {__u64 as_uint64; } __packed; +/* +* Device control API. +*/ +#define MSHV_CREATE_DEVICE_TEST 1 + +struct mshv_create_device { +__u32 type;/* in: MSHV_DEV_TYPE_xxx */ +__u32 fd;/* out: device handle */ +__u32 flags;/* in: MSHV_CREATE_DEVICE_xxx */ +}; + +#define MSHV_DEV_VFIO_GROUP 1 +#define MSHV_DEV_VFIO_GROUP_ADD 1 +#define MSHV_DEV_VFIO_GROUP_DEL 2 + +enum mshv_device_type { +MSHV_DEV_TYPE_VFIO , +#define MSHV_DEV_TYPE_VFIO MSHV_DEV_TYPE_VFIO + MSHV_DEV_TYPE_MAX, +}; + +struct mshv_device_attr { +__u32 flags;/* no flags currently defined */ +__u32 group;/* device-defined */ +__u64 attr;/* group-defined */ +__u64 addr;/* userspace address of attr data */ +}; +#endif
Signature · 1 line
2.30.2
Re: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 4 lines · Wei Liu
drivers/hv/mshv_main.c +4
-0+static long +mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *user_args) +{
[...]
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1060
Quoted 11 lines
drivers/hv/mshv_main.c +11
-0+mshv_partition_get (partition); + r =anon_inode_getfd (ops->name, &mshv_device_fops, dev,O_RDWR |O_CLOEXEC ); +if (r <0 ) { +mshv_partition_put_no_destroy (partition); +list_del (&dev->partition_node); + ops->destroy (dev); + goto out; + } + + cd->fd = r; + r =0 ;
Why return the fd in memory instead of returning the fd as the return value from the ioctl?
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1065
Quoted 4 lines
drivers/hv/mshv_main.c +4
-0+if (copy_to_user (user_args, &tmp,sizeof (tmp))) { + r = -EFAULT ; + goto out; + }
... this could then disappear.
Wei Liu
maintainerRe: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 21 lines · Matthew Wilcox
Quoted 4 lines · Wei Liu
drivers/hv/mshv_main.c +4
-0+static long +mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *user_args) +{[...]
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1060
Quoted 11 lines
drivers/hv/mshv_main.c +11
-0+mshv_partition_get (partition); + r =anon_inode_getfd (ops->name, &mshv_device_fops, dev,O_RDWR |O_CLOEXEC ); +if (r <0 ) { +mshv_partition_put_no_destroy (partition); +list_del (&dev->partition_node); + ops->destroy (dev); + goto out; + } + + cd->fd = r; + r =0 ;Why return the fd in memory instead of returning the fd as the return value from the ioctl?
Quoted 6 lines
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1065
Quoted 4 lines
drivers/hv/mshv_main.c +4
-0+if (copy_to_user (user_args, &tmp,sizeof (tmp))) { + r = -EFAULT ; + goto out; + }... this could then disappear.
Thanks for your comment, Matthew.
This is intentionally because I didn't want to deviate from KVM's API. The fewer differences the better.
Wei.
Quoted 1 line
Re: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 29 lines · Wei Liu
Quoted 21 lines · Matthew Wilcox
Quoted 4 lines · Wei Liu
drivers/hv/mshv_main.c +4
-0+static long +mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *user_args) +{[...]
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1060
Quoted 11 lines
drivers/hv/mshv_main.c +11
-0+mshv_partition_get (partition); + r =anon_inode_getfd (ops->name, &mshv_device_fops, dev,O_RDWR |O_CLOEXEC ); +if (r <0 ) { +mshv_partition_put_no_destroy (partition); +list_del (&dev->partition_node); + ops->destroy (dev); + goto out; + } + + cd->fd = r; + r =0 ;Why return the fd in memory instead of returning the fd as the return value from the ioctl?
Quoted 6 lines
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1065
Quoted 4 lines
drivers/hv/mshv_main.c +4
-0+if (copy_to_user (user_args, &tmp,sizeof (tmp))) { + r = -EFAULT ; + goto out; + }... this could then disappear.
Quoted 4 lines
Thanks for your comment, Matthew.
This is intentionally because I didn't want to deviate from KVM's API. The fewer differences the better.
Then don't define your own structure. Use theirs.
Wei Liu
maintainerRe: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 30 lines · Matthew Wilcox
Quoted 29 lines · Wei Liu
Quoted 27 lines · Matthew Wilcox
Quoted 4 lines · Wei Liu
drivers/hv/mshv_main.c +4
-0+static long +mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *user_args) +{[...]
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1060
Quoted 11 lines
drivers/hv/mshv_main.c +11
-0+mshv_partition_get (partition); + r =anon_inode_getfd (ops->name, &mshv_device_fops, dev,O_RDWR |O_CLOEXEC ); +if (r <0 ) { +mshv_partition_put_no_destroy (partition); +list_del (&dev->partition_node); + ops->destroy (dev); + goto out; + } + + cd->fd = r; + r =0 ;Why return the fd in memory instead of returning the fd as the return value from the ioctl?
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1065
Quoted 4 lines
drivers/hv/mshv_main.c +4
-0+if (copy_to_user (user_args, &tmp,sizeof (tmp))) { + r = -EFAULT ; + goto out; + }... this could then disappear.
Quoted 6 lines
Quoted 4 lines
Thanks for your comment, Matthew.
This is intentionally because I didn't want to deviate from KVM's API. The fewer differences the better.
Then don't define your own structure. Use theirs.
I specifically mentioned in the cover letter I didn't do it because I was not sure if that would be acceptable. I guess I will find out.
Wei.
Re: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 4 lines · Wei Liu
Quoted 1 line
Then don't define your own structure. Use theirs.
I specifically mentioned in the cover letter I didn't do it because I was not sure if that would be acceptable. I guess I will find out.
I only got patch 7/8. You can't blame me for not reading 0/8 if you didn't send me 0/8.
Wei Liu
maintainerRe: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 8 lines · Matthew Wilcox
Quoted 4 lines · Wei Liu
Quoted 1 line
Then don't define your own structure. Use theirs.
I specifically mentioned in the cover letter I didn't do it because I was not sure if that would be acceptable. I guess I will find out.
I only got patch 7/8. You can't blame me for not reading 0/8 if you didn't send me 0/8.
To be clear I did not blame you.
You were not CC'ed on this patch, so presumably you got it via one of the mailing lists. I'm not sure why you only got this one patch. Perhaps if you wait a bit you will get the rest.
But it is all good. I appreciate the fact that you took the time to go over this patch and voiced your opinion.
Wei.
Re: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 3 lines · Wei Liu
You were not CC'ed on this patch, so presumably you got it via one of the mailing lists. I'm not sure why you only got this one patch. Perhaps if you wait a bit you will get the rest.
No, I won't. You only cc'd linux-doc on this one patch and not on any of the others.
Wei Liu
maintainerRe: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 7 lines · Matthew Wilcox
Quoted 3 lines · Wei Liu
You were not CC'ed on this patch, so presumably you got it via one of the mailing lists. I'm not sure why you only got this one patch. Perhaps if you wait a bit you will get the rest.
No, I won't. You only cc'd linux-doc on this one patch and not on any of the others.
I see. That's because get_maintainers.pl only detected changes to the doc in this file.
If you're interested in having a look at the other patches, they can be fetched via at the following url.
https://lore.kernel.org/linux-hyperv/[email protected]/
Thanks for your attention.
Wei.
Re: [RFC v1 7/8] mshv: implement in-kernel device framework
In reply to Wei LiuQuoted 83 lines · Wei Liu
This is basically the same code adopted from KVM. The main user case is the future MSHV-VFIO bridge device. We don't have any plan to support in-kernel device emulation yet, but it wouldn't hurt to make the code more flexible.
Signed-off-by Wei Liu <[email protected]>--- Documentation/virt/mshv/api.rst | 12 +++ drivers/hv/mshv_main.c | 181 ++++++++++++++++++++++++++++++++ include/linux/mshv.h | 57 ++++++++++ include/uapi/linux/mshv.h | 36 +++++++ 4 files changed, 286 insertions(+)Documentation/virt/mshv/api.rst +12
-0@@ -170,4 +170,16 @@ Can be used to get/set various properties of a partition. Some properties can only be set at partition creation. For these, there are parameters in MSHV_CREATE_PARTITION. +3.12 MSHV_CREATE_DEVICE +----------------------- +:Type: partition ioctl +:Parameters: struct mshv_create_device +:Returns: 0 on success + +Can be used to create an in-kernel device. + +If the MSHV_CREATE_DEVICE_TEST flag is set, only test whether the +device type is supported (not necessarily whether it can be created +in the current vm). +Currently only supports VFIO type device.drivers/hv/mshv_main.c +27
-0@@ -20,6 +20,8 @@#include <linux/random.h> #include <linux/mshv.h> #include <linux/mshv_eventfd.h> +#include <linux/hyperv.h> +#include <linux/nospec.h> #include <asm/mshyperv.h> #include "mshv.h" @@ -33,6 +35,7 @@ static int mshv_vp_release(struct inode *inode, struct file *filp);static long mshv_vp_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg);static struct mshv_partition *mshv_partition_get (struct mshv_partition *partition);static void mshv_partition_put (struct mshv_partition *partition); +static void mshv_partition_put_no_destroy (struct mshv_partition *partition);static int mshv_partition_release (struct inode *inode,struct file *filp);static long mshv_partition_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg);static int mshv_dev_open (struct inode *inode,struct file *filp);@@ -912,6 +915,172 @@ mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition,return ret; } +static int mshv_device_ioctl_attr (struct mshv_device *dev, +int (*accessor)(struct mshv_device *dev, +struct mshv_device_attr *attr), +unsigned long arg) +{ +struct mshv_device_attr attr; + +if (!accessor) +return -EPERM ; + +if (copy_from_user (&attr, (void __user *)arg,sizeof (attr))) +return -EFAULT ; + +return accessor (dev, &attr); +} + +static long mshv_device_ioctl (struct file *filp,unsigned int ioctl, +unsigned long arg) +{ +struct mshv_device *dev = filp->private_data; + +switch (ioctl) { +case MSHV_SET_DEVICE_ATTR : +return mshv_device_ioctl_attr (dev, dev->ops->set_attr, arg);
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 951
Quoted 10 lines
drivers/hv/mshv_main.c +10
-0+case MSHV_GET_DEVICE_ATTR : +return mshv_device_ioctl_attr (dev, dev->ops->get_attr, arg); +case MSHV_HAS_DEVICE_ATTR : +return mshv_device_ioctl_attr (dev, dev->ops->has_attr, arg); +default : +if (dev->ops->ioctl) +return dev->ops->ioctl (dev, ioctl, arg); + +return -ENOTTY ; + }
Have seen some static analyzer tool cribbing here of not returning any error. If you feel OK, please move the 'return -ENOTTY' down after switch block. Thanks.
Quoted 94 lines
drivers/hv/mshv_main.c +94
-0+} + +static int mshv_device_release (struct inode *inode,struct file *filp) +{ +struct mshv_device *dev = filp->private_data; +struct mshv_partition *partition = dev->partition; + +if (dev->ops->release) { +mutex_lock (&partition->mutex); +list_del (&dev->partition_node); + dev->ops->release (dev); +mutex_unlock (&partition->mutex); + } + +mshv_partition_put (partition); +return 0 ; +} + +static const struct file_operations mshv_device_fops = { + .unlocked_ioctl = mshv_device_ioctl, + .release = mshv_device_release, +}; + +static const struct mshv_device_ops *mshv_device_ops_table[MSHV_DEV_TYPE_MAX ]; + +int mshv_register_device_ops (const struct mshv_device_ops *ops,u32 type) +{ +if (type >=ARRAY_SIZE (mshv_device_ops_table)) +return -ENOSPC ; + +if (mshv_device_ops_table[type] !=NULL ) +return -EEXIST ; + + mshv_device_ops_table[type] = ops; +return 0 ; +} + +void mshv_unregister_device_ops (u32 type) +{ +if (type >=ARRAY_SIZE (mshv_device_ops_table)) +return ; + mshv_device_ops_table[type] =NULL ; +} + +static long +mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *user_args) +{ +long r; +struct mshv_create_device tmp, *cd; +struct mshv_device *dev; +const struct mshv_device_ops *ops; +int type; + +if (copy_from_user (&tmp, user_args,sizeof (tmp))) { + r = -EFAULT ; + goto out; + } + + cd = &tmp; + +if (cd->type >=ARRAY_SIZE (mshv_device_ops_table)) { + r = -ENODEV ; + goto out; + } + + type =array_index_nospec (cd->type,ARRAY_SIZE (mshv_device_ops_table)); + ops = mshv_device_ops_table[type]; +if (ops ==NULL ) { + r = -ENODEV ; + goto out; + } + +if (cd->flags &MSHV_CREATE_DEVICE_TEST ) { + r =0 ; + goto out; + } + + dev =kzalloc (sizeof (*dev),GFP_KERNEL_ACCOUNT ); +if (!dev) { + r = -ENOMEM ; + goto out; + } + + dev->ops = ops; + dev->partition = partition; + + r = ops->create (dev, type); +if (r <0 ) { +kfree (dev); + goto out; + } + +list_add (&dev->partition_node, &partition->devices);
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1055
Quoted 10 lines
drivers/hv/mshv_main.c +10
-0+ +if (ops->init) + ops->init (dev); + +mshv_partition_get (partition); + r =anon_inode_getfd (ops->name, &mshv_device_fops, dev,O_RDWR |O_CLOEXEC ); +if (r <0 ) { +mshv_partition_put_no_destroy (partition); +list_del (&dev->partition_node); + ops->destroy (dev);
I hope ops->destroy will free dev as well ?
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1063
Quoted 8 lines
drivers/hv/mshv_main.c +8
-0+ goto out; + } + + cd->fd = r; + r =0 ; + +if (copy_to_user (user_args, &tmp,sizeof (tmp))) { + r = -EFAULT ;
I don't think we will be cleaning up anything ? Or do we need to?
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1069
Quoted 6 lines
drivers/hv/mshv_main.c +6
-0+ goto out; + } +out: +return r; +} +
Regards,
~Praveen.
Wei Liu
maintainerRe: [RFC v1 7/8] mshv: implement in-kernel device framework
Quoted 1 line · Praveen Kumar
On 09-07-2021 17:13, Wei Liu wrote:
[...]
Quoted 13 lines
Quoted 13 lines
drivers/hv/mshv_main.c +13
-0+static long mshv_device_ioctl (struct file *filp,unsigned int ioctl, +unsigned long arg) +{ +struct mshv_device *dev = filp->private_data; + +switch (ioctl) { +case MSHV_SET_DEVICE_ATTR : +return mshv_device_ioctl_attr (dev, dev->ops->set_attr, arg); +case MSHV_GET_DEVICE_ATTR : +return mshv_device_ioctl_attr (dev, dev->ops->get_attr, arg); +case MSHV_HAS_DEVICE_ATTR : +return mshv_device_ioctl_attr (dev, dev->ops->has_attr, arg); +default :
Quoted 9 lines
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 951
Quoted 5 lines
drivers/hv/mshv_main.c +5
-0+if (dev->ops->ioctl) +return dev->ops->ioctl (dev, ioctl, arg); + +return -ENOTTY ; + }Have seen some static analyzer tool cribbing here of not returning any error. If you feel OK, please move the 'return
-ENOTTY' down after switch block. Thanks.
Fair point. I will make the change.
Quoted 2 lines
Quoted 2 lines
(quoted diff) +2
-0+} +
[...]
Quoted 54 lines
Quoted 54 lines
drivers/hv/mshv_main.c +54
-0+static long +mshv_partition_ioctl_create_device (struct mshv_partition *partition, +void __user *user_args) +{ +long r; +struct mshv_create_device tmp, *cd; +struct mshv_device *dev; +const struct mshv_device_ops *ops; +int type; + +if (copy_from_user (&tmp, user_args,sizeof (tmp))) { + r = -EFAULT ; + goto out; + } + + cd = &tmp; + +if (cd->type >=ARRAY_SIZE (mshv_device_ops_table)) { + r = -ENODEV ; + goto out; + } + + type =array_index_nospec (cd->type,ARRAY_SIZE (mshv_device_ops_table)); + ops = mshv_device_ops_table[type]; +if (ops ==NULL ) { + r = -ENODEV ; + goto out; + } + +if (cd->flags &MSHV_CREATE_DEVICE_TEST ) { + r =0 ; + goto out; + } + + dev =kzalloc (sizeof (*dev),GFP_KERNEL_ACCOUNT ); +if (!dev) { + r = -ENOMEM ; + goto out; + } + + dev->ops = ops; + dev->partition = partition; + + r = ops->create (dev, type); +if (r <0 ) { +kfree (dev); + goto out; + } + +list_add (&dev->partition_node, &partition->devices); + +if (ops->init) + ops->init (dev); +
Quoted 8 lines
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1055
Quoted 6 lines
drivers/hv/mshv_main.c +6
-0+mshv_partition_get (partition); + r =anon_inode_getfd (ops->name, &mshv_device_fops, dev,O_RDWR |O_CLOEXEC ); +if (r <0 ) { +mshv_partition_put_no_destroy (partition); +list_del (&dev->partition_node); + ops->destroy (dev);I hope
ops->destroywill free dev as well ?
Yes. It is clearly written in the preceding comment of that hook. I hope that's prominent enough.
Quoted 11 lines
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1063
Quoted 9 lines
drivers/hv/mshv_main.c +8
-0+ goto out; + } + + cd->fd = r; + r =0 ; + +if (copy_to_user (user_args, &tmp,sizeof (tmp))) { + r = -EFAULT ;I don't think we will be cleaning up anything ? Or do we need to?
No need. Whatever residuals left will be cleaned up once the VM is destroyed.
Wei.
Quoted 10 lines
drivers/hv/mshv_main.c mshv_partition_ioctl_set_msi_routing(struct mshv_partition *partition, · line 1069
Quoted 6 lines
drivers/hv/mshv_main.c +6
-0+ goto out; + } +out: +return r; +} +Regards,
~Praveen.
Wei Liu
maintainerRFC 8/8 mshv: add vfio bridge device
In reply to Wei LiuThe main purpose of this device at this stage is to hold a reference to the vfio_group to avoid it disappearing under our feet.
---
Is there value in unifying with KVM?
--- drivers/hv/Kconfig | 4 + drivers/hv/Makefile | 2 +- drivers/hv/mshv_main.c | 5 + drivers/hv/vfio.c | 244 +++++++++++++++++++++++++++++++++++++++++ drivers/hv/vfio.h | 18 +++ 5 files changed, 272 insertions(+), 1 deletion(-) create mode 100644 drivers/hv/vfio.c create mode 100644 drivers/hv/vfio.h
drivers/hv/Kconfig +4 -0
@@ -2,6 +2,9 @@ menu "Microsoft Hyper-V guest support" +config MSHV_VFIO + bool + config HYPERV tristate "Microsoft Hyper-V client drivers" depends on X86 && ACPI && X86_LOCAL_APIC && HYPERVISOR_GUEST
@@ -31,6 +34,7 @@ config HYPERV_ROOT_API tristate "Microsoft Hypervisor root partition interfaces: /dev/mshv" depends on HYPERV select EVENTFD + select MSHV_VFIO help Provides access to interfaces for managing guest virtual machines running under the Microsoft Hypervisor.
drivers/hv/Makefile +1 -1
@@ -14,4 +14,4 @@ hv_vmbus-$(CONFIG_HYPERV_TESTING) += hv_debugfs.o hv_utils-y := hv_util.o hv_kvp.o hv_snapshot.o hv_fcopy.o hv_utils_transport.o mshv-y += mshv_main.o hv_call.o hv_synic.o hv_portid_table.o \ - hv_eventfd.o mshv_msi.o + hv_eventfd.o mshv_msi.o vfio.o
drivers/hv/mshv_main.c +5 -0
@@ -25,6 +25,7 @@#include <asm/mshyperv.h> #include "mshv.h" +#include "vfio.h" MODULE_AUTHOR ("Microsoft" );MODULE_LICENSE ("GPL" );
@@ -1439,6 +1440,8 @@ __init mshv_init(void)if (mshv_irqfd_wq_init ())mshv_irqfd_wq_cleanup (); +mshv_vfio_ops_init (); +return 0 ; }
@@ -1452,6 +1455,8 @@ __exit mshv_exit(void)hv_port_table_fini (); +mshv_vfio_ops_exit (); +misc_deregister (&mshv_dev); }
drivers/hv/vfio.c new file +244 -0
244 lines not on this page · raw message
drivers/hv/vfio.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __MSHV_VFIO_H +#define __MSHV_VFIO_H + +#ifdef CONFIG_MSHV_VFIO +int mshv_vfio_ops_init (void ); +void mshv_vfio_ops_exit (void ); +#else +static inline int mshv_vfio_ops_init (void ) +{ +return 0 ; +} +static inline void mshv_vfio_ops_exit (void ) +{ +} +#endif + +#endif
Signature · 1 line
2.30.2
Re: [RFC v1 8/8] mshv: add vfio bridge device
Quoted 47 lines · Wei Liu
drivers/hv/vfio.c new file +47
-0+ +static int mshv_vfio_set_group (struct mshv_device *dev,long attr,u64 arg) +{ +struct mshv_vfio *mv = dev->private; +struct vfio_group *vfio_group; +struct mshv_vfio_group *mvg; +int32_t __user *argp = (int32_t __user *)(unsigned long )arg; +struct fd f; +int32_t fd; +int ret; + +switch (attr) { +case MSHV_DEV_VFIO_GROUP_ADD : +if (get_user (fd, argp)) +return -EFAULT ; + + f =fdget (fd); +if (!f.file) +return -EBADF ; + + vfio_group =mshv_vfio_group_get_external_user (f.file); +fdput (f); + +if (IS_ERR (vfio_group)) +return PTR_ERR (vfio_group); + +mutex_lock (&mv->lock); + +list_for_each_entry (mvg, &mv->group_list, node) { +if (mvg->vfio_group == vfio_group) { +mutex_unlock (&mv->lock); +mshv_vfio_group_put_external_user (vfio_group); +return -EEXIST ; + } + } + + mvg =kzalloc (sizeof (*mvg),GFP_KERNEL_ACCOUNT ); +if (!mvg) { +mutex_unlock (&mv->lock); +mshv_vfio_group_put_external_user (vfio_group); +return -ENOMEM ; + } + +list_add_tail (&mvg->node, &mv->group_list); + mvg->vfio_group = vfio_group; + +mutex_unlock (&mv->lock);
drivers/hv/vfio.c line 133
Quoted 10 lines
drivers/hv/vfio.c new file +10
-0+ +return 0 ; + +case MSHV_DEV_VFIO_GROUP_DEL : +if (get_user (fd, argp)) +return -EFAULT ; + + f =fdget (fd); +if (!f.file) +return -EBADF ;
Can we move these both checks above switch statement and do fdput accordingly under both case statement accordingly?
Quoted 40 lines
drivers/hv/vfio.c new file +40
-0+ + ret = -ENOENT ; + +mutex_lock (&mv->lock); + +list_for_each_entry (mvg, &mv->group_list, node) { +if (!mshv_vfio_external_group_match_file (mvg->vfio_group, + f.file)) +continue ; + +list_del (&mvg->node); +mshv_vfio_group_put_external_user (mvg->vfio_group); +kfree (mvg); + ret =0 ; +break ; + } + +mutex_unlock (&mv->lock); + +fdput (f); + +return ret; + } + +return -ENXIO ; +} + +static int mshv_vfio_set_attr (struct mshv_device *dev, +struct mshv_device_attr *attr) +{ +switch (attr->group) { +case MSHV_DEV_VFIO_GROUP : +return mshv_vfio_set_group (dev, attr->attr, attr->addr); + } + +return -ENXIO ; +} + +static int mshv_vfio_has_attr (struct mshv_device *dev, +struct mshv_device_attr *attr)
drivers/hv/vfio.c line 183
Quoted 10 lines
drivers/hv/vfio.c new file +10
-0+{ +switch (attr->group) { +case MSHV_DEV_VFIO_GROUP : +switch (attr->attr) { +case MSHV_DEV_VFIO_GROUP_ADD : +case MSHV_DEV_VFIO_GROUP_DEL : +return 0 ; + } + +break ;
do we need this break statement ? If not, lets remove it.
Quoted 8 lines
drivers/hv/vfio.c new file +8
-0+ } + +return -ENXIO ; +} + +static void mshv_vfio_destroy (struct mshv_device *dev) +{ +struct mshv_vfio *mv = dev->private;
drivers/hv/vfio.c line 201
Quoted 10 lines
drivers/hv/vfio.c new file +10
-0+struct mshv_vfio_group *mvg, *tmp; + +list_for_each_entry_safe (mvg, tmp, &mv->group_list, node) { +mshv_vfio_group_put_external_user (mvg->vfio_group); +list_del (&mvg->node); +kfree (mvg); + } + +kfree (mv); +kfree (dev);
We are freeing up dev. Please ignore my comment in caller patch. Thanks.
drivers/hv/vfio.c line 212
Quoted 11 lines
drivers/hv/vfio.c new file +11
-0+} + +static int mshv_vfio_create (struct mshv_device *dev,u32 type); + +static struct mshv_device_ops mshv_vfio_ops = { + .name ="mshv-vfio" , + .create = mshv_vfio_create, + .destroy = mshv_vfio_destroy, + .set_attr = mshv_vfio_set_attr, + .has_attr = mshv_vfio_has_attr, +};
Regards,
~Praveen.
Wei Liu
maintainerRe: [RFC v1 8/8] mshv: add vfio bridge device
Quoted 53 lines · Praveen Kumar
Quoted 52 lines · Wei Liu
drivers/hv/vfio.c new file +52
-0+ +static int mshv_vfio_set_group (struct mshv_device *dev,long attr,u64 arg) +{ +struct mshv_vfio *mv = dev->private; +struct vfio_group *vfio_group; +struct mshv_vfio_group *mvg; +int32_t __user *argp = (int32_t __user *)(unsigned long )arg; +struct fd f; +int32_t fd; +int ret; + +switch (attr) { +case MSHV_DEV_VFIO_GROUP_ADD : +if (get_user (fd, argp)) +return -EFAULT ; + + f =fdget (fd); +if (!f.file) +return -EBADF ; + + vfio_group =mshv_vfio_group_get_external_user (f.file); +fdput (f); + +if (IS_ERR (vfio_group)) +return PTR_ERR (vfio_group); + +mutex_lock (&mv->lock); + +list_for_each_entry (mvg, &mv->group_list, node) { +if (mvg->vfio_group == vfio_group) { +mutex_unlock (&mv->lock); +mshv_vfio_group_put_external_user (vfio_group); +return -EEXIST ; + } + } + + mvg =kzalloc (sizeof (*mvg),GFP_KERNEL_ACCOUNT ); +if (!mvg) { +mutex_unlock (&mv->lock); +mshv_vfio_group_put_external_user (vfio_group); +return -ENOMEM ; + } + +list_add_tail (&mvg->node, &mv->group_list); + mvg->vfio_group = vfio_group; + +mutex_unlock (&mv->lock); + +return 0 ; + +case MSHV_DEV_VFIO_GROUP_DEL : +if (get_user (fd, argp))
Quoted 8 lines
drivers/hv/vfio.c line 95
Quoted 5 lines
drivers/hv/vfio.c new file +5
-0+return -EFAULT ; + + f =fdget (fd); +if (!f.file) +return -EBADF ;Can we move these both checks above switch statement and do fdput accordingly under both case statement accordingly?
Fair point. This can be done, albeit at the cost of having a rather different code structure.
I was waiting to see if we should somehow merge this with KVM's implementation so the code was deliberately kept close. If there is no further comment I can of course make the change you suggested.
Quoted 45 lines
Quoted 45 lines
drivers/hv/vfio.c new file +44
-0+ + ret = -ENOENT ; + +mutex_lock (&mv->lock); + +list_for_each_entry (mvg, &mv->group_list, node) { +if (!mshv_vfio_external_group_match_file (mvg->vfio_group, + f.file)) +continue ; + +list_del (&mvg->node); +mshv_vfio_group_put_external_user (mvg->vfio_group); +kfree (mvg); + ret =0 ; +break ; + } + +mutex_unlock (&mv->lock); + +fdput (f); + +return ret; + } + +return -ENXIO ; +} + +static int mshv_vfio_set_attr (struct mshv_device *dev, +struct mshv_device_attr *attr) +{ +switch (attr->group) { +case MSHV_DEV_VFIO_GROUP : +return mshv_vfio_set_group (dev, attr->attr, attr->addr); + } + +return -ENXIO ; +} + +static int mshv_vfio_has_attr (struct mshv_device *dev, +struct mshv_device_attr *attr) +{ +switch (attr->group) { +case MSHV_DEV_VFIO_GROUP : +switch (attr->attr) {
Quoted 8 lines
drivers/hv/vfio.c line 183
Quoted 6 lines
drivers/hv/vfio.c new file +6
-0+case MSHV_DEV_VFIO_GROUP_ADD : +case MSHV_DEV_VFIO_GROUP_DEL : +return 0 ; + } + +break ;do we need this break statement ? If not, lets remove it.
Will do.
Wei.