x86/Hyper-V: Add Hyper-V Isolation VM support
Subsystems (13)
v2 cover x86/Hyper-V: Add Hyper-V Isolation VM support
From: Tianyu Lan <[email protected]>
Hyper-V provides two kinds of Isolation VMs. VBS(Virtualization-based security) and AMD SEV-SNP unenlightened Isolation VMs. This patchset is to add support for these Isolation VM support in Linux.
The memory of these vms are encrypted and host can't access guest memory directly. Hyper-V provides new host visibility hvcall and the guest needs to call new hvcall to mark memory visible to host before sharing memory with host. For security, all network/storage stack memory should not be shared with host and so there is bounce buffer requests.
Vmbus channel ring buffer already plays bounce buffer role because all data from/to host needs to copy from/to between the ring buffer and IO stack memory. So mark vmbus channel ring buffer visible to host.
There are two exceptions - packets sent by vmbus_sendpacket_ pagebuffer() and vmbus_sendpacket_mpb_desc(). These packets contains IO stack memory address and host will access these memory. So add Hyper-V DMA Ops and use DMA API in the netvsc and storvsc drivers to allocate bounce buffer via swiotlb interface.
For SNP isolation VM, guest needs to access the shared memory via extra address space which is specified by Hyper-V CPUID HYPERV_CPUID_ ISOLATION_CONFIG. The access physical address of the shared memory should be bounce buffer memory GPA plus with shared_gpa_boundary reported by CPUID.
Change since v1:
* Add DMA API support in the netvsc and storvsc driver.
* Add Hyper-V DMA ops.
* Add static branch for the check of isolation type snp.
* Fix some code style comments.
Tianyu Lan (12):
x86/HV: Initialize GHCB page in Isolation VM
x86/HV: Initialize shared memory boundary in Isolation VM x86/Hyper-V: Add new hvcall guest address host visibility support
HV: Add Write/Read MSR registers via ghcb
HV: Add ghcb hvcall support for SNP VM
HV/Vmbus: Add SNP support for VMbus channel initiate message HV/Vmbus: Initialize VMbus ring buffer for Isolation VM UIO/Hyper-V: Not load UIO HV driver in the isolation VM.
swiotlb: Add bounce buffer remap address setting function HV/IOMMU: Add Hyper-V dma ops support
HV/Netvsc: Add Isolation VM support for netvsc driver HV/Storvsc: Add Isolation VM support for storvsc driver
arch/x86/hyperv/Makefile | 2 +- arch/x86/hyperv/hv_init.c | 70 +++++-- arch/x86/hyperv/ivm.c | 289 +++++++++++++++++++++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 22 +++ arch/x86/include/asm/mshyperv.h | 90 +++++++-- arch/x86/kernel/cpu/mshyperv.c | 5 + arch/x86/kernel/pci-swiotlb.c | 3 +- drivers/hv/channel.c | 44 ++++- drivers/hv/connection.c | 68 ++++++- drivers/hv/hv.c | 73 ++++++-- drivers/hv/hyperv_vmbus.h | 3 + drivers/hv/ring_buffer.c | 83 ++++++--- drivers/hv/vmbus_drv.c | 3 + drivers/iommu/hyperv-iommu.c | 127 +++++++++++++ drivers/net/hyperv/hyperv_net.h | 11 ++ drivers/net/hyperv/netvsc.c | 137 +++++++++++++- drivers/net/hyperv/rndis_filter.c | 3 + drivers/scsi/storvsc_drv.c | 67 ++++++- drivers/uio/uio_hv_generic.c | 5 + include/asm-generic/hyperv-tlfs.h | 1 + include/asm-generic/mshyperv.h | 18 +- include/linux/hyperv.h | 12 +- include/linux/swiotlb.h | 5 + kernel/dma/swiotlb.c | 13 +- mm/ioremap.c | 1 + mm/vmalloc.c | 1 + 26 files changed, 1068 insertions(+), 88 deletions(-) create mode 100644 arch/x86/hyperv/ivm.c
Signature · 1 line
2.25.1
RFC PATCH v2 1/12 x86/HV: Initialize GHCB page in Isolation VM
From: Tianyu Lan <[email protected]>
Hyper-V exposes GHCB page via SEV ES GHCB MSR for SNP guest to communicate with hypervisor. Map GHCB page for all cpus to read/write MSR register and submit hvcall request via GHCB.
--- arch/x86/hyperv/hv_init.c | 52 +++++++++++++++++++++++++++++++--- include/asm-generic/mshyperv.h | 1 + 2 files changed, 49 insertions(+), 4 deletions(-)
arch/x86/hyperv/hv_init.c +48 -4
@@ -82,6 +82,9 @@ static int hv_cpu_init(unsigned int cpu)struct hv_vp_assist_page **hvp = &hv_vp_assist_page[smp_processor_id ()];void **input_arg;struct page *pg; +u64 ghcb_gpa; +void *ghcb_va; +void **ghcb_base;/* hv_cpu_init() can be called with IRQs disabled from hv_resume() */ pg =alloc_pages (irqs_disabled () ?GFP_ATOMIC :GFP_KERNEL , hv_root_partition ?1 :0 );
@@ -128,6 +131,17 @@ static int hv_cpu_init(unsigned int cpu)wrmsrl (HV_X64_MSR_VP_ASSIST_PAGE , val); } +if (ms_hyperv.ghcb_base) { +rdmsrl (MSR_AMD64_SEV_ES_GHCB , ghcb_gpa); + + ghcb_va =ioremap_cache (ghcb_gpa,HV_HYP_PAGE_SIZE ); +if (!ghcb_va) +return -ENOMEM ; + + ghcb_base = (void **)this_cpu_ptr (ms_hyperv.ghcb_base); + *ghcb_base = ghcb_va; + } +return 0 ; }
@@ -223,6 +237,7 @@ static int hv_cpu_die(unsigned int cpu)unsigned long flags;void **input_arg;void *pg; +void **ghcb_va =NULL ;local_irq_save (flags); input_arg = (void **)this_cpu_ptr (hyperv_pcpu_input_arg);
@@ -236,6 +251,13 @@ static int hv_cpu_die(unsigned int cpu) *output_arg =NULL ; } +if (ms_hyperv.ghcb_base) { + ghcb_va = (void **)this_cpu_ptr (ms_hyperv.ghcb_base); +if (*ghcb_va) +iounmap (*ghcb_va); + *ghcb_va =NULL ; + } +local_irq_restore (flags);free_pages ((unsigned long )pg, hv_root_partition ?1 :0 );
@@ -372,6 +394,9 @@ void __init hyperv_init(void)u64 guest_id, required_msrs;union hv_x64_msr_hypercall_contents hypercall_msr;int cpuhp, i; +u64 ghcb_gpa; +void *ghcb_va; +void **ghcb_base;if (x86_hyper_type !=X86_HYPER_MS_HYPERV )return ;
@@ -432,9 +457,24 @@ void __init hyperv_init(void)VMALLOC_END ,GFP_KERNEL ,PAGE_KERNEL_ROX ,VM_FLUSH_RESET_PERMS ,NUMA_NO_NODE , __builtin_return_address(0)); -if (hv_hypercall_pg ==NULL ) { -wrmsrl (HV_X64_MSR_GUEST_OS_ID ,0 ); - goto remove_cpuhp_state; +if (hv_hypercall_pg ==NULL ) + goto clean_guest_os_id; + +if (hv_isolation_type_snp ()) { + ms_hyperv.ghcb_base = alloc_percpu(void *); +if (!ms_hyperv.ghcb_base) + goto clean_guest_os_id; + +rdmsrl (MSR_AMD64_SEV_ES_GHCB , ghcb_gpa); + ghcb_va =ioremap_cache (ghcb_gpa,HV_HYP_PAGE_SIZE ); +if (!ghcb_va) { +free_percpu (ms_hyperv.ghcb_base); + ms_hyperv.ghcb_base =NULL ; + goto clean_guest_os_id; + } + + ghcb_base = (void **)this_cpu_ptr (ms_hyperv.ghcb_base); + *ghcb_base = ghcb_va; }rdmsrl (HV_X64_MSR_HYPERCALL , hypercall_msr.as_uint64);
@@ -499,7 +539,8 @@ void __init hyperv_init(void)return ; -remove_cpuhp_state: +clean_guest_os_id: +wrmsrl (HV_X64_MSR_GUEST_OS_ID ,0 );cpuhp_remove_state (cpuhp); free_vp_assist_page:kfree (hv_vp_assist_page);
@@ -528,6 +569,9 @@ void hyperv_cleanup(void)*/ hv_hypercall_pg =NULL ; +if (ms_hyperv.ghcb_base) +free_percpu (ms_hyperv.ghcb_base); +/* Reset the hypercall page */ hypercall_msr.as_uint64 =0 ;wrmsrl (HV_X64_MSR_HYPERCALL , hypercall_msr.as_uint64);
include/asm-generic/mshyperv.h +1 -0
@@ -35,6 +35,7 @@ struct ms_hyperv_info {u32 max_lp_index;u32 isolation_config_a;u32 isolation_config_b; +void __percpu **ghcb_base; };extern struct ms_hyperv_info ms_hyperv;
Signature · 1 line
2.25.1
RFC PATCH v2 3/12 x86/Hyper-V: Add new hvcall guest address host visibility support
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Add new hvcall guest address host visibility support. Mark vmbus ring buffer visible to host when create gpadl buffer and mark back to not visible when tear down gpadl buffer.
--- arch/x86/hyperv/Makefile | 2 +- arch/x86/hyperv/ivm.c | 90 ++++++++++++++++++++++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 22 ++++++++ arch/x86/include/asm/mshyperv.h | 2 + drivers/hv/channel.c | 34 ++++++++++- include/asm-generic/hyperv-tlfs.h | 1 + include/linux/hyperv.h | 12 +++- 7 files changed, 159 insertions(+), 4 deletions(-) create mode 100644 arch/x86/hyperv/ivm.c
arch/x86/hyperv/Makefile +1 -1
@@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -obj-y := hv_init.o mmu.o nested.o irqdomain.o +obj-y := hv_init.o mmu.o nested.o irqdomain.o ivm.o obj-$(CONFIG_X86_64) += hv_apic.o hv_proc.o ifdef CONFIG_X86_64
arch/x86/hyperv/ivm.c new file +90 -0
90 lines not on this page · raw message
arch/x86/include/asm/hyperv-tlfs.h +22 -0
@@ -236,6 +236,15 @@ enum hv_isolation_type {/* TSC invariant control */ #define HV_X64_MSR_TSC_INVARIANT_CONTROL 0x40000118 +/* Hyper-V GPA map flags */ +#define HV_MAP_GPA_PERMISSIONS_NONE 0x0 +#define HV_MAP_GPA_READABLE 0x1 +#define HV_MAP_GPA_WRITABLE 0x2 + +#define VMBUS_PAGE_VISIBLE_READ_ONLY HV_MAP_GPA_READABLE +#define VMBUS_PAGE_VISIBLE_READ_WRITE (HV_MAP_GPA_READABLE|HV_MAP_GPA_WRITABLE) +#define VMBUS_PAGE_NOT_VISIBLE HV_MAP_GPA_PERMISSIONS_NONE +/* * Declare the MSR used to setup pages used to communicate with the hypervisor. */
@@ -564,4 +573,17 @@ enum hv_interrupt_type {#include <asm-generic/hyperv-tlfs.h> +/* All input parameters should be in single page. */ +#define HV_MAX_MODIFY_GPA_REP_COUNT \ + ((PAGE_SIZE / sizeof(u64)) - 2) + +/* HvCallModifySparseGpaPageHostVisibility hypercall */ +struct hv_input_modify_sparse_gpa_page_host_visibility { +u64 partition_id; +u32 host_visibility:2 ; +u32 reserved0:30 ; +u32 reserved1; +u64 gpa_page_list[HV_MAX_MODIFY_GPA_REP_COUNT ]; +} __packed; +#endif
arch/x86/include/asm/mshyperv.h +2 -0
@@ -269,6 +269,8 @@ int hv_map_ioapic_interrupt(int ioapic_id, bool level, int vcpu, int vector, struct hv_interrupt_entry *entry);int hv_unmap_ioapic_interrupt (int ioapic_id,struct hv_interrupt_entry *entry); +int hv_set_mem_host_visibility (void *kbuffer,u32 size,u32 visibility); +int hv_mark_gpa_visibility (u16 count,const u64 pfn[],u32 visibility);#else /* CONFIG_HYPERV */ static inline void hyperv_init (void ) {}static inline void hyperv_setup_mmu_ops (void ) {}
drivers/hv/channel.c +32 -2
@@ -400,7 +400,7 @@ static int __vmbus_establish_gpadl(struct vmbus_channel *channel,struct list_head *curr;u32 next_gpadl_handle;unsigned long flags; -int ret =0 ; +int ret =0 , index; next_gpadl_handle = (atomic_inc_return (&vmbus_connection.next_gpadl_handle) -1 );
@@ -409,6 +409,13 @@ static int __vmbus_establish_gpadl(struct vmbus_channel *channel,if (ret)return ret; + ret =hv_set_mem_host_visibility (kbuffer, size, +VMBUS_PAGE_VISIBLE_READ_WRITE ); +if (ret) { +pr_warn ("Failed to set host visibility.\n" ); +return ret; + } +init_completion (&msginfo->waitevent); msginfo->waiting_channel = channel;
@@ -484,6 +491,16 @@ static int __vmbus_establish_gpadl(struct vmbus_channel *channel, }kfree (msginfo); + +if (type ==HV_GPADL_BUFFER ) + index =0 ; +else + index = channel->gpadl_range[1 ].gpadlhandle ?2 :1 ; + + channel->gpadl_range[index].size = size; + channel->gpadl_range[index].buffer = kbuffer; + channel->gpadl_range[index].gpadlhandle = *gpadl_handle; +return ret; }
@@ -743,7 +760,7 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle)struct vmbus_channel_gpadl_teardown *msg;struct vmbus_channel_msginfo *info;unsigned long flags; -int ret; +int ret, i; info =kzalloc (sizeof (*info) +sizeof (struct vmbus_channel_gpadl_teardown ),GFP_KERNEL );
@@ -791,6 +808,19 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle)spin_unlock_irqrestore (&vmbus_connection.channelmsg_lock, flags);kfree (info); + +/* Find gpadl buffer virtual address and size. */ +for (i =0 ; i <VMBUS_GPADL_RANGE_COUNT ; i++) +if (channel->gpadl_range[i].gpadlhandle == gpadl_handle) +break ; + +if (hv_set_mem_host_visibility (channel->gpadl_range[i].buffer, + channel->gpadl_range[i].size, +VMBUS_PAGE_NOT_VISIBLE )) +pr_warn ("Fail to set mem host visibility.\n" ); + + channel->gpadl_range[i].gpadlhandle =0 ; +return ret; }EXPORT_SYMBOL_GPL (vmbus_teardown_gpadl);
include/asm-generic/hyperv-tlfs.h +1 -0
@@ -158,6 +158,7 @@ struct ms_hyperv_tsc_page {#define HVCALL_RETARGET_INTERRUPT 0x007e#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0 +#define HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY 0x00db#define HV_FLUSH_ALL_PROCESSORS BIT(0)#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES BIT(1)
include/linux/hyperv.h +11 -1
@@ -788,6 +788,14 @@ struct vmbus_device {bool allowed_in_isolated; }; +struct vmbus_gpadl_range { +u32 gpadlhandle; +u32 size; +void *buffer; +}; + +#define VMBUS_GPADL_RANGE_COUNT 3 +struct vmbus_channel {struct list_head listentry;
@@ -808,6 +816,8 @@ struct vmbus_channel {struct completion rescind_event;u32 ringbuffer_gpadlhandle; +/* GPADL_RING and Send/Receive GPADL_BUFFER. */ +struct vmbus_gpadl_range gpadl_range[VMBUS_GPADL_RANGE_COUNT ];/* Allocated memory for ring buffer */ struct page *ringbuffer_page;
@@ -1182,7 +1192,7 @@ extern int vmbus_establish_gpadl(struct vmbus_channel *channel, u32 *gpadl_handle);extern int vmbus_teardown_gpadl (struct vmbus_channel *channel, -u32 gpadl_handle); +u32 gpadl_handle);void vmbus_reset_channel_cb (struct vmbus_channel *channel);
Signature · 1 line
2.25.1
RFC PATCH v2 2/12 x86/HV: Initialize shared memory boundary in Isolation VM
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Hyper-V exposes shared memory boundary via cpuid HYPERV_ CPUID_ISOLATION_CONFIG and store it in the shared_gpa_ boundary of ms_hyperv struct. This prepares to share memory with host for AMD SEV SNP guest.
--- arch/x86/kernel/cpu/mshyperv.c | 2 ++ include/asm-generic/mshyperv.h | 13 ++++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/mshyperv.c +2 -0
@@ -328,6 +328,8 @@ static void __init ms_hyperv_init_platform(void)if (ms_hyperv.features_b &HV_ISOLATION ) { ms_hyperv.isolation_config_a =cpuid_eax (HYPERV_CPUID_ISOLATION_CONFIG ); ms_hyperv.isolation_config_b =cpuid_ebx (HYPERV_CPUID_ISOLATION_CONFIG ); + ms_hyperv.shared_gpa_boundary = + (u64 )1 << ms_hyperv.shared_gpa_boundary_bits;pr_info ("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n" , ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b);
include/asm-generic/mshyperv.h +12 -1
@@ -34,8 +34,19 @@ struct ms_hyperv_info {u32 max_vp_index;u32 max_lp_index;u32 isolation_config_a; -u32 isolation_config_b; +union + { +u32 isolation_config_b; +struct { +u32 cvm_type :4 ; +u32 Reserved11 :1 ; +u32 shared_gpa_boundary_active :1 ; +u32 shared_gpa_boundary_bits :6 ; +u32 Reserved12 :20 ; + }; + };void __percpu **ghcb_base; +u64 shared_gpa_boundary; };extern struct ms_hyperv_info ms_hyperv;
Signature · 1 line
2.25.1
RFC PATCH v2 4/12 HV: Add Write/Read MSR registers via ghcb
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Hyper-V provides GHCB protocol to write Synthetic Interrupt Controller MSR registers and these registers are emulated by Hypervisor rather than paravisor.
Hyper-V requests to write SINTx MSR registers twice(once via GHCB and once via wrmsr instruction including the proxy bit 21) Guest OS ID MSR also needs to be set via GHCB.
--- arch/x86/hyperv/hv_init.c | 18 +---- arch/x86/hyperv/ivm.c | 130 ++++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 87 +++++++++++++++++---- arch/x86/kernel/cpu/mshyperv.c | 3 + drivers/hv/hv.c | 65 +++++++++++----- include/asm-generic/mshyperv.h | 4 +- 6 files changed, 261 insertions(+), 46 deletions(-)
arch/x86/hyperv/hv_init.c +4 -14
@@ -475,6 +475,9 @@ void __init hyperv_init(void) ghcb_base = (void **)this_cpu_ptr (ms_hyperv.ghcb_base); *ghcb_base = ghcb_va; + +/* Hyper-V requires to write guest os id via ghcb in SNP IVM. */ +hv_ghcb_msr_write (HV_X64_MSR_GUEST_OS_ID , guest_id); }rdmsrl (HV_X64_MSR_HYPERCALL , hypercall_msr.as_uint64);
@@ -561,6 +564,7 @@ void hyperv_cleanup(void)/* Reset our OS id */ wrmsrl (HV_X64_MSR_GUEST_OS_ID ,0 ); +hv_ghcb_msr_write (HV_X64_MSR_GUEST_OS_ID ,0 );/* * Reset hypercall page reference before reset the page,
@@ -668,17 +672,3 @@ bool hv_is_hibernation_supported(void)return !hv_root_partition &&acpi_sleep_state_supported (ACPI_STATE_S4 ); }EXPORT_SYMBOL_GPL (hv_is_hibernation_supported); - -enum hv_isolation_type hv_get_isolation_type (void ) -{ -if (!(ms_hyperv.features_b &HV_ISOLATION )) -return HV_ISOLATION_TYPE_NONE ; -return FIELD_GET (HV_ISOLATION_TYPE , ms_hyperv.isolation_config_b); -} -EXPORT_SYMBOL_GPL (hv_get_isolation_type); - -bool hv_is_isolation_supported (void ) -{ -return hv_get_isolation_type () !=HV_ISOLATION_TYPE_NONE ; -} -EXPORT_SYMBOL_GPL (hv_is_isolation_supported);
arch/x86/hyperv/ivm.c +130 -0
148 lines not on this page · raw message
arch/x86/include/asm/mshyperv.h +74 -13
119 lines not on this page · raw message
arch/x86/kernel/cpu/mshyperv.c +3 -0
@@ -333,6 +333,9 @@ static void __init ms_hyperv_init_platform(void)pr_info ("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n" , ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); + +if (hv_get_isolation_type () ==HV_ISOLATION_TYPE_SNP ) +static_branch_enable (&isolation_type_snp); }if (ms_hyperv.hints &HV_X64_ENLIGHTENED_VMCS_RECOMMENDED ) {
drivers/hv/hv.c +47 -18
101 lines not on this page · raw message
include/asm-generic/mshyperv.h +3 -1
@@ -23,6 +23,7 @@#include <linux/bitops.h> #include <linux/cpumask.h> #include <asm/ptrace.h> +#include <asm/mshyperv.h> #include <asm/hyperv-tlfs.h> struct ms_hyperv_info {
@@ -52,7 +53,7 @@ extern struct ms_hyperv_info ms_hyperv;extern u64 hv_do_hypercall (u64 control,void *inputaddr,void *outputaddr);extern u64 hv_do_fast_hypercall8 (u16 control,u64 input8); - +extern bool hv_isolation_type_snp (void );/* Generate the guest OS identifier as described in the Hyper-V TLFS */ static inline __u64 generate_guest_id(__u64 d_info1, __u64 kernel_version,
@@ -186,6 +187,7 @@ bool hv_is_hyperv_initialized(void);bool hv_is_hibernation_supported (void );enum hv_isolation_type hv_get_isolation_type (void );bool hv_is_isolation_supported (void ); +bool hv_isolation_type_snp (void );void hyperv_cleanup (void );#else /* CONFIG_HYPERV */ static inline bool hv_is_hyperv_initialized (void ) {return false; }
Signature · 1 line
2.25.1
RFC PATCH v2 5/12 HV: Add ghcb hvcall support for SNP VM
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Hyper-V provides ghcb hvcall to handle VMBusHVCALL_SIGNAL_EVENT and HVCALL_POST_MESSAGE
msg in SNP Isolation VM. Add such support.
--- arch/x86/hyperv/ivm.c | 69 +++++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 1 + drivers/hv/connection.c | 6 ++- drivers/hv/hv.c | 8 +++- 4 files changed, 82 insertions(+), 2 deletions(-)
arch/x86/hyperv/ivm.c +69 -0
@@ -18,8 +18,77 @@union hv_ghcb {struct ghcb ghcb; +struct { +u64 hypercalldata[509 ]; +u64 outputgpa; +union { +union { +struct { +u32 callcode :16 ; +u32 isfast :1 ; +u32 reserved1 :14 ; +u32 isnested :1 ; +u32 countofelements :12 ; +u32 reserved2 :4 ; +u32 repstartindex :12 ; +u32 reserved3 :4 ; + }; +u64 asuint64; + } hypercallinput; +union { +struct { +u16 callstatus; +u16 reserved1; +u32 elementsprocessed :12 ; +u32 reserved2 :20 ; + }; +u64 asunit64; + } hypercalloutput; + }; +u64 reserved2; + } hypercall; } __packed__aligned(PAGE_SIZE) ; +u64 hv_ghcb_hypercall (u64 control,void *input,void *output,u32 input_size) +{ +union hv_ghcb *hv_ghcb; +void **ghcb_base; +unsigned long flags; + +if (!ms_hyperv.ghcb_base) +return -EFAULT ; + +local_irq_save (flags); + ghcb_base = (void **)this_cpu_ptr (ms_hyperv.ghcb_base); + hv_ghcb = (union hv_ghcb *)*ghcb_base; +if (!hv_ghcb) { +local_irq_restore (flags); +return -EFAULT ; + } + +memset (hv_ghcb,0x00 ,HV_HYP_PAGE_SIZE ); + hv_ghcb->ghcb.protocol_version =1 ; + hv_ghcb->ghcb.ghcb_usage =1 ; + + hv_ghcb->hypercall.outputgpa = (u64 )output; + hv_ghcb->hypercall.hypercallinput.asuint64 =0 ; + hv_ghcb->hypercall.hypercallinput.callcode = control; + +if (input_size) +memcpy (hv_ghcb->hypercall.hypercalldata, input, input_size); + +VMGEXIT (); + + hv_ghcb->ghcb.ghcb_usage =0xffffffff ; +memset (hv_ghcb->ghcb.save.valid_bitmap,0 , +sizeof (hv_ghcb->ghcb.save.valid_bitmap)); + +local_irq_restore (flags); + +return hv_ghcb->hypercall.hypercalloutput.callstatus; +} +EXPORT_SYMBOL_GPL (hv_ghcb_hypercall); +void hv_ghcb_msr_write (u64 msr,u64 value) {union hv_ghcb *hv_ghcb;
arch/x86/include/asm/mshyperv.h +1 -0
@@ -318,6 +318,7 @@ void hv_sint_rdmsrl_ghcb(u64 msr, u64 *value);void hv_signal_eom_ghcb (void );void hv_ghcb_msr_write (u64 msr,u64 value);void hv_ghcb_msr_read (u64 msr,u64 *value); +u64 hv_ghcb_hypercall (u64 control,void *input,void *output,u32 input_size);#define hv_get_synint_state_ghcb (int_num, val) \ hv_sint_rdmsrl_ghcb(HV_X64_MSR_SINT0 + int_num, val)
drivers/hv/connection.c +5 -1
@@ -442,6 +442,10 @@ void vmbus_set_event(struct vmbus_channel *channel) ++channel->sig_events; -hv_do_fast_hypercall8 (HVCALL_SIGNAL_EVENT , channel->sig_event); +if (hv_isolation_type_snp ()) +hv_ghcb_hypercall (HVCALL_SIGNAL_EVENT , &channel->sig_event, +NULL ,sizeof (u64)); +else +hv_do_fast_hypercall8 (HVCALL_SIGNAL_EVENT , channel->sig_event); }EXPORT_SYMBOL_GPL (vmbus_set_event);
drivers/hv/hv.c +7 -1
@@ -60,7 +60,13 @@ int hv_post_message(union hv_connection_id connection_id, aligned_msg->payload_size = payload_size;memcpy ((void *)aligned_msg->payload, payload, payload_size); - status =hv_do_hypercall (HVCALL_POST_MESSAGE , aligned_msg,NULL ); +if (hv_isolation_type_snp ()) + status =hv_ghcb_hypercall (HVCALL_POST_MESSAGE , + (void *)aligned_msg,NULL , +sizeof (struct hv_input_post_message )); +else + status =hv_do_hypercall (HVCALL_POST_MESSAGE , + aligned_msg,NULL );/* Preemption must remain disabled until after the hypercall * so some other thread can't get scheduled onto this cpu and
Signature · 1 line
2.25.1
RFC PATCH v2 7/12 HV/Vmbus: Initialize VMbus ring buffer for Isolation VM
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
VMbus ring buffer are shared with host and it's need to be accessed via extra address space of Isolation VM with SNP support. This patch is to map the ring buffer address in extra address space via ioremap(). HV host visibility hvcall smears data in the ring buffer and so reset the ring buffer memory to zero after calling visibility hvcall.
--- drivers/hv/channel.c | 10 +++++ drivers/hv/hyperv_vmbus.h | 2 + drivers/hv/ring_buffer.c | 83 +++++++++++++++++++++++++++++---------- mm/ioremap.c | 1 + mm/vmalloc.c | 1 + 5 files changed, 76 insertions(+), 21 deletions(-)
drivers/hv/channel.c +10 -0
@@ -634,6 +634,16 @@ static int __vmbus_open(struct vmbus_channel *newchannel,if (err) goto error_clean_ring; + err =hv_ringbuffer_post_init (&newchannel->outbound, + page, send_pages); +if (err) + goto error_free_gpadl; + + err =hv_ringbuffer_post_init (&newchannel->inbound, + &page[send_pages], recv_pages); +if (err) + goto error_free_gpadl; +/* Create and init the channel open message */ open_info = kzalloc(sizeof(*open_info) + sizeof(struct vmbus_channel_open_channel),
drivers/hv/hyperv_vmbus.h +2 -0
@@ -172,6 +172,8 @@ extern int hv_synic_cleanup(unsigned int cpu);/* Interface */ void hv_ringbuffer_pre_init (struct vmbus_channel *channel); +int hv_ringbuffer_post_init (struct hv_ring_buffer_info *ring_info, +struct page *pages,u32 page_cnt);int hv_ringbuffer_init (struct hv_ring_buffer_info *ring_info,struct page *pages,u32 pagecnt);
drivers/hv/ring_buffer.c +62 -21
107 lines not on this page · raw message
mm/ioremap.c +1 -0
@@ -248,6 +248,7 @@ int ioremap_page_range(unsigned long addr,return err; } +EXPORT_SYMBOL_GPL (ioremap_page_range);#ifdef CONFIG_GENERIC_IOREMAP void __iomem *ioremap_prot (phys_addr_t addr,size_t size,unsigned long prot)
mm/vmalloc.c +1 -0
@@ -2131,6 +2131,7 @@ struct vm_struct *get_vm_area(unsigned long size, unsigned long flags)NUMA_NO_NODE ,GFP_KERNEL , __builtin_return_address(0)); } +EXPORT_SYMBOL_GPL (get_vm_area);struct vm_struct *get_vm_area_caller (unsigned long size,unsigned long flags,const void *caller)
Signature · 1 line
2.25.1
RFC PATCH v2 6/12 HV/Vmbus: Add SNP support for VMbus channel initiate message
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
The physical address of monitor pages in the CHANNELMSG_INITIATE_CONTACT msg should be in the extra address space for SNP support and these pages also should be accessed via the extra address space inside Linux guest and remap the extra address by ioremap function.
--- drivers/hv/connection.c | 62 +++++++++++++++++++++++++++++++++++++++ drivers/hv/hyperv_vmbus.h | 1 + 2 files changed, 63 insertions(+)
drivers/hv/connection.c +62 -0
@@ -101,6 +101,12 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) msg->monitor_page1 =virt_to_phys (vmbus_connection.monitor_pages[0 ]); msg->monitor_page2 =virt_to_phys (vmbus_connection.monitor_pages[1 ]); + +if (hv_isolation_type_snp ()) { + msg->monitor_page1 += ms_hyperv.shared_gpa_boundary; + msg->monitor_page2 += ms_hyperv.shared_gpa_boundary; + } + msg->target_vcpu =hv_cpu_number_to_vp_number (VMBUS_CONNECT_CPU );/*
@@ -145,6 +151,29 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)return -ECONNREFUSED ; } +if (hv_isolation_type_snp ()) { + vmbus_connection.monitor_pages_va[0 ] + = vmbus_connection.monitor_pages[0 ]; + vmbus_connection.monitor_pages[0 ] + =ioremap_cache (msg->monitor_page1,HV_HYP_PAGE_SIZE ); +if (!vmbus_connection.monitor_pages[0 ]) +return -ENOMEM ; + + vmbus_connection.monitor_pages_va[1 ] + = vmbus_connection.monitor_pages[1 ]; + vmbus_connection.monitor_pages[1 ] + =ioremap_cache (msg->monitor_page2,HV_HYP_PAGE_SIZE ); +if (!vmbus_connection.monitor_pages[1 ]) { +vunmap (vmbus_connection.monitor_pages[0 ]); +return -ENOMEM ; + } + +memset (vmbus_connection.monitor_pages[0 ],0x00 , +HV_HYP_PAGE_SIZE ); +memset (vmbus_connection.monitor_pages[1 ],0x00 , +HV_HYP_PAGE_SIZE ); + } +return ret; }
@@ -156,6 +185,7 @@ int vmbus_connect(void)struct vmbus_channel_msginfo *msginfo =NULL ;int i, ret =0 ;__u32 version; +u64 pfn[2 ];/* Initialize the vmbus connection */ vmbus_connection.conn_state =CONNECTING ;
@@ -213,6 +243,16 @@ int vmbus_connect(void) goto cleanup; } +if (hv_isolation_type_snp ()) { + pfn[0 ] =virt_to_hvpfn (vmbus_connection.monitor_pages[0 ]); + pfn[1 ] =virt_to_hvpfn (vmbus_connection.monitor_pages[1 ]); +if (hv_mark_gpa_visibility (2 , pfn, +VMBUS_PAGE_VISIBLE_READ_WRITE )) { + ret = -EFAULT ; + goto cleanup; + } + } + msginfo =kzalloc (sizeof (*msginfo) +sizeof (struct vmbus_channel_initiate_contact ),GFP_KERNEL );
@@ -279,6 +319,8 @@ int vmbus_connect(void)void vmbus_disconnect (void ) { +u64 pfn[2 ]; +/* * First send the unload request to the host. */
@@ -298,6 +340,26 @@ void vmbus_disconnect(void) vmbus_connection.int_page =NULL ; } +if (hv_isolation_type_snp ()) { +if (vmbus_connection.monitor_pages_va[0 ]) { +vunmap (vmbus_connection.monitor_pages[0 ]); + vmbus_connection.monitor_pages[0 ] + = vmbus_connection.monitor_pages_va[0 ]; + vmbus_connection.monitor_pages_va[0 ] =NULL ; + } + +if (vmbus_connection.monitor_pages_va[1 ]) { +vunmap (vmbus_connection.monitor_pages[1 ]); + vmbus_connection.monitor_pages[1 ] + = vmbus_connection.monitor_pages_va[1 ]; + vmbus_connection.monitor_pages_va[1 ] =NULL ; + } + + pfn[0 ] =virt_to_hvpfn (vmbus_connection.monitor_pages[0 ]); + pfn[1 ] =virt_to_hvpfn (vmbus_connection.monitor_pages[1 ]); +hv_mark_gpa_visibility (2 , pfn,VMBUS_PAGE_NOT_VISIBLE ); + } +hv_free_hyperv_page ((unsigned long )vmbus_connection.monitor_pages[0 ]);hv_free_hyperv_page ((unsigned long )vmbus_connection.monitor_pages[1 ]); vmbus_connection.monitor_pages[0 ] =NULL ;
drivers/hv/hyperv_vmbus.h +1 -0
@@ -240,6 +240,7 @@ struct vmbus_connection {* is child->parent notification */ struct hv_monitor_page *monitor_pages[2 ]; +void *monitor_pages_va[2 ];struct list_head chn_msg_list;spinlock_t channelmsg_lock;
Signature · 1 line
2.25.1
RFC PATCH v2 8/12 UIO/Hyper-V: Not load UIO HV driver in the isolation VM.
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
UIO HV driver should not load in the isolation VM for security reason. Return ENOTSUPP in the hv_uio_probe() in the isolation VM.
--- drivers/uio/uio_hv_generic.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/uio/uio_hv_generic.c +5 -0
@@ -29,6 +29,7 @@#include <linux/hyperv.h> #include <linux/vmalloc.h> #include <linux/slab.h> +#include <asm/mshyperv.h> #include "../hv/hyperv_vmbus.h"
@@ -241,6 +242,10 @@ hv_uio_probe(struct hv_device *dev,void *ring_buffer;int ret; +/* UIO driver should not be loaded in the isolation VM.*/ +if (hv_is_isolation_supported ()) +return -ENOTSUPP ; +/* Communicating with host has to be via shared memory not hypercall */ if (!channel->offermsg.monitor_allocated) {dev_err (&dev->device,"vmbus channel requires hypercall\n" );
Signature · 1 line
2.25.1
RFC PATCH v2 11/12 HV/Netvsc: Add Isolation VM support for netvsc driver
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
In Isolation VM, all shared memory with host needs to mark visible to host via hvcall. vmbus_establish_gpadl() has already done it for netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_ pagebuffer() still need to handle. Use DMA API to map/umap these memory during sending/receiving packet and Hyper-V DMA ops callback will use swiotlb fucntion to allocate bounce buffer and copy data from/to bounce buffer.
--- drivers/net/hyperv/hyperv_net.h | 11 +++ drivers/net/hyperv/netvsc.c | 137 ++++++++++++++++++++++++++++-- drivers/net/hyperv/rndis_filter.c | 3 + 3 files changed, 144 insertions(+), 7 deletions(-)
drivers/net/hyperv/hyperv_net.h +11 -0
@@ -130,6 +130,7 @@ struct hv_netvsc_packet {u32 total_bytes;u32 send_buf_index;u32 total_data_buflen; +struct dma_range *dma_range; };#define NETVSC_HASH_KEYLEN 40
@@ -1026,6 +1027,7 @@ struct netvsc_device {/* Receive buffer allocated by us but manages by NetVSP */ void *recv_buf; +void *recv_original_buf;u32 recv_buf_size;/* allocated bytes */ u32 recv_buf_gpadl_handle;u32 recv_section_cnt;
@@ -1034,6 +1036,8 @@ struct netvsc_device {/* Send buffer allocated by us */ void *send_buf; +void *send_original_buf; +u32 send_buf_size;u32 send_buf_gpadl_handle;u32 send_section_cnt;u32 send_section_size;
@@ -1715,4 +1719,11 @@ struct rndis_message {#define TRANSPORT_INFO_IPV6_TCP 0x10#define TRANSPORT_INFO_IPV6_UDP 0x20 +struct dma_range { +dma_addr_t dma; +u32 mapping_size; +}; + +void netvsc_dma_unmap (struct hv_device *hv_dev, +struct hv_netvsc_packet *packet);#endif /* _HYPERV_NET_H */
drivers/net/hyperv/netvsc.c +130 -7
198 lines not on this page · raw message
drivers/net/hyperv/rndis_filter.c +3 -0
@@ -263,6 +263,8 @@ static void rndis_filter_receive_response(struct net_device *ndev, {struct rndis_device *dev = nvdev->extension;struct rndis_request *request =NULL ; +struct hv_device *hv_dev = ((struct net_device_context *) +netdev_priv (ndev))->device_ctx;bool found = false;unsigned long flags;
@@ -327,6 +329,7 @@ static void rndis_filter_receive_response(struct net_device *ndev, } } +netvsc_dma_unmap (hv_dev, &request->pkt);complete (&request->wait_event); }else { netdev_err(ndev,
Signature · 1 line
2.25.1
RFC PATCH v2 10/12 HV/IOMMU: Add Hyper-V dma ops support
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Hyper-V Isolation VM requires bounce buffer support. To use swiotlb bounce buffer, add Hyper-V dma ops and use swiotlb functions in the map and unmap callback.
Allocate bounce buffer in the Hyper-V code because bounce buffer needs to be accessed via extra address space(e.g, address above 39bit) in the AMD SEV SNP based Isolation VM.
ioremap_cache() can't use in the hyperv_iommu_swiotlb_init() which is too early place and remap bounce buffer in the hyperv_iommu_swiotlb_ later_init().
--- arch/x86/kernel/pci-swiotlb.c | 3 +- drivers/hv/vmbus_drv.c | 3 + drivers/iommu/hyperv-iommu.c | 127 ++++++++++++++++++++++++++++++++++ 3 files changed, 132 insertions(+), 1 deletion(-)
arch/x86/kernel/pci-swiotlb.c +2 -1
@@ -15,6 +15,7 @@#include <asm/iommu_table.h> int swiotlb__read_mostly ; +extern int hyperv_swiotlb;/* * pci_swiotlb_detect_override - set swiotlb to 1 if necessary
@@ -68,7 +69,7 @@ void __init pci_swiotlb_init(void)void __init pci_swiotlb_late_init (void ) {/* An IOMMU turned us off. */ -if (!swiotlb) +if (!swiotlb && !hyperv_swiotlb)swiotlb_exit ();else { printk(KERN_INFO "PCI-DMA: "
drivers/hv/vmbus_drv.c +3 -0
@@ -23,6 +23,7 @@#include <linux/cpu.h> #include <linux/sched/task_stack.h> +#include <linux/dma-map-ops.h> #include <linux/delay.h> #include <linux/notifier.h> #include <linux/ptrace.h>
@@ -2030,6 +2031,7 @@ struct hv_device *vmbus_device_create(const guid_t *type,return child_device_obj; } +static u64 vmbus_dma_mask =DMA_BIT_MASK (64 );/* * vmbus_device_register - Register the child device */
@@ -2070,6 +2072,7 @@ int vmbus_device_register(struct hv_device *child_device_obj) }hv_debug_add_dev_dir (child_device_obj); + child_device_obj->device.dma_mask = &vmbus_dma_mask;return 0 ; err_kset_unregister:
drivers/iommu/hyperv-iommu.c +127 -0
156 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v2 12/12 HV/Storvsc: Add Isolation VM support for storvsc driver
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
In Isolation VM, all shared memory with host needs to mark visible to host via hvcall. vmbus_establish_gpadl() has already done it for netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_ mpb_desc() still need to handle. Use DMA API to map/umap these memory during sending/receiving packet and Hyper-V DMA ops callback will use swiotlb fucntion to allocate bounce buffer and copy data from/to bounce buffer.
--- drivers/scsi/storvsc_drv.c | 67 +++++++++++++++++++++++++++++++++++++- 1 file changed, 66 insertions(+), 1 deletion(-)
drivers/scsi/storvsc_drv.c +66 -1
@@ -21,6 +21,8 @@#include <linux/device.h> #include <linux/hyperv.h> #include <linux/blkdev.h> +#include <linux/io.h> +#include <linux/dma-mapping.h> #include <scsi/scsi.h> #include <scsi/scsi_cmnd.h> #include <scsi/scsi_host.h>
@@ -414,6 +416,11 @@ static void storvsc_on_channel_callback(void *context);#define STORVSC_IDE_MAX_TARGETS 1#define STORVSC_IDE_MAX_CHANNELS 1 +struct dma_range { +dma_addr_t dma; +u32 mapping_size; +}; +struct storvsc_cmd_request {struct scsi_cmnd *cmd;
@@ -427,6 +434,8 @@ struct storvsc_cmd_request {u32 payload_sz;struct vstor_packet vstor_packet; +u32 hvpg_count; +struct dma_range *dma_range; };
@@ -1236,6 +1245,7 @@ static void storvsc_on_channel_callback(void *context)const struct vmpacket_descriptor *desc;struct hv_device *device;struct storvsc_device *stor_device; +int i;if (channel->primary_channel !=NULL ) device = channel->primary_channel->device_obj;
@@ -1249,6 +1259,8 @@ static void storvsc_on_channel_callback(void *context) foreach_vmbus_pkt(desc, channel) {void *packet =hv_pkt_data (desc);struct storvsc_cmd_request *request; +enum dma_data_direction dir; +u32 attrs;u64 cmd_rqst; cmd_rqst = vmbus_request_addr(&channel->requestor,
@@ -1261,6 +1273,22 @@ static void storvsc_on_channel_callback(void *context) request = (struct storvsc_cmd_request *)(unsigned long )cmd_rqst; +if (request->vstor_packet.vm_srb.data_in ==READ_TYPE ) + dir =DMA_FROM_DEVICE ; +else + dir =DMA_TO_DEVICE ; + +if (request->dma_range) { +for (i =0 ; i < request->hvpg_count; i++) +dma_unmap_page_attrs (&device->device, + request->dma_range[i].dma, + request->dma_range[i].mapping_size, + request->vstor_packet.vm_srb.data_in + ==READ_TYPE ? +DMA_FROM_DEVICE :DMA_TO_DEVICE , attrs); +kfree (request->dma_range); + } +if (request == &stor_device->init_request || request == &stor_device->reset_request) { memcpy(&request->vstor_packet, packet,
@@ -1682,8 +1710,10 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)struct vmscsi_request *vm_srb;struct scatterlist *cur_sgl;struct vmbus_packet_mpb_array *payload; +enum dma_data_direction dir;u32 payload_sz;u32 length; +u32 attrs;if (vmstor_proto_version <=VMSTOR_PROTO_VERSION_WIN8 ) {/*
@@ -1722,14 +1752,17 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)case DMA_TO_DEVICE : vm_srb->data_in =WRITE_TYPE ; vm_srb->win8_extension.srb_flags |=SRB_FLAGS_DATA_OUT ; + dir =DMA_TO_DEVICE ;break ;case DMA_FROM_DEVICE : vm_srb->data_in =READ_TYPE ; vm_srb->win8_extension.srb_flags |=SRB_FLAGS_DATA_IN ; + dir =DMA_FROM_DEVICE ;break ;case DMA_NONE : vm_srb->data_in =UNKNOWN_TYPE ; vm_srb->win8_extension.srb_flags |=SRB_FLAGS_NO_DATA_TRANSFER ; + dir =DMA_NONE ;break ;default :/*
@@ -1786,6 +1819,12 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) hvpgoff = sgl->offset >>HV_HYP_PAGE_SHIFT ; cur_sgl = sgl; + + cmd_request->dma_range =kzalloc (sizeof (struct dma_range ) * hvpg_count, +GFP_ATOMIC ); +if (!cmd_request->dma_range) +return -ENOMEM ; +for (i =0 ; i < hvpg_count; i++) {/* * 'i' is the index of hv pages in the payload and
@@ -1805,6 +1844,8 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)*/ unsigned int hvpgoff_in_page = (i + hvpgoff) %NR_HV_HYP_PAGES_IN_PAGE ; +dma_addr_t dma; +u32 size;/* * Two cases that we need to fetch a page:
@@ -1817,8 +1858,28 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) cur_sgl =sg_next (cur_sgl); } - payload->range.pfn_array[i] = hvpfn + hvpgoff_in_page; + size =min (HV_HYP_PAGE_SIZE - offset_in_hvpg, (unsigned long )length); + dma =dma_map_page_attrs (&dev->device, +pfn_to_page (hvpfn), + offset_in_hvpg, size, + scmnd->sc_data_direction, attrs); +if (dma_mapping_error (&dev->device, dma)) { +pr_warn ("dma map error.\n" ); + ret = -ENOMEM ; + goto free_dma_range; + } + +if (offset_in_hvpg) { + payload->range.offset = dma & ~HV_HYP_PAGE_MASK ; + offset_in_hvpg =0 ; + } + + cmd_request->dma_range[i].dma = dma; + cmd_request->dma_range[i].mapping_size = size; + payload->range.pfn_array[i] = dma >>HV_HYP_PAGE_SHIFT ; + length -= size; } + cmd_request->hvpg_count = hvpg_count; } cmd_request->payload = payload;
@@ -1836,6 +1897,10 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) }return 0 ; + +free_dma_range: +kfree (cmd_request->dma_range); +return ret; }static struct scsi_host_template scsi_driver = {
Signature · 1 line
2.25.1
RFC PATCH v2 9/12 swiotlb: Add bounce buffer remap address setting function
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
For Hyper-V isolation VM with AMD SEV SNP, the bounce buffer(shared memory) needs to be accessed via extra address space(e.g address above bit39). Hyper-V code may remap extra address space outside of swiotlb. swiotlb_bounce() needs to use remap virtual address to copy data from/to bounce buffer. Add new interface swiotlb_set_bounce_remap() to do that.
--- include/linux/swiotlb.h | 5 +++++ kernel/dma/swiotlb.c | 13 ++++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)
include/linux/swiotlb.h +5 -0
@@ -82,8 +82,13 @@ unsigned int swiotlb_max_segment(void);size_t swiotlb_max_mapping_size (struct device *dev);bool is_swiotlb_active (void );void __init swiotlb_adjust_size (unsigned long new_size); +void swiotlb_set_bounce_remap (unsigned char *vaddr);#else #define swiotlb_force SWIOTLB_NO_FORCE +static inline void swiotlb_set_bounce_remap (unsigned char *vaddr) +{ +} +static inline bool is_swiotlb_buffer (phys_addr_t paddr) {return false;
kernel/dma/swiotlb.c +12 -1
@@ -94,6 +94,7 @@ static unsigned int io_tlb_index;* not be bounced (unless SWIOTLB_FORCE is set). */ static unsigned int max_segment; +static unsigned char *swiotlb_bounce_remap_addr;/* * We need to save away the original address corresponding to a mapped entry
@@ -421,6 +422,11 @@ void __init swiotlb_exit(void)swiotlb_cleanup (); } +void swiotlb_set_bounce_remap (unsigned char *vaddr) +{ + swiotlb_bounce_remap_addr = vaddr; +} +/* * Bounce: copy the swiotlb buffer from or back to the original dma location */
@@ -428,7 +434,12 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr, size_t size, enum dma_data_direction dir) {unsigned long pfn =PFN_DOWN (orig_addr); -unsigned char *vaddr =phys_to_virt (tlb_addr); +unsigned char *vaddr; + +if (swiotlb_bounce_remap_addr) + vaddr = swiotlb_bounce_remap_addr + tlb_addr - io_tlb_start; +else + vaddr =phys_to_virt (tlb_addr);if (PageHighMem (pfn_to_page (pfn))) {/* The buffer does not have a mapping. Map it in and copy */
Signature · 1 line
2.25.1
Greg KH
maintainerRe: [RFC V2 PATCH 8/12] UIO/Hyper-V: Not load UIO HV driver in the isolation VM.
In reply to Tianyu LanQuoted 15 lines · Tianyu Lan
From: Tianyu Lan <[email protected]>
UIO HV driver should not load in the isolation VM for security reason. Return ENOTSUPP in the
hv_uio_probe()in the isolation VM.Signed-off-by Tianyu Lan <[email protected]>--- drivers/uio/uio_hv_generic.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/uio/uio_hv_generic.c +0
-0
drivers/uio/uio_hv_generic.c line 32
Quoted 4 lines
drivers/uio/uio_hv_generic.c +1
-0@@ -29,6 +29,7 @@#include <linux/hyperv.h> #include <linux/vmalloc.h> #include <linux/slab.h> +#include <asm/mshyperv.h>
No driver should be having to add "asm" include files.
Quoted 17 lines
drivers/uio/uio_hv_generic.c +4
-0#include "../hv/hyperv_vmbus.h" @@ -241,6 +242,10 @@ hv_uio_probe(struct hv_device *dev,void *ring_buffer;int ret; +/* UIO driver should not be loaded in the isolation VM.*/ +if (hv_is_isolation_supported ()) +return -ENOTSUPP ; +/* Communicating with host has to be via shared memory not hypercall */ if (!channel->offermsg.monitor_allocated) {dev_err (&dev->device,"vmbus channel requires hypercall\n" );Signature · 1 line
2.25.1
Always run checkpatch.pl on your patches so you do not get grumpy maintainers telling you to run checkpatch.pl on your patch :(
{sigh}
Greg KH
maintainerRe: [RFC V2 PATCH 8/12] UIO/Hyper-V: Not load UIO HV driver in the isolation VM.
In reply to Tianyu LanQuoted 3 lines · Tianyu Lan
From: Tianyu Lan <[email protected]>
UIO HV driver should not load in the isolation VM for security reason.
Why? I need a lot more excuse than that.
Why would the vm allow UIO devices to bind to it if it was not possible? Shouldn't the VM be handling this type of logic and not forcing all individual hyperv drivers to do this?
This feels wrong...
thanks,
greg k-h
Re: [RFC V2 PATCH 9/12] swiotlb: Add bounce buffer remap address setting function
In reply to Tianyu LanQuoted 7 lines · Tianyu Lan
From: Tianyu Lan <[email protected]>
For Hyper-V isolation VM with AMD SEV SNP, the bounce buffer(shared memory) needs to be accessed via extra address space(e.g address above bit39). Hyper-V code may remap extra address space outside of swiotlb.
swiotlb_bounce()needs to use remap virtual address to copy data from/to bounce buffer. Add new interfaceswiotlb_set_bounce_remap()to do that.
I have no way to review what this actually doing when you only Cc me on a single patch. Please make sure everyone is Cced on the whole series to enable proper review.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC V2 PATCH 9/12] swiotlb: Add bounce buffer remap address setting function
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Tianyu Lan
From: Tianyu Lan <[email protected]>
For Hyper-V isolation VM with AMD SEV SNP, the bounce buffer(shared memory) needs to be accessed via extra address space(e.g address above bit39). Hyper-V code may remap extra address space outside of swiotlb.
swiotlb_bounce()needs to use remap virtual address to copy data from/to bounce buffer. Add new interfaceswiotlb_set_bounce_remap()to do that.
Quoted 4 lines
I have no way to review what this actually doing when you only Cc me on a single patch. Please make sure everyone is Cced on the whole series to enable proper review.
Sure. I will resend all patches. Thanks for reminder.
Re: [RFC V2 PATCH 8/12] UIO/Hyper-V: Not load UIO HV driver in the isolation VM.
In reply to Greg KHHi Greg:
Thanks for your review.
Quoted 6 lines · Greg KH
Quoted 3 lines · Tianyu Lan
From: Tianyu Lan <[email protected]>
UIO HV driver should not load in the isolation VM for security reason.
Why? I need a lot more excuse than that.
The reason is that ring buffers have been marked as visible to host. UIO driver will expose these buffers to user space and user space driver hasn't done some secure check for data from host. This is considered as insecure in isolation VM.
Quoted 6 lines
Why would the vm allow UIO devices to bind to it if it was not possible? Shouldn't the VM be handling this type of logic and not forcing all individual hyperv drivers to do this?
This feels wrong...
Hypervisor exposes network and storage devices but can't prohibit guest from binding these devices to UIO driver.
You are right. This should not happen in the individual driver and will try handling this in the vmbus driver level.
Quoted 5 lines
thanks,
greg k-h
Greg KH
maintainerRe: [RFC V2 PATCH 8/12] UIO/Hyper-V: Not load UIO HV driver in the isolation VM.
Quoted 7 lines · Tianyu Lan
Hi Greg:
Thanks for your review.Quoted 3 lines · Greg KH
Quoted 2 lines · Tianyu Lan
From: Tianyu Lan <[email protected]>
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
UIO HV driver should not load in the isolation VM for security reason.
Why? I need a lot more excuse than that.
The reason is that ring buffers have been marked as visible to host. UIO driver will expose these buffers to user space and user space driver hasn't done some secure check for data from host. This is considered as insecure in isolation VM.
But as this is a VM choice, why did the VM mark those as visible in the first place?
thanks,
greg k-h
Re: [RFC V2 PATCH 11/12] HV/Netvsc: Add Isolation VM support for netvsc driver
In reply to Tianyu LanQuoted 16 lines · Tianyu Lan
From: Tianyu Lan <[email protected]>
In Isolation VM, all shared memory with host needs to mark visible to host via hvcall.
vmbus_establish_gpadl()has already done it for netvsc rx/tx ring buffer. The page buffer used byvmbus_sendpacket_pagebuffer()still need to handle. Use DMA API to map/umap these memory during sending/receiving packet and Hyper-V DMA ops callback will use swiotlb fucntion to allocate bounce buffer and copy data from/to bounce buffer.Signed-off-by Tianyu Lan <[email protected]>--- drivers/net/hyperv/hyperv_net.h | 11 +++ drivers/net/hyperv/netvsc.c | 137 ++++++++++++++++++++++++++++-- drivers/net/hyperv/rndis_filter.c | 3 + 3 files changed, 144 insertions(+), 7 deletions(-)
<...>
Quoted 4 lines
drivers/net/hyperv/netvsc.c +4
-0+ packet->dma_range =kzalloc (sizeof (struct dma_range ) * page_count, +GFP_KERNEL ); +if (!packet->dma_range) +return -ENOMEM ;
drivers/net/hyperv/netvsc.c static void netvsc_copy_to_send_buf(struct netvsc_device *net_device, · line 984
Quoted 10 lines
drivers/net/hyperv/netvsc.c +10
-0+ +for (i =0 ; i < page_count; i++) { +char *src =phys_to_virt ((pb[i].pfn <<HV_HYP_PAGE_SHIFT ) + + pb[i].offset); +u32 len = pb[i].len; + + dma =dma_map_single (&hv_dev->device, src, len, +DMA_TO_DEVICE ); +if (dma_mapping_error (&hv_dev->device, dma)) +return -ENOMEM ;
Don't you leak dma_range here?
BTW, It will be easier if you CC all on all patches, so we will be able to get whole context.
Thanks