x86/hyperv: Add AMD sev-snp enlightened guest support on hyperv
Subsystems (5)
v6 cover x86/hyperv: Add AMD sev-snp enlightened guest support on hyperv
From: Tianyu Lan <[email protected]>
Hyper-V provides two modes for running SEV-SNP VMs:
1) In vTOM mode with a paravisor (see Section 15.36.8 of [1])
2) In "fully enlightened" mode with normal "C" bit control over page encryption, and no paravisor
For #1, the paravisor runs in VMPL 0, while Linux runs in VMPL 2 (see Section 15.36.7 of [1]). The paravisor is typically provided by Hyper-V and handles most of the SNP-related functionality. As such, most of the SNP functionality in the Linux guest is bypassed. The guest operates in vTOM mode, where encryption is enabled by default. The guest must still request page transitions between private and shared, but there is relatively less SNP machinery required in the guest. Support for this mode of operation first went upstream in the 5.15 kernel.
For #2, this patch set provides the initial support. The existing SEV-SNP machinery in the kernel is fully used, but Hyper-V specific updates are required to properly share Hyper-V communication pages between the guest and host and to start APs at boot time.
In either mode, Hyper-V requires that the guest implement the SEV-SNP Restricted Interrupt Injection feature (see Section 15.36.16 of [1], and Section 5 of [2]). Without this feature, the guest is subject to attack by a compromised hypervisor that can inject any exception at any time, such as injecting an interrupt while the guest has interrupts disabled. In vTOM mode, Restricted Interrupt Injection is implemented by the paravisor, so no Linux guest changes are required. But in fully enlightened mode, the Linux guest must provide the implementation.
This patch set is derived from an earlier patch set that includes both the Hyper-V specific changes and Restricted Interrupt Injection support.[3] But it is now limited to only the Hyper-V specific changes. The Restricted Interrupt Injection support will come later in a separate patch set.
[1] https://www.amd.com/system/files/TechDocs/24593.pdf
[2] https://www.amd.com/system/files/TechDocs/56421-guest-hypervisor-communication-block-standardization.pdf
[3] https://lore.kernel.org/lkml/[email protected]/
Change since v5:
* Remove input page check in the get_vtl()
* Convert alternative back to if else check in the hypercall implementation.
Change since v3:
* Fix fossil comment
Change since v2:
* Update Change log.
* Rework Hyper-V hypercall implementation.
Change since v1:
* vTOM case uses paravisor_present flag and HV_ISOLATION_TYPE_SNP type.
* Rework some patches' change log
* Fix some comments in the patches
Tianyu Lan (8):
x86/hyperv: Add sev-snp enlightened guest static key x86/hyperv: Set Virtual Trust Level in VMBus init message x86/hyperv: Mark Hyper-V vp assist page unencrypted in SEV-SNP
enlightened guest
drivers: hv: Mark percpu hvcall input arg page unencrypted in SEV-SNP
enlightened guest
x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp
enlightened guest
clocksource: hyper-v: Mark hyperv tsc page unencrypted in sev-snp
enlightened guest
x86/hyperv: Add smp support for SEV-SNP guest
x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES
arch/x86/hyperv/hv_init.c | 50 +++++++++- arch/x86/hyperv/ivm.c | 149 +++++++++++++++++++++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 7 ++ arch/x86/include/asm/mshyperv.h | 55 ++++++++--- arch/x86/kernel/cpu/mshyperv.c | 43 ++++++++- drivers/clocksource/hyperv_timer.c | 2 +- drivers/hv/connection.c | 1 + drivers/hv/hv.c | 57 ++++++++++- drivers/hv/hv_common.c | 19 ++++ include/asm-generic/hyperv-tlfs.h | 1 + include/asm-generic/mshyperv.h | 14 ++- include/linux/hyperv.h | 4 +- 12 files changed, 377 insertions(+), 25 deletions(-)
Signature · 1 line
2.25.1
PATCH v6 1/8 x86/hyperv: Add sev-snp enlightened guest static key
From: Tianyu Lan <[email protected]>
Introduce static key isolation_type_en_snp for enlightened sev-snp guest check.
--- arch/x86/hyperv/ivm.c | 11 +++++++++++ arch/x86/include/asm/mshyperv.h | 2 ++ arch/x86/kernel/cpu/mshyperv.c | 9 +++++++-- drivers/hv/hv_common.c | 6 ++++++ include/asm-generic/mshyperv.h | 13 ++++++++++--- 5 files changed, 36 insertions(+), 5 deletions(-)
arch/x86/hyperv/ivm.c +11 -0
@@ -413,3 +413,14 @@ bool hv_isolation_type_snp(void) {return static_branch_unlikely (&isolation_type_snp); } + +DEFINE_STATIC_KEY_FALSE (isolation_type_en_snp); +/* +* hv_isolation_type_en_snp - Check system runs in the AMD SEV-SNP based +* isolation enlightened VM. +*/ +bool hv_isolation_type_en_snp (void ) +{ +return static_branch_unlikely (&isolation_type_en_snp); +} +
arch/x86/include/asm/mshyperv.h +2 -0
@@ -26,6 +26,7 @@union hv_ghcb ;DECLARE_STATIC_KEY_FALSE (isolation_type_snp); +DECLARE_STATIC_KEY_FALSE (isolation_type_en_snp); typedef int (*hyperv_fill_flush_list_func)( struct hv_guest_mapping_flush_list *flush,
@@ -239,6 +240,7 @@ static inline void hv_vtom_init(void) {}#endif extern bool hv_isolation_type_snp (void ); +extern bool hv_isolation_type_en_snp (void );static inline bool hv_is_synic_reg (unsigned int reg) {
arch/x86/kernel/cpu/mshyperv.c +7 -2
@@ -402,8 +402,12 @@ static void __init ms_hyperv_init_platform(void)pr_info ("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n" , ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); -if (hv_get_isolation_type () ==HV_ISOLATION_TYPE_SNP ) + +if (cc_platform_has (CC_ATTR_GUEST_SEV_SNP )) { +static_branch_enable (&isolation_type_en_snp); + }else if (hv_get_isolation_type () ==HV_ISOLATION_TYPE_SNP ) {static_branch_enable (&isolation_type_snp); + } }if (hv_max_functions_eax >=HYPERV_CPUID_NESTED_FEATURES ) {
@@ -473,7 +477,8 @@ static void __init ms_hyperv_init_platform(void)#if IS_ENABLED (CONFIG_HYPERV )if ((hv_get_isolation_type () ==HV_ISOLATION_TYPE_VBS ) || - (hv_get_isolation_type () ==HV_ISOLATION_TYPE_SNP )) + ((hv_get_isolation_type () ==HV_ISOLATION_TYPE_SNP ) && + ms_hyperv.paravisor_present))hv_vtom_init ();/* * Setup the hook to get control post apic initialization.
drivers/hv/hv_common.c +6 -0
@@ -502,6 +502,12 @@ bool __weak hv_isolation_type_snp(void) }EXPORT_SYMBOL_GPL (hv_isolation_type_snp); +bool __weak hv_isolation_type_en_snp (void ) +{ +return false; +} +EXPORT_SYMBOL_GPL (hv_isolation_type_en_snp); +void __weak hv_setup_vmbus_handler (void (*handler)(void )) { }
include/asm-generic/mshyperv.h +10 -3
@@ -36,15 +36,21 @@ struct ms_hyperv_info {u32 nested_features;u32 max_vp_index;u32 max_lp_index; -u32 isolation_config_a; +union { +u32 isolation_config_a; +struct { +u32 paravisor_present :1 ; +u32 reserved_a1 :31 ; + }; + };union {u32 isolation_config_b;struct {u32 cvm_type :4 ; -u32 reserved1 :1 ; +u32 reserved_b1 :1 ;u32 shared_gpa_boundary_active :1 ;u32 shared_gpa_boundary_bits :6 ; -u32 reserved2 :20 ; +u32 reserved_b2 :20 ; }; };u64 shared_gpa_boundary;
@@ -58,6 +64,7 @@ extern void * __percpu *hyperv_pcpu_output_arg;extern u64 hv_do_hypercall (u64 control,void *inputaddr,void *outputaddr);extern u64 hv_do_fast_hypercall8 (u16 control,u64 input8);extern bool hv_isolation_type_snp (void ); +extern bool hv_isolation_type_en_snp (void );/* Helper functions that provide a consistent pattern for checking Hyper-V hypercall status. */ static inline int hv_result (u64 status)
Signature · 1 line
2.25.1
RE: [PATCH v6 1/8] x86/hyperv: Add sev-snp enlightened guest static key
Quoted 8 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]
Introduce static keyisolation_type_en_snpfor enlightened sev-snp guest check.Reviewed-by Michael Kelley <[email protected]>Signed-off-by Tianyu Lan <[email protected]>
PATCH v6 2/8 x86/hyperv: Set Virtual Trust Level in VMBus init message
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
SEV-SNP guests on Hyper-V can run at multiple Virtual Trust Levels (VTL). During boot, get the VTL at which we're running using the GET_VP_REGISTERs hypercall, and save the value for future use. Then during VMBus initialization, set the VTL with the saved value as required in the VMBus init message.
---
* Change since v5:
Remove hvcall input parameter page check
* Change since v3:
Call get_vtl() when SEV-SNP is available and set vtl to 0 by default if fail to get VTL from Hyper-V.
* Change since v2:
Update the change log.
--- arch/x86/hyperv/hv_init.c | 34 ++++++++++++++++++++++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 7 ++++++ drivers/hv/connection.c | 1 + include/asm-generic/mshyperv.h | 1 + include/linux/hyperv.h | 4 ++-- 5 files changed, 45 insertions(+), 2 deletions(-)
arch/x86/hyperv/hv_init.c +34 -0
@@ -378,6 +378,36 @@ static void __init hv_get_partition_id(void)local_irq_restore (flags); } +static u8 __init get_vtl (void ) +{ +u64 control =HV_HYPERCALL_REP_COMP_1 |HVCALL_GET_VP_REGISTERS ; +struct hv_get_vp_registers_input *input; +struct hv_get_vp_registers_output *output; +unsigned long flags; +u64 ret =0 ; + +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg); + output = (struct hv_get_vp_registers_output *)input; + +memset (input,0 ,struct_size (input, element,1 )); + input->header.partitionid =HV_PARTITION_ID_SELF ; + input->header.vpindex =HV_VP_INDEX_SELF ; + input->header.inputvtl =0 ; + input->element[0 ].name0 =HV_X64_REGISTER_VSM_VP_STATUS ; + + ret =hv_do_hypercall (control, input, output); +if (hv_result_success (ret)) { + ret = output->as64.low &HV_X64_VTL_MASK ; + }else { +pr_err ("Failed to get VTL(%lld) and set VTL to zero by default.\n" , ret); + ret =0 ; + } + +local_irq_restore (flags); +return ret; +} +/* * This function is to be invoked early in the boot sequence after the * hypervisor has been detected.
@@ -506,6 +536,10 @@ void __init hyperv_init(void)/* Query the VMs extended capability once, so that it can be cached. */ hv_query_ext_cap (0 ); +/* Find the VTL */ +if (hv_isolation_type_en_snp ()) + ms_hyperv.vtl =get_vtl (); +return ; clean_guest_os_id:
arch/x86/include/asm/hyperv-tlfs.h +7 -0
@@ -301,6 +301,13 @@ enum hv_isolation_type {#define HV_X64_MSR_TIME_REF_COUNT HV_REGISTER_TIME_REF_COUNT#define HV_X64_MSR_REFERENCE_TSC HV_REGISTER_REFERENCE_TSC +/* +* Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and +* there is not associated MSR address. +*/ +#define HV_X64_REGISTER_VSM_VP_STATUS 0x000D0003 +#define HV_X64_VTL_MASK GENMASK(3, 0) +/* Hyper-V memory host visibility */ enum hv_mem_host_visibility {VMBUS_PAGE_NOT_VISIBLE =0 ,
drivers/hv/connection.c +1 -0
@@ -98,6 +98,7 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)*/ if (version >=VERSION_WIN10_V5 ) { msg->msg_sint =VMBUS_MESSAGE_SINT ; + msg->msg_vtl = ms_hyperv.vtl; vmbus_connection.msg_conn_id =VMBUS_MESSAGE_CONNECTION_ID_4 ; }else { msg->interrupt_page =virt_to_phys (vmbus_connection.int_page);
include/asm-generic/mshyperv.h +1 -0
@@ -54,6 +54,7 @@ struct ms_hyperv_info { }; };u64 shared_gpa_boundary; +u8 vtl; };extern struct ms_hyperv_info ms_hyperv;extern bool hv_nested;
include/linux/hyperv.h +2 -2
@@ -665,8 +665,8 @@ struct vmbus_channel_initiate_contact {u64 interrupt_page;struct {u8 msg_sint; -u8 padding1[3 ]; -u32 padding2; +u8 msg_vtl; +u8 reserved[6 ]; }; };u64 monitor_page1;
Signature · 1 line
2.25.1
RE: [PATCH v6 2/8] x86/hyperv: Set Virtual Trust Level in VMBus init message
Quoted 5 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]arch/x86/hyperv/hv_init.c +0
-0@@ -378,6 +378,36 @@ static void __init hv_get_partition_id(void)local_irq_restore (flags);
arch/x86/hyperv/hv_init.c static void __init hv_get_partition_id(void) · line 388
Quoted 10 lines
arch/x86/hyperv/hv_init.c +7
-0} +static u8 __init get_vtl (void ) +{ +u64 control =HV_HYPERCALL_REP_COMP_1 |HVCALL_GET_VP_REGISTERS ; +struct hv_get_vp_registers_input *input; +struct hv_get_vp_registers_output *output; +unsigned long flags; +u64 ret =0 ;
Nitpick: we don't have to initialize 'ret' here.
arch/x86/hyperv/hv_init.c static void __init hv_get_partition_id(void) · line 399
Quoted 12 lines
arch/x86/hyperv/hv_init.c +12
-0+ +local_irq_save (flags); + input = *this_cpu_ptr (hyperv_pcpu_input_arg); + output = (struct hv_get_vp_registers_output *)input; + +memset (input,0 ,struct_size (input, element,1 )); + input->header.partitionid =HV_PARTITION_ID_SELF ; + input->header.vpindex =HV_VP_INDEX_SELF ; + input->header.inputvtl =0 ; + input->element[0 ].name0 =HV_X64_REGISTER_VSM_VP_STATUS ; + + ret =hv_do_hypercall (control, input, output);
'ret' is always initialized here.
Quoted 11 lines
arch/x86/hyperv/hv_init.c +10
-0+if (hv_result_success (ret)) { + ret = output->as64.low &HV_X64_VTL_MASK ; + }else { + pr_err("Failed to get VTL(%lld) and set VTL to zero by default.\n", ret); + ret =0 ; + } + +local_irq_restore (flags); +return ret; +}
PATCH v6 3/8 x86/hyperv: Mark Hyper-V vp assist page unencrypted in SEV-SNP enlightened guest
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
hv vp assist page needs to be shared between SEV-SNP guest and Hyper-V. So mark the page unencrypted in the SEV-SNP guest.
--- arch/x86/hyperv/hv_init.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
arch/x86/hyperv/hv_init.c +15 -1
@@ -18,6 +18,7 @@#include <asm/hyperv-tlfs.h> #include <asm/mshyperv.h> #include <asm/idtentry.h> +#include <asm/set_memory.h> #include <linux/kexec.h> #include <linux/version.h> #include <linux/vmalloc.h>
@@ -106,8 +107,21 @@ static int hv_cpu_init(unsigned int cpu)* in hv_cpu_die(), otherwise a CPU may not be stopped in the * case of CPU offlining and the VM will hang. */ -if (!*hvp) +if (!*hvp) { *hvp =__vmalloc (PAGE_SIZE ,GFP_KERNEL | __GFP_ZERO); + +/* +* Hyper-V should never specify a VM that is a Confidential +* VM and also running in the root partition. Root partition +* is blocked to run in Confidential VM. So only decrypt assist +* page in non-root partition here. +*/ +if (*hvp &&hv_isolation_type_en_snp ()) { +WARN_ON_ONCE (set_memory_decrypted ((unsigned long )(*hvp),1 )); +memset (*hvp,0 ,PAGE_SIZE ); + } + } +if (*hvp) msr.pfn =vmalloc_to_pfn (*hvp);
Signature · 1 line
2.25.1
RE: [PATCH v6 3/8] x86/hyperv: Mark Hyper-V vp assist page unencrypted in SEV-SNP enlightened guest
Quoted 9 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]
hv vp assist page needs to be shared between SEV-SNP guest and Hyper-V. So mark the page unencrypted in the SEV-SNP guest.Reviewed-by Michael Kelley <[email protected]>Signed-off-by Tianyu Lan <[email protected]>---
PATCH v6 4/8 drivers: hv: Mark percpu hvcall input arg page unencrypted in SEV-SNP enlightened guest
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Hypervisor needs to access input arg, VMBus synic event and message pages. Mark these pages unencrypted in the SEV-SNP guest and free them only if they have been marked encrypted successfully.
--- drivers/hv/hv.c | 57 +++++++++++++++++++++++++++++++++++++++--- drivers/hv/hv_common.c | 13 ++++++++++ 2 files changed, 67 insertions(+), 3 deletions(-)
drivers/hv/hv.c +54 -3
@@ -20,6 +20,7 @@#include <linux/interrupt.h> #include <clocksource/hyperv_timer.h> #include <asm/mshyperv.h> +#include <linux/set_memory.h> #include "hyperv_vmbus.h" /* The one and only */
@@ -78,7 +79,7 @@ int hv_post_message(union hv_connection_id connection_id,int hv_synic_alloc (void ) { -int cpu; +int cpu, ret = -ENOMEM ;struct hv_per_cpu_context *hv_cpu;/*
@@ -123,26 +124,76 @@ int hv_synic_alloc(void) goto err; } } + +if (hv_isolation_type_en_snp ()) { + ret =set_memory_decrypted ((unsigned long ) + hv_cpu->synic_message_page,1 ); +if (ret) { +pr_err ("Failed to decrypt SYNIC msg page: %d\n" , ret); + hv_cpu->synic_message_page =NULL ; + +/* +* Free the event page here so that hv_synic_free() +* won't later try to re-encrypt it. +*/ +free_page ((unsigned long )hv_cpu->synic_event_page); + hv_cpu->synic_event_page =NULL ; + goto err; + } + + ret =set_memory_decrypted ((unsigned long ) + hv_cpu->synic_event_page,1 ); +if (ret) { +pr_err ("Failed to decrypt SYNIC event page: %d\n" , ret); + hv_cpu->synic_event_page =NULL ; + goto err; + } + +memset (hv_cpu->synic_message_page,0 ,PAGE_SIZE ); +memset (hv_cpu->synic_event_page,0 ,PAGE_SIZE ); + } }return 0 ; + err:/* * Any memory allocations that succeeded will be freed when * the caller cleans up by calling hv_synic_free() */ -return -ENOMEM ; +return ret; }void hv_synic_free (void ) { -int cpu; +int cpu, ret;for_each_present_cpu (cpu) {struct hv_per_cpu_context *hv_cpu =per_cpu_ptr (hv_context.cpu_context, cpu); +/* It's better to leak the page if the encryption fails. */ +if (hv_isolation_type_en_snp ()) { +if (hv_cpu->synic_message_page) { + ret =set_memory_encrypted ((unsigned long ) + hv_cpu->synic_message_page,1 ); +if (ret) { +pr_err ("Failed to encrypt SYNIC msg page: %d\n" , ret); + hv_cpu->synic_message_page =NULL ; + } + } + +if (hv_cpu->synic_event_page) { + ret =set_memory_encrypted ((unsigned long ) + hv_cpu->synic_event_page,1 ); +if (ret) { +pr_err ("Failed to encrypt SYNIC event page: %d\n" , ret); + hv_cpu->synic_event_page =NULL ; + } + } + } +free_page ((unsigned long )hv_cpu->synic_event_page);free_page ((unsigned long )hv_cpu->synic_message_page); }
drivers/hv/hv_common.c +13 -0
@@ -24,6 +24,7 @@#include <linux/kmsg_dump.h> #include <linux/slab.h> #include <linux/dma-map-ops.h> +#include <linux/set_memory.h> #include <asm/hyperv-tlfs.h> #include <asm/mshyperv.h>
@@ -359,6 +360,7 @@ int hv_common_cpu_init(unsigned int cpu)u64 msr_vp_index;gfp_t flags;int pgcount = hv_root_partition ?2 :1 ; +int ret;/* hv_cpu_init() can be called with IRQs disabled from hv_resume() */ flags =irqs_disabled () ?GFP_ATOMIC :GFP_KERNEL ;
@@ -378,6 +380,17 @@ int hv_common_cpu_init(unsigned int cpu) outputarg = (void **)this_cpu_ptr (hyperv_pcpu_output_arg); *outputarg = (char *)(*inputarg) +HV_HYP_PAGE_SIZE ; } + +if (hv_isolation_type_en_snp ()) { + ret =set_memory_decrypted ((unsigned long )*inputarg, pgcount); +if (ret) { +kfree (*inputarg); + *inputarg =NULL ; +return ret; + } + +memset (*inputarg,0x00 , pgcount *PAGE_SIZE ); + } } msr_vp_index =hv_get_register (HV_REGISTER_VP_INDEX );
Signature · 1 line
2.25.1
RE: [PATCH v6 4/8] drivers: hv: Mark percpu hvcall input arg page unencrypted in SEV-SNP enlightened guest
Quoted 11 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]
Hypervisor needs to access input arg, VMBus synic event and message pages. Mark these pages unencrypted in the SEV-SNP guest and free them only if they have been marked encrypted successfully.Reviewed-by Michael Kelley <[email protected]>Signed-off-by Tianyu Lan <[email protected]>---
PATCH v6 5/8 x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp enlightened guest
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
In sev-snp enlightened guest, Hyper-V hypercall needs to use vmmcall to trigger vmexit and notify hypervisor to handle hypercall request.
--- arch/x86/include/asm/mshyperv.h | 44 ++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 11 deletions(-)
arch/x86/include/asm/mshyperv.h +33 -11
@@ -59,16 +59,25 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output)u64 hv_status;#ifdef CONFIG_X86_64 -if (!hv_hypercall_pg) -return U64_MAX ; +if (hv_isolation_type_en_snp ()) { + __asm__ __volatile__("mov %4, %%r8\n" +"vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, +"+c" (control),"+d" (input_address) + :"r" (output_address) + :"cc" ,"memory" ,"r8" ,"r9" ,"r10" ,"r11" ); + }else { +if (!hv_hypercall_pg) +return U64_MAX ; - __asm__ __volatile__("mov %4, %%r8\n" -CALL_NOSPEC - : "=a" (hv_status), ASM_CALL_CONSTRAINT, -"+c" (control),"+d" (input_address) - : "r" (output_address), - THUNK_TARGET(hv_hypercall_pg) - :"cc" ,"memory" ,"r8" ,"r9" ,"r10" ,"r11" ); + __asm__ __volatile__("mov %4, %%r8\n" +CALL_NOSPEC + : "=a" (hv_status), ASM_CALL_CONSTRAINT, +"+c" (control),"+d" (input_address) + : "r" (output_address), + THUNK_TARGET(hv_hypercall_pg) + :"cc" ,"memory" ,"r8" ,"r9" ,"r10" ,"r11" ); + }#else u32 input_address_hi =upper_32_bits (input_address);u32 input_address_lo =lower_32_bits (input_address);
@@ -102,7 +111,13 @@ static inline u64 _hv_do_fast_hypercall8(u64 control, u64 input1)u64 hv_status;#ifdef CONFIG_X86_64 - { +if (hv_isolation_type_en_snp ()) { + __asm__ __volatile__( +"vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, +"+c" (control),"+d" (input1) + ::"cc" ,"r8" ,"r9" ,"r10" ,"r11" ); + }else { __asm__ __volatile__(CALL_NOSPEC : "=a" (hv_status), ASM_CALL_CONSTRAINT,"+c" (control),"+d" (input1)
@@ -147,7 +162,14 @@ static inline u64 _hv_do_fast_hypercall16(u64 control, u64 input1, u64 input2)u64 hv_status;#ifdef CONFIG_X86_64 - { +if (hv_isolation_type_en_snp ()) { + __asm__ __volatile__("mov %4, %%r8\n" +"vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, +"+c" (control),"+d" (input1) + :"r" (input2) + :"cc" ,"r8" ,"r9" ,"r10" ,"r11" ); + }else { __asm__ __volatile__("mov %4, %%r8\n" CALL_NOSPEC : "=a" (hv_status), ASM_CALL_CONSTRAINT,
Signature · 1 line
2.25.1
Dexuan Cui
maintainerRE: [PATCH v6 5/8] x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp enlightened guest
Quoted 38 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
To: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui
[...]
In sev-snp enlightened guest, Hyper-V hypercall needs to use vmmcall to trigger vmexit and notify hypervisor to handle hypercall request.Signed-off-by Tianyu Lan <[email protected]>---
arch/x86/include/asm/mshyperv.h +10
-8@@ -59,16 +59,25 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output)u64 hv_status;#ifdef CONFIG_X86_64 -if (!hv_hypercall_pg) -return U64_MAX ; +if (hv_isolation_type_en_snp ()) { + __asm__ __volatile__("mov %4, %%r8\n" +"vmmcall" + :"=a" (hv_status), ASM_CALL_CONSTRAINT, +"+c" (control),"+d" (input_address) + :"r" (output_address) + :"cc" ,"memory" ,"r8" ,"r9" ,"r10" ,"r11" ); + }else { +if (!hv_hypercall_pg) +return U64_MAX ; - __asm__ __volatile__("mov %4, %%r8\n" -CALL_NOSPEC - : "=a" (hv_status), ASM_CALL_CONSTRAINT, -"+c" (control),"+d" (input_address) - : "r" (output_address), - THUNK_TARGET(hv_hypercall_pg)
arch/x86/include/asm/mshyperv.h static inline u64 hv_do_hypercall(u64 control, void · line 83
Quoted 10 lines
arch/x86/include/asm/mshyperv.h +8
-1- : "cc", "memory", "r8", "r9", "r10", "r11"); + __asm__ __volatile__("mov %4, %%r8\n" +CALL_NOSPEC + :"=a" (hv_status), ASM_CALL_CONSTRAINT, +"+c" (control),"+d" (input_address) + : "r" (output_address), + THUNK_TARGET(hv_hypercall_pg) + :"cc" ,"memory" ,"r8" ,"r9" ,"r10" ,"r11" ); + }
IMO it's better if we add a "return hv_status;" for the SNP case, and don't move the assembly code for the regular VM. I made a patch:
https://github.com/dcui/tdx/commit/f81013578605aa02939a3186afa9fc76791b3acd
You may want to explain briefly why the earlier approach
ALTERNATIVE(CALL_NOSPEC, "vmmcall", X86_FEATURE_SEV_ES)
doesn't work:
start_kernel() calls hyperv_init() before alternative_instructions(), and hyperv_init() already uses hypercalls, e.g. the newly-added get_vtl() in your patch 2.
start_kernel:
late_time_init
x86_late_time_init
x86_init.irqs.intr_mode_init
apic_intr_mode_init
x86_platform.apic_post_init
hyperv_init ==> it already uses hypercalls, e.g. the newly-added get_vtl() in your patch 2. arch_cpu_finalize_init()
alternative_instructions()
We can move the get_vtl hypercall to a later place, but there are also other hypercalls before alternative_instructions(). IMO it may be unsafe to run ALTERNATIVE code before alternative_instructions() is called.
Dexuan Cui
maintainerRE: [PATCH v6 5/8] x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp enlightened guest
In reply to Tianyu Lanarch/x86/include/asm/mshyperv.h static inline u64 hv_do_hypercall(u64 control, void · line 63
Quoted 13 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]arch/x86/include/asm/mshyperv.h +1
-2@@ -59,16 +59,25 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output)u64 hv_status;#ifdef CONFIG_X86_64 -if (!hv_hypercall_pg) -return U64_MAX ; +if (hv_isolation_type_en_snp ()) {
I got a build failure:
In file included from arch/x86/hyperv/hv_spinlock.c:15:./arch/x86/include/asm/mshyperv.h: In function 'hv_do_hypercall':./arch/x86/include/asm/mshyperv.h:69:6: error: implicit declaration of function 'hv_isolation_type_en_snp' [-Werror=implicit-function-declaration]
69 | if (hv_isolation_type_en_snp()) {
| ^~~~~~~~~~~~~~~~~~~~~~~~In arch/x86/include/asm/mshyperv.h, we do have
extern bool hv_isolation_type_en_snp(void);
but it's defined at a late place. I think we need to move it to before hv_do_hypercall().
We also have
extern bool hv_isolation_type_en_snp(void);
in include/asm-generic/mshyperv.h, but that header file is included at the end of arch/x86/include/asm/mshyperv.h.
Quoted 7 lines
arch/x86/include/asm/mshyperv.h +6
-0+ __asm__ __volatile__("mov %4, %%r8\n" +"vmmcall" + :"=a" (hv_status), ASM_CALL_CONSTRAINT, +"+c" (control),"+d" (input_address) + :"r" (output_address) + :"cc" ,"memory" ,"r8" ,"r9" ,"r10" ,"r11" );
PATCH v6 6/8 clocksource: hyper-v: Mark hyperv tsc page unencrypted in sev-snp enlightened guest
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Hyper-V tsc page is shared with hypervisor and mark the page unencrypted in sev-snp enlightened guest when it's used.
--- drivers/clocksource/hyperv_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/clocksource/hyperv_timer.c +1 -1
@@ -390,7 +390,7 @@ static __always_inline u64 read_hv_clock_msr(void)static union {struct ms_hyperv_tsc_page page;u8 reserved[PAGE_SIZE ]; -} tsc_pg__aligned(PAGE_SIZE) ; +} tsc_pg __bss_decrypted __aligned(PAGE_SIZE);static struct ms_hyperv_tsc_page *tsc_page = &tsc_pg.page;static unsigned long tsc_pfn;
Signature · 1 line
2.25.1
RE: [PATCH v6 6/8] clocksource: hyper-v: Mark hyperv tsc page unencrypted in sev-snp enlightened guest
Quoted 8 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]
Hyper-V tsc page is shared with hypervisor and mark the page unencrypted in sev-snp enlightened guest when it's used.Reviewed-by Michael Kelley <[email protected]>Signed-off-by Tianyu Lan <[email protected]>
PATCH v6 7/8 x86/hyperv: Add smp support for SEV-SNP guest
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
In the AMD SEV-SNP guest, AP needs to be started up via sev es save area and Hyper-V requires to call HVCALL_START_VP hypercall to pass the gpa of sev es save area with AP's vp index and VTL(Virtual trust level) parameters. Override wakeup_secondary_cpu_64 callback with hv_snp_boot_ap.
--- arch/x86/hyperv/ivm.c | 138 ++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 9 ++ arch/x86/kernel/cpu/mshyperv.c | 13 ++- include/asm-generic/hyperv-tlfs.h | 1 + 4 files changed, 159 insertions(+), 2 deletions(-)
arch/x86/hyperv/ivm.c +138 -0
@@ -18,11 +18,20 @@#include <asm/mshyperv.h> #include <asm/hypervisor.h> #include <asm/mtrr.h> +#include <asm/coco.h> +#include <asm/io_apic.h> +#include <asm/sev.h> +#include <asm/realmode.h> +#include <asm/e820/api.h> +#include <asm/desc.h> #ifdef CONFIG_AMD_MEM_ENCRYPT #define GHCB_USAGE_HYPERV_CALL 1 +static u8 ap_start_input_arg[PAGE_SIZE] __bss_decrypted __aligned(PAGE_SIZE); +static u8 ap_start_stack[PAGE_SIZE] __aligned(PAGE_SIZE); + union hv_ghcb { struct ghcb ghcb; struct {
@@ -56,6 +65,8 @@ union hv_ghcb { } hypercall; } __packed__aligned(HV_HYP_PAGE_SIZE) ; +static DEFINE_PER_CPU(struct sev_es_save_area *, hv_sev_vmsa); +static u16 hv_ghcb_version__ro_after_init ;u64 hv_ghcb_hypercall (u64 control,void *input,void *output,u32 input_size)
@@ -357,6 +368,133 @@ static bool hv_is_private_mmio(u64 addr)return false; } +#define hv_populate_vmcb_seg (seg, gdtr_base) \ +do { \ + if (seg.selector) { \ + seg.base = 0; \ + seg.limit = HV_AP_SEGMENT_LIMIT; \ + seg.attrib = *(u16 *)(gdtr_base + seg.selector + 5); \ + seg.attrib = (seg.attrib & 0xFF) | ((seg.attrib >> 4) & 0xF00); \ + } \ +} while (0) \ + +static int snp_set_vmsa (void *va,bool vmsa) +{ +u64 attrs; + +/* +* Running at VMPL0 allows the kernel to change the VMSA bit for a page +* using the RMPADJUST instruction. However, for the instruction to +* succeed it must target the permissions of a lesser privileged +* (higher numbered) VMPL level, so use VMPL1 (refer to the RMPADJUST +* instruction in the AMD64 APM Volume 3). +*/ + attrs =1 ; +if (vmsa) + attrs |=RMPADJUST_VMSA_PAGE_BIT ; + +return rmpadjust ((unsigned long )va,RMP_PG_SIZE_4K , attrs); +} + +static void snp_cleanup_vmsa (struct sev_es_save_area *vmsa) +{ +int err; + + err =snp_set_vmsa (vmsa, false); +if (err) +pr_err ("clear VMSA page failed (%u), leaking page\n" , err); +else +free_page ((unsigned long )vmsa); +} + +int hv_snp_boot_ap (int cpu,unsigned long start_ip) +{ +struct sev_es_save_area *vmsa = (struct sev_es_save_area *) +__get_free_page (GFP_KERNEL | __GFP_ZERO); +struct sev_es_save_area *cur_vmsa; +struct desc_ptr gdtr; +u64 ret, retry =5 ; +struct hv_enable_vp_vtl *start_vp_input; +unsigned long flags; + +if (!vmsa) +return -ENOMEM ; + +native_store_gdt (&gdtr); + + vmsa->gdtr.base = gdtr.address; + vmsa->gdtr.limit = gdtr.size; + + asmvolatile ("movl %%es, %%eax;" :"=a" (vmsa->es.selector)); +hv_populate_vmcb_seg (vmsa->es, vmsa->gdtr.base); + + asmvolatile ("movl %%cs, %%eax;" :"=a" (vmsa->cs.selector)); +hv_populate_vmcb_seg (vmsa->cs, vmsa->gdtr.base); + + asmvolatile ("movl %%ss, %%eax;" :"=a" (vmsa->ss.selector)); +hv_populate_vmcb_seg (vmsa->ss, vmsa->gdtr.base); + + asmvolatile ("movl %%ds, %%eax;" :"=a" (vmsa->ds.selector)); +hv_populate_vmcb_seg (vmsa->ds, vmsa->gdtr.base); + + vmsa->efer =native_read_msr (MSR_EFER ); + + asmvolatile ("movq %%cr4, %%rax;" :"=a" (vmsa->cr4)); + asmvolatile ("movq %%cr3, %%rax;" :"=a" (vmsa->cr3)); + asmvolatile ("movq %%cr0, %%rax;" :"=a" (vmsa->cr0)); + + vmsa->xcr0 =1 ; + vmsa->g_pat =HV_AP_INIT_GPAT_DEFAULT ; + vmsa->rip = (u64 )secondary_startup_64_no_verify; + vmsa->rsp = (u64)&ap_start_stack[PAGE_SIZE ]; + +/* +* Set the SNP-specific fields for this VMSA: +* VMPL level +* SEV_FEATURES (matches the SEV STATUS MSR right shifted 2 bits) +*/ + vmsa->vmpl =0 ; + vmsa->sev_features = sev_status >>2 ; + + ret =snp_set_vmsa (vmsa, true); +if (!ret) { +pr_err ("RMPADJUST(%llx) failed: %llx\n" , (u64 )vmsa, ret); +free_page ((u64 )vmsa); +return ret; + } + +local_irq_save (flags); + start_vp_input = (struct hv_enable_vp_vtl *)ap_start_input_arg; +memset (start_vp_input,0 ,sizeof (*start_vp_input)); + start_vp_input->partition_id =-1 ; + start_vp_input->vp_index = cpu; + start_vp_input->target_vtl.target_vtl = ms_hyperv.vtl; + *(u64 *)&start_vp_input->vp_context =__pa (vmsa) |1 ; + +do { + ret =hv_do_hypercall (HVCALL_START_VP , + start_vp_input,NULL ); + }while (hv_result (ret) ==HV_STATUS_TIME_OUT && retry--); + +local_irq_restore (flags); + +if (!hv_result_success (ret)) { +pr_err ("HvCallStartVirtualProcessor failed: %llx\n" , ret); +snp_cleanup_vmsa (vmsa); + vmsa =NULL ; + } + + cur_vmsa =per_cpu (hv_sev_vmsa, cpu); +/* Free up any previous VMSA page */ +if (cur_vmsa) +snp_cleanup_vmsa (cur_vmsa); + +/* Record the current VMSA page */ +per_cpu (hv_sev_vmsa, cpu) = vmsa; + +return ret; +} +void __init hv_vtom_init (void ) {/*
arch/x86/include/asm/mshyperv.h +9 -0
@@ -48,6 +48,13 @@ extern u64 hv_current_partition_id;extern union hv_ghcb *__percpu *hv_ghcb_pg; +/* +* DEFAULT INIT GPAT and SEGMENT LIMIT value in struct VMSA +* to start AP in enlightened SEV guest. +*/ +#define HV_AP_INIT_GPAT_DEFAULT 0x0007040600070406ULL +#define HV_AP_SEGMENT_LIMIT 0xffffffff +int hv_call_deposit_pages (int node,u64 partition_id,u32 num_pages);int hv_call_add_logical_proc (int node,u32 lp_index,u32 acpi_id);int hv_call_create_vp (int node,u64 partition_id,u32 vp_index,u32 flags);
@@ -253,12 +260,14 @@ void hv_ghcb_msr_read(u64 msr, u64 *value);bool hv_ghcb_negotiate_protocol (void );void __noreturn hv_ghcb_terminate (unsigned int set,unsigned int reason);void hv_vtom_init (void ); +int hv_snp_boot_ap (int cpu,unsigned long start_ip);#else static inline void hv_ghcb_msr_write (u64 msr,u64 value) {}static inline void hv_ghcb_msr_read (u64 msr,u64 *value) {}static inline bool hv_ghcb_negotiate_protocol (void ) {return false; }static inline void hv_ghcb_terminate (unsigned int set,unsigned int reason) {}static inline void hv_vtom_init (void ) {} +static int hv_snp_boot_ap (int cpu,unsigned long start_ip) {return 0 ; }#endif extern bool hv_isolation_type_snp (void );
arch/x86/kernel/cpu/mshyperv.c +11 -2
@@ -295,6 +295,16 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus)native_smp_prepare_cpus (max_cpus); +/* +* Override wakeup_secondary_cpu_64 callback for SEV-SNP +* enlightened guest. +*/ +if (hv_isolation_type_en_snp ()) + apic->wakeup_secondary_cpu_64 = hv_snp_boot_ap; + +if (!hv_root_partition) +return ; +#ifdef CONFIG_X86_64 for_each_present_cpu (i) {if (i ==0 )
@@ -502,8 +512,7 @@ static void __init ms_hyperv_init_platform(void)# ifdef CONFIG_SMP smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu; -if (hv_root_partition) - smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus; + smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;# endif /*
include/asm-generic/hyperv-tlfs.h +1 -0
@@ -223,6 +223,7 @@ enum HV_GENERIC_SET_FORMAT {#define HV_STATUS_INVALID_PORT_ID 17#define HV_STATUS_INVALID_CONNECTION_ID 18#define HV_STATUS_INSUFFICIENT_BUFFERS 19 +#define HV_STATUS_TIME_OUT 120#define HV_STATUS_VTL_ALREADY_ENABLED 134/*
Signature · 1 line
2.25.1
Dexuan Cui
maintainerRE: [PATCH v6 7/8] x86/hyperv: Add smp support for SEV-SNP guest
Quoted 6 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]
arch/x86/kernel/cpu/mshyperv.c static void __init hv_smp_prepare_cpus(unsigned · line 306
Quoted 9 lines
arch/x86/kernel/cpu/mshyperv.c +5
-0@@ -295,6 +295,16 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus)native_smp_prepare_cpus (max_cpus ); +if (hv_isolation_type_en_snp ()) + apic->wakeup_secondary_cpu_64 = hv_snp_boot_ap; + +if (!hv_root_partition) +return ;
Can the above be changed to:
if (hv_isolation_type_en_snp()) {
apic->wakeup_secondary_cpu_64 = hv_snp_boot_ap;
return;
}(please also see the below)
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 515
Quoted 11 lines
arch/x86/kernel/cpu/mshyperv.c +2
-2+#ifdef CONFIG_X86_64 for_each_present_cpu (i) {if (i ==0 )@@ -502,8 +512,7 @@ static void __init ms_hyperv_init_platform(void)# ifdef CONFIG_SMP smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu; -if (hv_root_partition) - smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus; + smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
IMO it's better if we don't unconditionally change smp_ops.smp_prepare_cpus.
How about this:
if (hv_root_partition || hv_isolation_type_en_snp()) smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
PATCH v6 8/8 x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES
In reply to Tianyu LanFrom: Tianyu Lan <[email protected]>
Add Hyperv-specific handling for faults caused by VMMCALL instructions.
--- arch/x86/kernel/cpu/mshyperv.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
arch/x86/kernel/cpu/mshyperv.c +21 -0
@@ -32,6 +32,7 @@#include <asm/nmi.h> #include <clocksource/hyperv_timer.h> #include <asm/numa.h> +#include <asm/svm.h> /* Is Linux running as the root partition? */ bool hv_root_partition;
@@ -574,6 +575,22 @@ static bool __init ms_hyperv_msi_ext_dest_id(void)return eax &HYPERV_VS_PROPERTIES_EAX_EXTENDED_IOAPIC_RTE ; } +#ifdef CONFIG_AMD_MEM_ENCRYPT +static void hv_sev_es_hcall_prepare (struct ghcb *ghcb,struct pt_regs *regs) +{ +/* RAX and CPL are already in the GHCB */ +ghcb_set_rcx (ghcb, regs->cx); +ghcb_set_rdx (ghcb, regs->dx); +ghcb_set_r8 (ghcb, regs->r8); +} + +static bool hv_sev_es_hcall_finish (struct ghcb *ghcb,struct pt_regs *regs) +{ +/* No checking of the return state needed */ +return true; +} +#endif +const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv = { .name ="Microsoft Hyper-V" , .detect = ms_hyperv_platform,
@@ -581,4 +598,8 @@ const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv = { .init.x2apic_available = ms_hyperv_x2apic_available, .init.msi_ext_dest_id = ms_hyperv_msi_ext_dest_id, .init.init_platform = ms_hyperv_init_platform, +#ifdef CONFIG_AMD_MEM_ENCRYPT + .runtime.sev_es_hcall_prepare = hv_sev_es_hcall_prepare, + .runtime.sev_es_hcall_finish = hv_sev_es_hcall_finish, +#endif };
Signature · 1 line
2.25.1
RE: [PATCH v6 8/8] x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES
Quoted 8 lines
From: Tianyu Lan <[email protected]>
Sent: Wednesday, August 16, 2023 8:59 AM
[...]
Add Hyperv-specific handling for faults caused by VMMCALL instructions.Reviewed-by Michael Kelley <[email protected]>Signed-off-by Tianyu Lan <[email protected]>