Hyper-V VTL support
Subsystems (4)
v5 cover Hyper-V VTL support
This patch series introduces support for Virtual Trust Level (VTL) in Hyper-V systems. It provide a foundation for the implementation of Hyper-V VSM support in the Linux kernel, providing a secure platform for the development and deployment of applications.
Virtual Secure Mode (VSM) is a critical aspect of the security infrastructure in Hyper-V systems. It provides a set of hypervisor capabilities and enlightenments that enable the creation and
management of new security boundaries within operating system
software. The VSM achieves and maintains isolation through Virtual
Trust Levels, which are hierarchical, with higher levels being more privileged than lower levels. Please refer to this link for further
information: https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/vsm
This patch series adds the initialization of the x86 platform for VTL systems. This also adds the VTL early bootup code for initializing and bringing up secondary cpus to targeted VTL context. In VTL, AP has to start directly in the 64-bit mode, bypassing the usual 16-bit -> 32-bit -> 64-bit mode transition sequence that occurs after waking up an AP with SIPI whose vector points to the 16-bit AP startup trampoline code.
Currently only VTL level supprted is '2'. This patch series is tested extensively on VTL2 systems.
[V5]
- __u64/__u16 -> u64/u16
[V4]
- Move HYPERV_VTL_MODE definition from arch/x86/Kconfig to drivers/hv/Kconfig
- Move Kconfig changes before its getting used
- Replace initial_stack with current->thread.sp as per recent upstream changes
[V3]
- Break in to 5 patches
- hv_init_vp_context_t -> hv_init_vp_context
- HYPERV_VTL -> HYPERV_VTL_MODE
- Modify description of HYPERV_VTL_MODE
- VTL 0 and VTL 2 -> VTL0 and VTL2
- Remove casting for this_cpu_ptr pointer
[V2]
- Remove the code for reserve 1 IRQ.
- boot_cpu_has -> cpu_feature_enabled.
- Improved commit message for 0002 patch.
- Improved Kconfig flag description for HYPERV_VTL.
- Removed hv_result as a wrapper around hv_do_hypercall().
- The value of output[0] copied to a local variable before returning.
Saurabh Sengar (5):
x86/init: Make get/set_rtc_noop() public x86/hyperv: Add VTL specific structs and hypercalls x86/hyperv: Make hv_get_nmi_reason public
Drivers: hv: Kconfig: Add HYPERV_VTL_MODE x86/hyperv: VTL support for Hyper-V
arch/x86/hyperv/Makefile | 1 + arch/x86/hyperv/hv_vtl.c | 227 +++++++++++++++++++++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 75 ++++++++++ arch/x86/include/asm/mshyperv.h | 15 ++ arch/x86/include/asm/x86_init.h | 2 + arch/x86/kernel/cpu/mshyperv.c | 6 +- arch/x86/kernel/x86_init.c | 4 +- drivers/hv/Kconfig | 24 +++ include/asm-generic/hyperv-tlfs.h | 4 + 9 files changed, 351 insertions(+), 7 deletions(-) create mode 100644 arch/x86/hyperv/hv_vtl.c
Signature · 1 line
2.34.1
PATCH v5 1/5 x86/init: Make get/set_rtc_noop() public
Make get/set_rtc_noop() to be public so that they can be used in other modules as well.
--- arch/x86/include/asm/x86_init.h | 2 ++ arch/x86/kernel/x86_init.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/include/asm/x86_init.h +2 -0
@@ -330,5 +330,7 @@ extern void x86_init_uint_noop(unsigned int unused);extern bool bool_x86_init_noop (void );extern void x86_op_int_noop (int cpu);extern bool x86_pnpbios_disabled (void ); +extern int set_rtc_noop (const struct timespec64 *now); +extern void get_rtc_noop (struct timespec64 *now);#endif
arch/x86/kernel/x86_init.c +2 -2
@@ -33,8 +33,8 @@ static int __init iommu_init_noop(void) { return 0; }static void iommu_shutdown_noop (void ) { }bool __init bool_x86_init_noop (void ) {return false; }void x86_op_int_noop (int cpu) { } -static __init int set_rtc_noop (const struct timespec64 *now) {return -EINVAL ; } -static __init void get_rtc_noop (struct timespec64 *now) { } +int set_rtc_noop (const struct timespec64 *now) {return -EINVAL ; } +void get_rtc_noop (struct timespec64 *now) { }static __initconst const struct of_device_id of_cmos_match[] = { { .compatible ="motorola,mc146818" },
Signature · 1 line
2.34.1
PATCH v5 3/5 x86/hyperv: Make hv_get_nmi_reason public
In reply to Saurabh SengarMove hv_get_nmi_reason to .h file so it can be used in other modules as well.
--- arch/x86/include/asm/mshyperv.h | 5 +++++ arch/x86/kernel/cpu/mshyperv.c | 5 ----- 2 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/include/asm/mshyperv.h +5 -0
@@ -189,6 +189,11 @@ static inline struct hv_vp_assist_page *hv_get_vp_assist_page(unsigned int cpu)return hv_vp_assist_page[cpu]; } +static inline unsigned char hv_get_nmi_reason (void ) +{ +return 0 ; +} +void __init hyperv_init (void );void hyperv_setup_mmu_ops (void );void set_hv_tscchange_cb (void (*cb)(void ));
arch/x86/kernel/cpu/mshyperv.c +0 -5
@@ -249,11 +249,6 @@ static uint32_t __init ms_hyperv_platform(void)return HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS ; } -static unsigned char hv_get_nmi_reason (void ) -{ -return 0 ; -} -#ifdef CONFIG_X86_LOCAL_APIC /* * Prior to WS2016 Debug-VM sends NMIs to all CPUs which makes
Signature · 1 line
2.34.1
PATCH v5 2/5 x86/hyperv: Add VTL specific structs and hypercalls
In reply to Saurabh SengarAdd structs and hypercalls required to enable VTL support on x86.
---
[V5]
- __u64/__u16 -> u64/u16
arch/x86/include/asm/hyperv-tlfs.h | 75 ++++++++++++++++++++++++++++++ include/asm-generic/hyperv-tlfs.h | 4 ++ 2 files changed, 79 insertions(+)
arch/x86/include/asm/hyperv-tlfs.h +75 -0
@@ -713,6 +713,81 @@ union hv_msi_entry { } __packed; }; +struct hv_x64_segment_register { +u64 base; +u32 limit; +u16 selector; +union { +struct { +u16 segment_type :4 ; +u16 non_system_segment :1 ; +u16 descriptor_privilege_level :2 ; +u16 present :1 ; +u16 reserved :4 ; +u16 available :1 ; +u16 _long :1 ; +u16 _default :1 ; +u16 granularity :1 ; + } __packed; +u16 attributes; + }; +} __packed; + +struct hv_x64_table_register { +u16 pad[3 ]; +u16 limit; +u64 base; +} __packed; + +struct hv_init_vp_context { +u64 rip; +u64 rsp; +u64 rflags; + +struct hv_x64_segment_register cs; +struct hv_x64_segment_register ds; +struct hv_x64_segment_register es; +struct hv_x64_segment_register fs; +struct hv_x64_segment_register gs; +struct hv_x64_segment_register ss; +struct hv_x64_segment_register tr; +struct hv_x64_segment_register ldtr; + +struct hv_x64_table_register idtr; +struct hv_x64_table_register gdtr; + +u64 efer; +u64 cr0; +u64 cr3; +u64 cr4; +u64 msr_cr_pat; +} __packed; + +union hv_input_vtl { +u8 as_uint8; +struct { +u8 target_vtl:4 ; +u8 use_target_vtl:1 ; +u8 reserved_z:3 ; + }; +} __packed; + +struct hv_enable_vp_vtl { +u64 partition_id; +u32 vp_index; +union hv_input_vtl target_vtl; +u8 mbz0; +u16 mbz1; +struct hv_init_vp_context vp_context; +} __packed; + +struct hv_get_vp_from_apic_id_in { +u64 partition_id; +union hv_input_vtl target_vtl; +u8 res[7 ]; +u32 apic_ids[]; +} __packed; +#include <asm-generic/hyperv-tlfs.h> #endif
include/asm-generic/hyperv-tlfs.h +4 -0
@@ -146,6 +146,7 @@ union hv_reference_tsc_msr {/* Declare the various hypercall operations. */ #define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE 0x0002#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST 0x0003 +#define HVCALL_ENABLE_VP_VTL 0x000f#define HVCALL_NOTIFY_LONG_SPIN_WAIT 0x0008#define HVCALL_SEND_IPI 0x000b#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX 0x0013
@@ -165,6 +166,8 @@ union hv_reference_tsc_msr {#define HVCALL_MAP_DEVICE_INTERRUPT 0x007c#define HVCALL_UNMAP_DEVICE_INTERRUPT 0x007d#define HVCALL_RETARGET_INTERRUPT 0x007e +#define HVCALL_START_VP 0x0099 +#define HVCALL_GET_VP_ID_FROM_APIC_ID 0x009a#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0#define HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY 0x00db
@@ -218,6 +221,7 @@ enum HV_GENERIC_SET_FORMAT {#define HV_STATUS_INVALID_PORT_ID 17#define HV_STATUS_INVALID_CONNECTION_ID 18#define HV_STATUS_INSUFFICIENT_BUFFERS 19 +#define HV_STATUS_VTL_ALREADY_ENABLED 134/* * The Hyper-V TimeRefCount register and the TSC
Signature · 1 line
2.34.1
PATCH v5 4/5 Drivers: hv: Kconfig: Add HYPERV_VTL_MODE
In reply to Saurabh SengarAdd HYPERV_VTL_MODE Kconfig flag for VTL mode.
--- drivers/hv/Kconfig | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
drivers/hv/Kconfig +24 -0
@@ -13,6 +13,30 @@ config HYPERV Select this option to run Linux as a Hyper-V client operating system. +config HYPERV_VTL_MODE + bool "Enable Linux to boot in VTL context" + depends on X86_64 && HYPERV + default n + help + Virtual Secure Mode (VSM) is a set of hypervisor capabilities and + enlightenments offered to host and guest partitions which enables + the creation and management of new security boundaries within + operating system software. + + VSM achieves and maintains isolation through Virtual Trust Levels + (VTLs). Virtual Trust Levels are hierarchical, with higher levels + being more privileged than lower levels. VTL0 is the least privileged + level, and currently only other level supported is VTL2. + + Select this option to build a Linux kernel to run at a VTL other than + the normal VTL0, which currently is only VTL2. This option + initializes the x86 platform for VTL2, and adds the ability to boot + secondary CPUs directly into 64-bit context as required for VTLs other + than 0. A kernel built with this option must run at VTL2, and will + not run as a normal guest. + + If unsure, say N + config HYPERV_TIMER def_bool HYPERV && X86
Signature · 1 line
2.34.1
PATCH v5 5/5 x86/hyperv: VTL support for Hyper-V
In reply to Saurabh SengarVirtual Trust Levels (VTL) helps enable Hyper-V Virtual Secure Mode (VSM) feature. VSM is a set of hypervisor capabilities and enlightenments offered to host and guest partitions which enable the creation and management of new security boundaries within operating system software. VSM achieves and maintains isolation through VTLs.
Add early initialization for Virtual Trust Levels (VTL). This includes initializing the x86 platform for VTL and enabling boot support for secondary CPUs to start in targeted VTL context. For now, only enable the code for targeted VTL level as 2.
When starting an AP at a VTL other than VTL0, the AP must start directly in 64-bit mode, bypassing the usual 16-bit -> 32-bit -> 64-bit mode transition sequence that occurs after waking up an AP with SIPI whose vector points to the 16-bit AP startup trampoline code.
--- arch/x86/hyperv/Makefile | 1 + arch/x86/hyperv/hv_vtl.c | 227 ++++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 10 ++ arch/x86/kernel/cpu/mshyperv.c | 1 + 4 files changed, 239 insertions(+) create mode 100644 arch/x86/hyperv/hv_vtl.c
arch/x86/hyperv/Makefile +1 -0
@@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y := hv_init.o mmu.o nested.o irqdomain.o ivm.o obj-$(CONFIG_X86_64) += hv_apic.o hv_proc.o +obj-$(CONFIG_HYPERV_VTL_MODE) += hv_vtl.o ifdef CONFIG_X86_64 obj-$(CONFIG_PARAVIRT_SPINLOCKS) += hv_spinlock.o
arch/x86/hyperv/hv_vtl.c new file +227 -0
227 lines not on this page · raw message
arch/x86/include/asm/mshyperv.h +10 -0
@@ -19,6 +19,10 @@*/ #define HV_IOAPIC_BASE_ADDRESS 0xfec00000 +#define HV_VTL_NORMAL 0x0 +#define HV_VTL_SECURE 0x1 +#define HV_VTL_MGMT 0x2 +union hv_ghcb ;DECLARE_STATIC_KEY_FALSE (isolation_type_snp);
@@ -276,6 +280,12 @@ static inline u64 hv_get_non_nested_register(unsigned int reg) { return 0; }#endif /* CONFIG_HYPERV */ +#ifdef CONFIG_HYPERV_VTL_MODE +void __init hv_vtl_init_platform (void ); +#else +static inline void __init hv_vtl_init_platform (void ) {} +#endif +#include <asm-generic/mshyperv.h> #endif
arch/x86/kernel/cpu/mshyperv.c +1 -0
@@ -519,6 +519,7 @@ static void __init ms_hyperv_init_platform(void)/* Register Hyper-V specific clocksource */ hv_init_clocksource (); +hv_vtl_init_platform ();#endif /* * TSC should be marked as unstable only after Hyper-V
Signature · 1 line
2.34.1
Wei Liu
maintainerRe: [PATCH v5 1/5] x86/init: Make get/set_rtc_noop() public
In reply to Saurabh SengarQuoted 8 lines · Saurabh Sengar
Make get/set_rtc_noop() to be public so that they can be used in other modules as well.
Co-developed-by Tianyu Lan <[email protected]>Signed-off-by Tianyu Lan <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Reviewed-by Wei Liu <[email protected]>Reviewed-by Michael Kelley <[email protected]>
x86 maintainers, can you please ack or nack this patch?
This looks trivially correct, but I don't want to apply this patch without an ack since this is under arch/x86.
Thanks,
Wei.
Quoted 35 lines
--- arch/x86/include/asm/x86_init.h | 2 ++ arch/x86/kernel/x86_init.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)arch/x86/include/asm/x86_init.h +2
-0@@ -330,5 +330,7 @@ extern void x86_init_uint_noop(unsigned int unused);extern bool bool_x86_init_noop (void );extern void x86_op_int_noop (int cpu);extern bool x86_pnpbios_disabled (void ); +extern int set_rtc_noop (const struct timespec64 *now); +extern void get_rtc_noop (struct timespec64 *now);#endif arch/x86/kernel/x86_init.c +2
-2@@ -33,8 +33,8 @@ static int __init iommu_init_noop(void) { return 0; }static void iommu_shutdown_noop (void ) { }bool __init bool_x86_init_noop (void ) {return false; }void x86_op_int_noop (int cpu) { } -static __init int set_rtc_noop (const struct timespec64 *now) {return -EINVAL ; } -static __init void get_rtc_noop (struct timespec64 *now) { } +int set_rtc_noop (const struct timespec64 *now) {return -EINVAL ; } +void get_rtc_noop (struct timespec64 *now) { }static __initconst const struct of_device_id of_cmos_match[] = { { .compatible ="motorola,mc146818" },Signature · 1 line
2.34.1
Re: [PATCH v5 1/5] x86/init: Make get/set_rtc_noop() public
Quoted 11 lines · Wei Liu
Quoted 8 lines · Saurabh Sengar
Make get/set_rtc_noop() to be public so that they can be used in other modules as well.
Co-developed-by Tianyu Lan <[email protected]>Signed-off-by Tianyu Lan <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Reviewed-by Wei Liu <[email protected]>Reviewed-by Michael Kelley <[email protected]>x86 maintainers, can you please ack or nack this patch?
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteWei Liu
maintainerRe: [PATCH v5 1/5] x86/init: Make get/set_rtc_noop() public
Quoted 11 lines · Borislav Petkov
Quoted 10 lines · Wei Liu
Quoted 8 lines · Saurabh Sengar
Make get/set_rtc_noop() to be public so that they can be used in other modules as well.
Co-developed-by Tianyu Lan <[email protected]>Signed-off-by Tianyu Lan <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Reviewed-by Wei Liu <[email protected]>Reviewed-by Michael Kelley <[email protected]>
Quoted 3 lines
Quoted 1 line
x86 maintainers, can you please ack or nack this patch?
Thank you very much!
Borislav Petkov
maintainerRe: [PATCH v5 1/5] x86/init: Make get/set_rtc_noop() public
Quoted 12 lines · Wei Liu
Quoted 11 lines · Borislav Petkov
Quoted 10 lines · Wei Liu
Quoted 8 lines · Saurabh Sengar
Make get/set_rtc_noop() to be public so that they can be used in other modules as well.
Co-developed-by Tianyu Lan <[email protected]>Signed-off-by Tianyu Lan <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Reviewed-by Wei Liu <[email protected]>Reviewed-by Michael Kelley <[email protected]>
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
x86 maintainers, can you please ack or nack this patch?
Thank you very much!
Just a heads up:
https://git.kernel.org/tip/775d3c514c5b2763a50ab7839026d7561795924d
and that one is a fix so it'll go to Linus now.
Which means, you'll have to use Linus' tree with that fix as a base and queue everything ontop.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteWei Liu
maintainerRe: [PATCH v5 1/5] x86/init: Make get/set_rtc_noop() public
Quoted 19 lines · Borislav Petkov
Quoted 12 lines · Wei Liu
Quoted 11 lines · Borislav Petkov
Quoted 10 lines · Wei Liu
Quoted 8 lines · Saurabh Sengar
Make get/set_rtc_noop() to be public so that they can be used in other modules as well.
Co-developed-by Tianyu Lan <[email protected]>Signed-off-by Tianyu Lan <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Reviewed-by Wei Liu <[email protected]>Reviewed-by Michael Kelley <[email protected]>Quoted 5 lines
Quoted 3 lines
Quoted 1 line
x86 maintainers, can you please ack or nack this patch?
Thank you very much!
Quoted 8 lines
Just a heads up:
https://git.kernel.org/tip/775d3c514c5b2763a50ab7839026d7561795924d
and that one is a fix so it'll go to Linus now.
Which means, you'll have to use Linus' tree with that fix as a base and queue everything ontop.
Okay, understood. Thanks.
Quoted 6 lines
Signature · 5 lines
-- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
On Mon, Apr 10, 2023 at 10:55:27PM -0700, Saurabh Sengar wrote:
[...]
Quoted 7 lines
Saurabh Sengar (5):
x86/init: Make get/set_rtc_noop() public x86/hyperv: Add VTL specific structs and hypercalls x86/hyperv: Makehv_get_nmi_reasonpublic
Drivers: hv:Kconfig: AddHYPERV_VTL_MODEx86/hyperv: VTL support for Hyper-V
Applied to hyperv-next. Thanks.