Add Secure TSC support for SNP guests
v13 cover Add Secure TSC support for SNP guests
This patchset is also available at:
https://github.com/AMDESE/linux-kvm/tree/sectsc-guest-latest
and is based on tip/x86/sev
Overview
--------
Secure TSC allows guests to securely use RDTSC/RDTSCP instructions as the parameters being used cannot be changed by hypervisor once the guest is launched. More details in the AMD64 APM Vol 2, Section "Secure TSC".
In order to enable secure TSC, SEV-SNP guests need to send a TSC_INFO guest message before the APs are booted. Details from the TSC_INFO response will then be used to program the VMSA before the APs are brought up. See "SEV Secure Nested Paging Firmware ABI Specification" document (currently at
https://www.amd.com/system/files/TechDocs/56860.pdf) section "TSC Info"
SEV-guest driver has the implementation for guest and AMD Security Processor communication. As the TSC_INFO needs to be initialized during early boot before APs are started, move the guest messaging code from sev-guest driver to sev/core.c and provide well defined APIs to the sev-guest driver.
Patches:
01-02: Patches moving SNP guest messaging code from SEV guest driver to
SEV common code
03-09: SecureTSC enablement patches
10-11: Generic TSC/kvmclock improvements
12-13: SecureTSC enablement patches.
Testing SecureTSC
-----------------
SecureTSC hypervisor patches based on top of SEV-SNP Guest MEMFD series:
https://github.com/AMDESE/linux-kvm/tree/sectsc-host-latest
QEMU changes:
https://github.com/nikunjad/qemu/tree/snp-securetsc-latest
QEMU commandline SEV-SNP with SecureTSC:
qemu-system-x86_64 -cpu EPYC-Milan-v2 -smp 4 \
-object memory-backend-memfd,id=ram1,size=1G,share=true,prealloc=false,reserve=false \
-object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,secure-tsc=on \
-machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
...Changelog:
----------
v13:
* Rebased on top of tip/x86/sev
* Squashed CC_ATTR_GUEST_SNP_SNP_SECURE_TSC change to patch where it used. (Tom)
* Fixed sparse warnings (kernel test robot)
* Added patch to prevent GUEST_TSC_FREQ MSR interception(Tom)
* Fix sched_clock override in common code (Sean)
* Added GHCB_TERM_SECURE_TSC_KVMCLOCK reason code and use sev_es_terminate() to inform the hypervisor that KVMCLOCK was being instead of Secure TSC.
v12: https://lore.kernel.org/lkml/[email protected]/
* Rebased on top of v6.12-rc2
* Collected Reviewed-by (Tom)
* Improve error handling in sme_enable() (Boris)
* Drop handle_guest_request() copying routine (Boris)
* Move VMPCK empty check inside the lock (Tom)
* Drop export symbol for snp_issue_guest_request() (Tom)
* Rename CC_ATTR_GUEST_SECURE_TSC as CC_ATTR_GUEST_SNP_SECURE_TSC (Tom)
* Upgrade the tsc early and regular clock rating when TSC is invariant, non-stop and stable (tglx)
* Initialize kvm sched clock only when the kvmclock source is selected (Sean)
* Abort SecureTSC enabled guests when kvmclock is selected (Sean)
* Added patch to use TSC frequency using GUEST_TSC_FREQ MSR (Sean)
Nikunj A Dadhania (13):
x86/sev: Carve out and export SNP guest messaging init routines x86/sev: Relocate SNP guest messaging routines to common code x86/sev: Add Secure TSC support for SNP guests
x86/sev: Change TSC MSR behavior for Secure TSC enabled guests x86/sev: Prevent RDTSC/RDTSCP interception for Secure TSC enabled
guests
x86/sev: Prevent GUEST_TSC_FREQ MSR interception for Secure TSC
enabled guests
x86/sev: Mark Secure TSC as reliable clocksource x86/cpu/amd: Do not print FW_BUG for Secure TSC
tsc: Use the GUEST_TSC_FREQ MSR for discovering TSC frequency
tsc: Upgrade TSC clocksource rating
tsc: Switch to native sched clock
x86/kvmclock: Abort SecureTSC enabled guest when kvmclock is selected x86/sev: Allow Secure TSC feature for SNP guests
arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/sev-common.h | 2 + arch/x86/include/asm/sev.h | 134 +++++- arch/x86/include/asm/svm.h | 6 +- include/linux/cc_platform.h | 8 + arch/x86/boot/compressed/sev.c | 3 +- arch/x86/coco/core.c | 3 + arch/x86/coco/sev/core.c | 567 +++++++++++++++++++++++- arch/x86/coco/sev/shared.c | 13 +- arch/x86/kernel/cpu/amd.c | 3 +- arch/x86/kernel/kvmclock.c | 9 + arch/x86/kernel/tsc.c | 41 ++ arch/x86/mm/mem_encrypt.c | 4 + arch/x86/mm/mem_encrypt_amd.c | 4 + drivers/virt/coco/sev-guest/sev-guest.c | 485 +------------------- arch/x86/Kconfig | 1 + drivers/virt/coco/sev-guest/Kconfig | 1 - 17 files changed, 794 insertions(+), 491 deletions(-)
base-commit: 0a895c0d9b73d934de95aa0dd4e631c394bdd25d
Signature · 1 line
2.34.1
PATCH v13 1/13 x86/sev: Carve out and export SNP guest messaging init routines
Currently, the SEV guest driver is the only user of SNP guest messaging. All routines for initializing SNP guest messaging are implemented within the SEV guest driver. To add Secure TSC guest support, these initialization routines need to be available during early boot.
Carve out common SNP guest messaging buffer allocations and message initialization routines to core/sev.c and export them. These newly added APIs set up the SNP message context (snp_msg_desc), which contains all the necessary details for sending SNP guest messages.
At present, the SEV guest platform data structure is used to pass the secrets page physical address to SEV guest driver. Since the secrets page address is locally available to the initialization routine, use the cached address. Remove the unused SEV guest platform data structure.
--- arch/x86/include/asm/sev.h | 71 ++++++++- arch/x86/coco/sev/core.c | 133 +++++++++++++++- drivers/virt/coco/sev-guest/sev-guest.c | 195 +++--------------------- 3 files changed, 215 insertions(+), 184 deletions(-)
arch/x86/include/asm/sev.h +67 -4
@@ -14,6 +14,7 @@#include <asm/insn.h> #include <asm/sev-common.h> #include <asm/coco.h> +#include <asm/set_memory.h> #define GHCB_PROTOCOL_MIN 1ULL#define GHCB_PROTOCOL_MAX 2ULL
@@ -170,10 +171,6 @@ struct snp_guest_msg {u8 payload[PAGE_SIZE -sizeof (struct snp_guest_msg_hdr )]; } __packed; -struct sev_guest_platform_data { -u64 secrets_gpa; -}; -struct snp_guest_req {void *req_buf;size_t req_sz;
@@ -253,6 +250,7 @@ struct snp_msg_desc {u32 *os_area_msg_seqno;u8 *vmpck; +int vmpck_id; };/*
@@ -438,6 +436,63 @@ u64 sev_get_status(void);void sev_show_status (void );void snp_update_svsm_ca (void ); +static inline void free_shared_pages (void *buf,size_t sz) +{ +unsigned int npages =PAGE_ALIGN (sz) >>PAGE_SHIFT ; +int ret; + +if (!buf) +return ; + + ret =set_memory_encrypted ((unsigned long )buf, npages); +if (ret) { +WARN_ONCE (ret,"failed to restore encryption mask (leak it)\n" ); +return ; + } + +__free_pages (virt_to_page (buf),get_order (sz)); +} + +static inline void *alloc_shared_pages (size_t sz) +{ +unsigned int npages =PAGE_ALIGN (sz) >>PAGE_SHIFT ; +struct page *page; +int ret; + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,get_order (sz)); +if (!page) +return NULL ; + + ret =set_memory_decrypted ((unsigned long )page_address (page), npages); +if (ret) { +pr_err ("failed to mark page shared, ret=%d\n" , ret); +__free_pages (page,get_order (sz)); +return NULL ; + } + +return page_address (page); +} + +static inline bool is_vmpck_empty (struct snp_msg_desc *mdesc) +{ +char zero_key[VMPCK_KEY_LEN ] = {0 }; + +if (mdesc->vmpck) +return !memcmp (mdesc->vmpck, zero_key,VMPCK_KEY_LEN ); + +return true; +} + +int snp_msg_init (struct snp_msg_desc *mdesc,int vmpck_id); +struct snp_msg_desc *snp_msg_alloc (void ); + +static inline void snp_msg_cleanup (struct snp_msg_desc *mdesc) +{ + mdesc->vmpck =NULL ; + mdesc->os_area_msg_seqno =NULL ; +kfree (mdesc->ctx); +} +#else /* !CONFIG_AMD_MEM_ENCRYPT */ #define snp_vmpl 0
@@ -474,6 +529,14 @@ static inline u64 snp_get_unsupported_features(u64 status) { return 0; }static inline u64 sev_get_status (void ) {return 0 ; }static inline void sev_show_status (void ) { }static inline void snp_update_svsm_ca (void ) { } +static inline void free_shared_pages (void *buf,size_t sz) { } +static inline void *alloc_shared_pages (size_t sz) {return NULL ; } +static inline bool is_vmpck_empty (struct snp_msg_desc *mdesc) {return false; } + +static inline int snp_msg_init (struct snp_msg_desc *mdesc,int vmpck_id) {return -1 ; } +static inline struct snp_msg_desc *snp_msg_alloc (void ) {return NULL ; } + +static inline void snp_msg_cleanup (struct snp_msg_desc *mdesc) { }#endif /* CONFIG_AMD_MEM_ENCRYPT */
arch/x86/coco/sev/core.c +127 -6
157 lines not on this page · raw message
drivers/virt/coco/sev-guest/sev-guest.c +21 -174
292 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v13 01/13] x86/sev: Carve out and export SNP guest messaging init routines
Le 21/10/2024 à 07:51, Nikunj A Dadhania a écrit :
Quoted 18 lines
Currently, the SEV guest driver is the only user of SNP guest messaging. All routines for initializing SNP guest messaging are implemented within the SEV guest driver. To add Secure TSC guest support, these initialization routines need to be available during early boot.
Carve out common SNP guest messaging buffer allocations and message initialization routines to
core/sev.cand export them. These newly added APIs set up the SNP message context (snp_msg_desc), which contains all the necessary details for sending SNP guest messages.At present, the SEV guest platform data structure is used to pass the secrets page physical address to SEV guest driver. Since the secrets page address is locally available to the initialization routine, use the cached address. Remove the unused SEV guest platform data structure.
Signed-off-by Nikunj A Dadhania <[email protected]>Reviewed-by Tom Lendacky <[email protected]>---
...
arch/x86/include/asm/sev.h u64 sev_get_status(void); · line 478
Quoted 3 lines
arch/x86/include/asm/sev.h +3
-0+static inline bool is_vmpck_empty (struct snp_msg_desc *mdesc) +{ +char zero_key[VMPCK_KEY_LEN ] = {0 };
Nitpick: I think this could be a static const.
Quoted 6 lines
arch/x86/include/asm/sev.h +6
-0+ +if (mdesc->vmpck) +return !memcmp (mdesc->vmpck, zero_key,VMPCK_KEY_LEN ); + +return true; +}
...
Re: [PATCH v13 01/13] x86/sev: Carve out and export SNP guest messaging init routines
Quoted 20 lines · Christophe JAILLET
Le 21/10/2024 à 07:51, Nikunj A Dadhania a écrit :
Quoted 10 lines
Currently, the SEV guest driver is the only user of SNP guest messaging. All routines for initializing SNP guest messaging are implemented within the SEV guest driver. To add Secure TSC guest support, these initialization routines need to be available during early boot.
Carve out common SNP guest messaging buffer allocations and message initialization routines to
core/sev.cand export them. These newly added APIs set up the SNP message context (snp_msg_desc), which contains all the necessary details for sending SNP guest messages.Quoted 8 lines
At present, the SEV guest platform data structure is used to pass the secrets page physical address to SEV guest driver. Since the secrets page address is locally available to the initialization routine, use the cached address. Remove the unused SEV guest platform data structure.
Signed-off-by Nikunj A Dadhania <[email protected]>Reviewed-by Tom Lendacky <[email protected]>---
Quoted 8 lines
...
arch/x86/include/asm/sev.h u64 sev_get_status(void); · line 478
Quoted 3 lines
arch/x86/include/asm/sev.h +3
-0+static inline bool is_vmpck_empty (struct snp_msg_desc *mdesc) +{ + char zero_key[VMPCK_KEY_LEN] = {0};Nitpick: I think this could be a static const.
As the primary objective of this patch is to move the code, I have retained the previous implementation.
Regards
Nikunj
PATCH v13 2/13 x86/sev: Relocate SNP guest messaging routines to common code
In reply to Nikunj A DadhaniaAt present, the SEV guest driver exclusively handles SNP guest messaging. All routines for sending guest messages are embedded within the guest driver. To support Secure TSC, SEV-SNP guests must communicate with the AMD Security Processor during early boot. However, these guest messaging functions are not accessible during early boot since they are currently part of the guest driver.
Hence, relocate the core SNP guest messaging functions to SEV common code and provide an API for sending SNP guest messages.
No functional change, but just an export symbol added for sev_send_geust_message() and dropped the export symbol on snp_issue_guest_request() and made it static.
--- arch/x86/include/asm/sev.h | 15 +- arch/x86/coco/sev/core.c | 295 +++++++++++++++++++++++- drivers/virt/coco/sev-guest/sev-guest.c | 292 ----------------------- arch/x86/Kconfig | 1 + drivers/virt/coco/sev-guest/Kconfig | 1 - 5 files changed, 301 insertions(+), 303 deletions(-)
arch/x86/include/asm/sev.h +8 -7
@@ -125,6 +125,9 @@ struct snp_req_data {#define AAD_LEN 48#define MSG_HDR_VER 1 +#define SNP_REQ_MAX_RETRY_DURATION (60*HZ) +#define SNP_REQ_RETRY_DELAY (2*HZ) +/* See SNP spec SNP_GUEST_REQUEST section for the structure */ enum msg_type {SNP_MSG_TYPE_INVALID =0 ,
@@ -427,8 +430,6 @@ void snp_set_wakeup_secondary_cpu(void);bool snp_init (struct boot_params *bp);void __noreturn snp_abort (void );void snp_dmi_setup (void ); -int snp_issue_guest_request (struct snp_guest_req *req,struct snp_req_data *input, -struct snp_guest_request_ioctl *rio);int snp_issue_svsm_attest_req (u64 call_id,struct svsm_call *call,struct svsm_attest_call *input);void snp_accept_memory (phys_addr_t start,phys_addr_t end);u64 snp_get_unsupported_features (u64 status);
@@ -493,6 +494,9 @@ static inline void snp_msg_cleanup(struct snp_msg_desc *mdesc) kfree(mdesc->ctx); } +int snp_send_guest_request (struct snp_msg_desc *mdesc,struct snp_guest_req *req, +struct snp_guest_request_ioctl *rio); +#else /* !CONFIG_AMD_MEM_ENCRYPT */ #define snp_vmpl 0
@@ -515,11 +519,6 @@ static inline void snp_set_wakeup_secondary_cpu(void) { }static inline bool snp_init (struct boot_params *bp) {return false; }static inline void snp_abort (void ) { }static inline void snp_dmi_setup (void ) { } -static inline int snp_issue_guest_request (struct snp_guest_req *req,struct snp_req_data *input, -struct snp_guest_request_ioctl *rio) -{ -return -ENOTTY ; -}static inline int snp_issue_svsm_attest_req (u64 call_id,struct svsm_call *call,struct svsm_attest_call *input) {return -ENOTTY ;
@@ -537,6 +536,8 @@ static inline int snp_msg_init(struct snp_msg_desc *mdesc, int vmpck_id) { returstatic inline struct snp_msg_desc *snp_msg_alloc (void ) {return NULL ; }static inline void snp_msg_cleanup (struct snp_msg_desc *mdesc) { } +static inline int snp_send_guest_request (struct snp_msg_desc *mdesc,struct snp_guest_req *req, +struct snp_guest_request_ioctl *rio) {return -ENODEV ; }#endif /* CONFIG_AMD_MEM_ENCRYPT */
arch/x86/coco/sev/core.c +292 -3
310 lines not on this page · raw message
drivers/virt/coco/sev-guest/sev-guest.c +0 -292
310 lines not on this page · raw message
arch/x86/Kconfig +1 -0
@@ -1556,6 +1556,7 @@ config AMD_MEM_ENCRYPT select ARCH_HAS_CC_PLATFORM select X86_MEM_ENCRYPT select UNACCEPTED_MEMORY + select CRYPTO_LIB_AESGCM help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory
drivers/virt/coco/sev-guest/Kconfig +0 -1
@@ -2,7 +2,6 @@ config SEV_GUEST tristate "AMD SEV Guest driver" default m depends on AMD_MEM_ENCRYPT - select CRYPTO_LIB_AESGCM select TSM_REPORTS help SEV-SNP firmware provides the guest a mechanism to communicate with
Signature · 1 line
2.34.1
PATCH v13 3/13 x86/sev: Add Secure TSC support for SNP guests
In reply to Nikunj A DadhaniaAdd support for Secure TSC in SNP-enabled guests. Secure TSC allows guests to securely use RDTSC/RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched.
Secure TSC-enabled guests need to query TSC information from the AMD Security Processor. This communication channel is encrypted between the AMD Security Processor and the guest, with the hypervisor acting merely as a conduit to deliver the guest messages to the AMD Security Processor. Each message is protected with AEAD (AES-256 GCM). Use a minimal AES GCM library to encrypt and decrypt SNP guest messages for communication with the PSP.
Use mem_encrypt_init() to fetch SNP TSC information from the AMD Security Processor and initialize snp_tsc_scale and snp_tsc_offset. During secondary CPU initialization, set the VMSA fields GUEST_TSC_SCALE (offset 2F0h) and GUEST_TSC_OFFSET (offset 2F8h) with snp_tsc_scale and snp_tsc_offset, respectively.
Add confidential compute platform attribute CC_ATTR_GUEST_SNP_SECURE_TSC that can be used by the guest to query whether the Secure TSC feature is active.
Since handle_guest_request() is common routine used by both the SEV guest driver and Secure TSC code, move it to the SEV header file.
--- arch/x86/include/asm/sev-common.h | 1 + arch/x86/include/asm/sev.h | 46 ++++++++++++++++ arch/x86/include/asm/svm.h | 6 +- include/linux/cc_platform.h | 8 +++ arch/x86/coco/core.c | 3 + arch/x86/coco/sev/core.c | 91 +++++++++++++++++++++++++++++++ arch/x86/mm/mem_encrypt.c | 4 ++ 7 files changed, 157 insertions(+), 2 deletions(-)
arch/x86/include/asm/sev-common.h +1 -0
@@ -206,6 +206,7 @@ struct snp_psc_desc {#define GHCB_TERM_NO_SVSM 7/* SVSM is not advertised in the secrets page */ #define GHCB_TERM_SVSM_VMPL0 8/* SVSM is present but has set VMPL to 0 */ #define GHCB_TERM_SVSM_CAA 9/* SVSM is present but CAA is not page aligned */ +#define GHCB_TERM_SECURE_TSC 10/* Secure TSC initialization failed */ #define GHCB_RESP_CODE (v) ((v) & GHCB_MSR_INFO_MASK)
arch/x86/include/asm/sev.h +46 -0
70 lines not on this page · raw message
arch/x86/include/asm/svm.h +4 -2
@@ -417,7 +417,9 @@ struct sev_es_save_area {u8 reserved_0x298[80 ];u32 pkru;u32 tsc_aux; -u8 reserved_0x2f0[24 ]; +u64 tsc_scale; +u64 tsc_offset; +u8 reserved_0x300[8 ];u64 rcx;u64 rdx;u64 rbx;
@@ -564,7 +566,7 @@ static inline void __unused_size_checks(void)BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x1c0 );BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x248 );BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x298 ); -BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x2f0 ); +BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x300 );BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x320 );BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x380 );BUILD_BUG_RESERVED_OFFSET (sev_es_save_area,0x3f0 );
include/linux/cc_platform.h +8 -0
@@ -88,6 +88,14 @@ enum cc_attr {* enabled to run SEV-SNP guests. */ CC_ATTR_HOST_SEV_SNP , + +/** +* @CC_ATTR_GUEST_SNP_SECURE_TSC: SNP Secure TSC is active. +* +* The platform/OS is running as a guest/virtual machine and actively +* using AMD SEV-SNP Secure TSC feature. +*/ + CC_ATTR_GUEST_SNP_SECURE_TSC, };#ifdef CONFIG_ARCH_HAS_CC_PLATFORM
arch/x86/coco/core.c +3 -0
@@ -100,6 +100,9 @@ static bool noinstr amd_cc_platform_has(enum cc_attr attr)case CC_ATTR_HOST_SEV_SNP :return cc_flags.host_sev_snp; +case CC_ATTR_GUEST_SNP_SECURE_TSC : +return sev_status &MSR_AMD64_SNP_SECURE_TSC ; +default :return false; }
arch/x86/coco/sev/core.c +91 -0
106 lines not on this page · raw message
arch/x86/mm/mem_encrypt.c +4 -0
@@ -94,6 +94,10 @@ void __init mem_encrypt_init(void)/* Call into SWIOTLB to update the SWIOTLB DMA buffers */ swiotlb_update_mem_attributes (); +/* Initialize SNP Secure TSC */ +if (cc_platform_has (CC_ATTR_GUEST_SEV_SNP )) +snp_secure_tsc_prepare (); +print_mem_encrypt_feature_info (); }
Signature · 1 line
2.34.1
Re: [PATCH v13 03/13] x86/sev: Add Secure TSC support for SNP guests
Le 21/10/2024 à 07:51, Nikunj A Dadhania a écrit :
Quoted 28 lines
Add support for Secure TSC in SNP-enabled guests. Secure TSC allows guests to securely use RDTSC/RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched.
Secure TSC-enabled guests need to query TSC information from the AMD Security Processor. This communication channel is encrypted between the AMD Security Processor and the guest, with the hypervisor acting merely as a conduit to deliver the guest messages to the AMD Security Processor. Each message is protected with AEAD (AES-256 GCM). Use a minimal AES GCM library to encrypt and decrypt SNP guest messages for communication with the PSP.
Use
mem_encrypt_init()to fetch SNP TSC information from the AMD Security Processor and initializesnp_tsc_scaleandsnp_tsc_offset. During secondary CPU initialization, set the VMSA fieldsGUEST_TSC_SCALE(offset 2F0h) andGUEST_TSC_OFFSET(offset 2F8h) withsnp_tsc_scaleandsnp_tsc_offset, respectively.Add confidential compute platform attribute
CC_ATTR_GUEST_SNP_SECURE_TSCthat can be used by the guest to query whether the Secure TSC feature is active.Since
handle_guest_request()is common routine used by both the SEV guest driver and Secure TSC code, move it to the SEV header file.Signed-off-by Nikunj A Dadhania <[email protected]>Tested-by Peter Gonda <[email protected]>Reviewed-by Tom Lendacky <[email protected]>---
..
Quoted 54 lines
arch/x86/coco/sev/core.c +54
-0+static int __init snp_get_tsc_info (void ) +{ +static u8 buf[SNP_TSC_INFO_RESP_SZ +AUTHTAG_LEN ]; +struct snp_guest_request_ioctl rio; +struct snp_tsc_info_resp tsc_resp; +struct snp_tsc_info_req *tsc_req; +struct snp_msg_desc *mdesc; +struct snp_guest_req req; +int rc; + +/* +* The intermediate response buffer is used while decrypting the +* response payload. Make sure that it has enough space to cover the +* authtag. +*/ +BUILD_BUG_ON (sizeof (buf) < (sizeof (tsc_resp) +AUTHTAG_LEN )); + + mdesc =snp_msg_alloc (); +if (IS_ERR_OR_NULL (mdesc)) +return -ENOMEM ; + + rc =snp_msg_init (mdesc, snp_vmpl); +if (rc) +return rc; + + tsc_req =kzalloc (sizeof (struct snp_tsc_info_req ),GFP_KERNEL ); +if (!tsc_req) +return -ENOMEM ; + +memset (&req,0 ,sizeof (req)); +memset (&rio,0 ,sizeof (rio)); +memset (buf,0 ,sizeof (buf)); + + req.msg_version =MSG_HDR_VER ; + req.msg_type =SNP_MSG_TSC_INFO_REQ ; + req.vmpck_id = snp_vmpl; + req.req_buf = tsc_req; + req.req_sz =sizeof (*tsc_req); + req.resp_buf = buf; + req.resp_sz =sizeof (tsc_resp) +AUTHTAG_LEN ; + req.exit_code =SVM_VMGEXIT_GUEST_REQUEST ; + + rc =snp_send_guest_request (mdesc, &req, &rio); +if (rc) + goto err_req; + +memcpy (&tsc_resp, buf,sizeof (tsc_resp)); +pr_debug ("%s: response status %x scale %llx offset %llx factor %x\n" , + __func__, tsc_resp.status, tsc_resp.tsc_scale, tsc_resp.tsc_offset, + tsc_resp.tsc_factor); + +if (tsc_resp.status ==0 ) { + snp_tsc_scale = tsc_resp.tsc_scale; + snp_tsc_offset = tsc_resp.tsc_offset;
arch/x86/coco/sev/core.c int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req · line 3019
Quoted 10 lines
arch/x86/coco/sev/core.c +10
-0+ }else { +pr_err ("Failed to get TSC info, response status %x\n" , tsc_resp.status); + rc = -EIO ; + } + +err_req: +/* The response buffer contains the sensitive data, explicitly clear it. */ +memzero_explicit (buf,sizeof (buf)); +memzero_explicit (&tsc_resp,sizeof (tsc_resp)); +memzero_explicit (&req,sizeof (req));
req does not seem to hold sensitive data.
Is it needed, or maybe should it be tsc_req?
arch/x86/coco/sev/core.c int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req · line 3013
Quoted 3 lines
arch/x86/coco/sev/core.c +3
-0+ +return rc; +}
...
CJ
Re: [PATCH v13 03/13] x86/sev: Add Secure TSC support for SNP guests
Quoted 62 lines · Christophe JAILLET
Le 21/10/2024 à 07:51, Nikunj A Dadhania a écrit :
..
Quoted 58 lines
arch/x86/coco/sev/core.c +58
-0+static int __init snp_get_tsc_info (void ) +{ + static u8 buf[SNP_TSC_INFO_RESP_SZ + AUTHTAG_LEN]; + struct snp_guest_request_ioctl rio; + struct snp_tsc_info_resp tsc_resp; + struct snp_tsc_info_req *tsc_req; + struct snp_msg_desc *mdesc; + struct snp_guest_req req; + int rc; + + /* + * The intermediate response buffer is used while decrypting the + * response payload. Make sure that it has enough space to cover the + * authtag. + */ + BUILD_BUG_ON(sizeof(buf) < (sizeof(tsc_resp) + AUTHTAG_LEN)); + + mdesc = snp_msg_alloc(); + if (IS_ERR_OR_NULL(mdesc)) + return -ENOMEM; + + rc = snp_msg_init(mdesc, snp_vmpl); + if (rc) + return rc; + + tsc_req = kzalloc(sizeof(struct snp_tsc_info_req), GFP_KERNEL); + if (!tsc_req) + return -ENOMEM; + + memset(&req, 0, sizeof(req)); + memset(&rio, 0, sizeof(rio)); + memset(buf, 0, sizeof(buf)); + + req.msg_version = MSG_HDR_VER; + req.msg_type = SNP_MSG_TSC_INFO_REQ; + req.vmpck_id = snp_vmpl; + req.req_buf = tsc_req; + req.req_sz = sizeof(*tsc_req); + req.resp_buf = buf; + req.resp_sz = sizeof(tsc_resp) + AUTHTAG_LEN; + req.exit_code = SVM_VMGEXIT_GUEST_REQUEST; + + rc = snp_send_guest_request(mdesc, &req, &rio); + if (rc) + goto err_req; + + memcpy(&tsc_resp, buf, sizeof(tsc_resp)); + pr_debug("%s: response status %x scale %llx offset %llx factor %x\n", + __func__, tsc_resp.status, tsc_resp.tsc_scale, tsc_resp.tsc_offset, + tsc_resp.tsc_factor); + + if (tsc_resp.status == 0) { + snp_tsc_scale = tsc_resp.tsc_scale; + snp_tsc_offset = tsc_resp.tsc_offset; + } else { + pr_err("Failed to get TSC info, response status %x\n", tsc_resp.status); + rc = -EIO; + }
Quoted 8 lines
arch/x86/coco/sev/core.c int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req · line 3019
Quoted 6 lines
arch/x86/coco/sev/core.c +6
-0+ +err_req: + /* The response buffer contains the sensitive data, explicitly clear it. */ + memzero_explicit(buf, sizeof(buf)); + memzero_explicit(&tsc_resp, sizeof(tsc_resp)); + memzero_explicit(&req, sizeof(req));req does not seem to hold sensitive data.
That is correct, I will remove that.
Quoted 1 line
Is it needed, or maybe should it be
tsc_req?
No, and tsc_req is zeroed by the caller and is not updated by AMD security processor.
Regards
Nikunj
PATCH v13 4/13 x86/sev: Change TSC MSR behavior for Secure TSC enabled guests
In reply to Nikunj A DadhaniaSecure TSC enabled guests should not write to MSR_IA32_TSC(10H) register as the subsequent TSC value reads are undefined. MSR_IA32_TSC read/write accesses should not exit to the hypervisor for such guests.
Accesses to MSR_IA32_TSC needs special handling in the #VC handler for the guests with Secure TSC enabled. Writes to MSR_IA32_TSC should be ignored, and reads of MSR_IA32_TSC should return the result of the RDTSC instruction.
--- arch/x86/coco/sev/core.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
arch/x86/coco/sev/core.c +24 -0
@@ -1308,6 +1308,30 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)return ES_OK ; } +/* +* TSC related accesses should not exit to the hypervisor when a +* guest is executing with SecureTSC enabled, so special handling +* is required for accesses of MSR_IA32_TSC: +* +* Writes: Writing to MSR_IA32_TSC can cause subsequent reads +* of the TSC to return undefined values, so ignore all +* writes. +* Reads: Reads of MSR_IA32_TSC should return the current TSC +* value, use the value returned by RDTSC. +*/ +if (regs->cx ==MSR_IA32_TSC &&cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) { +u64 tsc; + +if (exit_info_1) +return ES_OK ; + + tsc =rdtsc (); + regs->ax =UINT_MAX & tsc; + regs->dx =UINT_MAX & (tsc >>32 ); + +return ES_OK ; + } +ghcb_set_rcx (ghcb, regs->cx);if (exit_info_1) {ghcb_set_rax (ghcb, regs->ax);
Signature · 1 line
2.34.1
Re: [PATCH v13 04/13] x86/sev: Change TSC MSR behavior for Secure TSC enabled guests
Quoted 25 lines · Nikunj A Dadhania
Secure TSC enabled guests should not write to
MSR_IA32_TSC(10H)register as the subsequent TSC value reads are undefined.MSR_IA32_TSCread/write accesses should not exit to the hypervisor for such guests.Accesses to
MSR_IA32_TSCneeds special handling in the #VC handler for the guests with Secure TSC enabled. Writes toMSR_IA32_TSCshould be ignored, and reads ofMSR_IA32_TSCshould return the result of the RDTSC instruction.Signed-off-by Nikunj A Dadhania <[email protected]>Reviewed-by Tom Lendacky <[email protected]>Tested-by Peter Gonda <[email protected]>--- arch/x86/coco/sev/core.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/x86/coco/sev/core.c +1
-0@@ -1308,6 +1308,30 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)return ES_OK ; } +/*
arch/x86/coco/sev/core.c static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) · line 1321
Quoted 10 lines
arch/x86/coco/sev/core.c +10
-0+* TSC related accesses should not exit to the hypervisor when a +* guest is executing with SecureTSC enabled, so special handling +* is required for accesses of MSR_IA32_TSC: +* +* Writes: Writing to MSR_IA32_TSC can cause subsequent reads +* of the TSC to return undefined values, so ignore all +* writes. +* Reads: Reads of MSR_IA32_TSC should return the current TSC +* value, use the value returned by RDTSC. +*/
Why doesn't handle it by returning ES_VMM_ERROR when hypervisor intercepts RD/WR of MSR_IA32_TSC? With SECURE_TSC enabled, it seems not need to be intercepted.
I think the reason is that SNP guest relies on interception to do the ignore behavior for WRMSR in #VC handler because the writing leads to undefined result. Then the question is what if the hypervisor doesn't intercept write to MSR_IA32_TSC in the first place?
Quoted 16 lines
arch/x86/coco/sev/core.c +13
-0+if (regs->cx ==MSR_IA32_TSC &&cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) { +u64 tsc; + +if (exit_info_1) +return ES_OK ; + + tsc =rdtsc (); + regs->ax =UINT_MAX & tsc; + regs->dx =UINT_MAX & (tsc >>32 ); + +return ES_OK ; + } +ghcb_set_rcx (ghcb, regs->cx);if (exit_info_1) {ghcb_set_rax (ghcb, regs->ax);
Re: [PATCH v13 04/13] x86/sev: Change TSC MSR behavior for Secure TSC enabled guests
Quoted 31 lines · Xiaoyao Li
Quoted 30 lines · Nikunj A Dadhania
Secure TSC enabled guests should not write to
MSR_IA32_TSC(10H)register as the subsequent TSC value reads are undefined.MSR_IA32_TSCread/write accesses should not exit to the hypervisor for such guests.Accesses to
MSR_IA32_TSCneeds special handling in the #VC handler for the guests with Secure TSC enabled. Writes toMSR_IA32_TSCshould be ignored, and reads ofMSR_IA32_TSCshould return the result of the RDTSC instruction.Signed-off-by Nikunj A Dadhania <[email protected]>Reviewed-by Tom Lendacky <[email protected]>Tested-by Peter Gonda <[email protected]>---
arch/x86/coco/sev/core.c| 24 ++++++++++++++++++++++++1 file changed, 24 insertions(+)arch/x86/coco/sev/core.c +0
-0@@ -1308,6 +1308,30 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) return ES_OK; }+ /*
+ * TSC related accesses should not exit to the hypervisor when a + * guest is executing with SecureTSC enabled, so special handling + * is required for accesses ofMSR_IA32_TSC:
+ *
+ * Writes: Writing toMSR_IA32_TSCcan cause subsequent reads + * of the TSC to return undefined values, so ignore all
Quoted 8 lines
Quoted 4 lines
+ * writes.
+ * Reads: Reads ofMSR_IA32_TSCshould return the current TSC + * value, use the value returned by RDTSC. + */Why doesn't handle it by returning
ES_VMM_ERRORwhen hypervisor intercepts RD/WR ofMSR_IA32_TSC? WithSECURE_TSCenabled, it seems not need to be intercepted.
ES_VMM_ERROR will terminate the guest, which is not the expected behaviour. As documented, writes to the MSR is ignored and reads are done using RDTSC.
Quoted 3 lines
I think the reason is that SNP guest relies on interception to do the ignore behavior for WRMSR in #VC handler because the writing leads to undefined result.
For legacy and secure guests MSR_IA32_TSC is always intercepted(for both RD/WR). Moreover, this is a legacy MSR, RDTSC and RDTSCP is the what modern OSes should use. The idea is the catch any writes to TSC MSR and handle them gracefully.
Quoted 2 lines
Then the question is what if the hypervisor doesn't intercept write to
MSR_IA32_TSCin the first place?
I have tried to disable interception of MSR_IA32_TSC, and writes are ignored by the HW as well. I would like to continue the current documented HW as per the APM.
Regards,
Nikunj
Re: [PATCH v13 04/13] x86/sev: Change TSC MSR behavior for Secure TSC enabled guests
Quoted 43 lines · Nikunj A. Dadhania
Quoted 31 lines · Xiaoyao Li
Quoted 30 lines · Nikunj A Dadhania
Secure TSC enabled guests should not write to
MSR_IA32_TSC(10H)register as the subsequent TSC value reads are undefined.MSR_IA32_TSCread/write accesses should not exit to the hypervisor for such guests.Accesses to
MSR_IA32_TSCneeds special handling in the #VC handler for the guests with Secure TSC enabled. Writes toMSR_IA32_TSCshould be ignored, and reads ofMSR_IA32_TSCshould return the result of the RDTSC instruction.Signed-off-by Nikunj A Dadhania <[email protected]>Reviewed-by Tom Lendacky <[email protected]>Tested-by Peter Gonda <[email protected]>--- arch/x86/coco/sev/core.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/x86/coco/sev/core.c +6
-0@@ -1308,6 +1308,30 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) return ES_OK; } + /* +* TSC related accesses should not exit to the hypervisor when a +* guest is executing with SecureTSC enabled, so special handling +* is required for accesses of MSR_IA32_TSC: +* +* Writes: Writing to MSR_IA32_TSC can cause subsequent reads +* of the TSC to return undefined values, so ignore all Quoted 8 lines
arch/x86/coco/sev/core.c static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) · line 1321
Quoted 4 lines
arch/x86/coco/sev/core.c +4
-0+* writes. +* Reads: Reads of MSR_IA32_TSC should return the current TSC +* value, use the value returned by RDTSC. +*/ Why doesn't handle it by returning
ES_VMM_ERRORwhen hypervisor intercepts RD/WR ofMSR_IA32_TSC? WithSECURE_TSCenabled, it seems not need to be intercepted.
Quoted 8 lines
ES_VMM_ERRORwill terminate the guest, which is not the expected behaviour. As documented, writes to the MSR is ignored and reads are done using RDTSC.Quoted 3 lines
I think the reason is that SNP guest relies on interception to do the ignore behavior for WRMSR in #VC handler because the writing leads to undefined result.
For legacy and secure guests
MSR_IA32_TSCis always intercepted(for both RD/WR).
We cannot make such assumption unless it's enforced by AMD HW.
Quoted 2 lines
Moreover, this is a legacy MSR, RDTSC and RDTSCP is the what modern OSes should use.
Again, this is your assumption and expectation only.
Quoted 1 line
The idea is the catch any writes to TSC MSR and handle them gracefully.
If SNP guest requires MSR_IA32_TSC being intercepted by hypervisor. It should come with a solution that guest kernel can check it certainly, just like the patch 5 and patch 6, that they can check the behavior of hypervisor.
If there is no clean way for guest to ensure MSR_IA32_TSC is intercepted by hypervisor, we at least need add some comment to call out that these code replies on the assumption that hypervisor intercepts MSR_IA32_TSC.
Quoted 5 lines
Quoted 2 lines
Then the question is what if the hypervisor doesn't intercept write to
MSR_IA32_TSCin the first place?I have tried to disable interception of
MSR_IA32_TSC, and writes are ignored by the HW as well. I would like to continue the current documented HW as per the APM.
I only means the writes are ignored in your testing HW. We don't know the result on other SNP-capable HW or future HW, unless it's documented in APM.
Current documented behavior is that write leads to a undefined value of subsequent read. So we need to avoid write. One solution is to intercept write and ignore it, but it depends on hypervisor to intercept it. Anther solution would be we fix all the place of writing MSR_IA32_TSC for SNP guest in linux.
Quoted 2 lines
Regards,
Nikunj
Re: [PATCH v13 04/13] x86/sev: Change TSC MSR behavior for Secure TSC enabled guests
Quoted 40 lines · Xiaoyao Li
Quoted 32 lines · Nikunj A. Dadhania
Quoted 16 lines · Xiaoyao Li
Quoted 15 lines · Nikunj A Dadhania
arch/x86/coco/sev/core.c +7
-0@@ -1308,6 +1308,30 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)return ES_OK ; } +/* +* TSC related accesses should not exit to the hypervisor when a +* guest is executing with SecureTSC enabled, so special handling +* is required for accesses of MSR_IA32_TSC: +* +* Writes: Writing to MSR_IA32_TSC can cause subsequent reads +* of the TSC to return undefined values, so ignore all +* writes. Quoted 8 lines
arch/x86/coco/sev/core.c static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) · line 1321
Quoted 3 lines
arch/x86/coco/sev/core.c +3
-0+* Reads: Reads of MSR_IA32_TSC should return the current TSC +* value, use the value returned by RDTSC. +*/ Why doesn't handle it by returning
ES_VMM_ERRORwhen hypervisor intercepts RD/WR ofMSR_IA32_TSC? WithSECURE_TSCenabled, it seems not need to be intercepted.
ES_VMM_ERRORwill terminate the guest, which is not the expected behaviour. As documented, writes to the MSR is ignored and reads are done using RDTSC.Quoted 6 lines
Quoted 3 lines
I think the reason is that SNP guest relies on interception to do the ignore behavior for WRMSR in #VC handler because the writing leads to undefined result.
For legacy and secure guests
MSR_IA32_TSCis always intercepted(for both RD/WR).
Quoted 6 lines
We cannot make such assumption unless it's enforced by AMD HW.
Quoted 2 lines
Moreover, this is a legacy MSR, RDTSC and RDTSCP is the what modern OSes should use.
Again, this is your assumption and expectation only.
Not my assumption, I could not find any reference to MSR_IA32_TSC read/write in the kernel code.
Quoted 11 lines
Quoted 3 lines
The idea is the catch any writes to TSC MSR and handle them gracefully.
If SNP guest requires
MSR_IA32_TSCbeing intercepted by hypervisor. It should come with a solution that guest kernel can check it certainly, just like the patch 5 and patch 6, that they can check the behavior of hypervisor.If there is no clean way for guest to ensure
MSR_IA32_TSCis intercepted by hypervisor,
Yes, that is my understanding as well.
Quoted 3 lines
we at least need add some comment to call
out that these code replies on the assumption that hypervisor interceptsMSR_IA32_TSC.
Sure, I will add a comment.
Quoted 11 lines
Quoted 6 lines
Quoted 2 lines
Then the question is what if the hypervisor doesn't intercept write to
MSR_IA32_TSCin the first place?I have tried to disable interception of
MSR_IA32_TSC, and writes are ignored by the HW as well. I would like to continue the current documented HW as per the APM.I only means the writes are ignored in your testing HW. We don't know the result on other SNP-capable HW or future HW, unless it's documented in APM.
Quoted 6 lines
Current documented behavior is that write leads to a undefined value of subsequent read. So we need to avoid write. One solution is to intercept write and ignore it, but it depends on hypervisor to intercept it.
Anther solution would be we fix all the place of writingMSR_IA32_TSCfor SNP guest in linux.
There is no MSR_IA32_TSC write in the kernel code, IMHO, so second solution is taken care.
Regards,
Nikunj
PATCH v13 5/13 x86/sev: Prevent RDTSC/RDTSCP interception for Secure TSC enabled guests
In reply to Nikunj A DadhaniaThe hypervisor should not be intercepting RDTSC/RDTSCP when Secure TSC is enabled. A #VC exception will be generated if the RDTSC/RDTSCP instructions are being intercepted. If this should occur and Secure TSC is enabled, terminate guest execution.
--- arch/x86/coco/sev/shared.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/x86/coco/sev/shared.c +10 -0
@@ -1140,6 +1140,16 @@ static enum es_result vc_handle_rdtsc(struct ghcb *ghcb,bool rdtscp = (exit_code ==SVM_EXIT_RDTSCP );enum es_result ret; +/* +* RDTSC and RDTSCP should not be intercepted when Secure TSC is +* enabled. Terminate the SNP guest when the interception is enabled. +* This file is included from kernel/sev.c and boot/compressed/sev.c, +* use sev_status here as cc_platform_has() is not available when +* compiling boot/compressed/sev.c. +*/ +if (sev_status &MSR_AMD64_SNP_SECURE_TSC ) +return ES_VMM_ERROR ; + ret =sev_es_ghcb_hv_call (ghcb, ctxt, exit_code,0 ,0 );if (ret !=ES_OK )return ret;
Signature · 1 line
2.34.1
Re: [PATCH v13 05/13] x86/sev: Prevent RDTSC/RDTSCP interception for Secure TSC enabled guests
Quoted 4 lines · Nikunj A Dadhania
The hypervisor should not be intercepting RDTSC/RDTSCP when Secure TSC is enabled. A #VC exception will be generated if the RDTSC/RDTSCP instructions are being intercepted. If this should occur and Secure TSC is enabled, terminate guest execution.
There is another option to ignore the interception and just return back to guest execution. I think it better to add some justification on why make it fatal and terminate the guest is better than ignoring the interception.
Quoted 28 lines
Signed-off-by Nikunj A Dadhania <[email protected]>Tested-by Peter Gonda <[email protected]>Reviewed-by Tom Lendacky <[email protected]>--- arch/x86/coco/sev/shared.c | 10 ++++++++++ 1 file changed, 10 insertions(+)arch/x86/coco/sev/shared.c +10
-0@@ -1140,6 +1140,16 @@ static enum es_result vc_handle_rdtsc(struct ghcb *ghcb,bool rdtscp = (exit_code ==SVM_EXIT_RDTSCP );enum es_result ret; +/* +* RDTSC and RDTSCP should not be intercepted when Secure TSC is +* enabled. Terminate the SNP guest when the interception is enabled. +* This file is included from kernel/sev.c and boot/compressed/sev.c, +* use sev_status here as cc_platform_has() is not available when +* compiling boot/compressed/sev.c. +*/ +if (sev_status &MSR_AMD64_SNP_SECURE_TSC ) +return ES_VMM_ERROR ; + ret =sev_es_ghcb_hv_call (ghcb, ctxt, exit_code,0 ,0 );if (ret !=ES_OK )return ret;
Re: [PATCH v13 05/13] x86/sev: Prevent RDTSC/RDTSCP interception for Secure TSC enabled guests
Quoted 8 lines · Xiaoyao Li
Quoted 4 lines · Nikunj A Dadhania
The hypervisor should not be intercepting RDTSC/RDTSCP when Secure TSC is enabled. A #VC exception will be generated if the RDTSC/RDTSCP instructions are being intercepted. If this should occur and Secure TSC is enabled, terminate guest execution.
There is another option to ignore the interception and just return back to guest execution.
That is not correct, RDTSC/RDTSCP should return the timestamp counter value computed using the GUEST_TSC_SCALE and GUEST_TSC_OFFSET part of VMSA.
Quoted 1 line
I think it better to add some justification on why make it> fatal and terminate the guest is better than ignoring the interception.
How about the below updated commit message:
The hypervisor should not be intercepting RDTSC/RDTSCP when Secure TSC is enabled. A #VC exception will be generated if the RDTSC/RDTSCP instructions are being intercepted. If this should occur and Secure TSC is enabled, terminate guest execution as the guest cannot rely on the TSC value provided by the hypervisor.
Regards
Nikunj
Re: [PATCH v13 05/13] x86/sev: Prevent RDTSC/RDTSCP interception for Secure TSC enabled guests
Quoted 9 lines · Nikunj A. Dadhania
Quoted 6 lines · Xiaoyao Li
Quoted 4 lines · Nikunj A Dadhania
The hypervisor should not be intercepting RDTSC/RDTSCP when Secure TSC is enabled. A #VC exception will be generated if the RDTSC/RDTSCP instructions are being intercepted. If this should occur and Secure TSC is enabled, terminate guest execution.
Quoted 5 lines
Quoted 2 lines
There is another option to ignore the interception and just return back to guest execution.
That is not correct, RDTSC/RDTSCP should return the timestamp counter value computed using the
GUEST_TSC_SCALEandGUEST_TSC_OFFSETpart of VMSA.
Ah, I missed this. Yes, if ignore the interception, guest needs to do TSC scale itself with GUEST_TSC_SCALE and GUEST_TSC_OFFSET to get the correct TSC. It's complicating things while making not intercepting RDTSC/RDTSP a hard requirement is much simple.
I think it's worth adding it as the justification.
Quoted 13 lines
Quoted 1 line
I think it better to add some justification on why make it> fatal and terminate the guest is better than ignoring the interception.
How about the below updated commit message:
The hypervisor should not be intercepting RDTSC/RDTSCP when Secure TSC is enabled. A #VC exception will be generated if the RDTSC/RDTSCP instructions are being intercepted. If this should occur and Secure TSC is enabled, terminate guest execution as the guest cannot rely on the TSC value provided by the hypervisor.
Regards
Nikunj
PATCH v13 6/13 x86/sev: Prevent GUEST_TSC_FREQ MSR interception for Secure TSC enabled guests
In reply to Nikunj A DadhaniaThe hypervisor should not be intercepting GUEST_TSC_FREQ MSR(0xcOO10134) when Secure TSC is enabled. A #VC exception will be generated if the GUEST_TSC_FREQ MSR is being intercepted. If this should occur and SecureTSC is enabled, terminate guest execution.
--- arch/x86/include/asm/msr-index.h | 1 + arch/x86/coco/sev/core.c | 8 ++++++++ 2 files changed, 9 insertions(+)
arch/x86/include/asm/msr-index.h +1 -0
@@ -608,6 +608,7 @@#define MSR_AMD_PERF_CTL 0xc0010062#define MSR_AMD_PERF_STATUS 0xc0010063#define MSR_AMD_PSTATE_DEF_BASE 0xc0010064 +#define MSR_AMD64_GUEST_TSC_FREQ 0xc0010134#define MSR_AMD64_OSVW_ID_LENGTH 0xc0010140#define MSR_AMD64_OSVW_STATUS 0xc0010141#define MSR_AMD_PPIN_CTL 0xc00102f0
arch/x86/coco/sev/core.c +8 -0
@@ -1332,6 +1332,14 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)return ES_OK ; } +/* +* GUEST_TSC_FREQ should not be intercepted when Secure TSC is +* enabled. Terminate the SNP guest when the interception is enabled. +*/ +if (regs->cx ==MSR_AMD64_GUEST_TSC_FREQ &&cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) +return ES_VMM_ERROR ; + +ghcb_set_rcx (ghcb, regs->cx);if (exit_info_1) {ghcb_set_rax (ghcb, regs->ax);
Signature · 1 line
2.34.1
Re: [PATCH v13 06/13] x86/sev: Prevent GUEST_TSC_FREQ MSR interception for Secure TSC enabled guests
Quoted 6 lines · Nikunj A Dadhania
The hypervisor should not be intercepting
GUEST_TSC_FREQMSR(0xcOO10134) when Secure TSC is enabled. A #VC exception will be generated if theGUEST_TSC_FREQMSR is being intercepted. If this should occur and SecureTSC is enabled, terminate guest execution.Signed-off-by Nikunj A Dadhania <[email protected]>
Just a minor comment/question below.
Quoted 22 lines
--- arch/x86/include/asm/msr-index.h | 1 + arch/x86/coco/sev/core.c | 8 ++++++++ 2 files changed, 9 insertions(+)arch/x86/include/asm/msr-index.h +1
-0@@ -608,6 +608,7 @@#define MSR_AMD_PERF_CTL 0xc0010062#define MSR_AMD_PERF_STATUS 0xc0010063#define MSR_AMD_PSTATE_DEF_BASE 0xc0010064 +#define MSR_AMD64_GUEST_TSC_FREQ 0xc0010134#define MSR_AMD64_OSVW_ID_LENGTH 0xc0010140#define MSR_AMD64_OSVW_STATUS 0xc0010141#define MSR_AMD_PPIN_CTL 0xc00102f0arch/x86/coco/sev/core.c +0
-0
arch/x86/coco/sev/core.c static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) · line 1339
Quoted 8 lines
arch/x86/coco/sev/core.c +5
-0@@ -1332,6 +1332,14 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)return ES_OK ; } +/* +* GUEST_TSC_FREQ should not be intercepted when Secure TSC is +* enabled. Terminate the SNP guest when the interception is enabled. +*/ +if (regs->cx ==MSR_AMD64_GUEST_TSC_FREQ &&cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC ))
Should the cc_platform_has() check be changed into a check against sev_status directly (similar to the DEBUG_SWAP support)? Just in case this handler ends up getting used in early code where cc_platform_has() can't be used.
Thanks,
Tom
Quoted 6 lines
arch/x86/coco/sev/core.c +3
-0+return ES_VMM_ERROR ; + +ghcb_set_rcx (ghcb, regs->cx);if (exit_info_1) {ghcb_set_rax (ghcb, regs->ax);
Re: [PATCH v13 06/13] x86/sev: Prevent GUEST_TSC_FREQ MSR interception for Secure TSC enabled guests
Quoted 12 lines · Tom Lendacky
Quoted 6 lines · Nikunj A Dadhania
The hypervisor should not be intercepting
GUEST_TSC_FREQMSR(0xcOO10134) when Secure TSC is enabled. A #VC exception will be generated if theGUEST_TSC_FREQMSR is being intercepted. If this should occur and SecureTSC is enabled, terminate guest execution.Signed-off-by Nikunj A Dadhania <[email protected]>Reviewed-by Tom Lendacky <[email protected]>Just a minor comment/question below.
..
Quoted 8 lines
Quoted 8 lines
arch/x86/coco/sev/core.c +0
-0@@ -1332,6 +1332,14 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)return ES_OK ; }
Quoted 10 lines
arch/x86/coco/sev/core.c static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) · line 1339
Quoted 5 lines
arch/x86/coco/sev/core.c +5
-0+/* +* GUEST_TSC_FREQ should not be intercepted when Secure TSC is +* enabled. Terminate the SNP guest when the interception is enabled. +*/ +if (regs->cx ==MSR_AMD64_GUEST_TSC_FREQ &&cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC ))Should the
cc_platform_has()check be changed into a check againstsev_statusdirectly (similar to theDEBUG_SWAPsupport)? Just in case this handler ends up getting used in early code wherecc_platform_has()can't be used.
Sure, that makes sense. I will change it in my next version.
Regards
Nikunj
PATCH v13 7/13 x86/sev: Mark Secure TSC as reliable clocksource
In reply to Nikunj A DadhaniaIn SNP guest environment with Secure TSC enabled, unlike other clock sources (such as HPET, ACPI timer, APIC, etc.), the RDTSC instruction is handled without causing a VM exit, resulting in minimal overhead and jitters. Hence, mark Secure TSC as the only reliable clock source, bypassing unstable calibration.
--- arch/x86/mm/mem_encrypt_amd.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/mm/mem_encrypt_amd.c +4 -0
@@ -516,6 +516,10 @@ void __init sme_early_init(void)* kernel mapped. */ snp_update_svsm_ca (); + +/* Mark the TSC as reliable when Secure TSC is enabled */ +if (sev_status &MSR_AMD64_SNP_SECURE_TSC ) +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE ); }void __init mem_encrypt_free_decrypted_mem (void )
Signature · 1 line
2.34.1
PATCH v13 8/13 x86/cpu/amd: Do not print FW_BUG for Secure TSC
In reply to Nikunj A DadhaniaWhen Secure TSC is enabled and TscInvariant (bit 8) in CPUID_8000_0007_edx is set, the kernel complains with the below firmware bug:
[Firmware Bug]: TSC doesn't count with P0 frequency!
Secure TSC does not need to run at P0 frequency; the TSC frequency is set by the VMM as part of the SNP_LAUNCH_START command. Skip this check when Secure TSC is enabled
--- arch/x86/kernel/cpu/amd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/amd.c +2 -1
@@ -370,7 +370,8 @@ static void bsp_determine_snp(struct cpuinfo_x86 *c)static void bsp_init_amd (struct cpuinfo_x86 *c) { -if (cpu_has (c,X86_FEATURE_CONSTANT_TSC )) { +if (cpu_has (c,X86_FEATURE_CONSTANT_TSC ) && + !cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) {if (c->x86 >0x10 || (c->x86 ==0x10 && c->x86_model >=0x2 )) {
Signature · 1 line
2.34.1
PATCH v13 9/13 tsc: Use the GUEST_TSC_FREQ MSR for discovering TSC frequency
In reply to Nikunj A DadhaniaCalibrating the TSC frequency using the kvmclock is not correct for SecureTSC enabled guests. Use the platform provided TSC frequency via the GUEST_TSC_FREQ MSR (C001_0134h).
--- arch/x86/include/asm/sev.h | 2 ++ arch/x86/coco/sev/core.c | 16 ++++++++++++++++ arch/x86/kernel/tsc.c | 5 +++++ 3 files changed, 23 insertions(+)
arch/x86/include/asm/sev.h +2 -0
@@ -536,6 +536,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code }void __init snp_secure_tsc_prepare (void ); +void __init securetsc_init (void );#else /* !CONFIG_AMD_MEM_ENCRYPT */
@@ -584,6 +585,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code u32 resp_sz) { return -ENODEV; }static inline void __init snp_secure_tsc_prepare (void ) { } +static inline void __init securetsc_init (void ) { }#endif /* CONFIG_AMD_MEM_ENCRYPT */
arch/x86/coco/sev/core.c +16 -0
@@ -3065,3 +3065,19 @@ void __init snp_secure_tsc_prepare(void)pr_debug ("SecureTSC enabled" ); } + +static unsigned long securetsc_get_tsc_khz (void ) +{ +unsigned long long tsc_freq_mhz; + +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +rdmsrl (MSR_AMD64_GUEST_TSC_FREQ , tsc_freq_mhz); + +return (unsigned long )(tsc_freq_mhz *1000 ); +} + +void __init securetsc_init (void ) +{ + x86_platform.calibrate_cpu = securetsc_get_tsc_khz; + x86_platform.calibrate_tsc = securetsc_get_tsc_khz; +}
arch/x86/kernel/tsc.c +5 -0
@@ -30,6 +30,7 @@#include <asm/i8259.h> #include <asm/topology.h> #include <asm/uv/uv.h> +#include <asm/sev.h> unsigned int __read_mostly cpu_khz;/* TSC clocks / usec, not used here */ EXPORT_SYMBOL (cpu_khz);
@@ -1514,6 +1515,10 @@ void __init tsc_early_init(void)/* Don't change UV TSC multi-chassis synchronization */ if (is_early_uv_system ())return ; + +if (cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) +securetsc_init (); +if (!determine_cpu_tsc_frequencies (true))return ;tsc_enable_sched_clock ();
Signature · 1 line
2.34.1
Re: [PATCH v13 09/13] tsc: Use the GUEST_TSC_FREQ MSR for discovering TSC frequency
Quoted 24 lines · Nikunj A Dadhania
Calibrating the TSC frequency using the kvmclock is not correct for SecureTSC enabled guests. Use the platform provided TSC frequency via the
GUEST_TSC_FREQMSR (C001_0134h).Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/sev.h | 2 ++ arch/x86/coco/sev/core.c | 16 ++++++++++++++++ arch/x86/kernel/tsc.c | 5 +++++ 3 files changed, 23 insertions(+)arch/x86/include/asm/sev.h +1
-0@@ -536,6 +536,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code }void __init snp_secure_tsc_prepare (void ); +void __init securetsc_init (void );#else /* !CONFIG_AMD_MEM_ENCRYPT */
arch/x86/include/asm/sev.h static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code · line 588
Quoted 4 lines
arch/x86/include/asm/sev.h +1
-0@@ -584,6 +585,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code u32 resp_sz) { return -ENODEV; }static inline void __init snp_secure_tsc_prepare (void ) { } +static inline void __init securetsc_init (void ) { }
This should probably be named snp_securetsc_init() or snp_secure_tsc_init() (to be consistent with the function above it) so that it is in the snp namespace.
Quoted 40 lines
arch/x86/include/asm/sev.h +0
-0#endif /* CONFIG_AMD_MEM_ENCRYPT */ arch/x86/coco/sev/core.c +16
-0@@ -3065,3 +3065,19 @@ void __init snp_secure_tsc_prepare(void)pr_debug ("SecureTSC enabled" ); } + +static unsigned long securetsc_get_tsc_khz (void ) +{ +unsigned long long tsc_freq_mhz; + +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +rdmsrl (MSR_AMD64_GUEST_TSC_FREQ , tsc_freq_mhz); + +return (unsigned long )(tsc_freq_mhz *1000 ); +} + +void __init securetsc_init (void ) +{ + x86_platform.calibrate_cpu = securetsc_get_tsc_khz; + x86_platform.calibrate_tsc = securetsc_get_tsc_khz; +}arch/x86/kernel/tsc.c +1
-0@@ -30,6 +30,7 @@#include <asm/i8259.h> #include <asm/topology.h> #include <asm/uv/uv.h> +#include <asm/sev.h> unsigned int __read_mostly cpu_khz;/* TSC clocks / usec, not used here */ EXPORT_SYMBOL (cpu_khz);
arch/x86/kernel/tsc.c void __init tsc_early_init(void) · line 1520
Quoted 6 lines
arch/x86/kernel/tsc.c +3
-0@@ -1514,6 +1515,10 @@ void __init tsc_early_init(void)/* Don't change UV TSC multi-chassis synchronization */ if (is_early_uv_system ())return ; + +if (cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) +securetsc_init ();
Would this call be better in kvm_init_platform() or kvmclock_init()? Any reason it has to be here?
Thanks,
Tom
Quoted 4 lines
arch/x86/kernel/tsc.c +1
-0+if (!determine_cpu_tsc_frequencies (true))return ;tsc_enable_sched_clock ();
Re: [PATCH v13 09/13] tsc: Use the GUEST_TSC_FREQ MSR for discovering TSC frequency
Quoted 27 lines · Tom Lendacky
Quoted 26 lines · Nikunj A Dadhania
Calibrating the TSC frequency using the kvmclock is not correct for SecureTSC enabled guests. Use the platform provided TSC frequency via the
GUEST_TSC_FREQMSR (C001_0134h).Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/sev.h | 2 ++ arch/x86/coco/sev/core.c | 16 ++++++++++++++++ arch/x86/kernel/tsc.c | 5 +++++ 3 files changed, 23 insertions(+)arch/x86/include/asm/sev.h +1
-0@@ -536,6 +536,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code }void __init snp_secure_tsc_prepare (void ); +void __init securetsc_init (void );#else /* !CONFIG_AMD_MEM_ENCRYPT */ @@ -584,6 +585,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code u32 resp_sz) { return -ENODEV; }
Quoted 6 lines
arch/x86/include/asm/sev.h static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_code · line 588
Quoted 2 lines
arch/x86/include/asm/sev.h +1
-0static inline void __init snp_secure_tsc_prepare (void ) { } +static inline void __init securetsc_init (void ) { }This should probably be named
snp_securetsc_init()orsnp_secure_tsc_init()(to be consistent with the function above it) so that it is in the snp namespace.
Ack.
Quoted 10 lines
Quoted 10 lines
arch/x86/kernel/tsc.c +1
-0@@ -30,6 +30,7 @@#include <asm/i8259.h> #include <asm/topology.h> #include <asm/uv/uv.h> +#include <asm/sev.h>
Quoted 12 lines
arch/x86/kernel/tsc.c void __init tsc_early_init(void) · line 1520
Quoted 9 lines
arch/x86/kernel/tsc.c +3
-0unsigned int __read_mostly cpu_khz;/* TSC clocks / usec, not used here */ EXPORT_SYMBOL (cpu_khz);@@ -1514,6 +1515,10 @@ void __init tsc_early_init(void)/* Don't change UV TSC multi-chassis synchronization */ if (is_early_uv_system ())return ; + +if (cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) +securetsc_init ();Would this call be better in
kvm_init_platform()orkvmclock_init()? Any reason it has to be here?
Added here to make sure that initialization happens on all the hypervisor, not just for KVM. Moreover, kvmclock can be disabled from the command line.
Regards,
Nikunj
PATCH v13 10/13 tsc: Upgrade TSC clocksource rating
In reply to Nikunj A DadhaniaIn virtualized environments running on modern CPUs, the underlying platforms guarantees to have a stable, always running TSC, i.e. that the TSC is a superior timesource as compared to other clock sources (such as kvmclock, HPET, ACPI timer, APIC, etc.).
Upgrade the rating of the early and regular clock source to prefer TSC over other clock sources when TSC is invariant, non-stop and stable.
--- arch/x86/kernel/tsc.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
arch/x86/kernel/tsc.c +17 -0
@@ -1264,6 +1264,21 @@ static void __init check_system_tsc_reliable(void)tsc_disable_clocksource_watchdog (); } +static void __init upgrade_clock_rating (struct clocksource *tsc_early, +struct clocksource *tsc) +{ +/* +* Upgrade the clock rating for TSC early and regular clocksource when +* the underlying platform provides non-stop, invaraint and stable TSC. +*/ +if (boot_cpu_has (X86_FEATURE_CONSTANT_TSC ) && +boot_cpu_has (X86_FEATURE_NONSTOP_TSC ) && + !tsc_unstable) { + tsc_early->rating =449 ; + tsc->rating =450 ; + } +} +/* * Make an educated guess if the TSC is trustworthy and synchronized * over all CPUs.
@@ -1565,6 +1580,8 @@ void __init tsc_init(void)if (tsc_clocksource_reliable || no_tsc_watchdog)tsc_disable_clocksource_watchdog (); +upgrade_clock_rating (&clocksource_tsc_early, &clocksource_tsc); +clocksource_register_khz (&clocksource_tsc_early, tsc_khz);detect_art (); }
Signature · 1 line
2.34.1
PATCH v13 11/13 tsc: Switch to native sched clock
In reply to Nikunj A DadhaniaAlthough the kernel switches over to stable TSC clocksource instead of PV clocksource, the scheduler still keeps on using PV clocks as the sched clock source. This is because the following KVM, Xen and VMWare, switches the paravirt sched clock handler in their init routines. The HyperV is the only PV clock source that checks if the platform provides invariant TSC and does not switch to PV sched clock.
When switching back to stable TSC, restore the scheduler clock to native_sched_clock().
As the clock selection happens in the stop machine context, schedule delayed work to update the static_call()
--- arch/x86/kernel/tsc.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
arch/x86/kernel/tsc.c +19 -0
@@ -272,10 +272,25 @@ bool using_native_sched_clock(void) {return static_call_query (pv_sched_clock) == native_sched_clock; } + +static void enable_native_sc_work (struct work_struct *work) +{ +pr_info ("using native sched clock\n" ); +paravirt_set_sched_clock (native_sched_clock); +} +static DECLARE_DELAYED_WORK(enable_native_sc, enable_native_sc_work); + +static void enable_native_sched_clock (void ) +{ +if (!using_native_sched_clock ()) +schedule_delayed_work (&enable_native_sc,0 ); +}#else u64 sched_clock_noinstr (void ) __attribute__((alias ("native_sched_clock" )));bool using_native_sched_clock (void ) {return true; } + +void enable_native_sched_clock (void ) { }#endif notrace u64 sched_clock(void)
@@ -1157,6 +1172,10 @@ static void tsc_cs_tick_stable(struct clocksource *cs)static int tsc_cs_enable (struct clocksource *cs) {vclocks_set_used (VDSO_CLOCKMODE_TSC ); + +/* Restore native_sched_clock() when switching to TSC */ +enable_native_sched_clock (); +return 0 ; }
Signature · 1 line
2.34.1
Re: [PATCH v13 11/13] tsc: Switch to native sched clock
Quoted 3 lines · Nikunj A Dadhania
Although the kernel switches over to stable TSC clocksource instead of PV clocksource, the scheduler still keeps on using PV clocks as the sched clock source. This is because the following KVM, Xen and VMWare, switches
s/the following//
s/switches/switch/
Quoted 1 line
the paravirt sched clock handler in their init routines. The HyperV is the
s/The HyperV/HyperV/
Quoted 1 line
only PV clock source that checks if the platform provides invariant TSC and
s/provides invariant/provides an invariant/
Thanks,
Tom
Quoted 57 lines
does not switch to PV sched clock.
When switching back to stable TSC, restore the scheduler clock to
native_sched_clock().As the clock selection happens in the stop machine context, schedule delayed work to update the
static_call()Cc Alexey Makhalov <[email protected]>Cc Juergen Gross <[email protected]>Cc Boris Ostrovsky <[email protected]>Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/kernel/tsc.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)arch/x86/kernel/tsc.c +19
-0@@ -272,10 +272,25 @@ bool using_native_sched_clock(void) {return static_call_query (pv_sched_clock) == native_sched_clock; } + +static void enable_native_sc_work (struct work_struct *work) +{ +pr_info ("using native sched clock\n" ); +paravirt_set_sched_clock (native_sched_clock); +} +static DECLARE_DELAYED_WORK(enable_native_sc, enable_native_sc_work); + +static void enable_native_sched_clock (void ) +{ +if (!using_native_sched_clock ()) +schedule_delayed_work (&enable_native_sc,0 ); +}#else u64 sched_clock_noinstr (void ) __attribute__((alias ("native_sched_clock" )));bool using_native_sched_clock (void ) {return true; } + +void enable_native_sched_clock (void ) { }#endif notrace u64 sched_clock(void)@@ -1157,6 +1172,10 @@ static void tsc_cs_tick_stable(struct clocksource *cs)static int tsc_cs_enable (struct clocksource *cs) {vclocks_set_used (VDSO_CLOCKMODE_TSC ); + +/* Restore native_sched_clock() when switching to TSC */ +enable_native_sched_clock (); +return 0 ; }
Re: [PATCH v13 11/13] tsc: Switch to native sched clock
Quoted 8 lines · Tom Lendacky
Quoted 3 lines · Nikunj A Dadhania
Although the kernel switches over to stable TSC clocksource instead of PV clocksource, the scheduler still keeps on using PV clocks as the sched clock source. This is because the following KVM, Xen and VMWare, switches
s/the following//
s/switches/switch/
Quoted 7 lines
Quoted 1 line
the paravirt sched clock handler in their init routines. The HyperV is the
s/The HyperV/HyperV/
Quoted 1 line
only PV clock source that checks if the platform provides invariant TSC and
s/provides invariant/provides an invariant/
Sure, I will update the message.
Regards
Nikunj
kernel test robot
build botRe: [PATCH v13 11/13] tsc: Switch to native sched clock
In reply to Nikunj A Dadhaniakernel test robot noticed the following build warnings:
Hi Nikunj,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 0a895c0d9b73d934de95aa0dd4e631c394bdd25d]
url: https://github.com/intel-lab-lkp/linux/commits/Nikunj-A-Dadhania/x86-sev-Carve-out-and-export-SNP-guest-messaging-init-routines/20241021-140125 base: 0a895c0d9b73d934de95aa0dd4e631c394bdd25d patch link: https://lore.kernel.org/r/20241021055156.2342564-12-nikunj%40amd.com patch subject: [PATCH v13 11/13] tsc: Switch to native sched clock config: x86_64-allnoconfig (https://download.01.org/0day-ci/archive/20241022/[email protected]/config) compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241022/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> arch/x86/kernel/tsc.c:293:6: warning: no previous prototype for function 'enable_native_sched_clock' [-Wmissing-prototypes]
293 | void enable_native_sched_clock(void) { }
| ^
arch/x86/kernel/tsc.c:293:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
293 | void enable_native_sched_clock(void) { }
| ^
| static 1 warning generated.
vim +/enable_native_sched_clock +293 arch/x86/kernel/tsc.c
292
> 293 void enable_native_sched_clock(void) { } 294 #endif
295
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH v13 11/13] tsc: Switch to native sched clock
In reply to Nikunj A Dadhaniakernel test robot noticed the following build warnings:
Hi Nikunj,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 0a895c0d9b73d934de95aa0dd4e631c394bdd25d]
url: https://github.com/intel-lab-lkp/linux/commits/Nikunj-A-Dadhania/x86-sev-Carve-out-and-export-SNP-guest-messaging-init-routines/20241021-140125 base: 0a895c0d9b73d934de95aa0dd4e631c394bdd25d patch link: https://lore.kernel.org/r/20241021055156.2342564-12-nikunj%40amd.com patch subject: [PATCH v13 11/13] tsc: Switch to native sched clock config: i386-buildonly-randconfig-002-20241022 (https://download.01.org/0day-ci/archive/20241022/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241022/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> arch/x86/kernel/tsc.c:293:6: warning: no previous prototype for 'enable_native_sched_clock' [-Wmissing-prototypes]
293 | void enable_native_sched_clock(void) { }
| ^~~~~~~~~~~~~~~~~~~~~~~~~vim +/enable_native_sched_clock +293 arch/x86/kernel/tsc.c
292
> 293 void enable_native_sched_clock(void) { } 294 #endif
295
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH v13 12/13 x86/kvmclock: Abort SecureTSC enabled guest when kvmclock is selected
In reply to Nikunj A DadhaniaSecureTSC enabled guests should use TSC as the only clock source, terminate the guest with appropriate code when clock source switches to hypervisor controlled kvmclock.
--- arch/x86/include/asm/sev-common.h | 1 + arch/x86/include/asm/sev.h | 2 ++ arch/x86/coco/sev/shared.c | 3 +-- arch/x86/kernel/kvmclock.c | 9 +++++++++ 4 files changed, 13 insertions(+), 2 deletions(-)
arch/x86/include/asm/sev-common.h +1 -0
@@ -207,6 +207,7 @@ struct snp_psc_desc {#define GHCB_TERM_SVSM_VMPL0 8/* SVSM is present but has set VMPL to 0 */ #define GHCB_TERM_SVSM_CAA 9/* SVSM is present but CAA is not page aligned */ #define GHCB_TERM_SECURE_TSC 10/* Secure TSC initialization failed */ +#define GHCB_TERM_SECURE_TSC_KVMCLOCK 11/* KVM clock selected instead of Secure TSC */ #define GHCB_RESP_CODE (v) ((v) & GHCB_MSR_INFO_MASK)
arch/x86/include/asm/sev.h +2 -0
@@ -537,6 +537,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_codevoid __init snp_secure_tsc_prepare (void );void __init securetsc_init (void ); +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason);#else /* !CONFIG_AMD_MEM_ENCRYPT */
@@ -586,6 +587,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_codestatic inline void __init snp_secure_tsc_prepare (void ) { }static inline void __init securetsc_init (void ) { } +static inline void sev_es_terminate (unsigned int set,unsigned int reason) { }#endif /* CONFIG_AMD_MEM_ENCRYPT */
arch/x86/coco/sev/shared.c +1 -2
@@ -117,8 +117,7 @@ static bool __init sev_es_check_cpu_features(void)return true; } -static void __head __noreturn -sev_es_terminate (unsigned int set,unsigned int reason) +void __head __noreturn sev_es_terminate(unsigned int set, unsigned int reason) {u64 val =GHCB_MSR_TERM_REQ ;
arch/x86/kernel/kvmclock.c +9 -0
@@ -21,6 +21,7 @@#include <asm/hypervisor.h> #include <asm/x86_init.h> #include <asm/kvmclock.h> +#include <asm/sev.h> static int kvmclock__initdata =1 ;static int kvmclock_vsyscall__initdata =1 ;
@@ -150,6 +151,14 @@ bool kvm_check_and_clear_guest_paused(void)static int kvm_cs_enable (struct clocksource *cs) { +/* +* For a guest with SecureTSC enabled, the TSC should be the only clock +* source. Abort the guest when kvmclock is selected as the clock +* source. +*/ +if (cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC )) +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SECURE_TSC_KVMCLOCK ); +vclocks_set_used (VDSO_CLOCKMODE_PVCLOCK );return 0 ; }
Signature · 1 line
2.34.1
Re: [PATCH v13 12/13] x86/kvmclock: Abort SecureTSC enabled guest when kvmclock is selected
Quoted 71 lines · Nikunj A Dadhania
SecureTSC enabled guests should use TSC as the only clock source, terminate the guest with appropriate code when clock source switches to hypervisor controlled kvmclock.
Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/sev-common.h | 1 + arch/x86/include/asm/sev.h | 2 ++ arch/x86/coco/sev/shared.c | 3 +-- arch/x86/kernel/kvmclock.c | 9 +++++++++ 4 files changed, 13 insertions(+), 2 deletions(-)arch/x86/include/asm/sev-common.h +1
-0@@ -207,6 +207,7 @@ struct snp_psc_desc {#define GHCB_TERM_SVSM_VMPL0 8/* SVSM is present but has set VMPL to 0 */ #define GHCB_TERM_SVSM_CAA 9/* SVSM is present but CAA is not page aligned */ #define GHCB_TERM_SECURE_TSC 10/* Secure TSC initialization failed */ +#define GHCB_TERM_SECURE_TSC_KVMCLOCK 11/* KVM clock selected instead of Secure TSC */ #define GHCB_RESP_CODE (v) ((v) & GHCB_MSR_INFO_MASK)arch/x86/include/asm/sev.h +2
-0@@ -537,6 +537,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_codevoid __init snp_secure_tsc_prepare (void );void __init securetsc_init (void ); +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason);#else /* !CONFIG_AMD_MEM_ENCRYPT */ @@ -586,6 +587,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_codestatic inline void __init snp_secure_tsc_prepare (void ) { }static inline void __init securetsc_init (void ) { } +static inline void sev_es_terminate (unsigned int set,unsigned int reason) { }#endif /* CONFIG_AMD_MEM_ENCRYPT */ arch/x86/coco/sev/shared.c +1
-2@@ -117,8 +117,7 @@ static bool __init sev_es_check_cpu_features(void)return true; } -static void __head __noreturn -sev_es_terminate (unsigned int set,unsigned int reason) +void __head __noreturn sev_es_terminate(unsigned int set, unsigned int reason) {u64 val =GHCB_MSR_TERM_REQ ;arch/x86/kernel/kvmclock.c +1
-0@@ -21,6 +21,7 @@#include <asm/hypervisor.h> #include <asm/x86_init.h> #include <asm/kvmclock.h> +#include <asm/sev.h> static int kvmclock__initdata =1 ;static int kvmclock_vsyscall__initdata =1 ;
arch/x86/kernel/kvmclock.c bool kvm_check_and_clear_guest_paused(void) · line 159
Quoted 9 lines
arch/x86/kernel/kvmclock.c +6
-0@@ -150,6 +151,14 @@ bool kvm_check_and_clear_guest_paused(void)static int kvm_cs_enable (struct clocksource *cs) { +/* +* For a guest with SecureTSC enabled, the TSC should be the only clock +* source. Abort the guest when kvmclock is selected as the clock +* source. +*/ +if (cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC ))
if (WARN_ON(cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)))
so that the guest sees something as well?
Thanks,
Tom
Quoted 5 lines
arch/x86/kernel/kvmclock.c +2
-0+sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SECURE_TSC_KVMCLOCK ); +vclocks_set_used (VDSO_CLOCKMODE_PVCLOCK );return 0 ; }
Re: [PATCH v13 12/13] x86/kvmclock: Abort SecureTSC enabled guest when kvmclock is selected
Quoted 84 lines · Tom Lendacky
Quoted 71 lines · Nikunj A Dadhania
SecureTSC enabled guests should use TSC as the only clock source, terminate the guest with appropriate code when clock source switches to hypervisor controlled kvmclock.
Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/sev-common.h | 1 + arch/x86/include/asm/sev.h | 2 ++ arch/x86/coco/sev/shared.c | 3 +-- arch/x86/kernel/kvmclock.c | 9 +++++++++ 4 files changed, 13 insertions(+), 2 deletions(-)arch/x86/include/asm/sev-common.h +1
-0@@ -207,6 +207,7 @@ struct snp_psc_desc {#define GHCB_TERM_SVSM_VMPL0 8/* SVSM is present but has set VMPL to 0 */ #define GHCB_TERM_SVSM_CAA 9/* SVSM is present but CAA is not page aligned */ #define GHCB_TERM_SECURE_TSC 10/* Secure TSC initialization failed */ +#define GHCB_TERM_SECURE_TSC_KVMCLOCK 11/* KVM clock selected instead of Secure TSC */ #define GHCB_RESP_CODE (v) ((v) & GHCB_MSR_INFO_MASK)arch/x86/include/asm/sev.h +2
-0@@ -537,6 +537,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_codevoid __init snp_secure_tsc_prepare (void );void __init securetsc_init (void ); +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason);#else /* !CONFIG_AMD_MEM_ENCRYPT */ @@ -586,6 +587,7 @@ static inline int handle_guest_request(struct snp_msg_desc *mdesc, u64 exit_codestatic inline void __init snp_secure_tsc_prepare (void ) { }static inline void __init securetsc_init (void ) { } +static inline void sev_es_terminate (unsigned int set,unsigned int reason) { }#endif /* CONFIG_AMD_MEM_ENCRYPT */ arch/x86/coco/sev/shared.c +1
-2@@ -117,8 +117,7 @@ static bool __init sev_es_check_cpu_features(void)return true; } -static void __head __noreturn -sev_es_terminate (unsigned int set,unsigned int reason) +void __head __noreturn sev_es_terminate(unsigned int set, unsigned int reason) {u64 val =GHCB_MSR_TERM_REQ ;arch/x86/kernel/kvmclock.c +1
-0@@ -21,6 +21,7 @@#include <asm/hypervisor.h> #include <asm/x86_init.h> #include <asm/kvmclock.h> +#include <asm/sev.h> static int kvmclock__initdata =1 ;static int kvmclock_vsyscall__initdata =1 ;arch/x86/kernel/kvmclock.c bool kvm_check_and_clear_guest_paused(void) · line 159
Quoted 9 lines
arch/x86/kernel/kvmclock.c +6
-0@@ -150,6 +151,14 @@ bool kvm_check_and_clear_guest_paused(void)static int kvm_cs_enable (struct clocksource *cs) { +/* +* For a guest with SecureTSC enabled, the TSC should be the only clock +* source. Abort the guest when kvmclock is selected as the clock +* source. +*/ +if (cc_platform_has (CC_ATTR_GUEST_SNP_SECURE_TSC ))if (WARN_ON(cc_platform_has(CC_ATTR_GUEST_SNP_SECURE_TSC)))
Quoted 1 line
so that the guest sees something as well?
Sure, will add.
Regards
Nikunj
PATCH v13 13/13 x86/sev: Allow Secure TSC feature for SNP guests
In reply to Nikunj A DadhaniaNow that all the required plumbing is done for enabling SNP Secure TSC feature, add Secure TSC to SNP features present list.
--- arch/x86/boot/compressed/sev.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/boot/compressed/sev.c +2 -1
@@ -401,7 +401,8 @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)* by the guest kernel. As and when a new feature is implemented in the * guest kernel, a corresponding bit should be added to the mask. */ -#define SNP_FEATURES_PRESENT MSR_AMD64_SNP_DEBUG_SWAP +#define SNP_FEATURES_PRESENT (MSR_AMD64_SNP_DEBUG_SWAP | \ + MSR_AMD64_SNP_SECURE_TSC)u64 snp_get_unsupported_features (u64 status) {
Signature · 1 line
2.34.1