Enable Secure TSC for SEV-SNP
v10 cover Enable Secure TSC for SEV-SNP
Patches are based on kvm/next with [1] applied
Testing Secure TSC
------------------
Secure TSC guest patches are available as part of v6.14.
QEMU changes:
https://github.com/AMDESE/qemu/tree/snp-securetsc-latest
QEMU command line SEV-SNP with Secure TSC:
qemu-system-x86_64 -cpu EPYC-Milan-v2 -smp 4 \
-object memory-backend-memfd,id=ram1,size=1G,share=true,prealloc=false,reserve=false \
-object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,secure-tsc=on,stsc-freq=2000000000 \
-machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
...Changelog:
----------
v10:
* Rebased on kvm/next
* Collect RB from Kai Huang
v9: https://lore.kernel.org/kvm/[email protected]/
* Set guest_tsc_protected during guest vCPU creation (Kai Huang)
* Improve error handling (Kai Huang)
* Disable MSR_AMD64_GUEST_TSC_FREQ write interception (Sean)
1. https://lore.kernel.org/kvm/[email protected]/
Nikunj A Dadhania (2):
x86/cpufeatures: Add SNP Secure TSC
KVM: SVM: Enable Secure TSC for SNP guests
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/svm.h | 1 + arch/x86/kvm/svm/sev.c | 27 +++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/svm/svm.h | 2 ++ 5 files changed, 33 insertions(+)
base-commit: 196d9e72c4b0bd68b74a4ec7f52d248f37d0f030
prerequisite-patch-id: 97788f545096df9ccc70cc99571d545fe4503f01
prerequisite-patch-id: 7a0edf8fa18231f19c781b4d412df4bafcb3d1ae
Signature · 1 line
2.43.0
PATCH v10 1/2 x86/cpufeatures: Add SNP Secure TSC
The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched. For more details, refer to the AMD64 APM Vol 2, Section "Secure TSC".
--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -443,6 +443,7 @@#define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2)/* VM Page Flush MSR is supported */ #define X86_FEATURE_SEV_ES (19*32+ 3)/* "sev_es" Secure Encrypted Virtualization - Encrypted State */ #define X86_FEATURE_SEV_SNP (19*32+ 4)/* "sev_snp" Secure Encrypted Virtualization - Secure Nested Paging */ +#define X86_FEATURE_SNP_SECURE_TSC (19*32+ 8)/* SEV-SNP Secure TSC */ #define X86_FEATURE_V_TSC_AUX (19*32+ 9)/* Virtual TSC_AUX */ #define X86_FEATURE_SME_COHERENT (19*32+10)/* hardware-enforced cache coherency */ #define X86_FEATURE_DEBUG_SWAP (19*32+14)/* "debug_swap" SEV-ES full debug state swap support */
Signature · 1 line
2.43.0
PATCH v10 2/2 KVM: SVM: Enable Secure TSC for SNP guests
In reply to Nikunj A DadhaniaAdd support for Secure TSC, allowing userspace to configure the Secure TSC feature for SNP guests. Use the SNP specification's desired TSC frequency parameter during the SNP_LAUNCH_START command to set the mean TSC frequency in KHz for Secure TSC enabled guests.
Always use kvm->arch.arch.default_tsc_khz as the TSC frequency that is passed to SNP guests in the SNP_LAUNCH_START command. The default value is the host TSC frequency. The userspace can optionally change the TSC frequency via the KVM_SET_TSC_KHZ ioctl before calling the SNP_LAUNCH_START ioctl.
Introduce the read-only MSR GUEST_TSC_FREQ (0xc0010134) that returns guest's effective frequency in MHZ when Secure TSC is enabled for SNP guests. Disable interception of this MSR when Secure TSC is enabled. Note that GUEST_TSC_FREQ MSR is accessible only to the guest and not from the hypervisor context.
--- arch/x86/include/asm/svm.h | 1 + arch/x86/kvm/svm/sev.c | 27 +++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/svm/svm.h | 2 ++ 4 files changed, 32 insertions(+)
arch/x86/include/asm/svm.h +1 -0
@@ -299,6 +299,7 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_#define SVM_SEV_FEAT_RESTRICTED_INJECTION BIT(3)#define SVM_SEV_FEAT_ALTERNATE_INJECTION BIT(4)#define SVM_SEV_FEAT_DEBUG_SWAP BIT(5) +#define SVM_SEV_FEAT_SECURE_TSC BIT(9)#define VMCB_ALLOWED_SEV_FEATURES_VALID BIT_ULL(63)
arch/x86/kvm/svm/sev.c +27 -0
@@ -146,6 +146,14 @@ static bool sev_vcpu_has_debug_swap(struct vcpu_svm *svm)return sev->vmsa_features &SVM_SEV_FEAT_DEBUG_SWAP ; } +bool snp_secure_tsc_enabled (struct kvm *kvm) +{ +struct kvm_sev_info *sev =to_kvm_sev_info (kvm); + +return (sev->vmsa_features &SVM_SEV_FEAT_SECURE_TSC ) && + !WARN_ON_ONCE (!sev_snp_guest (kvm)); +} +/* Must be called with the sev_bitmap_lock held */ static bool __sev_recycle_asids (unsigned int min_asid,unsigned int max_asid) {
@@ -415,6 +423,9 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,if (data->flags)return -EINVAL ; +if (!snp_active) + valid_vmsa_features &= ~SVM_SEV_FEAT_SECURE_TSC ; +if (data->vmsa_features & ~valid_vmsa_features)return -EINVAL ;
@@ -2195,6 +2206,16 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) start.gctx_paddr =__psp_pa (sev->snp_context); start.policy = params.policy; + +if (snp_secure_tsc_enabled (kvm)) { +if (WARN_ON (!kvm->arch.default_tsc_khz)) { + rc = -EINVAL ; + goto e_free_context; + } + + start.desired_tsc_khz = kvm->arch.default_tsc_khz; + } +memcpy (start.gosvw, params.gosvw,sizeof (params.gosvw)); rc =__sev_issue_cmd (argp->sev_fd,SEV_CMD_SNP_LAUNCH_START , &start, &argp->error);if (rc) {
@@ -3085,6 +3106,9 @@ void __init sev_hardware_setup(void) sev_supported_vmsa_features =0 ;if (sev_es_debug_swap_enabled) sev_supported_vmsa_features |=SVM_SEV_FEAT_DEBUG_SWAP ; + +if (sev_snp_enabled && tsc_khz &&cpu_feature_enabled (X86_FEATURE_SNP_SECURE_TSC )) + sev_supported_vmsa_features |=SVM_SEV_FEAT_SECURE_TSC ; }void sev_hardware_unsetup (void )
@@ -4455,6 +4479,9 @@ void sev_es_recalc_msr_intercepts(struct kvm_vcpu *vcpu) !guest_cpu_cap_has (vcpu,X86_FEATURE_RDTSCP ) && !guest_cpu_cap_has(vcpu, X86_FEATURE_RDPID)); +if (snp_secure_tsc_enabled (vcpu->kvm)) +svm_disable_intercept_for_msr (vcpu,MSR_AMD64_GUEST_TSC_FREQ ,MSR_TYPE_R ); +/* * For SEV-ES, accesses to MSR_IA32_XSS should not be intercepted if * the host/guest supports its use.
arch/x86/kvm/svm/svm.c +2 -0
@@ -1317,6 +1317,8 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu) svm->guest_state_loaded = false; + vcpu->arch.guest_tsc_protected =snp_secure_tsc_enabled (vcpu->kvm); +return 0 ; error_free_vmsa_page:
arch/x86/kvm/svm/svm.h +2 -0
@@ -869,6 +869,7 @@ void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end);int sev_private_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn);struct vmcb_save_area *sev_decrypt_vmsa (struct kvm_vcpu *vcpu);void sev_free_decrypted_vmsa (struct kvm_vcpu *vcpu,struct vmcb_save_area *vmsa); +bool snp_secure_tsc_enabled (struct kvm *kvm);#else static inline struct page *snp_safe_alloc_page_node (int node,gfp_t gfp) {
@@ -905,6 +906,7 @@ static inline struct vmcb_save_area *sev_decrypt_vmsa(struct kvm_vcpu *vcpu) return NULL; }static inline void sev_free_decrypted_vmsa (struct kvm_vcpu *vcpu,struct vmcb_save_area *vmsa) {} +static inline bool snp_secure_tsc_enabled (struct kvm *kvm) {return false; }#endif /* vmenter.S */
Signature · 1 line
2.43.0
Sean Christopherson
maintainerRe: [PATCH v10 2/2] KVM: SVM: Enable Secure TSC for SNP guests
Quoted 46 lines · Nikunj A Dadhania
Add support for Secure TSC, allowing userspace to configure the Secure TSC feature for SNP guests. Use the SNP specification's desired TSC frequency parameter during the
SNP_LAUNCH_STARTcommand to set the mean TSC frequency in KHz for Secure TSC enabled guests.Always use
kvm->arch.arch.default_tsc_khzas the TSC frequency that is passed to SNP guests in theSNP_LAUNCH_STARTcommand. The default value is the host TSC frequency. The userspace can optionally change the TSC frequency via theKVM_SET_TSC_KHZioctl before calling theSNP_LAUNCH_STARTioctl.Introduce the read-only MSR
GUEST_TSC_FREQ(0xc0010134) that returns guest's effective frequency in MHZ when Secure TSC is enabled for SNP guests. Disable interception of this MSR when Secure TSC is enabled. Note thatGUEST_TSC_FREQMSR is accessible only to the guest and not from the hypervisor context.Co-developed-by Ketan Chaturvedi <[email protected]>Signed-off-by Ketan Chaturvedi <[email protected]>Reviewed-by Kai Huang <[email protected]>Reviewed-by Tom Lendacky <[email protected]>Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/svm.h | 1 + arch/x86/kvm/svm/sev.c | 27 +++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/svm/svm.h | 2 ++ 4 files changed, 32 insertions(+)arch/x86/include/asm/svm.h +1
-0@@ -299,6 +299,7 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_#define SVM_SEV_FEAT_RESTRICTED_INJECTION BIT(3)#define SVM_SEV_FEAT_ALTERNATE_INJECTION BIT(4)#define SVM_SEV_FEAT_DEBUG_SWAP BIT(5) +#define SVM_SEV_FEAT_SECURE_TSC BIT(9)#define VMCB_ALLOWED_SEV_FEATURES_VALID BIT_ULL(63)arch/x86/kvm/svm/sev.c +0
-0
arch/x86/kvm/svm/sev.c static bool sev_vcpu_has_debug_swap(struct vcpu_svm *svm) · line 149
Quoted 4 lines
arch/x86/kvm/svm/sev.c +1
-0@@ -146,6 +146,14 @@ static bool sev_vcpu_has_debug_swap(struct vcpu_svm *svm)return sev->vmsa_features &SVM_SEV_FEAT_DEBUG_SWAP ; } +bool snp_secure_tsc_enabled (struct kvm *kvm)
snp_is_secure_tsc_enabled() to make it super obvious this is a predicate.
arch/x86/kvm/svm/sev.c static bool sev_vcpu_has_debug_swap(struct vcpu_svm *svm) · line 154
Quoted 5 lines
arch/x86/kvm/svm/sev.c +5
-0+{ +struct kvm_sev_info *sev =to_kvm_sev_info (kvm); + +return (sev->vmsa_features &SVM_SEV_FEAT_SECURE_TSC ) && + !WARN_ON_ONCE (!sev_snp_guest (kvm));
Align indentation.
arch/x86/kvm/svm/sev.c void sev_es_recalc_msr_intercepts(struct kvm_vcpu *vcpu) · line 4483
Quoted 7 lines
arch/x86/kvm/svm/sev.c +3
-0+}@@ -4455,6 +4479,9 @@ void sev_es_recalc_msr_intercepts(struct kvm_vcpu *vcpu) !guest_cpu_cap_has (vcpu,X86_FEATURE_RDTSCP ) && !guest_cpu_cap_has(vcpu, X86_FEATURE_RDPID)); +if (snp_secure_tsc_enabled (vcpu->kvm)) +svm_disable_intercept_for_msr (vcpu,MSR_AMD64_GUEST_TSC_FREQ ,MSR_TYPE_R );
I'm leaning towards:
svm_set_intercept_for_msr(vcpu, MSR_AMD64_GUEST_TSC_FREQ, MSR_TYPE_R, !snp_is_secure_tsc_enabled(vcpu->kvm));
because the cost of setting a bit is negligible.
arch/x86/kvm/svm/svm.c static int svm_vcpu_create(struct kvm_vcpu *vcpu) · line 1320
Quoted 13 lines
arch/x86/kvm/svm/sev.c +1
-0+/* * For SEV-ES, accesses to MSR_IA32_XSS should not be intercepted if * the host/guest supports its use. arch/x86/kvm/svm/svm.c +1
-0@@ -1317,6 +1317,8 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu) svm->guest_state_loaded = false; + vcpu->arch.guest_tsc_protected =snp_secure_tsc_enabled (vcpu->kvm);
Hmm, we can and should handle this in sev.c. If we add sev_vcpu_create(), then we don't need to expose snp_is_secure_tsc_enabled(), and we can move more code into that helper.
I'll post a combined series of this and the GHCB version patches.
Re: [PATCH v10 2/2] KVM: SVM: Enable Secure TSC for SNP guests
Quoted 50 lines · Sean Christopherson
Quoted 49 lines · Nikunj A Dadhania
Add support for Secure TSC, allowing userspace to configure the Secure TSC feature for SNP guests. Use the SNP specification's desired TSC frequency parameter during the
SNP_LAUNCH_STARTcommand to set the mean TSC frequency in KHz for Secure TSC enabled guests.Always use
kvm->arch.arch.default_tsc_khzas the TSC frequency that is passed to SNP guests in theSNP_LAUNCH_STARTcommand. The default value is the host TSC frequency. The userspace can optionally change the TSC frequency via theKVM_SET_TSC_KHZioctl before calling theSNP_LAUNCH_STARTioctl.Introduce the read-only MSR
GUEST_TSC_FREQ(0xc0010134) that returns guest's effective frequency in MHZ when Secure TSC is enabled for SNP guests. Disable interception of this MSR when Secure TSC is enabled. Note thatGUEST_TSC_FREQMSR is accessible only to the guest and not from the hypervisor context.Co-developed-by Ketan Chaturvedi <[email protected]>Signed-off-by Ketan Chaturvedi <[email protected]>Reviewed-by Kai Huang <[email protected]>Reviewed-by Tom Lendacky <[email protected]>Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/svm.h | 1 + arch/x86/kvm/svm/sev.c | 27 +++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/svm/svm.h | 2 ++ 4 files changed, 32 insertions(+)arch/x86/include/asm/svm.h +1
-0@@ -299,6 +299,7 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_#define SVM_SEV_FEAT_RESTRICTED_INJECTION BIT(3)#define SVM_SEV_FEAT_ALTERNATE_INJECTION BIT(4)#define SVM_SEV_FEAT_DEBUG_SWAP BIT(5) +#define SVM_SEV_FEAT_SECURE_TSC BIT(9)#define VMCB_ALLOWED_SEV_FEATURES_VALID BIT_ULL(63)arch/x86/kvm/svm/sev.c +0
-0@@ -146,6 +146,14 @@ static bool sev_vcpu_has_debug_swap(struct vcpu_svm *svm)return sev->vmsa_features &SVM_SEV_FEAT_DEBUG_SWAP ; }
Quoted 3 lines
arch/x86/kvm/svm/sev.c static bool sev_vcpu_has_debug_swap(struct vcpu_svm *svm) · line 149
Quoted 1 line
arch/x86/kvm/svm/sev.c +1
-0+bool snp_secure_tsc_enabled (struct kvm *kvm)
snp_is_secure_tsc_enabled()to make it super obvious this is a predicate.
Ack.
Quoted 19 lines
arch/x86/kvm/svm/sev.c static bool sev_vcpu_has_debug_swap(struct vcpu_svm *svm) · line 154
Quoted 6 lines
arch/x86/kvm/svm/sev.c +5
-0+{ +struct kvm_sev_info *sev =to_kvm_sev_info (kvm); + +return (sev->vmsa_features &SVM_SEV_FEAT_SECURE_TSC ) && + !WARN_ON_ONCE (!sev_snp_guest (kvm));Align indentation.
arch/x86/kvm/svm/sev.c void sev_es_recalc_msr_intercepts(struct kvm_vcpu *vcpu) · line 4483
Quoted 7 lines
arch/x86/kvm/svm/sev.c +3
-0+}@@ -4455,6 +4479,9 @@ void sev_es_recalc_msr_intercepts(struct kvm_vcpu *vcpu) !guest_cpu_cap_has (vcpu,X86_FEATURE_RDTSCP ) && !guest_cpu_cap_has(vcpu, X86_FEATURE_RDPID)); +if (snp_secure_tsc_enabled (vcpu->kvm)) +svm_disable_intercept_for_msr (vcpu,MSR_AMD64_GUEST_TSC_FREQ ,MSR_TYPE_R );I'm leaning towards:
Quoted 5 lines
svm_set_intercept_for_msr(vcpu, MSR_AMD64_GUEST_TSC_FREQ, MSR_TYPE_R, !snp_is_secure_tsc_enabled(vcpu->kvm));because the cost of setting a bit is negligible.
Ack.
Quoted 12 lines
Quoted 12 lines
arch/x86/kvm/svm/sev.c +1
-0+/* * For SEV-ES, accesses to MSR_IA32_XSS should not be intercepted if * the host/guest supports its use. arch/x86/kvm/svm/svm.c +0
-0@@ -1317,6 +1317,8 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu) svm->guest_state_loaded = false;
Quoted 7 lines
arch/x86/kvm/svm/svm.c static int svm_vcpu_create(struct kvm_vcpu *vcpu) · line 1320
Quoted 1 line
arch/x86/kvm/svm/svm.c +1
-0+ vcpu->arch.guest_tsc_protected =snp_secure_tsc_enabled (vcpu->kvm);Hmm, we can and should handle this in
sev.c. If we addsev_vcpu_create(), then we don't need to exposesnp_is_secure_tsc_enabled(), and we can move more code into that helper.I'll post a combined series of this and the GHCB version patches.
Thanks, I will test and get back on v11.
Regards
Nikunj
Quoted 24 lines · Nikunj A Dadhania
Patches are based on kvm/next with [1] applied
Testing Secure TSC
------------------Secure TSC guest patches are available as part of v6.14.
QEMU changes:
https://github.com/AMDESE/qemu/tree/snp-securetsc-latestQEMU command line SEV-SNP with Secure TSC:
qemu-system-x86_64 -cpu EPYC-Milan-v2 -smp 4 \ -object memory-backend-memfd,id=ram1,size=1G,share=true,prealloc=false,reserve=false \ -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,secure-tsc=on,stsc-freq=2000000000 \ -machine q35,confidential-guest-support=sev0,memory-backend=ram1 \ ...Changelog:
----------
v10:
* Rebased on kvm/next
* Collect RB from Kai Huang
Quoted 7 lines
v9: https://lore.kernel.org/kvm/[email protected]/
* Setguest_tsc_protectedduring guest vCPU creation (Kai Huang)
* Improve error handling (Kai Huang)
* DisableMSR_AMD64_GUEST_TSC_FREQwrite interception (Sean)
Hi Sean,
A gentle reminder for review/merge.
Regards,
Nikunj