Skip to contentSkip to conversation

Enable Secure TSC for SEV-SNP

PATCH Superseded Nikunj A Dadhania 5 participants 18 messages Feb 10, 2025 – Feb 14, 2025

Subsystems (3)

v2 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v2 cover Enable Secure TSC for SEV-SNP

The hypervisor controls TSC value calculations for the guest. A malicious hypervisor can prevent the guest from progressing. The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions. This ensures the guest has a consistent view of time and prevents a malicious hypervisor from manipulating time, such as making it appear to move backward or advance too quickly. For more details, refer to the "Secure Nested Paging (SEV-SNP)" section, subsection "Secure TSC" in APM Volume 2.

This patchset is also available at:

https://github.com/AMDESE/linux-kvm/tree/sectsc-host-latest

and is based on kvm/next

Testing SecureTSC
-----------------

SecureTSC guest patches are available as part of v6.14-rc1.

QEMU changes:
https://github.com/nikunjad/qemu/tree/snp-securetsc-latest

QEMU commandline SEV-SNP with SecureTSC:

  qemu-system-x86_64 -cpu EPYC-Milan-v2 -smp 4 \
    -object memory-backend-memfd,id=ram1,size=1G,share=true,prealloc=false,reserve=false \
    -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,secure-tsc=on,stsc-freq=2000000000 \
    -machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
    ...

Changelog:
----------
v2:
* Address cpufeatures comment from Boris
* Squashed Secure TSC enablement and setting frequency patch
* Set the default TSC KHz for proper calulation of guest offset/multiplier

Ketan Chaturvedi (1):
KVM: SVM: Enable Secure TSC for SNP guests

Nikunj A Dadhania (3):
x86/cpufeatures: Add SNP Secure TSC
KVM: SVM: Add GUEST_TSC_FREQ MSR for Secure TSC enabled guests
KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/include/asm/svm.h         |  1 +
 arch/x86/include/uapi/asm/kvm.h    |  3 ++-
 arch/x86/kvm/svm/sev.c             | 22 ++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             | 12 ++++++++++++
 arch/x86/kvm/svm/svm.h             | 14 +++++++++++++-
 include/linux/psp-sev.h            |  2 ++
 7 files changed, 53 insertions(+), 2 deletions(-)

base-commit: 43fb96ae78551d7bfa4ecca956b258f085d67c40

Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/4 x86/cpufeatures: Add SNP Secure TSC

The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched. For more details, refer to the AMD64 APM Vol 2, Section "Secure TSC".

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index 508c0dad116b..921ed26b0be7 100644
@@ -448,6 +448,7 @@
 #define X86_FEATURE_VM_PAGE_FLUSH	(19*32+ 2) /* VM Page Flush MSR is supported */
 #define X86_FEATURE_SEV_ES		(19*32+ 3) /* "sev_es" Secure Encrypted Virtualization - Encrypted State */
 #define X86_FEATURE_SEV_SNP		(19*32+ 4) /* "sev_snp" Secure Encrypted Virtualization - Secure Nested Paging */
+#define X86_FEATURE_SNP_SECURE_TSC	(19*32+ 8) /* SEV-SNP Secure TSC */
 #define X86_FEATURE_V_TSC_AUX		(19*32+ 9) /* Virtual TSC_AUX */
 #define X86_FEATURE_SME_COHERENT	(19*32+10) /* hardware-enforced cache coherency */
 #define X86_FEATURE_DEBUG_SWAP		(19*32+14) /* "debug_swap" SEV-ES full debug state swap support */
Signature · 1 line
2.43.0
maintainer Acked-by on v2 1/4
#2 lore raw

Re: [PATCH v2 1/4] x86/cpufeatures: Add SNP Secure TSC

Quoted 16 lines · Nikunj A Dadhania

The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched. For more details, refer to the AMD64 APM Vol 2, Section "Secure TSC".

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +0 -0
index 508c0dad116b..921ed26b0be7 100644

arch/x86/include/asm/cpufeatures.h line 454

Quoted 7 lines
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -448,6 +448,7 @@
 #define X86_FEATURE_VM_PAGE_FLUSH	(19*32+ 2) /* VM Page Flush MSR is supported */
 #define X86_FEATURE_SEV_ES		(19*32+ 3) /* "sev_es" Secure Encrypted Virtualization - Encrypted State */
 #define X86_FEATURE_SEV_SNP		(19*32+ 4) /* "sev_snp" Secure Encrypted Virtualization - Secure Nested Paging */
+#define X86_FEATURE_SNP_SECURE_TSC	(19*32+ 8) /* SEV-SNP Secure TSC */
 #define X86_FEATURE_V_TSC_AUX		(19*32+ 9) /* Virtual TSC_AUX */
 #define X86_FEATURE_SME_COHERENT	(19*32+10) /* hardware-enforced cache coherency */
 #define X86_FEATURE_DEBUG_SWAP		(19*32+14) /* "debug_swap" SEV-ES full debug state swap support */
Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#3 lore raw

PATCH v2 2/4 KVM: SVM: Add GUEST_TSC_FREQ MSR for Secure TSC enabled guests

In reply to Nikunj A Dadhania

Introduce the read-only MSR GUEST_TSC_FREQ (0xc0010134) that returns guest effective frequency in MHZ when secure TSC is enabled for SNP guests. Disable interception of this MSR when Secure TSC is enabled. Note that GUEST_TSC_FREQ MSR is accessible only to the guest and not from the hypervisor context.

---
 arch/x86/include/asm/svm.h |  1 +
 arch/x86/kvm/svm/sev.c     |  2 ++
 arch/x86/kvm/svm/svm.c     |  1 +
 arch/x86/kvm/svm/svm.h     | 14 +++++++++++++-
 4 files changed, 17 insertions(+), 1 deletion(-)
arch/x86/include/asm/svm.h +1 -0
index e2fac21471f5..a04346068c60 100644
@@ -289,6 +289,7 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_
 #define SVM_SEV_FEAT_RESTRICTED_INJECTION		BIT(3)
 #define SVM_SEV_FEAT_ALTERNATE_INJECTION		BIT(4)
 #define SVM_SEV_FEAT_DEBUG_SWAP				BIT(5)
+#define SVM_SEV_FEAT_SECURE_TSC				BIT(9)
 
 #define SVM_SEV_FEAT_INT_INJ_MODES		\
 	(SVM_SEV_FEAT_RESTRICTED_INJECTION |	\
arch/x86/kvm/svm/sev.c +2 -0
index a2a794c32050..0a1fd5c034e2 100644
@@ -849,6 +849,8 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm)
 	save->dr6  = svm->vcpu.arch.dr6;
 
 	save->sev_features = sev->vmsa_features;
+	if (snp_secure_tsc_enabled(vcpu->kvm))
+		set_msr_interception(&svm->vcpu, svm->msrpm, MSR_AMD64_GUEST_TSC_FREQ, 1, 1);
 
 	/*
 	 * Skip FPU and AVX setup with KVM_SEV_ES_INIT to avoid
arch/x86/kvm/svm/svm.c +1 -0
index 7640a84e554a..d7a0428aa2ae 100644
@@ -143,6 +143,7 @@ static const struct svm_direct_access_msrs {
 	{ .index = X2APIC_MSR(APIC_TMICT),		.always = false },
 	{ .index = X2APIC_MSR(APIC_TMCCT),		.always = false },
 	{ .index = X2APIC_MSR(APIC_TDCR),		.always = false },
+	{ .index = MSR_AMD64_GUEST_TSC_FREQ,		.always = false },
 	{ .index = MSR_INVALID,				.always = false },
 };
 
arch/x86/kvm/svm/svm.h +13 -1
index 9d7cdb8fbf87..8ef582c463e0 100644
@@ -44,7 +44,7 @@ static inline struct page *__sme_pa_to_page(unsigned long pa)
 #define	IOPM_SIZE PAGE_SIZE * 3
 #define	MSRPM_SIZE PAGE_SIZE * 2
 
-#define MAX_DIRECT_ACCESS_MSRS	48
+#define MAX_DIRECT_ACCESS_MSRS	49
 #define MSRPM_OFFSETS	32
 extern u32 msrpm_offsets[MSRPM_OFFSETS] __read_mostly;
 extern bool npt_enabled;
@@ -385,6 +385,18 @@ static __always_inline bool sev_snp_guest(struct kvm *kvm)
 #define sev_snp_guest(kvm) false
 #endif
 
+static inline bool snp_secure_tsc_enabled(struct kvm *kvm)
+{
+#ifdef CONFIG_KVM_AMD_SEV
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+
+	return (sev->vmsa_features & SVM_SEV_FEAT_SECURE_TSC) &&
+	       !WARN_ON_ONCE(!sev_snp_guest(kvm));
+#else
+	return false;
+#endif
+}
+
 static inline bool ghcb_gpa_is_registered(struct vcpu_svm *svm, u64 val)
 {
 	return svm->sev_es.ghcb_registered_gpa == val;
Signature · 1 line
2.43.0
#4 lore raw

PATCH v2 3/4 KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

In reply to Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

---
 arch/x86/kvm/svm/svm.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/kvm/svm/svm.c +11 -0
index d7a0428aa2ae..929f35a2f542 100644
@@ -3161,6 +3161,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 
 		svm->tsc_aux = data;
 		break;
+	case MSR_IA32_TSC:
+		/*
+		 * If Secure TSC is enabled, KVM doesn't expect to receive
+		 * a VMEXIT for a TSC write, record the error and return a
+		 * #GP
+		 */
+		if (vcpu->arch.guest_state_protected && snp_secure_tsc_enabled(vcpu->kvm)) {
+			vcpu_unimpl(vcpu, "unimplemented IA32_TSC for secure tsc\n");
+			return 1;
+		}
+		break;
 	case MSR_IA32_DEBUGCTLMSR:
 		if (!lbrv) {
 			kvm_pr_unimpl_wrmsr(vcpu, ecx, data);
Signature · 1 line
2.43.0
#5 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 2 lines · Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

Re-word this to make it a bit clearer about why this is needed. It is expected that the guest won't write to MSR_IA32_TSC or, if it does, it will ignore any writes to it and not exit to the HV. So this is catching the case where that behavior is not occurring.

Quoted 11 lines
---
 arch/x86/kvm/svm/svm.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/kvm/svm/svm.c +0 -0
index d7a0428aa2ae..929f35a2f542 100644

arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 3170

Quoted 10 lines
arch/x86/kvm/svm/svm.c +7 -0
@@ -3161,6 +3161,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 
 		svm->tsc_aux = data;
 		break;
+	case MSR_IA32_TSC:
+		/*
+		 * If Secure TSC is enabled, KVM doesn't expect to receive
+		 * a VMEXIT for a TSC write, record the error and return a
+		 * #GP
+		 */
+		if (vcpu->arch.guest_state_protected && snp_secure_tsc_enabled(vcpu->kvm)) {

Does it matter if the VMSA has already been encrypted? Can this just be

  if (sev_snp_guest() && snp_secure_tsc_enabled(vcpu->kvm)) {

?

Quoted 7 lines
arch/x86/kvm/svm/svm.c +4 -0
+			vcpu_unimpl(vcpu, "unimplemented IA32_TSC for secure tsc\n");
+			return 1;
+		}
+		break;
 	case MSR_IA32_DEBUGCTLMSR:
 		if (!lbrv) {
 			kvm_pr_unimpl_wrmsr(vcpu, ecx, data);
#6 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 9 lines · Tom Lendacky
Quoted 2 lines · Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

Re-word this to make it a bit clearer about why this is needed. It is expected that the guest won't write to MSR_IA32_TSC or, if it does, it will ignore any writes to it and not exit to the HV. So this is catching the case where that behavior is not occurring.

Sure, will update.

Quoted 24 lines
Quoted 11 lines
---
 arch/x86/kvm/svm/svm.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/kvm/svm/svm.c +0 -0
index d7a0428aa2ae..929f35a2f542 100644

arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 3170

Quoted 10 lines
arch/x86/kvm/svm/svm.c +7 -0
@@ -3161,6 +3161,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 
 		svm->tsc_aux = data;
 		break;
+	case MSR_IA32_TSC:
+		/*
+		 * If Secure TSC is enabled, KVM doesn't expect to receive
+		 * a VMEXIT for a TSC write, record the error and return a
+		 * #GP
+		 */
+		if (vcpu->arch.guest_state_protected && snp_secure_tsc_enabled(vcpu->kvm)) {

Does it matter if the VMSA has already been encrypted? Can this just be

Quoted 4 lines
  if (sev_snp_guest() && snp_secure_tsc_enabled(vcpu->kvm)) {

?

QEMU initializes the IA32_TSC MSR to zero resulting in the below error if I use the above.

qemu-system-x86_64: error: failed to set MSR 0x10 to 0x0
qemu-system-x86_64: ../target/i386/kvm/kvm.c:3849: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

Once the guest state is protected, we do not expect any writes from VMM.

Regards,
Nikunj

#7 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 39 lines · Nikunj A Dadhania
Quoted 9 lines · Tom Lendacky
Quoted 2 lines · Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

Re-word this to make it a bit clearer about why this is needed. It is expected that the guest won't write to MSR_IA32_TSC or, if it does, it will ignore any writes to it and not exit to the HV. So this is catching the case where that behavior is not occurring.

Sure, will update.

Quoted 26 lines
Quoted 11 lines
---
 arch/x86/kvm/svm/svm.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/kvm/svm/svm.c +0 -0
index d7a0428aa2ae..929f35a2f542 100644

arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 3170

Quoted 10 lines
arch/x86/kvm/svm/svm.c +7 -0
@@ -3161,6 +3161,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 
 		svm->tsc_aux = data;
 		break;
+	case MSR_IA32_TSC:
+		/*
+		 * If Secure TSC is enabled, KVM doesn't expect to receive
+		 * a VMEXIT for a TSC write, record the error and return a
+		 * #GP
+		 */
+		if (vcpu->arch.guest_state_protected && snp_secure_tsc_enabled(vcpu->kvm)) {

Does it matter if the VMSA has already been encrypted? Can this just be

  if (sev_snp_guest() && snp_secure_tsc_enabled(vcpu->kvm)) {
Quoted 8 lines
Quoted 2 lines

?

QEMU initializes the IA32_TSC MSR to zero resulting in the below error if I use the above.

qemu-system-x86_64: error: failed to set MSR 0x10 to 0x0
qemu-system-x86_64: ../target/i386/kvm/kvm.c:3849: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

Should KVM be doing anything related to MSR_IA32_TSC for a Secure TSC guest, even handling this Qemu write? That Qemu write takes it through the kvm_synchronize_tsc() path, does it need to? I'm just wondering if the Secure TSC HV support needs more handling of MSR_IA32_TSC (in both set and get) than what's here. Thoughts?

Thanks,
Tom

Quoted 5 lines

Once the guest state is protected, we do not expect any writes from VMM.

Regards,
Nikunj

#8 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 46 lines · Tom Lendacky
Quoted 45 lines · Nikunj A Dadhania
Quoted 9 lines · Tom Lendacky
Quoted 2 lines · Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

Re-word this to make it a bit clearer about why this is needed. It is expected that the guest won't write to MSR_IA32_TSC or, if it does, it will ignore any writes to it and not exit to the HV. So this is catching the case where that behavior is not occurring.

Sure, will update.

Quoted 24 lines
Quoted 11 lines
---
 arch/x86/kvm/svm/svm.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/kvm/svm/svm.c +0 -0
index d7a0428aa2ae..929f35a2f542 100644

arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 3170

Quoted 10 lines
arch/x86/kvm/svm/svm.c +7 -0
@@ -3161,6 +3161,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 
 		svm->tsc_aux = data;
 		break;
+	case MSR_IA32_TSC:
+		/*
+		 * If Secure TSC is enabled, KVM doesn't expect to receive
+		 * a VMEXIT for a TSC write, record the error and return a
+		 * #GP
+		 */
+		if (vcpu->arch.guest_state_protected && snp_secure_tsc_enabled(vcpu->kvm)) {

Does it matter if the VMSA has already been encrypted? Can this just be

Quoted 4 lines
  if (sev_snp_guest() && snp_secure_tsc_enabled(vcpu->kvm)) {

?

QEMU initializes the IA32_TSC MSR to zero resulting in the below error if I use the above.

Quoted 6 lines
Quoted 2 lines

qemu-system-x86_64: error: failed to set MSR 0x10 to 0x0
qemu-system-x86_64: ../target/i386/kvm/kvm.c:3849: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

Should KVM be doing anything related to MSR_IA32_TSC for a Secure TSC guest, even handling this Qemu write? That Qemu write takes it through the kvm_synchronize_tsc() path, does it need to? I'm just wondering if

Ah, strike that, the write from Qemu is just ignored in this situation. But doesn't that break existing support since we no longer call into kvm_set_msr_common() for MSR_IA32_TSC? This looks like it needs to invoke kvm_set_msr_common() if it isn't a Secure TSC guest.

Thanks,
Tom

Quoted 11 lines

the Secure TSC HV support needs more handling of MSR_IA32_TSC (in both set and get) than what's here. Thoughts?

Thanks,
Tom

Quoted 5 lines

Once the guest state is protected, we do not expect any writes from VMM.

Regards,
Nikunj

#9 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 10 lines · Tom Lendacky
Quoted 4 lines · Tom Lendacky
Quoted 3 lines · Nikunj A Dadhania
Quoted 1 line · Tom Lendacky

On 2/10/25 03:22, Nikunj A Dadhania wrote:

Ah, strike that, the write from Qemu is just ignored in this situation. But doesn't that break existing support since we no longer call into kvm_set_msr_common() for MSR_IA32_TSC? This looks like it needs to invoke kvm_set_msr_common() if it isn't a Secure TSC guest.

Right, I will add that.

Regards
Nikunj

#10 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

In reply to Tom Lendacky
Quoted 8 lines · Tom Lendacky
Quoted 2 lines · Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

Re-word this to make it a bit clearer about why this is needed. It is expected that the guest won't write to MSR_IA32_TSC or, if it does, it will ignore any writes to it and not exit to the HV. So this is catching the case where that behavior is not occurring.

Unless it's architectural impossible for KVM to modify MSR_IA32_TSC, I don't see any reason for KVM to care. If the guest wants to modify TSC, that's the guest's prerogative.

If KVM _can't_ honor the write, then that's something else entirely, and the changelog should pretty much write itself.

#11 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 10 lines · Sean Christopherson
Quoted 8 lines · Tom Lendacky
Quoted 2 lines · Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

Re-word this to make it a bit clearer about why this is needed. It is expected that the guest won't write to MSR_IA32_TSC or, if it does, it will ignore any writes to it and not exit to the HV. So this is catching the case where that behavior is not occurring.

Quoted 6 lines

Unless it's architectural impossible for KVM to modify MSR_IA32_TSC, I don't see any reason for KVM to care. If the guest wants to modify TSC, that's the guest's prerogative.

If KVM _can't_ honor the write, then that's something else entirely, and the changelog should pretty much write itself.

How about the below changelog:

KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Secure TSC enabled SNP guests should not write to the TSC MSR. Any such writes should be identified and ignored by the guest kernel in the #VC handler. As a safety measure, detect and disallow writes to MSR_IA32_TSC by Secure TSC enabled guests, as these writes are not expected to reach the hypervisor. Log the error and return #GP to the guest.

Additionally, incorporate a check for protected guest state to allow the VMM to initialize the TSC MSR.

Regards,
Nikunj

#12 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 16 lines · Nikunj A Dadhania
Quoted 14 lines · Sean Christopherson
Quoted 8 lines · Tom Lendacky
Quoted 2 lines · Nikunj A Dadhania

Disallow writes to MSR_IA32_TSC for Secure TSC enabled SNP guests, as such writes are not expected. Log the error and return #GP to the guest.

Re-word this to make it a bit clearer about why this is needed. It is expected that the guest won't write to MSR_IA32_TSC or, if it does, it will ignore any writes to it and not exit to the HV. So this is catching the case where that behavior is not occurring.

Unless it's architectural impossible for KVM to modify MSR_IA32_TSC, I don't see any reason for KVM to care. If the guest wants to modify TSC, that's the guest's prerogative.

Quoted 8 lines
Quoted 2 lines

If KVM _can't_ honor the write, then that's something else entirely, and the changelog should pretty much write itself.

How about the below changelog:

KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Secure TSC enabled SNP guests should not write to the TSC MSR. Any such

This is a host write, not a guest write. What guest's "should" or should not do is irrelevant.

Quoted 1 line

writes should be identified and ignored by the guest kernel in the #VC

Again, I don't care what the guest does. Talking about #VC just adds noise. E.g. if the guest requests WRMSR emulation without ever doing WRMSR, there will be no #VC.

Quoted 1 line

handler. As a safety measure, detect and disallow writes to MSR_IA32_TSC by

No, KVM is not the trusted monitor. "safety measure" makes it sound like KVM is protecting the guest from a malicious VMM. That is not KVM's responsibility.

Quoted 2 lines

Secure TSC enabled guests, as these writes are not expected to reach the hypervisor. Log the error and return #GP to the guest.

Again, none of this ever says what actually happens if KVM tries to emulate a write to MSR_IA32_TSC. Based on what the APM says, the TSC fields in the control area are ignored. _That's_ what's relevant.

The TSC value is first scaled with the GUEST_TSC_SCALE value from the VMSA and then is added to the VMSA GUEST_TSC_OFFSET value. The P0 frequency, TSC_RATIO (C001_0104h) and TSC_OFFSET (VMCB offset 50h) values are not used in the calculation.

#13 lore raw

Re: [PATCH v2 3/4] KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled

Quoted 11 lines · Sean Christopherson
Quoted 3 lines · Nikunj A Dadhania

Sean Christopherson <[email protected]> writes:
Secure TSC enabled guests, as these writes are not expected to reach the hypervisor. Log the error and return #GP to the guest.

Again, none of this ever says what actually happens if KVM tries to emulate a write to MSR_IA32_TSC. Based on what the APM says, the TSC fields in the control area are ignored. _That's_ what's relevant.

Quoted 1 line

The TSC value is first scaled with the GUEST_TSC_SCALE value from the VMSA and

then is added to the VMSA GUEST_TSC_OFFSET value. The P0 frequency, TSC_RATIO (C001_0104h) and TSC_OFFSET (VMCB offset 50h) values are not used in the calculation.

Sure, I will update the commit.

Thanks,
Nikunj

#15 lore raw

PATCH v2 4/4 KVM: SVM: Enable Secure TSC for SNP guests

In reply to Nikunj A Dadhania

From: Ketan Chaturvedi <[email protected]>

Add support for Secure TSC, allowing userspace to configure the Secure TSC feature for SNP guests. Use the SNP specification's desired TSC frequency parameter during the SNP_LAUNCH_START command to set the mean TSC frequency in KHz for Secure TSC enabled guests. If the frequency is not specified by the VMM, default to tsc_khz.

---
 arch/x86/include/uapi/asm/kvm.h |  3 ++-
 arch/x86/kvm/svm/sev.c          | 20 ++++++++++++++++++++
 include/linux/psp-sev.h         |  2 ++
 3 files changed, 24 insertions(+), 1 deletion(-)
arch/x86/include/uapi/asm/kvm.h +2 -1
index 9e75da97bce0..8e090cab9aa0 100644
@@ -836,7 +836,8 @@ struct kvm_sev_snp_launch_start {
 	__u64 policy;
 	__u8 gosvw[16];
 	__u16 flags;
-	__u8 pad0[6];
+	__u32 desired_tsc_khz;
+	__u8 pad0[2];
 	__u64 pad1[4];
 };
 
arch/x86/kvm/svm/sev.c +20 -0
index 0a1fd5c034e2..0edd473749f7 100644
@@ -2228,6 +2228,20 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 	start.gctx_paddr = __psp_pa(sev->snp_context);
 	start.policy = params.policy;
+
+	if (snp_secure_tsc_enabled(kvm)) {
+		u32 user_tsc_khz = params.desired_tsc_khz;
+
+		/* Use tsc_khz if the VMM has not provided the TSC frequency */
+		if (!user_tsc_khz)
+			user_tsc_khz = tsc_khz;
+
+		start.desired_tsc_khz = user_tsc_khz;
+
+		/* Set the arch default TSC for the VM*/
+		kvm->arch.default_tsc_khz = user_tsc_khz;
+	}
+
 	memcpy(start.gosvw, params.gosvw, sizeof(params.gosvw));
 	rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_START, &start, &argp->error);
 	if (rc) {
@@ -2949,6 +2963,9 @@ void __init sev_set_cpu_caps(void)
 	if (sev_snp_enabled) {
 		kvm_cpu_cap_set(X86_FEATURE_SEV_SNP);
 		kvm_caps.supported_vm_types |= BIT(KVM_X86_SNP_VM);
+
+		if (cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
+			kvm_cpu_cap_set(X86_FEATURE_SNP_SECURE_TSC);
 	}
 }
 
@@ -3071,6 +3088,9 @@ void __init sev_hardware_setup(void)
 	sev_supported_vmsa_features = 0;
 	if (sev_es_debug_swap_enabled)
 		sev_supported_vmsa_features |= SVM_SEV_FEAT_DEBUG_SWAP;
+
+	if (sev_snp_enabled && cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
+		sev_supported_vmsa_features |= SVM_SEV_FEAT_SECURE_TSC;
 }
 
 void sev_hardware_unsetup(void)
include/linux/psp-sev.h +2 -0
index 903ddfea8585..613a8209bed2 100644
@@ -594,6 +594,7 @@ struct sev_data_snp_addr {
  * @imi_en: launch flow is launching an IMI (Incoming Migration Image) for the
  *          purpose of guest-assisted migration.
  * @rsvd: reserved
+ * @desired_tsc_khz: hypervisor desired mean TSC freq in kHz of the guest
  * @gosvw: guest OS-visible workarounds, as defined by hypervisor
  */
 struct sev_data_snp_launch_start {
@@ -603,6 +604,7 @@ struct sev_data_snp_launch_start {
 	u32 ma_en:1;				/* In */
 	u32 imi_en:1;				/* In */
 	u32 rsvd:30;
+	u32 desired_tsc_khz;			/* In */
 	u8 gosvw[16];				/* In */
 } __packed;
 
Signature · 1 line
2.43.0
#16 lore raw

Re: [PATCH v2 4/4] KVM: SVM: Enable Secure TSC for SNP guests

Quoted 22 lines · Nikunj A Dadhania

From: Ketan Chaturvedi <[email protected]>

Add support for Secure TSC, allowing userspace to configure the Secure TSC feature for SNP guests. Use the SNP specification's desired TSC frequency parameter during the SNP_LAUNCH_START command to set the mean TSC frequency in KHz for Secure TSC enabled guests. If the frequency is not specified by the VMM, default to tsc_khz.

---
 arch/x86/include/uapi/asm/kvm.h |  3 ++-
 arch/x86/kvm/svm/sev.c          | 20 ++++++++++++++++++++
 include/linux/psp-sev.h         |  2 ++
 3 files changed, 24 insertions(+), 1 deletion(-)
arch/x86/include/uapi/asm/kvm.h +0 -0
index 9e75da97bce0..8e090cab9aa0 100644

arch/x86/include/uapi/asm/kvm.h struct kvm_sev_snp_launch_start { · line 839

Quoted 5 lines
arch/x86/include/uapi/asm/kvm.h +1 -1
@@ -836,7 +836,8 @@ struct kvm_sev_snp_launch_start {
 	__u64 policy;
 	__u8 gosvw[16];
 	__u16 flags;
-	__u8 pad0[6];
+	__u32 desired_tsc_khz;

This will put the __u32 field misaligned in the struct. You should probably move the now 2-byte pad0 field to before the desired_tsc_khz field.

Quoted 62 lines
arch/x86/include/uapi/asm/kvm.h +1 -0
+	__u8 pad0[2];
 	__u64 pad1[4];
 };
 
arch/x86/kvm/svm/sev.c +20 -0
index 0a1fd5c034e2..0edd473749f7 100644
@@ -2228,6 +2228,20 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 	start.gctx_paddr = __psp_pa(sev->snp_context);
 	start.policy = params.policy;
+
+	if (snp_secure_tsc_enabled(kvm)) {
+		u32 user_tsc_khz = params.desired_tsc_khz;
+
+		/* Use tsc_khz if the VMM has not provided the TSC frequency */
+		if (!user_tsc_khz)
+			user_tsc_khz = tsc_khz;
+
+		start.desired_tsc_khz = user_tsc_khz;
+
+		/* Set the arch default TSC for the VM*/
+		kvm->arch.default_tsc_khz = user_tsc_khz;
+	}
+
 	memcpy(start.gosvw, params.gosvw, sizeof(params.gosvw));
 	rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_START, &start, &argp->error);
 	if (rc) {
@@ -2949,6 +2963,9 @@ void __init sev_set_cpu_caps(void)
 	if (sev_snp_enabled) {
 		kvm_cpu_cap_set(X86_FEATURE_SEV_SNP);
 		kvm_caps.supported_vm_types |= BIT(KVM_X86_SNP_VM);
+
+		if (cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
+			kvm_cpu_cap_set(X86_FEATURE_SNP_SECURE_TSC);
 	}
 }
 
@@ -3071,6 +3088,9 @@ void __init sev_hardware_setup(void)
 	sev_supported_vmsa_features = 0;
 	if (sev_es_debug_swap_enabled)
 		sev_supported_vmsa_features |= SVM_SEV_FEAT_DEBUG_SWAP;
+
+	if (sev_snp_enabled && cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
+		sev_supported_vmsa_features |= SVM_SEV_FEAT_SECURE_TSC;
 }
 
 void sev_hardware_unsetup(void)
include/linux/psp-sev.h +1 -0
index 903ddfea8585..613a8209bed2 100644
@@ -594,6 +594,7 @@ struct sev_data_snp_addr {
  * @imi_en: launch flow is launching an IMI (Incoming Migration Image) for the
  *          purpose of guest-assisted migration.
  * @rsvd: reserved
+ * @desired_tsc_khz: hypervisor desired mean TSC freq in kHz of the guest
  * @gosvw: guest OS-visible workarounds, as defined by hypervisor
  */
 struct sev_data_snp_launch_start {

include/linux/psp-sev.h struct sev_data_snp_launch_start { · line 607

Quoted 4 lines
include/linux/psp-sev.h +1 -0
@@ -603,6 +604,7 @@ struct sev_data_snp_launch_start {
 	u32 ma_en:1;				/* In */
 	u32 imi_en:1;				/* In */
 	u32 rsvd:30;
+	u32 desired_tsc_khz;			/* In */

Shouldn't there be a separate fix for this before this patch? The desired_tsc_freq should have been here all along, so before this patch, the gosvw field is off by 4 bytes with sev_data_snp_launch_start not being large enough compared to what the firmware is accessing, right?

Thanks,
Tom

Quoted 3 lines
include/linux/psp-sev.h +0 -0
 	u8 gosvw[16];				/* In */
 } __packed;
 
#17 lore raw

Re: [PATCH v2 4/4] KVM: SVM: Enable Secure TSC for SNP guests

Quoted 18 lines · Tom Lendacky
Quoted 17 lines · Nikunj A Dadhania

From: Ketan Chaturvedi <[email protected]>

Add support for Secure TSC, allowing userspace to configure the Secure TSC feature for SNP guests. Use the SNP specification's desired TSC frequency parameter during the SNP_LAUNCH_START command to set the mean TSC frequency in KHz for Secure TSC enabled guests. If the frequency is not specified by the VMM, default to tsc_khz.

---
 arch/x86/include/uapi/asm/kvm.h |  3 ++-
 arch/x86/kvm/svm/sev.c          | 20 ++++++++++++++++++++
 include/linux/psp-sev.h         |  2 ++
 3 files changed, 24 insertions(+), 1 deletion(-)
Quoted 14 lines

arch/x86/include/uapi/asm/kvm.h struct kvm_sev_snp_launch_start { · line 839

Quoted 10 lines
arch/x86/include/uapi/asm/kvm.h +1 -1
index 9e75da97bce0..8e090cab9aa0 100644
@@ -836,7 +836,8 @@ struct kvm_sev_snp_launch_start {
 	__u64 policy;
 	__u8 gosvw[16];
 	__u16 flags;
-	__u8 pad0[6];
+	__u32 desired_tsc_khz;

This will put the __u32 field misaligned in the struct. You should probably move the now 2-byte pad0 field to before the desired_tsc_khz field.

Sure, will update.

Quoted 63 lines
Quoted 63 lines
arch/x86/include/uapi/asm/kvm.h +1 -0
+	__u8 pad0[2];
 	__u64 pad1[4];
 };
 
arch/x86/kvm/svm/sev.c +20 -0
index 0a1fd5c034e2..0edd473749f7 100644
@@ -2228,6 +2228,20 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 	start.gctx_paddr = __psp_pa(sev->snp_context);
 	start.policy = params.policy;
+
+	if (snp_secure_tsc_enabled(kvm)) {
+		u32 user_tsc_khz = params.desired_tsc_khz;
+
+		/* Use tsc_khz if the VMM has not provided the TSC frequency */
+		if (!user_tsc_khz)
+			user_tsc_khz = tsc_khz;
+
+		start.desired_tsc_khz = user_tsc_khz;
+
+		/* Set the arch default TSC for the VM*/
+		kvm->arch.default_tsc_khz = user_tsc_khz;
+	}
+
 	memcpy(start.gosvw, params.gosvw, sizeof(params.gosvw));
 	rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_START, &start, &argp->error);
 	if (rc) {
@@ -2949,6 +2963,9 @@ void __init sev_set_cpu_caps(void)
 	if (sev_snp_enabled) {
 		kvm_cpu_cap_set(X86_FEATURE_SEV_SNP);
 		kvm_caps.supported_vm_types |= BIT(KVM_X86_SNP_VM);
+
+		if (cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
+			kvm_cpu_cap_set(X86_FEATURE_SNP_SECURE_TSC);
 	}
 }
 
@@ -3071,6 +3088,9 @@ void __init sev_hardware_setup(void)
 	sev_supported_vmsa_features = 0;
 	if (sev_es_debug_swap_enabled)
 		sev_supported_vmsa_features |= SVM_SEV_FEAT_DEBUG_SWAP;
+
+	if (sev_snp_enabled && cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
+		sev_supported_vmsa_features |= SVM_SEV_FEAT_SECURE_TSC;
 }
 
 void sev_hardware_unsetup(void)
include/linux/psp-sev.h +1 -0
index 903ddfea8585..613a8209bed2 100644
@@ -594,6 +594,7 @@ struct sev_data_snp_addr {
  * @imi_en: launch flow is launching an IMI (Incoming Migration Image) for the
  *          purpose of guest-assisted migration.
  * @rsvd: reserved
+ * @desired_tsc_khz: hypervisor desired mean TSC freq in kHz of the guest
  * @gosvw: guest OS-visible workarounds, as defined by hypervisor
  */
 struct sev_data_snp_launch_start {
@@ -603,6 +604,7 @@ struct sev_data_snp_launch_start {
 	u32 ma_en:1;				/* In */
Quoted 8 lines

include/linux/psp-sev.h struct sev_data_snp_launch_start { · line 607

Quoted 3 lines
include/linux/psp-sev.h +1 -0
 	u32 imi_en:1;				/* In */
 	u32 rsvd:30;
+	u32 desired_tsc_khz;			/* In */

Shouldn't there be a separate fix for this before this patch? The desired_tsc_freq should have been here all along, so before this patch, the gosvw field is off by 4 bytes with sev_data_snp_launch_start not being large enough compared to what the firmware is accessing, right?

Yes, this should have been part of SNP series, I will separate this change in a new patch with Fixes + stable tag.

Thanks
Nikunj