Enable Secure TSC for SEV-SNP
cover Enable Secure TSC for SEV-SNP
TSC value calculations for the guest are controlled by the hypervisor. A malicious hypervisor can prevent guest from moving forward. The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions. This ensures the guest gets a consistent view of time and prevents a malicious hypervisor from making it appear that time rolls backwards, advancing at an unusually fast rate, or employing similar tricks. For more details, please refer to "Secure Nested Paging (SEV-SNP)" section, subsection "Secure TSC" of APM Volume 2
This patchset is also available at:
https://github.com/AMDESE/linux-kvm/tree/sectsc-host-latest
and is based on v6.11-rc5
Testing SecureTSC
-----------------
SecureTSC Guest patches:
https://github.com/AMDESE/linux-kvm/tree/sectsc-guest-latest
QEMU changes:
https://github.com/nikunjad/qemu/tree/snp-securetsc-latest
QEMU commandline SEV-SNP with SecureTSC:
qemu-system-x86_64 -cpu EPYC-Milan-v2 -smp 4 \
-object memory-backend-memfd,id=ram1,size=1G,share=true,prealloc=false,reserve=false \
-object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,secure-tsc=on \
-machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
...Ketan Chaturvedi (2):
KVM: SVM: Set TSC frequency for SecureTSC-enabled guests
KVM: SVM: Add Secure TSC support for SNP Guest
Nikunj A Dadhania (3):
x86/cpufeatures: Add SNP Secure TSC
KVM: SVM: Add GUEST_TSC_FREQ MSR
KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/kvm.h | 3 ++- arch/x86/kvm/svm/sev.c | 15 +++++++++++++++ arch/x86/kvm/svm/svm.c | 12 ++++++++++++ arch/x86/kvm/svm/svm.h | 14 +++++++++++++- include/linux/psp-sev.h | 2 ++ 8 files changed, 47 insertions(+), 2 deletions(-)
base-commit: 5be63fc19fcaa4c236b307420483578a56986a37
Signature · 1 line
2.34.1
RFC PATCH 1/5 x86/cpufeatures: Add SNP Secure TSC
The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched. More details in the AMD64 APM Vol 2, Section "Secure TSC".
--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -444,6 +444,7 @@#define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2)/* VM Page Flush MSR is supported */ #define X86_FEATURE_SEV_ES (19*32+ 3)/* "sev_es" AMD Secure Encrypted Virtualization - Encrypted State */ #define X86_FEATURE_SEV_SNP (19*32+ 4)/* "sev_snp" AMD Secure Encrypted Virtualization - Secure Nested Paging */ +#define X86_FEATURE_SNP_SECURE_TSC (19*32+ 8)/* "" AMD SEV-SNP Secure TSC */ #define X86_FEATURE_V_TSC_AUX (19*32+ 9)/* Virtual TSC_AUX */ #define X86_FEATURE_SME_COHERENT (19*32+10)/* AMD hardware-enforced cache coherency */ #define X86_FEATURE_DEBUG_SWAP (19*32+14)/* "debug_swap" AMD SEV-ES full debug state swap support */
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [RFC PATCH 1/5] x86/cpufeatures: Add SNP Secure TSC
Quoted 15 lines · Nikunj A Dadhania
The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched. More details in the AMD64 APM Vol 2, Section "Secure TSC".
Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)arch/x86/include/asm/cpufeatures.h +0
-0
arch/x86/include/asm/cpufeatures.h line 447
Quoted 4 lines
arch/x86/include/asm/cpufeatures.h +1
-0@@ -444,6 +444,7 @@#define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2)/* VM Page Flush MSR is supported */ #define X86_FEATURE_SEV_ES (19*32+ 3)/* "sev_es" AMD Secure Encrypted Virtualization - Encrypted State */ #define X86_FEATURE_SEV_SNP (19*32+ 4)/* "sev_snp" AMD Secure Encrypted Virtualization - Secure Nested Paging */ +#define X86_FEATURE_SNP_SECURE_TSC (19*32+ 8)/* "" AMD SEV-SNP Secure TSC */
There's a newline here on purpose - keep it.
Also, you don't need "" anymore.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC PATCH 1/5] x86/cpufeatures: Add SNP Secure TSC
Quoted 11 lines · Borislav Petkov
Quoted 10 lines · Nikunj A Dadhania
The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions, ensuring that the parameters used cannot be altered by the hypervisor once the guest is launched. More details in the AMD64 APM Vol 2, Section "Secure TSC".
Signed-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
Quoted 11 lines
arch/x86/include/asm/cpufeatures.h line 447
Quoted 9 lines
arch/x86/include/asm/cpufeatures.h +1
-0@@ -444,6 +444,7 @@#define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2)/* VM Page Flush MSR is supported */ #define X86_FEATURE_SEV_ES (19*32+ 3)/* "sev_es" AMD Secure Encrypted Virtualization - Encrypted State */ #define X86_FEATURE_SEV_SNP (19*32+ 4)/* "sev_snp" AMD Secure Encrypted Virtualization - Secure Nested Paging */ +#define X86_FEATURE_SNP_SECURE_TSC (19*32+ 8)/* "" AMD SEV-SNP Secure TSC */ There's a newline here on purpose - keep it.
Sure
Quoted 1 line
Also, you don't need "" anymore.
Ok, do we need to add an entry to tools/arch/x86/kcpuid/cpuid.csv ?
Regards
Nikunj
Borislav Petkov
maintainerRe: [RFC PATCH 1/5] x86/cpufeatures: Add SNP Secure TSC
Quoted 1 line · Nikunj A. Dadhania
Ok, do we need to add an entry to
tools/arch/x86/kcpuid/cpuid.csv?
Already there:
# Leaf 8000001FH
# AMD encrypted memory capabilities enumeration (SME/SEV)
0x8000001f, 0, eax, 0, sme , Secure Memory Encryption supported 0x8000001f, 0, eax, 1, sev , Secure Encrypted Virtualization supported 0x8000001f, 0, eax, 2, vm_page_flush , VM Page Flush MSR (0xc001011e) available 0x8000001f, 0, eax, 3, sev_es , SEV Encrypted State supported 0x8000001f, 0, eax, 4, sev_nested_paging , SEV secure nested paging supported 0x8000001f, 0, eax, 5, vm_permission_levels , VMPL supported 0x8000001f, 0, eax, 6, rpmquery , RPMQUERY instruction supported 0x8000001f, 0, eax, 7, vmpl_sss , VMPL supervisor shadwo stack supported 0x8000001f, 0, eax, 8, secure_tsc , Secure TSC supported
^^^^
but in general if it is not there, most definitely.
This list should contain *all* CPUID definitions.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC PATCH 1/5] x86/cpufeatures: Add SNP Secure TSC
Quoted 4 lines · Borislav Petkov
Quoted 1 line · Nikunj A. Dadhania
Ok, do we need to add an entry to
tools/arch/x86/kcpuid/cpuid.csv?Already there:
Ah ok, in tip/master
Quoted 6 lines
0x8000001f, 0, eax, 8,secure_tsc, Secure TSC supported
^^^^but in general if it is not there, most definitely.
This list should contain *all* CPUID definitions.
Sure
Regards
Nikunj
Borislav Petkov
maintainerRe: [RFC PATCH 1/5] x86/cpufeatures: Add SNP Secure TSC
Quoted 1 line · Nikunj A. Dadhania
Ah ok, in tip/master
What else?
See, we document these things not to waste space on people's hdds:
Documentation/process/maintainer-tip.rst
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRFC PATCH 2/5 KVM: SVM: Set TSC frequency for SecureTSC-enabled guests
In reply to Nikunj A DadhaniaFrom: Ketan Chaturvedi <[email protected]>
The SNP specification provides a desired TSC frequency parameter that can be set as part of the SNP_LAUNCH_START command. This field has effect only in Secure TSC enabled guests and can be used by the hypervisor to set desired mean TSC frequency in KHz of the guest.
--- arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/kvm.h | 3 ++- arch/x86/kvm/svm/sev.c | 7 +++++++ arch/x86/kvm/svm/svm.h | 12 ++++++++++++ include/linux/psp-sev.h | 2 ++ 5 files changed, 24 insertions(+), 1 deletion(-)
arch/x86/include/asm/svm.h +1 -0
@@ -289,6 +289,7 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_#define SVM_SEV_FEAT_RESTRICTED_INJECTION BIT(3)#define SVM_SEV_FEAT_ALTERNATE_INJECTION BIT(4)#define SVM_SEV_FEAT_DEBUG_SWAP BIT(5) +#define SVM_SEV_FEAT_SECURE_TSC BIT(9)#define SVM_SEV_FEAT_INT_INJ_MODES \ (SVM_SEV_FEAT_RESTRICTED_INJECTION | \
arch/x86/include/uapi/asm/kvm.h +2 -1
@@ -834,7 +834,8 @@ struct kvm_sev_snp_launch_start {__u64 policy;__u8 gosvw[16 ];__u16 flags; -__u8 pad0[6 ]; +__u32 desired_tsc_freq; +__u8 pad0[2 ];__u64 pad1[4 ]; };
arch/x86/kvm/svm/sev.c +7 -0
@@ -2212,6 +2212,9 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)if (sev->snp_context)return -EINVAL ; +if (snp_secure_tsc_enabled (kvm) && !params.desired_tsc_freq) +return -EINVAL ; + sev->snp_context =snp_context_create (kvm, argp);if (!sev->snp_context)return -ENOTTY ;
@@ -2232,6 +2235,10 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) start.gctx_paddr =__psp_pa (sev->snp_context); start.policy = params.policy; + +if (snp_secure_tsc_enabled (kvm)) + start.desired_tsc_freq = params.desired_tsc_freq; +memcpy (start.gosvw, params.gosvw,sizeof (params.gosvw)); rc =__sev_issue_cmd (argp->sev_fd,SEV_CMD_SNP_LAUNCH_START , &start, &argp->error);if (rc) {
arch/x86/kvm/svm/svm.h +12 -0
@@ -378,6 +378,18 @@ static __always_inline bool sev_snp_guest(struct kvm *kvm)#endif } +static inline bool snp_secure_tsc_enabled (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_AMD_SEV +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; + +return (sev->vmsa_features &SVM_SEV_FEAT_SECURE_TSC ) && + !WARN_ON_ONCE (!sev_snp_guest (kvm)); +#else +return false; +#endif +} +static inline bool ghcb_gpa_is_registered (struct vcpu_svm *svm,u64 val) {return svm->sev_es.ghcb_registered_gpa == val;
include/linux/psp-sev.h +2 -0
@@ -594,6 +594,7 @@ struct sev_data_snp_addr {* @imi_en: launch flow is launching an IMI (Incoming Migration Image) for the * purpose of guest-assisted migration. * @rsvd: reserved +* @desired_tsc_freq: hypervisor desired mean TSC freq in kHz of the guest * @gosvw: guest OS-visible workarounds, as defined by hypervisor */ struct sev_data_snp_launch_start {
@@ -603,6 +604,7 @@ struct sev_data_snp_launch_start { u32 ma_en:1; /* In */ u32 imi_en:1; /* In */ u32 rsvd:30; +u32 desired_tsc_freq;/* In */ u8 gosvw[16 ];/* In */ } __packed;
Signature · 1 line
2.34.1
RFC PATCH 3/5 KVM: SVM: Add GUEST_TSC_FREQ MSR
In reply to Nikunj A DadhaniaTSC calculation in Secure TSC enabled guests uses a new read-only MSR 0xc0010134 (GUEST_TSC_FREQ). Add the GUEST_TSC_FREQ MSR and disable the interception when secure TSC is enabled. Moreover, GUEST_TSC_FREQ MSR is only available to the guest and is not accessible from hypervisor context.
--- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kvm/svm/sev.c | 2 ++ arch/x86/kvm/svm/svm.c | 1 + arch/x86/kvm/svm/svm.h | 2 +- 4 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/include/asm/msr-index.h +1 -0
@@ -586,6 +586,7 @@#define MSR_AMD_PERF_CTL 0xc0010062#define MSR_AMD_PERF_STATUS 0xc0010063#define MSR_AMD_PSTATE_DEF_BASE 0xc0010064 +#define MSR_AMD64_GUEST_TSC_FREQ 0xc0010134#define MSR_AMD64_OSVW_ID_LENGTH 0xc0010140#define MSR_AMD64_OSVW_STATUS 0xc0010141#define MSR_AMD_PPIN_CTL 0xc00102f0
arch/x86/kvm/svm/sev.c +2 -0
@@ -851,6 +851,8 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm) save->dr6 = svm->vcpu.arch.dr6; save->sev_features = sev->vmsa_features; +if (save->sev_features &SVM_SEV_FEAT_SECURE_TSC ) +set_msr_interception (&svm->vcpu, svm->msrpm,MSR_AMD64_GUEST_TSC_FREQ ,1 ,1 );/* * Skip FPU and AVX setup with KVM_SEV_ES_INIT to avoid
arch/x86/kvm/svm/svm.c +1 -0
@@ -142,6 +142,7 @@ static const struct svm_direct_access_msrs { { .index = X2APIC_MSR(APIC_TMICT), .always = false }, { .index = X2APIC_MSR(APIC_TMCCT), .always = false }, { .index = X2APIC_MSR(APIC_TDCR), .always = false }, + { .index = MSR_AMD64_GUEST_TSC_FREQ, .always = false }, { .index = MSR_INVALID, .always = false }, };
arch/x86/kvm/svm/svm.h +1 -1
@@ -30,7 +30,7 @@#define IOPM_SIZE PAGE_SIZE * 3#define MSRPM_SIZE PAGE_SIZE * 2 -#define MAX_DIRECT_ACCESS_MSRS 48 +#define MAX_DIRECT_ACCESS_MSRS 49#define MSRPM_OFFSETS 32extern u32 msrpm_offsets[MSRPM_OFFSETS ]__read_mostly ;extern bool npt_enabled;
Signature · 1 line
2.34.1
RFC PATCH 4/5 KVM: SVM: Prevent writes to TSC MSR when Secure TSC is enabled
In reply to Nikunj A DadhaniaFor a Secure TSC enabled SNP guest, writes to MSR_IA32_TSC is not expected. Log the error and return #GP.
--- arch/x86/kvm/svm/svm.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/x86/kvm/svm/svm.c +11 -0
@@ -3151,6 +3151,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) svm->tsc_aux = data;break ; +case MSR_IA32_TSC : +/* +* If Secure TSC is enabled, KVM doesn't expect to receive +* a VMEXIT for a TSC write, record the error and return a +* #GP +*/ +if (vcpu->arch.guest_state_protected &&snp_secure_tsc_enabled (vcpu->kvm)) { +vcpu_unimpl (vcpu,"unimplemented IA32_TSC for secure tsc\n" ); +return 1 ; + } +break ;case MSR_IA32_DEBUGCTLMSR :if (!lbrv) {kvm_pr_unimpl_wrmsr (vcpu, ecx, data);
Signature · 1 line
2.34.1
RFC PATCH 5/5 KVM: SVM: Add Secure TSC support for SNP Guest
In reply to Nikunj A DadhaniaFrom: Ketan Chaturvedi <[email protected]>
Enable the Secure TSC capability and add it to SEV supported features to allow the userspave to set the Secure TSC feature for SNP guests.
--- arch/x86/kvm/svm/sev.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/x86/kvm/svm/sev.c +6 -0
@@ -2966,6 +2966,9 @@ void __init sev_set_cpu_caps(void)if (sev_snp_enabled) {kvm_cpu_cap_set (X86_FEATURE_SEV_SNP ); kvm_caps.supported_vm_types |=BIT (KVM_X86_SNP_VM ); + +if (cpu_feature_enabled (X86_FEATURE_SNP_SECURE_TSC )) +kvm_cpu_cap_set (X86_FEATURE_SNP_SECURE_TSC ); } }
@@ -3088,6 +3091,9 @@ void __init sev_hardware_setup(void) sev_supported_vmsa_features =0 ;if (sev_es_debug_swap_enabled) sev_supported_vmsa_features |=SVM_SEV_FEAT_DEBUG_SWAP ; + +if (sev_snp_enabled &&cpu_feature_enabled (X86_FEATURE_SNP_SECURE_TSC )) + sev_supported_vmsa_features |=SVM_SEV_FEAT_SECURE_TSC ; }void sev_hardware_unsetup (void )
Signature · 1 line
2.34.1
Quoted 8 lines · Nikunj A Dadhania
TSC value calculations for the guest are controlled by the hypervisor. A malicious hypervisor can prevent guest from moving forward. The Secure TSC feature for SEV-SNP allows guests to securely use the RDTSC and RDTSCP instructions. This ensures the guest gets a consistent view of time and prevents a malicious hypervisor from making it appear that time rolls backwards, advancing at an unusually fast rate, or employing similar tricks. For more details, please refer to "Secure Nested Paging (SEV-SNP)" section, subsection "Secure TSC" of APM Volume 2
Hello. Although I replied at [1], let raise this here too.
Don't we need to prevent the KVM from modifying KVM vcpu tsc offset/multiplier (vcpu->arch.tsc_offset etc.)?
As long as I understand, the spec (APM volume2) says the timer interrupt (TSC deadline timer or local APIC timer) is not virtualized by hardware so that KVM emulates timer interrupt.
If KVM modifies guest offset/multiplier from the original value (the SEV-SNP secure tsc uses or the TDX module uses), the timer interrupt emulation by KVM will be inaccurate. It's injected late or early than the guest expects.
Please notice that kvm_arch_vcpu_create() calls kvm_set_tsc_khz(vcpu, vcpu->kvm->arch.default_tsc_khz) after kvm_x86_call(vcpu_create)().
[1] https://lore.kernel.org/kvm/Zz%2FDGOoo%[email protected]/
Quoted 15 lines
This patchset is also available at:
https://github.com/AMDESE/linux-kvm/tree/sectsc-host-latest
and is based on v6.11-rc5
Testing SecureTSC
-----------------SecureTSC Guest patches:
https://github.com/AMDESE/linux-kvm/tree/sectsc-guest-latestQEMU changes:
https://github.com/nikunjad/qemu/tree/snp-securetsc-latest
Quoted 7 lines
QEMU commandline SEV-SNP with SecureTSC:
qemu-system-x86_64 -cpu EPYC-Milan-v2 -smp 4 \ -object memory-backend-memfd,id=ram1,size=1G,share=true,prealloc=false,reserve=false \ -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,secure-tsc=on \ -machine q35,confidential-guest-support=sev0,memory-backend=ram1 \ ...
Did you test it with tsc frequency/offset different from the kvm system default value (max_tsc_khz or kvm_caps.default_tsc_scaling_ranio etc.)?
Signature · 1 line
Isaku Yamahata <[email protected]>