Add TDX Guest Support
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Dave HansenQuoted 7 lines · Dave Hansen
Because the code is already separate. You're actually going to some trouble to move the SEV-specific code and then combine it with the TDX-specific code.
Anyway, please just give it a shot. Should take all of ten minutes. If it doesn't work out in practice, fine. You'll have a good paragraph for the changelog.
After reviewing the code again, I have noticed that we don't really have much common code between AMD and TDX. So I don't see any justification for creating this common layer. So, I have decided to drop this patch and move Intel TDX specific memory encryption init code to patch titled "[RFC v2 30/32] x86/tdx: Make DMA pages shared". This model is similar to how AMD-SEV does the initialization.
I have sent the modified patch as reply to patch titled "[RFC v2 30/32] x86/tdx: Make DMA pages shared". Please check and let me know your comments.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 15 lines · Sathyanarayanan
Quoted 7 lines
Because the code is already separate. You're actually going to some trouble to move the SEV-specific code and then combine it with the TDX-specific code.
Anyway, please just give it a shot. Should take all of ten minutes. If it doesn't work out in practice, fine. You'll have a good paragraph for the changelog.
After reviewing the code again, I have noticed that we don't really have much common code between AMD and TDX. So I don't see any justification for creating this common layer. So, I have decided to drop this patch and move Intel TDX specific memory encryption init code to patch titled "[RFC v2 30/32] x86/tdx: Make DMA pages shared". This model is similar to how AMD-SEV does the initialization.
Quoted 2 lines
I have sent the modified patch as reply to patch titled "[RFC v2 30/32] x86/tdx: Make DMA pages shared". Please check and let me know your comment
My method of using separate initialization file for Intel only code will not work if we want to support both AMD SEV and TDX guest support in same binary. So please ignore my previous reply. I will address the issue as per your original comments and send you an updated patch.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
RFC 1/1 x86/mm: Move force_dma_unencrypted() to common code
From: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to access guest private memory. Any memory that is required for communication with VMM must be shared explicitly by setting the bit in page table entry. After setting the shared bit, the conversion must be completed with MapGPA TDVMALL. The call informs VMM about the conversion between private/shared mappings. The shared memory is similar to unencrypted memory in AMD SME/SEV terminology but the underlying process of sharing/un-sharing the memory is different for Intel TDX guest platform.
SEV assumes that I/O devices can only do DMA to "decrypted" physical addresses without the C-bit set. In order for the CPU to interact with this memory, the CPU needs a decrypted mapping. To add this support, AMD SME code forces force_dma_unencrypted() to return true for platforms that support AMD SEV feature. It will be used for DMA memory allocation API to trigger set_memory_decrypted() for platforms that support AMD SEV feature.
TDX is similar. So, to communicate with I/O devices, related pages need to be marked as shared. As mentioned above, shared memory in TDX architecture is similar to decrypted memory in AMD SME/SEV. So similar to AMD SEV, force_dma_unencrypted() has to forced to return true. This support is added in other patches in this series.
So move force_dma_unencrypted() out of AMD specific code and call AMD specific (amd_force_dma_unencrypted()) initialization function from it. force_dma_unencrypted() will be modified by later patches to include Intel TDX guest platform specific initialization.
Also, introduce new config option X86_MEM_ENCRYPT_COMMON that has to be selected by all x86 memory encryption features. This will be selected by both AMD SEV and Intel TDX guest config options.
This is preparation for TDX changes in DMA code and it has not functional change.
---
Changes since RFC v2:
* Instead of moving all the contents of force_dma_unencrypted() to mem_encrypt_common.c, create sub function for AMD and call it from common code.
* Fixed commit log as per review comments.
arch/x86/Kconfig | 8 ++++++-- arch/x86/mm/Makefile | 2 ++ arch/x86/mm/mem_encrypt.c | 4 ++-- arch/x86/mm/mem_encrypt_common.c | 22 ++++++++++++++++++++++ 4 files changed, 32 insertions(+), 4 deletions(-) create mode 100644 arch/x86/mm/mem_encrypt_common.c
arch/x86/Kconfig +6 -2
@@ -1531,14 +1531,18 @@ config X86_CPA_STATISTICS helps to determine the effectiveness of preserving large and huge page mappings when mapping protections are changed. +config X86_MEM_ENCRYPT_COMMON + select ARCH_HAS_FORCE_DMA_UNENCRYPTED + select DYNAMIC_PHYSICAL_MASK + def_bool n + config AMD_MEM_ENCRYPT bool "AMD Secure Memory Encryption (SME) support" depends on X86_64 && CPU_SUP_AMD select DMA_COHERENT_POOL - select DYNAMIC_PHYSICAL_MASK select ARCH_USE_MEMREMAP_PROT - select ARCH_HAS_FORCE_DMA_UNENCRYPTED select INSTRUCTION_DECODER + select X86_MEM_ENCRYPT_COMMON help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory
arch/x86/mm/Makefile +2 -0
@@ -52,6 +52,8 @@ obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o +obj-$(CONFIG_X86_MEM_ENCRYPT_COMMON) += mem_encrypt_common.o + obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o
arch/x86/mm/mem_encrypt.c +2 -2
@@ -390,8 +390,8 @@ bool noinstr sev_es_active(void)return sev_status &MSR_AMD64_SEV_ES_ENABLED ; } -/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ -bool force_dma_unencrypted (struct device *dev) +/* Override for DMA direct allocation check - AMD specific initialization */ +bool amd_force_dma_unencrypted (struct device *dev) {/* * For SEV, all DMA must be to unencrypted addresses.
arch/x86/mm/mem_encrypt_common.c new file +22 -0
@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Memory Encryption Support Common Code +* +* Copyright (C) 2021 Intel Corporation +* +* Author: Kuppuswamy Sathyanarayanan <[email protected]> +*/ + +#include <linux/mem_encrypt.h> +#include <linux/dma-mapping.h> + +bool amd_force_dma_unencrypted (struct device *dev); + +/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ +bool force_dma_unencrypted (struct device *dev) +{ +if (sev_active () ||sme_active ()) +return amd_force_dma_unencrypted (dev); + +return false; +}
Signature · 1 line
2.25.1
RFC 1/1 x86/mm: Move force_dma_unencrypted() to common code
From: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to access guest private memory. Any memory that is required for communication with VMM must be shared explicitly by setting the bit in page table entry. After setting the shared bit, the conversion must be completed with MapGPA TDVMALL. The call informs VMM about the conversion between private/shared mappings. The shared memory is similar to unencrypted memory in AMD SME/SEV terminology but the underlying process of sharing/un-sharing the memory is different for Intel TDX guest platform.
SEV assumes that I/O devices can only do DMA to "decrypted" physical addresses without the C-bit set. In order for the CPU to interact with this memory, the CPU needs a decrypted mapping. To add this support, AMD SME code forces force_dma_unencrypted() to return true for platforms that support AMD SEV feature. It will be used for DMA memory allocation API to trigger set_memory_decrypted() for platforms that support AMD SEV feature.
TDX is similar. So, to communicate with I/O devices, related pages need to be marked as shared. As mentioned above, shared memory in TDX architecture is similar to decrypted memory in AMD SME/SEV. So similar to AMD SEV, force_dma_unencrypted() has to forced to return true. This support is added in other patches in this series.
So move force_dma_unencrypted() out of AMD specific code and call AMD specific (amd_force_dma_unencrypted()) initialization function from it. force_dma_unencrypted() will be modified by later patches to include Intel TDX guest platform specific initialization.
Also, introduce new config option X86_MEM_ENCRYPT_COMMON that has to be selected by all x86 memory encryption features. This will be selected by both AMD SEV and Intel TDX guest config options.
This is preparation for TDX changes in DMA code and it has not functional change.
---
Changes since RFC v2-fix-v1:
* Added mem_encrypt_common.h and moved common encryption related function declarations to it.
Changes since RFC v2:
* Instead of moving all the contents of force_dma_unencrypted() to mem_encrypt_common.c, create sub function for AMD and call it from common code.
* Fixed commit log as per review comments.
arch/x86/Kconfig | 8 ++++++-- arch/x86/include/asm/mem_encrypt_common.h | 18 ++++++++++++++++++ arch/x86/mm/Makefile | 2 ++ arch/x86/mm/mem_encrypt.c | 5 +++-- arch/x86/mm/mem_encrypt_common.c | 20 ++++++++++++++++++++ 5 files changed, 49 insertions(+), 4 deletions(-) create mode 100644 arch/x86/include/asm/mem_encrypt_common.h create mode 100644 arch/x86/mm/mem_encrypt_common.c
arch/x86/Kconfig +6 -2
@@ -1531,14 +1531,18 @@ config X86_CPA_STATISTICS helps to determine the effectiveness of preserving large and huge page mappings when mapping protections are changed. +config X86_MEM_ENCRYPT_COMMON + select ARCH_HAS_FORCE_DMA_UNENCRYPTED + select DYNAMIC_PHYSICAL_MASK + def_bool n + config AMD_MEM_ENCRYPT bool "AMD Secure Memory Encryption (SME) support" depends on X86_64 && CPU_SUP_AMD select DMA_COHERENT_POOL - select DYNAMIC_PHYSICAL_MASK select ARCH_USE_MEMREMAP_PROT - select ARCH_HAS_FORCE_DMA_UNENCRYPTED select INSTRUCTION_DECODER + select X86_MEM_ENCRYPT_COMMON help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory
arch/x86/include/asm/mem_encrypt_common.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2020 Intel Corporation */ +#ifndef _ASM_X86_MEM_ENCRYPT_COMMON_H +#define _ASM_X86_MEM_ENCRYPT_COMMON_H + +#include <linux/mem_encrypt.h> +#include <linux/device.h> + +#ifdef CONFIG_AMD_MEM_ENCRYPT +bool amd_force_dma_unencrypted (struct device *dev); +#else /* CONFIG_AMD_MEM_ENCRYPT */ +static inline bool amd_force_dma_unencrypted (struct device *dev) +{ +return false; +} +#endif /* CONFIG_AMD_MEM_ENCRYPT */ + +#endif
arch/x86/mm/Makefile +2 -0
@@ -52,6 +52,8 @@ obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o +obj-$(CONFIG_X86_MEM_ENCRYPT_COMMON) += mem_encrypt_common.o + obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o
arch/x86/mm/mem_encrypt.c +3 -2
@@ -29,6 +29,7 @@#include <asm/processor-flags.h> #include <asm/msr.h> #include <asm/cmdline.h> +#include <asm/mem_encrypt_common.h> #include "mm_internal.h"
@@ -390,8 +391,8 @@ bool noinstr sev_es_active(void)return sev_status &MSR_AMD64_SEV_ES_ENABLED ; } -/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ -bool force_dma_unencrypted (struct device *dev) +/* Override for DMA direct allocation check - AMD specific initialization */ +bool amd_force_dma_unencrypted (struct device *dev) {/* * For SEV, all DMA must be to unencrypted addresses.
arch/x86/mm/mem_encrypt_common.c new file +20 -0
@@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Memory Encryption Support Common Code +* +* Copyright (C) 2021 Intel Corporation +* +* Author: Kuppuswamy Sathyanarayanan <[email protected]> +*/ + +#include <asm/mem_encrypt_common.h> +#include <linux/dma-mapping.h> + +/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ +bool force_dma_unencrypted (struct device *dev) +{ +if (sev_active () ||sme_active ()) +return amd_force_dma_unencrypted (dev); + +return false; +}
Signature · 1 line
2.25.1
RFC v2 27/32 x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
tdx_shared_mask() returns the mask that has to be set in a page table entry to make page shared with VMM.
Also, note that we cannot club shared mapping configuration between AMD SME and Intel TDX Guest platforms in common function. SME has to do it very early in __startup_64() as it sets the bit on all memory, except what is used for communication. TDX can postpone as we don't need any shared mapping in very early boot.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/kernel/tdx.c | 9 +++++++++ 3 files changed, 16 insertions(+)
arch/x86/Kconfig +1 -0
@@ -882,6 +882,7 @@ config INTEL_TDX_GUEST select PARAVIRT_XL select X86_X2APIC select SECURITY_LOCKDOWN_LSM + select X86_MEM_ENCRYPT_COMMON help Provide support for running in a trusted domain on Intel processors equipped with Trusted Domain eXtenstions. TDX is an new Intel
arch/x86/include/asm/tdx.h +6 -0
@@ -111,6 +111,8 @@ unsigned char tdg_inb(unsigned short port);unsigned short tdg_inw (unsigned short port);unsigned int tdg_inl (unsigned short port); +extern phys_addr_t tdg_shared_mask (void ); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )
@@ -149,6 +151,10 @@ static inline long tdx_kvm_hypercall4(unsigned int nr, unsigned long p1,return -ENODEV ; } +static inline phys_addr_t tdg_shared_mask (void ) +{ +return 0 ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */
arch/x86/kernel/tdx.c +9 -0
@@ -76,6 +76,12 @@ bool is_tdx_guest(void) }EXPORT_SYMBOL_GPL (is_tdx_guest); +/* The highest bit of a guest physical address is the "sharing" bit */ +phys_addr_t tdg_shared_mask (void ) +{ +return 1ULL << (td_info.gpa_width -1 ); +} +static void tdg_get_info (void ) {u64 ret;
@@ -87,6 +93,9 @@ static void tdg_get_info(void) td_info.gpa_width = out.rcx &GENMASK (5 ,0 ); td_info.attributes = out.rdx; + +/* Exclude Shared bit from the __PHYSICAL_MASK */ + physical_mask &= ~tdg_shared_mask (); } static __cpuidle void tdg_halt(void)
Signature · 1 line
2.25.1
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Hi Dave,
Quoted 11 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov"<[email protected]>
tdx_shared_mask()returns the mask that has to be set in a page table entry to make page shared with VMM.Also, note that we cannot club shared mapping configuration between AMD SME and Intel TDX Guest platforms in common function. SME has to do it very early in
__startup_64()as it sets the bit on all memory, except what is used for communication. TDX can postpone as we don't need any shared mapping in very early boot.
Quoted 3 lines
Signed-off-by Kirill A. Shutemov<[email protected]>Reviewed-by Andi Kleen<[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan<[email protected]>
Any comments on this patch?
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
In reply to Kuppuswamy SathyanarayananQuoted 4 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
tdx_shared_mask()returns the mask that has to be set in a page table entry to make page shared with VMM.
Here's a rewrite:
Just like MKTME, TDX reassigns bits of the physical address for metadata. MKTME used several bits for an encryption KeyID. TDX uses a single bit in guests to communicate whether a physical page should be protected by TDX as private memory (bit set to 0) or unprotected and shared with the VMM (bit set to 1).
Add a helper, tdg_shared_mask() (bad name please fix it) to generate the mask. The processor enumerates its physical address width to include the shared bit, which means it gets included in __PHYSICAL_MASK by default.
Remove the shared mask from 'physical_mask' since any bits in tdg_shared_mask() are not used for physical addresses in page table entries.
--
BTW, do you find it confusing that the subject says: '__PHYSICAL_MASK' and yet the code only modifies 'physical_mask'?
Quoted 56 lines
Also, note that we cannot club shared mapping configuration between AMD SME and Intel TDX Guest platforms in common function. SME has to do it very early in
__startup_64()as it sets the bit on all memory, except what is used for communication. TDX can postpone as we don't need any shared mapping in very early boot.Signed-off-by Kirill A. Shutemov <[email protected]>Reviewed-by Andi Kleen <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>--- arch/x86/Kconfig | 1 + arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/kernel/tdx.c | 9 +++++++++ 3 files changed, 16 insertions(+)arch/x86/Kconfig +1
-0@@ -882,6 +882,7 @@ config INTEL_TDX_GUEST select PARAVIRT_XL select X86_X2APIC select SECURITY_LOCKDOWN_LSM + select X86_MEM_ENCRYPT_COMMON help Provide support for running in a trusted domain on Intel processors equipped with Trusted Domain eXtenstions. TDX is an new Intelarch/x86/include/asm/tdx.h +6
-0@@ -111,6 +111,8 @@ unsigned char tdg_inb(unsigned short port);unsigned short tdg_inw (unsigned short port);unsigned int tdg_inl (unsigned short port); +extern phys_addr_t tdg_shared_mask (void ); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )@@ -149,6 +151,10 @@ static inline long tdx_kvm_hypercall4(unsigned int nr, unsigned long p1,return -ENODEV ; } +static inline phys_addr_t tdg_shared_mask (void ) +{ +return 0 ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */ arch/x86/kernel/tdx.c +0
-0
arch/x86/kernel/tdx.c bool is_tdx_guest(void) · line 83
Quoted 8 lines
arch/x86/kernel/tdx.c +5
-0@@ -76,6 +76,12 @@ bool is_tdx_guest(void) }EXPORT_SYMBOL_GPL (is_tdx_guest); +/* The highest bit of a guest physical address is the "sharing" bit */ +phys_addr_t tdg_shared_mask (void ) +{ +return 1ULL << (td_info.gpa_width -1 ); +}
Why not just inline this thing? Functions don't get any smaller than that. Or does it not get used anywhere else? Or are you concerned about exporting td_info?
Quoted 14 lines
arch/x86/kernel/tdx.c +3
-0static void tdg_get_info (void ) {u64 ret;@@ -87,6 +93,9 @@ static void tdg_get_info(void) td_info.gpa_width = out.rcx &GENMASK (5 ,0 ); td_info.attributes = out.rdx; + +/* Exclude Shared bit from the __PHYSICAL_MASK */ + physical_mask &= ~tdg_shared_mask (); } static __cpuidle void tdg_halt(void)
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Hi Dave,
Quoted 8 lines · Dave Hansen
Quoted 4 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
tdx_shared_mask()returns the mask that has to be set in a page table entry to make page shared with VMM.Here's a rewrite:
Quoted 7 lines
Just like MKTME, TDX reassigns bits of the physical address for metadata. MKTME used several bits for an encryption KeyID. TDX uses a single bit in guests to communicate whether a physical page should be protected by TDX as private memory (bit set to 0) or unprotected and shared with the VMM (bit set to 1).
Add a helper,
tdg_shared_mask()(bad name please fix it) to generate the
Initially we have used tdx_* prefix for the guest code. But when the code from host side got merged together, we came across many name conflicts. So to avoid such issues in future, we were asked not to use the "tdx_" prefix and our alternative choice was "tdg_".
Also, IMO, "tdg" prefix is more meaningful for guest code (Trusted Domain Guest) compared to "tdx" (Trusted Domain eXtensions). I know that it gets confusing when grepping for TDX related changes. But since these functions are only used inside arch/x86 it should not be too confusing.
Even if rename is requested, IMO, it is easier to do it in one patch over making changes in all the patches. So if it is required, we can do it later once these initial patches were merged.
Quoted 7 lines
mask. The processor enumerates its physical address width to include the shared bit, which means it gets included in
__PHYSICAL_MASKby default.Remove the shared mask from '
physical_mask' since any bits intdg_shared_mask()are not used for physical addresses in page table entries.--
Thanks. I will include it in next version.
Quoted 63 lines
BTW, do you find it confusing that the subject says: '
__PHYSICAL_MASK' and yet the code only modifies 'physical_mask'?Quoted 59 lines
Also, note that we cannot club shared mapping configuration between AMD SME and Intel TDX Guest platforms in common function. SME has to do it very early in
__startup_64()as it sets the bit on all memory, except what is used for communication. TDX can postpone as we don't need any shared mapping in very early boot.Signed-off-by Kirill A. Shutemov <[email protected]>Reviewed-by Andi Kleen <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>--- arch/x86/Kconfig | 1 + arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/kernel/tdx.c | 9 +++++++++ 3 files changed, 16 insertions(+)arch/x86/Kconfig +1
-0@@ -882,6 +882,7 @@ config INTEL_TDX_GUEST select PARAVIRT_XL select X86_X2APIC select SECURITY_LOCKDOWN_LSM + select X86_MEM_ENCRYPT_COMMON help Provide support for running in a trusted domain on Intel processors equipped with Trusted Domain eXtenstions. TDX is an new Intelarch/x86/include/asm/tdx.h +6
-0@@ -111,6 +111,8 @@ unsigned char tdg_inb(unsigned short port);unsigned short tdg_inw (unsigned short port);unsigned int tdg_inl (unsigned short port); +extern phys_addr_t tdg_shared_mask (void ); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )@@ -149,6 +151,10 @@ static inline long tdx_kvm_hypercall4(unsigned int nr, unsigned long p1,return -ENODEV ; } +static inline phys_addr_t tdg_shared_mask (void ) +{ +return 0 ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */ arch/x86/kernel/tdx.c +0
-0@@ -76,6 +76,12 @@ bool is_tdx_guest(void) }EXPORT_SYMBOL_GPL (is_tdx_guest);
Quoted 9 lines
arch/x86/kernel/tdx.c bool is_tdx_guest(void) · line 83
Quoted 5 lines
arch/x86/kernel/tdx.c +5
-0+/* The highest bit of a guest physical address is the "sharing" bit */ +phys_addr_t tdg_shared_mask (void ) +{ +return 1ULL << (td_info.gpa_width -1 ); +}Why not just inline this thing? Functions don't get any smaller than that. Or does it not get used anywhere else? Or are you concerned about exporting
td_info?
We don't want to export td_info. It has more information additional to shared mask details. Any reason for suggesting to use inline?
This function is only used in following files.
arch/x86/include/asm/pgtable.h:25:#define pgprot_tdg_shared(prot) __pgprot(pgprot_val(prot) | tdg_shared_mask())arch/x86/mm/pat/set_memory.c:1997: mem_plain_bits = __pgprot(tdg_shared_mask());
arch/x86/kernel/tdx.c:134:phys_addr_t tdg_shared_mask(void)arch/x86/kernel/tdx.c:274: physical_mask &= ~tdg_shared_mask();
Quoted 16 lines
Quoted 15 lines
arch/x86/kernel/tdx.c +3
-0static void tdg_get_info (void ) {u64 ret;@@ -87,6 +93,9 @@ static void tdg_get_info(void) td_info.gpa_width = out.rcx &GENMASK (5 ,0 ); td_info.attributes = out.rdx; + +/* Exclude Shared bit from the __PHYSICAL_MASK */ + physical_mask &= ~tdg_shared_mask (); } static __cpuidle void tdg_halt(void)
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 2 lines · Sathyanarayanan
BTW, do you find it confusing that the subject says: '
__PHYSICAL_MASK' and yet the code only modifies 'physical_mask'?
"physical_mask" is defined as __PHYSICAL_MASK in page_types.h. MM code seems to use __PHYSICAL_MASK for common usage. But for our use case, if it makes it more readable, I am fine with using "physical_mask".
arch/x86/include/asm/page_types.h:57:#define __PHYSICAL_MASK physical_maskarch/x86/mm/pat/memtype.c:560: return address & __PHYSICAL_MASK;
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Sean Christopherson
maintainerRe: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
In reply to Kuppuswamy, SathyanarayananQuoted 17 lines · Sathyanarayanan
Hi Dave,
Quoted 14 lines · Dave Hansen
Quoted 4 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
tdx_shared_mask()returns the mask that has to be set in a page table entry to make page shared with VMM.Here's a rewrite:
Just like MKTME, TDX reassigns bits of the physical address for metadata. MKTME used several bits for an encryption KeyID. TDX uses a single bit in guests to communicate whether a physical page should be protected by TDX as private memory (bit set to 0) or unprotected and shared with the VMM (bit set to 1).
Quoted 4 lines
Quoted 1 line
Add a helper,
tdg_shared_mask()(bad name please fix it) to generate theInitially we have used tdx_* prefix for the guest code. But when the code from host side got merged together, we came across many name conflicts.
Whatever the conflicts are, they are by no means an unsolvable problem. I am more than happy to end up with slightly verbose names in KVM if that's what it takes to avoid "tdg".
Quoted 2 lines
So to avoid such issues in future, we were asked not to use the "
tdx_" prefix and our alternative choice was "tdg_".
Who asked you not to use tdx_? More specifically, did that feedback come from a maintainer (or anyone on-list), or was it an Intel-internal decision?
Quoted 8 lines
Also, IMO, "tdg" prefix is more meaningful for guest code (Trusted Domain Guest) compared to "tdx" (Trusted Domain eXtensions). I know that it gets confusing when grepping for TDX related changes. But since these functions are only used inside
arch/x86it should not be too confusing.Even if rename is requested, IMO, it is easier to do it in one patch over making changes in all the patches. So if it is required, we can do it later once these initial patches were merged.
Hell no, we are not merging known bad crud that requires useless churn to get things right.
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 10 lines · Sean Christopherson
Quoted 2 lines
Initially we have used tdx_* prefix for the guest code. But when the code from host side got merged together, we came across many name conflicts.
Whatever the conflicts are, they are by no means an unsolvable problem. I am more than happy to end up with slightly verbose names in KVM if that's what it takes to avoid "tdg".
Quoted 2 lines
So to avoid such issues in future, we were asked not to use the "
tdx_" prefix and our alternative choice was "tdg_".Who asked you not to use
tdx_? More specifically, did that feedback come from a maintainer (or anyone on-list), or was it an Intel-internal decision?
It is the Intel internal feedback.
Quoted 11 lines
Quoted 9 lines
Also, IMO, "tdg" prefix is more meaningful for guest code (Trusted Domain Guest) compared to "tdx" (Trusted Domain eXtensions). I know that it gets confusing when grepping for TDX related changes. But since these functions are only used inside
arch/x86it should not be too confusing.Even if rename is requested, IMO, it is easier to do it in one patch over making changes in all the patches. So if it is required, we can do it later once these initial patches were merged.
Hell no, we are not merging known bad crud that requires useless churn to get things right.
So what is your proposal? "tdx_guest_" / "tdx_host_" ?
If there is supposed be a rename, lets wait till we know about maintainers feedback as well. If possible I would prefer not to go through another rename.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Sean Christopherson
maintainerRe: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 1 line · Sathyanarayanan
So what is your proposal? "
tdx_guest_" / "tdx_host_" ?
1. Abstract things where appropriate, e.g. I'm guessing there is a clever way to deal with the shared vs. private inversion and avoid tdg_shared_mask altogether.
2. Steal what SEV-ES did for the #VC handlers and use ve_ as the prefix for handlers.
3. Use tdx_ everywhere else and handle the conflicts on a case-by-case basis with a healthy dose of common sense. E.g. there should be no need to worry about "static __cpuidle void tdg_safe_halt(void)" colliding because neither the guest nor KVM should be exposing tdx_safe_halt() outside of its compilation unit.
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 6 lines · Sean Christopherson
Quoted 1 line · Sathyanarayanan
So what is your proposal? "
tdx_guest_" / "tdx_host_" ?1. Abstract things where appropriate, e.g. I'm guessing there is a clever way to deal with the shared vs. private inversion and avoid
tdg_shared_maskaltogether.
Quoted 8 lines
2. Steal what SEV-ES did for the #VC handlers and use
ve_as the prefix for handlers.3. Use
tdx_everywhere else and handle the conflicts on a case-by-case basis with a healthy dose of common sense. E.g. there should be no need to worry about "static__cpuidlevoidtdg_safe_halt(void)" colliding because neither the guest nor KVM should be exposingtdx_safe_halt()outside of its compilation unit.
Sorry Sean, but your suggestion is against all good code hygiene practices. Normally we try to pick unique prefixes for every module, and trying to coordinate with lots of other code that is maintained by other people is just a long term recipe for annoying merging problems. Same with coordinating with SEV-ES for ve_.
Is it really that hard to adjust your grep patterns?
I'm not against changing tdg_, but if it's changed it should be something unique, and also not too long. Today tdg_ fits that criteria nicely.
-Andi
Sean Christopherson
maintainerRe: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 18 lines · Andi Kleen
Quoted 6 lines · Sean Christopherson
Quoted 1 line · Sathyanarayanan
So what is your proposal? "
tdx_guest_" / "tdx_host_" ?1. Abstract things where appropriate, e.g. I'm guessing there is a clever way to deal with the shared vs. private inversion and avoid
tdg_shared_maskaltogether.Quoted 8 lines
2. Steal what SEV-ES did for the #VC handlers and use
ve_as the prefix for handlers.3. Use
tdx_everywhere else and handle the conflicts on a case-by-case basis with a healthy dose of common sense. E.g. there should be no need to worry about "static__cpuidlevoidtdg_safe_halt(void)" colliding because neither the guest nor KVM should be exposingtdx_safe_halt()outside of its compilation unit.
Quoted 5 lines
Sorry Sean, but your suggestion is against all good code hygiene practices. Normally we try to pick unique prefixes for every module, and trying to coordinate with lots of other code that is maintained by other people is just a long term recipe for annoying merging problems. Same with coordinating with SEV-ES for
ve_.
For ve_? SEV-ES uses vc_...
I'd buy that argument if series as a whole was consistent, but there are individual function prototypes that aren't consistent, e.g.
+static int __tdg_map_gpa(phys_addr_t gpa, int numpages,
+ enum tdx_map_type map_type)
a number of functions that use tdx_ isntead of tdg_ (I'll give y'all a break on is_tdx_guest()), the files are all tdx.{c,h}, the shortlogs all use x86/tdx, the comments all use TDX, and so on and so forth.
I understand the desire to have a unique prefix, but tdg is is _too_ close to tdx. I don't want to spend the next N years wondering if tdg is a typo or intended.
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 2 lines · Sean Christopherson
I understand the desire to have a unique prefix, but tdg is is _too_ close to tdx. I don't want to spend the next N years wondering if tdg is a typo or intended.
Sathya has even mis-typed "tdx" instead of "tdg" this in his own changelogs up to this point. That massively weakens the argument that "tdg" is a good idea.
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 3 lines · Dave Hansen
Sathya has even mis-typed "tdx" instead of "tdg" this in his own changelogs up to this point. That massively weakens the argument that "tdg" is a good idea.
It is not a typo. But when we did the initial rename from "tdx_" -> "tdg_", somehow I missed the change log change. That's why I am bit reluctant to go for another rename (since we have scan change log, comments and code) in all the patches.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 11 lines · Sathyanarayanan
Quoted 3 lines · Dave Hansen
Sathya has even mis-typed "tdx" instead of "tdg" this in his own changelogs up to this point. That massively weakens the argument that "tdg" is a good idea.
It is not a typo. But when we did the initial rename from "
tdx_" -> "tdg_", somehow I missed the change log change. That's why I am bit reluctant to go for another rename (since we have scan change log, comments and code) in all the patches.
Yes I agree. If there's another rename it should be after a full review by all the maintainers. If there is still consensus that a rename is needed then it can be done then.
And we'll just hope that Sean's brain will get used to tdg_ by then.
-Andi
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
In reply to Sean ChristophersonQuoted 5 lines · Sean Christopherson
Quoted 1 line · Sathyanarayanan
So what is your proposal? "
tdx_guest_" / "tdx_host_" ?1. Abstract things where appropriate, e.g. I'm guessing there is a clever way to deal with the shared vs. private inversion and avoid
tdg_shared_maskaltogether.
One example here would be to keep a structure like:
struct protected_mem_config
{
unsigned long p_set_bits;
unsigned long p_clear_bits;
}Where 'p_set_bits' are the bits that need to be set to establish memory protection and 'p_clear_bits' are the bits that need to be cleared. physical_mask would clear both of them:
physical_mask &= ~(pmc.p_set_bits & pmc.p_set_bits);
Then, in a place like __set_memory_enc_dec(), you would query whether memory protection was in place or not:
+ if (protect) {
+ cpa.mask_set = pmc.p_set_bits;
+ cpa.mask_clr = pmc.p_clear_bits;
+ map_type = TDX_MAP_PRIVATE;
+ } else {
+ cpa.mask_set = pmc.p_clear_bits;
+ cpa.mask_clr = pmc.p_set_bits;
+ map_type = TDX_MAP_SHARED;
+ }
The is_tdx_guest() if()'s would just go away.
Basically, if there's a is_tdx_guest() check in common code, it's a place that might need an abstraction.
This, for instance:
Review
Quoted 3 lines
(quoted diff) +3
-0+ if (!ret && is_tdx_guest()) { + ret = tdg_map_gpa(__pa(addr), numpages, map_type); + }
could probably just be:
if (!ret && is_protected_guest()) {
ret = x86_vmm_protect(__pa(addr), numpages, protected);
}Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
Quoted 15 lines · Dave Hansen
Quoted 5 lines · Sean Christopherson
Quoted 1 line · Sathyanarayanan
So what is your proposal? "
tdx_guest_" / "tdx_host_" ?1. Abstract things where appropriate, e.g. I'm guessing there is a clever way to deal with the shared vs. private inversion and avoid
tdg_shared_maskaltogether.One example here would be to keep a structure like:
struct
protected_mem_config{ unsigned long p_set_bits; unsigned long p_clear_bits; }
Quoted 5 lines
Where '
p_set_bits' are the bits that need to be set to establish memory protection and 'p_clear_bits' are the bits that need to be cleared.physical_maskwould clear both of them:physical_mask &= ~(pmc.p_set_bits & pmc.p_set_bits);
For me it looks like an abstraction for sake of abstraction. More levels of indirection without clear benefit. It doesn't add any more readability:
would you know what 'p_set_bits' stands for in two month? I'm not sure.
I would rather leave explicit check for protection flavour. It provides better context for a reader.
Signature · 1 line
Kirill A. Shutemov
RFC 1/1 x86/tdx: Exclude Shared bit from physical_mask
From: "Kirill A. Shutemov" <[email protected]>
Just like MKTME, TDX reassigns bits of the physical address for metadata. MKTME used several bits for an encryption KeyID. TDX uses a single bit in guests to communicate whether a physical page should be protected by TDX as private memory (bit set to 0) or unprotected and shared with the VMM (bit set to 1).
Add a helper, tdg_shared_mask() to generate the mask. The processor enumerates its physical address width to include the shared bit, which means it gets included in __PHYSICAL_MASK by default.
Remove the shared mask from 'physical_mask' since any bits in tdg_shared_mask() are not used for physical addresses in page table entries.
Also, note that we cannot club shared mapping configuration between AMD SME and Intel TDX Guest platforms in common function. SME has to do it very early in __startup_64() as it sets the bit on all memory, except what is used for communication. TDX can postpone it, as it don't need any shared mapping in very early boot.
---
Changes since RFC-v2:
* Renamed __PHYSICAL_MASK to physical_mask in commit subject.
* Fixed commit log as per review comments.
arch/x86/Kconfig | 1 + arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/kernel/tdx.c | 9 +++++++++ 3 files changed, 16 insertions(+)
arch/x86/Kconfig +1 -0
@@ -879,6 +879,7 @@ config INTEL_TDX_GUEST select X86_X2APIC select SECURITY_LOCKDOWN_LSM select ARCH_HAS_PROTECTED_GUEST + select X86_MEM_ENCRYPT_COMMON help Provide support for running in a trusted domain on Intel processors equipped with Trusted Domain eXtenstions. TDX is a new Intel
arch/x86/include/asm/tdx.h +6 -0
@@ -118,6 +118,8 @@ do { \ } while (0)#endif +extern phys_addr_t tdg_shared_mask (void ); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )
@@ -137,6 +139,10 @@ static inline bool tdg_early_handle_ve(struct pt_regs *regs) return false; } +static inline phys_addr_t tdg_shared_mask (void ) +{ +return 0 ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #ifdef CONFIG_INTEL_TDX_GUEST_KVM
arch/x86/kernel/tdx.c +9 -0
@@ -101,6 +101,12 @@ bool tdx_protected_guest_has(unsigned long flag) }EXPORT_SYMBOL_GPL (tdx_protected_guest_has); +/* The highest bit of a guest physical address is the "sharing" bit */ +phys_addr_t tdg_shared_mask (void ) +{ +return 1ULL << (td_info.gpa_width -1 ); +} +static void tdg_get_info (void ) {u64 ret;
@@ -112,6 +118,9 @@ static void tdg_get_info(void) td_info.gpa_width = out.rcx &GENMASK (5 ,0 ); td_info.attributes = out.rdx; + +/* Exclude Shared bit from the __PHYSICAL_MASK */ + physical_mask &= ~tdg_shared_mask (); } static __cpuidle void tdg_halt(void)
Signature · 1 line
2.25.1
Re: [RFC v2 27/32] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
In reply to Kuppuswamy, SathyanarayananQuoted 11 lines · Sathyanarayanan
Quoted 10 lines
arch/x86/kernel/tdx.c bool is_tdx_guest(void) · line 83
Quoted 6 lines
+/* The highest bit of a guest physical address is the "sharing" bit */
arch/x86/kernel/tdx.c +4
-0+phys_addr_t tdg_shared_mask (void ) +{ + return 1ULL << (td_info.gpa_width - 1); +}Why not just inline this thing? Functions don't get any smaller than that. Or does it not get used anywhere else? Or are you concerned about exporting
td_info?
Quoted 2 lines
We don't want to export
td_info. It has more information additional to shared mask details. Any reason for suggesting to use inline?
My favorite reason is that it eliminates the need for three declarations:
1. An extern for the header
2. A stub for the header
3. The real function in the .c file.
An inline removes two places that might get out of sync in some way and eliminates the need to check two implementation sites when grepping.
Not in this case, but in general, inlines also result in faster, more compact code since the compiler has more visibility into what the function does at its call sites.
Not wanting to export td_info _is_ a reasonable argument, though.
RFC v2 28/32 x86/tdx: Make pages shared in ioremap()
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
All ioremap()ed pages that are not backed by normal memory (NONE or RESERVED) have to be mapped as shared.
Reuse the infrastructure we have for AMD SEV.
Note that DMA code doesn't use ioremap() to convert memory to shared as DMA buffers backed by normal memory. DMA code make buffer shared with set_memory_decrypted().
--- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/mm/ioremap.c | 8 +++++--- 2 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/include/asm/pgtable.h +3 -0
@@ -21,6 +21,9 @@#define pgprot_encrypted (prot) __pgprot(__sme_set(pgprot_val(prot)))#define pgprot_decrypted (prot) __pgprot(__sme_clr(pgprot_val(prot))) +/* Make the page accesable by VMM */ +#define pgprot_tdg_shared (prot) __pgprot(pgprot_val(prot) | tdg_shared_mask()) +#ifndef __ASSEMBLY__#include <asm/x86_init.h> #include <asm/fpu/xstate.h>
arch/x86/mm/ioremap.c +5 -3
@@ -87,12 +87,12 @@ static unsigned int __ioremap_check_ram(struct resource *res) }/* -* In a SEV guest, NONE and RESERVED should not be mapped encrypted because -* there the whole memory is already encrypted. +* In a SEV or TDX guest, NONE and RESERVED should not be mapped encrypted (or +* private in TDX case) because there the whole memory is already encrypted. */ static unsigned int __ioremap_check_encrypted (struct resource *res) { -if (!sev_active ()) +if (!sev_active () && !is_tdx_guest ())return 0 ;switch (res->desc) {
@@ -244,6 +244,8 @@ __ioremap_caller(resource_size_t phys_addr, unsigned long size, prot =PAGE_KERNEL_IO ;if ((io_desc.flags &IORES_MAP_ENCRYPTED ) || encrypted) prot =pgprot_encrypted (prot); +else if (is_tdx_guest ()) + prot =pgprot_tdg_shared (prot);switch (pcm) {case _PAGE_CACHE_MODE_UC:
Signature · 1 line
2.25.1
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
arch/x86/mm/ioremap.c static unsigned int __ioremap_check_ram(struct resource *res) · line 96
Quoted 5 lines · Kuppuswamy Sathyanarayanan
static unsigned int
__ioremap_check_encrypted(struct resource *res){arch/x86/mm/ioremap.c +1
-1-if (!sev_active ()) +if (!sev_active () && !is_tdx_guest ())return 0 ;
I think it's time to come up with a real name for all of the code that's
under: (sev_active() || is_tdx_guest()).
"encrypted" isn't it, for sure.
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 10 lines · Dave Hansen
arch/x86/mm/ioremap.c static unsigned int __ioremap_check_ram(struct resource *res) · line 96
Quoted 5 lines · Kuppuswamy Sathyanarayanan
static unsigned int __ioremap_check_encrypted(struct resource *res) {arch/x86/mm/ioremap.c +1
-1-if (!sev_active ()) +if (!sev_active () && !is_tdx_guest ())return 0 ;I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest())."encrypted" isn't it, for sure.
I called it protected_guest() in some other patches.
-Andi
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Hi Dave,
Quoted 8 lines · Andi Kleen
Quoted 6 lines · Dave Hansen
Quoted 5 lines · Kuppuswamy Sathyanarayanan
static unsigned int
__ioremap_check_encrypted(struct resource *res){
- if (!sev_active())
+ if (!sev_active() && !is_tdx_guest())
return 0;
Quoted 6 lines
Quoted 4 lines
I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest())."encrypted" isn't it, for sure.
I called it
protected_guest()in some other patches.
If you are also fine with above mentioned function name, I can include it in this series. Since we have many use cases of above condition, it will be useful define it as helper function.
Quoted 3 lines
-Andi
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 7 lines · Sathyanarayanan
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
- if (!sev_active())
+ if (!sev_active() && !is_tdx_guest())
return 0;I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest()).
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
"encrypted" isn't it, for sure.
I called it
protected_guest()in some other patches.If you are also fine with above mentioned function name, I can include it in this series. Since we have many use cases of above condition, it will be useful define it as helper function.
FWIW, I think sev_active() has a horrible name. Shouldn't that be "is_sev_guest()"? "sev_active()" could be read as "I'm a SEV host" or "I'm a SEV guest" and "SEV is active".
protected_guest() seems fine to cover both, despite the horrid SEV naming. It'll actually be nice to banish it from appearing in many of its uses. :)
Sean Christopherson
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
+Boris, who has similar opinions on sev_active().
Quoted 12 lines · Dave Hansen
Quoted 11 lines · Sathyanarayanan
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
- if (!sev_active())
+ if (!sev_active() && !is_tdx_guest())
return 0;I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest())."encrypted" isn't it, for sure.
I called it
protected_guest()in some other patches.
Quoted 7 lines
Quoted 3 lines
If you are also fine with above mentioned function name, I can include it in this series. Since we have many use cases of above condition, it will be useful define it as helper function.
FWIW, I think
sev_active()has a horrible name. Shouldn't that be "is_sev_guest()"? "sev_active()" could be read as "I'm a SEV host" or "I'm a SEV guest" and "SEV is active".
I can't find the thread offhand, but Boris proposed something along the lines of cpu_has(), but specific to a given flavor of protected guest. IIRC, it was sev_guest_has(SEV_ES) or something like that.
I 100% agree that we should have actual feature bits somewhere for the various protected guest flavors.
Quoted 3 lines
protected_guest()seems fine to cover both, despite the horrid SEV naming. It'll actually be nice to banish it from appearing in many of its uses. :)
Borislav Petkov
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 6 lines · Sean Christopherson
I can't find the thread offhand, but Boris proposed something along the lines of
cpu_has(), but specific to a given flavor of protected guest. IIRC, it wassev_guest_has(SEV_ES)or something like that.I 100% agree that we should have actual feature bits somewhere for the various protected guest flavors.
Preach brother! :)
/me goes and greps mailboxes...
ah, do you mean this, per chance:
https://lore.kernel.org/kvm/[email protected]/
?
And yes, this has "sev" in the name and dhansen makes sense to me in wishing to unify all the protected guest feature queries under a common name. And then depending on the vendor, that common name will call the respective vendor's helper to answer the protected guest aspect asked about.
This way, generic code will call
protected_guest_has()
or so and be nicely abstracted away from the underlying implementation.
Hohumm, yap, sounds nice to me.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 18 lines · Borislav Petkov
Preach brother!:)
/me goes and greps mailboxes...
ah, do you mean this, per chance:
https://lore.kernel.org/kvm/[email protected]/
?
And yes, this has "sev" in the name and dhansen makes sense to me in wishing to unify all the protected guest feature queries under a common name. And then depending on the vendor, that common name will call the respective vendor's helper to answer the protected guest aspect asked about.
This way, generic code will call
Quoted 7 lines
protected_guest_has()or so and be nicely abstracted away from the underlying implementation.
Hohumm, yap, sounds nice to me.
Thx.
I see many variants of SEV/SME related checks in the common code path between TDX and SEV/SME. Can a generic call like protected_guest_has(MEMORY_ENCRYPTION) or is_protected_guest() replace all these variants?
We will not be able to test AMD related features. So I need to confirm it with AMD code maintainers/developers before making this change.
arch/x86/include/asm/io.h:313: if (sev_key_active() || is_tdx_guest()) { \
arch/x86/include/asm/io.h:329: if (sev_key_active() || is_tdx_guest()) { \
arch/x86/kernel/pci-swiotlb.c:52: if (sme_active() || is_tdx_guest())arch/x86/mm/ioremap.c:96: if (!sev_active() && !is_tdx_guest())arch/x86/mm/pat/set_memory.c:1984: if (!mem_encrypt_active() && !is_tdx_guest())
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Borislav Petkov
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 4 lines · Sathyanarayanan
I see many variants of SEV/SME related checks in the common code path between TDX and SEV/SME. Can a generic call like
protected_guest_has(MEMORY_ENCRYPTION)oris_protected_guest()replace all these variants?
It depends...
Quoted 2 lines
We will not be able to test AMD related features. So I need to confirm it with AMD code maintainers/developers before making this change.
Lemme add two to Cc.
So looking at those examples, you guys are making it not very suspenceful for TDX - it is the same function in all. :)
Quoted 2 lines
arch/x86/include/asm/io.h:313: if (sev_key_active() || is_tdx_guest()) { \
arch/x86/include/asm/io.h:329: if (sev_key_active() || is_tdx_guest()) { \So I think the static key on the AMD side is not really needed and it could be replaced with
sev_active() && !sev_es_active()
i.e. SEV but but not SEV-ES. A vendor-agnostic function would do here probably something like:
protected_guest_has(ENC_UNROLL_STRING_IO)
and inside it, it would do:
if (AMD) amd_protected_guest_has(...) else if (Intel) intel_protected_guest_has(...) else WARN()
and both vendors would each implement that function with the respective low-level query functions.
Quoted 1 line
arch/x86/kernel/pci-swiotlb.c:52: if (sme_active()||is_tdx_guest())
That can be probably
protected_guest_has(ENC_HOST_MEM_ENCRYPT);
as on AMD that means SME but not SEV. I guess on Intel you guys want to do bounce buffers in the guest? or so...
Quoted 1 line
arch/x86/mm/ioremap.c:96: if (!sev_active() && !is_tdx_guest())
So that function should simply be replaced with:
if (!(desc->flags & IORES_MAP_ENCRYPTED)) {
/* ... comment bla explaining what this is... */
if ((sev_active() || is_tdx_guest()) &&
(res->desc != IORES_DESC_NONE &&
res->desc != IORES_DESC_RESERVED))
desc->flags |= IORES_MAP_ENCRYPTED;
}as to the first check I guess:
protected_guest_has(ENC_GUEST_ENABLED)
or so to mean, kernel is running as an encrypted guest...
Quoted 1 line
arch/x86/mm/pat/set_memory.c:1984: if (!mem_encrypt_active() && !is_tdx_guest())
That should probably be
protected_guest_has(ENC_ACTIVE);
to denote the generic "I'm running some sort of memory encryption..."
Yeah, this is all rough and should show the main idea - to have a vendor-agnostic accessor in such common code paths and then abstract away the differences in cpu/amd.c and cpu/intel.c, respectively and thus keep the code sane.
How does that sound?
ENC_ being an ENCryption prefix, ofc.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 41 lines · Borislav Petkov
Quoted 4 lines · Sathyanarayanan
I see many variants of SEV/SME related checks in the common code path between TDX and SEV/SME. Can a generic call like
protected_guest_has(MEMORY_ENCRYPTION)oris_protected_guest()replace all these variants?It depends...
Quoted 2 lines
We will not be able to test AMD related features. So I need to confirm it with AMD code maintainers/developers before making this change.
Lemme add two to Cc.
So looking at those examples, you guys are making it not very suspenceful for TDX - it is the same function in all. :)
Quoted 2 lines
arch/x86/include/asm/io.h:313: if (sev_key_active() || is_tdx_guest()) { \ arch/x86/include/asm/io.h:329: if (sev_key_active() || is_tdx_guest()) { \So I think the static key on the AMD side is not really needed and it could be replaced with
sev_active() && !sev_es_active()i.e. SEV but but not SEV-ES. A vendor-agnostic function would do here probably something like:
protected_guest_has(ENC_UNROLL_STRING_IO)and inside it, it would do:
if (AMD) amd_protected_guest_has(...) else if (Intel) intel_protected_guest_has(...) else WARN()and both vendors would each implement that function with the respective low-level query functions.
Quoted 8 lines
Quoted 1 line
arch/x86/kernel/pci-swiotlb.c:52: if (sme_active()||is_tdx_guest())That can be probably
protected_guest_has(ENC_HOST_MEM_ENCRYPT);as on AMD that means SME but not SEV. I guess on Intel you guys want to do bounce buffers in the guest? or so...
In arch/x86/mm/mem_encrypt.c, sme_early_init() (should have renamed that when SEV support was added), we do:
if (sev_active()) swiotlb_force = SWIOTLB_FORCE;
TDX should be able to do a similar thing without having to touch arch/x86/kernel/pci-swiotlb.c.
That would remove any confusion over SME being part of a protected_guest_has() call.
Quoted 5 lines
Quoted 2 lines
arch/x86/mm/ioremap.c:96: if (!sev_active() && !is_tdx_guest())So that function should simply be replaced with:
Quoted 7 lines
if (!(desc->flags & IORES_MAP_ENCRYPTED)) {
/* ... comment bla explaining what this is... */
if ((sev_active() || is_tdx_guest()) &&
(res->desc != IORES_DESC_NONE &&
res->desc != IORES_DESC_RESERVED))
desc->flags |= IORES_MAP_ENCRYPTED;
}I kinda like the separate function, though.
Quoted 7 lines
as to the first check I guess:
protected_guest_has(ENC_GUEST_ENABLED)or so to mean, kernel is running as an encrypted guest...
Quoted 7 lines
Quoted 1 line
arch/x86/mm/pat/set_memory.c:1984: if (!mem_encrypt_active() && !is_tdx_guest())That should probably be
protected_guest_has(ENC_ACTIVE);to denote the generic "I'm running some sort of memory encryption..."
Except mem_encrypt_active() covers both SME and SEV, so protected_guest_has() would be confusing.
Thanks,
Tom
Quoted 10 lines
Yeah, this is all rough and should show the main idea - to have a vendor-agnostic accessor in such common code paths and then abstract away the differences in
cpu/amd.candcpu/intel.c, respectively and thus keep the code sane.How does that sound?
ENC_being an ENCryption prefix, ofc.
Borislav Petkov
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 10 lines · Tom Lendacky
In
arch/x86/mm/mem_encrypt.c,sme_early_init()(should have renamed that when SEV support was added), we do:if (sev_active()) swiotlb_force = SWIOTLB_FORCE;TDX should be able to do a similar thing without having to touch
arch/x86/kernel/pci-swiotlb.c.That would remove any confusion over SME being part of a
protected_guest_has()call.
Even better.
Quoted 1 line
I kinda like the separate function, though.
Only if you clean it up and get rid of the inverted logic and drop that silly switch-case.
Quoted 2 lines
Except
mem_encrypt_active()covers both SME and SEV, soprotected_guest_has()would be confusing.
I don't understand - the AMD-specific function amd_protected_guest_has() would return sme_me_mask just like mem_encrypt_active() does and we can get rid of latter.
Or do you have a problem with the name protected_guest_has() containing "guest" while we're talking about SME here?
If so, feel free to suggest a better one - the name does not have to have "guest" in it.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 22 lines · Borislav Petkov
Quoted 10 lines · Tom Lendacky
In
arch/x86/mm/mem_encrypt.c,sme_early_init()(should have renamed that when SEV support was added), we do:if (sev_active()) swiotlb_force = SWIOTLB_FORCE;TDX should be able to do a similar thing without having to touch
arch/x86/kernel/pci-swiotlb.c.That would remove any confusion over SME being part of a
protected_guest_has()call.Even better.
Quoted 1 line
I kinda like the separate function, though.
Only if you clean it up and get rid of the inverted logic and drop that silly switch-case.
Quoted 2 lines
Except
mem_encrypt_active()covers both SME and SEV, soprotected_guest_has()would be confusing.
Quoted 6 lines
I don't understand - the AMD-specific function
amd_protected_guest_has()would returnsme_me_maskjust likemem_encrypt_active()does and we can get rid of latter.Or do you have a problem with the name
protected_guest_has()containing "guest" while we're talking about SME here?
The latter.
Quoted 3 lines
If so, feel free to suggest a better one - the name does not have to have "guest" in it.
Let me see if I can come up with something that will make sense.
Thanks,
Tom
Quoted 4 lines
Thx.
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Hi,
Quoted 35 lines · Tom Lendacky
Quoted 22 lines · Borislav Petkov
Quoted 10 lines · Tom Lendacky
In
arch/x86/mm/mem_encrypt.c,sme_early_init()(should have renamed that when SEV support was added), we do:if (sev_active()) swiotlb_force = SWIOTLB_FORCE;TDX should be able to do a similar thing without having to touch
arch/x86/kernel/pci-swiotlb.c.That would remove any confusion over SME being part of a
protected_guest_has()call.Even better.
Quoted 1 line
I kinda like the separate function, though.
Only if you clean it up and get rid of the inverted logic and drop that silly switch-case.
Quoted 2 lines
Except
mem_encrypt_active()covers both SME and SEV, soprotected_guest_has()would be confusing.Quoted 6 lines
I don't understand - the AMD-specific function
amd_protected_guest_has()would returnsme_me_maskjust likemem_encrypt_active()does and we can get rid of latter.Or do you have a problem with the name
protected_guest_has()containing "guest" while we're talking about SME here?The latter.
Quoted 1 line
Quoted 12 lines
Quoted 2 lines
If so, feel free to suggest a better one - the name does not have to have "guest" in it.
Let me see if I can come up with something that will make sense.
Thanks,
TomQuoted 4 lines
Thx.
Following is the sample implementation. Please let me know your comments.
tdx: Introduce generic protected_guest abstraction
Add a generic way to check if we run with an encrypted guest, without requiring x86 specific ifdefs. This can then be used in non architecture specific code.
is_protected_guest() helper function can be implemented using arch specific CPU feature flags.
protected_guest_has() is used to check for protected guest feature flags.
arch/Kconfig +3 -0
@@ -956,6 +956,9 @@ config HAVE_ARCH_NVRAM_OPS config ISA_BUS_API def_bool ISA +config ARCH_HAS_PROTECTED_GUEST + bool + # # ABI hall of shame #
arch/x86/Kconfig +1 -0
@@ -879,6 +879,7 @@ config INTEL_TDX_GUEST select X86_X2APIC select SECURITY_LOCKDOWN_LSM select X86_MEM_ENCRYPT_COMMON + select ARCH_HAS_PROTECTED_GUEST help Provide support for running in a trusted domain on Intel processors equipped with Trusted Domain eXtenstions. TDX is a new Intel
arch/x86/include/asm/protected_guest.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2020 Intel Corporation */ +#ifndef _ASM_PROTECTED_GUEST +#define _ASM_PROTECTED_GUEST 1 + +#include <asm/cpufeature.h> +#include <asm/tdx.h> + +/* Only include through linux/protected_guest.h */ + +static inline bool is_protected_guest (void ) +{ +return boot_cpu_has (X86_FEATURE_TDX_GUEST ); +} + +static inline bool protected_guest_has (unsigned long flag) +{ +if (boot_cpu_has (X86_FEATURE_TDX_GUEST )) +return tdx_protected_guest_has (flag); + +return false; +} + +#endif
arch/x86/include/asm/tdx.h +6 -0
@@ -147,6 +147,7 @@ do { \extern phys_addr_t tdg_shared_mask (void );extern int tdx_hcall_gpa_intent (phys_addr_t gpa,int numpages,enum tdx_map_type map_type); +bool tdx_protected_guest_has (unsigned long flag);#else // !CONFIG_INTEL_TDX_GUEST
@@ -167,6 +168,11 @@ static inline int tdx_hcall_gpa_intent(phys_addr_t gpa, int numpages, {return -ENODEV ; } + +static inline bool tdx_protected_guest_has (unsigned long flag) +{ +return false; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #ifdef CONFIG_INTEL_TDX_GUEST_KVM
arch/x86/kernel/tdx.c +18 -0
@@ -11,6 +11,7 @@#include <linux/sched/signal.h> /* force_sig_fault() */ #include <linux/swiotlb.h> #include <linux/security.h> +#include <linux/protected_guest.h> #include <linux/cpu.h>
@@ -122,6 +123,23 @@ bool is_tdx_guest(void) }EXPORT_SYMBOL_GPL (is_tdx_guest); +bool tdx_protected_guest_has (unsigned long flag) +{ +if (!is_tdx_guest ()) +return false; + +switch (flag) { +case VM_MEM_ENCRYPT : +case VM_MEM_ENCRYPT_ACTIVE : +case VM_UNROLL_STRING_IO : +case VM_HOST_MEM_ENCRYPT : +return true; + } + +return false; +} +EXPORT_SYMBOL_GPL (tdx_protected_guest_has); +/* The highest bit of a guest physical address is the "sharing" bit */ phys_addr_t tdg_shared_mask (void ) {
include/linux/protected_guest.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _LINUX_PROTECTED_GUEST_H +#define _LINUX_PROTECTED_GUEST_H 1 + +/* Protected Guest Feature Flags (leave 0-0xff for arch specific flags) */ + +/* Support for guest encryption */ +#define VM_MEM_ENCRYPT 0x100 +/* Encryption support is active */ +#define VM_MEM_ENCRYPT_ACTIVE 0x101 +/* Support for unrolled string IO */ +#define VM_UNROLL_STRING_IO 0x102 +/* Support for host memory encryption */ +#define VM_HOST_MEM_ENCRYPT 0x103 + +#ifdef CONFIG_ARCH_HAS_PROTECTED_GUEST +#include <asm/protected_guest.h> +#else +static inline bool is_protected_guest (void ) {return false; } +static inline bool protected_guest_has (unsigned long flag) {return false; } +#endif + +#endif
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Borislav Petkov
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 2 lines · Sathyanarayanan
Following is the sample implementation. Please let me know your comments.
Doesn't look like what I suggested here:
https://lkml.kernel.org/r/[email protected]
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 7 lines · Borislav Petkov
Quoted 2 lines · Sathyanarayanan
Following is the sample implementation. Please let me know your comments.
Doesn't look like what I suggested here:
IIUC, following are your design suggestions:
1. Define generic flags.
I think following flags are defined as you have suggested.
+++ b/include/linux/protected_guest.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LINUX_PROTECTED_GUEST_H
+#define _LINUX_PROTECTED_GUEST_H 1
+
+/* Protected Guest Feature Flags (leave 0-0xff for arch specific flags) */ +
+/* Support for guest encryption */
+#define VM_MEM_ENCRYPT 0x100
+/* Encryption support is active */
+#define VM_MEM_ENCRYPT_ACTIVE 0x101
+/* Support for unrolled string IO */
+#define VM_UNROLL_STRING_IO 0x102
+/* Support for host memory encryption */
+#define VM_HOST_MEM_ENCRYPT 0x103
2. Define generic functions and allow calls to arch specific implementations.
For above requirement, instead of calling arch specific functions from include/linux/protected_guest.h, I have directly included the arch specific file in linux/protected_guest.h
+#ifdef CONFIG_ARCH_HAS_PROTECTED_GUEST
+#include <asm/protected_guest.h>
+#else
+static inline bool is_protected_guest(void) { return false; } +static inline bool protected_guest_has(unsigned long flag) { return false; } +#endif
3. Implement arch specific implementations respond to protected_guest_has() calls right?
I think above requirement is satisfied in following implementation.
+++ b/arch/x86/include/asm/protected_guest.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2020 Intel Corporation */ +#ifndef _ASM_PROTECTED_GUEST
+#define _ASM_PROTECTED_GUEST 1
+
+#include <asm/cpufeature.h>
+#include <asm/tdx.h>
+
+/* Only include through linux/protected_guest.h */ +
+static inline bool is_protected_guest(void)
+{
+ return boot_cpu_has(X86_FEATURE_TDX_GUEST);
+}
+
+static inline bool protected_guest_has(unsigned long flag) +{
+ if (boot_cpu_has(X86_FEATURE_TDX_GUEST))
+ return tdx_protected_guest_has(flag);
+
+ return false;
+}
+
Did I misunderstand anything ? Please let me know your comments.
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Borislav Petkov
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 1 line · Sathyanarayanan
I think above requirement is satisfied in following implementation.
Well, I suggested a single protected_guest_has() function which does:
if (AMD)
amd_protected_guest_has(...)
else if (Intel)
intel_protected_guest_has(...)
else
WARN()where amd_protected_guest_has() is implemented in arch/x86/kernel/sev.c and intel_protected_guest_has() is implemented in, as far as I can follow your paths in the diff, in arch/x86/kernel/tdx.c.
No is_protected_guest() and no ARCH_HAS_PROTECTED_GUEST.
Just the above controlled by CONFIG_INTEL_TDX_GUEST or whatever the TDX config item is gonna end up being and on the AMD side by CONFIG_AMD_MEM_ENCRYPT.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 12 lines · Borislav Petkov
Quoted 1 line · Sathyanarayanan
I think above requirement is satisfied in following implementation.
Well, I suggested a single
protected_guest_has()function which does:if (AMD) amd_protected_guest_has(...) else if (Intel) intel_protected_guest_has(...) else WARN()
Quoted 5 lines
where
amd_protected_guest_has()is implemented inarch/x86/kernel/sev.candintel_protected_guest_has()is implemented in, as far as I can follow your paths in the diff, inarch/x86/kernel/tdx.c.No
is_protected_guest()
is_protected_guest() is a helper function added to check for VM guest type (protected or normal). Andi is going to add some security hardening code in virto and other some other generic drivers. He wants a helper function to selective enable them for all protected guests. Since these are generic drivers we need generic (non arch specific) helper call. is_protected_guest() is proposed for this purpose.
We can also use protected_guest_has(VM_VIRTIO_SECURE_FIX) or something similar for this purpose. Andi, any comments?
Quoted 1 line
and no
ARCH_HAS_PROTECTED_GUEST.
IMHO, its better to use above generic config option in common header file (linux/protected_guest.h). Any architecture that implements protected guest feature can enable it. This will help is hide arch specific config options in arch specific header file.
This seems to be a cleaner solution than including ARCH specific CONFIG option options in common header file (linux/protected_guest.h)
#ifdef CONFIG_ARCH_HAS_PROTECTED_GUEST
#include <asm/protected_guest.h>
#else
blah
#endif
is better than
#ifdef (AMD)amd_call()
#endif
#ifdef (INTEL)intel_call()
#endif
#ifdef (ARM)arm_call()
#endif
Quoted 7 lines
Just the above controlled by
CONFIG_INTEL_TDX_GUESTor whatever the TDX config item is gonna end up being and on the AMD side byCONFIG_AMD_MEM_ENCRYPT.Thx.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Borislav Petkov
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 2 lines · Sathyanarayanan
We can also use
protected_guest_has(VM_VIRTIO_SECURE_FIX)or something similar for this purpose. Andi, any comments?
protected_guest_has() is enough for that - no need for two functions.
Quoted 4 lines
IMHO, its better to use above generic config option in common header file (
linux/protected_guest.h). Any architecture that implements protected guest feature can enable it. This will help is hide arch specific config options in arch specific header file.
You define empty function stubs for when the arch config option is not enabled. Everything else is unnecessary. When another architecture needs this, then another architecture will generalize it like it is usually done.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRFC 1/1 x86/tdx: Make pages shared in ioremap()
From: "Kirill A. Shutemov" <[email protected]>
All ioremap()ed pages that are not backed by normal memory (NONE or RESERVED) have to be mapped as shared.
Reuse the infrastructure we have for AMD SEV.
Note that DMA code doesn't use ioremap() to convert memory to shared as DMA buffers backed by normal memory. DMA code make buffer shared with set_memory_decrypted().
---
Changes since RFC v2:
* Replaced is_tdx_guest() checks with protected_guest_has() calls.
* Renamed pgprot_tdg_shared() to pgprot_protected_guest()
arch/x86/include/asm/pgtable.h | 4 ++++ arch/x86/mm/ioremap.c | 9 ++++++--- 2 files changed, 10 insertions(+), 3 deletions(-)
arch/x86/include/asm/pgtable.h +4 -0
@@ -21,6 +21,10 @@#define pgprot_encrypted (prot) __pgprot(__sme_set(pgprot_val(prot)))#define pgprot_decrypted (prot) __pgprot(__sme_clr(pgprot_val(prot))) +/* Make the page accesable by VMM for protected guests */ +#define pgprot_protected_guest (prot) __pgprot(pgprot_val(prot) | \ + tdg_shared_mask()) +#ifndef __ASSEMBLY__#include <asm/x86_init.h> #include <asm/fpu/xstate.h>
arch/x86/mm/ioremap.c +6 -3
@@ -17,6 +17,7 @@#include <linux/mem_encrypt.h> #include <linux/efi.h> #include <linux/pgtable.h> +#include <linux/protected_guest.h> #include <asm/set_memory.h> #include <asm/e820/api.h>
@@ -87,12 +88,12 @@ static unsigned int __ioremap_check_ram(struct resource *res) }/* -* In a SEV guest, NONE and RESERVED should not be mapped encrypted because -* there the whole memory is already encrypted. +* In a SEV or TDX guest, NONE and RESERVED should not be mapped encrypted (or +* private in TDX case) because there the whole memory is already encrypted. */ static unsigned int __ioremap_check_encrypted (struct resource *res) { -if (!sev_active ()) +if (!sev_active () && !protected_guest_has (VM_MEM_ENCRYPT ))return 0 ;switch (res->desc) {
@@ -244,6 +245,8 @@ __ioremap_caller(resource_size_t phys_addr, unsigned long size, prot =PAGE_KERNEL_IO ;if ((io_desc.flags &IORES_MAP_ENCRYPTED ) || encrypted) prot =pgprot_encrypted (prot); +else if (protected_guest_has (VM_SHARED_MAPPING_INIT )) + prot =pgprot_protected_guest (prot);switch (pcm) {case _PAGE_CACHE_MODE_UC:
Signature · 1 line
2.25.1
kernel test robot
build botRe: [RFC v2-fix-v1 1/1] x86/tdx: Make pages shared in ioremap()
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Kuppuswamy,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on tip/x86/core]
[also build test ERROR on tip/x86/mm v5.13-rc4 next-20210528] [cannot apply to luto/next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Kuppuswamy-Sathyanarayanan/x86-tdx-Make-pages-shared-in-ioremap/20210601-100825 base: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git 2c88d45edbb89029c1190bb3b136d2602f057c98 config: i386-tinyconfig (attached as .config) compiler: gcc-9 (Debian 9.3.0-22) 9.3.0 reproduce (this is a W=1 build): # https://github.com/0day-ci/linux/commit/b1140e0d6eaf92621223679adab3f2b90048a731 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Kuppuswamy-Sathyanarayanan/x86-tdx-Make-pages-shared-in-ioremap/20210601-100825 git checkout b1140e0d6eaf92621223679adab3f2b90048a731 # save the attached .config to linux build tree make W=1 ARCH=i386
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
>> arch/x86/mm/ioremap.c:20:10: fatal error: linux/protected_guest.h: No such file or directory
20 | #include <linux/protected_guest.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
vim +20 arch/x86/mm/ioremap.c
> 20 #include <linux/protected_guest.h>
21 ---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kernel test robot
build botRe: [RFC v2-fix-v1 1/1] x86/tdx: Make pages shared in ioremap()
In reply to Kuppuswamy Sathyanarayanan[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Kuppuswamy,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on tip/x86/core]
[also build test ERROR on tip/x86/mm v5.13-rc4 next-20210528] [cannot apply to luto/next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Kuppuswamy-Sathyanarayanan/x86-tdx-Make-pages-shared-in-ioremap/20210601-100825 base: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git 2c88d45edbb89029c1190bb3b136d2602f057c98 config: x86_64-randconfig-a002-20210601 (attached as .config) compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project db26cd30b6dd65e88d786e97a1e453af5cd48966)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/b1140e0d6eaf92621223679adab3f2b90048a731
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Kuppuswamy-Sathyanarayanan/x86-tdx-Make-pages-shared-in-ioremap/20210601-100825 git checkout b1140e0d6eaf92621223679adab3f2b90048a731
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
>> arch/x86/mm/ioremap.c:20:10: fatal error: 'linux/protected_guest.h' file not found
#include <linux/protected_guest.h>
^~~~~~~~~~~~~~~~~~~~~~~~~1 error generated.
vim +20 arch/x86/mm/ioremap.c
> 20 #include <linux/protected_guest.h>
21 ---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
In reply to Borislav PetkovHi,
Quoted 4 lines · Borislav Petkov
Quoted 2 lines
We can also use
protected_guest_has(VM_VIRTIO_SECURE_FIX)or something similar for this purpose. Andi, any comments?
protected_guest_has()is enough for that - no need for two functions.
Quoted 8 lines
Quoted 4 lines
IMHO, its better to use above generic config option in common header file (
linux/protected_guest.h). Any architecture that implements protected guest feature can enable it. This will help is hide arch specific config options in arch specific header file.You define empty function stubs for when the arch config option is not enabled. Everything else is unnecessary. When another architecture needs this, then another architecture will generalize it like it is usually done.
Please check the updated version in email titled "[RFC v2-fix-v2 1/1] x86:
Introduce generic protected guest abstraction".
We can continue the rest of the discussion in that email.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
In reply to Tom LendackyQuoted 10 lines · Tom Lendacky
In
arch/x86/mm/mem_encrypt.c,sme_early_init()(should have renamed that when SEV support was added), we do:if (sev_active()) swiotlb_force = SWIOTLB_FORCE;TDX should be able to do a similar thing without having to touch
arch/x86/kernel/pci-swiotlb.c.That would remove any confusion over SME being part of a
protected_guest_has()call.
You mean sme_active() check in arch/x86/kernel/pci-swiotlb.c is redundant?
41 int __init pci_swiotlb_detect_4gb(void)
42 {
43 /* don't initialize swiotlb if iommu=off (no_iommu=1) */
44 if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
45 swiotlb = 1;
46
47 /*
48 * If SME is active then swiotlb will be set to 1 so that bounce 49 * buffers are allocated and used for devices that do not support 50 * the addressing range required for the encryption mask. 51 */
52 if (sme_active() || is_tdx_guest())
53 swiotlb = 1;
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 8 lines · Sathyanarayanan
Quoted 5 lines · Tom Lendacky
In
arch/x86/mm/mem_encrypt.c,sme_early_init()(should have renamed that when SEV support was added), we do:
if (sev_active())
swiotlb_force=SWIOTLB_FORCE;
Quoted 7 lines
Quoted 5 lines
TDX should be able to do a similar thing without having to touch
arch/x86/kernel/pci-swiotlb.c.That would remove any confusion over SME being part of a
protected_guest_has()call.You mean
sme_active()check inarch/x86/kernel/pci-swiotlb.cis redundant?
No, the sme_active() check is required to make sure that SWIOTLB is available under SME. Encrypted DMA is supported under SME if the device supports 64-bit DMA. But if the device doesn't support 64-bit DMA and the IOMMU is not active, then DMA will be bounced through SWIOTLB.
As compared to SEV, where all DMA has to be bounced through SWIOTLB or unencrypted memory. For that, swiotlb_force is used.
Thanks,
Tom
Quoted 16 lines
41 int
__initpci_swiotlb_detect_4gb(void)
42 {
43 /* don't initialize swiotlb if iommu=off (no_iommu=1) */ 44 if (!no_iommu &&max_possible_pfn>MAX_DMA32_PFN)
45 swiotlb = 1;
46
47 /*
48 * If SME is active then swiotlb will be set to 1 so that bounce 49 * buffers are allocated and used for devices that do not support 50 * the addressing range required for the encryption mask. 51 */
52 if (sme_active()||is_tdx_guest())
53 swiotlb = 1;
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 17 lines · Tom Lendacky
Quoted 8 lines · Sathyanarayanan
Quoted 5 lines · Tom Lendacky
In
arch/x86/mm/mem_encrypt.c,sme_early_init()(should have renamed that when SEV support was added), we do:
if (sev_active())
swiotlb_force=SWIOTLB_FORCE;Quoted 7 lines
Quoted 5 lines
TDX should be able to do a similar thing without having to touch
arch/x86/kernel/pci-swiotlb.c.That would remove any confusion over SME being part of a
protected_guest_has()call.You mean
sme_active()check inarch/x86/kernel/pci-swiotlb.cis redundant?
Quoted 7 lines
No, the
sme_active()check is required to make sure that SWIOTLB is available under SME. Encrypted DMA is supported under SME if the device supports 64-bit DMA. But if the device doesn't support 64-bit DMA and the IOMMU is not active, then DMA will be bounced through SWIOTLB.As compared to SEV, where all DMA has to be bounced through SWIOTLB or unencrypted memory. For that,
swiotlb_forceis used.
I should probably add that SME is memory encryption support for host/hypervisor/bare-metal, while SEV is memory encryption support for virtualization.
Thanks,
Tom
Quoted 20 lines
Thanks,
TomQuoted 16 lines
41 int
__initpci_swiotlb_detect_4gb(void)
42 {
43 /* don't initialize swiotlb if iommu=off (no_iommu=1) */ 44 if (!no_iommu &&max_possible_pfn>MAX_DMA32_PFN)
45 swiotlb = 1;
46
47 /*
48 * If SME is active then swiotlb will be set to 1 so that bounce 49 * buffers are allocated and used for devices that do not support 50 * the addressing range required for the encryption mask. 51 */
52 if (sme_active()||is_tdx_guest())
53 swiotlb = 1;
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 23 lines · Tom Lendacky
Quoted 22 lines · Tom Lendacky
Quoted 8 lines · Sathyanarayanan
Quoted 5 lines · Tom Lendacky
In
arch/x86/mm/mem_encrypt.c,sme_early_init()(should have renamed that when SEV support was added), we do:
if (sev_active())
swiotlb_force=SWIOTLB_FORCE;Quoted 7 lines
Quoted 5 lines
TDX should be able to do a similar thing without having to touch
arch/x86/kernel/pci-swiotlb.c.That would remove any confusion over SME being part of a
protected_guest_has()call.You mean
sme_active()check inarch/x86/kernel/pci-swiotlb.cis redundant?No, the
sme_active()check is required to make sure that SWIOTLB is available under SME. Encrypted DMA is supported under SME if the device supports 64-bit DMA. But if the device doesn't support 64-bit DMA and the IOMMU is not active, then DMA will be bounced through SWIOTLB.
Quoted 6 lines
Quoted 2 lines
As compared to SEV, where all DMA has to be bounced through SWIOTLB or unencrypted memory. For that,
swiotlb_forceis used.I should probably add that SME is memory encryption support for host/hypervisor/bare-metal, while SEV is memory encryption support for virtualization.
Got it. Thanks for clarification.
Quoted 24 lines
Thanks,
TomQuoted 20 lines
Thanks,
TomQuoted 16 lines
41 int
__initpci_swiotlb_detect_4gb(void)
42 {
43 /* don't initialize swiotlb if iommu=off (no_iommu=1) */ 44 if (!no_iommu &&max_possible_pfn>MAX_DMA32_PFN) 45 swiotlb = 1;
46
47 /*
48 * If SME is active then swiotlb will be set to 1 so that bounce 49 * buffers are allocated and used for devices that do not support 50 * the addressing range required for the encryption mask. 51 */
52 if (sme_active()||is_tdx_guest())
53 swiotlb = 1;
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
RFC v2 29/32 x86/tdx: Add helper to do MapGPA TDVMALL
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
MapGPA TDVMCALL requests the host VMM to map a GPA range as private or shared memory mappings. Shared GPA mappings can be used for communication beteen TD guest and host VMM, for example for paravirtualized IO.
The new helper tdx_map_gpa() provides access to the operation.
--- arch/x86/include/asm/tdx.h | 13 +++++++++++++ arch/x86/kernel/tdx.c | 13 +++++++++++++ 2 files changed, 26 insertions(+)
arch/x86/include/asm/tdx.h +13 -0
@@ -7,6 +7,11 @@#ifndef __ASSEMBLY__ +enum tdx_map_type { +TDX_MAP_PRIVATE , +TDX_MAP_SHARED , +}; +#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h>
@@ -112,6 +117,8 @@ unsigned short tdg_inw(unsigned short port);unsigned int tdg_inl (unsigned short port);extern phys_addr_t tdg_shared_mask (void ); +extern int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type);#else // !CONFIG_INTEL_TDX_GUEST
@@ -155,6 +162,12 @@ static inline phys_addr_t tdg_shared_mask(void) {return 0 ; } + +static inline int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) +{ +return -ENODEV ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */
arch/x86/kernel/tdx.c +13 -0
@@ -15,6 +15,8 @@#include "tdx-kvm.c" #endif +#define TDVMCALL_MAP_GPA 0x10001 + static struct { unsigned int gpa_width; unsigned long attributes;
@@ -98,6 +100,17 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } +int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +u64 ret; + +if (map_type ==TDX_MAP_SHARED ) + gpa |=tdg_shared_mask (); + + ret =tdvmcall (TDVMCALL_MAP_GPA , gpa,PAGE_SIZE * numpages,0 ,0 ); +return ret ? -EIO :0 ; +} + static __cpuidle void tdg_halt(void) {u64 ret;
Signature · 1 line
2.25.1
Re: [RFC v2 29/32] x86/tdx: Add helper to do MapGPA TDVMALL
Quoted 6 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
MapGPA TDVMCALL requests the host VMM to map a GPA range as private or shared memory mappings. Shared GPA mappings can be used for communication beteen TD guest and host VMM, for example for paravirtualized IO.
As usual, I hate the changelog. This appears to just be regurgitating the spec.
Is this just for part of converting an existing mapping between private and shared? If so, please say that.
Quoted 1 line
The new helper
tdx_map_gpa()provides access to the operation.
<sigh> You got your own name wrong. It's tdg_map_gpa() in the patch.
BTW, I agree with Sean on this one: "tdg" is a horrible prefix. You just proved Sean's point by mistyping it. *EVERYONE* is going to rpeat that mistake: tdg -> tdx.
arch/x86/include/asm/tdx.h line 13
Quoted 12 lines
arch/x86/include/asm/tdx.h +4
-0@@ -7,6 +7,11 @@#ifndef __ASSEMBLY__ +enum tdx_map_type { +TDX_MAP_PRIVATE , +TDX_MAP_SHARED , +};
I like the enum, but please call out that this is a software construct, not a part of any hardware or VMM ABI.
Quoted 13 lines
arch/x86/include/asm/tdx.h +2
-0#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h> @@ -112,6 +117,8 @@ unsigned short tdg_inw(unsigned short port);unsigned int tdg_inl (unsigned short port);extern phys_addr_t tdg_shared_mask (void ); +extern int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type);#else // !CONFIG_INTEL_TDX_GUEST
arch/x86/include/asm/tdx.h static inline phys_addr_t tdg_shared_mask(void) · line 170
Quoted 9 lines
arch/x86/include/asm/tdx.h +6
-0@@ -155,6 +162,12 @@ static inline phys_addr_t tdg_shared_mask(void) {return 0 ; } + +static inline int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) +{ +return -ENODEV ; +}
FWIW, you could probably get away with just inlining tdg_map_gpa():
static inline int tdg_map_gpa(phys_addr_t gpa, int numpages, ...
{
u64 ret;
if (!IS_ENABLED(CONFIG_INTEL_TDX_GUEST))
return -ENODEV;
if (map_type == TDX_MAP_SHARED)
gpa |= tdg_shared_mask();
ret = tdvmcall(TDVMCALL_MAP_GPA, gpa, ...
return ret ? -EIO : 0;
}Then you don't have three copies of the function signature that can get out of sync.
Quoted 20 lines
arch/x86/include/asm/tdx.h +0
-0#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */ arch/x86/kernel/tdx.c +2
-0@@ -15,6 +15,8 @@#include "tdx-kvm.c" #endif +#define TDVMCALL_MAP_GPA 0x10001 + static struct { unsigned int gpa_width; unsigned long attributes;@@ -98,6 +100,17 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); }
arch/x86/kernel/tdx.c static void tdg_get_info(void) · line 112
Quoted 10 lines
arch/x86/kernel/tdx.c +10
-0+int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +u64 ret; + +if (map_type ==TDX_MAP_SHARED ) + gpa |=tdg_shared_mask (); + + ret =tdvmcall (TDVMCALL_MAP_GPA , gpa,PAGE_SIZE * numpages,0 ,0 ); +return ret ? -EIO :0 ; +}
The naming Intel chose here is nasty. This doesn't "map" anything. It modifies an existing mapping from what I can tell. We could name it much better than the spec, perhaps:
tdx_hcall_gpa_intent()
BTW, all of these hypercalls need a consistent prefix.
It also needs a comment:
/*
* Inform the VMM of the guest's intent for this physical page:
* shared with the VMM or private to the guest. The VMM is
* expected to change its mapping of the page in response. *
* Note: shared->private conversions require further guest
* action to accept the page.
*/
The intent here is important. It makes it clear that this function really only plays a role in the conversion process.
Re: [RFC v2 29/32] x86/tdx: Add helper to do MapGPA TDVMALL
Quoted 8 lines · Dave Hansen
Quoted 6 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
MapGPA TDVMCALL requests the host VMM to map a GPA range as private or shared memory mappings. Shared GPA mappings can be used for communication beteen TD guest and host VMM, for example for paravirtualized IO.
Quoted 6 lines
As usual, I hate the changelog. This appears to just be regurgitating the spec.
Is this just for part of converting an existing mapping between private and shared? If so, please say that.
How about following change?
x86/tdx: Add helper to do MapGPA hypercall
MapGPA hypercall is used by TDX guests to request VMM convert the existing mapping of given GPA address range between private/shared.
tdx_hcall_gpa_intent() is the wrapper used for making MapGPA hypercall.
Quoted 3 lines
Quoted 1 line
The new helper
tdx_map_gpa()provides access to the operation.<sigh> You got your own name wrong. It's
tdg_map_gpa()in the patch.
I can use tdx_hcall_gpa_intent().
Quoted 61 lines
BTW, I agree with Sean on this one: "tdg" is a horrible prefix. You just proved Sean's point by mistyping it. *EVERYONE* is going to rpeat that mistake: tdg -> tdx.
arch/x86/include/asm/tdx.h line 13
Quoted 12 lines
arch/x86/include/asm/tdx.h +4
-0@@ -7,6 +7,11 @@#ifndef __ASSEMBLY__ +enum tdx_map_type { +TDX_MAP_PRIVATE , +TDX_MAP_SHARED , +};I like the enum, but please call out that this is a software construct, not a part of any hardware or VMM ABI.
Quoted 13 lines
arch/x86/include/asm/tdx.h +2
-0#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h> @@ -112,6 +117,8 @@ unsigned short tdg_inw(unsigned short port);unsigned int tdg_inl (unsigned short port);extern phys_addr_t tdg_shared_mask (void ); +extern int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type);#else // !CONFIG_INTEL_TDX_GUEST arch/x86/include/asm/tdx.h static inline phys_addr_t tdg_shared_mask(void) · line 170
Quoted 9 lines
arch/x86/include/asm/tdx.h +6
-0@@ -155,6 +162,12 @@ static inline phys_addr_t tdg_shared_mask(void) {return 0 ; } + +static inline int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) +{ +return -ENODEV ; +}FWIW, you could probably get away with just inlining
tdg_map_gpa():static inline int tdg_map_gpa(phys_addr_t gpa, int numpages, ...
{ u64 ret; if (!IS_ENABLED(CONFIG_INTEL_TDX_GUEST)) return -ENODEV; if (map_type == TDX_MAP_SHARED) gpa |= tdg_shared_mask(); ret = tdvmcall(TDVMCALL_MAP_GPA, gpa, ... return ret ? -EIO : 0; }
Quoted 2 lines
Then you don't have three copies of the function signature that can get out of sync.
I agree that this simplifies the function definition. But, there are other TDX hypercalls definitions in tdx.c. I can't move all of them to the header file. If possible, I would like to group all hypercalls in the same place.
Also, IMO, it is better to hide hypercall internal implementation details in C file. For example, user of MapGPA hypercall does not care about the TDVMCALL_MAP_GPA leaf id value. If we inline this function we have to move such details to header file.
Quoted 36 lines
Quoted 21 lines
arch/x86/include/asm/tdx.h +0
-0#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */ arch/x86/kernel/tdx.c +2
-0@@ -15,6 +15,8 @@#include "tdx-kvm.c" #endif +#define TDVMCALL_MAP_GPA 0x10001 + static struct { unsigned int gpa_width; unsigned long attributes;@@ -98,6 +100,17 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); }arch/x86/kernel/tdx.c static void tdg_get_info(void) · line 112
Quoted 10 lines
arch/x86/kernel/tdx.c +10
-0+int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +u64 ret; + +if (map_type ==TDX_MAP_SHARED ) + gpa |=tdg_shared_mask (); + + ret =tdvmcall (TDVMCALL_MAP_GPA , gpa,PAGE_SIZE * numpages,0 ,0 ); +return ret ? -EIO :0 ; +}The naming Intel chose here is nasty. This doesn't "map" anything. It modifies an existing mapping from what I can tell. We could name it much better than the spec, perhaps:
Quoted 1 line
tdx_hcall_gpa_intent()
I will use this function name in next version.
Quoted 2 lines
BTW, all of these hypercalls need a consistent prefix.
I can include _hcall in other hypercall helper functions as well.
Quoted 12 lines
It also needs a comment:
/*
* Inform the VMM of the guest's intent for this physical page:
* shared with the VMM or private to the guest. The VMM is
* expected to change its mapping of the page in response. *
* Note:shared->privateconversions require further guest
* action to accept the page.
*/
Quoted 2 lines
The intent here is important. It makes it clear that this function really only plays a role in the conversion process.
Thanks. I will include it in next version.
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
RFC 1/1 x86/tdx: Add helper to do MapGPA hypercall
From: "Kirill A. Shutemov" <[email protected]>
MapGPA hypercall is used by TDX guests to request VMM convert the existing mapping of given GPA address range between private/shared.
tdx_hcall_gpa_intent() is the wrapper used for making MapGPA hypercall.
---
Changes since RFC v2:
* Renamed tdg_map_gpa() to tdx_hcall_gpa_intent().
* Fixed commit log and comments as per review comments.
arch/x86/include/asm/tdx.h | 17 +++++++++++++++++ arch/x86/kernel/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 41 insertions(+)
arch/x86/include/asm/tdx.h +17 -0
@@ -7,6 +7,15 @@#ifndef __ASSEMBLY__ +/* +* Page mapping type enum. This is software construct not +* part of any hardware or VMM ABI. +*/ +enum tdx_map_type { +TDX_MAP_PRIVATE , +TDX_MAP_SHARED , +}; +#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h>
@@ -119,6 +128,8 @@ do { \#endif extern phys_addr_t tdg_shared_mask (void ); +extern int tdx_hcall_gpa_intent (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type);#else // !CONFIG_INTEL_TDX_GUEST
@@ -143,6 +154,12 @@ static inline phys_addr_t tdg_shared_mask(void) {return 0 ; } + +static inline int tdx_hcall_gpa_intent (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) +{ +return -ENODEV ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #ifdef CONFIG_INTEL_TDX_GUEST_KVM
arch/x86/kernel/tdx.c +24 -0
@@ -16,6 +16,9 @@#define TDINFO 1#define TDGETVEINFO 3 +/* TDX hypercall Leaf IDs */ +#define TDVMCALL_MAP_GPA 0x10001 +#define VE_GET_IO_TYPE (exit_qual) (((exit_qual) & 8) ? 0 : 1)#define VE_GET_IO_SIZE (exit_qual) (((exit_qual) & 7) + 1)#define VE_GET_PORT_NUM (exit_qual) ((exit_qual) >> 16)
@@ -122,6 +125,27 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } +/* +* Inform the VMM of the guest's intent for this physical page: +* shared with the VMM or private to the guest. The VMM is +* expected to change its mapping of the page in response. +* +* Note: shared->private conversions require further guest +* action to accept the page. +*/ +int tdx_hcall_gpa_intent (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) +{ +u64 ret; + +if (map_type ==TDX_MAP_SHARED ) + gpa |=tdg_shared_mask (); + + ret =tdx_hypercall (TDVMCALL_MAP_GPA , gpa,PAGE_SIZE * numpages,0 ,0 ); + +return ret ? -EIO :0 ; +} + static __cpuidle void tdg_halt(void) {u64 ret;
Signature · 1 line
2.25.1
RFC v2 30/32 x86/tdx: Make DMA pages shared
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Make force_dma_unencrypted() return true for TDX to get DMA pages mapped as shared.
__set_memory_enc_dec() is now aware about TDX and sets Shared bit accordingly following with relevant TDVMCALL.
Also, Do TDACCEPTPAGE on every 4k page after mapping the GPA range when converting memory to private. If the VMM uses a common pool for private and shared memory, it will likely do TDAUGPAGE in response to MAP_GPA (or on the first access to the private GPA), in which case TDX-Module will hold the page in a non-present "pending" state until it is explicitly accepted.
BUG() if TDACCEPTPAGE fails (except the above case), as the guest is completely hosed if it can't access memory.
--- arch/x86/include/asm/tdx.h | 3 ++ arch/x86/kernel/tdx.c | 26 ++++++++++++++++- arch/x86/mm/mem_encrypt_common.c | 4 +-- arch/x86/mm/pat/set_memory.c | 48 ++++++++++++++++++++++++++------ 4 files changed, 70 insertions(+), 11 deletions(-)
arch/x86/include/asm/tdx.h +3 -0
@@ -19,6 +19,9 @@ enum tdx_map_type {#define TDINFO 1#define TDGETVEINFO 3 +#define TDACCEPTPAGE 6 + +#define TDX_PAGE_ALREADY_ACCEPTED 0x8000000000000001struct tdcall_output {u64 rcx;
arch/x86/kernel/tdx.c +25 -1
@@ -100,7 +100,8 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } -int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +static int __tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) {u64 ret;
@@ -111,6 +112,29 @@ int tdg_map_gpa(phys_addr_t gpa, int numpages, enum tdx_map_type map_type)return ret ? -EIO :0 ; } +static void tdg_accept_page (phys_addr_t gpa) +{ +u64 ret; + + ret =__tdcall (TDACCEPTPAGE , gpa,0 ,0 ,0 ,NULL ); + +BUG_ON (ret && ret !=TDX_PAGE_ALREADY_ACCEPTED ); +} + +int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +int ret, i; + + ret =__tdg_map_gpa (gpa, numpages, map_type); +if (ret || map_type ==TDX_MAP_SHARED ) +return ret; + +for (i =0 ; i < numpages; i++) +tdg_accept_page (gpa + i*PAGE_SIZE ); + +return 0 ; +} + static __cpuidle void tdg_halt(void) {u64 ret;
arch/x86/mm/mem_encrypt_common.c +2 -2
@@ -15,9 +15,9 @@bool force_dma_unencrypted (struct device *dev) {/* -* For SEV, all DMA must be to unencrypted/shared addresses. +* For SEV and TDX, all DMA must be to unencrypted/shared addresses. */ -if (sev_active ()) +if (sev_active () ||is_tdx_guest ())return true;/*
arch/x86/mm/pat/set_memory.c +40 -8
93 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC 1/1 x86/tdx: Make DMA pages shared
From: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to access guest memory. Any memory that is required for communication with VMM must be shared explicitly by setting the bit in page table entry. And, after setting the shared bit, the conversion must be completed with MapGPA TDVMALL. The call informs VMM about the conversion and makes it remove the GPA from the S-EPT mapping. The shared memory is similar to unencrypted memory in AMD SME/SEV terminology but the underlying process of sharing/un-sharing the memory is different for Intel TDX guest platform.
SEV assumes that I/O devices can only do DMA to "decrypted" physical addresses without the C-bit set. In order for the CPU to interact with this memory, the CPU needs a decrypted mapping. To add this support, AMD SME code forces force_dma_unencrypted() to return true for platforms that support AMD SEV feature. It will be used for DMA memory allocation API to trigger set_memory_decrypted() for platforms that support AMD SEV feature.
TDX is similar. TDX architecturally prevents access to private guest memory by anything other than the guest itself. This means that any DMA buffers must be shared.
So create a new file mem_encrypt_tdx.c to hold TDX specific memory initialization code, and re-define force_dma_unencrypted() for TDX guest and make it return true to get DMA pages mapped as shared.
__set_memory_enc_dec() is now aware about TDX and sets Shared bit accordingly following with relevant TDVMCALL.
Also, Do TDACCEPTPAGE on every 4k page after mapping the GPA range when converting memory to private. If the VMM uses a common pool for private and shared memory, it will likely do TDAUGPAGE in response to MAP_GPA (or on the first access to the private GPA), in which case TDX-Module will hold the page in a non-present "pending" state until it is explicitly accepted.
BUG() if TDACCEPTPAGE fails (except the above case), as the guest is completely hosed if it can't access memory.
---
Changes since RFC v2:
* Since the common code between AMD-SEV and TDX is very minimal, defining a new config (X86_MEM_ENCRYPT_COMMON) for common code is not very useful. So createed a seperate file for Intel TDX specific memory initialization (similar to AMD SEV).
* Removed patch titled "x86/mm: Move force_dma_unencrypted() to common code" from this series. And merged required changes in this patch.
arch/x86/Kconfig | 1 + arch/x86/include/asm/tdx.h | 3 +++ arch/x86/kernel/tdx.c | 26 ++++++++++++++++++- arch/x86/mm/Makefile | 1 + arch/x86/mm/mem_encrypt_tdx.c | 19 ++++++++++++++ arch/x86/mm/pat/set_memory.c | 48 +++++++++++++++++++++++++++++------ 6 files changed, 89 insertions(+), 9 deletions(-) create mode 100644 arch/x86/mm/mem_encrypt_tdx.c
arch/x86/Kconfig +1 -0
@@ -879,6 +879,7 @@ config INTEL_TDX_GUEST select X86_X2APIC select SECURITY_LOCKDOWN_LSM select ARCH_HAS_PROTECTED_GUEST + select ARCH_HAS_FORCE_DMA_UNENCRYPTED select DYNAMIC_PHYSICAL_MASK help Provide support for running in a trusted domain on Intel processors
arch/x86/include/asm/tdx.h +3 -0
@@ -19,6 +19,9 @@ enum tdx_map_type {#define TDINFO 1#define TDGETVEINFO 3 +#define TDACCEPTPAGE 6 + +#define TDX_PAGE_ALREADY_ACCEPTED 0x8000000000000001struct tdx_module_output {u64 rcx;
arch/x86/kernel/tdx.c +25 -1
@@ -100,7 +100,8 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } -int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +static int __tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) {u64 ret;
@@ -111,6 +112,29 @@ int tdg_map_gpa(phys_addr_t gpa, int numpages, enum tdx_map_type map_type)return ret ? -EIO :0 ; } +static void tdg_accept_page (phys_addr_t gpa) +{ +u64 ret; + + ret =__tdx_module_call (TDACCEPTPAGE , gpa,0 ,0 ,0 ,NULL ); + +BUG_ON (ret && ret !=TDX_PAGE_ALREADY_ACCEPTED ); +} + +int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +int ret, i; + + ret =__tdg_map_gpa (gpa, numpages, map_type); +if (ret || map_type ==TDX_MAP_SHARED ) +return ret; + +for (i =0 ; i < numpages; i++) +tdg_accept_page (gpa + i*PAGE_SIZE ); + +return 0 ; +} + static __cpuidle void tdg_halt(void) {u64 ret;
arch/x86/mm/Makefile +1 -0
@@ -55,3 +55,4 @@ obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o +obj-$(CONFIG_INTEL_TDX_GUEST) += mem_encrypt_tdx.o
arch/x86/mm/mem_encrypt_tdx.c new file +19 -0
@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Intel TDX Memory Encryption Support +* +* Copyright (C) 2020 Intel Corporation +* +* Author: Kuppuswamy Sathyanarayanan <[email protected]> +*/ + +#include <linux/mm.h> +#include <linux/dma-mapping.h> + +#include <asm/tdx.h> + +/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ +bool force_dma_unencrypted (struct device *dev) +{ +return is_tdx_guest (); +}
arch/x86/mm/pat/set_memory.c +40 -8
93 lines not on this page · raw message
Signature · 1 line
2.25.1
Sean Christopherson
maintainerRe: [RFC v2-fix 1/1] x86/tdx: Make DMA pages shared
Quoted 3 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to access guest memory. Any memory
^
|- privateAnd to be pedantic, the VMM can _access_ guest private memory all it wants, it just can't decrypt guest private memory.
Quoted 5 lines
that is required for communication with VMM must be shared explicitly by setting the bit in page table entry. And, after setting the shared bit, the conversion must be completed with MapGPA TDVMALL. The call informs VMM about the conversion and makes it remove the GPA from the S-EPT mapping.
The VMM is _not_ required to remove the GPA from the S-EPT. E.g. if the VMM wants to, it can leave a 2mb private page intact and create a 4kb shared page translation within the same range (ignoring the shared bit).
Quoted 12 lines
The shared memory is similar to unencrypted memory in AMD SME/SEV terminology but the underlying process of sharing/un-sharing the memory is different for Intel TDX guest platform.
SEV assumes that I/O devices can only do DMA to "decrypted" physical addresses without the C-bit set. In order for the CPU to interact with this memory, the CPU needs a decrypted mapping. To add this support, AMD SME code forces
force_dma_unencrypted()to return true for platforms that support AMD SEV feature. It will be used for DMA memory allocation API to triggerset_memory_decrypted()for platforms that support AMD SEV feature.
Quoted 1 line
TDX is similar. TDX architecturally prevents access to private
TDX doesn't prevent accesses. If hardware _prevented_ accesses then we wouldn't have to deal with the #MC mess.
Quoted 11 lines
guest memory by anything other than the guest itself. This means that any DMA buffers must be shared.
So create a new file
mem_encrypt_tdx.cto hold TDX specific memory initialization code, and re-defineforce_dma_unencrypted()for TDX guest and make it return true to get DMA pages mapped as shared.
__set_memory_enc_dec()is now aware about TDX and sets Shared bit accordingly following with relevant TDVMCALL.Also, Do TDACCEPTPAGE on every 4k page after mapping the GPA range when
This should call out that the current TDX spec only supports 4kb AUG/ACCEPT.
On that topic... are there plans to support 2mb and/or 1gb TDH.MEM.PAGE.AUG? If so, will TDG.MEM.PAGE.ACCEPT also support 2mb/1gb granularity?
Quoted 3 lines
converting memory to private. If the VMM uses a common pool for private and shared memory, it will likely do TDAUGPAGE in response to
MAP_GPA(or on the first access to the private GPA),
What the VMM does or does not do is irrelevant. What matters is what the VMM is _allowed_ to do without violating the GHCI. Specifically, the VMM is allowed to unmap a private page in response to MAP_GPA to convert to a shared page.
If the GPA (range) was already mapped as an active, private page, the host VMM may remove the private page from the TD by following the “Removing TD Private Pages” sequence in the Intel TDX-module specification [3] to safely block the mapping(s), flush the TLB and cache, and remove the mapping(s).
That would also provide a nice segue into the "already accepted" error below.
Quoted 4 lines
in which case TDX-Module will hold the page in a non-present "pending" state until it is explicitly accepted.
BUG()if TDACCEPTPAGE fails (except the above case)
What above case? The code handles the case where the page was already accepted, but the changelog doesn't talk about that at all.
Quoted 1 line
as the guest is completely hosed if it can't access memory.
Re: [RFC v2-fix 1/1] x86/tdx: Make DMA pages shared
Quoted 9 lines · Sean Christopherson
Quoted 3 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to access guest memory. Any memory
^ |- privateAnd to be pedantic, the VMM can _access_ guest private memory all it wants, it just can't decrypt guest private memory.
Ok. I will use "guest private memory".
Quoted 10 lines
Quoted 6 lines
that is required for communication with VMM must be shared explicitly by setting the bit in page table entry. And, after setting the shared bit, the conversion must be completed with MapGPA TDVMALL. The call informs VMM about the conversion and makes it remove the GPA from the S-EPT mapping.
The VMM is _not_ required to remove the GPA from the S-EPT. E.g. if the VMM wants to, it can leave a 2mb private page intact and create a 4kb shared page translation within the same range (ignoring the shared bit).
So does removing "makes it remove the GPA from the S-EPT mapping" be sufficient? Or you want to add more detail?
Quoted 13 lines
Quoted 13 lines
The shared memory is similar to unencrypted memory in AMD SME/SEV terminology but the underlying process of sharing/un-sharing the memory is different for Intel TDX guest platform.
SEV assumes that I/O devices can only do DMA to "decrypted" physical addresses without the C-bit set. In order for the CPU to interact with this memory, the CPU needs a decrypted mapping. To add this support, AMD SME code forces
force_dma_unencrypted()to return true for platforms that support AMD SEV feature. It will be used for DMA memory allocation API to triggerset_memory_decrypted()for platforms that support AMD SEV feature.
Quoted 4 lines
Quoted 1 line
TDX is similar. TDX architecturally prevents access to private
TDX doesn't prevent accesses. If hardware _prevented_ accesses then we wouldn't have to deal with the #MC mess.
How about following change?
"TDX is similar. TDX architecturally prevents access to private guest memory by
anything other than the guest itself.This means that any DMA buffers must be shared."
modified to =>
"TDX is similar. In TDX architecture, the private guest memory is encrypted, which prevents anything other than guest from accessing/modifying it. So to communicate with I/O devices, we need to create decrypted mapping and make the pages shared."
Quoted 8 lines
Quoted 8 lines
guest memory by anything other than the guest itself. This means that any DMA buffers must be shared.
So create a new file
mem_encrypt_tdx.cto hold TDX specific memory initialization code, and re-defineforce_dma_unencrypted()for TDX guest and make it return true to get DMA pages mapped as shared.
Quoted 6 lines
Quoted 4 lines
__set_memory_enc_dec()is now aware about TDX and sets Shared bit accordingly following with relevant TDVMCALL.Also, Do TDACCEPTPAGE on every 4k page after mapping the GPA range when
This should call out that the current TDX spec only supports 4kb AUG/ACCEPT.
Ok. I will add this spec detail.
Quoted 12 lines
On that topic... are there plans to support 2mb and/or 1gb TDH.MEM.PAGE.AUG? If so, will TDG.MEM.PAGE.ACCEPT also support 2mb/1gb granularity?
Quoted 3 lines
converting memory to private. If the VMM uses a common pool for private and shared memory, it will likely do TDAUGPAGE in response to
MAP_GPA(or on the first access to the private GPA),What the VMM does or does not do is irrelevant. What matters is what the VMM is _allowed_ to do without violating the GHCI. Specifically, the VMM is allowed to unmap a private page in response to
MAP_GPAto convert to a shared page.
Quoted 6 lines
If the GPA (range) was already mapped as an active, private page, the host VMM may remove the private page from the TD by following the “Removing TD Private Pages” sequence in the Intel TDX-module specification [3] to safely block the mapping(s), flush the TLB and cache, and remove the mapping(s).
That would also provide a nice segue into the "already accepted" error below.
Ok. I will add the above detail.
Quoted 8 lines
Quoted 5 lines
in which case TDX-Module will hold the page in a non-present "pending" state until it is explicitly accepted.
BUG()if TDACCEPTPAGE fails (except the above case)What above case? The code handles the case where the page was already accepted, but the changelog doesn't talk about that at all.
I think it meant about "already accepted" page case. With your above suggestion, we can ignore this error. Or I can change it to,
BUG() if TDACCEPTPAGE fails (except "previously accepted page" case)
Quoted 3 lines
Quoted 2 lines
as the guest is completely hosed if it can't access memory.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2-fix 1/1] x86/tdx: Make DMA pages shared
Quoted 4 lines · Sathyanarayanan
"TDX is similar. In TDX architecture, the private guest memory is encrypted, which prevents anything other than guest from accessing/modifying it. So to communicate with I/O devices, we need to create decrypted mapping and make the pages shared."
That's actually even more wrong. :(
Check out "Machine Check Architecture Background" in the TDX architecture spec.
Modification is totally permitted in the architecture. A host can write all day long to guest memory. Depending on how you use the word, "access" can also include writes.
TDX really just prevents guests from *consuming* the gunk that an attacker might write.
Also, don't say "decrypted". The memory is probably still TME-enabled and probably encrypted on the DIMM. It's still encrypted even if shared, it's just using the TME key, not the TD key.
RFC 1/1 x86/tdx: Make DMA pages shared
From: "Kirill A. Shutemov" <[email protected]>
Just like MKTME, TDX reassigns bits of the physical address for metadata. MKTME used several bits for an encryption KeyID. TDX uses a single bit in guests to communicate whether a physical page should be protected by TDX as private memory (bit set to 0) or unprotected and shared with the VMM (bit set to 1).
__set_memory_enc_dec() is now aware about TDX and sets Shared bit accordingly following with relevant TDX hypercall.
Also, Do TDACCEPTPAGE on every 4k page after mapping the GPA range when converting memory to private. Using 4k page size limit is due to current TDX spec restriction. Also, If the GPA (range) was already mapped as an active, private page, the host VMM may remove the private page from the TD by following the “Removing TD Private Pages” sequence in the Intel TDX-module specification [1] to safely block the mapping(s), flush the TLB and cache, and remove the mapping(s).
BUG() if TDACCEPTPAGE fails (except "previously accepted page" case) , as the guest is completely hosed if it can't access memory.
---
Changes since RFC v2-fix:
* Using seperate file for Intel TDX specific memory initialization breaks binary compatibility (between AMD/TDX). So revert back to older version.
* Fixed the commit log to reflect the above change.
* Replaced is_tdx_guest() checks with appropriate protected_guest_has() checks.
* Used tdx_hcall_gpa_intent() instead of __tdg_map_gpa() call.
* Removed __tdg_map_gpa() helper function and added tdg_accept_page() related changes to tdx_hcall_gpa_intent().
* Used pgprot_pg_shared_mask() macro for __pgprot(tdg_shared_mask()).
* Fixed commit log as per review comments.
Changes since RFC v2:
* Since the common code between AMD-SEV and TDX is very minimal, defining a new config (X86_MEM_ENCRYPT_COMMON) for common code is not very useful. So createed a seperate file for Intel TDX specific memory initialization (similar to AMD SEV).
* Removed patch titled "x86/mm: Move force_dma_unencrypted() to common code" from this series. And merged required changes in this patch.
arch/x86/include/asm/pgtable.h | 1 + arch/x86/kernel/tdx.c | 34 ++++++++++++++++++----- arch/x86/mm/mem_encrypt_common.c | 3 +++ arch/x86/mm/pat/set_memory.c | 46 +++++++++++++++++++++++++++----- 4 files changed, 71 insertions(+), 13 deletions(-)
arch/x86/include/asm/pgtable.h +1 -0
@@ -24,6 +24,7 @@/* Make the page accesable by VMM for protected guests */ #define pgprot_protected_guest (prot) __pgprot(pgprot_val(prot) | \ tdg_shared_mask()) +#define pgprot_pg_shared_mask () __pgprot(tdg_shared_mask())#ifndef __ASSEMBLY__#include <asm/x86_init.h>
arch/x86/kernel/tdx.c +28 -6
@@ -15,10 +15,14 @@/* TDX Module call Leaf IDs */ #define TDINFO 1#define TDGETVEINFO 3 +#define TDACCEPTPAGE 6/* TDX hypercall Leaf IDs */ #define TDVMCALL_MAP_GPA 0x10001 +/* TDX Module call error codes */ +#define TDX_PAGE_ALREADY_ACCEPTED 0x8000000000000001 +#define VE_GET_IO_TYPE (exit_qual) (((exit_qual) & 8) ? 0 : 1)#define VE_GET_IO_SIZE (exit_qual) (((exit_qual) & 7) + 1)#define VE_GET_PORT_NUM (exit_qual) ((exit_qual) >> 16)
@@ -126,25 +130,43 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } +static void tdg_accept_page (phys_addr_t gpa) +{ +u64 ret; + + ret =__tdx_module_call (TDACCEPTPAGE , gpa,0 ,0 ,0 ,NULL ); + +BUG_ON (ret && ret !=TDX_PAGE_ALREADY_ACCEPTED ); +} +/* * Inform the VMM of the guest's intent for this physical page: * shared with the VMM or private to the guest. The VMM is * expected to change its mapping of the page in response. -* -* Note: shared->private conversions require further guest -* action to accept the page. */ int tdx_hcall_gpa_intent (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) { -u64 ret; +u64 ret =0 ; +int i;if (map_type ==TDX_MAP_SHARED ) gpa |=tdg_shared_mask (); - ret =tdx_hypercall (TDVMCALL_MAP_GPA , gpa,PAGE_SIZE * numpages,0 ,0 ); +if (tdx_hypercall (TDVMCALL_MAP_GPA , gpa,PAGE_SIZE * numpages,0 ,0 )) + ret = -EIO ; -return ret ? -EIO :0 ; +if (ret || map_type ==TDX_MAP_SHARED ) +return ret; + +/* +* For shared->private conversion, accept the page using TDACCEPTPAGE +* TDX module call. +*/ +for (i =0 ; i < numpages; i++) +tdg_accept_page (gpa + i *PAGE_SIZE ); + +return 0 ; } static __cpuidle void tdg_halt(void)
arch/x86/mm/mem_encrypt_common.c +3 -0
@@ -16,5 +16,8 @@ bool force_dma_unencrypted(struct device *dev)if (sev_active () ||sme_active ())return amd_force_dma_unencrypted (dev); +if (protected_guest_has (VM_MEM_ENCRYPT )) +return true; +return false; }
arch/x86/mm/pat/set_memory.c +39 -7
92 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC v2 31/32 x86/kvm: Use bounce buffers for TD guest
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
TDX doesn't allow to perform DMA access to guest private memory. In order for DMA to work properly in TD guest, user SWIOTLB bounce buffers.
Move AMD SEV initialization into common code and adopt for TDX.
--- arch/x86/include/asm/io.h | 3 +- arch/x86/kernel/pci-swiotlb.c | 2 +- arch/x86/kernel/tdx.c | 3 ++ arch/x86/mm/mem_encrypt.c | 45 ------------------------------ arch/x86/mm/mem_encrypt_common.c | 47 ++++++++++++++++++++++++++++++++ 5 files changed, 53 insertions(+), 47 deletions(-)
arch/x86/include/asm/io.h +2 -1
@@ -257,10 +257,11 @@ static inline void slow_down_io(void)#endif +extern struct static_key_false sev_enable_key; +#ifdef CONFIG_AMD_MEM_ENCRYPT #include <linux/jump_label.h> -extern struct static_key_false sev_enable_key;static inline bool sev_key_active (void ) {return static_branch_unlikely (&sev_enable_key);
arch/x86/kernel/pci-swiotlb.c +1 -1
@@ -49,7 +49,7 @@ int __init pci_swiotlb_detect_4gb(void)* buffers are allocated and used for devices that do not support * the addressing range required for the encryption mask. */ -if (sme_active ()) +if (sme_active () ||is_tdx_guest ()) swiotlb =1 ;return swiotlb;
arch/x86/kernel/tdx.c +3 -0
@@ -8,6 +8,7 @@#include <asm/vmx.h> #include <asm/insn.h> #include <linux/sched/signal.h> /* force_sig_fault() */ +#include <linux/swiotlb.h> #include <linux/cpu.h>
@@ -470,6 +471,8 @@ void __init tdx_early_init(void) legacy_pic = &null_legacy_pic; + swiotlb_force =SWIOTLB_FORCE ; +cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"tdg:cpu_hotplug" ,NULL , tdg_cpu_offline_prepare);
arch/x86/mm/mem_encrypt.c +0 -45
48 lines not on this page · raw message
arch/x86/mm/mem_encrypt_common.c +47 -0
56 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC 1/1 x86/kvm: Use bounce buffers for TD guest
From: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to directly access guest private memory. Any memory that is required for communication with VMM must be shared explicitly. The same rule applies for any any DMA to and fromTDX guest. All DMA pages had to marked as shared pages. A generic way to achieve this without any changes to device drivers is to use the SWIOTLB framework.
This method of handling is similar to AMD SEV. So extend this support for TDX guest as well. Also since there are some common code between AMD SEV and TDX guest in mem_encrypt_init(), move it to mem_encrypt_common.c and call AMD specific init function from it
---
Changes since RFC v2:
* Fixed commit log as per review comments.
* Instead of moving all AMD related changes to mem_encrypt_common.c, created a AMD specific helper function amd_mem_encrypt_init() and called it from mem_encrypt_init().
* Removed redundant changes in arch/x86/kernel/pci-swiotlb.c.
arch/x86/include/asm/mem_encrypt_common.h | 2 ++ arch/x86/kernel/tdx.c | 3 +++ arch/x86/mm/mem_encrypt.c | 5 +---- arch/x86/mm/mem_encrypt_common.c | 16 ++++++++++++++++ 4 files changed, 22 insertions(+), 4 deletions(-)
arch/x86/include/asm/mem_encrypt_common.h +2 -0
@@ -8,11 +8,13 @@#ifdef CONFIG_AMD_MEM_ENCRYPT bool amd_force_dma_unencrypted (struct device *dev); +void __init amd_mem_encrypt_init (void );#else /* CONFIG_AMD_MEM_ENCRYPT */ static inline bool amd_force_dma_unencrypted (struct device *dev) {return false; } +static inline void amd_mem_encrypt_init (void ) {}#endif /* CONFIG_AMD_MEM_ENCRYPT */ #endif
arch/x86/kernel/tdx.c +3 -0
@@ -8,6 +8,7 @@#include <asm/vmx.h> #include <asm/insn.h> #include <linux/sched/signal.h> /* force_sig_fault() */ +#include <linux/swiotlb.h> #include <linux/cpu.h> #include <linux/protected_guest.h>
@@ -536,6 +537,8 @@ void __init tdx_early_init(void) legacy_pic = &null_legacy_pic; + swiotlb_force =SWIOTLB_FORCE ; +cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"tdg:cpu_hotplug" ,NULL , tdg_cpu_offline_prepare);
arch/x86/mm/mem_encrypt.c +1 -4
@@ -467,14 +467,11 @@ static void print_mem_encrypt_feature_info(void) }/* Architecture __weak replacement functions */ -void __init mem_encrypt_init (void ) +void __init amd_mem_encrypt_init (void ) {if (!sme_me_mask)return ; -/* Call into SWIOTLB to update the SWIOTLB DMA buffers */ -swiotlb_update_mem_attributes (); -/* * With SEV, we need to unroll the rep string I/O instructions, * but SEV-ES supports them through the #VC handler.
arch/x86/mm/mem_encrypt_common.c +16 -0
@@ -9,6 +9,7 @@#include <asm/mem_encrypt_common.h> #include <linux/dma-mapping.h> +#include <linux/swiotlb.h> /* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ bool force_dma_unencrypted (struct device *dev)
@@ -21,3 +22,18 @@ bool force_dma_unencrypted(struct device *dev)return false; } + +/* Architecture __weak replacement functions */ +void __init mem_encrypt_init (void ) +{ +/* +* For TDX guest or SEV/SME, call into SWIOTLB to update +* the SWIOTLB DMA buffers +*/ +if (sme_me_mask ||protected_guest_has (VM_MEM_ENCRYPT )) +swiotlb_update_mem_attributes (); + +if (sme_me_mask) +amd_mem_encrypt_init (); +} +
Signature · 1 line
2.25.1
RFC v2 32/32 x86/tdx: ioapic: Add shared bit for IOAPIC base address
In reply to Kuppuswamy SathyanarayananFrom: Isaku Yamahata <[email protected]>
IOAPIC is emulated by KVM which means its MMIO address is shared by host. Add shared bit for base address of IOAPIC. Most MMIO region is handled by ioremap which is already marked as shared for TDX guest platform, but IOAPIC is an exception which uses fixed map.
--- arch/x86/kernel/apic/io_apic.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kernel/apic/io_apic.c +10 -2
@@ -2675,6 +2675,14 @@ static struct resource * __init ioapic_setup_resources(void)return res; } +static void io_apic_set_fixmap_nocache (enum fixed_addresses idx,phys_addr_t phys) +{ +pgprot_t flags =FIXMAP_PAGE_NOCACHE ; +if (is_tdx_guest ()) + flags =pgprot_tdg_shared (flags); +__set_fixmap (idx, phys, flags); +} +void __init io_apic_init_mappings (void ) {unsigned long ioapic_phys, idx =FIX_IO_APIC_BASE_0 ;
@@ -2707,7 +2715,7 @@ void __init io_apic_init_mappings(void) __func__, PAGE_SIZE, PAGE_SIZE); ioapic_phys =__pa (ioapic_phys); } -set_fixmap_nocache (idx, ioapic_phys); +io_apic_set_fixmap_nocache (idx, ioapic_phys);apic_printk (APIC_VERBOSE ,"mapped IOAPIC to %08lx (%08lx)\n" ,__fix_to_virt (idx) + (ioapic_phys & ~PAGE_MASK ), ioapic_phys);
@@ -2836,7 +2844,7 @@ int mp_register_ioapic(int id, u32 address, u32 gsi_base, ioapics[idx].mp_config.flags =MPC_APIC_USABLE ; ioapics[idx].mp_config.apicaddr = address; -set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address); +io_apic_set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address);if (bad_ioapic_register (idx)) {clear_fixmap (FIX_IO_APIC_BASE_0 + idx);return -ENODEV ;
Signature · 1 line
2.25.1
Re: [RFC v2 32/32] x86/tdx: ioapic: Add shared bit for IOAPIC base address
Quoted 7 lines · Kuppuswamy Sathyanarayanan
From: Isaku Yamahata <[email protected]>
IOAPIC is emulated by KVM which means its MMIO address is shared by host. Add shared bit for base address of IOAPIC. Most MMIO region is handled by ioremap which is already marked as shared for TDX guest platform, but IOAPIC is an exception which uses fixed map.
Ho hum... I guess I'll rewrite the changelog:
The kernel interacts with each bare-metal IOAPIC with a special MMIO page. When running under KVM, the guest's IOAPICs are emulated by KVM.
When running as a TDX guest, the guest needs to mark each IOAPIC mapping as "shared" with the host. This ensures that TDX private protections are not applied to the page, which allows the TDX host emulation to work.
Earlier patches in this series modified ioremap() so that ioremap()-created mappings such as virtio will be marked as shared. However, the IOAPIC code does not use ioremap() and instead uses the fixmap mechanism.
Introduce a special fixmap helper just for the IOAPIC code. Ensure that it marks IOAPIC pages as "shared". This replaces set_fixmap_nocache() with __set_fixmap() since __set_fixmap() allows custom 'prot' values.
Quoted 8 lines
arch/x86/kernel/apic/io_apic.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)arch/x86/kernel/apic/io_apic.c +0
-0
arch/x86/kernel/apic/io_apic.c static struct resource * __init ioapic_setup_resources(void) · line 2684
Quoted 10 lines
arch/x86/kernel/apic/io_apic.c +7
-0@@ -2675,6 +2675,14 @@ static struct resource * __init ioapic_setup_resources(void)return res; } +static void io_apic_set_fixmap_nocache (enum fixed_addresses idx,phys_addr_t phys) +{ +pgprot_t flags =FIXMAP_PAGE_NOCACHE ; +if (is_tdx_guest ()) + flags =pgprot_tdg_shared (flags); +__set_fixmap (idx, phys, flags); +}
^ This seems like it could at least use a one-liner comment.
Quoted 22 lines
arch/x86/kernel/apic/io_apic.c +2
-2void __init io_apic_init_mappings (void ) {unsigned long ioapic_phys, idx =FIX_IO_APIC_BASE_0 ;@@ -2707,7 +2715,7 @@ void __init io_apic_init_mappings(void) __func__, PAGE_SIZE, PAGE_SIZE); ioapic_phys =__pa (ioapic_phys); } -set_fixmap_nocache (idx, ioapic_phys); +io_apic_set_fixmap_nocache (idx, ioapic_phys);apic_printk (APIC_VERBOSE ,"mapped IOAPIC to %08lx (%08lx)\n" ,__fix_to_virt (idx) + (ioapic_phys & ~PAGE_MASK ), ioapic_phys);@@ -2836,7 +2844,7 @@ int mp_register_ioapic(int id, u32 address, u32 gsi_base, ioapics[idx].mp_config.flags =MPC_APIC_USABLE ; ioapics[idx].mp_config.apicaddr = address; -set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address); +io_apic_set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address);if (bad_ioapic_register (idx)) {clear_fixmap (FIX_IO_APIC_BASE_0 + idx);return -ENODEV ;
RFC 1/1 x86/tdx: ioapic: Add shared bit for IOAPIC base address
From: Isaku Yamahata <[email protected]>
The kernel interacts with each bare-metal IOAPIC with a special MMIO page. When running under KVM, the guest's IOAPICs are emulated by KVM.
When running as a TDX guest, the guest needs to mark each IOAPIC mapping as "shared" with the host. This ensures that TDX private protections are not applied to the page, which allows the TDX host emulation to work.
Earlier patches in this series modified ioremap() so that ioremap()-created mappings such as virtio will be marked as shared. However, the IOAPIC code does not use ioremap() and instead uses the fixmap mechanism.
Introduce a special fixmap helper just for the IOAPIC code. Ensure that it marks IOAPIC pages as "shared". This replaces set_fixmap_nocache() with __set_fixmap() since __set_fixmap() allows custom 'prot' values.
---
Changes since RFC v2:
* Fixed commit log and comment as per review comments.
arch/x86/kernel/apic/io_apic.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
arch/x86/kernel/apic/io_apic.c +14 -2
@@ -2675,6 +2675,18 @@ static struct resource * __init ioapic_setup_resources(void)return res; } +static void io_apic_set_fixmap_nocache (enum fixed_addresses idx, +phys_addr_t phys) +{ +pgprot_t flags =FIXMAP_PAGE_NOCACHE ; + +/* Set TDX guest shared bit in pgprot flags */ +if (is_tdx_guest ()) + flags =pgprot_tdg_shared (flags); + +__set_fixmap (idx, phys, flags); +} +void __init io_apic_init_mappings (void ) {unsigned long ioapic_phys, idx =FIX_IO_APIC_BASE_0 ;
@@ -2707,7 +2719,7 @@ void __init io_apic_init_mappings(void) __func__, PAGE_SIZE, PAGE_SIZE); ioapic_phys =__pa (ioapic_phys); } -set_fixmap_nocache (idx, ioapic_phys); +io_apic_set_fixmap_nocache (idx, ioapic_phys);apic_printk (APIC_VERBOSE ,"mapped IOAPIC to %08lx (%08lx)\n" ,__fix_to_virt (idx) + (ioapic_phys & ~PAGE_MASK ), ioapic_phys);
@@ -2836,7 +2848,7 @@ int mp_register_ioapic(int id, u32 address, u32 gsi_base, ioapics[idx].mp_config.flags =MPC_APIC_USABLE ; ioapics[idx].mp_config.apicaddr = address; -set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address); +io_apic_set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address);if (bad_ioapic_register (idx)) {clear_fixmap (FIX_IO_APIC_BASE_0 + idx);return -ENODEV ;
Signature · 1 line
2.25.1
RFC 1/1 x86/tdx: ioapic: Add shared bit for IOAPIC base address
From: Isaku Yamahata <[email protected]>
The kernel interacts with each bare-metal IOAPIC with a special MMIO page. When running under KVM, the guest's IOAPICs are emulated by KVM.
When running as a TDX guest, the guest needs to mark each IOAPIC mapping as "shared" with the host. This ensures that TDX private protections are not applied to the page, which allows the TDX host emulation to work.
Earlier patches in this series modified ioremap() so that ioremap()-created mappings such as virtio will be marked as shared. However, the IOAPIC code does not use ioremap() and instead uses the fixmap mechanism.
Introduce a special fixmap helper just for the IOAPIC code. Ensure that it marks IOAPIC pages as "shared". This replaces set_fixmap_nocache() with __set_fixmap() since __set_fixmap() allows custom 'prot' values.
---
Changes since RFC v2-fix-v2:
* Replaced is_tdx_guest() call with protected_guest_has() call.
* Used pgprot_protected_guest() instead of prot_tdg_shared().
Changes since RFC v2:
* Fixed commit log and comment as per review comment
arch/x86/kernel/apic/io_apic.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
arch/x86/kernel/apic/io_apic.c +15 -2
@@ -49,6 +49,7 @@#include <linux/slab.h> #include <linux/memblock.h> #include <linux/msi.h> +#include <linux/protected_guest.h> #include <asm/irqdomain.h> #include <asm/io.h>
@@ -2675,6 +2676,18 @@ static struct resource * __init ioapic_setup_resources(void)return res; } +static void io_apic_set_fixmap_nocache (enum fixed_addresses idx, +phys_addr_t phys) +{ +pgprot_t flags =FIXMAP_PAGE_NOCACHE ; + +/* Set TDX guest shared bit in pgprot flags */ +if (protected_guest_has (VM_SHARED_MAPPING_INIT )) + flags =pgprot_protected_guest (flags); + +__set_fixmap (idx, phys, flags); +} +void __init io_apic_init_mappings (void ) {unsigned long ioapic_phys, idx =FIX_IO_APIC_BASE_0 ;
@@ -2707,7 +2720,7 @@ void __init io_apic_init_mappings(void) __func__, PAGE_SIZE, PAGE_SIZE); ioapic_phys =__pa (ioapic_phys); } -set_fixmap_nocache (idx, ioapic_phys); +io_apic_set_fixmap_nocache (idx, ioapic_phys);apic_printk (APIC_VERBOSE ,"mapped IOAPIC to %08lx (%08lx)\n" ,__fix_to_virt (idx) + (ioapic_phys & ~PAGE_MASK ), ioapic_phys);
@@ -2836,7 +2849,7 @@ int mp_register_ioapic(int id, u32 address, u32 gsi_base, ioapics[idx].mp_config.flags =MPC_APIC_USABLE ; ioapics[idx].mp_config.apicaddr = address; -set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address); +io_apic_set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address);if (bad_ioapic_register (idx)) {clear_fixmap (FIX_IO_APIC_BASE_0 + idx);return -ENODEV ;
Signature · 1 line
2.25.1
kernel test robot
build botRe: [RFC v2-fix-v3 1/1] x86/tdx: ioapic: Add shared bit for IOAPIC base address
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Kuppuswamy,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on tip/x86/core]
[also build test ERROR on tip/master linux/master linus/master v5.13-rc4 next-20210528] [cannot apply to luto/next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Kuppuswamy-Sathyanarayanan/x86-tdx-ioapic-Add-shared-bit-for-IOAPIC-base-address/20210601-102809 base: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git 2c88d45edbb89029c1190bb3b136d2602f057c98 config: x86_64-randconfig-a002-20210601 (attached as .config) compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project db26cd30b6dd65e88d786e97a1e453af5cd48966)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/837965b9dfc7504b80b412e75288d2558e7c36b7
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Kuppuswamy-Sathyanarayanan/x86-tdx-ioapic-Add-shared-bit-for-IOAPIC-base-address/20210601-102809 git checkout 837965b9dfc7504b80b412e75288d2558e7c36b7
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
>> arch/x86/kernel/apic/io_apic.c:52:10: fatal error: 'linux/protected_guest.h' file not found
#include <linux/protected_guest.h>
^~~~~~~~~~~~~~~~~~~~~~~~~1 error generated.
vim +52 arch/x86/kernel/apic/io_apic.c
> 52 #include <linux/protected_guest.h>
53 ---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Hi Peter/Andy,
Quoted 1 line · Kuppuswamy Sathyanarayanan
Hi All,
Just a gentle ping. Please let me know your comments on this patch set. I hope it addressed concerns raised by you in RFC v1.
Quoted 164 lines
NOTE: This series is not ready for wide public review. It is being specifically posted so that Peter Z and other experts on the entry code can look for problems with the new exception handler (#VE). That's also why x86@ is not being spammed.
Intel's Trust Domain Extensions (TDX) protect guest VMs from malicious hosts and some physical attacks. This series adds the bare-minimum support to run a TDX guest. The host-side support will be submitted separately. Also support for advanced TD guest features like attestation or debug-mode will be submitted separately. Also, at this point it is not secure with some known holes in drivers, and also hasn’t been fully audited and fuzzed yet.
TDX has a lot of similarities to SEV. It enhances confidentiality and of guest memory and state (like registers) and includes a new exception (#VE) for the same basic reasons as SEV-ES. Like SEV-SNP (not merged yet), TDX limits the host's ability to effect changes in the guest physical address space.
In contrast to the SEV code in the kernel, TDX guest memory is integrity protected and isolated; the host is prevented from accessing guest memory (even ciphertext).
The TDX architecture also includes a new CPU mode called Secure-Arbitration Mode (SEAM). The software (TDX module) running in this mode arbitrates interactions between host and guest and implements many of the guarantees of the TDX architecture.
Some of the key differences between TD and regular VM is,
1. Multi CPU bring-up is done using the ACPI MADT wake-up table.
2. A new #VE exception handler is added. The TDX module injects #VE exception to the guest TD in cases of instructions that need to be emulated, disallowed MSR accesses, subset of CPUID leaves, etc.
3. By default memory is marked as private, and TD will selectively share it with VMM based on need.
4. Remote attestation is supported to enable a third party (either the owner of the workload or a user of the services provided by the workload) to establish that the workload is running on an Intel-TDX-enabled platform located within a TD prior to providing that workload data.You can find TDX related documents in the following link.
https://software.intel.com/content/www/br/pt/develop/articles/intel-trust-domain-extensions.html
Changes since v1:
* Implementedtdcall()andtdvmcall()helper functions in assembly and renamed them as__tdcall()and__tdvmcall().
* Addeddo_general_protection()helper function to re-use protection code between #GP exception and TDX #VE exception handlers.
* Addressed syscall gap issue in #VE handler support (for details check the commit log in "x86/traps: Add #VE support for TDX guest").
* Modified patch titled "x86/tdx: Handle port I/O" to re-use commontdvmcall()helper function.
* Added error handling support to MADT CPU wakeup code.
* Introduced enumtdx_map_typeto identify SHARED vs PRIVATE memory type.
* Enabled shared memory in IOAPIC driver.
* Added BINUTILS version info for TDCALL.
* Changed the TDVMCALL vendor id from 0 to "TDX.KVM".
* ReplacedWARN()withpr_warn_ratelimited()in__tdvmcall()wrappers.
* Fixed commit log and code comments related review comments.
* Renamed patch titled # "x86/topology: Disable CPU hotplug support for TDX platforms" to "x86/topology: Disable CPU online/offline control for TDX guest"
* Rebased on top of v5.12 kernel.Erik Kaneda (1):
ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup StructureIsaku Yamahata (1):
x86/tdx: ioapic: Add shared bit for IOAPIC base addressKirill A. Shutemov (16):
x86/paravirt: IntroduceCONFIG_PARAVIRT_XLx86/tdx: Get TD execution environment information via TDINFO x86/traps: Add #VE support for TDX guest
x86/tdx: Add HLT support for TDX guest
x86/tdx: Wire up KVM hypercalls
x86/tdx: Add MSR support for TDX guest
x86/tdx: Handle CPUID via #VE
x86/io: Allow to overrideinX()andoutX()implementation x86/tdx: Handle port I/O
x86/tdx: Handle in-kernel MMIO
x86/mm: Moveforce_dma_unencrypted()to common code x86/tdx: Exclude Shared bit from__PHYSICAL_MASKx86/tdx: Make pages shared inioremap()
x86/tdx: Add helper to do MapGPA TDVMALL
x86/tdx: Make DMA pages shared
x86/kvm: Use bounce buffers for TD guestKuppuswamy Sathyanarayanan (10):
x86/tdx: IntroduceINTEL_TDX_GUESTconfig option x86/cpufeatures: Add TDX Guest CPU feature
x86/x86: Addis_tdx_guest()interface
x86/tdx: Add__tdcall()and__tdvmcall()helper functions x86/traps: Add do_general_protection() helper function x86/tdx: Handle MWAIT, MONITOR and WBINVD
ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup Mailbox Structure ACPI/table: Print MADT Wake table information
x86/acpi, x86/boot: Add multiprocessor wake-up support x86/topology: Disable CPU online/offline control for TDX guestSean Christopherson (4):
x86/boot: Add a trampoline for APs booting in 64-bit mode x86/boot: Avoid #VE during compressed boot for TDX platforms x86/boot: Avoid unnecessary #VE during boot process x86/tdx: Forcefully disable legacy PIC for TDX guestsarch/x86/Kconfig | 28 +- arch/x86/boot/compressed/Makefile | 2 + arch/x86/boot/compressed/head_64.S | 10 +- arch/x86/boot/compressed/misc.h | 1 + arch/x86/boot/compressed/pgtable.h | 2 +- arch/x86/boot/compressed/tdcall.S | 9 + arch/x86/boot/compressed/tdx.c | 32 ++ arch/x86/include/asm/apic.h | 3 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/idtentry.h | 4 + arch/x86/include/asm/io.h | 24 +- arch/x86/include/asm/irqflags.h | 38 +- arch/x86/include/asm/kvm_para.h | 21 + arch/x86/include/asm/paravirt.h | 22 +- arch/x86/include/asm/paravirt_types.h | 3 +- arch/x86/include/asm/pgtable.h | 3 + arch/x86/include/asm/realmode.h | 1 + arch/x86/include/asm/tdx.h | 176 +++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/acpi/boot.c | 79 ++++ arch/x86/kernel/apic/apic.c | 8 + arch/x86/kernel/apic/io_apic.c | 12 +- arch/x86/kernel/asm-offsets.c | 22 ++ arch/x86/kernel/head64.c | 3 + arch/x86/kernel/head_64.S | 13 +- arch/x86/kernel/idt.c | 6 + arch/x86/kernel/paravirt.c | 4 +- arch/x86/kernel/pci-swiotlb.c | 2 +- arch/x86/kernel/smpboot.c | 5 + arch/x86/kernel/tdcall.S | 361 +++++++++++++++++ arch/x86/kernel/tdx-kvm.c | 45 +++ arch/x86/kernel/tdx.c | 480 +++++++++++++++++++++++ arch/x86/kernel/topology.c | 3 +- arch/x86/kernel/traps.c | 81 ++-- arch/x86/mm/Makefile | 2 + arch/x86/mm/ioremap.c | 8 +- arch/x86/mm/mem_encrypt.c | 75 ---- arch/x86/mm/mem_encrypt_common.c | 85 ++++ arch/x86/mm/mem_encrypt_identity.c | 1 + arch/x86/mm/pat/set_memory.c | 48 ++- arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 49 ++- arch/x86/realmode/rm/trampoline_common.S | 5 +- drivers/acpi/tables.c | 11 + include/acpi/actbl2.h | 26 +- 45 files changed, 1654 insertions(+), 162 deletions(-)create mode 100644
arch/x86/boot/compressed/tdcall.Screate mode 100644arch/x86/boot/compressed/tdx.ccreate mode 100644arch/x86/include/asm/tdx.hcreate mode 100644arch/x86/kernel/tdcall.Screate mode 100644arch/x86/kernel/tdx-kvm.ccreate mode 100644arch/x86/kernel/tdx.ccreate mode 100644arch/x86/mm/mem_encrypt_common.c
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer