Skip to contentSkip to conversation

Introduce support for guest CET feature

PATCH Superseded Yang Weijiang 2 participants 17 messages Mar 20, 2020

Subsystems (3)

v10 of 15 Superseded Latest is v15 →

lore
Share & tools
lore
#0 lore raw

v10 cover Introduce support for guest CET feature

Control-flow Enforcement Technology (CET) provides protection against Return/Jump-Oriented Programming (ROP/JOP) attack. It includes two
sub-features: Shadow Stack (SHSTK) and Indirect Branch Tracking (IBT).

KVM needs to update to enable guest CET feature.
This patchset implements CET related CPUID/XSAVES enumeration, MSRs and vmentry/vmexit configuration etc.so that guest kernel can setup CET runtime infrastructure based on them. Some CET MSRs and related feature flags used reference the definitions in kernel patchset.

CET kernel patches are here:
https://lkml.org/lkml/2020/2/5/593
https://lkml.org/lkml/2020/2/5/604

v9 -> v10
- Refactored code per Sean's review feedback.
- Added CET support for nested VM.
- Removed fix-patch for CPUID(0xd,N) enumeration as this part is done by Paolo and Sean.
- This new patchset is based on Paolo's queued cpu_caps branch.
- Modified patch per XSAVES related change.
- Consolidated KVM unit-test patch with KVM patches.

v8 -> v9:
- Refactored msr-check functions per Sean's feedback.
- Fixed a few issues per Sean's suggestion.
- Rebased patch to kernel-v5.4.
- Moved CET CPUID feature bits and CR4.CET to last patch.

v7 -> v8:
- Addressed Jim and Sean's feedback on: 1) CPUID(0xD,i) enumeration. 2) sanity check when configure guest CET. 3) function improvement.
- Added more sanity check functions.
- Set host vmexit default status so that guest won't leak CET status to host when vmexit.
- Added CR0.WP vs. CR4.CET mutual constrains.

v6 -> v7:
- Rebased patch to kernel v5.3
- Sean suggested to change CPUID(0xd, n) enumeration code as alined with existing one, and I think it's better to make the fix as an independent patch since XSS MSR are being used widely on X86 platforms.
- Check more host and guest status before configure guest CET
per Sean's feedback.
- Add error-check before guest accesses CET MSRs per Sean's feedback.
- Other minor fixes suggested by Sean.

v5 -> v6:
- Rebase patch to kernel v5.2.
- Move CPUID(0xD, n>=1) helper to a seperate patch.
- Merge xsave size fix with other patch.
- Other minor fixes per community feedback.

v4 -> v5:
- Rebase patch to kernel v5.1.
- Wrap CPUID(0xD, n>=1) code to a helper function.
- Pass through MSR_IA32_PL1_SSP and MSR_IA32_PL2_SSP to Guest.
- Add Co-developed-by expression in patch description.
- Refine patch description.

v3 -> v4:
- Add Sean's patch for loading Guest fpu state before access XSAVES managed CET MSRs.
- Melt down CET bits setting into CPUID configuration patch.
- Add VMX interface to query Host XSS.
- Check Host and Guest XSS support bits before set Guest XSS.
- Make Guest SHSTK and IBT feature enabling independent.
- Do not report CET support to Guest when Host CET feature is Disabled.

v2 -> v3:
- Modified patches to make Guest CET independent to Host enabling.
- Added patch 8 to add user space access for Guest CET MSR access.
- Modified code comments and patch description to reflect changes.

v1 -> v2:
- Re-ordered patch sequence, combined one patch.
- Added more description for CET related VMCS fields.
- Added Host CET capability check while enabling Guest CET loading bit.
- Added Host CET capability check while reporting Guest CPUID(EAX=7, EXC=0).
- Modified code in reporting Guest CPUID(EAX=D,ECX>=1), make it clearer.
- Added Host and Guest XSS mask check while setting bits for Guest XSS.

Sean Christopherson (1):
KVM: X86: Load guest fpu state when accessing MSRs managed by XSAVES

Yang Weijiang (7):
KVM: VMX: Introduce CET VMX fields and flags
KVM: VMX: Set up guest CET MSRs per KVM and host configuration
KVM: VMX: Load CET states on vmentry/vmexit
KVM: X86: Refresh CPUID on guest XSS change
KVM: X86: Add userspace access interface for CET MSRs
KVM: VMX: Enable CET support for nested VM
KVM: X86: Set CET feature bits for CPUID enumeration

 arch/x86/include/asm/kvm_host.h |   3 +-
 arch/x86/include/asm/vmx.h      |   8 ++
 arch/x86/include/uapi/asm/kvm.h |   1 +
 arch/x86/kvm/cpuid.c            |  22 +++-
 arch/x86/kvm/vmx/capabilities.h |  10 ++
 arch/x86/kvm/vmx/nested.c       |  41 ++++++-
 arch/x86/kvm/vmx/vmcs12.c       |   6 +
 arch/x86/kvm/vmx/vmcs12.h       |  14 ++-
 arch/x86/kvm/vmx/vmx.c          | 202 +++++++++++++++++++++++++++++++-
 arch/x86/kvm/x86.c              |  44 ++++++-
 arch/x86/kvm/x86.h              |   2 +-
 11 files changed, 341 insertions(+), 12 deletions(-)
Signature · 1 line
2.17.2
#1 lore raw

PATCH v10 1/8 KVM: VMX: Introduce CET VMX fields and flags

CET(Control-flow Enforcement Technology) is a CPU feature used to prevent Return/Jump-Oriented Programming(ROP/JOP) attacks. It provides the following sub-features to defend against ROP/JOP style control-flow subversion attacks:

Shadow Stack (SHSTK):
A second stack for program which is used exclusively for control transfer operations.

Indirect Branch Tracking (IBT):
Code branching protection to defend against jump/call oriented programming.

Several new CET MSRs are defined in kernel to support CET:
MSR_IA32_{U,S}_CET: Controls the CET settings for user
mode and suervisor mode respectively.

MSR_IA32_PL{0,1,2,3}_SSP: Stores shadow stack pointers for

                            CPL-0,1,2,3 level respectively.

MSR_IA32_INT_SSP_TAB: Stores base address of shadow stack
pointer table.

Two XSAVES state bits are introduced for CET:
IA32_XSS:[bit 11]: For saving/restoring user mode CET states IA32_XSS:[bit 12]: For saving/restoring supervisor mode CET states.

Six VMCS fields are introduced for CET:
{HOST,GUEST}_S_CET: Stores CET settings for supervisor mode. {HOST,GUEST}_SSP: Stores shadow stack pointer for supervisor mode. {HOST,GUEST}_INTR_SSP_TABLE: Stores base address of shadow stack pointer
table.

If VM_EXIT_LOAD_HOST_CET_STATE = 1, the host's CET MSRs are restored from below VMCS fields at VM-Exit:
HOST_S_CET
HOST_SSP
HOST_INTR_SSP_TABLE

If VM_ENTRY_LOAD_GUEST_CET_STATE = 1, the guest's CET MSRs are loaded from below VMCS fields at VM-Entry:
GUEST_S_CET
GUEST_SSP
GUEST_INTR_SSP_TABLE

---
 arch/x86/include/asm/vmx.h      | 8 ++++++++
 arch/x86/include/uapi/asm/kvm.h | 1 +
 arch/x86/kvm/x86.c              | 4 ++++
 arch/x86/kvm/x86.h              | 2 +-
 4 files changed, 14 insertions(+), 1 deletion(-)
arch/x86/include/asm/vmx.h +8 -0
index 5e090d1f03f8..e938bc6c37aa 100644
@@ -94,6 +94,7 @@
 #define VM_EXIT_CLEAR_BNDCFGS                   0x00800000
 #define VM_EXIT_PT_CONCEAL_PIP			0x01000000
 #define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
+#define VM_EXIT_LOAD_HOST_CET_STATE             0x10000000
 
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
@@ -107,6 +108,7 @@
 #define VM_ENTRY_LOAD_BNDCFGS                   0x00010000
 #define VM_ENTRY_PT_CONCEAL_PIP			0x00020000
 #define VM_ENTRY_LOAD_IA32_RTIT_CTL		0x00040000
+#define VM_ENTRY_LOAD_GUEST_CET_STATE           0x00100000
 
 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR	0x000011ff
 
@@ -328,6 +330,9 @@ enum vmcs_field {
 	GUEST_PENDING_DBG_EXCEPTIONS    = 0x00006822,
 	GUEST_SYSENTER_ESP              = 0x00006824,
 	GUEST_SYSENTER_EIP              = 0x00006826,
+	GUEST_S_CET                     = 0x00006828,
+	GUEST_SSP                       = 0x0000682a,
+	GUEST_INTR_SSP_TABLE            = 0x0000682c,
 	HOST_CR0                        = 0x00006c00,
 	HOST_CR3                        = 0x00006c02,
 	HOST_CR4                        = 0x00006c04,
@@ -340,6 +345,9 @@ enum vmcs_field {
 	HOST_IA32_SYSENTER_EIP          = 0x00006c12,
 	HOST_RSP                        = 0x00006c14,
 	HOST_RIP                        = 0x00006c16,
+	HOST_S_CET                      = 0x00006c18,
+	HOST_SSP                        = 0x00006c1a,
+	HOST_INTR_SSP_TABLE             = 0x00006c1c
 };
 
 /*
arch/x86/include/uapi/asm/kvm.h +1 -0
index 3f3f780c8c65..78e5c4266270 100644
@@ -31,6 +31,7 @@
 #define MC_VECTOR 18
 #define XM_VECTOR 19
 #define VE_VECTOR 20
+#define CP_VECTOR 21
 
 /* Select x86 specific features in <linux/kvm.h> */
 #define __KVM_HAVE_PIT
arch/x86/kvm/x86.c +4 -0
index 40c6768942ae..830afe5038d1 100644
@@ -186,6 +186,9 @@ static struct kvm_shared_msrs __percpu *shared_msrs;
 				| XFEATURE_MASK_BNDCSR | XFEATURE_MASK_AVX512 \
 				| XFEATURE_MASK_PKRU)
 
+#define KVM_SUPPORTED_XSS	(XFEATURE_MASK_CET_USER | \
+				 XFEATURE_MASK_CET_KERNEL)
+
 u64 __read_mostly host_efer;
 EXPORT_SYMBOL_GPL(host_efer);
 
@@ -402,6 +405,7 @@ static int exception_class(int vector)
 	case NP_VECTOR:
 	case SS_VECTOR:
 	case GP_VECTOR:
+	case CP_VECTOR:
 		return EXCPT_CONTRIBUTORY;
 	default:
 		break;
arch/x86/kvm/x86.h +1 -1
index c1954e216b41..8f0baa6fa72f 100644
@@ -115,7 +115,7 @@ static inline bool x86_exception_has_error_code(unsigned int vector)
 {
 	static u32 exception_has_error_code = BIT(DF_VECTOR) | BIT(TS_VECTOR) |
 			BIT(NP_VECTOR) | BIT(SS_VECTOR) | BIT(GP_VECTOR) |
-			BIT(PF_VECTOR) | BIT(AC_VECTOR);
+			BIT(PF_VECTOR) | BIT(AC_VECTOR) | BIT(CP_VECTOR);
 
 	return (1U << vector) & exception_has_error_code;
 }
Signature · 1 line
2.17.2
#2 lore raw

PATCH v10 2/8 KVM: VMX: Set up guest CET MSRs per KVM and host configuration

In reply to Yang Weijiang

CET MSRs pass through guest directly to enhance performance. CET runtime control settings are stored in MSR_IA32_{U,S}_CET, Shadow Stack Pointer(SSP) are stored in MSR_IA32_PL{0,1,2,3}_SSP, SSP table base address is stored in MSR_IA32_INT_SSP_TAB, these MSRs are defined in kernel and re-used here.

MSR_IA32_U_CET and MSR_IA32_PL3_SSP are used for user-mode protection, the MSR contents are switched between threads during scheduling, it makes sense to pass through them so that the guest kernel can use xsaves/xrstors to operate them efficiently. Other MSRs are used for non-user mode protection. See SDM for detailed info.

The difference between CET VMCS fields and CET MSRs is that,the former are used during VMEnter/VMExit, whereas the latter are used for CET state storage between task/thread scheduling.

---
 arch/x86/kvm/vmx/vmx.c | 43 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
arch/x86/kvm/vmx/vmx.c +43 -0
index 860e5f4a9f7b..61d2a4bf9eb6 100644
@@ -3033,6 +3033,19 @@ void vmx_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
 		vmcs_writel(GUEST_CR3, guest_cr3);
 }
 
+static bool is_cet_mode_allowed(struct kvm_vcpu *vcpu, u32 mode_mask)
+{
+	return ((supported_xss & mode_mask) &&
+		(guest_cpuid_has(vcpu, X86_FEATURE_SHSTK) ||
+		guest_cpuid_has(vcpu, X86_FEATURE_IBT)));
+}
+
+static bool is_cet_supported(struct kvm_vcpu *vcpu)
+{
+	return is_cet_mode_allowed(vcpu, XFEATURE_MASK_CET_USER |
+				   XFEATURE_MASK_CET_KERNEL);
+}
+
 int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
@@ -7064,6 +7077,35 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu)
 		vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i * 4));
 }
 
+static void vmx_update_intercept_for_cet_msr(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	unsigned long *msr_bitmap = vmx->vmcs01.msr_bitmap;
+	bool flag;
+
+	flag = !is_cet_mode_allowed(vcpu, XFEATURE_MASK_CET_USER);
+	/*
+	 * U_CET is required for USER CET, and U_CET, PL3_SPP are bound as
+	 * one component and controlled by IA32_XSS[bit 11].
+	 */
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_U_CET, MSR_TYPE_RW, flag);
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL3_SSP, MSR_TYPE_RW, flag);
+
+	flag = !is_cet_mode_allowed(vcpu, XFEATURE_MASK_CET_KERNEL);
+	/*
+	 * S_CET is required for KERNEL CET, and PL0_SSP ... PL2_SSP are
+	 * bound as one component and controlled by IA32_XSS[bit 12].
+	 */
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_S_CET, MSR_TYPE_RW, flag);
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL0_SSP, MSR_TYPE_RW, flag);
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL1_SSP, MSR_TYPE_RW, flag);
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL2_SSP, MSR_TYPE_RW, flag);
+
+	flag |= !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK);
+	/* SSP_TAB is only available for KERNEL SHSTK.*/
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_INT_SSP_TAB, MSR_TYPE_RW, flag);
+}
+
 static void vmx_cpuid_update(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
@@ -7102,6 +7144,7 @@ static void vmx_cpuid_update(struct kvm_vcpu *vcpu)
 			vmx_set_guest_msr(vmx, msr, enabled ? 0 : TSX_CTRL_RTM_DISABLE);
 		}
 	}
+	vmx_update_intercept_for_cet_msr(vcpu);
 }
 
 static __init void vmx_set_cpu_caps(void)
Signature · 1 line
2.17.2
#3 lore raw

PATCH v10 3/8 KVM: VMX: Load CET states on vmentry/vmexit

In reply to Yang Weijiang

"Load {guest,host} CET state" bit controls whether guest/host CET states will be loaded at VM entry/exit.
There're mutual constrains between CR0.WP and CR4.CET, so need to check the dependent bit while changing the control registers.

Note:
1)The processor does not allow CR4.CET to be set if CR0.WP = 0,
similarly, it does not allow CR0.WP to be cleared while CR4.CET = 1. In either case, KVM would inject #GP to guest.

2)SHSTK and IBT features share one control MSR:
MSR_IA32_{U,S}_CET, which means it's difficult to hide one feature from another in the case of SHSTK != IBT, after discussed in community, it's agreed to allow Guest control two features independently as it won't introduce security hole.

---
 arch/x86/kvm/vmx/capabilities.h | 10 ++++++++++
 arch/x86/kvm/vmx/vmx.c          | 25 +++++++++++++++++++++++--
 arch/x86/kvm/x86.c              |  3 +++
 3 files changed, 36 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/capabilities.h +10 -0
index 8903475f751e..565340352260 100644
@@ -107,6 +107,16 @@ static inline bool cpu_has_vmx_mpx(void)
 		(vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_BNDCFGS);
 }
 
+static inline bool cpu_has_cet_guest_load_ctrl(void)
+{
+	return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_GUEST_CET_STATE);
+}
+
+static inline bool cpu_has_cet_host_load_ctrl(void)
+{
+	return (vmcs_config.vmexit_ctrl & VM_EXIT_LOAD_HOST_CET_STATE);
+}
+
 static inline bool cpu_has_vmx_tpr_shadow(void)
 {
 	return vmcs_config.cpu_based_exec_ctrl & CPU_BASED_TPR_SHADOW;
arch/x86/kvm/vmx/vmx.c +23 -2
index 61d2a4bf9eb6..e7ac776c808f 100644
@@ -44,6 +44,7 @@
 #include <asm/spec-ctrl.h>
 #include <asm/virtext.h>
 #include <asm/vmx.h>
+#include <asm/cet.h>
 
 #include "capabilities.h"
 #include "cpuid.h"
@@ -2456,7 +2457,8 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 	      VM_EXIT_LOAD_IA32_EFER |
 	      VM_EXIT_CLEAR_BNDCFGS |
 	      VM_EXIT_PT_CONCEAL_PIP |
-	      VM_EXIT_CLEAR_IA32_RTIT_CTL;
+	      VM_EXIT_CLEAR_IA32_RTIT_CTL |
+	      VM_EXIT_LOAD_HOST_CET_STATE;
 	if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_EXIT_CTLS,
 				&_vmexit_control) < 0)
 		return -EIO;
@@ -2480,7 +2482,8 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 	      VM_ENTRY_LOAD_IA32_EFER |
 	      VM_ENTRY_LOAD_BNDCFGS |
 	      VM_ENTRY_PT_CONCEAL_PIP |
-	      VM_ENTRY_LOAD_IA32_RTIT_CTL;
+	      VM_ENTRY_LOAD_IA32_RTIT_CTL |
+	      VM_ENTRY_LOAD_GUEST_CET_STATE;
 	if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_ENTRY_CTLS,
 				&_vmentry_control) < 0)
 		return -EIO;
@@ -3086,6 +3089,10 @@ int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
 			return 1;
 	}
 
+	if ((cr4 & X86_CR4_CET) && (!is_cet_supported(vcpu) ||
+	    !(kvm_read_cr0(vcpu) & X86_CR0_WP)))
+		return 1;
+
 	if (vmx->nested.vmxon && !nested_cr4_valid(vcpu, cr4))
 		return 1;
 
@@ -3945,6 +3952,12 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx)
 
 	if (cpu_has_load_ia32_efer())
 		vmcs_write64(HOST_IA32_EFER, host_efer);
+
+	if (cpu_has_cet_host_load_ctrl()) {
+		vmcs_writel(HOST_S_CET, 0);
+		vmcs_writel(HOST_INTR_SSP_TABLE, 0);
+		vmcs_writel(HOST_SSP, 0);
+	}
 }
 
 void set_cr4_guest_host_mask(struct vcpu_vmx *vmx)
@@ -6541,6 +6554,14 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)
 		vmx->loaded_vmcs->host_state.cr3 = cr3;
 	}
 
+	if (cpu_has_cet_guest_load_ctrl() && is_cet_supported(vcpu))
+		vmcs_set_bits(VM_ENTRY_CONTROLS,
+			      VM_ENTRY_LOAD_GUEST_CET_STATE);
+
+	if (cpu_has_cet_host_load_ctrl() && is_cet_supported(vcpu))
+		vmcs_set_bits(VM_EXIT_CONTROLS,
+			      VM_EXIT_LOAD_HOST_CET_STATE);
+
 	cr4 = cr4_read_shadow();
 	if (unlikely(cr4 != vmx->loaded_vmcs->host_state.cr4)) {
 		vmcs_writel(HOST_CR4, cr4);
arch/x86/kvm/x86.c +3 -0
index 830afe5038d1..90acdbbb8a5a 100644
@@ -804,6 +804,9 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)
 	if (!(cr0 & X86_CR0_PG) && kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE))
 		return 1;
 
+	if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET))
+		return 1;
+
 	kvm_x86_ops->set_cr0(vcpu, cr0);
 
 	if ((cr0 ^ old_cr0) & X86_CR0_PG) {
Signature · 1 line
2.17.2
#4 lore raw

Re: [PATCH v10 3/8] KVM: VMX: Load CET states on vmentry/vmexit

Thank you for the patch! Yet something to improve:

Hi Yang,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on kvm/linux-next]
[also build test ERROR on next-20200319]
[cannot apply to vhost/linux-next tip/auto-latest linux/master linus/master v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
config: i386-allyesconfig (attached as .config)
compiler: gcc-7 (Debian 7.5.0-5) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag

All errors (new ones prefixed by >>):

arch/x86/kvm/x86.c: In function 'kvm_set_cr0':

>> arch/x86/kvm/x86.c:807:53: error: 'X86_CR4_CET' undeclared (first use in this function); did you mean 'X86_CR4_DE'?
     if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET))
                                                        ^~~~~~~~~~~
                                                        X86_CR4_DE
   arch/x86/kvm/x86.c:807:53: note: each undeclared identifier is reported only once for each function it appears in
--
>> arch/x86/kvm/vmx/vmx.c:47:10: fatal error: asm/cet.h: No such file or directory
    #include <asm/cet.h>
             ^~~~~~~~~~~

compilation terminated.

vim +807 arch/x86/kvm/x86.c

778
779 cr0 &= ~CR0_RESERVED_BITS;
780
781 if ((cr0 & X86_CR0_NW) && !(cr0 & X86_CR0_CD))
782 return 1;
783
784 if ((cr0 & X86_CR0_PG) && !(cr0 & X86_CR0_PE))
785 return 1;
786
787 if (!is_paging(vcpu) && (cr0 & X86_CR0_PG)) {
788 #ifdef CONFIG_X86_64
789 if ((vcpu->arch.efer & EFER_LME)) {
790 int cs_db, cs_l;
791
792 if (!is_pae(vcpu))
793 return 1;
794 kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l);
795 if (cs_l)
796 return 1;
797 } else
798 #endif
799 if (is_pae(vcpu) && !load_pdptrs(vcpu, vcpu->arch.walk_mmu,
800 kvm_read_cr3(vcpu)))
801 return 1;
802 }
803
804 if (!(cr0 & X86_CR0_PG) && kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE))
805 return 1;
806

 > 807		if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET))

808 return 1;
809
810 kvm_x86_ops->set_cr0(vcpu, cr0);
811
812 if ((cr0 ^ old_cr0) & X86_CR0_PG) {
813 kvm_clear_async_pf_completion_queue(vcpu);
814 kvm_async_pf_hash_reset(vcpu);
815 }
816
817 if ((cr0 ^ old_cr0) & update_bits)
818 kvm_mmu_reset_context(vcpu);
819
820 if (((cr0 ^ old_cr0) & X86_CR0_CD) &&
821 kvm_arch_has_noncoherent_dma(vcpu->kvm) &&
822 !kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_CD_NW_CLEARED))
823 kvm_zap_gfn_range(vcpu->kvm, 0, ~0ULL);
824
825 return 0;
826 }
827 EXPORT_SYMBOL_GPL(kvm_set_cr0);
828

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#5 lore raw

Re: [PATCH v10 3/8] KVM: VMX: Load CET states on vmentry/vmexit

In reply to Yang Weijiang
Thank you for the patch! Yet something to improve:

Hi Yang,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on kvm/linux-next]
[also build test ERROR on next-20200319]
[cannot apply to vhost/linux-next tip/auto-latest linux/master linus/master v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
config: x86_64-rhel (attached as .config)
compiler: gcc-7 (Debian 7.5.0-5) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

If you fix the issue, kindly add following tag

Note: the linux-review/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517 HEAD 8410cecc9c65bb83a88a1bd5d46ea7ebdd947367 builds fine.
It only hurts bisectibility.

All errors (new ones prefixed by >>):

arch/x86/kvm/x86.c: In function 'kvm_set_cr0':

>> arch/x86/kvm/x86.c:807:53: error: 'X86_CR4_CET' undeclared (first use in this function); did you mean 'X86_CR0_ET'?
     if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET))
                                                        ^~~~~~~~~~~
                                                        X86_CR0_ET

arch/x86/kvm/x86.c:807:53: note: each undeclared identifier is reported only once for each function it appears in

vim +807 arch/x86/kvm/x86.c

778
779 cr0 &= ~CR0_RESERVED_BITS;
780
781 if ((cr0 & X86_CR0_NW) && !(cr0 & X86_CR0_CD))
782 return 1;
783
784 if ((cr0 & X86_CR0_PG) && !(cr0 & X86_CR0_PE))
785 return 1;
786
787 if (!is_paging(vcpu) && (cr0 & X86_CR0_PG)) {
788 #ifdef CONFIG_X86_64
789 if ((vcpu->arch.efer & EFER_LME)) {
790 int cs_db, cs_l;
791
792 if (!is_pae(vcpu))
793 return 1;
794 kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l);
795 if (cs_l)
796 return 1;
797 } else
798 #endif
799 if (is_pae(vcpu) && !load_pdptrs(vcpu, vcpu->arch.walk_mmu,
800 kvm_read_cr3(vcpu)))
801 return 1;
802 }
803
804 if (!(cr0 & X86_CR0_PG) && kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE))
805 return 1;
806

 > 807		if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET))

808 return 1;
809
810 kvm_x86_ops->set_cr0(vcpu, cr0);
811
812 if ((cr0 ^ old_cr0) & X86_CR0_PG) {
813 kvm_clear_async_pf_completion_queue(vcpu);
814 kvm_async_pf_hash_reset(vcpu);
815 }
816
817 if ((cr0 ^ old_cr0) & update_bits)
818 kvm_mmu_reset_context(vcpu);
819
820 if (((cr0 ^ old_cr0) & X86_CR0_CD) &&
821 kvm_arch_has_noncoherent_dma(vcpu->kvm) &&
822 !kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_CD_NW_CLEARED))
823 kvm_zap_gfn_range(vcpu->kvm, 0, ~0ULL);
824
825 return 0;
826 }
827 EXPORT_SYMBOL_GPL(kvm_set_cr0);
828

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#6 lore raw

PATCH v10 4/8 KVM: X86: Refresh CPUID on guest XSS change

In reply to Yang Weijiang

CPUID(0xd, 1) reports the current required storage size of XCR0 | XSS, when guest updates the XSS, it's necessary to update the CPUID leaf, otherwise guest will fetch stale state, this result into some WARNs during guest running.

---
 arch/x86/kvm/cpuid.c | 18 +++++++++++++++---
 arch/x86/kvm/x86.c   |  7 ++++++-
 2 files changed, 21 insertions(+), 4 deletions(-)
arch/x86/kvm/cpuid.c +15 -3
index 78d461be2102..71703d9277ee 100644
@@ -95,9 +95,21 @@ int kvm_update_cpuid(struct kvm_vcpu *vcpu)
 	}
 
 	best = kvm_find_cpuid_entry(vcpu, 0xD, 1);
-	if (best && (cpuid_entry_has(best, X86_FEATURE_XSAVES) ||
-		     cpuid_entry_has(best, X86_FEATURE_XSAVEC)))
-		best->ebx = xstate_required_size(vcpu->arch.xcr0, true);
+	if (best) {
+		if (best->eax & (F(XSAVES) | F(XSAVEC))) {
+			u64 xstate = vcpu->arch.xcr0 | vcpu->arch.ia32_xss;
+
+			best->ebx = xstate_required_size(xstate, true);
+		}
+
+		if (best->eax & F(XSAVES)) {
+			supported_xss &= best->ecx | ((u64)best->edx << 32);
+		} else {
+			best->ecx = 0;
+			best->edx = 0;
+			supported_xss = 0;
+		}
+	}
 
 	/*
 	 * The existing code assumes virtual address is 48-bit or 57-bit in the
arch/x86/kvm/x86.c +6 -1
index 90acdbbb8a5a..5be6fad6e08d 100644
@@ -2838,7 +2838,10 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		 */
 		if (data & ~supported_xss)
 			return 1;
-		vcpu->arch.ia32_xss = data;
+		if (vcpu->arch.ia32_xss != data) {
+			vcpu->arch.ia32_xss = data;
+			kvm_update_cpuid(vcpu);
+		}
 		break;
 	case MSR_SMI_COUNT:
 		if (!msr_info->host_initiated)
@@ -9635,6 +9638,8 @@ int kvm_arch_hardware_setup(void)
 
 	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
 		supported_xss = 0;
+	else
+		supported_xss = host_xss & KVM_SUPPORTED_XSS;
 
 	cr4_reserved_bits = kvm_host_cr4_reserved_bits(&boot_cpu_data);
 
Signature · 1 line
2.17.2
#7 lore raw

Re: [PATCH v10 4/8] KVM: X86: Refresh CPUID on guest XSS change

Thank you for the patch! Yet something to improve:

Hi Yang,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on kvm/linux-next]
[also build test ERROR on next-20200319]
[cannot apply to vhost/linux-next tip/auto-latest linux/master linus/master v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
config: i386-allyesconfig (attached as .config)
compiler: gcc-7 (Debian 7.5.0-5) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag

All error/warnings (new ones prefixed by >>):

arch/x86/kvm/x86.c: In function 'kvm_set_cr0':
arch/x86/kvm/x86.c:807:53: error: 'X86_CR4_CET' undeclared (first use in this function); did you mean 'X86_CR4_DE'?

     if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET))
                                                        ^~~~~~~~~~~
                                                        X86_CR4_DE
   arch/x86/kvm/x86.c:807:53: note: each undeclared identifier is reported only once for each function it appears in
   arch/x86/kvm/x86.c: In function 'kvm_arch_hardware_setup':
>> arch/x86/kvm/x86.c:189:28: error: 'XFEATURE_MASK_CET_USER' undeclared (first use in this function); did you mean 'XFEATURE_MASK_BNDCSR'?
    #define KVM_SUPPORTED_XSS (XFEATURE_MASK_CET_USER | \
                               ^
>> arch/x86/kvm/x86.c:9650:30: note: in expansion of macro 'KVM_SUPPORTED_XSS'
      supported_xss = host_xss & KVM_SUPPORTED_XSS;
                                 ^~~~~~~~~~~~~~~~~
>> arch/x86/kvm/x86.c:190:6: error: 'XFEATURE_MASK_CET_KERNEL' undeclared (first use in this function); did you mean 'XFEATURE_MASK_CET_USER'?
         XFEATURE_MASK_CET_KERNEL)
         ^
>> arch/x86/kvm/x86.c:9650:30: note: in expansion of macro 'KVM_SUPPORTED_XSS'
      supported_xss = host_xss & KVM_SUPPORTED_XSS;
                                 ^~~~~~~~~~~~~~~~~

vim +189 arch/x86/kvm/x86.c

18863bdd60f895 Avi Kivity          2009-09-07  183  
cfc481810c903a Sean Christopherson 2020-03-02  184  #define KVM_SUPPORTED_XCR0     (XFEATURE_MASK_FP | XFEATURE_MASK_SSE \
cfc481810c903a Sean Christopherson 2020-03-02  185  				| XFEATURE_MASK_YMM | XFEATURE_MASK_BNDREGS \
cfc481810c903a Sean Christopherson 2020-03-02  186  				| XFEATURE_MASK_BNDCSR | XFEATURE_MASK_AVX512 \
cfc481810c903a Sean Christopherson 2020-03-02  187  				| XFEATURE_MASK_PKRU)
cfc481810c903a Sean Christopherson 2020-03-02  188  
4d0c7566d56266 Yang Weijiang       2020-03-20 @189  #define KVM_SUPPORTED_XSS	(XFEATURE_MASK_CET_USER | \
4d0c7566d56266 Yang Weijiang       2020-03-20 @190  				 XFEATURE_MASK_CET_KERNEL)
4d0c7566d56266 Yang Weijiang       2020-03-20  191  

:::::: The code at line 189 was first introduced by commit
:::::: 4d0c7566d56266fc80759b1780e660edbb6f2fa8 KVM: VMX: Introduce CET VMX fields and flags

:::::: TO: Yang Weijiang <[email protected]> :::::: CC: 0day robot <[email protected]>

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#8 lore raw

PATCH v10 5/8 KVM: X86: Load guest fpu state when accessing MSRs managed by XSAVES

In reply to Yang Weijiang

From: Sean Christopherson <[email protected]>

A handful of CET MSRs are not context switched through "traditional" methods, e.g. VMCS or manual switching, but rather are passed through to the guest and are saved and restored by XSAVES/XRSTORS, i.e. in the guest's FPU state.

Load the guest's FPU state if userspace is accessing MSRs whose values are managed by XSAVES so that the MSR helper, e.g. vmx_{get,set}_msr(), can simply do {RD,WR}MSR to access the guest's value.

Note that guest_cpuid_has() is not queried as host userspace is allowed to access MSRs that have not been exposed to the guest, e.g. it might do KVM_SET_MSRS prior to KVM_SET_CPUID2.

---
 arch/x86/kvm/x86.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +18 -1
index 5be6fad6e08d..99e5b1df8555 100644
@@ -109,6 +109,8 @@ static void enter_smm(struct kvm_vcpu *vcpu);
 static void __kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags);
 static void store_regs(struct kvm_vcpu *vcpu);
 static int sync_regs(struct kvm_vcpu *vcpu);
+static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu);
+static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu);
 
 struct kvm_x86_ops *kvm_x86_ops __read_mostly;
 EXPORT_SYMBOL_GPL(kvm_x86_ops);
@@ -3249,6 +3251,12 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 }
 EXPORT_SYMBOL_GPL(kvm_get_msr_common);
 
+static bool is_xsaves_msr(u32 index)
+{
+	return index == MSR_IA32_U_CET ||
+	       (index >= MSR_IA32_PL0_SSP && index <= MSR_IA32_PL3_SSP);
+}
+
 /*
  * Read or write a bunch of msrs. All parameters are kernel addresses.
  *
@@ -3259,11 +3267,20 @@ static int __msr_io(struct kvm_vcpu *vcpu, struct kvm_msrs *msrs,
 		    int (*do_msr)(struct kvm_vcpu *vcpu,
 				  unsigned index, u64 *data))
 {
+	bool fpu_loaded = false;
 	int i;
 
-	for (i = 0; i < msrs->nmsrs; ++i)
+	for (i = 0; i < msrs->nmsrs; ++i) {
+		if (vcpu && !fpu_loaded && supported_xss &&
+		    is_xsaves_msr(entries[i].index)) {
+			kvm_load_guest_fpu(vcpu);
+			fpu_loaded = true;
+		}
 		if (do_msr(vcpu, entries[i].index, &entries[i].data))
 			break;
+	}
+	if (fpu_loaded)
+		kvm_put_guest_fpu(vcpu);
 
 	return i;
 }
Signature · 1 line
2.17.2
#9 lore raw

PATCH v10 6/8 KVM: X86: Add userspace access interface for CET MSRs

In reply to Yang Weijiang

There're two different places storing Guest CET states, states managed with XSAVES/XRSTORS, as restored/saved
in previous patch, can be read/write directly from/to the MSRs. For those stored in VMCS fields, they're access via vmcs_read/ vmcs_write.

To correctly read/write the CET MSRs, it's necessary to check whether the kernel FPU context switch happened and reload guest FPU context if needed.

---
 arch/x86/kvm/vmx/vmx.c | 133 +++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/x86.c     |  11 ++++
 2 files changed, 144 insertions(+)
arch/x86/kvm/vmx/vmx.c +133 -0
index e7ac776c808f..2654bd099fe6 100644
@@ -1809,6 +1809,91 @@ static int vmx_get_msr_feature(struct kvm_msr_entry *msr)
 	}
 }
 
+static void vmx_get_xsave_msr(struct msr_data *msr_info)
+{
+	local_irq_disable();
+	if (test_thread_flag(TIF_NEED_FPU_LOAD))
+		switch_fpu_return();
+	rdmsrl(msr_info->index, msr_info->data);
+	local_irq_enable();
+}
+
+static void vmx_set_xsave_msr(struct msr_data *msr_info)
+{
+	local_irq_disable();
+	if (test_thread_flag(TIF_NEED_FPU_LOAD))
+		switch_fpu_return();
+	wrmsrl(msr_info->index, msr_info->data);
+	local_irq_enable();
+}
+
+#define CET_MSR_RSVD_BITS_1  GENMASK(1, 0)
+#define CET_MSR_RSVD_BITS_2  GENMASK(9, 6)
+
+static bool cet_check_msr_write(struct kvm_vcpu *vcpu,
+				struct msr_data *msr,
+				u64 mask)
+{
+	u64 data = msr->data;
+	u32 high_word = data >> 32;
+
+	if (data & mask)
+		return false;
+
+	if (!is_64_bit_mode(vcpu) && high_word)
+		return false;
+
+	return true;
+}
+
+static bool cet_check_ssp_msr_access(struct kvm_vcpu *vcpu,
+				     struct msr_data *msr)
+{
+	u32 index = msr->index;
+
+	if (!boot_cpu_has(X86_FEATURE_SHSTK))
+		return false;
+
+	if (!msr->host_initiated &&
+	    !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK))
+		return false;
+
+	if (index == MSR_IA32_INT_SSP_TAB)
+		return true;
+
+	if (index == MSR_IA32_PL3_SSP) {
+		if (!(supported_xss & XFEATURE_MASK_CET_USER))
+			return false;
+	} else if (!(supported_xss & XFEATURE_MASK_CET_KERNEL)) {
+		return false;
+	}
+
+	return true;
+}
+
+static bool cet_check_ctl_msr_access(struct kvm_vcpu *vcpu,
+				     struct msr_data *msr)
+{
+	u32 index = msr->index;
+
+	if (!boot_cpu_has(X86_FEATURE_SHSTK) &&
+	    !boot_cpu_has(X86_FEATURE_IBT))
+		return false;
+
+	if (!msr->host_initiated &&
+	    !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK) &&
+	    !guest_cpuid_has(vcpu, X86_FEATURE_IBT))
+		return false;
+
+	if (index == MSR_IA32_U_CET) {
+		if (!(supported_xss & XFEATURE_MASK_CET_USER))
+			return false;
+	} else if (!(supported_xss & XFEATURE_MASK_CET_KERNEL)) {
+		return false;
+	}
+
+	return true;
+}
 /*
  * Reads an msr value (of 'msr_index') into 'pdata'.
  * Returns 0 on success, non-0 otherwise.
@@ -1941,6 +2026,26 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		else
 			msr_info->data = vmx->pt_desc.guest.addr_a[index / 2];
 		break;
+	case MSR_IA32_S_CET:
+		if (!cet_check_ctl_msr_access(vcpu, msr_info))
+			return 1;
+		msr_info->data = vmcs_readl(GUEST_S_CET);
+		break;
+	case MSR_IA32_INT_SSP_TAB:
+		if (!cet_check_ssp_msr_access(vcpu, msr_info))
+			return 1;
+		msr_info->data = vmcs_readl(GUEST_INTR_SSP_TABLE);
+		break;
+	case MSR_IA32_U_CET:
+		if (!cet_check_ctl_msr_access(vcpu, msr_info))
+			return 1;
+		vmx_get_xsave_msr(msr_info);
+		break;
+	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
+		if (!cet_check_ssp_msr_access(vcpu, msr_info))
+			return 1;
+		vmx_get_xsave_msr(msr_info);
+		break;
 	case MSR_TSC_AUX:
 		if (!msr_info->host_initiated &&
 		    !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP))
@@ -2197,6 +2302,34 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		else
 			vmx->pt_desc.guest.addr_a[index / 2] = data;
 		break;
+	case MSR_IA32_S_CET:
+		if (!cet_check_ctl_msr_access(vcpu, msr_info))
+			return 1;
+		if (!cet_check_msr_write(vcpu, msr_info, CET_MSR_RSVD_BITS_2))
+			return 1;
+		vmcs_writel(GUEST_S_CET, data);
+		break;
+	case MSR_IA32_INT_SSP_TAB:
+		if (!cet_check_ctl_msr_access(vcpu, msr_info))
+			return 1;
+		if (!is_64_bit_mode(vcpu))
+			return 1;
+		vmcs_writel(GUEST_INTR_SSP_TABLE, data);
+		break;
+	case MSR_IA32_U_CET:
+		if (!cet_check_ctl_msr_access(vcpu, msr_info))
+			return 1;
+		if (!cet_check_msr_write(vcpu, msr_info, CET_MSR_RSVD_BITS_2))
+			return 1;
+		vmx_set_xsave_msr(msr_info);
+		break;
+	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
+		if (!cet_check_ssp_msr_access(vcpu, msr_info))
+			return 1;
+		if (!cet_check_msr_write(vcpu, msr_info, CET_MSR_RSVD_BITS_1))
+			return 1;
+		vmx_set_xsave_msr(msr_info);
+		break;
 	case MSR_TSC_AUX:
 		if (!msr_info->host_initiated &&
 		    !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP))
arch/x86/kvm/x86.c +11 -0
index 99e5b1df8555..3d5049faac58 100644
@@ -1229,6 +1229,10 @@ static const u32 msrs_to_save_all[] = {
 	MSR_ARCH_PERFMON_EVENTSEL0 + 12, MSR_ARCH_PERFMON_EVENTSEL0 + 13,
 	MSR_ARCH_PERFMON_EVENTSEL0 + 14, MSR_ARCH_PERFMON_EVENTSEL0 + 15,
 	MSR_ARCH_PERFMON_EVENTSEL0 + 16, MSR_ARCH_PERFMON_EVENTSEL0 + 17,
+
+	MSR_IA32_XSS, MSR_IA32_U_CET, MSR_IA32_S_CET,
+	MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP,
+	MSR_IA32_PL3_SSP, MSR_IA32_INT_SSP_TAB,
 };
 
 static u32 msrs_to_save[ARRAY_SIZE(msrs_to_save_all)];
@@ -1504,6 +1508,13 @@ static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data,
 		 * invokes 64-bit SYSENTER.
 		 */
 		data = get_canonical(data, vcpu_virt_addr_bits(vcpu));
+		break;
+	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
+	case MSR_IA32_U_CET:
+	case MSR_IA32_S_CET:
+	case MSR_IA32_INT_SSP_TAB:
+		if (is_noncanonical_address(data, vcpu))
+			return 1;
 	}
 
 	msr.data = data;
Signature · 1 line
2.17.2
#10 lore raw

Re: [PATCH v10 6/8] KVM: X86: Add userspace access interface for CET MSRs

Thank you for the patch! Yet something to improve:

Hi Yang,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on kvm/linux-next]
[also build test ERROR on next-20200319]
[cannot apply to vhost/linux-next tip/auto-latest linux/master linus/master v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
config: i386-allyesconfig (attached as .config)
compiler: gcc-7 (Debian 7.5.0-5) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag

All errors (new ones prefixed by >>):

arch/x86/kvm/x86.c: In function 'kvm_set_cr0':
arch/x86/kvm/x86.c:809:53: error: 'X86_CR4_CET' undeclared (first use in this function); did you mean 'X86_CR4_DE'?

     if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET))
                                                        ^~~~~~~~~~~
                                                        X86_CR4_DE
   arch/x86/kvm/x86.c:809:53: note: each undeclared identifier is reported only once for each function it appears in
   arch/x86/kvm/x86.c: At top level:
>> arch/x86/kvm/x86.c:1233:16: error: 'MSR_IA32_U_CET' undeclared here (not in a function); did you mean 'MSR_IA32_TSC'?
     MSR_IA32_XSS, MSR_IA32_U_CET, MSR_IA32_S_CET,
                   ^~~~~~~~~~~~~~
                   MSR_IA32_TSC
>> arch/x86/kvm/x86.c:1233:32: error: 'MSR_IA32_S_CET' undeclared here (not in a function); did you mean 'MSR_IA32_U_CET'?
     MSR_IA32_XSS, MSR_IA32_U_CET, MSR_IA32_S_CET,
                                   ^~~~~~~~~~~~~~
                                   MSR_IA32_U_CET
>> arch/x86/kvm/x86.c:1234:2: error: 'MSR_IA32_PL0_SSP' undeclared here (not in a function); did you mean 'MSR_IA32_MCG_ESP'?
     MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP,
     ^~~~~~~~~~~~~~~~
     MSR_IA32_MCG_ESP
>> arch/x86/kvm/x86.c:1234:20: error: 'MSR_IA32_PL1_SSP' undeclared here (not in a function); did you mean 'MSR_IA32_PL0_SSP'?
     MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP,
                       ^~~~~~~~~~~~~~~~
                       MSR_IA32_PL0_SSP
>> arch/x86/kvm/x86.c:1234:38: error: 'MSR_IA32_PL2_SSP' undeclared here (not in a function); did you mean 'MSR_IA32_PL1_SSP'?
     MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP,
                                         ^~~~~~~~~~~~~~~~
                                         MSR_IA32_PL1_SSP
>> arch/x86/kvm/x86.c:1235:2: error: 'MSR_IA32_PL3_SSP' undeclared here (not in a function); did you mean 'MSR_IA32_PL2_SSP'?
     MSR_IA32_PL3_SSP, MSR_IA32_INT_SSP_TAB,
     ^~~~~~~~~~~~~~~~
     MSR_IA32_PL2_SSP
>> arch/x86/kvm/x86.c:1235:20: error: 'MSR_IA32_INT_SSP_TAB' undeclared here (not in a function); did you mean 'MSR_IA32_PL3_SSP'?
     MSR_IA32_PL3_SSP, MSR_IA32_INT_SSP_TAB,
                       ^~~~~~~~~~~~~~~~~~~~
                       MSR_IA32_PL3_SSP
   arch/x86/kvm/x86.c: In function 'is_xsaves_msr':
   arch/x86/kvm/x86.c:3267:15: error: comparison between pointer and integer [-Werror]
     return index == MSR_IA32_U_CET ||
                  ^~
   arch/x86/kvm/x86.c:3268:16: error: comparison between pointer and integer [-Werror]
            (index >= MSR_IA32_PL0_SSP && index <= MSR_IA32_PL3_SSP);
                   ^~
   arch/x86/kvm/x86.c:3268:45: error: comparison between pointer and integer [-Werror]
            (index >= MSR_IA32_PL0_SSP && index <= MSR_IA32_PL3_SSP);
                                                ^~
   arch/x86/kvm/x86.c: In function 'kvm_arch_hardware_setup':
   arch/x86/kvm/x86.c:191:28: error: 'XFEATURE_MASK_CET_USER' undeclared (first use in this function); did you mean 'XFEATURE_MASK_BNDCSR'?
    #define KVM_SUPPORTED_XSS (XFEATURE_MASK_CET_USER | \
                               ^
   arch/x86/kvm/x86.c:9678:30: note: in expansion of macro 'KVM_SUPPORTED_XSS'
      supported_xss = host_xss & KVM_SUPPORTED_XSS;
                                 ^~~~~~~~~~~~~~~~~
   arch/x86/kvm/x86.c:192:6: error: 'XFEATURE_MASK_CET_KERNEL' undeclared (first use in this function); did you mean 'XFEATURE_MASK_CET_USER'?
         XFEATURE_MASK_CET_KERNEL)
         ^
   arch/x86/kvm/x86.c:9678:30: note: in expansion of macro 'KVM_SUPPORTED_XSS'
      supported_xss = host_xss & KVM_SUPPORTED_XSS;
                                 ^~~~~~~~~~~~~~~~~
>> arch/x86/kvm/x86.c:191:51: error: invalid operands to binary | (have 'const u32 * {aka const unsigned int *}' and 'const u32 * {aka const unsigned int *}')
    #define KVM_SUPPORTED_XSS (XFEATURE_MASK_CET_USER | \
                               ~                      ^
   arch/x86/kvm/x86.c:9678:30: note: in expansion of macro 'KVM_SUPPORTED_XSS'
      supported_xss = host_xss & KVM_SUPPORTED_XSS;
                                 ^~~~~~~~~~~~~~~~~
>> arch/x86/kvm/x86.c:9678:28: error: invalid operands to binary & (have 'u64 {aka long long unsigned int}' and 'const u32 * {aka const unsigned int *}')
      supported_xss = host_xss & KVM_SUPPORTED_XSS;
                               ^
>> arch/x86/kvm/x86.c:9678:17: error: assignment makes integer from pointer without a cast [-Werror=int-conversion]
      supported_xss = host_xss & KVM_SUPPORTED_XSS;
                    ^

cc1: all warnings being treated as errors

vim +1233 arch/x86/kvm/x86.c

1180
1181 /*
1182 * List of msr numbers which we expose to userspace through KVM_GET_MSRS
1183 * and KVM_SET_MSRS, and KVM_GET_MSR_INDEX_LIST.
1184 *
1185 * The three MSR lists(msrs_to_save, emulated_msrs, msr_based_features)
1186 * extract the supported MSRs from the related const lists.
1187 * msrs_to_save is selected from the msrs_to_save_all to reflect the
1188 * capabilities of the host cpu. This capabilities test skips MSRs that are
1189 * kvm-specific. Those are put in emulated_msrs_all; filtering of emulated_msrs
1190 * may depend on host virtualization features rather than host cpu features.
1191 */
1192
1193 static const u32 msrs_to_save_all[] = {
1194 MSR_IA32_SYSENTER_CS, MSR_IA32_SYSENTER_ESP, MSR_IA32_SYSENTER_EIP,
1195 MSR_STAR,
1196 #ifdef CONFIG_X86_64
1197 MSR_CSTAR, MSR_KERNEL_GS_BASE, MSR_SYSCALL_MASK, MSR_LSTAR,
1198 #endif
1199 MSR_IA32_TSC, MSR_IA32_CR_PAT, MSR_VM_HSAVE_PA,
1200 MSR_IA32_FEAT_CTL, MSR_IA32_BNDCFGS, MSR_TSC_AUX,
1201 MSR_IA32_SPEC_CTRL,
1202 MSR_IA32_RTIT_CTL, MSR_IA32_RTIT_STATUS, MSR_IA32_RTIT_CR3_MATCH,
1203 MSR_IA32_RTIT_OUTPUT_BASE, MSR_IA32_RTIT_OUTPUT_MASK,
1204 MSR_IA32_RTIT_ADDR0_A, MSR_IA32_RTIT_ADDR0_B,
1205 MSR_IA32_RTIT_ADDR1_A, MSR_IA32_RTIT_ADDR1_B,
1206 MSR_IA32_RTIT_ADDR2_A, MSR_IA32_RTIT_ADDR2_B,
1207 MSR_IA32_RTIT_ADDR3_A, MSR_IA32_RTIT_ADDR3_B,
1208 MSR_IA32_UMWAIT_CONTROL,
1209
1210 MSR_ARCH_PERFMON_FIXED_CTR0, MSR_ARCH_PERFMON_FIXED_CTR1,
1211 MSR_ARCH_PERFMON_FIXED_CTR0 + 2, MSR_ARCH_PERFMON_FIXED_CTR0 + 3,
1212 MSR_CORE_PERF_FIXED_CTR_CTRL, MSR_CORE_PERF_GLOBAL_STATUS,
1213 MSR_CORE_PERF_GLOBAL_CTRL, MSR_CORE_PERF_GLOBAL_OVF_CTRL,
1214 MSR_ARCH_PERFMON_PERFCTR0, MSR_ARCH_PERFMON_PERFCTR1,
1215 MSR_ARCH_PERFMON_PERFCTR0 + 2, MSR_ARCH_PERFMON_PERFCTR0 + 3,
1216 MSR_ARCH_PERFMON_PERFCTR0 + 4, MSR_ARCH_PERFMON_PERFCTR0 + 5,
1217 MSR_ARCH_PERFMON_PERFCTR0 + 6, MSR_ARCH_PERFMON_PERFCTR0 + 7,
1218 MSR_ARCH_PERFMON_PERFCTR0 + 8, MSR_ARCH_PERFMON_PERFCTR0 + 9,
1219 MSR_ARCH_PERFMON_PERFCTR0 + 10, MSR_ARCH_PERFMON_PERFCTR0 + 11,
1220 MSR_ARCH_PERFMON_PERFCTR0 + 12, MSR_ARCH_PERFMON_PERFCTR0 + 13,
1221 MSR_ARCH_PERFMON_PERFCTR0 + 14, MSR_ARCH_PERFMON_PERFCTR0 + 15,
1222 MSR_ARCH_PERFMON_PERFCTR0 + 16, MSR_ARCH_PERFMON_PERFCTR0 + 17,
1223 MSR_ARCH_PERFMON_EVENTSEL0, MSR_ARCH_PERFMON_EVENTSEL1,
1224 MSR_ARCH_PERFMON_EVENTSEL0 + 2, MSR_ARCH_PERFMON_EVENTSEL0 + 3,
1225 MSR_ARCH_PERFMON_EVENTSEL0 + 4, MSR_ARCH_PERFMON_EVENTSEL0 + 5,
1226 MSR_ARCH_PERFMON_EVENTSEL0 + 6, MSR_ARCH_PERFMON_EVENTSEL0 + 7,
1227 MSR_ARCH_PERFMON_EVENTSEL0 + 8, MSR_ARCH_PERFMON_EVENTSEL0 + 9,
1228 MSR_ARCH_PERFMON_EVENTSEL0 + 10, MSR_ARCH_PERFMON_EVENTSEL0 + 11,
1229 MSR_ARCH_PERFMON_EVENTSEL0 + 12, MSR_ARCH_PERFMON_EVENTSEL0 + 13,
1230 MSR_ARCH_PERFMON_EVENTSEL0 + 14, MSR_ARCH_PERFMON_EVENTSEL0 + 15,
1231 MSR_ARCH_PERFMON_EVENTSEL0 + 16, MSR_ARCH_PERFMON_EVENTSEL0 + 17,
1232

> 1233		MSR_IA32_XSS, MSR_IA32_U_CET, MSR_IA32_S_CET,
> 1234		MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP,
> 1235		MSR_IA32_PL3_SSP, MSR_IA32_INT_SSP_TAB,

1236 };
1237

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#11 lore raw

Re: [PATCH v10 6/8] KVM: X86: Add userspace access interface for CET MSRs

In reply to Yang Weijiang
Thank you for the patch! Perhaps something to improve:

Hi Yang,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on kvm/linux-next]
[also build test WARNING on next-20200319]
[cannot apply to vhost/linux-next tip/auto-latest linux/master linus/master v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.1-181-g83789bbc-dirty
        make ARCH=x86_64 allmodconfig
        make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'

If you fix the issue, kindly add following tag

sparse warnings: (new ones prefixed by >>)

arch/x86/kvm/x86.c:809:60: sparse: sparse: undefined identifier 'X86_CR4_CET' arch/x86/kvm/x86.c:1233:23: sparse: sparse: undefined identifier 'MSR_IA32_U_CET' arch/x86/kvm/x86.c:1233:39: sparse: sparse: undefined identifier 'MSR_IA32_S_CET' arch/x86/kvm/x86.c:1234:9: sparse: sparse: undefined identifier 'MSR_IA32_PL0_SSP' arch/x86/kvm/x86.c:1234:27: sparse: sparse: undefined identifier 'MSR_IA32_PL1_SSP' arch/x86/kvm/x86.c:1234:45: sparse: sparse: undefined identifier 'MSR_IA32_PL2_SSP' arch/x86/kvm/x86.c:1235:9: sparse: sparse: undefined identifier 'MSR_IA32_PL3_SSP' arch/x86/kvm/x86.c:1235:27: sparse: sparse: undefined identifier 'MSR_IA32_INT_SSP_TAB' arch/x86/kvm/x86.c:1512:14: sparse: sparse: undefined identifier 'MSR_IA32_PL0_SSP' arch/x86/kvm/x86.c:1512:35: sparse: sparse: undefined identifier 'MSR_IA32_PL3_SSP' arch/x86/kvm/x86.c:1513:14: sparse: sparse: undefined identifier 'MSR_IA32_U_CET' arch/x86/kvm/x86.c:1514:14: sparse: sparse: undefined identifier 'MSR_IA32_S_CET' arch/x86/kvm/x86.c:1515:14: sparse: sparse: undefined identifier 'MSR_IA32_INT_SSP_TAB'

>> arch/x86/kvm/x86.c:1512:14: sparse: sparse: incompatible types for 'case' statement

arch/x86/kvm/x86.c:1512:35: sparse: sparse: incompatible types for 'case' statement arch/x86/kvm/x86.c:1513:14: sparse: sparse: incompatible types for 'case' statement arch/x86/kvm/x86.c:1514:14: sparse: sparse: incompatible types for 'case' statement arch/x86/kvm/x86.c:1515:14: sparse: sparse: incompatible types for 'case' statement
arch/x86/kvm/x86.c:2646:38: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected void const [noderef] <asn:1> * @@ got const [noderef] <asn:1> * @@
arch/x86/kvm/x86.c:2646:38: sparse: expected void const [noderef] <asn:1> *
arch/x86/kvm/x86.c:2646:38: sparse: got unsigned char [usertype] *
arch/x86/kvm/x86.c:3267:25: sparse: sparse: undefined identifier 'MSR_IA32_U_CET' arch/x86/kvm/x86.c:7549:15: sparse: sparse: incompatible types in comparison expression (different address spaces):
arch/x86/kvm/x86.c:7549:15: sparse: struct kvm_apic_map [noderef] <asn:4> *
arch/x86/kvm/x86.c:7549:15: sparse: struct kvm_apic_map *
arch/x86/kvm/x86.c:9678:44: sparse: sparse: undefined identifier 'XFEATURE_MASK_CET_USER' arch/x86/kvm/x86.c:9678:44: sparse: sparse: undefined identifier 'XFEATURE_MASK_CET_KERNEL' arch/x86/kvm/x86.c:9912:16: sparse: sparse: incompatible types in comparison expression (different address spaces):
arch/x86/kvm/x86.c:9912:16: sparse: struct kvm_apic_map [noderef] <asn:4> *
arch/x86/kvm/x86.c:9912:16: sparse: struct kvm_apic_map *
arch/x86/kvm/x86.c:9913:15: sparse: sparse: incompatible types in comparison expression (different address spaces):
arch/x86/kvm/x86.c:9913:15: sparse: struct kvm_pmu_event_filter [noderef] <asn:4> *
arch/x86/kvm/x86.c:9913:15: sparse: struct kvm_pmu_event_filter *
arch/x86/kvm/x86.c:1512:14: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1512:35: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1513:14: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1514:14: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1515:14: sparse: sparse: Expected constant expression in case statement

vim +/case +1512 arch/x86/kvm/x86.c

1475
1476 /*
1477 * Write @data into the MSR specified by @index. Select MSR specific fault
1478 * checks are bypassed if @host_initiated is %true.
1479 * Returns 0 on success, non-0 otherwise.
1480 * Assumes vcpu_load() was already called.
1481 */
1482 static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data,
1483 bool host_initiated)
1484 {
1485 struct msr_data msr;
1486
1487 switch (index) {
1488 case MSR_FS_BASE:
1489 case MSR_GS_BASE:
1490 case MSR_KERNEL_GS_BASE:
1491 case MSR_CSTAR:
1492 case MSR_LSTAR:
1493 if (is_noncanonical_address(data, vcpu))
1494 return 1;
1495 break;
1496 case MSR_IA32_SYSENTER_EIP:
1497 case MSR_IA32_SYSENTER_ESP:
1498 /*
1499 * IA32_SYSENTER_ESP and IA32_SYSENTER_EIP cause #GP if
1500 * non-canonical address is written on Intel but not on
1501 * AMD (which ignores the top 32-bits, because it does
1502 * not implement 64-bit SYSENTER).
1503 *
1504 * 64-bit code should hence be able to write a non-canonical
1505 * value on AMD. Making the address canonical ensures that
1506 * vmentry does not fail on Intel after writing a non-canonical
1507 * value, and that something deterministic happens if the guest
1508 * invokes 64-bit SYSENTER.
1509 */
1510 data = get_canonical(data, vcpu_virt_addr_bits(vcpu));
1511 break;

> 1512		case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:

1513 case MSR_IA32_U_CET:
1514 case MSR_IA32_S_CET:
1515 case MSR_IA32_INT_SSP_TAB:
1516 if (is_noncanonical_address(data, vcpu))
1517 return 1;
1518 }
1519
1520 msr.data = data;
1521 msr.index = index;
1522 msr.host_initiated = host_initiated;
1523
1524 return kvm_x86_ops->set_msr(vcpu, &msr);
1525 }
1526

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#12 lore raw

PATCH 1/1 x86: Add tests for user-mode CET

In reply to Yang Weijiang

This unit-test is intended to test user-mode CET support of KVM, it's tested on Intel new platform. Two CET features: Shadow Stack Protection(SHSTK) and Indirect-Branch Tracking(IBT) are enclosed.

In SHSTK test, if the function return-address in normal stack is tampered with a value not equal to the one on shadow-stack, #CP (Control Protection Exception)will generated on function returning. This feature is supported by processor itself, no compiler/link option is required.

However, to enabled IBT, we need to add -fcf-protection=full in compiler options, this makes the compiler insert endbr64 at the very beginning of each jmp/call target given the binary is for x86_64.

To get PASS results, the following conditions must be met:
1) The processor is powered with CET feature.
2) The kernel is patched with the latest CET kernel patches.
3) The KVM and QEMU are patched with the latest CET patches.
4) Use CET-enabled gcc to compile the test app.

---
 lib/x86/desc.c      |   2 +
 lib/x86/msr.h       |   2 +
 lib/x86/processor.h |   3 +
 x86/Makefile.common |   3 +-
 x86/Makefile.x86_64 |   3 +-
 x86/cet.c           | 212 ++++++++++++++++++++++++++++++++++++++++++++
 x86/unittests.cfg   |   6 ++
 7 files changed, 229 insertions(+), 2 deletions(-)
 create mode 100644 x86/cet.c
lib/x86/desc.c +2 -0
index 451f504..983d4d8 100644
@@ -179,6 +179,7 @@ EX(mf, 16);
 EX_E(ac, 17);
 EX(mc, 18);
 EX(xm, 19);
+EX_E(cp, 21);
 
 asm (".pushsection .text \n\t"
      "__handle_exception: \n\t"
@@ -224,6 +225,7 @@ static void *idt_handlers[32] = {
 	[17] = &ac_fault,
 	[18] = &mc_fault,
 	[19] = &xm_fault,
+	[21] = &cp_fault,
 };
 
 void setup_idt(void)
lib/x86/msr.h +2 -0
index 8dca964..98489e0 100644
@@ -208,6 +208,8 @@
 #define MSR_IA32_EBL_CR_POWERON		0x0000002a
 #define MSR_IA32_FEATURE_CONTROL        0x0000003a
 #define MSR_IA32_TSC_ADJUST		0x0000003b
+#define MSR_IA32_U_CET                  0x000006a0
+#define MSR_IA32_PL3_SSP                0x000006a7
 
 #define FEATURE_CONTROL_LOCKED				(1<<0)
 #define FEATURE_CONTROL_VMXON_ENABLED_INSIDE_SMX	(1<<1)
lib/x86/processor.h +3 -0
index 03fdf64..5763d62 100644
@@ -44,6 +44,7 @@
 #define X86_CR4_SMEP   0x00100000
 #define X86_CR4_SMAP   0x00200000
 #define X86_CR4_PKE    0x00400000
+#define X86_CR4_CET    0x00800000
 
 #define X86_EFLAGS_CF    0x00000001
 #define X86_EFLAGS_FIXED 0x00000002
@@ -149,8 +150,10 @@ static inline u8 cpuid_maxphyaddr(void)
 #define	X86_FEATURE_PKU			(CPUID(0x7, 0, ECX, 3))
 #define	X86_FEATURE_LA57		(CPUID(0x7, 0, ECX, 16))
 #define	X86_FEATURE_RDPID		(CPUID(0x7, 0, ECX, 22))
+#define	X86_FEATURE_SHSTK		(CPUID(0x7, 0, ECX, 7))
 #define	X86_FEATURE_SPEC_CTRL		(CPUID(0x7, 0, EDX, 26))
 #define	X86_FEATURE_ARCH_CAPABILITIES	(CPUID(0x7, 0, EDX, 29))
+#define	X86_FEATURE_IBT			(CPUID(0x7, 0, EDX, 20))
 #define	X86_FEATURE_NX			(CPUID(0x80000001, 0, EDX, 20))
 #define	X86_FEATURE_RDPRU		(CPUID(0x80000008, 0, EBX, 4))
 
x86/Makefile.common +2 -1
index ab67ca0..c5b4d2c 100644
@@ -58,7 +58,8 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \
                $(TEST_DIR)/init.flat $(TEST_DIR)/smap.flat \
                $(TEST_DIR)/hyperv_synic.flat $(TEST_DIR)/hyperv_stimer.flat \
                $(TEST_DIR)/hyperv_connections.flat \
-               $(TEST_DIR)/umip.flat $(TEST_DIR)/tsx-ctrl.flat
+               $(TEST_DIR)/umip.flat $(TEST_DIR)/tsx-ctrl.flat \
+               $(TEST_DIR)/cet.flat
 
 test_cases: $(tests-common) $(tests)
 
x86/Makefile.x86_64 +2 -1
index 010102b..43d9706 100644
@@ -1,7 +1,7 @@
 cstart.o = $(TEST_DIR)/cstart64.o
 bits = 64
 ldarch = elf64-x86-64
-COMMON_CFLAGS += -mno-red-zone -mno-sse -mno-sse2
+COMMON_CFLAGS += -mno-red-zone -mno-sse -mno-sse2 -fcf-protection=full
 
 cflatobjs += lib/x86/setjmp64.o
 cflatobjs += lib/x86/intel-iommu.o
@@ -20,6 +20,7 @@ tests += $(TEST_DIR)/tscdeadline_latency.flat
 tests += $(TEST_DIR)/intel-iommu.flat
 tests += $(TEST_DIR)/vmware_backdoors.flat
 tests += $(TEST_DIR)/rdpru.flat
+tests += $(TEST_DIR)/cet.flat
 
 include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
x86/cet.c new file +212 -0

212 lines not on this page · raw message

x86/unittests.cfg +6 -0
index f2401eb..87d412f 100644
@@ -346,3 +346,9 @@ extra_params = -M q35,kernel-irqchip=split -device intel-iommu,intremap=on,eim=o
 file = tsx-ctrl.flat
 extra_params = -cpu host
 groups = tsx-ctrl
+
+[intel_cet]
+file = cet.flat
+arch = x86_64
+smp = 2
+extra_params = -enable-kvm -m 2048 -cpu host
Signature · 1 line
2.17.2
#13 lore raw

PATCH v10 7/8 KVM: VMX: Enable CET support for nested VM

In reply to Yang Weijiang

CET MSRs are passed to guests for performance consideration. Configure the MSRs to match L0/L1 settings so that nested VM is able to run with CET.

---
 arch/x86/kvm/vmx/nested.c | 41 +++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/vmx/vmcs12.c |  6 ++++++
 arch/x86/kvm/vmx/vmcs12.h | 14 ++++++++++++-
 arch/x86/kvm/vmx/vmx.c    |  1 +
 4 files changed, 59 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/nested.c +39 -2
index e47eb7c0fbae..a71ef33de55f 100644
@@ -627,6 +627,41 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,
 	nested_vmx_disable_intercept_for_msr(msr_bitmap_l1, msr_bitmap_l0,
 					     MSR_KERNEL_GS_BASE, MSR_TYPE_RW);
 
+	/* Pass CET MSRs to nested VM if L0 and L1 are set to pass-through. */
+	if (!msr_write_intercepted_l01(vcpu, MSR_IA32_U_CET))
+		nested_vmx_disable_intercept_for_msr(
+					msr_bitmap_l1, msr_bitmap_l0,
+					MSR_IA32_U_CET, MSR_TYPE_RW);
+
+	if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL3_SSP))
+		nested_vmx_disable_intercept_for_msr(
+					msr_bitmap_l1, msr_bitmap_l0,
+					MSR_IA32_PL3_SSP, MSR_TYPE_RW);
+
+	if (!msr_write_intercepted_l01(vcpu, MSR_IA32_S_CET))
+		nested_vmx_disable_intercept_for_msr(
+					msr_bitmap_l1, msr_bitmap_l0,
+					MSR_IA32_S_CET, MSR_TYPE_RW);
+
+	if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL0_SSP))
+		nested_vmx_disable_intercept_for_msr(
+					msr_bitmap_l1, msr_bitmap_l0,
+					MSR_IA32_PL0_SSP, MSR_TYPE_RW);
+
+	if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL1_SSP))
+		nested_vmx_disable_intercept_for_msr(
+					msr_bitmap_l1, msr_bitmap_l0,
+					MSR_IA32_PL1_SSP, MSR_TYPE_RW);
+
+	if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL2_SSP))
+		nested_vmx_disable_intercept_for_msr(
+					msr_bitmap_l1, msr_bitmap_l0,
+					MSR_IA32_PL2_SSP, MSR_TYPE_RW);
+
+	if (!msr_write_intercepted_l01(vcpu, MSR_IA32_INT_SSP_TAB))
+		nested_vmx_disable_intercept_for_msr(
+					msr_bitmap_l1, msr_bitmap_l0,
+					MSR_IA32_INT_SSP_TAB, MSR_TYPE_RW);
 	/*
 	 * Checking the L0->L1 bitmap is trying to verify two things:
 	 *
@@ -6040,7 +6075,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps)
 	msrs->exit_ctls_high |=
 		VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR |
 		VM_EXIT_LOAD_IA32_EFER | VM_EXIT_SAVE_IA32_EFER |
-		VM_EXIT_SAVE_VMX_PREEMPTION_TIMER | VM_EXIT_ACK_INTR_ON_EXIT;
+		VM_EXIT_SAVE_VMX_PREEMPTION_TIMER | VM_EXIT_ACK_INTR_ON_EXIT |
+		VM_EXIT_LOAD_HOST_CET_STATE;
 
 	/* We support free control of debug control saving. */
 	msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS;
@@ -6057,7 +6093,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps)
 #endif
 		VM_ENTRY_LOAD_IA32_PAT;
 	msrs->entry_ctls_high |=
-		(VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER);
+		(VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER |
+		 VM_ENTRY_LOAD_GUEST_CET_STATE);
 
 	/* We support free control of debug control loading. */
 	msrs->entry_ctls_low &= ~VM_ENTRY_LOAD_DEBUG_CONTROLS;
arch/x86/kvm/vmx/vmcs12.c +6 -0
index 53dfb401316d..82b82bebeee0 100644
@@ -141,6 +141,9 @@ const unsigned short vmcs_field_to_offset_table[] = {
 	FIELD(GUEST_PENDING_DBG_EXCEPTIONS, guest_pending_dbg_exceptions),
 	FIELD(GUEST_SYSENTER_ESP, guest_sysenter_esp),
 	FIELD(GUEST_SYSENTER_EIP, guest_sysenter_eip),
+	FIELD(GUEST_S_CET, guest_s_cet),
+	FIELD(GUEST_SSP, guest_ssp),
+	FIELD(GUEST_INTR_SSP_TABLE, guest_ssp_tbl),
 	FIELD(HOST_CR0, host_cr0),
 	FIELD(HOST_CR3, host_cr3),
 	FIELD(HOST_CR4, host_cr4),
@@ -153,5 +156,8 @@ const unsigned short vmcs_field_to_offset_table[] = {
 	FIELD(HOST_IA32_SYSENTER_EIP, host_ia32_sysenter_eip),
 	FIELD(HOST_RSP, host_rsp),
 	FIELD(HOST_RIP, host_rip),
+	FIELD(HOST_S_CET, host_s_cet),
+	FIELD(HOST_SSP, host_ssp),
+	FIELD(HOST_INTR_SSP_TABLE, host_ssp_tbl),
 };
 const unsigned int nr_vmcs12_fields = ARRAY_SIZE(vmcs_field_to_offset_table);
arch/x86/kvm/vmx/vmcs12.h +13 -1
index d0c6df373f67..62b7be68f05c 100644
@@ -118,7 +118,13 @@ struct __packed vmcs12 {
 	natural_width host_ia32_sysenter_eip;
 	natural_width host_rsp;
 	natural_width host_rip;
-	natural_width paddingl[8]; /* room for future expansion */
+	natural_width host_s_cet;
+	natural_width host_ssp;
+	natural_width host_ssp_tbl;
+	natural_width guest_s_cet;
+	natural_width guest_ssp;
+	natural_width guest_ssp_tbl;
+	natural_width paddingl[2]; /* room for future expansion */
 	u32 pin_based_vm_exec_control;
 	u32 cpu_based_vm_exec_control;
 	u32 exception_bitmap;
@@ -301,6 +307,12 @@ static inline void vmx_check_vmcs12_offsets(void)
 	CHECK_OFFSET(host_ia32_sysenter_eip, 656);
 	CHECK_OFFSET(host_rsp, 664);
 	CHECK_OFFSET(host_rip, 672);
+	CHECK_OFFSET(host_s_cet, 680);
+	CHECK_OFFSET(host_ssp, 688);
+	CHECK_OFFSET(host_ssp_tbl, 696);
+	CHECK_OFFSET(guest_s_cet, 704);
+	CHECK_OFFSET(guest_ssp, 712);
+	CHECK_OFFSET(guest_ssp_tbl, 720);
 	CHECK_OFFSET(pin_based_vm_exec_control, 744);
 	CHECK_OFFSET(cpu_based_vm_exec_control, 748);
 	CHECK_OFFSET(exception_bitmap, 752);
arch/x86/kvm/vmx/vmx.c +1 -0
index 2654bd099fe6..1de03da2bda6 100644
@@ -7141,6 +7141,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
 	cr4_fixed1_update(X86_CR4_PKE,        ecx, feature_bit(PKU));
 	cr4_fixed1_update(X86_CR4_UMIP,       ecx, feature_bit(UMIP));
 	cr4_fixed1_update(X86_CR4_LA57,       ecx, feature_bit(LA57));
+	cr4_fixed1_update(X86_CR4_CET,	      ecx, feature_bit(SHSTK));
 
 #undef cr4_fixed1_update
 }
Signature · 1 line
2.17.2
#14 lore raw

Re: [PATCH v10 7/8] KVM: VMX: Enable CET support for nested VM

Thank you for the patch! Yet something to improve:

Hi Yang,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on kvm/linux-next]
[also build test ERROR on next-20200319]
[cannot apply to vhost/linux-next tip/auto-latest linux/master linus/master v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
config: i386-allyesconfig (attached as .config)
compiler: gcc-7 (Debian 7.5.0-5) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag

All errors (new ones prefixed by >>):

arch/x86/kvm/vmx/nested.c: In function 'nested_vmx_prepare_msr_bitmap':

>> arch/x86/kvm/vmx/nested.c:630:39: error: 'MSR_IA32_U_CET' undeclared (first use in this function); did you mean 'MSR_IA32_TSC'?
     if (!msr_write_intercepted_l01(vcpu, MSR_IA32_U_CET))
                                          ^~~~~~~~~~~~~~
                                          MSR_IA32_TSC
   arch/x86/kvm/vmx/nested.c:630:39: note: each undeclared identifier is reported only once for each function it appears in
>> arch/x86/kvm/vmx/nested.c:635:39: error: 'MSR_IA32_PL3_SSP' undeclared (first use in this function); did you mean 'MSR_IA32_MCG_ESP'?
     if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL3_SSP))
                                          ^~~~~~~~~~~~~~~~
                                          MSR_IA32_MCG_ESP
>> arch/x86/kvm/vmx/nested.c:640:39: error: 'MSR_IA32_S_CET' undeclared (first use in this function); did you mean 'MSR_IA32_U_CET'?
     if (!msr_write_intercepted_l01(vcpu, MSR_IA32_S_CET))
                                          ^~~~~~~~~~~~~~
                                          MSR_IA32_U_CET
>> arch/x86/kvm/vmx/nested.c:645:39: error: 'MSR_IA32_PL0_SSP' undeclared (first use in this function); did you mean 'MSR_IA32_PL3_SSP'?
     if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL0_SSP))
                                          ^~~~~~~~~~~~~~~~
                                          MSR_IA32_PL3_SSP
>> arch/x86/kvm/vmx/nested.c:650:39: error: 'MSR_IA32_PL1_SSP' undeclared (first use in this function); did you mean 'MSR_IA32_PL0_SSP'?
     if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL1_SSP))
                                          ^~~~~~~~~~~~~~~~
                                          MSR_IA32_PL0_SSP
>> arch/x86/kvm/vmx/nested.c:655:39: error: 'MSR_IA32_PL2_SSP' undeclared (first use in this function); did you mean 'MSR_IA32_PL1_SSP'?
     if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL2_SSP))
                                          ^~~~~~~~~~~~~~~~
                                          MSR_IA32_PL1_SSP
>> arch/x86/kvm/vmx/nested.c:660:39: error: 'MSR_IA32_INT_SSP_TAB' undeclared (first use in this function); did you mean 'MSR_IA32_PL2_SSP'?
     if (!msr_write_intercepted_l01(vcpu, MSR_IA32_INT_SSP_TAB))
                                          ^~~~~~~~~~~~~~~~~~~~
                                          MSR_IA32_PL2_SSP

vim +630 arch/x86/kvm/vmx/nested.c

557
558 /*
559 * Merge L0's and L1's MSR bitmap, return false to indicate that
560 * we do not use the hardware.
561 */
562 static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,
563 struct vmcs12 *vmcs12)
564 {
565 int msr;
566 unsigned long *msr_bitmap_l1;
567 unsigned long *msr_bitmap_l0 = to_vmx(vcpu)->nested.vmcs02.msr_bitmap;
568 struct kvm_host_map *map = &to_vmx(vcpu)->nested.msr_bitmap_map;
569
570 /* Nothing to do if the MSR bitmap is not in use. */
571 if (!cpu_has_vmx_msr_bitmap() ||
572 !nested_cpu_has(vmcs12, CPU_BASED_USE_MSR_BITMAPS))
573 return false;
574
575 if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), map))
576 return false;
577
578 msr_bitmap_l1 = (unsigned long *)map->hva;
579
580 /*
581 * To keep the control flow simple, pay eight 8-byte writes (sixteen
582 * 4-byte writes on 32-bit systems) up front to enable intercepts for
583 * the x2APIC MSR range and selectively disable them below.
584 */
585 enable_x2apic_msr_intercepts(msr_bitmap_l0);
586
587 if (nested_cpu_has_virt_x2apic_mode(vmcs12)) {
588 if (nested_cpu_has_apic_reg_virt(vmcs12)) {
589 /*
590 * L0 need not intercept reads for MSRs between 0x800
591 * and 0x8ff, it just lets the processor take the value
592 * from the virtual-APIC page; take those 256 bits
593 * directly from the L1 bitmap.
594 */
595 for (msr = 0x800; msr <= 0x8ff; msr += BITS_PER_LONG) {
596 unsigned word = msr / BITS_PER_LONG;
597
598 msr_bitmap_l0[word] = msr_bitmap_l1[word];
599 }
600 }
601
602 nested_vmx_disable_intercept_for_msr(
603 msr_bitmap_l1, msr_bitmap_l0,
604 X2APIC_MSR(APIC_TASKPRI),
605 MSR_TYPE_R | MSR_TYPE_W);
606
607 if (nested_cpu_has_vid(vmcs12)) {
608 nested_vmx_disable_intercept_for_msr(
609 msr_bitmap_l1, msr_bitmap_l0,
610 X2APIC_MSR(APIC_EOI),
611 MSR_TYPE_W);
612 nested_vmx_disable_intercept_for_msr(
613 msr_bitmap_l1, msr_bitmap_l0,
614 X2APIC_MSR(APIC_SELF_IPI),
615 MSR_TYPE_W);
616 }
617 }
618
619 /* KVM unconditionally exposes the FS/GS base MSRs to L1. */
620 nested_vmx_disable_intercept_for_msr(msr_bitmap_l1, msr_bitmap_l0,
621 MSR_FS_BASE, MSR_TYPE_RW);
622
623 nested_vmx_disable_intercept_for_msr(msr_bitmap_l1, msr_bitmap_l0,
624 MSR_GS_BASE, MSR_TYPE_RW);
625
626 nested_vmx_disable_intercept_for_msr(msr_bitmap_l1, msr_bitmap_l0,
627 MSR_KERNEL_GS_BASE, MSR_TYPE_RW);
628
629 /* Pass CET MSRs to nested VM if L0 and L1 are set to pass-through. */

 > 630		if (!msr_write_intercepted_l01(vcpu, MSR_IA32_U_CET))
   631			nested_vmx_disable_intercept_for_msr(
   632						msr_bitmap_l1, msr_bitmap_l0,
   633						MSR_IA32_U_CET, MSR_TYPE_RW);
   634	
 > 635		if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL3_SSP))
   636			nested_vmx_disable_intercept_for_msr(
   637						msr_bitmap_l1, msr_bitmap_l0,
   638						MSR_IA32_PL3_SSP, MSR_TYPE_RW);
   639	
 > 640		if (!msr_write_intercepted_l01(vcpu, MSR_IA32_S_CET))
   641			nested_vmx_disable_intercept_for_msr(
   642						msr_bitmap_l1, msr_bitmap_l0,
   643						MSR_IA32_S_CET, MSR_TYPE_RW);
   644	
 > 645		if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL0_SSP))
   646			nested_vmx_disable_intercept_for_msr(
   647						msr_bitmap_l1, msr_bitmap_l0,
   648						MSR_IA32_PL0_SSP, MSR_TYPE_RW);
   649	
 > 650		if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL1_SSP))
   651			nested_vmx_disable_intercept_for_msr(
   652						msr_bitmap_l1, msr_bitmap_l0,
   653						MSR_IA32_PL1_SSP, MSR_TYPE_RW);
   654	
 > 655		if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL2_SSP))
   656			nested_vmx_disable_intercept_for_msr(
   657						msr_bitmap_l1, msr_bitmap_l0,
   658						MSR_IA32_PL2_SSP, MSR_TYPE_RW);
   659	
 > 660		if (!msr_write_intercepted_l01(vcpu, MSR_IA32_INT_SSP_TAB))
   661			nested_vmx_disable_intercept_for_msr(
   662						msr_bitmap_l1, msr_bitmap_l0,
   663						MSR_IA32_INT_SSP_TAB, MSR_TYPE_RW);
   664		/*
   665		 * Checking the L0->L1 bitmap is trying to verify two things:
   666		 *
   667		 * 1. L0 gave a permission to L1 to actually passthrough the MSR. This
   668		 *    ensures that we do not accidentally generate an L02 MSR bitmap
   669		 *    from the L12 MSR bitmap that is too permissive.
   670		 * 2. That L1 or L2s have actually used the MSR. This avoids
   671		 *    unnecessarily merging of the bitmap if the MSR is unused. This
   672		 *    works properly because we only update the L01 MSR bitmap lazily.
   673		 *    So even if L0 should pass L1 these MSRs, the L01 bitmap is only
   674		 *    updated to reflect this when L1 (or its L2s) actually write to
   675		 *    the MSR.

676 */
677 if (!msr_write_intercepted_l01(vcpu, MSR_IA32_SPEC_CTRL))
678 nested_vmx_disable_intercept_for_msr(
679 msr_bitmap_l1, msr_bitmap_l0,
680 MSR_IA32_SPEC_CTRL,
681 MSR_TYPE_R | MSR_TYPE_W);
682
683 if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PRED_CMD))
684 nested_vmx_disable_intercept_for_msr(
685 msr_bitmap_l1, msr_bitmap_l0,
686 MSR_IA32_PRED_CMD,
687 MSR_TYPE_W);
688
689 kvm_vcpu_unmap(vcpu, &to_vmx(vcpu)->nested.msr_bitmap_map, false);
690
691 return true;
692 }
693

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#15 lore raw

PATCH v10 8/8 KVM: X86: Set CET feature bits for CPUID enumeration

In reply to Yang Weijiang

Set the feature bits so that CET capabilities can be see in guest via CPUID enumeration. Add CR4.CET bit support in order to allow guest set CET master control bit.

---
 arch/x86/include/asm/kvm_host.h | 3 ++-
 arch/x86/kvm/cpuid.c            | 4 ++++
 2 files changed, 6 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +2 -1
index 24c90ea5ddbd..3b6dba7e610e 100644
@@ -95,7 +95,8 @@
 			  | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \
 			  | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \
 			  | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \
-			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP))
+			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \
+			  | X86_CR4_CET))
 
 #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)
 
arch/x86/kvm/cpuid.c +4 -0
index 71703d9277ee..9e19775c4105 100644
@@ -363,6 +363,10 @@ void kvm_set_cpu_caps(void)
 		kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP);
 	if (boot_cpu_has(X86_FEATURE_AMD_SSBD))
 		kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD);
+	if (boot_cpu_has(X86_FEATURE_IBT))
+		kvm_cpu_cap_set(X86_FEATURE_IBT);
+	if (boot_cpu_has(X86_FEATURE_SHSTK))
+		kvm_cpu_cap_set(X86_FEATURE_SHSTK);
 
 	kvm_cpu_cap_mask(CPUID_7_1_EAX,
 		F(AVX512_BF16)
Signature · 1 line
2.17.2
#16 lore raw

Re: [PATCH v10 8/8] KVM: X86: Set CET feature bits for CPUID enumeration

Thank you for the patch! Perhaps something to improve:

Hi Yang,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on kvm/linux-next]
[also build test WARNING on next-20200320]
[cannot apply to vhost/linux-next tip/auto-latest linux/master linus/master v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Yang-Weijiang/Introduce-support-for-guest-CET-feature/20200320-155517
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.1-181-g83789bbc-dirty
        make ARCH=x86_64 allmodconfig
        make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'

If you fix the issue, kindly add following tag

sparse warnings: (new ones prefixed by >>)

arch/x86/kvm/x86.c:98:46: sparse: sparse: undefined identifier 'X86_CR4_CET'

>> arch/x86/kvm/x86.c:98:46: sparse: sparse: cast from unknown type

arch/x86/kvm/x86.c:809:60: sparse: sparse: undefined identifier 'X86_CR4_CET' arch/x86/kvm/x86.c:940:29: sparse: sparse: undefined identifier 'X86_CR4_CET' arch/x86/kvm/x86.c:940:29: sparse: sparse: cast from unknown type
arch/x86/kvm/x86.c:956:19: sparse: sparse: undefined identifier 'X86_CR4_CET' arch/x86/kvm/x86.c:956:19: sparse: sparse: cast from unknown type
arch/x86/kvm/x86.c:1233:23: sparse: sparse: undefined identifier 'MSR_IA32_U_CET' arch/x86/kvm/x86.c:1233:39: sparse: sparse: undefined identifier 'MSR_IA32_S_CET' arch/x86/kvm/x86.c:1234:9: sparse: sparse: undefined identifier 'MSR_IA32_PL0_SSP' arch/x86/kvm/x86.c:1234:27: sparse: sparse: undefined identifier 'MSR_IA32_PL1_SSP' arch/x86/kvm/x86.c:1234:45: sparse: sparse: undefined identifier 'MSR_IA32_PL2_SSP' arch/x86/kvm/x86.c:1235:9: sparse: sparse: undefined identifier 'MSR_IA32_PL3_SSP' arch/x86/kvm/x86.c:1235:27: sparse: sparse: undefined identifier 'MSR_IA32_INT_SSP_TAB' arch/x86/kvm/x86.c:1512:14: sparse: sparse: undefined identifier 'MSR_IA32_PL0_SSP' arch/x86/kvm/x86.c:1512:35: sparse: sparse: undefined identifier 'MSR_IA32_PL3_SSP' arch/x86/kvm/x86.c:1513:14: sparse: sparse: undefined identifier 'MSR_IA32_U_CET' arch/x86/kvm/x86.c:1514:14: sparse: sparse: undefined identifier 'MSR_IA32_S_CET' arch/x86/kvm/x86.c:1515:14: sparse: sparse: undefined identifier 'MSR_IA32_INT_SSP_TAB' arch/x86/kvm/x86.c:1512:14: sparse: sparse: incompatible types for 'case' statement arch/x86/kvm/x86.c:1512:35: sparse: sparse: incompatible types for 'case' statement arch/x86/kvm/x86.c:1513:14: sparse: sparse: incompatible types for 'case' statement arch/x86/kvm/x86.c:1514:14: sparse: sparse: incompatible types for 'case' statement arch/x86/kvm/x86.c:1515:14: sparse: sparse: incompatible types for 'case' statement
arch/x86/kvm/x86.c:2646:38: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected void const [noderef] <asn:1> * @@ got const [noderef] <asn:1> * @@
arch/x86/kvm/x86.c:2646:38: sparse: expected void const [noderef] <asn:1> *
arch/x86/kvm/x86.c:2646:38: sparse: got unsigned char [usertype] *
arch/x86/kvm/x86.c:3267:25: sparse: sparse: undefined identifier 'MSR_IA32_U_CET' arch/x86/kvm/x86.c:7549:15: sparse: sparse: incompatible types in comparison expression (different address spaces):
arch/x86/kvm/x86.c:7549:15: sparse: struct kvm_apic_map [noderef] <asn:4> *
arch/x86/kvm/x86.c:7549:15: sparse: struct kvm_apic_map *
arch/x86/kvm/x86.c:9678:44: sparse: sparse: undefined identifier 'XFEATURE_MASK_CET_USER' arch/x86/kvm/x86.c:9678:44: sparse: sparse: undefined identifier 'XFEATURE_MASK_CET_KERNEL' arch/x86/kvm/x86.c:9912:16: sparse: sparse: incompatible types in comparison expression (different address spaces):
arch/x86/kvm/x86.c:9912:16: sparse: struct kvm_apic_map [noderef] <asn:4> *
arch/x86/kvm/x86.c:9912:16: sparse: struct kvm_apic_map *
arch/x86/kvm/x86.c:9913:15: sparse: sparse: incompatible types in comparison expression (different address spaces):
arch/x86/kvm/x86.c:9913:15: sparse: struct kvm_pmu_event_filter [noderef] <asn:4> *
arch/x86/kvm/x86.c:9913:15: sparse: struct kvm_pmu_event_filter *
arch/x86/kvm/x86.c:1512:14: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1512:35: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1513:14: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1514:14: sparse: sparse: Expected constant expression in case statement arch/x86/kvm/x86.c:1515:14: sparse: sparse: Expected constant expression in case statement
--
arch/x86/kvm/emulate.c:5495:21: sparse: sparse: arithmetics on pointers to functions arch/x86/kvm/emulate.c:4206:17: sparse: sparse: undefined identifier 'X86_CR4_CET'

>> arch/x86/kvm/emulate.c:4206:17: sparse: sparse: cast from unknown type

vim +98 arch/x86/kvm/x86.c

313a3dc75da206 drivers/kvm/x86.c Carsten Otte 2007-10-11 97
b11306b53b2540 arch/x86/kvm/x86.c Sean Christopherson 2019-12-10 @98 static u64 __read_mostly cr4_reserved_bits = CR4_RESERVED_BITS; b11306b53b2540 arch/x86/kvm/x86.c Sean Christopherson 2019-12-10 99

:::::: The code at line 98 was first introduced by commit
:::::: b11306b53b2540c6ba068c4deddb6a17d9f8d95b KVM: x86: Don't let userspace set host-reserved cr4 bits

:::::: TO: Sean Christopherson <[email protected]> :::::: CC: Paolo Bonzini <[email protected]>

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org