Skip to contentSkip to conversation

LASS KVM virtualization support

PATCH In review Zeng Guang 4 participants 25 messages Apr 20, 2023 – Apr 26, 2023

Subsystems (3)
lore
Share & tools
lore

Zeng Guang

author
#0 lore raw

cover LASS KVM virtualization support

Linear Address Space Separation (LASS)[1] is a new mechanism that enforces the same mode-based protections as paging, i.e. SMAP/SMEP but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, "probes" by malicious software will provide no paging-based timing information.

LASS works in long mode and partitions the 64-bit canonical linear address space into two halves:
1. Lower half (LA[63]=0) --> user space
2. Upper half (LA[63]=1) --> kernel space

When LASS is enabled, a general protection #GP fault or a stack fault #SS will be generated if software accesses the address from the half in which it resides to another half, e.g., either from user space to upper half, or from kernel space to lower half. This protection applies to data access, code execution.

This series add KVM LASS virtualization support.

When platform has LASS capability, KVM requires to expose this feature to guest VM enumerated by CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], and allow guest to enable it via CR4.LASS[bit 27] on demand. For instruction executed in the guest directly, hardware will perform the LASS violation check, while KVM also needs to apply LASS to instructions emulated by software and injects #GP or #SS fault to the guest.

Following LASS voilations check will be taken on KVM emulation path. User-mode access to supervisor space address:

        LA[bit 63] && (CPL == 3)
Supervisor-mode access to user space address:
        Instruction fetch: !LA[bit 63] && (CPL < 3)
        Data access: !LA[bit 63] && (CR4.SMAP==1) && ((RFLAGS.AC == 0 &&
                     CPL < 3) || Implicit supervisor access)

We tested the basic function of LASS virtualization including LASS enumeration and enabling in non-root and nested environment. As current KVM unittest framework is not compatible to LASS rule that kernel should run in the upper half, we use kernel module and application test to verify LASS functionalities in guest instead. The data access related x86 emulator code is verified with forced emulation prefix (FEP) mechanism. Other test cases are working in progress.

How to add tests for LASS in KUT or kselftest is still under investigation.

[1] Intel Architecutre Instruction Set Extensions and Future Features Programming Reference: Chapter Linear Address Space Separation (LASS)
https://cdrdv2.intel.com/v1/dl/getContent/671368

Zeng Guang (6):
KVM: x86: Virtualize CR4.LASS
KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check
KVM: x86: Add emulator helper for LASS violation check
KVM: x86: LASS protection on KVM emulation when LASS enabled
KVM: x86: Advertise LASS CPUID to user space
KVM: x86: Set KVM LASS based on hardware capability

 arch/x86/include/asm/cpuid.h       | 36 +++++++++++++++++++
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  7 +++-
 arch/x86/kvm/cpuid.c               |  8 +++--
 arch/x86/kvm/emulate.c             | 36 ++++++++++++++++---
 arch/x86/kvm/kvm_emulate.h         |  1 +
 arch/x86/kvm/vmx/nested.c          |  3 ++
 arch/x86/kvm/vmx/sgx.c             |  2 ++
 arch/x86/kvm/vmx/vmx.c             | 58 ++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.h             |  2 ++
 arch/x86/kvm/x86.c                 |  9 +++++
 arch/x86/kvm/x86.h                 |  2 ++
 12 files changed, 157 insertions(+), 8 deletions(-)
Signature · 1 line
2.27.0
author
#1 lore raw

PATCH 1/6 KVM: x86: Virtualize CR4.LASS

Virtualize CR4.LASS[bit 27] under KVM control instead of being guest-owned as CR4.LASS generally set once for each vCPU at boot time and won't be toggled at runtime. Besides, only if VM has LASS capability enumerated with CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], KVM allows guest software to be able to set CR4.LASS. By design CR4.LASS can be manipulated by nested guest as well.

Notes: Setting CR4.LASS to 1 enable LASS in IA-32e mode. It doesn't take effect in legacy mode even if CR4.LASS is set.

---
 arch/x86/include/asm/kvm_host.h | 2 +-
 arch/x86/kvm/vmx/vmx.c          | 3 +++
 arch/x86/kvm/x86.h              | 2 ++
 3 files changed, 6 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -1
index 6aaae18f1854..8ff89a52ef66 100644
@@ -125,7 +125,7 @@
 			  | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \
 			  | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \
 			  | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \
-			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP))
+			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP | X86_CR4_LASS))
 
 #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)
 
arch/x86/kvm/vmx/vmx.c +3 -0
index 43ff3276918b..c923d7599d71 100644
@@ -7569,6 +7569,9 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
 	cr4_fixed1_update(X86_CR4_UMIP,       ecx, feature_bit(UMIP));
 	cr4_fixed1_update(X86_CR4_LA57,       ecx, feature_bit(LA57));
 
+	entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1);
+	cr4_fixed1_update(X86_CR4_LASS,       eax, feature_bit(LASS));
+
 #undef cr4_fixed1_update
 }
 
arch/x86/kvm/x86.h +2 -0
index 7c8a30d44c29..218f4c73789a 100644
@@ -475,6 +475,8 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type);
 		__reserved_bits |= X86_CR4_VMXE;        \
 	if (!__cpu_has(__c, X86_FEATURE_PCID))          \
 		__reserved_bits |= X86_CR4_PCIDE;       \
+	if (!__cpu_has(__c, X86_FEATURE_LASS))          \
+		__reserved_bits |= X86_CR4_LASS;        \
 	__reserved_bits;                                \
 })
 
Signature · 1 line
2.27.0
Reviewed-by on v1 1/6
#2 lore raw

Re: [PATCH 1/6] KVM: x86: Virtualize CR4.LASS

one nit below

Quoted 1 line · Zeng Guang

Virtualize CR4.LASS[bit 27] under KVM control instead of being guest-owned

under control of KVM or under KVM's control

Or just simply use "intercept"?

Quoted 56 lines

as CR4.LASS generally set once for each vCPU at boot time and won't be toggled at runtime. Besides, only if VM has LASS capability enumerated with CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], KVM allows guest software to be able to set CR4.LASS. By design CR4.LASS can be manipulated by nested guest as well.

Notes: Setting CR4.LASS to 1 enable LASS in IA-32e mode. It doesn't take effect in legacy mode even if CR4.LASS is set.

---
  arch/x86/include/asm/kvm_host.h | 2 +-
  arch/x86/kvm/vmx/vmx.c          | 3 +++
  arch/x86/kvm/x86.h              | 2 ++
  3 files changed, 6 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -1
index 6aaae18f1854..8ff89a52ef66 100644
@@ -125,7 +125,7 @@
  			  | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \
  			  | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \
  			  | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \
-			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP))
+			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP | X86_CR4_LASS))
 
  #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)
 
arch/x86/kvm/vmx/vmx.c +3 -0
index 43ff3276918b..c923d7599d71 100644
@@ -7569,6 +7569,9 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
  	cr4_fixed1_update(X86_CR4_UMIP,       ecx, feature_bit(UMIP));
  	cr4_fixed1_update(X86_CR4_LA57,       ecx, feature_bit(LA57));
 
+	entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1);
+	cr4_fixed1_update(X86_CR4_LASS,       eax, feature_bit(LASS));
+
  #undef cr4_fixed1_update
  }
 
arch/x86/kvm/x86.h +2 -0
index 7c8a30d44c29..218f4c73789a 100644
@@ -475,6 +475,8 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type);
  		__reserved_bits |= X86_CR4_VMXE;        \
  	if (!__cpu_has(__c, X86_FEATURE_PCID))          \
  		__reserved_bits |= X86_CR4_PCIDE;       \
+	if (!__cpu_has(__c, X86_FEATURE_LASS))          \
+		__reserved_bits |= X86_CR4_LASS;        \
  	__reserved_bits;                                \
  })
 
author
#3 lore raw

Re: [PATCH 1/6] KVM: x86: Virtualize CR4.LASS

Quoted 9 lines · Binbin Wu

one nit below

Quoted 1 line · Zeng Guang

Virtualize CR4.LASS[bit 27] under KVM control instead of being guest-owned

under control of KVM or under KVM's control

Or just simply use "intercept"?

OK. Will change it.

Quoted 56 lines
Quoted 56 lines

as CR4.LASS generally set once for each vCPU at boot time and won't be toggled at runtime. Besides, only if VM has LASS capability enumerated with CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], KVM allows guest software to be able to set CR4.LASS. By design CR4.LASS can be manipulated by nested guest as well.

Notes: Setting CR4.LASS to 1 enable LASS in IA-32e mode. It doesn't take effect in legacy mode even if CR4.LASS is set.

---
   arch/x86/include/asm/kvm_host.h | 2 +-
   arch/x86/kvm/vmx/vmx.c          | 3 +++
   arch/x86/kvm/x86.h              | 2 ++
   3 files changed, 6 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -1
index 6aaae18f1854..8ff89a52ef66 100644
@@ -125,7 +125,7 @@
   			  | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \
   			  | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \
   			  | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \
-			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP))
+			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP | X86_CR4_LASS))
 
   #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)
 
arch/x86/kvm/vmx/vmx.c +3 -0
index 43ff3276918b..c923d7599d71 100644
@@ -7569,6 +7569,9 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
   	cr4_fixed1_update(X86_CR4_UMIP,       ecx, feature_bit(UMIP));
   	cr4_fixed1_update(X86_CR4_LA57,       ecx, feature_bit(LA57));
 
+	entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1);
+	cr4_fixed1_update(X86_CR4_LASS,       eax, feature_bit(LASS));
+
   #undef cr4_fixed1_update
   }
 
arch/x86/kvm/x86.h +2 -0
index 7c8a30d44c29..218f4c73789a 100644
@@ -475,6 +475,8 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type);
   		__reserved_bits |= X86_CR4_VMXE;        \
   	if (!__cpu_has(__c, X86_FEATURE_PCID))          \
   		__reserved_bits |= X86_CR4_PCIDE;       \
+	if (!__cpu_has(__c, X86_FEATURE_LASS))          \
+		__reserved_bits |= X86_CR4_LASS;        \
   	__reserved_bits;                                \
   })
 
#4 lore raw

Re: [PATCH 1/6] KVM: x86: Virtualize CR4.LASS

In reply to Zeng Guang
Quoted 8 lines · Zeng Guang

Virtualize CR4.LASS[bit 27] under KVM control instead of being guest-owned as CR4.LASS generally set once for each vCPU at boot time and won't be toggled at runtime. Besides, only if VM has LASS capability enumerated with CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], KVM allows guest software to be able to set CR4.LASS.

By design CR4.LASS can be manipulated by nested guest as well.

This is inaccurate. The change in nested_vmx_cr_fixed1_bits_update() is to allow L1 guests to set CR4.LASS in VMX operation. I would say:

Set the CR4.LASS bit in the emulated IA32_VMX_CR4_FIXED1 MSR for guests to allow guests to enable LASS in nested VMX operation.

author
#5 lore raw

Re: [PATCH 1/6] KVM: x86: Virtualize CR4.LASS

Quoted 10 lines · Chao
Quoted 7 lines · Zeng Guang

Virtualize CR4.LASS[bit 27] under KVM control instead of being guest-owned as CR4.LASS generally set once for each vCPU at boot time and won't be toggled at runtime. Besides, only if VM has LASS capability enumerated with CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], KVM allows guest software to be able to set CR4.LASS.
By design CR4.LASS can be manipulated by nested guest as
well.

This is inaccurate. The change in nested_vmx_cr_fixed1_bits_update() is to allow L1 guests to set CR4.LASS in VMX operation.

Essentially it allows nested guest to set CR4.LASS. L1 guest uses cr4_fixed1 to check cr4 value requested to set by nested guest valid or not. Nested guest will get #GP fault if it's not allowed.

Quoted 4 lines

I would say:

Set the CR4.LASS bit in the emulated IA32_VMX_CR4_FIXED1 MSR for guests to allow guests to enable LASS in nested VMX operation.

#6 lore raw

Re: [PATCH 1/6] KVM: x86: Virtualize CR4.LASS

Quoted 10 lines · Zeng Guang
Quoted 8 lines · Chao
Quoted 7 lines · Zeng Guang

Virtualize CR4.LASS[bit 27] under KVM control instead of being guest-owned as CR4.LASS generally set once for each vCPU at boot time and won't be toggled at runtime. Besides, only if VM has LASS capability enumerated with CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], KVM allows guest software to be able to set CR4.LASS.
By design CR4.LASS can be manipulated by nested guest as
well.

Quoted 8 lines
Quoted 2 lines

This is inaccurate. The change in nested_vmx_cr_fixed1_bits_update() is to allow L1 guests to set CR4.LASS in VMX operation.

Essentially it allows nested guest to set CR4.LASS. L1 guest uses cr4_fixed1 to check
cr4 value requested to set by nested guest valid or not. Nested guest will get #GP
fault if it's not allowed.

The change to CR4_FIXED1 has more impacts. Without CR4_FIXED1 change, guest shouldn't enable LASS in VMX operation; it means:

1. before VMXON, LASS should be disabled
2. in VMX operation, LASS cannot be enabled

What you said (i.e., L1 guest allows L2 to enable LASS) belongs in #2. But #1 isn't covered. That's why I said "inaccurate".

Quoted 6 lines
Quoted 5 lines

I would say:

Set the CR4.LASS bit in the emulated IA32_VMX_CR4_FIXED1 MSR for guests to allow guests to enable LASS in nested VMX operation.

author
#7 lore raw

PATCH 2/6 KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check

In reply to Zeng Guang

Intel introduce LASS (Linear Address Separation) feature providing an independent mechanism to achieve the mode-based protection.

LASS partitions 64-bit linear address space into two halves, user-mode address (LA[bit 63]=0) and supervisor-mode address (LA[bit 63]=1). It stops any code execution or data access
1. from user mode to supervisor-mode address space
2. from supervisor mode to user-mode address space
and generates LASS violation fault accordingly.

A supervisor mode data access causes a LASS violation only if supervisor mode access protection is enabled (CR4.SMAP = 1) and either RFLAGS.AC = 0 or the access implicitly accesses a system data structure.

Following are the rule of LASS violation check on the linear address(LA). User access to supervisor-mode address space:

    LA[bit 63] && (CPL == 3)
Supervisor access to user-mode address space:
    Instruction fetch: !LA[bit 63] && (CPL < 3)
    Data access: !LA[bit 63] && (CR4.SMAP==1) && ((RFLAGS.AC == 0 &&
		 CPL < 3) || Implicit supervisor access)

Add new ops in kvm_x86_ops to do LASS violation check.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  5 +++
 arch/x86/kvm/vmx/vmx.c             | 55 ++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.h             |  2 ++
 4 files changed, 63 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index abccd51dcfca..f76c07f2674b 100644
@@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed)
 KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
+KVM_X86_OP_OPTIONAL_RET0(check_lass);
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +5 -0
index 8ff89a52ef66..31fb8699a1ff 100644
@@ -69,6 +69,9 @@
 #define KVM_X86_NOTIFY_VMEXIT_VALID_BITS	(KVM_X86_NOTIFY_VMEXIT_ENABLED | \
 						 KVM_X86_NOTIFY_VMEXIT_USER)
 
+/* x86-specific emulation flags */
+#define KVM_X86_EMULFLAG_SKIP_LASS	_BITULL(1)
+
 /* x86-specific vcpu->requests bit members */
 #define KVM_REQ_MIGRATE_TIMER		KVM_ARCH_REQ(0)
 #define KVM_REQ_REPORT_TPR_ACCESS	KVM_ARCH_REQ(1)
@@ -1706,6 +1709,8 @@ struct kvm_x86_ops {
 	 * Returns vCPU specific APICv inhibit reasons
 	 */
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
+
+	bool (*check_lass)(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/vmx/vmx.c +55 -0

67 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.h +2 -0
index a3da84f4ea45..6569385a5978 100644
@@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type);
 u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu);
 u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
 
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);
+
 static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr,
 					     int type, bool value)
 {
Signature · 1 line
2.27.0
#8 lore raw

Re: [PATCH 2/6] KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check

Quoted 1 line · Zeng Guang

Intel introduce LASS (Linear Address Separation) feature providing

/s/introduce/introduces

Quoted 8 lines

an independent mechanism to achieve the mode-based protection.

LASS partitions 64-bit linear address space into two halves, user-mode address (LA[bit 63]=0) and supervisor-mode address (LA[bit 63]=1). It stops any code execution or data access
1. from user mode to supervisor-mode address space
2. from supervisor mode to user-mode address space
and generates LASS violation fault accordingly.

IMO, the description of the point 2 may be misleading that LASS stops any data access from supervisor mode to user mode address space, although the description following adds the conditions.

Quoted 6 lines

A supervisor mode data access causes a LASS violation only if supervisor mode access protection is enabled (CR4.SMAP = 1) and either RFLAGS.AC = 0 or the access implicitly accesses a system data structure.

Following are the rule of LASS violation check on the linear address(LA).

/s/rule/rules

Quoted 34 lines

User access to supervisor-mode address space:

     LA[bit 63] && (CPL == 3)
Supervisor access to user-mode address space:
     Instruction fetch: !LA[bit 63] && (CPL < 3)
     Data access: !LA[bit 63] && (CR4.SMAP==1) && ((RFLAGS.AC == 0 &&
		 CPL < 3) || Implicit supervisor access)

Add new ops in kvm_x86_ops to do LASS violation check.

---
  arch/x86/include/asm/kvm-x86-ops.h |  1 +
  arch/x86/include/asm/kvm_host.h    |  5 +++
  arch/x86/kvm/vmx/vmx.c             | 55 ++++++++++++++++++++++++++++++
  arch/x86/kvm/vmx/vmx.h             |  2 ++
  4 files changed, 63 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index abccd51dcfca..f76c07f2674b 100644
@@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed)
  KVM_X86_OP(complete_emulated_msr)
  KVM_X86_OP(vcpu_deliver_sipi_vector)
  KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
+KVM_X86_OP_OPTIONAL_RET0(check_lass);
 
  #undef KVM_X86_OP
  #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +0 -0
index 8ff89a52ef66..31fb8699a1ff 100644

arch/x86/include/asm/kvm_host.h line 73

Quoted 5 lines
arch/x86/include/asm/kvm_host.h +2 -0
@@ -69,6 +69,9 @@
  #define KVM_X86_NOTIFY_VMEXIT_VALID_BITS	(KVM_X86_NOTIFY_VMEXIT_ENABLED | \
  						 KVM_X86_NOTIFY_VMEXIT_USER)
 
+/* x86-specific emulation flags */
+#define KVM_X86_EMULFLAG_SKIP_LASS	_BITULL(1)

Do you use the flag outside of emulator?
For LAM patch, it's planned to move the flags inside emulator.

arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1713

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +3 -0
+
  /* x86-specific vcpu->requests bit members */
  #define KVM_REQ_MIGRATE_TIMER		KVM_ARCH_REQ(0)
  #define KVM_REQ_REPORT_TPR_ACCESS	KVM_ARCH_REQ(1)
@@ -1706,6 +1709,8 @@ struct kvm_x86_ops {
  	 * Returns vCPU specific APICv inhibit reasons
  	 */
  	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
+
+	bool (*check_lass)(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);

The flags may be dropped if the caller knows to skip it or not.

Quoted 8 lines
arch/x86/include/asm/kvm_host.h +0 -0
  };
 
  struct kvm_x86_nested_ops {
arch/x86/kvm/vmx/vmx.c +0 -0
index c923d7599d71..581327ede66a 100644

arch/x86/kvm/vmx/vmx.c static void vmx_vm_destroy(struct kvm *kvm) · line 8076

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +4 -0
@@ -8070,6 +8070,59 @@ static void vmx_vm_destroy(struct kvm *kvm)
  	free_pages((unsigned long)kvm_vmx->pid_table, vmx_get_pid_table_order(kvm));
  }
 
+/*
+ * Determine whether an access to the linear address causes a LASS violation.
+ * LASS protection is only effective in long mode. As a prerequisite, caller
+ * should make sure VM

Should be vCPU?

Quoted 75 lines

running in long mode and invoke this api to do LASS
+ * violation check.
+ */

arch/x86/kvm/vmx/vmx.c +49 -0
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	bool user_mode, user_as, rflags_ac;
+
+	if (!!(flags & KVM_X86_EMULFLAG_SKIP_LASS) ||
+	    !kvm_is_cr4_bit_set(vcpu, X86_CR4_LASS))
+		return false;
+
+	WARN_ON_ONCE(!is_long_mode(vcpu));
+
+	user_as = !(la >> 63);
+
+	/*
+	 * An access is a supervisor-mode access if CPL < 3 or if it implicitly
+	 * accesses a system data structure. For implicit accesses to system
+	 * data structure, the processor acts as if RFLAGS.AC is clear.
+	 */
+	if (access & PFERR_IMPLICIT_ACCESS) {
+		user_mode = false;
+		rflags_ac = false;
+	} else {
+		user_mode = vmx_get_cpl(vcpu) == 3;
+		if (!user_mode)
+			rflags_ac = !!(kvm_get_rflags(vcpu) & X86_EFLAGS_AC);
+	}
+
+	if (user_mode != user_as) {
+		/*
+		 * Supervisor-mode _data_ accesses to user address space
+		 * cause LASS violations only if SMAP is enabled.
+		 */
+		if (!user_mode && !(access & PFERR_FETCH_MASK)) {
+			return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) &&
+			       !rflags_ac;
+		} else {
+			return true;
+		}
+	}
+
+	return false;
+}
+
+static bool vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	return is_long_mode(vcpu) && __vmx_check_lass(vcpu, access, la, flags);
+}
+
  static struct kvm_x86_ops vmx_x86_ops __initdata = {
  	.name = "kvm_intel",
@@ -8207,6 +8260,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {
  	.complete_emulated_msr = kvm_complete_insn_gp,
 
  	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
+
+	.check_lass = vmx_check_lass,
  };
 
  static unsigned int vmx_handle_intel_pt_intr(void)
arch/x86/kvm/vmx/vmx.h +2 -0
index a3da84f4ea45..6569385a5978 100644
@@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type);
  u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu);
  u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
 
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);
+
  static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr,
  					     int type, bool value)
  {
author
#9 lore raw

Re: [PATCH 2/6] KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check

Quoted 4 lines · Binbin Wu
Quoted 1 line · Zeng Guang

Intel introduce LASS (Linear Address Separation) feature providing

/s/introduce/introduces

OK.

Quoted 9 lines
Quoted 9 lines

an independent mechanism to achieve the mode-based protection.

LASS partitions 64-bit linear address space into two halves, user-mode address (LA[bit 63]=0) and supervisor-mode address (LA[bit 63]=1). It stops any code execution or data access
1. from user mode to supervisor-mode address space
2. from supervisor mode to user-mode address space
and generates LASS violation fault accordingly.

Quoted 3 lines

IMO, the description of the point 2 may be misleading that LASS stops any data access from supervisor mode to user mode address space, although the description following adds the conditions.

May change to " It stops any code execution or conditional data access". The condition is illustrated in next paragraph.

Quoted 7 lines
Quoted 6 lines

A supervisor mode data access causes a LASS violation only if supervisor mode access protection is enabled (CR4.SMAP = 1) and either RFLAGS.AC = 0 or the access implicitly accesses a system data structure.

Following are the rule of LASS violation check on the linear address(LA).

/s/rule/rules

OK.

Quoted 37 lines
Quoted 37 lines

User access to supervisor-mode address space:

      LA[bit 63] && (CPL == 3)
Supervisor access to user-mode address space:
      Instruction fetch: !LA[bit 63] && (CPL < 3)
      Data access: !LA[bit 63] && (CR4.SMAP==1) && ((RFLAGS.AC == 0 &&
		 CPL < 3) || Implicit supervisor access)

Add new ops in kvm_x86_ops to do LASS violation check.

---
   arch/x86/include/asm/kvm-x86-ops.h |  1 +
   arch/x86/include/asm/kvm_host.h    |  5 +++
   arch/x86/kvm/vmx/vmx.c             | 55 ++++++++++++++++++++++++++++++
   arch/x86/kvm/vmx/vmx.h             |  2 ++
   4 files changed, 63 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index abccd51dcfca..f76c07f2674b 100644
@@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed)
   KVM_X86_OP(complete_emulated_msr)
   KVM_X86_OP(vcpu_deliver_sipi_vector)
   KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
+KVM_X86_OP_OPTIONAL_RET0(check_lass);
 
   #undef KVM_X86_OP
   #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +0 -0
index 8ff89a52ef66..31fb8699a1ff 100644
@@ -69,6 +69,9 @@
   #define KVM_X86_NOTIFY_VMEXIT_VALID_BITS	(KVM_X86_NOTIFY_VMEXIT_ENABLED | \
   						 KVM_X86_NOTIFY_VMEXIT_USER)
 
Quoted 4 lines

arch/x86/include/asm/kvm_host.h line 73

Quoted 2 lines
arch/x86/include/asm/kvm_host.h +2 -0
+/* x86-specific emulation flags */
+#define KVM_X86_EMULFLAG_SKIP_LASS	_BITULL(1)

Do you use the flag outside of emulator?
For LAM patch, it's planned to move the flags inside emulator.

IMO, the detailed flag is implementation specific. Is it necessary to bind with emulator though it's only used inside emulator ?

Quoted 11 lines

arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1713

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +3 -0
+
   /* x86-specific vcpu->requests bit members */
   #define KVM_REQ_MIGRATE_TIMER		KVM_ARCH_REQ(0)
   #define KVM_REQ_REPORT_TPR_ACCESS	KVM_ARCH_REQ(1)
@@ -1706,6 +1709,8 @@ struct kvm_x86_ops {
   	 * Returns vCPU specific APICv inhibit reasons
   	 */
   	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
+
+	bool (*check_lass)(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);

The flags may be dropped if the caller knows to skip it or not.

Probably I don't get you right. Do you mean it need define another function without flags ?

Quoted 11 lines
Quoted 11 lines
arch/x86/include/asm/kvm_host.h +0 -0
   };
 
   struct kvm_x86_nested_ops {
arch/x86/kvm/vmx/vmx.c +0 -0
index c923d7599d71..581327ede66a 100644
@@ -8070,6 +8070,59 @@ static void vmx_vm_destroy(struct kvm *kvm)
   	free_pages((unsigned long)kvm_vmx->pid_table, vmx_get_pid_table_order(kvm));
   }
 
Quoted 5 lines

arch/x86/kvm/vmx/vmx.c static void vmx_vm_destroy(struct kvm *kvm)

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +4 -0
+/*
+ * Determine whether an access to the linear address causes a LASS violation.
+ * LASS protection is only effective in long mode. As a prerequisite, caller
+ * should make sure VM

Should be vCPU?

Similar meaning, I think. :)

Quoted 75 lines
Quoted 75 lines

running in long mode and invoke this api to do LASS
+ * violation check.
+ */

arch/x86/kvm/vmx/vmx.c +49 -0
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	bool user_mode, user_as, rflags_ac;
+
+	if (!!(flags & KVM_X86_EMULFLAG_SKIP_LASS) ||
+	    !kvm_is_cr4_bit_set(vcpu, X86_CR4_LASS))
+		return false;
+
+	WARN_ON_ONCE(!is_long_mode(vcpu));
+
+	user_as = !(la >> 63);
+
+	/*
+	 * An access is a supervisor-mode access if CPL < 3 or if it implicitly
+	 * accesses a system data structure. For implicit accesses to system
+	 * data structure, the processor acts as if RFLAGS.AC is clear.
+	 */
+	if (access & PFERR_IMPLICIT_ACCESS) {
+		user_mode = false;
+		rflags_ac = false;
+	} else {
+		user_mode = vmx_get_cpl(vcpu) == 3;
+		if (!user_mode)
+			rflags_ac = !!(kvm_get_rflags(vcpu) & X86_EFLAGS_AC);
+	}
+
+	if (user_mode != user_as) {
+		/*
+		 * Supervisor-mode _data_ accesses to user address space
+		 * cause LASS violations only if SMAP is enabled.
+		 */
+		if (!user_mode && !(access & PFERR_FETCH_MASK)) {
+			return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) &&
+			       !rflags_ac;
+		} else {
+			return true;
+		}
+	}
+
+	return false;
+}
+
+static bool vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	return is_long_mode(vcpu) && __vmx_check_lass(vcpu, access, la, flags);
+}
+
   static struct kvm_x86_ops vmx_x86_ops __initdata = {
   	.name = "kvm_intel",
@@ -8207,6 +8260,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {
   	.complete_emulated_msr = kvm_complete_insn_gp,
 
   	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
+
+	.check_lass = vmx_check_lass,
   };
 
   static unsigned int vmx_handle_intel_pt_intr(void)
arch/x86/kvm/vmx/vmx.h +2 -0
index a3da84f4ea45..6569385a5978 100644
@@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type);
   u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu);
   u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
 
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);
+
   static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr,
   					     int type, bool value)
   {
#10 lore raw

Re: [PATCH 2/6] KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check

Quoted 3 lines · Zeng Guang
Quoted 1 line · Binbin Wu

[...]

Quoted 18 lines
Quoted 18 lines · Zeng Guang
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index abccd51dcfca..f76c07f2674b 100644
@@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed)
    KVM_X86_OP(complete_emulated_msr)
    KVM_X86_OP(vcpu_deliver_sipi_vector)
    KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
+KVM_X86_OP_OPTIONAL_RET0(check_lass);
       #undef KVM_X86_OP
    #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +0 -0
index 8ff89a52ef66..31fb8699a1ff 100644
@@ -69,6 +69,9 @@
Quoted 8 lines
Quoted 6 lines
arch/x86/include/asm/kvm_host.h +0 -0
    #define KVM_X86_NOTIFY_VMEXIT_VALID_BITS (KVM_X86_NOTIFY_VMEXIT_ENABLED | \
                             KVM_X86_NOTIFY_VMEXIT_USER)

   +/* x86-specific emulation flags */
+#define KVM_X86_EMULFLAG_SKIP_LASS    _BITULL(1)
Do you use the flag outside of emulator?
For LAM patch, it's planned to move the flags inside emulator.

IMO, the detailed flag is implementation specific. Is it necessary to bind with emulator though it's only used inside emulator ?

For the rest part (i.e., VMExit handlings), the code is already in the vendor specific implementations. The callers are aware of the information to skip LASS check or not.

I plan to do a cleanup to consolidate the flags into one parameter for __linearize(). And the consolidated flags value will be extended for LAM and other features (e.g. LASS).

I post the proposed patch as following, could you help to check wether it is OK for LASS to follow?

arch/x86/kvm/emulate.c     | 20 ++++++++++++++------ arch/x86/kvm/kvm_emulate.h |  4 ++++

 2 files changed, 18 insertions(+), 6 deletions(-)
arch/x86/kvm/emulate.c +0 -0
index a20bec931764..5fb516bc5731 100644
@@ -687,8 +687,8 @@ static unsigned insn_alignment(struct x86_emulate_ctxt *ctxt, unsigned size)

 static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt, struct segmented_address addr,
                        unsigned *max_size, unsigned size,
-                       bool write, bool fetch,
-                       enum x86emul_mode mode, ulong *linear)
+                       u64 flags, enum x86emul_mode mode,
+                       ulong *linear)
 {
     struct desc_struct desc;
     bool usable;
@@ -696,6 +696,8 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt, u32 lim;
     u16 sel;
     u8  va_bits;
+    bool fetch = !!(flags & X86_EMULFLAG_FETCH);
+    bool write = !!(flags & X86_EMULFLAG_WRITE);

     la = seg_base(ctxt, addr.seg) + addr.ea;
     *max_size = 0;
@@ -757,7 +759,12 @@ static int linearize(struct x86_emulate_ctxt *ctxt, ulong *linear)
 {
     unsigned max_size;
-    return __linearize(ctxt, addr, &max_size, size, write, false, +    u64 flags = 0;
+
+    if (write)
+        flags |= X86_EMULFLAG_WRITE;
+
+    return __linearize(ctxt, addr, &max_size, size, flags, ctxt->mode, linear);
 }

@@ -768,10 +775,11 @@ static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst) unsigned max_size;
     struct segmented_address addr = { .seg = VCPU_SREG_CS,
                        .ea = dst };
+    u64 flags = X86_EMULFLAG_FETCH;

     if (ctxt->op_bytes != sizeof(unsigned long))
         addr.ea = dst & ((1UL << (ctxt->op_bytes << 3)) - 1);
-    rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, &linear); +    rc = __linearize(ctxt, addr, &max_size, 1, flags, ctxt->mode, &linear);
     if (rc == X86EMUL_CONTINUE)
         ctxt->_eip = addr.ea;
     return rc;
@@ -896,6 +904,7 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctxt *ctxt, int op_size) int cur_size = ctxt->fetch.end - ctxt->fetch.data;
     struct segmented_address addr = { .seg = VCPU_SREG_CS,
                        .ea = ctxt->eip + cur_size };
+    u64 flags = X86_EMULFLAG_FETCH;

     /*
      * We do not know exactly how many bytes will be needed, and
@@ -907,8 +916,7 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctxt *ctxt, int op_size)
      * boundary check itself.  Instead, we use max_size to check
      * against op_size.
      */
-    rc = __linearize(ctxt, addr, &max_size, 0, false, true, ctxt->mode,
-             &linear);
+    rc = __linearize(ctxt, addr, &max_size, 0, flags, ctxt->mode, &linear);
     if (unlikely(rc != X86EMUL_CONTINUE))
         return rc;

arch/x86/kvm/kvm_emulate.h +0 -0
index ab65f3a47dfd..5451a37f135f 100644
@@ -88,6 +88,10 @@ struct x86_instruction_info {

 #define X86EMUL_IO_NEEDED       5 /* IO is needed to complete emulation */ #define X86EMUL_INTERCEPTED     6 /* Intercepted by nested VMCB/VMCS */

+/* x86-specific emulation flags */
+#define X86_EMULFLAG_FETCH            _BITULL(0) +#define X86_EMULFLAG_WRITE            _BITULL(1) +
 struct x86_emulate_ops {
     void (*vm_bugged)(struct x86_emulate_ctxt *ctxt); /*

Quoted 108 lines
Quoted 4 lines
Quoted 4 lines

+
   /* x86-specific vcpu->requests bit members */ #define KVM_REQ_MIGRATE_TIMER        KVM_ARCH_REQ(0) #define KVM_REQ_REPORT_TPR_ACCESS    KVM_ARCH_REQ(1)

Quoted 8 lines

Review struct kvm_x86_ops { · line 1714

Quoted 7 lines
(quoted diff) +2 -0
@@ -1706,6 +1709,8 @@ struct kvm_x86_ops {
         * Returns vCPU specific APICv inhibit reasons
         */
        unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
+
+    bool (*check_lass)(struct kvm_vcpu *vcpu, u64 access, u64 la, 
 u64 flags);

The flags may be dropped if the caller knows to skip it or not.

Probably I don't get you right. Do you mean it need define another function without flags ?

Quoted 8 lines
Quoted 8 lines
(quoted diff) +0 -0
    };
       struct kvm_x86_nested_ops {
arch/x86/kvm/vmx/vmx.c +0 -0
index c923d7599d71..581327ede66a 100644
@@ -8070,6 +8070,59 @@ static void vmx_vm_destroy(struct kvm *kvm)
        free_pages((unsigned long)kvm_vmx->pid_table, vmx_get_pid_table_order(kvm));
Quoted 8 lines
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +0 -0
    }

   +/*
+ * Determine whether an access to the linear address causes a LASS violation.
+ * LASS protection is only effective in long mode. As a prerequisite, caller
+ * should make sure VM

Should be vCPU?

Similar meaning, I think. :)

Quoted 77 lines
Quoted 77 lines

running in long mode and invoke this api to do LASS
+ * violation check.
+ */
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)

arch/x86/kvm/vmx/vmx.c +13 -0
+{
+    bool user_mode, user_as, rflags_ac;
+
+    if (!!(flags & KVM_X86_EMULFLAG_SKIP_LASS) ||
+        !kvm_is_cr4_bit_set(vcpu, X86_CR4_LASS))
+        return false;
+
+    WARN_ON_ONCE(!is_long_mode(vcpu));
+
+    user_as = !(la >> 63);
+
+    /*
+     * An access is a supervisor-mode access if CPL < 3 or if it 

implicitly
+     * accesses a system data structure. For implicit accesses to system
+     * data structure, the processor acts as if RFLAGS.AC is clear. +     */
+    if (access & PFERR_IMPLICIT_ACCESS) {
+        user_mode = false;
+        rflags_ac = false;
+    } else {
+        user_mode = vmx_get_cpl(vcpu) == 3;
+        if (!user_mode)
+            rflags_ac = !!(kvm_get_rflags(vcpu) & X86_EFLAGS_AC); +    }

arch/x86/kvm/vmx/vmx.c +18 -0
+
+    if (user_mode != user_as) {
+        /*
+         * Supervisor-mode _data_ accesses to user address space
+         * cause LASS violations only if SMAP is enabled.
+         */
+        if (!user_mode && !(access & PFERR_FETCH_MASK)) {
+            return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) &&
+                   !rflags_ac;
+        } else {
+            return true;
+        }
+    }
+
+    return false;
+}
+
+static bool vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 

la, u64 flags)
+{
+    return is_long_mode(vcpu) && __vmx_check_lass(vcpu, access, la, flags);

arch/x86/kvm/vmx/vmx.c +2 -0
+}
+

   static struct kvm_x86_ops vmx_x86_ops __initdata = {
       .name = "kvm_intel",
   @@ -8207,6 +8260,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .complete_emulated_msr = kvm_complete_insn_gp, .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, +
+    .check_lass = vmx_check_lass,
   };
      static unsigned int vmx_handle_intel_pt_intr(void)

arch/x86/kvm/vmx/vmx.h +1 -0
index a3da84f4ea45..6569385a5978 100644
@@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct 
 kvm_vcpu *vcpu, u32 msr, int type);
    u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu);
    u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
    +bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);
+
    static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr,
                             int type, bool value)

   {

#11 lore raw

Re: [PATCH 2/6] KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check

In reply to Zeng Guang
Quoted 5 lines · Zeng Guang

+/*
+ * Determine whether an access to the linear address causes a LASS violation.
+ * LASS protection is only effective in long mode. As a prerequisite, caller
+ * should make sure VM running in long mode and invoke this api to do LASS
+ * violation check.

Could you place the comment above vmx_check_lass()?

And for __vmx_check_lass(), just add:

A variant of vmx_check_lass() without the check for long mode.

Quoted 20 lines

+ */

(quoted diff) +17 -0
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	bool user_mode, user_as, rflags_ac;
+
+	if (!!(flags & KVM_X86_EMULFLAG_SKIP_LASS) ||
+	    !kvm_is_cr4_bit_set(vcpu, X86_CR4_LASS))
+		return false;
+
+	WARN_ON_ONCE(!is_long_mode(vcpu));
+
+	user_as = !(la >> 63);
+
 
 
+	/*
+	 * An access is a supervisor-mode access if CPL < 3 or if it implicitly
+	 * accesses a system data structure. For implicit accesses to system
+	 * data structure, the processor acts as if RFLAGS.AC is clear.
+	 */

Review

Quoted 10 lines
(quoted diff) +10 -0
+	if (access & PFERR_IMPLICIT_ACCESS) {
+		user_mode = false;
+		rflags_ac = false;
+	} else {
+		user_mode = vmx_get_cpl(vcpu) == 3;
+		if (!user_mode)
+			rflags_ac = !!(kvm_get_rflags(vcpu) & X86_EFLAGS_AC);
+	}
+
+	if (user_mode != user_as) {

to reduce one level of indentation, how about:

	if (user_mode == user_as)
		return false;

	/*
	 * Supervisor-mode _data_ accesses to user address space
	 * cause LASS violations only if SMAP is enabled.
	 */
	if (!user_mode && !(access & PFERR_FETCH_MASK)) {
		return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) && !rflags_ac;

	return true;
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +8 -0
+		/*
+		 * Supervisor-mode _data_ accesses to user address space
+		 * cause LASS violations only if SMAP is enabled.
+		 */
+		if (!user_mode && !(access & PFERR_FETCH_MASK)) {
+			return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) &&
+			       !rflags_ac;
+		} else {

arch/x86/kvm/vmx/vmx.c static void vmx_vm_destroy(struct kvm *kvm) · line 8123

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+			return true;
+		}
+	}
+
+	return false;
+}
+
+static bool vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	return is_long_mode(vcpu) && __vmx_check_lass(vcpu, access, la, flags);

Why not request all callers to check if vcpu is in long mode?

e.g.,

	return is_long_mode(vcpu) && static_call(kvm_x86_check_lass)(...);

then you can rename __vmx_check_lass() to vmx_check_lass() and drop the original one.

Quoted 19 lines
(quoted diff) +2 -0
+}
+
static struct kvm_x86_ops vmx_x86_ops __initdata = {
	.name = "kvm_intel",
arch/x86/kvm/vmx/vmx.c +2 -0
@@ -8207,6 +8260,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {
 	.complete_emulated_msr = kvm_complete_insn_gp,
 
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
+
+	.check_lass = vmx_check_lass,
 };
 
 static unsigned int vmx_handle_intel_pt_intr(void)
arch/x86/kvm/vmx/vmx.h +0 -0
index a3da84f4ea45..6569385a5978 100644

arch/x86/kvm/vmx/vmx.h void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type); · line 437

Quoted 5 lines
arch/x86/kvm/vmx/vmx.h +2 -0
@@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type);
 u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu);
 u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
 
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);
+

no one uses this function. You can defer exporting it to when the first external caller is added.

Quoted 6 lines
arch/x86/kvm/vmx/vmx.h +0 -0
 static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr,
 					     int type, bool value)
 {
Signature · 1 line
2.27.0
author
#12 lore raw

Re: [PATCH 2/6] KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check

Quoted 43 lines · Chao
Quoted 5 lines · Zeng Guang

+/*
+ * Determine whether an access to the linear address causes a LASS violation.
+ * LASS protection is only effective in long mode. As a prerequisite, caller
+ * should make sure VM running in long mode and invoke this api to do LASS
+ * violation check.

Could you place the comment above vmx_check_lass()?

And for __vmx_check_lass(), just add:

A variant of vmx_check_lass() without the check for long mode.

Quoted 19 lines

+ */

(quoted diff) +17 -0
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	bool user_mode, user_as, rflags_ac;
+
+	if (!!(flags & KVM_X86_EMULFLAG_SKIP_LASS) ||
+	    !kvm_is_cr4_bit_set(vcpu, X86_CR4_LASS))
+		return false;
+
+	WARN_ON_ONCE(!is_long_mode(vcpu));
+
+	user_as = !(la >> 63);
+
 
+	/*
+	 * An access is a supervisor-mode access if CPL < 3 or if it implicitly
+	 * accesses a system data structure. For implicit accesses to system
+	 * data structure, the processor acts as if RFLAGS.AC is clear.
+	 */

Review

Quoted 10 lines
(quoted diff) +10 -0
+	if (access & PFERR_IMPLICIT_ACCESS) {
+		user_mode = false;
+		rflags_ac = false;
+	} else {
+		user_mode = vmx_get_cpl(vcpu) == 3;
+		if (!user_mode)
+			rflags_ac = !!(kvm_get_rflags(vcpu) & X86_EFLAGS_AC);
+	}
+
+	if (user_mode != user_as) {

to reduce one level of indentation, how about:

Quoted 12 lines
	if (user_mode == user_as)
		return false;

	/*
	 * Supervisor-mode _data_ accesses to user address space
	 * cause LASS violations only if SMAP is enabled.
	 */
	if (!user_mode && !(access & PFERR_FETCH_MASK)) {
		return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) && !rflags_ac;

	return true;

Looks better.

Quoted 20 lines
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +8 -0
+		/*
+		 * Supervisor-mode _data_ accesses to user address space
+		 * cause LASS violations only if SMAP is enabled.
+		 */
+		if (!user_mode && !(access & PFERR_FETCH_MASK)) {
+			return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) &&
+			       !rflags_ac;
+		} else {

arch/x86/kvm/vmx/vmx.c static void vmx_vm_destroy(struct kvm *kvm) · line 8123

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+			return true;
+		}
+	}
+
+	return false;
+}
+
+static bool vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags)
+{
+	return is_long_mode(vcpu) && __vmx_check_lass(vcpu, access, la, flags);

Why not request all callers to check if vcpu is in long mode?

Quoted 5 lines

e.g.,

	return is_long_mode(vcpu) && static_call(kvm_x86_check_lass)(...);

then you can rename __vmx_check_lass() to vmx_check_lass() and drop the original one.

By design, __vmx_check_lass() is standalone to be used for checking LASS violation only. In some cases, cpu mode is already identified prior to performing LASS protection. please refer to patch 4. So we provide two interfaces, vmx_check_lass() with cpu mode check wrapped for other modules usage, e.g. kvm emulator and __vmx_check_lass() dedicated for VMX.

I would check the feasibility to re-organize code to be more optimal.

Thanks.

Quoted 33 lines
Quoted 19 lines
(quoted diff) +2 -0
+}
+
static struct kvm_x86_ops vmx_x86_ops __initdata = {
	.name = "kvm_intel",
arch/x86/kvm/vmx/vmx.c +2 -0
@@ -8207,6 +8260,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {
 	.complete_emulated_msr = kvm_complete_insn_gp,
 
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
+
+	.check_lass = vmx_check_lass,
 };
 
 static unsigned int vmx_handle_intel_pt_intr(void)
arch/x86/kvm/vmx/vmx.h +0 -0
index a3da84f4ea45..6569385a5978 100644

arch/x86/kvm/vmx/vmx.h void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type); · line 437

Quoted 5 lines
arch/x86/kvm/vmx/vmx.h +2 -0
@@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type);
 u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu);
 u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
 
+bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags);
+

no one uses this function. You can defer exporting it to when the first external caller is added.

Quoted 6 lines
arch/x86/kvm/vmx/vmx.h +0 -0
 static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr,
 					     int type, bool value)
 {
Signature · 1 line
2.27.0
author
#13 lore raw

PATCH 3/6 KVM: x86: Add emulator helper for LASS violation check

In reply to Zeng Guang

When LASS is enabled, KVM need apply LASS violation check to instruction emulations. Add helper for the usage of x86 emulator to perform LASS protection.

---
 arch/x86/kvm/kvm_emulate.h | 1 +
 arch/x86/kvm/x86.c         | 9 +++++++++
 2 files changed, 10 insertions(+)
arch/x86/kvm/kvm_emulate.h +1 -0
index 2d9662be8333..1c55247d52d7 100644
@@ -224,6 +224,7 @@ struct x86_emulate_ops {
 	int (*leave_smm)(struct x86_emulate_ctxt *ctxt);
 	void (*triple_fault)(struct x86_emulate_ctxt *ctxt);
 	int (*set_xcr)(struct x86_emulate_ctxt *ctxt, u32 index, u64 xcr);
+	bool (*check_lass)(struct x86_emulate_ctxt *ctxt, u64 access, u64 la, u64 flags);
 };
 
 /* Type, address-of, and value of an instruction's operand. */
arch/x86/kvm/x86.c +9 -0
index 87feb1249ad6..704c5e4b9e76 100644
@@ -8193,6 +8193,14 @@ static void emulator_vm_bugged(struct x86_emulate_ctxt *ctxt)
 		kvm_vm_bugged(kvm);
 }
 
+static bool emulator_check_lass(struct x86_emulate_ctxt *ctxt,
+				u64 access, u64 la, u64 flags)
+{
+	struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
+
+	return static_call(kvm_x86_check_lass)(vcpu, access, la, flags);
+}
+
 static const struct x86_emulate_ops emulate_ops = {
 	.vm_bugged           = emulator_vm_bugged,
 	.read_gpr            = emulator_read_gpr,
@@ -8237,6 +8245,7 @@ static const struct x86_emulate_ops emulate_ops = {
 	.leave_smm           = emulator_leave_smm,
 	.triple_fault        = emulator_triple_fault,
 	.set_xcr             = emulator_set_xcr,
+	.check_lass          = emulator_check_lass,
 };
 
 static void toggle_interruptibility(struct kvm_vcpu *vcpu, u32 mask)
Signature · 1 line
2.27.0
author
#14 lore raw

PATCH 4/6 KVM: x86: LASS protection on KVM emulation when LASS enabled

In reply to Zeng Guang

Do LASS violation check for instructions emulated by KVM. Note that for instructions executed in the guest directly, hardware will perform the check.

Not all instruction emulation leads to accesses to guest linear addresses because 1) some instrutions like CPUID, RDMSR, don't take memory as operands 2) instruction fetch in most cases is already done inside the guest.

Four cases in which kvm may access guest linear addresses are identified by code inspection:
- KVM emulator uses segmented address for instruction fetches or data accesses.
- For implicit data access, KVM emulator gets address to a system data structure(GDT/LDT/IDT/TR).
- For VMX instruction emulation, KVM gets the address from "VM-exit instruction information" field in VMCS.
- For SGX ENCLS instruction emulation, KVM gets the address from registers.

LASS violation check applies to these linear address so as to enforce mode-based protections as hardware behaves.

As exceptions, the target memory address of emulation of invlpg, branch and call instructions doesn't require LASS violation check.

---
 arch/x86/kvm/emulate.c    | 36 +++++++++++++++++++++++++++++++-----
 arch/x86/kvm/vmx/nested.c |  3 +++
 arch/x86/kvm/vmx/sgx.c    |  2 ++
 3 files changed, 36 insertions(+), 5 deletions(-)
arch/x86/kvm/emulate.c +31 -5
index 5cc3efa0e21c..a9a022fd712e 100644
@@ -687,7 +687,8 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 				       struct segmented_address addr,
 				       unsigned *max_size, unsigned size,
 				       bool write, bool fetch,
-				       enum x86emul_mode mode, ulong *linear)
+				       enum x86emul_mode mode, ulong *linear,
+				       u64 flags)
 {
 	struct desc_struct desc;
 	bool usable;
@@ -695,6 +696,7 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 	u32 lim;
 	u16 sel;
 	u8  va_bits;
+	u64 access = fetch ? PFERR_FETCH_MASK : 0;
 
 	la = seg_base(ctxt, addr.seg) + addr.ea;
 	*max_size = 0;
@@ -740,6 +742,10 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 		}
 		break;
 	}
+
+	if (ctxt->ops->check_lass(ctxt, access, *linear, flags))
+		goto bad;
+
 	if (la & (insn_alignment(ctxt, size) - 1))
 		return emulate_gp(ctxt, 0);
 	return X86EMUL_CONTINUE;
@@ -757,7 +763,7 @@ static int linearize(struct x86_emulate_ctxt *ctxt,
 {
 	unsigned max_size;
 	return __linearize(ctxt, addr, &max_size, size, write, false,
-			   ctxt->mode, linear);
+			   ctxt->mode, linear, 0);
 }
 
 static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
@@ -770,7 +776,10 @@ static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
 
 	if (ctxt->op_bytes != sizeof(unsigned long))
 		addr.ea = dst & ((1UL << (ctxt->op_bytes << 3)) - 1);
-	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, &linear);
+
+	/* LASS doesn't apply to address for branch and call instructions */
+	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode,
+	     &linear, KVM_X86_EMULFLAG_SKIP_LASS);
 	if (rc == X86EMUL_CONTINUE)
 		ctxt->_eip = addr.ea;
 	return rc;
@@ -845,6 +854,13 @@ static inline int jmp_rel(struct x86_emulate_ctxt *ctxt, int rel)
 static int linear_read_system(struct x86_emulate_ctxt *ctxt, ulong linear,
 			      void *data, unsigned size)
 {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
 	return ctxt->ops->read_std(ctxt, linear, data, size, &ctxt->exception, true);
 }
 
@@ -852,6 +868,13 @@ static int linear_write_system(struct x86_emulate_ctxt *ctxt,
 			       ulong linear, void *data,
 			       unsigned int size)
 {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
 	return ctxt->ops->write_std(ctxt, linear, data, size, &ctxt->exception, true);
 }
 
@@ -907,7 +930,7 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctxt *ctxt, int op_size)
 	 * against op_size.
 	 */
 	rc = __linearize(ctxt, addr, &max_size, 0, false, true, ctxt->mode,
-			 &linear);
+			 &linear, 0);
 	if (unlikely(rc != X86EMUL_CONTINUE))
 		return rc;
 
@@ -3432,8 +3455,11 @@ static int em_invlpg(struct x86_emulate_ctxt *ctxt)
 {
 	int rc;
 	ulong linear;
+	unsigned max_size;
 
-	rc = linearize(ctxt, ctxt->src.addr.mem, 1, false, &linear);
+	/* LASS doesn't apply to the memory address for invlpg */
+	rc = __linearize(ctxt, ctxt->src.addr.mem, &max_size, 1, false, false,
+	     ctxt->mode, &linear, KVM_X86_EMULFLAG_SKIP_LASS);
 	if (rc == X86EMUL_CONTINUE)
 		ctxt->ops->invlpg(ctxt, linear);
 	/* Disable writeback. */
arch/x86/kvm/vmx/nested.c +3 -0
index c8ae9d0e59b3..55c88c4593a6 100644
@@ -4974,6 +4974,9 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification,
 		 * destination for long mode!
 		 */
 		exn = is_noncanonical_address(*ret, vcpu);
+
+		if (!exn)
+			exn = __vmx_check_lass(vcpu, 0, *ret, 0);
 	} else {
 		/*
 		 * When not in long mode, the virtual/linear address is
arch/x86/kvm/vmx/sgx.c +2 -0
index b12da2a6dec9..30cb5d0980be 100644
@@ -37,6 +37,8 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset,
 		fault = true;
 	} else if (likely(is_long_mode(vcpu))) {
 		fault = is_noncanonical_address(*gva, vcpu);
+		if (!fault)
+			fault = __vmx_check_lass(vcpu, 0, *gva, 0);
 	} else {
 		*gva &= 0xffffffff;
 		fault = (s.unusable) ||
Signature · 1 line
2.27.0
#15 lore raw

Re: [PATCH 4/6] KVM: x86: LASS protection on KVM emulation when LASS enabled

Quoted 6 lines · Zeng Guang

Do LASS violation check for instructions emulated by KVM. Note that for instructions executed in the guest directly, hardware will perform the check.

Not all instruction emulation leads to accesses to guest linear addresses because 1) some instrutions like CPUID, RDMSR, don't take memory as

/s/instrutions/instructions

Quoted 2 lines

operands 2) instruction fetch in most cases is already done inside the guest.

What are the instruction fetch cases not covered in non-root mode?
And IIUC, the patch actually doesn't distinguish them and alway checks LASS voilation for instruction fetch in instruction emulation, right?

Quoted 6 lines

Four cases in which kvm may access guest linear addresses are identified by code inspection:
- KVM emulator uses segmented address for instruction fetches or data accesses.
- For implicit data access, KVM emulator gets address to a system data

to or from?

Quoted 6 lines

structure(GDT/LDT/IDT/TR).
- For VMX instruction emulation, KVM gets the address from "VM-exit instruction information" field in VMCS.
- For SGX ENCLS instruction emulation, KVM gets the address from registers.

LASS violation check applies to these linear address so as to enforce

address -> addresses

Quoted 143 lines

mode-based protections as hardware behaves.

As exceptions, the target memory address of emulation of invlpg, branch and call instructions doesn't require LASS violation check.

---
  arch/x86/kvm/emulate.c    | 36 +++++++++++++++++++++++++++++++-----
  arch/x86/kvm/vmx/nested.c |  3 +++
  arch/x86/kvm/vmx/sgx.c    |  2 ++
  3 files changed, 36 insertions(+), 5 deletions(-)
arch/x86/kvm/emulate.c +31 -5
index 5cc3efa0e21c..a9a022fd712e 100644
@@ -687,7 +687,8 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
  				       struct segmented_address addr,
  				       unsigned *max_size, unsigned size,
  				       bool write, bool fetch,
-				       enum x86emul_mode mode, ulong *linear)
+				       enum x86emul_mode mode, ulong *linear,
+				       u64 flags)
  {
  	struct desc_struct desc;
  	bool usable;
@@ -695,6 +696,7 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
  	u32 lim;
  	u16 sel;
  	u8  va_bits;
+	u64 access = fetch ? PFERR_FETCH_MASK : 0;
 
  	la = seg_base(ctxt, addr.seg) + addr.ea;
  	*max_size = 0;
@@ -740,6 +742,10 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
  		}
  		break;
  	}
+
+	if (ctxt->ops->check_lass(ctxt, access, *linear, flags))
+		goto bad;
+
  	if (la & (insn_alignment(ctxt, size) - 1))
  		return emulate_gp(ctxt, 0);
  	return X86EMUL_CONTINUE;
@@ -757,7 +763,7 @@ static int linearize(struct x86_emulate_ctxt *ctxt,
  {
  	unsigned max_size;
  	return __linearize(ctxt, addr, &max_size, size, write, false,
-			   ctxt->mode, linear);
+			   ctxt->mode, linear, 0);
  }
 
  static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
@@ -770,7 +776,10 @@ static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
 
  	if (ctxt->op_bytes != sizeof(unsigned long))
  		addr.ea = dst & ((1UL << (ctxt->op_bytes << 3)) - 1);
-	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, &linear);
+
+	/* LASS doesn't apply to address for branch and call instructions */
+	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode,
+	     &linear, KVM_X86_EMULFLAG_SKIP_LASS);
  	if (rc == X86EMUL_CONTINUE)
  		ctxt->_eip = addr.ea;
  	return rc;
@@ -845,6 +854,13 @@ static inline int jmp_rel(struct x86_emulate_ctxt *ctxt, int rel)
  static int linear_read_system(struct x86_emulate_ctxt *ctxt, ulong linear,
  			      void *data, unsigned size)
  {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
  	return ctxt->ops->read_std(ctxt, linear, data, size, &ctxt->exception, true);
  }
 
@@ -852,6 +868,13 @@ static int linear_write_system(struct x86_emulate_ctxt *ctxt,
  			       ulong linear, void *data,
  			       unsigned int size)
  {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
  	return ctxt->ops->write_std(ctxt, linear, data, size, &ctxt->exception, true);
  }
 
@@ -907,7 +930,7 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctxt *ctxt, int op_size)
  	 * against op_size.
  	 */
  	rc = __linearize(ctxt, addr, &max_size, 0, false, true, ctxt->mode,
-			 &linear);
+			 &linear, 0);
  	if (unlikely(rc != X86EMUL_CONTINUE))
  		return rc;
 
@@ -3432,8 +3455,11 @@ static int em_invlpg(struct x86_emulate_ctxt *ctxt)
  {
  	int rc;
  	ulong linear;
+	unsigned max_size;
 
-	rc = linearize(ctxt, ctxt->src.addr.mem, 1, false, &linear);
+	/* LASS doesn't apply to the memory address for invlpg */
+	rc = __linearize(ctxt, ctxt->src.addr.mem, &max_size, 1, false, false,
+	     ctxt->mode, &linear, KVM_X86_EMULFLAG_SKIP_LASS);
  	if (rc == X86EMUL_CONTINUE)
  		ctxt->ops->invlpg(ctxt, linear);
  	/* Disable writeback. */
arch/x86/kvm/vmx/nested.c +3 -0
index c8ae9d0e59b3..55c88c4593a6 100644
@@ -4974,6 +4974,9 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification,
  		 * destination for long mode!
  		 */
  		exn = is_noncanonical_address(*ret, vcpu);
+
+		if (!exn)
+			exn = __vmx_check_lass(vcpu, 0, *ret, 0);
  	} else {
  		/*
  		 * When not in long mode, the virtual/linear address is
arch/x86/kvm/vmx/sgx.c +2 -0
index b12da2a6dec9..30cb5d0980be 100644
@@ -37,6 +37,8 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset,
  		fault = true;
  	} else if (likely(is_long_mode(vcpu))) {
  		fault = is_noncanonical_address(*gva, vcpu);
+		if (!fault)
+			fault = __vmx_check_lass(vcpu, 0, *gva, 0);
  	} else {
  		*gva &= 0xffffffff;
  		fault = (s.unusable) ||
author
#16 lore raw

Re: [PATCH 4/6] KVM: x86: LASS protection on KVM emulation when LASS enabled

Quoted 9 lines · Binbin Wu
Quoted 6 lines · Zeng Guang

Do LASS violation check for instructions emulated by KVM. Note that for instructions executed in the guest directly, hardware will perform the check.

Not all instruction emulation leads to accesses to guest linear addresses because 1) some instrutions like CPUID, RDMSR, don't take memory as

/s/instrutions/instructions

Oops. :P

Quoted 6 lines
Quoted 2 lines

operands 2) instruction fetch in most cases is already done inside the guest.

What are the instruction fetch cases not covered in non-root mode? And IIUC, the patch actually doesn't distinguish them and alway checks LASS voilation
for instruction fetch in instruction emulation, right?

Here states most of instruction needn't be fetched by KVM. KVM intercept the most of privileged instructions and complete the function emulation directly. But some instructions requires KVM to fetch the code and emulate further, e.g. lgdt/sgdt etc. KVM will always do LASS violation check on instruction fetch once it happens.

Quoted 6 lines
Quoted 5 lines

Four cases in which kvm may access guest linear addresses are identified by code inspection:
- KVM emulator uses segmented address for instruction fetches or data accesses.
- For implicit data access, KVM emulator gets address to a system data

to or from?

It means the address pointing *to* a system data structure.

Quoted 7 lines
Quoted 6 lines

structure(GDT/LDT/IDT/TR).
- For VMX instruction emulation, KVM gets the address from "VM-exit instruction information" field in VMCS.
- For SGX ENCLS instruction emulation, KVM gets the address from registers.

LASS violation check applies to these linear address so as to enforce

address -> addresses

OK.

Quoted 144 lines
Quoted 144 lines

mode-based protections as hardware behaves.

As exceptions, the target memory address of emulation of invlpg, branch and call instructions doesn't require LASS violation check.

---
   arch/x86/kvm/emulate.c    | 36 +++++++++++++++++++++++++++++++-----
   arch/x86/kvm/vmx/nested.c |  3 +++
   arch/x86/kvm/vmx/sgx.c    |  2 ++
   3 files changed, 36 insertions(+), 5 deletions(-)
arch/x86/kvm/emulate.c +31 -5
index 5cc3efa0e21c..a9a022fd712e 100644
@@ -687,7 +687,8 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
   				       struct segmented_address addr,
   				       unsigned *max_size, unsigned size,
   				       bool write, bool fetch,
-				       enum x86emul_mode mode, ulong *linear)
+				       enum x86emul_mode mode, ulong *linear,
+				       u64 flags)
   {
   	struct desc_struct desc;
   	bool usable;
@@ -695,6 +696,7 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
   	u32 lim;
   	u16 sel;
   	u8  va_bits;
+	u64 access = fetch ? PFERR_FETCH_MASK : 0;
 
   	la = seg_base(ctxt, addr.seg) + addr.ea;
   	*max_size = 0;
@@ -740,6 +742,10 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
   		}
   		break;
   	}
+
+	if (ctxt->ops->check_lass(ctxt, access, *linear, flags))
+		goto bad;
+
   	if (la & (insn_alignment(ctxt, size) - 1))
   		return emulate_gp(ctxt, 0);
   	return X86EMUL_CONTINUE;
@@ -757,7 +763,7 @@ static int linearize(struct x86_emulate_ctxt *ctxt,
   {
   	unsigned max_size;
   	return __linearize(ctxt, addr, &max_size, size, write, false,
-			   ctxt->mode, linear);
+			   ctxt->mode, linear, 0);
   }
 
   static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
@@ -770,7 +776,10 @@ static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
 
   	if (ctxt->op_bytes != sizeof(unsigned long))
   		addr.ea = dst & ((1UL << (ctxt->op_bytes << 3)) - 1);
-	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, &linear);
+
+	/* LASS doesn't apply to address for branch and call instructions */
+	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode,
+	     &linear, KVM_X86_EMULFLAG_SKIP_LASS);
   	if (rc == X86EMUL_CONTINUE)
   		ctxt->_eip = addr.ea;
   	return rc;
@@ -845,6 +854,13 @@ static inline int jmp_rel(struct x86_emulate_ctxt *ctxt, int rel)
   static int linear_read_system(struct x86_emulate_ctxt *ctxt, ulong linear,
   			      void *data, unsigned size)
   {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
   	return ctxt->ops->read_std(ctxt, linear, data, size, &ctxt->exception, true);
   }
 
@@ -852,6 +868,13 @@ static int linear_write_system(struct x86_emulate_ctxt *ctxt,
   			       ulong linear, void *data,
   			       unsigned int size)
   {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
   	return ctxt->ops->write_std(ctxt, linear, data, size, &ctxt->exception, true);
   }
 
@@ -907,7 +930,7 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctxt *ctxt, int op_size)
   	 * against op_size.
   	 */
   	rc = __linearize(ctxt, addr, &max_size, 0, false, true, ctxt->mode,
-			 &linear);
+			 &linear, 0);
   	if (unlikely(rc != X86EMUL_CONTINUE))
   		return rc;
 
@@ -3432,8 +3455,11 @@ static int em_invlpg(struct x86_emulate_ctxt *ctxt)
   {
   	int rc;
   	ulong linear;
+	unsigned max_size;
 
-	rc = linearize(ctxt, ctxt->src.addr.mem, 1, false, &linear);
+	/* LASS doesn't apply to the memory address for invlpg */
+	rc = __linearize(ctxt, ctxt->src.addr.mem, &max_size, 1, false, false,
+	     ctxt->mode, &linear, KVM_X86_EMULFLAG_SKIP_LASS);
   	if (rc == X86EMUL_CONTINUE)
   		ctxt->ops->invlpg(ctxt, linear);
   	/* Disable writeback. */
arch/x86/kvm/vmx/nested.c +3 -0
index c8ae9d0e59b3..55c88c4593a6 100644
@@ -4974,6 +4974,9 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification,
   		 * destination for long mode!
   		 */
   		exn = is_noncanonical_address(*ret, vcpu);
+
+		if (!exn)
+			exn = __vmx_check_lass(vcpu, 0, *ret, 0);
   	} else {
   		/*
   		 * When not in long mode, the virtual/linear address is
arch/x86/kvm/vmx/sgx.c +2 -0
index b12da2a6dec9..30cb5d0980be 100644
@@ -37,6 +37,8 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset,
   		fault = true;
   	} else if (likely(is_long_mode(vcpu))) {
   		fault = is_noncanonical_address(*gva, vcpu);
+		if (!fault)
+			fault = __vmx_check_lass(vcpu, 0, *gva, 0);
   	} else {
   		*gva &= 0xffffffff;
   		fault = (s.unusable) ||
#17 lore raw

Re: [PATCH 4/6] KVM: x86: LASS protection on KVM emulation when LASS enabled

In reply to Zeng Guang
Quoted 75 lines · Zeng Guang

Do LASS violation check for instructions emulated by KVM. Note that for instructions executed in the guest directly, hardware will perform the check.

Not all instruction emulation leads to accesses to guest linear addresses because 1) some instrutions like CPUID, RDMSR, don't take memory as operands 2) instruction fetch in most cases is already done inside the guest.

Four cases in which kvm may access guest linear addresses are identified by code inspection:
- KVM emulator uses segmented address for instruction fetches or data accesses.
- For implicit data access, KVM emulator gets address to a system data structure(GDT/LDT/IDT/TR).
- For VMX instruction emulation, KVM gets the address from "VM-exit instruction information" field in VMCS.
- For SGX ENCLS instruction emulation, KVM gets the address from registers.

LASS violation check applies to these linear address so as to enforce mode-based protections as hardware behaves.

As exceptions, the target memory address of emulation of invlpg, branch and call instructions doesn't require LASS violation check.

---
 arch/x86/kvm/emulate.c    | 36 +++++++++++++++++++++++++++++++-----
 arch/x86/kvm/vmx/nested.c |  3 +++
 arch/x86/kvm/vmx/sgx.c    |  2 ++
 3 files changed, 36 insertions(+), 5 deletions(-)
arch/x86/kvm/emulate.c +8 -2
index 5cc3efa0e21c..a9a022fd712e 100644
@@ -687,7 +687,8 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 				       struct segmented_address addr,
 				       unsigned *max_size, unsigned size,
 				       bool write, bool fetch,
-				       enum x86emul_mode mode, ulong *linear)
+				       enum x86emul_mode mode, ulong *linear,
+				       u64 flags)
 {
 	struct desc_struct desc;
 	bool usable;
@@ -695,6 +696,7 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 	u32 lim;
 	u16 sel;
 	u8  va_bits;
+	u64 access = fetch ? PFERR_FETCH_MASK : 0;
 
 	la = seg_base(ctxt, addr.seg) + addr.ea;
 	*max_size = 0;
@@ -740,6 +742,10 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 		}
 		break;
 	}
+
+	if (ctxt->ops->check_lass(ctxt, access, *linear, flags))
+		goto bad;
+
 	if (la & (insn_alignment(ctxt, size) - 1))
 		return emulate_gp(ctxt, 0);
 	return X86EMUL_CONTINUE;
@@ -757,7 +763,7 @@ static int linearize(struct x86_emulate_ctxt *ctxt,
 {
 	unsigned max_size;
 	return __linearize(ctxt, addr, &max_size, size, write, false,
-			   ctxt->mode, linear);
+			   ctxt->mode, linear, 0);
 }
 
 static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)

arch/x86/kvm/emulate.c static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst) · line 782

Quoted 8 lines
arch/x86/kvm/emulate.c +4 -1
@@ -770,7 +776,10 @@ static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
 
 	if (ctxt->op_bytes != sizeof(unsigned long))
 		addr.ea = dst & ((1UL << (ctxt->op_bytes << 3)) - 1);
-	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, &linear);
+
+	/* LASS doesn't apply to address for branch and call instructions */
+	rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode,
+	     &linear, KVM_X86_EMULFLAG_SKIP_LASS);

The emulator.c is common part of x86, so may more common abstraction like permiession_check_before_paging better ? Let's also wait other guy's input for this.

Quoted 83 lines
arch/x86/kvm/emulate.c +19 -2
 	if (rc == X86EMUL_CONTINUE)
 		ctxt->_eip = addr.ea;
 	return rc;
@@ -845,6 +854,13 @@ static inline int jmp_rel(struct x86_emulate_ctxt *ctxt, int rel)
 static int linear_read_system(struct x86_emulate_ctxt *ctxt, ulong linear,
 			      void *data, unsigned size)
 {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
 	return ctxt->ops->read_std(ctxt, linear, data, size, &ctxt->exception, true);
 }
 
@@ -852,6 +868,13 @@ static int linear_write_system(struct x86_emulate_ctxt *ctxt,
 			       ulong linear, void *data,
 			       unsigned int size)
 {
+	if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) {
+		ctxt->exception.vector = GP_VECTOR;
+		ctxt->exception.error_code = 0;
+		ctxt->exception.error_code_valid = true;
+		return X86EMUL_PROPAGATE_FAULT;
+	}
+
 	return ctxt->ops->write_std(ctxt, linear, data, size, &ctxt->exception, true);
 }
 
@@ -907,7 +930,7 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctxt *ctxt, int op_size)
 	 * against op_size.
 	 */
 	rc = __linearize(ctxt, addr, &max_size, 0, false, true, ctxt->mode,
-			 &linear);
+			 &linear, 0);
 	if (unlikely(rc != X86EMUL_CONTINUE))
 		return rc;
 
@@ -3432,8 +3455,11 @@ static int em_invlpg(struct x86_emulate_ctxt *ctxt)
 {
 	int rc;
 	ulong linear;
+	unsigned max_size;
 
-	rc = linearize(ctxt, ctxt->src.addr.mem, 1, false, &linear);
+	/* LASS doesn't apply to the memory address for invlpg */
+	rc = __linearize(ctxt, ctxt->src.addr.mem, &max_size, 1, false, false,
+	     ctxt->mode, &linear, KVM_X86_EMULFLAG_SKIP_LASS);
 	if (rc == X86EMUL_CONTINUE)
 		ctxt->ops->invlpg(ctxt, linear);
 	/* Disable writeback. */
arch/x86/kvm/vmx/nested.c +3 -0
index c8ae9d0e59b3..55c88c4593a6 100644
@@ -4974,6 +4974,9 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification,
 		 * destination for long mode!
 		 */
 		exn = is_noncanonical_address(*ret, vcpu);
+
+		if (!exn)
+			exn = __vmx_check_lass(vcpu, 0, *ret, 0);
 	} else {
 		/*
 		 * When not in long mode, the virtual/linear address is
arch/x86/kvm/vmx/sgx.c +2 -1
index b12da2a6dec9..30cb5d0980be 100644
@@ -37,6 +37,8 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset,
 		fault = true;
 	} else if (likely(is_long_mode(vcpu))) {
 		fault = is_noncanonical_address(*gva, vcpu);
+		if (!fault)
+			fault = __vmx_check_lass(vcpu, 0, *gva, 0);
 	} else {
 		*gva &= 0xffffffff;
 		fault = (s.unusable) ||
--

2.27.0

author
#18 lore raw

PATCH 5/6 KVM: x86: Advertise LASS CPUID to user space

In reply to Zeng Guang

LASS (Linear-address space separation) is an independent mechanism to enforce the mode-based protection that can prevent user-mode accesses to supervisor-mode addresses, and vice versa. Because the LASS protections are applied before paging, malicious software can not acquire any paging-based timing information to compromise the security of system.

The CPUID bit definition to support LASS:
CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6]

Advertise LASS to user space to support LASS virtualization.

---
 arch/x86/kvm/cpuid.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +2 -2
index ba7f7abc8964..5facb8037140 100644
@@ -663,8 +663,8 @@ void kvm_set_cpu_caps(void)
 		kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD);
 
 	kvm_cpu_cap_mask(CPUID_7_1_EAX,
-		F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | F(AMX_FP16) |
-		F(AVX_IFMA)
+		F(AVX_VNNI) | F(AVX512_BF16) | F(LASS) | F(CMPCCXADD) |
+		F(AMX_FP16) | F(AVX_IFMA)
 	);
 
 	kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX,
Signature · 1 line
2.27.0
author
#19 lore raw

PATCH 6/6 KVM: x86: Set KVM LASS based on hardware capability

In reply to Zeng Guang

Host kernel may clear LASS capability in boot_cpu_data.x86_capability besides explicitly using clearcpuid parameter. That will cause guest not being able to manage LASS independently. So set KVM LASS directly based on hardware capability to eliminate the dependency.

Add new helper functions to facilitate getting result of CPUID sub-leaf.

---
 arch/x86/include/asm/cpuid.h | 36 ++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/cpuid.c         |  4 ++++
 2 files changed, 40 insertions(+)
arch/x86/include/asm/cpuid.h +36 -0
index 9bee3e7bf973..a25dd00b7c0a 100644
@@ -127,6 +127,42 @@ static inline unsigned int cpuid_edx(unsigned int op)
 	return edx;
 }
 
+static inline unsigned int cpuid_count_eax(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return eax;
+}
+
+static inline unsigned int cpuid_count_ebx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return ebx;
+}
+
+static inline unsigned int cpuid_count_ecx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return ecx;
+}
+
+static inline unsigned int cpuid_count_edx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return edx;
+}
+
 static __always_inline bool cpuid_function_is_indexed(u32 function)
 {
 	switch (function) {
arch/x86/kvm/cpuid.c +4 -0
index 5facb8037140..e99b99ebe1fe 100644
@@ -667,6 +667,10 @@ void kvm_set_cpu_caps(void)
 		F(AMX_FP16) | F(AVX_IFMA)
 	);
 
+	/* Set LASS based on hardware capability */
+	if (cpuid_count_eax(7, 1) & F(LASS))
+		kvm_cpu_cap_set(X86_FEATURE_LASS);
+
 	kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX,
 		F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) | F(PREFETCHITI)
 	);
Signature · 1 line
2.27.0
#20 lore raw

Re: [PATCH 6/6] KVM: x86: Set KVM LASS based on hardware capability

Quoted 1 line · Zeng Guang

Host kernel may clear LASS capability in boot_cpu_data.x86_capability

Is there some option to do it?

Quoted 74 lines

besides explicitly using clearcpuid parameter. That will cause guest not being able to manage LASS independently. So set KVM LASS directly based on hardware capability to eliminate the dependency.

Add new helper functions to facilitate getting result of CPUID sub-leaf.

---
  arch/x86/include/asm/cpuid.h | 36 ++++++++++++++++++++++++++++++++++++
  arch/x86/kvm/cpuid.c         |  4 ++++
  2 files changed, 40 insertions(+)
arch/x86/include/asm/cpuid.h +36 -0
index 9bee3e7bf973..a25dd00b7c0a 100644
@@ -127,6 +127,42 @@ static inline unsigned int cpuid_edx(unsigned int op)
  	return edx;
  }
 
+static inline unsigned int cpuid_count_eax(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return eax;
+}
+
+static inline unsigned int cpuid_count_ebx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return ebx;
+}
+
+static inline unsigned int cpuid_count_ecx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return ecx;
+}
+
+static inline unsigned int cpuid_count_edx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return edx;
+}
+
  static __always_inline bool cpuid_function_is_indexed(u32 function)
  {
  	switch (function) {
arch/x86/kvm/cpuid.c +4 -0
index 5facb8037140..e99b99ebe1fe 100644
@@ -667,6 +667,10 @@ void kvm_set_cpu_caps(void)
  		F(AMX_FP16) | F(AVX_IFMA)
  	);
 
+	/* Set LASS based on hardware capability */
+	if (cpuid_count_eax(7, 1) & F(LASS))
+		kvm_cpu_cap_set(X86_FEATURE_LASS);
+
  	kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX,
  		F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) | F(PREFETCHITI)
  	);
author
#21 lore raw

Re: [PATCH 6/6] KVM: x86: Set KVM LASS based on hardware capability

Quoted 4 lines · Binbin Wu
Quoted 1 line · Zeng Guang

Host kernel may clear LASS capability in boot_cpu_data.x86_capability

Is there some option to do it?

Kernel supporting LASS will turn off the LASS capability with specific option, e.g. "vsyscall=emulate".

Quoted 74 lines
Quoted 74 lines

besides explicitly using clearcpuid parameter. That will cause guest not being able to manage LASS independently. So set KVM LASS directly based on hardware capability to eliminate the dependency.

Add new helper functions to facilitate getting result of CPUID sub-leaf.

---
   arch/x86/include/asm/cpuid.h | 36 ++++++++++++++++++++++++++++++++++++
   arch/x86/kvm/cpuid.c         |  4 ++++
   2 files changed, 40 insertions(+)
arch/x86/include/asm/cpuid.h +36 -0
index 9bee3e7bf973..a25dd00b7c0a 100644
@@ -127,6 +127,42 @@ static inline unsigned int cpuid_edx(unsigned int op)
   	return edx;
   }
 
+static inline unsigned int cpuid_count_eax(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return eax;
+}
+
+static inline unsigned int cpuid_count_ebx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return ebx;
+}
+
+static inline unsigned int cpuid_count_ecx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return ecx;
+}
+
+static inline unsigned int cpuid_count_edx(unsigned int op, int count)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	cpuid_count(op, count, &eax, &ebx, &ecx, &edx);
+
+	return edx;
+}
+
   static __always_inline bool cpuid_function_is_indexed(u32 function)
   {
   	switch (function) {
arch/x86/kvm/cpuid.c +4 -0
index 5facb8037140..e99b99ebe1fe 100644
@@ -667,6 +667,10 @@ void kvm_set_cpu_caps(void)
   		F(AMX_FP16) | F(AVX_IFMA)
   	);
 
+	/* Set LASS based on hardware capability */
+	if (cpuid_count_eax(7, 1) & F(LASS))
+		kvm_cpu_cap_set(X86_FEATURE_LASS);
+
   	kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX,
   		F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) | F(PREFETCHITI)
   	);
#22 lore raw

Re: [PATCH 6/6] KVM: x86: Set KVM LASS based on hardware capability

In reply to Zeng Guang
Quoted 4 lines · Zeng Guang

Host kernel may clear LASS capability in boot_cpu_data.x86_capability besides explicitly using clearcpuid parameter. That will cause guest not being able to manage LASS independently. So set KVM LASS directly based on hardware capability to eliminate the dependency.

...

arch/x86/kvm/cpuid.c void kvm_set_cpu_caps(void) · line 673

Quoted 4 lines
arch/x86/kvm/cpuid.c +4 -0
+	/* Set LASS based on hardware capability */
+	if (cpuid_count_eax(7, 1) & F(LASS))
+		kvm_cpu_cap_set(X86_FEATURE_LASS);
+

What if LASS is cleared in boot_cpu_data because not all CPUs support LASS?

In arch/x86/kernel/cpu/common.c, identify_cpu() clears features which are not supported by all CPUs:

	/*
	 * On SMP, boot_cpu_data holds the common feature set between
	 * all CPUs; so make sure that we indicate which features are
	 * common between the CPUs.  The first time this routine gets
	 * executed, c == &boot_cpu_data.
	 */
	if (c != &boot_cpu_data) {
		/* AND the already accumulated flags with these */
		for (i = 0; i < NCAPINTS; i++)
			boot_cpu_data.x86_capability[i] &= c->x86_capability[i];

LA57 seems to have the same issue. We may need to add some checks for LA57 in KVM's cpu hotplug callback.

Quoted 6 lines
	kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX,
		F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) | F(PREFETCHITI)
	);
Signature · 1 line
2.27.0
#23 lore raw
In reply to Zeng Guang
Quoted 6 lines · Zeng Guang

Linear Address Space Separation (LASS)[1] is a new mechanism that enforces the same mode-based protections as paging, i.e. SMAP/SMEP but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, "probes" by malicious software will provide no paging-based timing information.

Quoted 8 lines

LASS works in long mode and partitions the 64-bit canonical linear address space into two halves:
1. Lower half (LA[63]=0) --> user space
2. Upper half (LA[63]=1) --> kernel space

When LASS is enabled, a general protection #GP fault or a stack fault #SS will be generated if software accesses the address from the half in which it resides to another half,

The accessor's mode is based on CPL, not the address range, so it feels a bit inaccurate of descripton "in which it resides".

Quoted 6 lines

e.g., either from user space to
upper half, or from kernel space to lower half. This protection applies to data access, code execution.

This series add KVM LASS virtualization support.

Quoted 8 lines

When platform has LASS capability, KVM requires to expose this feature to guest VM enumerated by CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], and allow guest to enable it via CR4.LASS[bit 27] on demand. For instruction executed in the guest directly, hardware will perform the LASS violation check, while KVM also needs to apply LASS to instructions emulated by software and injects #GP or #SS fault to the guest.

Following LASS voilations check will be taken on KVM emulation path.

/s/voilations/violations

Quoted 15 lines

User-mode access to supervisor space address:

         LA[bit 63] && (CPL == 3)
Supervisor-mode access to user space address:
         Instruction fetch: !LA[bit 63] && (CPL < 3)
         Data access: !LA[bit 63] && (CR4.SMAP==1) && ((RFLAGS.AC == 0 &&
                      CPL < 3) || Implicit supervisor access)

We tested the basic function of LASS virtualization including LASS enumeration and enabling in non-root and nested environment. As current KVM unittest framework is not compatible to LASS rule that kernel should run in the upper half, we use kernel module and application test to verify LASS functionalities in guest instead. The data access related x86 emulator code is verified with forced emulation prefix (FEP) mechanism. Other test cases are working in progress.

Quoted 3 lines

How to add tests for LASS in KUT or kselftest is still under investigation.

[1] Intel Architecutre Instruction Set Extensions and Future Features

/s/Architecutre/Architecture

Quoted 25 lines

Programming Reference: Chapter Linear Address Space Separation (LASS)
https://cdrdv2.intel.com/v1/dl/getContent/671368

Zeng Guang (6):
KVM: x86: Virtualize CR4.LASS
KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check
KVM: x86: Add emulator helper for LASS violation check
KVM: x86: LASS protection on KVM emulation when LASS enabled
KVM: x86: Advertise LASS CPUID to user space
KVM: x86: Set KVM LASS based on hardware capability

  arch/x86/include/asm/cpuid.h       | 36 +++++++++++++++++++
  arch/x86/include/asm/kvm-x86-ops.h |  1 +
  arch/x86/include/asm/kvm_host.h    |  7 +++-
  arch/x86/kvm/cpuid.c               |  8 +++--
  arch/x86/kvm/emulate.c             | 36 ++++++++++++++++---
  arch/x86/kvm/kvm_emulate.h         |  1 +
  arch/x86/kvm/vmx/nested.c          |  3 ++
  arch/x86/kvm/vmx/sgx.c             |  2 ++
  arch/x86/kvm/vmx/vmx.c             | 58 ++++++++++++++++++++++++++++++
  arch/x86/kvm/vmx/vmx.h             |  2 ++
  arch/x86/kvm/x86.c                 |  9 +++++
  arch/x86/kvm/x86.h                 |  2 ++
  12 files changed, 157 insertions(+), 8 deletions(-)
author
#24 lore raw
Quoted 12 lines · Binbin Wu
Quoted 11 lines · Zeng Guang

Linear Address Space Separation (LASS)[1] is a new mechanism that enforces the same mode-based protections as paging, i.e. SMAP/SMEP but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, "probes" by malicious software will provide no paging-based timing information.

LASS works in long mode and partitions the 64-bit canonical linear address space into two halves:
1. Lower half (LA[63]=0) --> user space
2. Upper half (LA[63]=1) --> kernel space

Quoted 6 lines
Quoted 3 lines

When LASS is enabled, a general protection #GP fault or a stack fault #SS will be generated if software accesses the address from the half in which it resides to another half,

The accessor's mode is based on CPL, not the address range, so it feels a bit inaccurate of descripton "in which it resides".

This is alternative description to implicitly signify the privilege level, i.e. code running in upper half means it is in supervisor mode, otherwise it's in user mode.  :)

Quoted 32 lines
Quoted 6 lines

e.g., either from user space to
upper half, or from kernel space to lower half. This protection applies to data access, code execution.

This series add KVM LASS virtualization support.

Quoted 8 lines

When platform has LASS capability, KVM requires to expose this feature to guest VM enumerated by CPUID.(EAX=07H.ECX=1):EAX.LASS[bit 6], and allow guest to enable it via CR4.LASS[bit 27] on demand. For instruction executed in the guest directly, hardware will perform the LASS violation check, while KVM also needs to apply LASS to instructions emulated by software and injects #GP or #SS fault to the guest.

Following LASS voilations check will be taken on KVM emulation path.

/s/voilations/violations

Quoted 15 lines

User-mode access to supervisor space address:

          LA[bit 63] && (CPL == 3)
Supervisor-mode access to user space address:
          Instruction fetch: !LA[bit 63] && (CPL < 3)
          Data access: !LA[bit 63] && (CR4.SMAP==1) && ((RFLAGS.AC == 0 &&
                       CPL < 3) || Implicit supervisor access)

We tested the basic function of LASS virtualization including LASS enumeration and enabling in non-root and nested environment. As current KVM unittest framework is not compatible to LASS rule that kernel should run in the upper half, we use kernel module and application test to verify LASS functionalities in guest instead. The data access related x86 emulator code is verified with forced emulation prefix (FEP) mechanism. Other test cases are working in progress.

Quoted 5 lines
Quoted 3 lines

How to add tests for LASS in KUT or kselftest is still under investigation.

[1] Intel Architecutre Instruction Set Extensions and Future Features

/s/Architecutre/Architecture

Sorry for typos above. Thanks.

Quoted 25 lines
Quoted 25 lines

Programming Reference: Chapter Linear Address Space Separation (LASS)
https://cdrdv2.intel.com/v1/dl/getContent/671368

Zeng Guang (6):
KVM: x86: Virtualize CR4.LASS
KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check
KVM: x86: Add emulator helper for LASS violation check
KVM: x86: LASS protection on KVM emulation when LASS enabled
KVM: x86: Advertise LASS CPUID to user space
KVM: x86: Set KVM LASS based on hardware capability

   arch/x86/include/asm/cpuid.h       | 36 +++++++++++++++++++
   arch/x86/include/asm/kvm-x86-ops.h |  1 +
   arch/x86/include/asm/kvm_host.h    |  7 +++-
   arch/x86/kvm/cpuid.c               |  8 +++--
   arch/x86/kvm/emulate.c             | 36 ++++++++++++++++---
   arch/x86/kvm/kvm_emulate.h         |  1 +
   arch/x86/kvm/vmx/nested.c          |  3 ++
   arch/x86/kvm/vmx/sgx.c             |  2 ++
   arch/x86/kvm/vmx/vmx.c             | 58 ++++++++++++++++++++++++++++++
   arch/x86/kvm/vmx/vmx.h             |  2 ++
   arch/x86/kvm/x86.c                 |  9 +++++
   arch/x86/kvm/x86.h                 |  2 ++
   12 files changed, 157 insertions(+), 8 deletions(-)