Skip to contentSkip to conversation

x86/kvm: Basic split lock #AC handling

PATCH Applied Thomas Gleixner 4 participants 10 messages Apr 10, 2020 – Apr 14, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover x86/kvm: Basic split lock #AC handling

This is a reworked version of the patches posted by Sean:

https://lore.kernel.org/r/[email protected]

The changes vs. this are:

1) Use a separate function for guest split lock handling

2) Force SIGBUS when SLD mode fatal

3) Rename the misnomed helper function which decides whether #AC is injected into the guest or not and move the feature check and the comments into that helper.

Thanks,

tglx

#1 lore raw

PATCH 1/3 x86/split_lock: Provide handle_guest_split_lock()

From: Thomas Gleixner <[email protected]>

Without at least minimal handling for split lock detection induced #AC, VMX will just run into the same problem as the VMWare hypervisor, which was reported by Kenneth.

It will inject the #AC blindly into the guest whether the guest is prepared or not.

Provide a function for guest mode which acts depending on the host SLD mode. If mode == sld_warn, treat it like user space, i.e. emit a warning disable SLD and mark the task accordingly. Otherwise force SIGBUS.

Cc "Kenneth R. Crudup" <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc Xiaoyao Li <[email protected]>
Cc Nadav Amit <[email protected]>
Cc Thomas Hellstrom <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc Tony Luck <[email protected]>
---
 arch/x86/include/asm/cpu.h  |    1 +
 arch/x86/kernel/cpu/intel.c |   33 ++++++++++++++++++++++++++++-----
 2 files changed, 29 insertions(+), 5 deletions(-)
arch/x86/include/asm/cpu.h +1 -0
@@ -44,6 +44,7 @@ unsigned int x86_stepping(unsigned int s
 extern void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c);
 extern void switch_to_sld(unsigned long tifn);
 extern bool handle_user_split_lock(struct pt_regs *regs, long error_code);
+extern bool handle_guest_split_lock(unsigned long ip);
 #else
 static inline void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c) {}
 static inline void switch_to_sld(unsigned long tifn) {}
arch/x86/kernel/cpu/intel.c +28 -5
@@ -21,6 +21,7 @@
 #include <asm/elf.h>
 #include <asm/cpu_device_id.h>
 #include <asm/cmdline.h>
+#include <asm/traps.h>
 
 #ifdef CONFIG_X86_64
 #include <linux/topology.h>
@@ -1066,13 +1067,10 @@ static void split_lock_init(void)
 	split_lock_verify_msr(sld_state != sld_off);
 }
 
-bool handle_user_split_lock(struct pt_regs *regs, long error_code)
+static void split_lock_warn(unsigned long ip)
 {
-	if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
-		return false;
-
 	pr_warn_ratelimited("#AC: %s/%d took a split_lock trap at address: 0x%lx\n",
-			    current->comm, current->pid, regs->ip);
+			    current->comm, current->pid, ip);
 
 	/*
 	 * Disable the split lock detection for this task so it can make
@@ -1081,6 +1079,31 @@ bool handle_user_split_lock(struct pt_re
 	 */
 	sld_update_msr(false);
 	set_tsk_thread_flag(current, TIF_SLD);
+}
+
+bool handle_guest_split_lock(unsigned long ip)
+{
+	if (sld_state == sld_warn) {
+		split_lock_warn(ip);
+		return true;
+	}
+
+	pr_warn_once("#AC: %s/%d %s split_lock trap at address: 0x%lx\n",
+		     current->comm, current->pid,
+		     sld_state == sld_fatal ? "fatal" : "bogus", ip);
+
+	current->thread.error_code = 0;
+	current->thread.trap_nr = X86_TRAP_AC;
+	force_sig_fault(SIGBUS, BUS_ADRALN, NULL);
+	return false;
+}
+EXPORT_SYMBOL_GPL(handle_guest_split_lock);
+
+bool handle_user_split_lock(struct pt_regs *regs, long error_code)
+{
+	if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
+		return false;
+	split_lock_warn(regs->ip);
 	return true;
 }
 
#2 lore raw

[tip: x86/urgent] x86/split_lock: Provide handle_guest_split_lock()

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     d7e94dbdac1a40924626b0efc7ff530c8baf5e4a
Gitweb:        https://git.kernel.org/tip/d7e94dbdac1a40924626b0efc7ff530c8baf5e4a
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Fri, 10 Apr 2020 13:54:00 +02:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Sat, 11 Apr 2020 16:39:30 +02:00

x86/split_lock: Provide handle_guest_split_lock()

Without at least minimal handling for split lock detection induced #AC, VMX will just run into the same problem as the VMWare hypervisor, which was reported by Kenneth.

It will inject the #AC blindly into the guest whether the guest is prepared or not.

Provide a function for guest mode which acts depending on the host SLD mode. If mode == sld_warn, treat it like user space, i.e. emit a warning, disable SLD and mark the task accordingly. Otherwise force SIGBUS.

[ bp: Add a !CPU_SUP_INTEL stub for handle_guest_split_lock(). ]

---
 arch/x86/include/asm/cpu.h  |  6 ++++++
 arch/x86/kernel/cpu/intel.c | 33 ++++++++++++++++++++++++++++-----
 2 files changed, 34 insertions(+), 5 deletions(-)
arch/x86/include/asm/cpu.h +6 -0
index ff6f3ca..dd17c2d 100644
@@ -44,6 +44,7 @@ unsigned int x86_stepping(unsigned int sig);
 extern void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c);
 extern void switch_to_sld(unsigned long tifn);
 extern bool handle_user_split_lock(struct pt_regs *regs, long error_code);
+extern bool handle_guest_split_lock(unsigned long ip);
 #else
 static inline void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c) {}
 static inline void switch_to_sld(unsigned long tifn) {}
@@ -51,5 +52,10 @@ static inline bool handle_user_split_lock(struct pt_regs *regs, long error_code)
 {
 	return false;
 }
+
+static inline bool handle_guest_split_lock(unsigned long ip)
+{
+	return false;
+}
 #endif
 #endif /* _ASM_X86_CPU_H */
arch/x86/kernel/cpu/intel.c +28 -5
index 9a26e97..bf08d45 100644
@@ -21,6 +21,7 @@
 #include <asm/elf.h>
 #include <asm/cpu_device_id.h>
 #include <asm/cmdline.h>
+#include <asm/traps.h>
 
 #ifdef CONFIG_X86_64
 #include <linux/topology.h>
@@ -1066,13 +1067,10 @@ static void split_lock_init(void)
 	split_lock_verify_msr(sld_state != sld_off);
 }
 
-bool handle_user_split_lock(struct pt_regs *regs, long error_code)
+static void split_lock_warn(unsigned long ip)
 {
-	if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
-		return false;
-
 	pr_warn_ratelimited("#AC: %s/%d took a split_lock trap at address: 0x%lx\n",
-			    current->comm, current->pid, regs->ip);
+			    current->comm, current->pid, ip);
 
 	/*
 	 * Disable the split lock detection for this task so it can make
@@ -1081,6 +1079,31 @@ bool handle_user_split_lock(struct pt_regs *regs, long error_code)
 	 */
 	sld_update_msr(false);
 	set_tsk_thread_flag(current, TIF_SLD);
+}
+
+bool handle_guest_split_lock(unsigned long ip)
+{
+	if (sld_state == sld_warn) {
+		split_lock_warn(ip);
+		return true;
+	}
+
+	pr_warn_once("#AC: %s/%d %s split_lock trap at address: 0x%lx\n",
+		     current->comm, current->pid,
+		     sld_state == sld_fatal ? "fatal" : "bogus", ip);
+
+	current->thread.error_code = 0;
+	current->thread.trap_nr = X86_TRAP_AC;
+	force_sig_fault(SIGBUS, BUS_ADRALN, NULL);
+	return false;
+}
+EXPORT_SYMBOL_GPL(handle_guest_split_lock);
+
+bool handle_user_split_lock(struct pt_regs *regs, long error_code)
+{
+	if ((regs->flags & X86_EFLAGS_AC) || sld_state == sld_fatal)
+		return false;
+	split_lock_warn(regs->ip);
 	return true;
 }
 
#3 lore raw

PATCH 2/3 KVM: x86: Emulate split-lock access as a write in emulator

In reply to Thomas Gleixner

From: Xiaoyao Li <[email protected]>

Emulate split-lock accesses as writes if split lock detection is on to avoid #AC during emulation, which will result in a panic(). This should never occur for a well behaved guest, but a malicious guest can manipulate the TLB to trigger emulation of a locked instruction[1].

More discussion can be found [2][3].

[1] https://lkml.kernel.org/r/[email protected]
[2] https://lkml.kernel.org/r/[email protected]
[3] https://lkml.kernel.org/r/[email protected]

---
 arch/x86/kvm/x86.c |   12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +11 -1
@@ -5838,6 +5838,7 @@ static int emulator_cmpxchg_emulated(str
 {
 	struct kvm_host_map map;
 	struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
+	u64 page_line_mask;
 	gpa_t gpa;
 	char *kaddr;
 	bool exchanged;
@@ -5852,7 +5853,16 @@ static int emulator_cmpxchg_emulated(str
 	    (gpa & PAGE_MASK) == APIC_DEFAULT_PHYS_BASE)
 		goto emul_write;
 
-	if (((gpa + bytes - 1) & PAGE_MASK) != (gpa & PAGE_MASK))
+	/*
+	 * Emulate the atomic as a straight write to avoid #AC if SLD is
+	 * enabled in the host and the access splits a cache line.
+	 */
+	if (boot_cpu_has(X86_FEATURE_SPLIT_LOCK_DETECT))
+		page_line_mask = ~(cache_line_size() - 1);
+	else
+		page_line_mask = PAGE_MASK;
+
+	if (((gpa + bytes - 1) & page_line_mask) != (gpa & page_line_mask))
 		goto emul_write;
 
 	if (kvm_vcpu_map(vcpu, gpa_to_gfn(gpa), &map))
merge bot Applied
#4 lore raw

[tip: x86/urgent] KVM: x86: Emulate split-lock access as a write in emulator

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     9de6fe3c28d6d8feadfad907961f1f31b85c6985
Gitweb:        https://git.kernel.org/tip/9de6fe3c28d6d8feadfad907961f1f31b85c6985
Author:        Xiaoyao Li <[email protected]>
AuthorDate:    Fri, 10 Apr 2020 13:54:01 +02:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Sat, 11 Apr 2020 16:40:55 +02:00

KVM: x86: Emulate split-lock access as a write in emulator

Emulate split-lock accesses as writes if split lock detection is on to avoid #AC during emulation, which will result in a panic(). This should never occur for a well-behaved guest, but a malicious guest can manipulate the TLB to trigger emulation of a locked instruction[1].

More discussion can be found at [2][3].

[1] https://lkml.kernel.org/r/[email protected]
[2] https://lkml.kernel.org/r/[email protected]
[3] https://lkml.kernel.org/r/[email protected]

---
 arch/x86/kvm/x86.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +11 -1
index 027dfd2..3bf2eca 100644
@@ -5839,6 +5839,7 @@ static int emulator_cmpxchg_emulated(struct x86_emulate_ctxt *ctxt,
 {
 	struct kvm_host_map map;
 	struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
+	u64 page_line_mask;
 	gpa_t gpa;
 	char *kaddr;
 	bool exchanged;
@@ -5853,7 +5854,16 @@ static int emulator_cmpxchg_emulated(struct x86_emulate_ctxt *ctxt,
 	    (gpa & PAGE_MASK) == APIC_DEFAULT_PHYS_BASE)
 		goto emul_write;
 
-	if (((gpa + bytes - 1) & PAGE_MASK) != (gpa & PAGE_MASK))
+	/*
+	 * Emulate the atomic as a straight write to avoid #AC if SLD is
+	 * enabled in the host and the access splits a cache line.
+	 */
+	if (boot_cpu_has(X86_FEATURE_SPLIT_LOCK_DETECT))
+		page_line_mask = ~(cache_line_size() - 1);
+	else
+		page_line_mask = PAGE_MASK;
+
+	if (((gpa + bytes - 1) & page_line_mask) != (gpa & page_line_mask))
 		goto emul_write;
 
 	if (kvm_vcpu_map(vcpu, gpa_to_gfn(gpa), &map))
#5 lore raw

PATCH 3/3 KVM: VMX: Extend VMXs #AC interceptor to handle split lock #AC in guest

In reply to Thomas Gleixner

From: Xiaoyao Li <[email protected]>

Two types of #AC can be generated in Intel CPUs:
1. legacy alignment check #AC
2. split lock #AC

Reflect #AC back into the guest if the guest has legacy alignment checks enabled or if split lock detection is disabled.

If the #AC is not a legacy one and split lock detection is enabled, then invoke handle_guest_split_lock() which will either warn and disable split lock detection for this task or force SIGBUS on it.

[ tglx: Switch it to handle_guest_split_lock() and renamed the misnomed
helper function ]

---
 arch/x86/kvm/vmx/vmx.c |   37 ++++++++++++++++++++++++++++++++++---
 1 file changed, 34 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/vmx.c +34 -3
@@ -4592,6 +4592,26 @@ static int handle_machine_check(struct k
 	return 1;
 }
 
+/*
+ * If the host has split lock detection disabled, then #AC is
+ * unconditionally injected into the guest, which is the pre split lock
+ * detection behaviour.
+ *
+ * If the host has split lock detection enabled then #AC is
+ * only injected into the guest when:
+ *  - Guest CPL == 3 (user mode)
+ *  - Guest has #AC detection enabled in CR0
+ *  - Guest EFLAGS has AC bit set
+ */
+static inline bool guest_inject_ac(struct kvm_vcpu *vcpu)
+{
+	if (!boot_cpu_has(X86_FEATURE_SPLIT_LOCK_DETECT))
+		return true;
+
+	return vmx_get_cpl(vcpu) == 3 && kvm_read_cr0_bits(vcpu, X86_CR0_AM) &&
+	       (kvm_get_rflags(vcpu) & X86_EFLAGS_AC);
+}
+
 static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
@@ -4657,9 +4677,6 @@ static int handle_exception_nmi(struct k
 		return handle_rmode_exception(vcpu, ex_no, error_code);
 
 	switch (ex_no) {
-	case AC_VECTOR:
-		kvm_queue_exception_e(vcpu, AC_VECTOR, error_code);
-		return 1;
 	case DB_VECTOR:
 		dr6 = vmcs_readl(EXIT_QUALIFICATION);
 		if (!(vcpu->guest_debug &
@@ -4688,6 +4705,20 @@ static int handle_exception_nmi(struct k
 		kvm_run->debug.arch.pc = vmcs_readl(GUEST_CS_BASE) + rip;
 		kvm_run->debug.arch.exception = ex_no;
 		break;
+	case AC_VECTOR:
+		if (guest_inject_ac(vcpu)) {
+			kvm_queue_exception_e(vcpu, AC_VECTOR, error_code);
+			return 1;
+		}
+
+		/*
+		 * Handle split lock. Depending on detection mode this will
+		 * either warn and disable split lock detection for this
+		 * task or force SIGBUS on it.
+		 */
+		if (handle_guest_split_lock(kvm_rip_read(vcpu)))
+			return 1;
+		fallthrough;
 	default:
 		kvm_run->exit_reason = KVM_EXIT_EXCEPTION;
 		kvm_run->ex.exception = ex_no;
merge bot Applied
#6 lore raw

[tip: x86/urgent] KVM: VMX: Extend VMXs #AC interceptor to handle split lock #AC in guest

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     e6f8b6c12f03818baacc5f504fe83fa5e20771d6
Gitweb:        https://git.kernel.org/tip/e6f8b6c12f03818baacc5f504fe83fa5e20771d6
Author:        Xiaoyao Li <[email protected]>
AuthorDate:    Fri, 10 Apr 2020 13:54:02 +02:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Sat, 11 Apr 2020 16:42:41 +02:00

KVM: VMX: Extend VMXs #AC interceptor to handle split lock #AC in guest

Two types of #AC can be generated in Intel CPUs:
1. legacy alignment check #AC
2. split lock #AC

Reflect #AC back into the guest if the guest has legacy alignment checks enabled or if split lock detection is disabled.

If the #AC is not a legacy one and split lock detection is enabled, then invoke handle_guest_split_lock() which will either warn and disable split lock detection for this task or force SIGBUS on it.

[ tglx: Switch it to handle_guest_split_lock() and rename the misnamed
helper function. ]

---
 arch/x86/kvm/vmx/vmx.c | 37 ++++++++++++++++++++++++++++++++++---
 1 file changed, 34 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/vmx.c +34 -3
index 8959514..8305097 100644
@@ -4588,6 +4588,26 @@ static int handle_machine_check(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+/*
+ * If the host has split lock detection disabled, then #AC is
+ * unconditionally injected into the guest, which is the pre split lock
+ * detection behaviour.
+ *
+ * If the host has split lock detection enabled then #AC is
+ * only injected into the guest when:
+ *  - Guest CPL == 3 (user mode)
+ *  - Guest has #AC detection enabled in CR0
+ *  - Guest EFLAGS has AC bit set
+ */
+static inline bool guest_inject_ac(struct kvm_vcpu *vcpu)
+{
+	if (!boot_cpu_has(X86_FEATURE_SPLIT_LOCK_DETECT))
+		return true;
+
+	return vmx_get_cpl(vcpu) == 3 && kvm_read_cr0_bits(vcpu, X86_CR0_AM) &&
+	       (kvm_get_rflags(vcpu) & X86_EFLAGS_AC);
+}
+
 static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
@@ -4653,9 +4673,6 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 		return handle_rmode_exception(vcpu, ex_no, error_code);
 
 	switch (ex_no) {
-	case AC_VECTOR:
-		kvm_queue_exception_e(vcpu, AC_VECTOR, error_code);
-		return 1;
 	case DB_VECTOR:
 		dr6 = vmcs_readl(EXIT_QUALIFICATION);
 		if (!(vcpu->guest_debug &
@@ -4684,6 +4701,20 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 		kvm_run->debug.arch.pc = vmcs_readl(GUEST_CS_BASE) + rip;
 		kvm_run->debug.arch.exception = ex_no;
 		break;
+	case AC_VECTOR:
+		if (guest_inject_ac(vcpu)) {
+			kvm_queue_exception_e(vcpu, AC_VECTOR, error_code);
+			return 1;
+		}
+
+		/*
+		 * Handle split lock. Depending on detection mode this will
+		 * either warn and disable split lock detection for this
+		 * task or force SIGBUS on it.
+		 */
+		if (handle_guest_split_lock(kvm_rip_read(vcpu)))
+			return 1;
+		fallthrough;
 	default:
 		kvm_run->exit_reason = KVM_EXIT_EXCEPTION;
 		kvm_run->ex.exception = ex_no;
maintainer Acked-by on the v1 series
#7 lore raw
In reply to Thomas Gleixner
Quoted 10 lines · Thomas Gleixner

This is a reworked version of the patches posted by Sean:

https://lore.kernel.org/r/[email protected]

The changes vs. this are:

1) Use a separate function for guest split lock handling

2) Force SIGBUS when SLD mode fatal

Quoted 7 lines

3) Rename the misnomed helper function which decides whether #AC is injected into the guest or not and move the feature check and the comments into that helper.

Thanks,

tglx

Or just tell me if you want me to send it Linus's way.

Paolo

#8 lore raw
In reply to Thomas Gleixner
Quoted 9 lines · Thomas Gleixner

This is a reworked version of the patches posted by Sean:

https://lore.kernel.org/r/[email protected]

The changes vs. this are:

1) Use a separate function for guest split lock handling

2) Force SIGBUS when SLD mode fatal

Not that it matters as the code is correct, but I think you meant "when SLD mode off" here.

Quoted 3 lines

3) Rename the misnomed helper function which decides whether #AC is injected into the guest or not and move the feature check and the comments into that helper.

Thanks a bunch for helping push this along!

#9 lore raw

Sean,

Quoted 5 lines · Sean Christopherson
Quoted 4 lines · Thomas Gleixner

This is a reworked version of the patches posted by Sean:

https://lore.kernel.org/r/[email protected]

Quoted 8 lines
Quoted 5 lines

The changes vs. this are:

1) Use a separate function for guest split lock handling

2) Force SIGBUS when SLD mode fatal

Not that it matters as the code is correct, but I think you meant "when SLD mode off" here.

Actually for both fatal and off. The latter should never happen :)

Thanks,

tglx