Skip to contentSkip to conversation

x86/bus_lock: Enable bus lock detection

PATCH Superseded Fenghua Yu 2 participants 14 messages Nov 24, 2020 – Jan 27, 2021

Subsystems (2)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Fenghua Yu

author
#0 lore raw

v4 cover x86/bus_lock: Enable bus lock detection

A bus lock [1] is acquired through either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically >1000 cycles slower than an atomic operation within a cache line. It also disrupts performance on other cores.

Although split lock can be detected by #AC trap, the trap is triggered before the instruction acquires bus lock. This makes it difficult to mitigate bus lock (e.g. throttle the user application).

Some CPUs have ability to notify the kernel by an #DB trap after a user instruction acquires a bus lock and is executed. This allows the kernel to enforce user application throttling or mitigations.

#DB for bus lock detect fixes issues in #AC for split lock detect:
1) It's architectural ... just need to look at one CPUID bit to know it exists
2) The IA32_DEBUGCTL MSR, which reports bus lock in #DB, is per-thread. So each process or guest can have different behavior.
3) It has support for VMM/guests (new VMEXIT codes, etc).
4) It detects not only split locks but also bus locks from non-WB.

Hardware only generates #DB for bus lock detect when CPL>0 to avoid nested #DB from multiple bus locks while the first #DB is being handled.

Use the existing kernel command line option "split_lock_detect=" to handle #DB for bus lock:

split_lock_detect=
#AC for split lock #DB for bus lock

off Do nothing Do nothing

warn Kernel OOPs Warn once per task and
Warn once per task and and continues to run.
disable future checking When both features are
supported, warn in #DB

fatal Kernel OOPs Send SIGBUS to user.
Send SIGBUS to user When both features are
supported, split lock triggers #AC and bus lock from non-WB triggers #DB.

ratelimit:N Do nothing Limit bus lock rate to
N per second in the current non-root user.

Default split_lock_detect is "warn".

[1] Intel Instruction Set Extension Chapter 8: https://software.intel.com/sites/default/files/managed/c5/15/architecture-instruction-set-extensions-programming-reference.pdf

Change Log:
v4:
- Fix a ratelimit wording issue in the doc (Randy).
- Patch 4 is acked by Randy (Randy).

v3:
- Enable Bus Lock Detection when fatal to handle bus lock from non-WB (PeterZ).
- Add Acked-by: PeterZ in patch 2.

v2:
- Send SIGBUS in fatal case for bus lock #DB (PeterZ).

v1:
- Check bus lock bit by its positive polarity (Xiaoyao).
- Fix a few wording issues in the documentation (Randy).
[RFC v3 can be found at: https://lore.kernel.org/patchwork/cover/1329943/]

RFC v3:
- Remove DR6_RESERVED change (PeterZ).
- Simplify the documentation (Randy).

RFC v2:
- Architecture changed based on feedback from Thomas and PeterZ. #DB is
no longer generated for bus lock in ring0.
- Split the one single patch into four patches.
[RFC v1 can be found at: https://lore.kernel.org/lkml/[email protected]/]

Fenghua Yu (4):
x86/cpufeatures: Enumerate #DB for bus lock detection x86/bus_lock: Handle warn and fatal in #DB for bus lock x86/bus_lock: Set rate limit for bus lock Documentation/admin-guide: Change doc for split_lock_detect parameter

 .../admin-guide/kernel-parameters.txt         |  30 +++-
 arch/x86/include/asm/cpu.h                    |   9 +-
 arch/x86/include/asm/cpufeatures.h            |   1 +
 arch/x86/include/asm/msr-index.h              |   1 +
 arch/x86/include/uapi/asm/debugreg.h          |   1 +
 arch/x86/kernel/cpu/common.c                  |   2 +-
 arch/x86/kernel/cpu/intel.c                   | 155 +++++++++++++++---
 arch/x86/kernel/traps.c                       |   7 +
 include/linux/sched/user.h                    |   4 +-
 kernel/user.c                                 |   7 +
 10 files changed, 185 insertions(+), 32 deletions(-)
Signature · 1 line
2.29.2
author
#1 lore raw

PATCH v4 1/4 x86/cpufeatures: Enumerate #DB for bus lock detection

A bus lock is acquired though either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically >1000 cycles slower than an atomic operation within a cache line. It also disrupts performance on other cores.

Some CPUs have ability to notify the kernel by an #DB trap after a user instruction acquires a bus lock and is executed. This allows the kernel to enforce user application throttling or mitigations.

The CPU feature flag to be shown in /proc/cpuinfo will be "bus_lock_detect".

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index dad350d42ecf..f375d9cb8123 100644
@@ -352,6 +352,7 @@
 #define X86_FEATURE_AVX512_VPOPCNTDQ	(16*32+14) /* POPCNT for vectors of DW/QW */
 #define X86_FEATURE_LA57		(16*32+16) /* 5-level page tables */
 #define X86_FEATURE_RDPID		(16*32+22) /* RDPID instruction */
+#define X86_FEATURE_BUS_LOCK_DETECT	(16*32+24) /* Bus Lock detect */
 #define X86_FEATURE_CLDEMOTE		(16*32+25) /* CLDEMOTE instruction */
 #define X86_FEATURE_MOVDIRI		(16*32+27) /* MOVDIRI instruction */
 #define X86_FEATURE_MOVDIR64B		(16*32+28) /* MOVDIR64B instruction */
Signature · 1 line
2.29.2
#2 lore raw

Re: [PATCH v4 1/4] x86/cpufeatures: Enumerate #DB for bus lock detection

Quoted 8 lines · Fenghua Yu

A bus lock is acquired though either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically >1000 cycles slower than an atomic operation within a cache line. It also disrupts performance on other cores.

Some CPUs have ability to notify the kernel by an #DB trap after a user instruction acquires a bus lock and is executed. This allows the kernel to enforce user application throttling or mitigations.

That's nice, but how does that interact with a data breakpoint on the same location?

Also the information you pointed to in the cover letter

Quoted 2 lines

[1] Intel Instruction Set Extension Chapter 8:
https://software.intel.com/sites/default/files/managed/c5/15/architecture-instruction-set-extensions-programming-reference.pdf

does not contain anything which is even remotely related to this patch series. That chapter describes another bit in TEST_CTRL_MSR ...

Thanks,

tglx

#3 lore raw

RE: [PATCH v4 1/4] x86/cpufeatures: Enumerate #DB for bus lock detection

Hi, Thomas,

Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · Fenghua Yu

A bus lock is acquired though either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically

Quoted 1 line

1000 cycles slower than an atomic operation within a cache line. It

Quoted 8 lines
Quoted 5 lines

also disrupts performance on other cores.

Some CPUs have ability to notify the kernel by an #DB trap after a user instruction acquires a bus lock and is executed. This allows the kernel to enforce user application throttling or mitigations.

That's nice, but how does that interact with a data breakpoint on the same location?

If both data breakpoint and bus lock happen on the same location, the bus lock is handled first and then the data breakpoint is handled in the same exception:

1. If warn on bus lock, a rate limited warning is printed for the bus lock and then a SIGTRAP is sent to the user process.
2. If fatal on bus lock, a SIGBUS is sent to the user process for the bus lock and a SIGTRAP is also sent to the user process. I think the SIGBUS will be delivered first to the process and then SIGTRAP will be delivered to the process.
3. If ratelimit on bus lock, first the tasks in the user sleep for specified time, then SIGTRAP is sent to the user process.

Is the interaction OK?

Quoted 9 lines

Also the information you pointed to in the cover letter

Quoted 3 lines

[1] Intel Instruction Set Extension Chapter 8:
https://software.intel.com/sites/default/files/managed/c5/15/architect
ure-instruction-set-extensions-programming-reference.pdf

does not contain anything which is even remotely related to this patch series. That chapter describes another bit in TEST_CTRL_MSR ...

I think either I gave an old link or the content in the link was changed to an older version of ISE doc after this series was released.

Here is the new ISE doc and the bus lock exception is described in Chapter 9.
https://software.intel.com/content/dam/develop/public/us/en/documents/architecture-instruction-set-extensions-programming-reference.pdf

I'll update the link in the next version.

Thanks!

-Fenghua

#4 lore raw

RE: [PATCH v4 1/4] x86/cpufeatures: Enumerate #DB for bus lock detection

Fenghua,

Quoted 26 lines · Fenghua Yu
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · Fenghua Yu

A bus lock is acquired though either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically

Quoted 1 line

1000 cycles slower than an atomic operation within a cache line. It

Quoted 8 lines
Quoted 5 lines

also disrupts performance on other cores.

Some CPUs have ability to notify the kernel by an #DB trap after a user instruction acquires a bus lock and is executed. This allows the kernel to enforce user application throttling or mitigations.

That's nice, but how does that interact with a data breakpoint on the same location?

If both data breakpoint and bus lock happen on the same location, the bus lock is handled first and then the data breakpoint is handled in the same exception:

1. If warn on bus lock, a rate limited warning is printed for the bus lock and then a SIGTRAP is sent to the user process.
2. If fatal on bus lock, a SIGBUS is sent to the user process for the bus lock and a SIGTRAP is also sent to the user process. I think the SIGBUS will be delivered first to the process and then SIGTRAP will be delivered to the process.
3. If ratelimit on bus lock, first the tasks in the user sleep for specified time, then SIGTRAP is sent to the user process.

Quoted 1 line

Is the interaction OK?

The ordering is a software choice and fine with me as long as the hardware actually delivers both.

All of this information needs to be in the changelog of the relevant patches.

Thanks,

tglx

author
#5 lore raw

PATCH v4 2/4 x86/bus_lock: Handle warn and fatal in #DB for bus lock

In reply to Fenghua Yu

#DB for bus lock is enabled by bus lock detection bit 2 in DEBUGCTL MSR while #AC for split lock is enabled by split lock detection bit 29 in TEST_CTRL MSR.

Delivery of #DB for bus lock in userspace clears DR6[11]. To avoid confusion in identifying #DB, #DB handler sets the bit to 1 before returning to the interrupted task.

Use the existing kernel command line option "split_lock_detect=" to handle #DB for bus lock:

split_lock_detect=
#AC for split lock #DB for bus lock

off Do nothing Do nothing

warn Kernel OOPs Warn once per task and
Warn once per task and and continues to run.
disable future checking When both features are
supported, warn in #DB

fatal Kernel OOPs Send SIGBUS to user
Send SIGBUS to user When both features are
supported, split lock triggers #AC and bus lock from non-WB triggers #DB.

Default option is "warn".

Hardware only generates #DB for bus lock detect when CPL>0 to avoid nested #DB from multiple bus locks while the first #DB is being handled. So no need to handle #DB for bus lock detected in the kernel.

---
Change Log:
v3:
- Enable Bus Lock Detection when fatal to handle bus lock from non-WB (PeterZ).

v2:
- Send SIGBUS in fatal case for bus lock #DB (PeterZ).

v1::
- Check bus lock bit by its positive polarity (Xiaoyao).

RFC v3:
- Remove DR6_RESERVED change (PeterZ).

 arch/x86/include/asm/cpu.h           |   9 +-
 arch/x86/include/asm/msr-index.h     |   1 +
 arch/x86/include/uapi/asm/debugreg.h |   1 +
 arch/x86/kernel/cpu/common.c         |   2 +-
 arch/x86/kernel/cpu/intel.c          | 126 ++++++++++++++++++++++-----
 arch/x86/kernel/traps.c              |   7 ++
 6 files changed, 123 insertions(+), 23 deletions(-)
arch/x86/include/asm/cpu.h +7 -2
index da78ccbd493b..c16b9bab502f 100644
@@ -41,12 +41,13 @@ unsigned int x86_family(unsigned int sig);
 unsigned int x86_model(unsigned int sig);
 unsigned int x86_stepping(unsigned int sig);
 #ifdef CONFIG_CPU_SUP_INTEL
-extern void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c);
+extern void __init sld_setup(struct cpuinfo_x86 *c);
 extern void switch_to_sld(unsigned long tifn);
 extern bool handle_user_split_lock(struct pt_regs *regs, long error_code);
 extern bool handle_guest_split_lock(unsigned long ip);
+extern void handle_bus_lock(struct pt_regs *regs);
 #else
-static inline void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c) {}
+static inline void __init sld_setup(struct cpuinfo_x86 *c) {}
 static inline void switch_to_sld(unsigned long tifn) {}
 static inline bool handle_user_split_lock(struct pt_regs *regs, long error_code)
 {
@@ -57,6 +58,10 @@ static inline bool handle_guest_split_lock(unsigned long ip)
 {
 	return false;
 }
+
+static inline void handle_bus_lock(struct pt_regs *regs)
+{
+}
 #endif
 #ifdef CONFIG_IA32_FEAT_CTL
 void init_ia32_feat_ctl(struct cpuinfo_x86 *c);
arch/x86/include/asm/msr-index.h +1 -0
index 972a34d93505..62f7534e0855 100644
@@ -264,6 +264,7 @@
 #define DEBUGCTLMSR_LBR			(1UL <<  0) /* last branch recording */
 #define DEBUGCTLMSR_BTF_SHIFT		1
 #define DEBUGCTLMSR_BTF			(1UL <<  1) /* single-step on branches */
+#define DEBUGCTLMSR_BUS_LOCK_DETECT	(1UL <<  2) /* Bus lock detect */
 #define DEBUGCTLMSR_TR			(1UL <<  6)
 #define DEBUGCTLMSR_BTS			(1UL <<  7)
 #define DEBUGCTLMSR_BTINT		(1UL <<  8)
arch/x86/include/uapi/asm/debugreg.h +1 -0
index d95d080b30e3..0007ba077c0c 100644
@@ -24,6 +24,7 @@
 #define DR_TRAP3	(0x8)		/* db3 */
 #define DR_TRAP_BITS	(DR_TRAP0|DR_TRAP1|DR_TRAP2|DR_TRAP3)
 
+#define DR_BUS_LOCK	(0x800)		/* bus_lock */
 #define DR_STEP		(0x4000)	/* single-step */
 #define DR_SWITCH	(0x8000)	/* task switch */
 
arch/x86/kernel/cpu/common.c +1 -1
index 35ad8480c464..92705ac301ec 100644
@@ -1327,7 +1327,7 @@ static void __init early_identify_cpu(struct cpuinfo_x86 *c)
 
 	cpu_set_bug_bits(c);
 
-	cpu_set_core_cap_bits(c);
+	sld_setup(c);
 
 	fpu__init_system(c);
 
arch/x86/kernel/cpu/intel.c +106 -20

203 lines not on this page · raw message

arch/x86/kernel/traps.c +7 -0
index e19df6cde35d..dea26135e10b 100644
@@ -970,6 +970,13 @@ static __always_inline void exc_debug_user(struct pt_regs *regs,
 		goto out_irq;
 	}
 
+	/*
+	 * Handle bus lock. #DB for bus lock can only be triggered from
+	 * userspace. The bus lock bit was flipped to positive polarity.
+	 */
+	if (dr6 & DR_BUS_LOCK)
+		handle_bus_lock(regs);
+
 	/* Add the virtual_dr6 bits for signals. */
 	dr6 |= current->thread.virtual_dr6;
 	if (dr6 & (DR_STEP | DR_TRAP_BITS) || icebp)
Signature · 1 line
2.29.2
#6 lore raw

Re: [PATCH v4 2/4] x86/bus_lock: Handle warn and fatal in #DB for bus lock

Quoted 14 lines · Fenghua Yu

#DB for bus lock is enabled by bus lock detection bit 2 in DEBUGCTL MSR while #AC for split lock is enabled by split lock detection bit 29 in TEST_CTRL MSR.

Delivery of #DB for bus lock in userspace clears DR6[11]. To avoid confusion in identifying #DB, #DB handler sets the bit to 1 before returning to the interrupted task.

Use the existing kernel command line option "split_lock_detect=" to handle #DB for bus lock:

split_lock_detect=
#AC for split lock #DB for bus lock

Quoted 6 lines

off Do nothing Do nothing

warn Kernel OOPs Warn once per task and
Warn once per task and and continues to run.
disable future checking When both features are
supported, warn in #DB

Which means that we don't catch kernel split locks anymore with 'warn' if bus lock detection is supported. WHY? There is zero rationale for this change in the changelog.

Quoted 17 lines

fatal Kernel OOPs Send SIGBUS to user
Send SIGBUS to user When both features are
supported, split lock triggers #AC and bus lock from non-WB triggers #DB.

/*
- * Default to sld_off because most systems do not support split lock detection
- * split_lock_setup() will switch this to sld_warn on systems that support
- * split lock detect, unless there is a command line override.
+ * Default to sld_off because most systems do not support split lock detection.
+ * sld_state_setup() will switch this to sld_warn on systems that support
+ * split lock/bus lock detect, unless there is a command line override. */
static enum split_lock_detect_state sld_state __ro_after_init = sld_off; static u64 msr_test_ctrl_cache __ro_after_init;

arch/x86/kernel/cpu/intel.c enum split_lock_detect_state { · line 53

Quoted 2 lines
arch/x86/kernel/cpu/intel.c +2 -0
+/* Split lock detection is enabled if it's true. */
+static bool sld;

Why did you bother with 3 letters? bool s, b; along with comments explaining what it means would have been sufficient, right?

sld_enable/bld_enable would be too self explaining and this also lacks __ro_after_init

Aside of that it's beyond silly because bld and sld are just shadowing the corresponding CPU feature bits. So what are these variables gaining aside of confusion?

arch/x86/kernel/cpu/intel.c enum split_lock_detect_state {

Quoted 4 lines
arch/x86/kernel/cpu/intel.c +3 -0
+/* Bus lock detection is enabled if it's true. */
+static bool bld;
 
+static void __init sld_state_setup(void)

This is confusing as hell. sld_state_setup() is used for bus lock as well and split_lock_detect_state is not less confusing. It took me five reads to figure out how all of that works.

Quoted 1 line

+static void __init _split_lock_setup(void)

We generally use two underscores for readability sake.

arch/x86/kernel/cpu/intel.c static void sld_update_msr(bool on) · line 1089

Quoted 12 lines
arch/x86/kernel/cpu/intel.c +6 -0
+{
+	if (!split_lock_verify_msr(false)) {
+		pr_info("MSR access failed: Disabled\n");
 
 /*
@@ -1079,6 +1084,15 @@ static void sld_update_msr(bool on)
 
 static void split_lock_init(void)
 {
+	/*
+	 * If supported, #DB for bus lock will handle warn
+	 * and #AC for split lock is disabled.

Why does this disable the kernel detection? Just because?

arch/x86/kernel/cpu/intel.c bool handle_guest_split_lock(unsigned long ip) · line 1159

Quoted 4 lines
arch/x86/kernel/cpu/intel.c +4 -0
+void handle_bus_lock(struct pt_regs *regs)
+{
+	if (!bld)
+		return;

How is #DB ever calling this function when the debug MSR bit is not set?

Quoted 17 lines
arch/x86/kernel/cpu/intel.c +8 -2
-void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c)
+static void __init split_lock_setup(struct cpuinfo_x86 *c)
 {
 	const struct x86_cpu_id *m;
 	u64 ia32_core_caps;
@@ -1189,5 +1237,43 @@ void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c)
 	}
 
 	cpu_model_supports_sld = true;
-	split_lock_setup();
+	_split_lock_setup();
+}
+
+static void sld_state_show(void)
+{
+	if (!bld && !sld)
+		return;

arch/x86/kernel/cpu/intel.c void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c) · line 1256

Quoted 10 lines
arch/x86/kernel/cpu/intel.c +10 -0
+
+	switch (sld_state) {
+	case sld_off:
+		pr_info("disabled\n");
+		break;
+	case sld_warn:
+		if (bld)
+			pr_info("#DB: warning about user-space bus_locks\n");
+		else
+			pr_info("#AC: crashing the kernel about kernel split_locks and warning about user-space split_locks\n");

crashing about?

arch/x86/kernel/cpu/intel.c void __init cpu_set_core_cap_bits(struct cpuinfo_x86 *c) · line 1264

Quoted 8 lines
arch/x86/kernel/cpu/intel.c +8 -0
+		break;
+	case sld_fatal:
+		if (sld)
+			pr_info("#AC: crashing the kernel on kernel split_locks and sending SIGBUS on user-space split_locks\n");
+		if (bld)
+			pr_info("#DB: sending SIGBUS on user-space bus_locks%s\n", sld ? " from non-WB" : "");
+		break;
+	}

Thanks,

tglx

author
#7 lore raw

PATCH v4 3/4 x86/bus_lock: Set rate limit for bus lock

In reply to Fenghua Yu

To enforce user application throttling or mitigations, extend the existing split lock detect kernel parameter:
split_lock_detect=ratelimit:N

It limits bus lock rate to N per second for non-root users.

---
 arch/x86/kernel/cpu/intel.c | 35 ++++++++++++++++++++++++++++++-----
 include/linux/sched/user.h  |  4 +++-
 kernel/user.c               |  7 +++++++
 3 files changed, 40 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/intel.c +30 -5
index 77c3e33f41c7..5eb5822a446d 100644
@@ -10,6 +10,9 @@
 #include <linux/thread_info.h>
 #include <linux/init.h>
 #include <linux/uaccess.h>
+#include <linux/cred.h>
+#include <linux/delay.h>
+#include <linux/sched/user.h>
 
 #include <asm/cpufeature.h>
 #include <asm/msr.h>
@@ -40,6 +43,7 @@ enum split_lock_detect_state {
 	sld_off = 0,
 	sld_warn,
 	sld_fatal,
+	sld_ratelimit,
 };
 
 /*
@@ -998,13 +1002,25 @@ static const struct {
 	{ "off",	sld_off   },
 	{ "warn",	sld_warn  },
 	{ "fatal",	sld_fatal },
+	{ "ratelimit:", sld_ratelimit },
 };
 
 static inline bool match_option(const char *arg, int arglen, const char *opt)
 {
-	int len = strlen(opt);
 
-	return len == arglen && !strncmp(arg, opt, len);
+	int len = strlen(opt), ratelimit;
+
+	if (strncmp(arg, opt, len))
+		return false;
+
+	if (sscanf(arg, "ratelimit:%d", &ratelimit) == 1 && ratelimit > 0 &&
+	    ratelimit_bl <= HZ / 2) {
+		ratelimit_bl = ratelimit;
+
+		return true;
+	}
+
+	return len == arglen;
 }
 
 static bool split_lock_verify_msr(bool on)
@@ -1085,10 +1101,10 @@ static void sld_update_msr(bool on)
 static void split_lock_init(void)
 {
 	/*
-	 * If supported, #DB for bus lock will handle warn
+	 * If supported, #DB for bus lock will handle warn or ratelimit
 	 * and #AC for split lock is disabled.
 	 */
-	if (bld && sld_state == sld_warn) {
+	if ((bld && sld_state == sld_warn) || sld_state == sld_ratelimit) {
 		split_lock_verify_msr(false);
 		return;
 	}
@@ -1147,7 +1163,8 @@ static void bus_lock_init(void)
 
 bool handle_user_split_lock(struct pt_regs *regs, long error_code)
 {
-	if ((regs->flags & X86_EFLAGS_AC) || !sld || sld_state == sld_fatal)
+	if ((regs->flags & X86_EFLAGS_AC) || !sld || sld_state == sld_fatal ||
+	    sld_state == sld_ratelimit)
 		return false;
 	split_lock_warn(regs->ip);
 	return true;
@@ -1168,6 +1185,10 @@ void handle_bus_lock(struct pt_regs *regs)
 	case sld_fatal:
 		force_sig_fault(SIGBUS, BUS_ADRALN, NULL);
 		break;
+	case sld_ratelimit:
+		while (!__ratelimit(&get_current_user()->ratelimit_bl))
+			msleep(1000 / ratelimit_bl);
+		break;
 	}
 }
 
@@ -1261,6 +1282,10 @@ static void sld_state_show(void)
 		if (bld)
 			pr_info("#DB: sending SIGBUS on user-space bus_locks%s\n", sld ? " from non-WB" : "");
 		break;
+	case sld_ratelimit:
+		if (bld)
+			pr_info("#DB: setting rate limit to %d/sec per user on non-root user-space bus_locks\n", ratelimit_bl);
+		break;
 	}
 }
 
include/linux/sched/user.h +3 -1
index a8ec3b6093fc..79f95002a123 100644
@@ -40,8 +40,9 @@ struct user_struct {
 	atomic_t nr_watches;	/* The number of watches this user currently has */
 #endif
 
-	/* Miscellaneous per-user rate limit */
+	/* Miscellaneous per-user rate limits */
 	struct ratelimit_state ratelimit;
+	struct ratelimit_state ratelimit_bl;
 };
 
 extern int uids_sysfs_init(void);
@@ -51,6 +52,7 @@ extern struct user_struct *find_user(kuid_t);
 extern struct user_struct root_user;
 #define INIT_USER (&root_user)
 
+extern int ratelimit_bl;
 
 /* per-UID process charging. */
 extern struct user_struct * alloc_uid(kuid_t);
kernel/user.c +7 -0
index b1635d94a1f2..8fc19706bc91 100644
@@ -103,6 +103,7 @@ struct user_struct root_user = {
 	.locked_shm     = 0,
 	.uid		= GLOBAL_ROOT_UID,
 	.ratelimit	= RATELIMIT_STATE_INIT(root_user.ratelimit, 0, 0),
+	.ratelimit_bl	= RATELIMIT_STATE_INIT(root_user.ratelimit_bl, 0, 0),
 };
 
 /*
@@ -172,6 +173,9 @@ void free_uid(struct user_struct *up)
 		free_user(up, flags);
 }
 
+/* Architectures (e.g. X86) may set this for rate-limited bus locks. */
+int ratelimit_bl;
+
 struct user_struct *alloc_uid(kuid_t uid)
 {
 	struct hlist_head *hashent = uidhashentry(uid);
@@ -190,6 +194,9 @@ struct user_struct *alloc_uid(kuid_t uid)
 		refcount_set(&new->__count, 1);
 		ratelimit_state_init(&new->ratelimit, HZ, 100);
 		ratelimit_set_flags(&new->ratelimit, RATELIMIT_MSG_ON_RELEASE);
+		ratelimit_state_init(&new->ratelimit_bl, HZ, ratelimit_bl);
+		ratelimit_set_flags(&new->ratelimit_bl,
+				    RATELIMIT_MSG_ON_RELEASE);
 
 		/*
 		 * Before adding this, check whether we raced
Signature · 1 line
2.29.2
#8 lore raw

Re: [PATCH v4 3/4] x86/bus_lock: Set rate limit for bus lock

Quoted 5 lines · Fenghua Yu

To enforce user application throttling or mitigations, extend the existing split lock detect kernel parameter:
split_lock_detect=ratelimit:N

It limits bus lock rate to N per second for non-root users.

Yet another changelog which describes what it does without giving some clue why. It also lacks the information that the ratelimiting is per UID and not per task and why this was chosen to be per UID...

Quoted 39 lines
---
 arch/x86/kernel/cpu/intel.c | 35 ++++++++++++++++++++++++++++++-----
 include/linux/sched/user.h  |  4 +++-
 kernel/user.c               |  7 +++++++
 3 files changed, 40 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/intel.c +5 -0
index 77c3e33f41c7..5eb5822a446d 100644
@@ -10,6 +10,9 @@
 #include <linux/thread_info.h>
 #include <linux/init.h>
 #include <linux/uaccess.h>
+#include <linux/cred.h>
+#include <linux/delay.h>
+#include <linux/sched/user.h>
 
 #include <asm/cpufeature.h>
 #include <asm/msr.h>
@@ -40,6 +43,7 @@ enum split_lock_detect_state {
 	sld_off = 0,
 	sld_warn,
 	sld_fatal,
+	sld_ratelimit,
 };
 
 /*
@@ -998,13 +1002,25 @@ static const struct {
 	{ "off",	sld_off   },
 	{ "warn",	sld_warn  },
 	{ "fatal",	sld_fatal },
+	{ "ratelimit:", sld_ratelimit },
 };
 
 static inline bool match_option(const char *arg, int arglen, const char *opt)
 {

arch/x86/kernel/cpu/intel.c static const struct { · line 1017

Quoted 10 lines
arch/x86/kernel/cpu/intel.c +7 -2
-	int len = strlen(opt);
 
-	return len == arglen && !strncmp(arg, opt, len);
+	int len = strlen(opt), ratelimit;
+
+	if (strncmp(arg, opt, len))
+		return false;
+
+	if (sscanf(arg, "ratelimit:%d", &ratelimit) == 1 && ratelimit > 0 &&
+	    ratelimit_bl <= HZ / 2) {

That last part condition of the condition is always true because ratelimit_bl is a global variable in the BSS and nothing else than the below writes to it. So INT_MAX is a valid argument ....

arch/x86/kernel/cpu/intel.c static const struct { · line 1018

Quoted 1 line
arch/x86/kernel/cpu/intel.c +1 -0
+		ratelimit_bl = ratelimit;

Can you please use consistent name spaces? bld_XXX all over the place?

Quoted 9 lines
arch/x86/kernel/cpu/intel.c +2 -2
 static void split_lock_init(void)
 {
 	/*
-	 * If supported, #DB for bus lock will handle warn
+	 * If supported, #DB for bus lock will handle warn or ratelimit
 	 * and #AC for split lock is disabled.
 	 */
-	if (bld && sld_state == sld_warn) {
+	if ((bld && sld_state == sld_warn) || sld_state == sld_ratelimit) {

...

arch/x86/kernel/cpu/intel.c static void bus_lock_init(void) · line 1167

Quoted 3 lines
arch/x86/kernel/cpu/intel.c +2 -1
-	if ((regs->flags & X86_EFLAGS_AC) || !sld || sld_state == sld_fatal)
+	if ((regs->flags & X86_EFLAGS_AC) || !sld || sld_state == sld_fatal ||
+	    sld_state == sld_ratelimit)

These conditions are uncomprehensible by now

arch/x86/kernel/cpu/intel.c void handle_bus_lock(struct pt_regs *regs) · line 1190

Quoted 10 lines
arch/x86/kernel/cpu/intel.c +3 -0
 		return false;
 	split_lock_warn(regs->ip);
 	return true;
@@ -1168,6 +1185,10 @@ void handle_bus_lock(struct pt_regs *regs)
 	case sld_fatal:
 		force_sig_fault(SIGBUS, BUS_ADRALN, NULL);
 		break;
+	case sld_ratelimit:
+		while (!__ratelimit(&get_current_user()->ratelimit_bl))
+			msleep(1000 / ratelimit_bl);

Yikes. That's really creative abuse of the ratelimit mechanics of course without any comment how this is supposed to work.

include/linux/sched/user.h struct user_struct { · line 45

Quoted 12 lines
include/linux/sched/user.h +2 -1
index a8ec3b6093fc..79f95002a123 100644
@@ -40,8 +40,9 @@ struct user_struct {
 	atomic_t nr_watches;	/* The number of watches this user currently has */
 #endif
 
-	/* Miscellaneous per-user rate limit */
+	/* Miscellaneous per-user rate limits */
 	struct ratelimit_state ratelimit;
+	struct ratelimit_state ratelimit_bl;

Why has every architecture to carry this if only one out of N uses it?

Thanks,

tglx

author
#9 lore raw

PATCH v4 4/4 Documentation/admin-guide: Change doc for split_lock_detect parameter

In reply to Fenghua Yu

Since #DB for bus lock detect changes the split_lock_detect parameter, update the documentation for the changes.

---
Change Log:
v4:
- Fix a ratelimit wording issue in the doc (Randy).
- Patch 4 is acked by Randy (Randy).

v3:
- Enable Bus Lock Detection when fatal to handle bus lock from non-WB (PeterZ).

v1:
- Fix a few wording issues (Randy).

RFC v2:
- Simplify the documentation (Randy).

 .../admin-guide/kernel-parameters.txt         | 30 +++++++++++++++----
 1 file changed, 24 insertions(+), 6 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +24 -6
index 44fde25bb221..28228539b02a 100644
@@ -5051,27 +5051,45 @@
 	spia_peddr=
 
 	split_lock_detect=
-			[X86] Enable split lock detection
+			[X86] Enable split lock detection or bus lock detection
 
 			When enabled (and if hardware support is present), atomic
 			instructions that access data across cache line
-			boundaries will result in an alignment check exception.
+			boundaries will result in an alignment check exception
+			for split lock detection or a debug exception for
+			bus lock detection.
 
 			off	- not enabled
 
-			warn	- the kernel will emit rate limited warnings
+			warn	- the kernel will emit rate-limited warnings
 				  about applications triggering the #AC
-				  exception. This mode is the default on CPUs
-				  that supports split lock detection.
+				  exception or the #DB exception. This mode is
+				  the default on CPUs that support split lock
+				  detection or bus lock detection. Default
+				  behavior is by #DB if both features are
+				  enabled in hardware.
 
 			fatal	- the kernel will send SIGBUS to applications
-				  that trigger the #AC exception.
+				  that trigger the #AC exception or the #DB
+				  exception. If both features are enabled in
+				  hardware, split lock triggers #AC and bus
+				  lock from non-WB triggers #DB.
+
+			ratelimit:N -
+				  Set rate limit to N bus locks per second
+				  for bus lock detection. 0 < N <= HZ/2.
+				  Only applied to non-root users.
+
+				  N/A for split lock detection.
 
 			If an #AC exception is hit in the kernel or in
 			firmware (i.e. not while executing in user mode)
 			the kernel will oops in either "warn" or "fatal"
 			mode.
 
+			#DB exception for bus lock is triggered only when
+			CPL > 0.
+
 	srbds=		[X86,INTEL]
 			Control the Special Register Buffer Data Sampling
 			(SRBDS) mitigation.
Signature · 1 line
2.29.2
#10 lore raw

Re: [PATCH v4 4/4] Documentation/admin-guide: Change doc for split_lock_detect parameter

Quoted 2 lines · Fenghua Yu

Since #DB for bus lock detect changes the split_lock_detect parameter, update the documentation for the changes.

Why is this seperate and an all in one thing? patch 2/4 changes the parameter first and 3/4 adds a new option....

Quoted 28 lines

---
Change Log:
v4:
- Fix a ratelimit wording issue in the doc (Randy).
- Patch 4 is acked by Randy (Randy).

v3:
- Enable Bus Lock Detection when fatal to handle bus lock from non-WB (PeterZ).

v1:
- Fix a few wording issues (Randy).

RFC v2:
- Simplify the documentation (Randy).

 .../admin-guide/kernel-parameters.txt         | 30 +++++++++++++++----
 1 file changed, 24 insertions(+), 6 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +0 -0
index 44fde25bb221..28228539b02a 100644

Documentation/admin-guide/kernel-parameters.txt line 5055

Quoted 5 lines
Documentation/admin-guide/kernel-parameters.txt +4 -0
@@ -5051,27 +5051,45 @@
 	spia_peddr=
+
+			ratelimit:N -
+				  Set rate limit to N bus locks per second
+				  for bus lock detection. 0 < N <= HZ/2.

Oh well. So if I have ratelimit:500 on the kernel command line and then this works for CONFIG_HZ=1000, but after rebuilding the kernel with HZ=250 it fails. What?

If I allow 500 hits per second then CONFIG_HZ has absolutely nothing to do with it. A second stays a second independent of CONFIG_HZ.

So what's the purpose of this HZ business?

Thanks,

tglx

#11 lore raw
In reply to Fenghua Yu

Hi, Dear maintainers,

Quoted 4 lines

A bus lock [1] is acquired through either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically >1000 cycles slower than an atomic operation within a cache line. It also disrupts performance on other cores.

...

Quoted 4 lines

Change Log:
v4:
- Fix a ratelimit wording issue in the doc (Randy).
- Patch 4 is acked by Randy (Randy).

Friendly reminder about this series...

Thank you very much in advance for your time!

-Fenghua

author
#12 lore raw
In reply to Fenghua Yu

Hi, Dear X86 Maintainers,

Quoted 4 lines · Fenghua Yu

A bus lock [1] is acquired through either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically >1000 cycles slower than an atomic operation within a cache line. It also disrupts performance on other cores.

This is a friendly reminder. Any comment on this series?

Thanks.

-Fenghua

author
#13 lore raw

Hi, Dear X86 Maintainers,

Quoted 7 lines · Fenghua Yu
Quoted 4 lines · Fenghua Yu

A bus lock [1] is acquired through either split locked access to writeback (WB) memory or any locked access to non-WB memory. This is typically >1000 cycles slower than an atomic operation within a cache line. It also disrupts performance on other cores.

Just a friendly reminder. Any comment on this series? There hasn't been any comment since it was posted.

This series can be applied cleanly to v5.11-rc5 and pass all of our tests. If you want me to repost this series, I can do that too.

Thanks!

-Fenghua