Skip to contentSkip to conversation

x86, mpx updates for 4.2 (take 6)

PATCH In review Dave Hansen 2 participants 45 messages May 8, 2015 – May 19, 2015

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover x86, mpx updates for 4.2 (take 6)

Changes from take 5 / v20:

* Fix get_xsave_addr() to consult xstate_bv in anticipation of fixes to xsave code.
* Bug fix for when an VMA being unmapped has neighbors which are bounds tables.
* Rewrite unmapping code. I didn't do this lightly. It was not originally my own code, and I resisted changing it because it worked. But, I started bug chasing and decided it was unmaintainable. The rewrite ended up removing about 20% of the unmapping code and made it much simpler.

Changes from take 4 / v19:

* Do not pass a task_struct around when we are
really just going to operate on current

Changes from take 3 / v18 (all minor):

* use DECLARE_EVENT_CLASS()/DEFINE_EVENT() for the ranged tracepoints to save 10 lines of code.

Changes from take 2 / v17 (all minor):

* fix a couple of whitespace borkages caught by checkpatch, and a spelling error or two.
* replace printk with pr_info() for boot disable
* change trace print format for address intervals
* fix up variable name in tsk_get_xsave_addr() comment
* remove tsk_get_xsave_field() GPL export
* fix up Qiaowei's From:

--

Hi x86 maintainers,

There are a few basic things going on here:
1. Make FPU/xsave code preempt safe and work properly
2. Add trace points to make kernel and app debugging easier
3. Add a boot-time disable for mpx
4. Rewrite the unmapping code.
5. Support 32-bit binaries to run on 64-bit kernels

This sees breakage unless either booted with 'noxsaves' or if it has Fenghua's set from here applied:

	http://lkml.kernel.org/r/[email protected]

This set is also available against 4.1-rc2 in git:

git://git.kernel.org/pub/scm/linux/kernel/git/daveh/x86-mpx.git mpx-v21

 Documentation/kernel-parameters.txt |    4
 arch/x86/include/asm/mmu_context.h  |   13
 arch/x86/include/asm/mpx.h          |   76 ++--
 arch/x86/include/asm/processor.h    |   12
 arch/x86/include/asm/xsave.h        |    1
 arch/x86/kernel/cpu/common.c        |   16
 arch/x86/kernel/traps.c             |   18 -
 arch/x86/kernel/uprobes.c           |   10
 arch/x86/kernel/xsave.c             |   73 +++-
 arch/x86/mm/mpx.c                   |  634 +++++++++++++++++++++---------------
 kernel/sys.c                        |    8
 11 files changed, 543 insertions(+), 322 deletions(-)
#1 lore raw

PATCH 1/19 x86, mpx, xsave: fix up bad get_xsave_addr() assumptions

From: Dave Hansen <[email protected]>

get_xsave_addr() assumes that if an xsave bit is present in the hardware (pcntxt_mask) that it is present in a given xsave buffer. Due to an bug in the xsave code on all of the systems that have MPX (and thus all the users of this code), that has been a true assumption.

But, the bug is getting fixed, so our assumption is not going to hold any more.

It's quite possible (and normal) for an enabled state to be present on 'pcntxt_mask', but *not* in 'xstate_bv'. We need to consult 'xstate_bv'.

---

 b/arch/x86/kernel/xsave.c |   41 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 34 insertions(+), 7 deletions(-)
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~consullt-xstate_bv +34 -7
diff -puN arch/x86/kernel/xsave.c~consullt-xstate_bv arch/x86/kernel/xsave.c
@@ -706,19 +706,46 @@ void __init_refok eager_fpu_init(void)
  * This is the API that is called to get xstate address in either
  * standard format or compacted format of xsave area.
  *
+ * Note that if there is no data for the field in the xsave buffer
+ * this will return NULL.
+ *
  * Inputs:
- *	xsave: base address of the xsave area;
- *	xstate: state which is defined in xsave.h (e.g. XSTATE_FP, XSTATE_SSE,
- *	etc.)
+ *	xstate: the thread's storage area for all FPU data
+ *	xstate_field: state which is defined in xsave.h (e.g. XSTATE_FP,
+ *	XSTATE_SSE, etc...)
  * Output:
  *	address of the state in the xsave area.
  */
-void *get_xsave_addr(struct xsave_struct *xsave, int xstate)
+void *get_xsave_addr(struct xsave_struct *xsave, int xstate_field)
 {
-	int feature = fls64(xstate) - 1;
-	if (!test_bit(feature, (unsigned long *)&pcntxt_mask))
+	int feature_nr = fls64(xstate_field) - 1;
+	/*
+	 * Do we even *have* xsave state?
+	 */
+	if (!boot_cpu_has(X86_FEATURE_XSAVE))
+		return NULL;
+
+	xsave = &current->thread.fpu.state->xsave;
+	/*
+	 * We should not ever be requesting fields that we
+	 * have not enabled.  Remember that pcntxt_mask is
+	 * what we write to the XCR0 register.
+	 */
+	WARN_ONCE(!(pcntxt_mask & xstate_field), "get of unsupported state");
+	/*
+	 * This assumes the last 'xsave*' instruction to
+	 * have requested that 'xstate_field' be saved.
+	 * If it did not, we might be seeing and old value
+	 * of the field in the buffer.
+	 *
+	 * This can happen because the last 'xsave' did not
+	 * request that this feature be saved (unlikely)
+	 * or because the "init optimization" caused it
+	 * to not be saved.
+	 */
+	if (!(xsave->xsave_hdr.xstate_bv & xstate_field))
 		return NULL;
 
-	return (void *)xsave + xstate_comp_offsets[feature];
+	return (void *)xsave + xstate_comp_offsets[feature_nr];
 }
 EXPORT_SYMBOL_GPL(get_xsave_addr);

_

Reviewed-by on v1 1/19
#2 lore raw

Re: [PATCH 01/19] x86, mpx, xsave: fix up bad get_xsave_addr() assumptions

Quoted 8 lines · Dave Hansen

From: Dave Hansen <[email protected]>

get_xsave_addr() assumes that if an xsave bit is present in the hardware (pcntxt_mask) that it is present in a given xsave buffer. Due to an bug in the xsave code on all of the systems that have MPX (and thus all the users of this code), that has been a true assumption.

Quoted 8 lines

But, the bug is getting fixed, so our assumption is not going to hold any more.

It's quite possible (and normal) for an enabled state to be present on 'pcntxt_mask', but *not* in 'xstate_bv'. We need to consult 'xstate_bv'.

#3 lore raw

PATCH 2/19 x86, fpu: wrap get_xsave_addr() to make it safer

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

Changes from "v19":
* remove 'tsk' argument to get_xsave_addr() since the code can only realistically work on 'current', and fix up the comment a bit to match.

Changes from "v17":
* fix s/xstate/xsave_field/ in the function comment
* remove EXPORT_SYMBOL_GPL()

---
From: Dave Hansen <[email protected]>

The MPX code appears to be saving off the FPU in an unsafe
way. It does not disable preemption or ensure that the
FPU state has been allocated.

This patch introduces a new helper which will do both of those things internally.

Note that this requires a patch from Oleg in order to work properly. It is currently in tip/x86/fpu.

Quoted 5 lines

commit f893959b0898bd876673adbeb6798bdf25c034d7

Author: Oleg Nesterov <[email protected]>
Date:   Fri Mar 13 18:30:30 2015 +0100

x86/fpu: Don't abuse drop_init_fpu() in flush_thread()

Cc Oleg Nesterov <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Suresh Siddha <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc the arch/x86 maintainers <[email protected]>

---

 b/arch/x86/include/asm/xsave.h |    1 +
 b/arch/x86/kernel/xsave.c      |   32 ++++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)
arch/x86/include/asm/xsave.h renamed from arch/x86/include/asm/xsave.h~tsk_get_xsave_addr +1 -0
diff -puN arch/x86/include/asm/xsave.h~tsk_get_xsave_addr arch/x86/include/asm/xsave.h
@@ -252,6 +252,7 @@ static inline int xrestore_user(struct x
 }
 
 void *get_xsave_addr(struct xsave_struct *xsave, int xstate);
+void *get_xsave_field(int xstate_field);
 void setup_xstate_comp(void);
 
 #endif
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +32 -0
diff -puN arch/x86/kernel/xsave.c~tsk_get_xsave_addr arch/x86/kernel/xsave.c
@@ -749,3 +749,35 @@ void *get_xsave_addr(struct xsave_struct
 	return (void *)xsave + xstate_comp_offsets[feature_nr];
 }
 EXPORT_SYMBOL_GPL(get_xsave_addr);
+
+/*
+ * This wraps up the common operations that need to occur when retrieving
+ * data from xsave state.  It first ensures that the current task was
+ * using the FPU and retrieves the data in to a buffer.  It then calculates
+ * the offset of the requested field in the buffer.
+ *
+ * This function is safe to call whether the FPU is in use or not.
+ *
+ * Note that this only works on the current task.
+ *
+ * Inputs:
+ *	@xsave_field: state which is defined in xsave.h (e.g. XSTATE_FP,
+ *	XSTATE_SSE, etc...)
+ * Output:
+ *	address of the state in the xsave area.
+ */
+void *get_xsave_field(int xsave_field)
+{
+	union thread_xstate *xstate;
+
+	if (!tsk_used_math(current))
+		return NULL;
+	/*
+	 * unlazy_fpu() is poorly named and will actually
+	 * save the xstate off in to the memory buffer.
+	 */
+	unlazy_fpu(current);
+	xstate = current->thread.fpu.state;
+
+	return get_xsave_addr(&xstate->xsave, xsave_field);
+}

_

#4 lore raw

Re: [PATCH 02/19] x86, fpu: wrap get_xsave_addr() to make it safer

Quoted 6 lines · Dave Hansen

The MPX code appears to be saving off the FPU in an unsafe
way. It does not disable preemption or ensure that the
FPU state has been allocated.

This patch introduces a new helper which will do both of those things internally.

This changelog does not really match the implementation. Unless I'm missing something I can't find anything preemption related.

Quoted 6 lines
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +6 -0
+
+/*
+ * This wraps up the common operations that need to occur when retrieving
+ * data from xsave state.  It first ensures that the current task was
+ * using the FPU and retrieves the data in to a buffer.  It then calculates
+ * the offset of the requested field in the buffer.

arch/x86/kernel/xsave.c void *get_xsave_addr(struct xsave_struct · line 767

Quoted 10 lines
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +10 -0
+ *
+ * This function is safe to call whether the FPU is in use or not.
+ *
+ * Note that this only works on the current task.
+ *
+ * Inputs:
+ *	@xsave_field: state which is defined in xsave.h (e.g. XSTATE_FP,
+ *	XSTATE_SSE, etc...)
+ * Output:
+ *	address of the state in the xsave area.

or NULL in case of .....

Quoted 6 lines

+ */

arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +5 -0
+void *get_xsave_field(int xsave_field)
+{
+	union thread_xstate *xstate;
+
+	if (!tsk_used_math(current))

arch/x86/kernel/xsave.c void *get_xsave_addr(struct xsave_struct · line 783

Quoted 10 lines
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +10 -0
+		return NULL;
+	/*
+	 * unlazy_fpu() is poorly named and will actually
+	 * save the xstate off in to the memory buffer.
+	 */
+	unlazy_fpu(current);
+	xstate = current->thread.fpu.state;
+
+	return get_xsave_addr(&xstate->xsave, xsave_field);
+}

Thanks,

tglx

#5 lore raw

Re: [PATCH 02/19] x86, fpu: wrap get_xsave_addr() to make it safer

Quoted 10 lines · Thomas Gleixner
Quoted 6 lines · Dave Hansen

The MPX code appears to be saving off the FPU in an unsafe
way. It does not disable preemption or ensure that the
FPU state has been allocated.

This patch introduces a new helper which will do both of those things internally.

This changelog does not really match the implementation. Unless I'm missing something I can't find anything preemption related.

Gah. Hit send before finishing the mail.

It's unlazy_fpu (which I agree is a horrible name) which does the right thing.

Quoted 40 lines
Quoted 6 lines
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +6 -0
+
+/*
+ * This wraps up the common operations that need to occur when retrieving
+ * data from xsave state.  It first ensures that the current task was
+ * using the FPU and retrieves the data in to a buffer.  It then calculates
+ * the offset of the requested field in the buffer.

arch/x86/kernel/xsave.c void *get_xsave_addr(struct xsave_struct · line 767

Quoted 10 lines
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +10 -0
+ *
+ * This function is safe to call whether the FPU is in use or not.
+ *
+ * Note that this only works on the current task.
+ *
+ * Inputs:
+ *	@xsave_field: state which is defined in xsave.h (e.g. XSTATE_FP,
+ *	XSTATE_SSE, etc...)
+ * Output:
+ *	address of the state in the xsave area.

or NULL in case of .....

Quoted 6 lines

+ */

arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +5 -0
+void *get_xsave_field(int xsave_field)
+{
+	union thread_xstate *xstate;
+
+	if (!tsk_used_math(current))

arch/x86/kernel/xsave.c void *get_xsave_addr(struct xsave_struct · line 783

Quoted 10 lines
arch/x86/kernel/xsave.c renamed from arch/x86/kernel/xsave.c~tsk_get_xsave_addr +10 -0
+		return NULL;
+	/*
+	 * unlazy_fpu() is poorly named and will actually
+	 * save the xstate off in to the memory buffer.
+	 */
+	unlazy_fpu(current);
+	xstate = current->thread.fpu.state;
+
+	return get_xsave_addr(&xstate->xsave, xsave_field);
+}

Thanks,

tglx

#6 lore raw

PATCH 3/19 x86, mpx: use new tsk_get_xsave_addr()

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

The MPX registers (bndcsr/bndcfgu/bndstatus) are not directly accessible via normal instructions. They essentially act as if they were floating point registers and are saved/restored along with those registers.

There are two main paths in the MPX code where we care about the contents of these registers:
1. #BR (bounds) faults
2. the prctl() code where we are setting MPX up

Both of those paths _might_ be called without the FPU having been used. That means that 'tsk->thread.fpu.state' might never be allocated.

Also, fpu_save_init() is not preempt-safe. It was a bug to call it without disabling preemption. The new tsk_get_xsave_addr() calls unlazy_fpu() instead and properly disables preemption.

Cc Oleg Nesterov <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Suresh Siddha <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc the arch/x86 maintainers <[email protected]>
Cc linux-kernel <[email protected]>

---

 b/arch/x86/include/asm/mpx.h |    8 ++++----
 b/arch/x86/kernel/traps.c    |   15 +++++++--------
 b/arch/x86/mm/mpx.c          |   23 +++++++++++------------
 3 files changed, 22 insertions(+), 24 deletions(-)
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~use-new-tsk_get_xsave_addr +4 -4
diff -puN arch/x86/include/asm/mpx.h~use-new-tsk_get_xsave_addr arch/x86/include/asm/mpx.h
@@ -60,8 +60,8 @@
 
 #ifdef CONFIG_X86_INTEL_MPX
 siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-				struct xsave_struct *xsave_buf);
-int mpx_handle_bd_fault(struct xsave_struct *xsave_buf);
+				struct task_struct *tsk);
+int mpx_handle_bd_fault(struct task_struct *tsk);
 static inline int kernel_managing_mpx_tables(struct mm_struct *mm)
 {
 	return (mm->bd_addr != MPX_INVALID_BOUNDS_DIR);
@@ -78,11 +78,11 @@ void mpx_notify_unmap(struct mm_struct *
 		      unsigned long start, unsigned long end);
 #else
 static inline siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-					      struct xsave_struct *xsave_buf)
+					      struct task_struct *tsk)
 {
 	return NULL;
 }
-static inline int mpx_handle_bd_fault(struct xsave_struct *xsave_buf)
+static inline int mpx_handle_bd_fault(struct task_struct *tsk)
 {
 	return -EINVAL;
 }
arch/x86/kernel/traps.c renamed from arch/x86/kernel/traps.c~use-new-tsk_get_xsave_addr +7 -8
diff -puN arch/x86/kernel/traps.c~use-new-tsk_get_xsave_addr arch/x86/kernel/traps.c
@@ -61,6 +61,7 @@
 #include <asm/mach_traps.h>
 #include <asm/alternative.h>
 #include <asm/mpx.h>
+#include <asm/xsave.h>
 
 #ifdef CONFIG_X86_64
 #include <asm/x86_init.h>
@@ -372,7 +373,6 @@ dotraplinkage void do_double_fault(struc
 dotraplinkage void do_bounds(struct pt_regs *regs, long error_code)
 {
 	struct task_struct *tsk = current;
-	struct xsave_struct *xsave_buf;
 	enum ctx_state prev_state;
 	struct bndcsr *bndcsr;
 	siginfo_t *info;
@@ -393,12 +393,11 @@ dotraplinkage void do_bounds(struct pt_r
 
 	/*
 	 * We need to look at BNDSTATUS to resolve this exception.
-	 * It is not directly accessible, though, so we need to
-	 * do an xsave and then pull it out of the xsave buffer.
+	 * A NULL here might mean that it is in its 'init state',
+	 * which is all zeros which indicates MPX was not
+	 * responsible for the exception.
 	 */
-	fpu_save_init(&tsk->thread.fpu);
-	xsave_buf = &(tsk->thread.fpu.state->xsave);
-	bndcsr = get_xsave_addr(xsave_buf, XSTATE_BNDCSR);
+	bndcsr = get_xsave_field(XSTATE_BNDCSR);
 	if (!bndcsr)
 		goto exit_trap;
 
@@ -409,11 +408,11 @@ dotraplinkage void do_bounds(struct pt_r
 	 */
 	switch (bndcsr->bndstatus & MPX_BNDSTA_ERROR_CODE) {
 	case 2:	/* Bound directory has invalid entry. */
-		if (mpx_handle_bd_fault(xsave_buf))
+		if (mpx_handle_bd_fault(tsk))
 			goto exit_trap;
 		break; /* Success, it was handled */
 	case 1: /* Bound violation. */
-		info = mpx_generate_siginfo(regs, xsave_buf);
+		info = mpx_generate_siginfo(regs, tsk);
 		if (IS_ERR(info)) {
 			/*
 			 * We failed to decode the MPX instruction.  Act as if
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~use-new-tsk_get_xsave_addr +11 -12
diff -puN arch/x86/mm/mpx.c~use-new-tsk_get_xsave_addr arch/x86/mm/mpx.c
@@ -273,7 +273,7 @@ bad_opcode:
  * The caller is expected to kfree() the returned siginfo_t.
  */
 siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-				struct xsave_struct *xsave_buf)
+				struct task_struct *tsk)
 {
 	struct bndreg *bndregs, *bndreg;
 	siginfo_t *info = NULL;
@@ -295,8 +295,8 @@ siginfo_t *mpx_generate_siginfo(struct p
 		err = -EINVAL;
 		goto err_out;
 	}
-	/* get the bndregs _area_ of the xsave structure */
-	bndregs = get_xsave_addr(xsave_buf, XSTATE_BNDREGS);
+	/* get bndregs field from urrent task's xsave area */
+	bndregs = get_xsave_field(XSTATE_BNDREGS);
 	if (!bndregs) {
 		err = -EINVAL;
 		goto err_out;
@@ -358,8 +358,7 @@ static __user void *task_get_bounds_dir(
 	 * The bounds directory pointer is stored in a register
 	 * only accessible if we first do an xsave.
 	 */
-	fpu_save_init(&tsk->thread.fpu);
-	bndcsr = get_xsave_addr(&tsk->thread.fpu.state->xsave, XSTATE_BNDCSR);
+	bndcsr = get_xsave_field(XSTATE_BNDCSR);
 	if (!bndcsr)
 		return MPX_INVALID_BOUNDS_DIR;
 
@@ -390,9 +389,9 @@ int mpx_enable_management(struct task_st
 	 * directory into XSAVE/XRSTOR Save Area and enable MPX through
 	 * XRSTOR instruction.
 	 *
-	 * fpu_xsave() is expected to be very expensive. Storing the bounds
-	 * directory here means that we do not have to do xsave in the unmap
-	 * path; we can just use mm->bd_addr instead.
+	 * xsaves are expected to be very expensive.  Storing the bounds
+	 * directory here means that we do not have to do xsave in the
+	 * unmap path; we can just use mm->bd_addr instead.
 	 */
 	bd_base = task_get_bounds_dir(tsk);
 	down_write(&mm->mmap_sem);
@@ -498,12 +497,12 @@ out_unmap:
  * bound table is 16KB. With 64-bit mode, the size of BD is 2GB,
  * and the size of each bound table is 4MB.
  */
-static int do_mpx_bt_fault(struct xsave_struct *xsave_buf)
+static int do_mpx_bt_fault(struct task_struct *tsk)
 {
 	unsigned long bd_entry, bd_base;
 	struct bndcsr *bndcsr;
 
-	bndcsr = get_xsave_addr(xsave_buf, XSTATE_BNDCSR);
+	bndcsr = get_xsave_field(XSTATE_BNDCSR);
 	if (!bndcsr)
 		return -EINVAL;
 	/*
@@ -526,7 +525,7 @@ static int do_mpx_bt_fault(struct xsave_
 	return allocate_bt((long __user *)bd_entry);
 }
 
-int mpx_handle_bd_fault(struct xsave_struct *xsave_buf)
+int mpx_handle_bd_fault(struct task_struct *tsk)
 {
 	/*
 	 * Userspace never asked us to manage the bounds tables,
@@ -535,7 +534,7 @@ int mpx_handle_bd_fault(struct xsave_str
 	if (!kernel_managing_mpx_tables(current->mm))
 		return -EINVAL;
 
-	if (do_mpx_bt_fault(xsave_buf)) {
+	if (do_mpx_bt_fault(tsk)) {
 		force_sig(SIGSEGV, current);
 		/*
 		 * The force_sig() is essentially "handling" this

_

#7 lore raw

Re: [PATCH 03/19] x86, mpx: use new tsk_get_xsave_addr()

Quoted 1 line · Dave Hansen

Subject: x86, mpx: use new tsk_get_xsave_addr()

-NOSUCHFUNCTION

And please let the sentence start with an upper case letter.

x86, mpx: Use ...

arch/x86/include/asm/mpx.h line 64

Quoted 6 lines

#ifdef CONFIG_X86_INTEL_MPX
siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,

arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~use-new-tsk_get_xsave_addr +2 -2
-				struct xsave_struct *xsave_buf);
-int mpx_handle_bd_fault(struct xsave_struct *xsave_buf);
+				struct task_struct *tsk);
+int mpx_handle_bd_fault(struct task_struct *tsk);

So you still keep *tsk and remove it in the next patch. I guess either ordering has its value, so no problem to do it this way.

arch/x86/include/asm/mpx.h void mpx_notify_unmap(struct mm_struct * · line 81

Quoted 9 lines

static inline int kernel_managing_mpx_tables(struct mm_struct *mm) {

arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~use-new-tsk_get_xsave_addr +1 -1
 	return (mm->bd_addr != MPX_INVALID_BOUNDS_DIR);
@@ -78,11 +78,11 @@ void mpx_notify_unmap(struct mm_struct *
 		      unsigned long start, unsigned long end);
 #else
 static inline siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-					      struct xsave_struct *xsave_buf)
+					      struct task_struct *tsk)

But that makes no sense. Introduce a new instance just to remove it one patch later. So maybe cleaning up the *tsk thing first might be the better approach, but I let you decide.

Quoted 19 lines
arch/x86/kernel/traps.c renamed from arch/x86/kernel/traps.c~use-new-tsk_get_xsave_addr +1 -1
 #include <asm/mach_traps.h>
 #include <asm/alternative.h>
 #include <asm/mpx.h>
+#include <asm/xsave.h>
 
 #ifdef CONFIG_X86_64
 #include <asm/x86_init.h>
@@ -372,7 +373,6 @@ dotraplinkage void do_double_fault(struc
 dotraplinkage void do_bounds(struct pt_regs *regs, long error_code)
 {
 	struct task_struct *tsk = current;
-	struct xsave_struct *xsave_buf;
 	enum ctx_state prev_state;
 	struct bndcsr *bndcsr;
 	siginfo_t *info;
@@ -393,12 +393,11 @@ dotraplinkage void do_bounds(struct pt_r
 
 	/*
 	 * We need to look at BNDSTATUS to resolve this exception.

arch/x86/kernel/traps.c dotraplinkage void do_bounds(struct pt_r · line 400

Quoted 10 lines
arch/x86/kernel/traps.c renamed from arch/x86/kernel/traps.c~use-new-tsk_get_xsave_addr +4 -5
-	 * It is not directly accessible, though, so we need to
-	 * do an xsave and then pull it out of the xsave buffer.
+	 * A NULL here might mean that it is in its 'init state',
+	 * which is all zeros which indicates MPX was not
+	 * responsible for the exception.
 	 */
-	fpu_save_init(&tsk->thread.fpu);
-	xsave_buf = &(tsk->thread.fpu.state->xsave);
-	bndcsr = get_xsave_addr(xsave_buf, XSTATE_BNDCSR);
+	bndcsr = get_xsave_field(XSTATE_BNDCSR);

Hmm, I did not notice when I looked at the patch which introduced get_xsave_field(). But with the removed get_xsave_addr() I really had to look twice.

get_xsave_field() suggest that you get the field content. So wouldn't it be better to name it get_xsave_field_ptr() or such?

Quoted 21 lines
arch/x86/kernel/traps.c renamed from arch/x86/kernel/traps.c~use-new-tsk_get_xsave_addr +2 -2
 	if (!bndcsr)
 		goto exit_trap;
 
@@ -409,11 +408,11 @@ dotraplinkage void do_bounds(struct pt_r
 	 */
 	switch (bndcsr->bndstatus & MPX_BNDSTA_ERROR_CODE) {
 	case 2:	/* Bound directory has invalid entry. */
-		if (mpx_handle_bd_fault(xsave_buf))
+		if (mpx_handle_bd_fault(tsk))
 			goto exit_trap;
 		break; /* Success, it was handled */
 	case 1: /* Bound violation. */
-		info = mpx_generate_siginfo(regs, xsave_buf);
+		info = mpx_generate_siginfo(regs, tsk);
 		if (IS_ERR(info)) {
 			/*
 			 * We failed to decode the MPX instruction.  Act as if

diff -puN arch/x86/mm/mpx.c~use-new-tsk_get_xsave_addr arch/x86/mm/mpx.c

arch/x86/mm/mpx.c bad_opcode: · line 276

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~use-new-tsk_get_xsave_addr +1 -1
@@ -273,7 +273,7 @@ bad_opcode:
  * The caller is expected to kfree() the returned siginfo_t.
  */
 siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-				struct xsave_struct *xsave_buf)
+				struct task_struct *tsk)

Looking at the implementation makes me wonder about the extra *tsk argument some more ....

arch/x86/mm/mpx.c siginfo_t *mpx_generate_siginfo(struct p · line 298

Quoted 10 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~use-new-tsk_get_xsave_addr +1 -2
 {
 	struct bndreg *bndregs, *bndreg;
 	siginfo_t *info = NULL;
@@ -295,8 +295,8 @@ siginfo_t *mpx_generate_siginfo(struct p
 		err = -EINVAL;
 		goto err_out;
 	}
-	/* get the bndregs _area_ of the xsave structure */
-	bndregs = get_xsave_addr(xsave_buf, XSTATE_BNDREGS);
+	/* get bndregs field from urrent task's xsave area */

urrent?

arch/x86/mm/mpx.c siginfo_t *mpx_generate_siginfo(struct p · line 302

Quoted 4 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~use-new-tsk_get_xsave_addr +1 -0
+	bndregs = get_xsave_field(XSTATE_BNDREGS);
 	if (!bndregs) {
 		err = -EINVAL;
 		goto err_out;

Still no user of *tsk ....

arch/x86/mm/mpx.c int mpx_handle_bd_fault(struct xsave_str · line 537

Quoted 12 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~use-new-tsk_get_xsave_addr +2 -2
@@ -358,8 +358,7 @@ static __user void *task_get_bounds_dir(
-int mpx_handle_bd_fault(struct xsave_struct *xsave_buf)
+int mpx_handle_bd_fault(struct task_struct *tsk)
 {
 	/*
 	 * Userspace never asked us to manage the bounds tables,
@@ -535,7 +534,7 @@ int mpx_handle_bd_fault(struct xsave_str
 	if (!kernel_managing_mpx_tables(current->mm))
 		return -EINVAL;
 
-	if (do_mpx_bt_fault(xsave_buf)) {
+	if (do_mpx_bt_fault(tsk)) {

Same problem actually with do_mpx_bt_fault().

Thanks,

tglx

#8 lore raw

Re: [PATCH 03/19] x86, mpx: use new tsk_get_xsave_addr()

Quoted 8 lines · Thomas Gleixner
Quoted 6 lines

arch/x86/include/asm/mpx.h line 64

Quoted 6 lines

#ifdef CONFIG_X86_INTEL_MPX
siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,

arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~use-new-tsk_get_xsave_addr +2 -2
-				struct xsave_struct *xsave_buf);
-int mpx_handle_bd_fault(struct xsave_struct *xsave_buf);
+				struct task_struct *tsk);
+int mpx_handle_bd_fault(struct task_struct *tsk);

So you still keep *tsk and remove it in the next patch. I guess either ordering has its value, so no problem to do it this way.

I agree that this looks a bit goofy.

Originally, this patch just passed around 'tsk' instead of 'xsave_buf'.
But I realized we could go a step further.

The way it stands, it is bisectable and been built in this configuration for a month or two. I'd really like to keep it this way if possible, even though it looks a bit goofy.

#9 lore raw

PATCH 4/19 x86, mpx: cleanup: do not pass task around when unnecessary

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

The MPX code can largely only work on the current task. You can not, for instance, enable MPX management in another process or thread. You can also not handle a fault for another process or thread.

Despite this, we pass a task_struct around prolifically. This patch removes all of the task struct passing for code paths where the code can not deal with another task (which turns out to be all of them).

This has no functional changes. It's just a cleanup.

Cc Oleg Nesterov <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc the arch/x86 maintainers <[email protected]>
Cc linux-kernel <[email protected]>

---

 b/arch/x86/include/asm/mpx.h       |   10 ++++------
 b/arch/x86/include/asm/processor.h |   12 ++++++------
 b/arch/x86/kernel/traps.c          |    5 ++---
 b/arch/x86/mm/mpx.c                |   19 +++++++++----------
 b/kernel/sys.c                     |    8 ++++----
 5 files changed, 25 insertions(+), 29 deletions(-)
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~x86-mpx-dont-pass-current-around +4 -6
diff -puN arch/x86/include/asm/mpx.h~x86-mpx-dont-pass-current-around arch/x86/include/asm/mpx.h
@@ -59,9 +59,8 @@
 		MPX_BT_ENTRY_MASK) << MPX_BT_ENTRY_SHIFT)
 
 #ifdef CONFIG_X86_INTEL_MPX
-siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-				struct task_struct *tsk);
-int mpx_handle_bd_fault(struct task_struct *tsk);
+siginfo_t *mpx_generate_siginfo(struct pt_regs *regs);
+int mpx_handle_bd_fault(void);
 static inline int kernel_managing_mpx_tables(struct mm_struct *mm)
 {
 	return (mm->bd_addr != MPX_INVALID_BOUNDS_DIR);
@@ -77,12 +76,11 @@ static inline void mpx_mm_init(struct mm
 void mpx_notify_unmap(struct mm_struct *mm, struct vm_area_struct *vma,
 		      unsigned long start, unsigned long end);
 #else
-static inline siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-					      struct task_struct *tsk)
+static inline siginfo_t *mpx_generate_siginfo(struct pt_regs *regs)
 {
 	return NULL;
 }
-static inline int mpx_handle_bd_fault(struct task_struct *tsk)
+static inline int mpx_handle_bd_fault(void)
 {
 	return -EINVAL;
 }
arch/x86/include/asm/processor.h renamed from arch/x86/include/asm/processor.h~x86-mpx-dont-pass-current-around +6 -6
diff -puN arch/x86/include/asm/processor.h~x86-mpx-dont-pass-current-around arch/x86/include/asm/processor.h
@@ -928,18 +928,18 @@ extern int get_tsc_mode(unsigned long ad
 extern int set_tsc_mode(unsigned int val);
 
 /* Register/unregister a process' MPX related resource */
-#define MPX_ENABLE_MANAGEMENT(tsk)	mpx_enable_management((tsk))
-#define MPX_DISABLE_MANAGEMENT(tsk)	mpx_disable_management((tsk))
+#define MPX_ENABLE_MANAGEMENT()	mpx_enable_management()
+#define MPX_DISABLE_MANAGEMENT()	mpx_disable_management()
 
 #ifdef CONFIG_X86_INTEL_MPX
-extern int mpx_enable_management(struct task_struct *tsk);
-extern int mpx_disable_management(struct task_struct *tsk);
+extern int mpx_enable_management(void);
+extern int mpx_disable_management(void);
 #else
-static inline int mpx_enable_management(struct task_struct *tsk)
+static inline int mpx_enable_management(void)
 {
 	return -EINVAL;
 }
-static inline int mpx_disable_management(struct task_struct *tsk)
+static inline int mpx_disable_management(void)
 {
 	return -EINVAL;
 }
arch/x86/kernel/traps.c renamed from arch/x86/kernel/traps.c~x86-mpx-dont-pass-current-around +2 -3
diff -puN arch/x86/kernel/traps.c~x86-mpx-dont-pass-current-around arch/x86/kernel/traps.c
@@ -372,7 +372,6 @@ dotraplinkage void do_double_fault(struc
 
 dotraplinkage void do_bounds(struct pt_regs *regs, long error_code)
 {
-	struct task_struct *tsk = current;
 	enum ctx_state prev_state;
 	struct bndcsr *bndcsr;
 	siginfo_t *info;
@@ -408,11 +407,11 @@ dotraplinkage void do_bounds(struct pt_r
 	 */
 	switch (bndcsr->bndstatus & MPX_BNDSTA_ERROR_CODE) {
 	case 2:	/* Bound directory has invalid entry. */
-		if (mpx_handle_bd_fault(tsk))
+		if (mpx_handle_bd_fault())
 			goto exit_trap;
 		break; /* Success, it was handled */
 	case 1: /* Bound violation. */
-		info = mpx_generate_siginfo(regs, tsk);
+		info = mpx_generate_siginfo(regs);
 		if (IS_ERR(info)) {
 			/*
 			 * We failed to decode the MPX instruction.  Act as if
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~x86-mpx-dont-pass-current-around +9 -10
diff -puN arch/x86/mm/mpx.c~x86-mpx-dont-pass-current-around arch/x86/mm/mpx.c
@@ -272,8 +272,7 @@ bad_opcode:
  *
  * The caller is expected to kfree() the returned siginfo_t.
  */
-siginfo_t *mpx_generate_siginfo(struct pt_regs *regs,
-				struct task_struct *tsk)
+siginfo_t *mpx_generate_siginfo(struct pt_regs *regs)
 {
 	struct bndreg *bndregs, *bndreg;
 	siginfo_t *info = NULL;
@@ -341,7 +340,7 @@ err_out:
 	return ERR_PTR(err);
 }
 
-static __user void *task_get_bounds_dir(struct task_struct *tsk)
+static __user void *mpx_get_bounds_dir(void)
 {
 	struct bndcsr *bndcsr;
 
@@ -377,10 +376,10 @@ static __user void *task_get_bounds_dir(
 		(bndcsr->bndcfgu & MPX_BNDCFG_ADDR_MASK);
 }
 
-int mpx_enable_management(struct task_struct *tsk)
+int mpx_enable_management(void)
 {
 	void __user *bd_base = MPX_INVALID_BOUNDS_DIR;
-	struct mm_struct *mm = tsk->mm;
+	struct mm_struct *mm = current->mm;
 	int ret = 0;
 
 	/*
@@ -393,7 +392,7 @@ int mpx_enable_management(struct task_st
 	 * directory here means that we do not have to do xsave in the
 	 * unmap path; we can just use mm->bd_addr instead.
 	 */
-	bd_base = task_get_bounds_dir(tsk);
+	bd_base = mpx_get_bounds_dir();
 	down_write(&mm->mmap_sem);
 	mm->bd_addr = bd_base;
 	if (mm->bd_addr == MPX_INVALID_BOUNDS_DIR)
@@ -403,7 +402,7 @@ int mpx_enable_management(struct task_st
 	return ret;
 }
 
-int mpx_disable_management(struct task_struct *tsk)
+int mpx_disable_management(void)
 {
 	struct mm_struct *mm = current->mm;
 
@@ -497,7 +496,7 @@ out_unmap:
  * bound table is 16KB. With 64-bit mode, the size of BD is 2GB,
  * and the size of each bound table is 4MB.
  */
-static int do_mpx_bt_fault(struct task_struct *tsk)
+static int do_mpx_bt_fault(void)
 {
 	unsigned long bd_entry, bd_base;
 	struct bndcsr *bndcsr;
@@ -525,7 +524,7 @@ static int do_mpx_bt_fault(struct task_s
 	return allocate_bt((long __user *)bd_entry);
 }
 
-int mpx_handle_bd_fault(struct task_struct *tsk)
+int mpx_handle_bd_fault(void)
 {
 	/*
 	 * Userspace never asked us to manage the bounds tables,
@@ -534,7 +533,7 @@ int mpx_handle_bd_fault(struct task_stru
 	if (!kernel_managing_mpx_tables(current->mm))
 		return -EINVAL;
 
-	if (do_mpx_bt_fault(tsk)) {
+	if (do_mpx_bt_fault()) {
 		force_sig(SIGSEGV, current);
 		/*
 		 * The force_sig() is essentially "handling" this
kernel/sys.c renamed from kernel/sys.c~x86-mpx-dont-pass-current-around +4 -4
diff -puN kernel/sys.c~x86-mpx-dont-pass-current-around kernel/sys.c
@@ -92,10 +92,10 @@
 # define SET_TSC_CTL(a)		(-EINVAL)
 #endif
 #ifndef MPX_ENABLE_MANAGEMENT
-# define MPX_ENABLE_MANAGEMENT(a)	(-EINVAL)
+# define MPX_ENABLE_MANAGEMENT()	(-EINVAL)
 #endif
 #ifndef MPX_DISABLE_MANAGEMENT
-# define MPX_DISABLE_MANAGEMENT(a)	(-EINVAL)
+# define MPX_DISABLE_MANAGEMENT()	(-EINVAL)
 #endif
 #ifndef GET_FP_MODE
 # define GET_FP_MODE(a)		(-EINVAL)
@@ -2230,12 +2230,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsi
 	case PR_MPX_ENABLE_MANAGEMENT:
 		if (arg2 || arg3 || arg4 || arg5)
 			return -EINVAL;
-		error = MPX_ENABLE_MANAGEMENT(me);
+		error = MPX_ENABLE_MANAGEMENT();
 		break;
 	case PR_MPX_DISABLE_MANAGEMENT:
 		if (arg2 || arg3 || arg4 || arg5)
 			return -EINVAL;
-		error = MPX_DISABLE_MANAGEMENT(me);
+		error = MPX_DISABLE_MANAGEMENT();
 		break;
 	case PR_SET_FP_MODE:
 		error = SET_FP_MODE(me, arg2);

_

#10 lore raw

Re: [PATCH 04/19] x86, mpx: cleanup: do not pass task around when unnecessary

Quoted 1 line · Dave Hansen

The MPX code can largely only work on the current task. You can

s/largely// ?

Quoted 10 lines

not, for instance, enable MPX management in another process or thread. You can also not handle a fault for another process or thread.

Despite this, we pass a task_struct around prolifically. This patch removes all of the task struct passing for code paths where the code can not deal with another task (which turns out to be all of them).

This has no functional changes. It's just a cleanup.

See also reply to previous patch.

Thanks,

tglx

#11 lore raw

PATCH 5/19 x86, mpx: remove redundant MPX_BNDCFG_ADDR_MASK

In reply to Dave Hansen

From: Qiaowei Ren <[email protected]>

MPX_BNDCFG_ADDR_MASK is defined two times, so this patch removes redundant one.

---

 b/arch/x86/include/asm/mpx.h |    1 -
 1 file changed, 1 deletion(-)
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0001-x86-mpx-remove-redundant-MPX_BNDCFG_ADDR_MASK +0 -1
diff -puN arch/x86/include/asm/mpx.h~0001-x86-mpx-remove-redundant-MPX_BNDCFG_ADDR_MASK arch/x86/include/asm/mpx.h
@@ -45,7 +45,6 @@
 #define MPX_BNDSTA_TAIL		2
 #define MPX_BNDCFG_TAIL		12
 #define MPX_BNDSTA_ADDR_MASK	(~((1UL<<MPX_BNDSTA_TAIL)-1))
-#define MPX_BNDCFG_ADDR_MASK	(~((1UL<<MPX_BNDCFG_TAIL)-1))
 #define MPX_BT_ADDR_MASK	(~((1UL<<MPX_BD_ENTRY_TAIL)-1))
 
 #define MPX_BNDCFG_ADDR_MASK	(~((1UL<<MPX_BNDCFG_TAIL)-1))

_

#13 lore raw

PATCH 6/19 x86, mpx: we do not allocate the bounds directory

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

The comment and code here are confusing. We do not currently allocate the bounds directory in the kernel.

---

 b/arch/x86/mm/mpx.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~x86-mpx-we-do-not-allocate-the-bounds-directory +2 -2
diff -puN arch/x86/mm/mpx.c~x86-mpx-we-do-not-allocate-the-bounds-directory arch/x86/mm/mpx.c
@@ -47,8 +47,8 @@ static unsigned long mpx_mmap(unsigned l
 	vm_flags_t vm_flags;
 	struct vm_area_struct *vma;
 
-	/* Only bounds table and bounds directory can be allocated here */
-	if (len != MPX_BD_SIZE_BYTES && len != MPX_BT_SIZE_BYTES)
+	/* Only bounds table can be allocated here */
+	if (len != MPX_BT_SIZE_BYTES)
 		return -EINVAL;
 
 	down_write(&mm->mmap_sem);

_

Reviewed-by on v1 6/19
#14 lore raw

Re: [PATCH 06/19] x86, mpx: we do not allocate the bounds directory

Quoted 1 line · Dave Hansen

Subject: x86, mpx: we do not allocate the bounds directory

led me to expect, that we actually do something like this.

x86, mpx: Restrict mmap size check to bounds tables

might be more accurate.

Quoted 7 lines

From: Dave Hansen <[email protected]>

The comment and code here are confusing. We do not currently allocate the bounds directory in the kernel.

Otherwise

#15 lore raw

PATCH 7/19 x86, mpx: boot-time disable

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

MPX has the _potential_ to cause some issues. Say part of your init system tried to protect one of its components from buffer overflows with MPX. If there were a false positive, it's possible that MPX could keep a system from booting.

MPX could also potentially cause performance issues since it is present in hot paths like the unmap path.

Allow it to be disabled at boot time.

---

 b/Documentation/kernel-parameters.txt |    4 ++++
 b/arch/x86/kernel/cpu/common.c        |   16 ++++++++++++++++
 2 files changed, 20 insertions(+)
arch/x86/kernel/cpu/common.c renamed from arch/x86/kernel/cpu/common.c~x86-mpx-disable-boot-time +16 -0
diff -puN arch/x86/kernel/cpu/common.c~x86-mpx-disable-boot-time arch/x86/kernel/cpu/common.c
@@ -172,6 +172,22 @@ static int __init x86_xsaves_setup(char
 }
 __setup("noxsaves", x86_xsaves_setup);
 
+static int __init x86_mpx_setup(char *s)
+{
+	/* require an exact match without trailing characters */
+	if (strlen(s))
+		return 0;
+
+	/* do not emit a message if the feature is not present */
+	if (!boot_cpu_has(X86_FEATURE_MPX))
+		return 1;
+
+	setup_clear_cpu_cap(X86_FEATURE_MPX);
+	pr_info("nompx: Intel Memory Protection Extensions (MPX) disabled\n");
+	return 1;
+}
+__setup("nompx", x86_mpx_setup);
+
 #ifdef CONFIG_X86_32
 static int cachesize_override = -1;
 static int disable_x86_serial_nr = 1;
Documentation/kernel-parameters.txt renamed from Documentation/kernel-parameters.txt~x86-mpx-disable-boot-time +4 -0
diff -puN Documentation/kernel-parameters.txt~x86-mpx-disable-boot-time Documentation/kernel-parameters.txt
@@ -937,6 +937,10 @@ bytes respectively. Such letter suffixes
 			Enable debug messages at boot time.  See
 			Documentation/dynamic-debug-howto.txt for details.
 
+	nompx		[X86] Disables Intel Memory Protection Extensions.
+			See Documentation/x86/intel_mpx.txt for more
+			information about the feature.
+
 	eagerfpu=	[X86]
 			on	enable eager fpu restore
 			off	disable eager fpu restore

_

Reviewed-by on v1 7/19
#16 lore raw

Re: [PATCH 07/19] x86, mpx: boot-time disable

Quoted 11 lines · Dave Hansen

MPX has the _potential_ to cause some issues. Say part of your init system tried to protect one of its components from buffer overflows with MPX. If there were a false positive, it's possible that MPX could keep a system from booting.

MPX could also potentially cause performance issues since it is present in hot paths like the unmap path.

Allow it to be disabled at boot time.

#17 lore raw

PATCH 8/19 x86, mpx: trace #BR exceptions

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

This is the first in a series of MPX tracing patches. I've found these extremely useful in the process of debugging applications and the kernel code itself.

This exception hooks in to the bounds (#BR) exception very early and allows capturing the key registers which would influence how the exception is handled.

Note that bndcfgu/bndstatus are technically still 64-bit registers even in 32-bit mode.

---

 b/arch/x86/include/asm/trace/mpx.h |   49 +++++++++++++++++++++++++++++++++++++
 b/arch/x86/kernel/traps.c          |    2 +
 b/arch/x86/mm/mpx.c                |    3 ++
 3 files changed, 54 insertions(+)
arch/x86/include/asm/trace/mpx.h new file +49 -0
diff -puN /dev/null arch/x86/include/asm/trace/mpx.h
@@ -0,0 +1,49 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM mpx
+
+#if !defined(_TRACE_MPX_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_MPX_H
+
+#include <linux/tracepoint.h>
+
+#ifdef CONFIG_X86_INTEL_MPX
+
+TRACE_EVENT(bounds_exception_mpx,
+
+	TP_PROTO(struct bndcsr *bndcsr),
+	TP_ARGS(bndcsr),
+
+	TP_STRUCT__entry(
+		__field(u64, bndcfgu)
+		__field(u64, bndstatus)
+	),
+
+	TP_fast_assign(
+		__entry->bndcfgu   = bndcsr->bndcfgu;
+		__entry->bndstatus = bndcsr->bndstatus;
+	),
+
+	TP_printk("bndcfgu:0x%llx bndstatus:0x%llx",
+		__entry->bndcfgu,
+		__entry->bndstatus)
+);
+
+#else
+
+/*
+ * This gets used outside of MPX-specific code, so we need a stub.
+ */
+static inline void trace_bounds_exception_mpx(struct bndcsr *bndcsr)
+{
+}
+
+#endif /* CONFIG_X86_INTEL_MPX */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH asm/trace/
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE mpx
+#endif /* _TRACE_MPX_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
arch/x86/kernel/traps.c renamed from arch/x86/kernel/traps.c~x86-br-exception-trace +2 -0
diff -puN arch/x86/kernel/traps.c~x86-br-exception-trace arch/x86/kernel/traps.c
@@ -61,6 +61,7 @@
 #include <asm/mach_traps.h>
 #include <asm/alternative.h>
 #include <asm/mpx.h>
+#include <asm/trace/mpx.h>
 #include <asm/xsave.h>
 
 #ifdef CONFIG_X86_64
@@ -400,6 +401,7 @@ dotraplinkage void do_bounds(struct pt_r
 	if (!bndcsr)
 		goto exit_trap;
 
+	trace_bounds_exception_mpx(bndcsr);
 	/*
 	 * The error code field of the BNDSTATUS register communicates status
 	 * information of a bound range exception #BR or operation involving
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~x86-br-exception-trace +3 -0
diff -puN arch/x86/mm/mpx.c~x86-br-exception-trace arch/x86/mm/mpx.c
@@ -18,6 +18,9 @@
 #include <asm/processor.h>
 #include <asm/fpu-internal.h>
 
+#define CREATE_TRACE_POINTS
+#include <asm/trace/mpx.h>
+
 static const char *mpx_mapping_name(struct vm_area_struct *vma)
 {
 	return "[mpx]";

_

Reviewed-by on v1 8/19
#18 lore raw

Re: [PATCH 08/19] x86, mpx: trace #BR exceptions

Quoted 4 lines · Dave Hansen

This is the first in a series of MPX tracing patches. I've found these extremely useful in the process of debugging applications and the kernel code itself.

Quoted 8 lines

This exception hooks in to the bounds (#BR) exception very early and allows capturing the key registers which would influence how the exception is handled.

Note that bndcfgu/bndstatus are technically still 64-bit registers even in 32-bit mode.

#19 lore raw

PATCH 9/19 x86, mpx: trace entry to bounds exception paths

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

There are two basic things that can happen as the result of a bounds exception (#BR):

1. We allocate a new bounds table
2. We pass up a bounds exception to userspace.

This patch adds a trace point for the case where we are passing the exception up to userspace with a signal.

We are also explicit that we're printing out the inverse of the 'upper' that we encounter. If you want to filter, for instance, you need to ~ the value first.

---

 b/arch/x86/include/asm/trace/mpx.h |   25 +++++++++++++++++++++++++
 b/arch/x86/mm/mpx.c                |    1 +
 2 files changed, 26 insertions(+)
arch/x86/include/asm/trace/mpx.h renamed from arch/x86/include/asm/trace/mpx.h~x86-mpx-trace-1 +25 -0
diff -puN arch/x86/include/asm/trace/mpx.h~x86-mpx-trace-1 arch/x86/include/asm/trace/mpx.h
@@ -8,6 +8,31 @@
 
 #ifdef CONFIG_X86_INTEL_MPX
 
+TRACE_EVENT(mpx_bounds_register_exception,
+
+	TP_PROTO(void *addr_referenced,
+		 struct bndreg *bndreg),
+	TP_ARGS(addr_referenced, bndreg),
+
+	TP_STRUCT__entry(
+		__field(void *, addr_referenced)
+		__field(u64, lower_bound)
+		__field(u64, upper_bound)
+	),
+
+	TP_fast_assign(
+		__entry->addr_referenced = addr_referenced;
+		__entry->lower_bound = bndreg->lower_bound;
+		__entry->upper_bound = bndreg->upper_bound;
+	),
+
+	TP_printk("address referenced: 0x%p bounds: lower: 0x%llx ~upper: 0x%llx",
+		__entry->addr_referenced,
+		__entry->lower_bound,
+		~__entry->upper_bound
+	)
+);
+
 TRACE_EVENT(bounds_exception_mpx,
 
 	TP_PROTO(struct bndcsr *bndcsr),
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~x86-mpx-trace-1 +1 -0
diff -puN arch/x86/mm/mpx.c~x86-mpx-trace-1 arch/x86/mm/mpx.c
@@ -336,6 +336,7 @@ siginfo_t *mpx_generate_siginfo(struct p
 		err = -EINVAL;
 		goto err_out;
 	}
+	trace_mpx_bounds_register_exception(info->si_addr, bndreg);
 	return info;
 err_out:
 	/* info might be NULL, but kfree() handles that */

_

#20 lore raw

Re: [PATCH 09/19] x86, mpx: trace entry to bounds exception paths

Quoted 6 lines · Dave Hansen

This patch adds a trace point for the case where we are passing the exception up to userspace with a signal.

We are also explicit that we're printing out the inverse of the 'upper' that we encounter. If you want to filter, for instance, you need to ~ the value first.

Confused. What is the point of printing the complement of upper?

Lets assume that my boundaries are: lower = 0x10000 upper = 0x1ffff

So the trace will tell me:

pointer: 0x40000 lower: 0x10000 ~upper: 0xfffffffffff70000

Even if the upper bound is not inclusive (and I can't be bothered to look it up right now), i.e. in the example above upper = 0x20000 we would get

pointer: 0x40000 lower: 0x10000 ~upper: 0xfffffffffffdffff

I have a hard time to understand the value of this over:

pointer: 0x40000 lower: 0x10000 upper: 0x20000

Thanks,

tglx

#21 lore raw

Re: [PATCH 09/19] x86, mpx: trace entry to bounds exception paths

Quoted 8 lines · Thomas Gleixner
Quoted 6 lines · Dave Hansen
Quoted 6 lines

This patch adds a trace point for the case where we are passing the exception up to userspace with a signal.

We are also explicit that we're printing out the inverse of the 'upper' that we encounter. If you want to filter, for instance, you need to ~ the value first.

Confused. What is the point of printing the complement of upper?

If a pointer's range is

	0x1000 -> 0x2000

it is stored in the bounds table as (32-bits here for brevity):

	lower: 0x00001000
	upper: 0xffffdfff

That is so that an all 0's entry:

	lower: 0x00000000
	upper: 0x00000000

corresponds to the "init" bounds which store a *range* of:

	0x00000000 -> 0xffffffff

The 'upper' stored in the table is gibberish to print by itself, so we print ~upper to get the *actual*, logical value.

#22 lore raw

Re: [PATCH 09/19] x86, mpx: trace entry to bounds exception paths

Quoted 24 lines · Dave Hansen
Quoted 8 lines · Thomas Gleixner
Quoted 6 lines · Dave Hansen
Quoted 6 lines

This patch adds a trace point for the case where we are passing the exception up to userspace with a signal.

We are also explicit that we're printing out the inverse of the 'upper' that we encounter. If you want to filter, for instance, you need to ~ the value first.

Confused. What is the point of printing the complement of upper?

If a pointer's range is

	0x1000 -> 0x2000

it is stored in the bounds table as (32-bits here for brevity):

	lower: 0x00001000
	upper: 0xffffdfff

That is so that an all 0's entry:

	lower: 0x00000000
	upper: 0x00000000
Quoted 6 lines

corresponds to the "init" bounds which store a *range* of:

	0x00000000 -> 0xffffffff

The 'upper' stored in the table is gibberish to print by itself, so we print ~upper to get the *actual*, logical value.

Some comment in the code and the changelog might be helpful then :)

Thanks,

tglx

#23 lore raw

PATCH 10/19 x86, mpx: trace ranged MPX operations

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

trace when MPX is zapping pages:

When MPX can not free an entire bounds table, it will instead try to zap unused parts of a bounds table to free the backing memory. This decreases RSS (resident set size) without decreasing the virtual space allocated for bounds tables.

trace attempts to find bounds tables:

This event traces any time we go looking to unmap a bounds table for a given virtual address range. This is useful to ensure that the kernel actually "tried" to free a bounds table versus times it succeeded.

It might try and fail if it realized that a table was shared with an adjacent VMA which is not being unmapped.

---

 b/arch/x86/include/asm/trace/mpx.h |   32 ++++++++++++++++++++++++++++++++
 b/arch/x86/mm/mpx.c                |    2 ++
 2 files changed, 34 insertions(+)
arch/x86/include/asm/trace/mpx.h renamed from arch/x86/include/asm/trace/mpx.h~mpx-trace_unmap_zap +32 -0
diff -puN arch/x86/include/asm/trace/mpx.h~mpx-trace_unmap_zap arch/x86/include/asm/trace/mpx.h
@@ -53,6 +53,38 @@ TRACE_EVENT(bounds_exception_mpx,
 		__entry->bndstatus)
 );
 
+DECLARE_EVENT_CLASS(mpx_range_trace,
+
+	TP_PROTO(unsigned long start,
+		 unsigned long end),
+	TP_ARGS(start, end),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, start)
+		__field(unsigned long, end)
+	),
+
+	TP_fast_assign(
+		__entry->start = start;
+		__entry->end   = end;
+	),
+
+	TP_printk("[0x%p:0x%p]",
+		(void *)__entry->start,
+		(void *)__entry->end
+	)
+);
+
+DEFINE_EVENT(mpx_range_trace, mpx_unmap_zap,
+	TP_PROTO(unsigned long start, unsigned long end),
+	TP_ARGS(start, end)
+);
+
+DEFINE_EVENT(mpx_range_trace, mpx_unmap_search,
+	TP_PROTO(unsigned long start, unsigned long end),
+	TP_ARGS(start, end)
+);
+
 #else
 
 /*
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-trace_unmap_zap +2 -0
diff -puN arch/x86/mm/mpx.c~mpx-trace_unmap_zap arch/x86/mm/mpx.c
@@ -668,6 +668,7 @@ static int zap_bt_entries(struct mm_stru
 
 		len = min(vma->vm_end, end) - addr;
 		zap_page_range(vma, addr, len, NULL);
+		trace_mpx_unmap_zap(addr, addr+len);
 
 		vma = vma->vm_next;
 		addr = vma->vm_start;
@@ -840,6 +841,7 @@ static int mpx_unmap_tables(struct mm_st
 	long __user *bd_entry, *bde_start, *bde_end;
 	unsigned long bt_addr;
 
+	trace_mpx_unmap_search(start, end);
 	/*
 	 * "Edge" bounds tables are those which are being used by the region
 	 * (start -> end), but that may be shared with adjacent areas.  If they

_

#24 lore raw

Re: [PATCH 10/19] x86, mpx: trace ranged MPX operations

Quoted 6 lines · Dave Hansen

When MPX can not free an entire bounds table, it will instead try to zap unused parts of a bounds table to free the backing memory. This decreases RSS (resident set size) without decreasing the virtual space allocated for bounds tables.

trace attempts to find bounds tables:

My first reading was: "The trace attempts to find bounds tables"

"Trace the attempts to find bounds tables" is what you actually mean.

Quoted 1 line

This event traces any time we go looking to unmap a bounds table

You have two events ....

Quoted 6 lines

for a given virtual address range. This is useful to ensure that the kernel actually "tried" to free a bounds table versus times it succeeded.

It might try and fail if it realized that a table was shared with an adjacent VMA which is not being unmapped.

So rewording that would probably create a useful changelog.

Thanks,

tglx

#25 lore raw

PATCH 11/19 x86, mpx: trace allocation of new bounds tables

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

Bounds tables are a significant consumer of memory. It is important to know when they are being allocated. Add a trace point to trace whenever an allocation occurs and also its virtual address.

---

 b/arch/x86/include/asm/trace/mpx.h |   16 ++++++++++++++++
 b/arch/x86/mm/mpx.c                |    1 +
 2 files changed, 17 insertions(+)
arch/x86/include/asm/trace/mpx.h renamed from arch/x86/include/asm/trace/mpx.h~trace_mpx_new_bounds_table +16 -0
diff -puN arch/x86/include/asm/trace/mpx.h~trace_mpx_new_bounds_table arch/x86/include/asm/trace/mpx.h
@@ -85,6 +85,22 @@ DEFINE_EVENT(mpx_range_trace, mpx_unmap_
 	TP_ARGS(start, end)
 );
 
+TRACE_EVENT(mpx_new_bounds_table,
+
+	TP_PROTO(unsigned long table_vaddr),
+	TP_ARGS(table_vaddr),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, table_vaddr)
+	),
+
+	TP_fast_assign(
+		__entry->table_vaddr = table_vaddr;
+	),
+
+	TP_printk("table vaddr:%p", (void *)__entry->table_vaddr)
+);
+
 #else
 
 /*
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~trace_mpx_new_bounds_table +1 -0
diff -puN arch/x86/mm/mpx.c~trace_mpx_new_bounds_table arch/x86/mm/mpx.c
@@ -483,6 +483,7 @@ static int allocate_bt(long __user *bd_e
 		ret = -EINVAL;
 		goto out_unmap;
 	}
+	trace_mpx_new_bounds_table(bt_addr);
 	return 0;
 out_unmap:
 	vm_munmap(bt_addr & MPX_BT_ADDR_MASK, MPX_BT_SIZE_BYTES);

_

#27 lore raw

PATCH 12/19 x86: make is_64bit_mm() widely available

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

The uprobes code has a nice helper, is_64bit_mm(), that consults both the runtime and compile-time flags for 32-bit support. Instead of reinventing the wheel, pull it in to an x86 header so we can use it for MPX.

I prefer passing the mm around to test_thread_flag(TIF_IA32) because it makes it explicit where the context is coming from. It will also make it easier if we ever need to access the MPX structures from another process.

---

 b/arch/x86/include/asm/mmu_context.h |   13 +++++++++++++
 b/arch/x86/kernel/uprobes.c          |   10 +---------
 2 files changed, 14 insertions(+), 9 deletions(-)
arch/x86/include/asm/mmu_context.h renamed from arch/x86/include/asm/mmu_context.h~x86-make-is_64bit_mm-available +13 -0
diff -puN arch/x86/include/asm/mmu_context.h~x86-make-is_64bit_mm-available arch/x86/include/asm/mmu_context.h
@@ -142,6 +142,19 @@ static inline void arch_exit_mmap(struct
 	paravirt_arch_exit_mmap(mm);
 }
 
+#ifdef CONFIG_X86_64
+static inline bool is_64bit_mm(struct mm_struct *mm)
+{
+	return	!config_enabled(CONFIG_IA32_EMULATION) ||
+		!(mm->context.ia32_compat == TIF_IA32);
+}
+#else
+static inline bool is_64bit_mm(struct mm_struct *mm)
+{
+	return false;
+}
+#endif
+
 static inline void arch_bprm_mm_init(struct mm_struct *mm,
 		struct vm_area_struct *vma)
 {
arch/x86/kernel/uprobes.c renamed from arch/x86/kernel/uprobes.c~x86-make-is_64bit_mm-available +1 -9
diff -puN arch/x86/kernel/uprobes.c~x86-make-is_64bit_mm-available arch/x86/kernel/uprobes.c
@@ -29,6 +29,7 @@
 #include <linux/kdebug.h>
 #include <asm/processor.h>
 #include <asm/insn.h>
+#include <asm/mmu_context.h>
 
 /* Post-execution fixups. */
 
@@ -312,11 +313,6 @@ static int uprobe_init_insn(struct arch_
 }
 
 #ifdef CONFIG_X86_64
-static inline bool is_64bit_mm(struct mm_struct *mm)
-{
-	return	!config_enabled(CONFIG_IA32_EMULATION) ||
-		!(mm->context.ia32_compat == TIF_IA32);
-}
 /*
  * If arch_uprobe->insn doesn't use rip-relative addressing, return
  * immediately.  Otherwise, rewrite the instruction so that it accesses
@@ -497,10 +493,6 @@ static void riprel_post_xol(struct arch_
 	}
 }
 #else /* 32-bit: */
-static inline bool is_64bit_mm(struct mm_struct *mm)
-{
-	return false;
-}
 /*
  * No RIP-relative addressing on 32-bit
  */

_

#28 lore raw

Re: [PATCH 12/19] x86: make is_64bit_mm() widely available

Quoted 7 lines · Dave Hansen

The uprobes code has a nice helper, is_64bit_mm(), that consults both the runtime and compile-time flags for 32-bit support. Instead of reinventing the wheel, pull it in to an x86 header so we can use it for MPX.

I prefer passing the mm around to test_thread_flag(TIF_IA32) because it makes it explicit where the context is coming from.

Ack.

Quoted 2 lines

It will also make it easier if we ever need to access the MPX structures from another process.

Do we really want to go there?

Otherwise: Reviewed-by: Thomas Gleixner <[email protected]>

#29 lore raw

PATCH 13/19 x86, mpx: Add temporary variable to reduce masking

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

When we allocate a bounds table, we call mmap(), then add a "valid" bit to the value before storing it in to the bounds directory.

If we fail along the way, we go and mask that valid bit _back_ out. That seems a little silly, and this makes it much more clear when we have a plain address versus an actual table _entry_.

---

 b/arch/x86/mm/mpx.c |    7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-remove-unnecessary-masking +4 -3
diff -puN arch/x86/mm/mpx.c~mpx-remove-unnecessary-masking arch/x86/mm/mpx.c
@@ -429,6 +429,7 @@ static int allocate_bt(long __user *bd_e
 	unsigned long expected_old_val = 0;
 	unsigned long actual_old_val = 0;
 	unsigned long bt_addr;
+	unsigned long bd_new_entry;
 	int ret = 0;
 
 	/*
@@ -441,7 +442,7 @@ static int allocate_bt(long __user *bd_e
 	/*
 	 * Set the valid flag (kinda like _PAGE_PRESENT in a pte)
 	 */
-	bt_addr = bt_addr | MPX_BD_ENTRY_VALID_FLAG;
+	bd_new_entry = bt_addr | MPX_BD_ENTRY_VALID_FLAG;
 
 	/*
 	 * Go poke the address of the new bounds table in to the
@@ -455,7 +456,7 @@ static int allocate_bt(long __user *bd_e
 	 * of the MPX code that have to pagefault_disable().
 	 */
 	ret = user_atomic_cmpxchg_inatomic(&actual_old_val, bd_entry,
-					   expected_old_val, bt_addr);
+					   expected_old_val, bd_new_entry);
 	if (ret)
 		goto out_unmap;
 
@@ -486,7 +487,7 @@ static int allocate_bt(long __user *bd_e
 	trace_mpx_new_bounds_table(bt_addr);
 	return 0;
 out_unmap:
-	vm_munmap(bt_addr & MPX_BT_ADDR_MASK, MPX_BT_SIZE_BYTES);
+	vm_munmap(bt_addr, MPX_BT_SIZE_BYTES);
 	return ret;
 }
 

_

Reviewed-by on v1 13/19
#30 lore raw

Re: [PATCH 13/19] x86, mpx: Add temporary variable to reduce masking

Quoted 10 lines · Dave Hansen

When we allocate a bounds table, we call mmap(), then add a "valid" bit to the value before storing it in to the bounds directory.

If we fail along the way, we go and mask that valid bit _back_ out. That seems a little silly, and this makes it much more clear when we have a plain address versus an actual table _entry_.

#31 lore raw

PATCH 14/19 x86, mpx: new directory entry to addr helper

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

Currently, to get from a bounds directory entry to the virtual address of a bounds table, we simply mask off a few low bits. However, the set of bits we mask off is different for 32 and 64-bit binaries.

This breaks the operation out in to a helper function and also adds a temporary variable to store the result until we are sure we are returning one.

---

 b/arch/x86/include/asm/mpx.h |    1 -
 b/arch/x86/mm/mpx.c          |   41 ++++++++++++++++++++++++++++++++++-------
 2 files changed, 34 insertions(+), 8 deletions(-)
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~mpx-new-entry-to-addr-helper +0 -1
diff -puN arch/x86/include/asm/mpx.h~mpx-new-entry-to-addr-helper arch/x86/include/asm/mpx.h
@@ -45,7 +45,6 @@
 #define MPX_BNDSTA_TAIL		2
 #define MPX_BNDCFG_TAIL		12
 #define MPX_BNDSTA_ADDR_MASK	(~((1UL<<MPX_BNDSTA_TAIL)-1))
-#define MPX_BT_ADDR_MASK	(~((1UL<<MPX_BD_ENTRY_TAIL)-1))
 
 #define MPX_BNDCFG_ADDR_MASK	(~((1UL<<MPX_BNDCFG_TAIL)-1))
 #define MPX_BNDSTA_ERROR_CODE	0x3
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-new-entry-to-addr-helper +34 -7

84 lines not on this page · raw message

_

#32 lore raw

Re: [PATCH 14/19] x86, mpx: new directory entry to addr helper

arch/x86/mm/mpx.c static int mpx_resolve_fault(long __user · line 580

Quoted 13 lines · Dave Hansen

Currently, to get from a bounds directory entry to the virtual address of a bounds table, we simply mask off a few low bits. However, the set of bits we mask off is different for 32 and 64-bit binaries.

This breaks the operation out in to a helper function and also adds a temporary variable to store the result until we are sure we are returning one.

arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-new-entry-to-addr-helper +2 -0
+static unsigned long mpx_bd_entry_to_bt_addr(struct mm_struct *mm,
+		unsigned long bd_entry)

Nit: Please adjust the second line so the arguments are horizontally aligned.

static unsigned long mpx_bd_entry_to_bt_addr(struct mm_struct *mm,
unsigned long bd_entry)

Aside of that: Reviewed-by: Thomas Gleixner <[email protected]>

#33 lore raw

PATCH 15/19 x86, mpx: do 32-bit-only cmpxchg for 32-bit apps

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

user_atomic_cmpxchg_inatomic() actually looks at sizeof(*ptr) to figure out how many bytes to copy. If we run it on a 64-bit kernel with a 64-bit pointer, it will copy a 64-bit bounds directory entry. That's fine, except when we have 32-bit programs with 32-bit bounds directory entries and we only *want* 32-bits.

This patch breaks the cmpxchg operation out in to its own function and performs the 32-bit type swizzling in there.

Note, the "64-bit" version of this code _would_ work on a 32-bit-only kernel. The issue this patch addresses is only for when the kernel's 'long' is mismatched from the size of the bounds directory entry of the process we are working on.

The new helper modifies 'actual_old_val' or returns an error. But gcc doesn't know this, so it warns about 'actual_old_val' being unused. Shut it up with an uninitialized_var().

---

 b/arch/x86/mm/mpx.c |   41 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 36 insertions(+), 5 deletions(-)
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-variable-sized-userspace-pokes +36 -5
diff -puN arch/x86/mm/mpx.c~mpx-variable-sized-userspace-pokes arch/x86/mm/mpx.c
@@ -419,6 +419,35 @@ int mpx_disable_management(void)
 	return 0;
 }
 
+static int mpx_cmpxchg_bd_entry(struct mm_struct *mm,
+		unsigned long *actual_old_val_ptr, long __user *bd_entry_addr,
+		unsigned long expected_old_val, unsigned long new_bd_entry)
+{
+	int ret;
+	/*
+	 * user_atomic_cmpxchg_inatomic() actually uses sizeof()
+	 * the pointer that we pass to it to figure out how much
+	 * data to cmpxchg.  We have to be careful here not to
+	 * pass a pointer to a 64-bit data type when we only want
+	 * a 32-bit copy.
+	 */
+	if (is_64bit_mm(mm)) {
+		ret = user_atomic_cmpxchg_inatomic(actual_old_val_ptr,
+				bd_entry_addr, expected_old_val, new_bd_entry);
+	} else {
+		u32 uninitialized_var(actual_old_val_32);
+		u32 expected_old_val_32 = expected_old_val;
+		u32 new_bd_entry_32 = new_bd_entry;
+		u32 __user *bd_entry_32 = (u32 __user *)bd_entry_addr;
+		ret = user_atomic_cmpxchg_inatomic(&actual_old_val_32,
+				bd_entry_32, expected_old_val_32,
+				new_bd_entry_32);
+		if (!ret)
+			*actual_old_val_ptr = actual_old_val_32;
+	}
+	return ret;
+}
+
 /*
  * With 32-bit mode, MPX_BT_SIZE_BYTES is 4MB, and the size of each
  * bounds table is 16KB. With 64-bit mode, MPX_BT_SIZE_BYTES is 2GB,
@@ -426,6 +455,7 @@ int mpx_disable_management(void)
  */
 static int allocate_bt(long __user *bd_entry)
 {
+	struct mm_struct *mm = current->mm;
 	unsigned long expected_old_val = 0;
 	unsigned long actual_old_val = 0;
 	unsigned long bt_addr;
@@ -455,8 +485,8 @@ static int allocate_bt(long __user *bd_e
 	 * mmap_sem at this point, unlike some of the other part
 	 * of the MPX code that have to pagefault_disable().
 	 */
-	ret = user_atomic_cmpxchg_inatomic(&actual_old_val, bd_entry,
-					   expected_old_val, bd_new_entry);
+	ret = mpx_cmpxchg_bd_entry(mm, &actual_old_val,	bd_entry,
+				   expected_old_val, bd_new_entry);
 	if (ret)
 		goto out_unmap;
 
@@ -710,15 +740,16 @@ static int unmap_single_bt(struct mm_str
 		long __user *bd_entry, unsigned long bt_addr)
 {
 	unsigned long expected_old_val = bt_addr | MPX_BD_ENTRY_VALID_FLAG;
-	unsigned long actual_old_val = 0;
+	unsigned long uninitialized_var(actual_old_val);
 	int ret;
 
 	while (1) {
 		int need_write = 1;
+		unsigned long cleared_bd_entry = 0;
 
 		pagefault_disable();
-		ret = user_atomic_cmpxchg_inatomic(&actual_old_val, bd_entry,
-						   expected_old_val, 0);
+		ret = mpx_cmpxchg_bd_entry(mm, &actual_old_val,
+				bd_entry, expected_old_val, cleared_bd_entry);
 		pagefault_enable();
 		if (!ret)
 			break;

_

#34 lore raw

Re: [PATCH 15/19] x86, mpx: do 32-bit-only cmpxchg for 32-bit apps

arch/x86/mm/mpx.c int mpx_disable_management(void) · line 424

Quoted 3 lines · Dave Hansen
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-variable-sized-userspace-pokes +3 -0
+static int mpx_cmpxchg_bd_entry(struct mm_struct *mm,
+		unsigned long *actual_old_val_ptr, long __user *bd_entry_addr,
+		unsigned long expected_old_val, unsigned long new_bd_entry)

I know that you tried to use descriptive argument names, but the result is close to camel case argument lists :)

Also why is *bd_entry_addr a long? Shouldn't that be unsigned long?

static int mpx_cmpxchg_bd_entry(struct mm_struct *mm,
unsigned long __user *addr, unsigned long oldval, unsigned long newval, unsigned long *curval)

or something like that should be sufficient, methinks.

Quoted 7 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-variable-sized-userspace-pokes +7 -0
+{
+	int ret;
+	/*
+	 * user_atomic_cmpxchg_inatomic() actually uses sizeof()
+	 * the pointer that we pass to it to figure out how much
+	 * data to cmpxchg.  We have to be careful here not to
+	 * pass a pointer to a 64-bit data type when we only want

arch/x86/mm/mpx.c int mpx_disable_management(void) · line 441

Quoted 10 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-variable-sized-userspace-pokes +10 -0
+	 * a 32-bit copy.
+	 */
+	if (is_64bit_mm(mm)) {
+		ret = user_atomic_cmpxchg_inatomic(actual_old_val_ptr,
+				bd_entry_addr, expected_old_val, new_bd_entry);
+	} else {
+		u32 uninitialized_var(actual_old_val_32);
+		u32 expected_old_val_32 = expected_old_val;
+		u32 new_bd_entry_32 = new_bd_entry;
+		u32 __user *bd_entry_32 = (u32 __user *)bd_entry_addr;

Newline between variables and code please.

arch/x86/mm/mpx.c int mpx_disable_management(void) · line 446

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-variable-sized-userspace-pokes +5 -0
+		ret = user_atomic_cmpxchg_inatomic(&actual_old_val_32,
+				bd_entry_32, expected_old_val_32,
+				new_bd_entry_32);
+		if (!ret)
+			*actual_old_val_ptr = actual_old_val_32;

You can update that unconditionally.

Thanks,

tglx

#35 lore raw

PATCH 16/19 x86, mpx: support 32-bit binaries on 64-bit kernel

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

Changes from v20:

* Fix macro confusion between BD and BT
* Add accessor for bt_entry_size_bytes()

Right now, the kernel can only switch between 64-bit and 32-bit binaries at compile time. This patch adds support for 32-bit binaries on 64-bit kernels when we support ia32 emulation.

We essentially choose which set of table sizes to use when doing arithmetic for the bounds table calculations.

This also uses a different approach for calculating the table indexes than before. I think the new one makes it much more clear what is going on, and allows us to share more code between the 32 and 64-bit cases.

---

 b/arch/x86/include/asm/mpx.h |   68 ++++++++--------
 b/arch/x86/mm/mpx.c          |  178 +++++++++++++++++++++++++++++++++++++------
 2 files changed, 191 insertions(+), 55 deletions(-)
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +34 -34
diff -puN arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel arch/x86/include/asm/mpx.h
@@ -13,49 +13,49 @@
 #define MPX_BNDCFG_ENABLE_FLAG	0x1
 #define MPX_BD_ENTRY_VALID_FLAG	0x1
 
-#ifdef CONFIG_X86_64
-
-/* upper 28 bits [47:20] of the virtual address in 64-bit used to
- * index into bounds directory (BD).
+/*
+ * The upper 28 bits [47:20] of the virtual address in 64-bit
+ * are used to index into bounds directory (BD).
+ *
+ * The directory is 2G (2^31) in size, and with 8-byte entries
+ * it has 2^28 entries.
  */
-#define MPX_BD_ENTRY_OFFSET	28
-#define MPX_BD_ENTRY_SHIFT	3
-/* bits [19:3] of the virtual address in 64-bit used to index into
- * bounds table (BT).
+#define MPX_BD_SIZE_BYTES_64	(1UL<<31)
+/* An entry is a long, so 8 bytes and a shift of 3 */
+#define MPX_BD_ENTRY_BYTES_64	8
+#define MPX_BD_NR_ENTRIES_64	(MPX_BD_SIZE_BYTES_64/MPX_BD_ENTRY_BYTES_64)
+
+/*
+ * The 32-bit directory is 4MB (2^22) in size, and with 4-byte
+ * entries it has 2^20 entries.
  */
-#define MPX_BT_ENTRY_OFFSET	17
-#define MPX_BT_ENTRY_SHIFT	5
-#define MPX_IGN_BITS		3
-#define MPX_BD_ENTRY_TAIL	3
-
-#else
-
-#define MPX_BD_ENTRY_OFFSET	20
-#define MPX_BD_ENTRY_SHIFT	2
-#define MPX_BT_ENTRY_OFFSET	10
-#define MPX_BT_ENTRY_SHIFT	4
-#define MPX_IGN_BITS		2
-#define MPX_BD_ENTRY_TAIL	2
-
-#endif
-
-#define MPX_BD_SIZE_BYTES (1UL<<(MPX_BD_ENTRY_OFFSET+MPX_BD_ENTRY_SHIFT))
-#define MPX_BT_SIZE_BYTES (1UL<<(MPX_BT_ENTRY_OFFSET+MPX_BT_ENTRY_SHIFT))
+#define MPX_BD_SIZE_BYTES_32	(1UL<<22)
+/* An entry is a long, so 4 bytes and a shift of 2 */
+#define MPX_BD_ENTRY_BYTES_32	4
+#define MPX_BD_NR_ENTRIES_32	(MPX_BD_SIZE_BYTES_32/MPX_BD_ENTRY_BYTES_32)
+
+/*
+ * A 64-bit table is 4MB total in size, and an entry is
+ * 4 64-bit pointers in size.
+ */
+#define MPX_BT_SIZE_BYTES_64	(1UL<<22)
+#define MPX_BT_ENTRY_BYTES_64	32
+#define MPX_BT_NR_ENTRIES_64	(MPX_BT_SIZE_BYTES_64/MPX_BT_ENTRY_BYTES_64)
+
+/*
+ * A 32-bit table is 16kB total in size, and an entry is
+ * 4 32-bit pointers in size.
+ */
+#define MPX_BT_SIZE_BYTES_32	(1UL<<14)
+#define MPX_BT_ENTRY_BYTES_32	16
+#define MPX_BT_NR_ENTRIES_32	(MPX_BT_SIZE_BYTES_32/MPX_BT_ENTRY_BYTES_32)
 
 #define MPX_BNDSTA_TAIL		2
 #define MPX_BNDCFG_TAIL		12
 #define MPX_BNDSTA_ADDR_MASK	(~((1UL<<MPX_BNDSTA_TAIL)-1))
-
 #define MPX_BNDCFG_ADDR_MASK	(~((1UL<<MPX_BNDCFG_TAIL)-1))
 #define MPX_BNDSTA_ERROR_CODE	0x3
 
-#define MPX_BD_ENTRY_MASK	((1<<MPX_BD_ENTRY_OFFSET)-1)
-#define MPX_BT_ENTRY_MASK	((1<<MPX_BT_ENTRY_OFFSET)-1)
-#define MPX_GET_BD_ENTRY_OFFSET(addr)	((((addr)>>(MPX_BT_ENTRY_OFFSET+ \
-		MPX_IGN_BITS)) & MPX_BD_ENTRY_MASK) << MPX_BD_ENTRY_SHIFT)
-#define MPX_GET_BT_ENTRY_OFFSET(addr)	((((addr)>>MPX_IGN_BITS) & \
-		MPX_BT_ENTRY_MASK) << MPX_BT_ENTRY_SHIFT)
-
 #ifdef CONFIG_X86_INTEL_MPX
 siginfo_t *mpx_generate_siginfo(struct pt_regs *regs);
 int mpx_handle_bd_fault(void);
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +157 -21

269 lines not on this page · raw message

_

#36 lore raw

Re: [PATCH 16/19] x86, mpx: support 32-bit binaries on 64-bit kernel

Quoted 7 lines · Dave Hansen

From: Dave Hansen <[email protected]>

Changes from v20:

* Fix macro confusion between BD and BT
* Add accessor for bt_entry_size_bytes()

Forgot to say this earlier. Please put the changes after the changelog itself, i.e. after the '---'

Quoted 7 lines
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +3 -4
-#ifdef CONFIG_X86_64
-
-/* upper 28 bits [47:20] of the virtual address in 64-bit used to
- * index into bounds directory (BD).
+/*
+ * The upper 28 bits [47:20] of the virtual address in 64-bit
+ * are used to index into bounds directory (BD).

arch/x86/include/asm/mpx.h line 24

Quoted 10 lines
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +5 -4
+ *
+ * The directory is 2G (2^31) in size, and with 8-byte entries
+ * it has 2^28 entries.
  */
-#define MPX_BD_ENTRY_OFFSET	28
-#define MPX_BD_ENTRY_SHIFT	3
-/* bits [19:3] of the virtual address in 64-bit used to index into
- * bounds table (BT).
+#define MPX_BD_SIZE_BYTES_64	(1UL<<31)
+/* An entry is a long, so 8 bytes and a shift of 3 */

I can see the 8 bytes, but where is the shift constant?

arch/x86/include/asm/mpx.h line 33

Quoted 10 lines
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +8 -0
+#define MPX_BD_ENTRY_BYTES_64	8
+#define MPX_BD_NR_ENTRIES_64	(MPX_BD_SIZE_BYTES_64/MPX_BD_ENTRY_BYTES_64)
 
+
+/*
+ * The 32-bit directory is 4MB (2^22) in size, and with 4-byte
+ * entries it has 2^20 entries.
  */
+#define MPX_BD_SIZE_BYTES_32	(1UL<<22)
+/* An entry is a long, so 4 bytes and a shift of 2 */

Ditto.

arch/x86/include/asm/mpx.h line 35

Quoted 2 lines
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +2 -0
+#define MPX_BD_ENTRY_BYTES_32	4
+#define MPX_BD_NR_ENTRIES_32	(MPX_BD_SIZE_BYTES_32/MPX_BD_ENTRY_BYTES_32)

Otherwise this macro zoo looks good.

Quoted 27 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +27 -0
+static inline int bt_entry_size_bytes(struct mm_struct *mm)
+{
+	if (is_64bit_mm(mm))
+		return MPX_BT_ENTRY_BYTES_64;
+	else
+		return MPX_BT_ENTRY_BYTES_32;
+}
+
+/*
+ * Take a virtual address and turns it in to the offset in bytes
+ * inside of the bounds table where the bounds table entry
+ * controlling 'addr' can be found.
+ */
+static unsigned long mpx_get_bt_entry_offset_bytes(struct mm_struct *mm,
+		unsigned long addr)
+{
+	unsigned long bt_table_nr_entries;
+	unsigned long offset = addr;
+
+	if (is_64bit_mm(mm)) {
+		/* Bottom 3 bits are ignored on 64-bit */
+		offset >>= 3;
+		bt_table_nr_entries = MPX_BT_NR_ENTRIES_64;
+	} else {
+		/* Bottom 2 bits are ignored on 32-bit */
+		offset >>= 2;
+		bt_table_nr_entries = MPX_BT_NR_ENTRIES_32;

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 847

Quoted 10 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +10 -0
+	}
+	/*
+	 * We know the size of the table in to which we are
+	 * indexing, and we have eliminated all the low bits
+	 * which are ignored for indexing.
+	 *
+	 * Mask out all the high bits which we do not need
+	 * to index in to the table.
+	 */
+	offset &= (bt_table_nr_entries-1);

.... entries - 1);

And you might explain why nr_entries - 1 is a proper mask, i.e. nr_entries is always a power of 2.

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 852

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +5 -0
+	/*
+	 * We now have an entry offset in terms of *entries* in
+	 * the table.  We need to scale it back up to bytes.
+	 */
+	offset *= bt_entry_size_bytes(mm);

You could store the scale value out in the if () construct above, but I guess the compiler can figure that out as well :)

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +5 -0
+	return offset;
+}
+
+/*
+ * Total size of the process's virtual address space

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 867

Quoted 10 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +10 -0
+ * Use a u64 because 4GB (for 32-bit) won't fit in a long.
+ *
+ * __VIRTUAL_MASK does not work here.  It only covers the
+ * user address space and the tables cover the *entire*
+ * virtual address space supported on the CPU.
+ */
+static inline unsigned long long mm_virt_space(struct mm_struct *mm)
+{
+	if (is_64bit_mm(mm))
+		return 1ULL << 48;

cpu_info->x86_phys_bits will tell you the proper value

Review

Quoted 2 lines
(quoted diff) +2 -0
+	else
+		return 1ULL << 32;

And for a 32bit kernel 32 might be wrong because with PAE you have 36 bits.

Quoted 29 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +29 -0
+/*
+ * How much virtual address space does a single bounds
+ * directory entry cover?
+ */
+static inline unsigned long bd_entry_virt_space(struct mm_struct *mm)
+{
+	if (is_64bit_mm(mm))
+		return mm_virt_space(mm) / MPX_BD_NR_ENTRIES_64;
+	else
+		return mm_virt_space(mm) / MPX_BD_NR_ENTRIES_32;
+}
+
+/*
+ * Return an offset in terms of bytes in to the bounds
+ * directory where the bounds directory entry for a given
+ * virtual address resides.
+ *
+ * This has to be in bytes because the directory entries
+ * are different sizes on 64/32 bit.
+ */
+static unsigned long mpx_get_bd_entry_offset(struct mm_struct *mm,
+		unsigned long addr)
+{
+	/*
+	 * There are several ways to derive the bd offsets.  We
+	 * use the following approach here:
+	 * 1. We know the size of the virtual address space
+	 * 2. We know the number of entries in a bounds table
+	 * 3. We know that each entry covers a fixed amount of

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 910

Quoted 10 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +10 -0
+	 *    virtual address space.
+	 * So, we can just divide the virtual address by the
+	 * virtual space used by one entry to determine which
+	 * entry "controls" the given virtual address.
+	 */
+	if (is_64bit_mm(mm)) {
+		int bd_entry_size = 8; /* 64-bit pointer */
+		/*
+		 * Take the 64-bit addressing hole in to account.
+		 * This is a noop on 32-bit since it has no hole.

But a 32bit kernel will not take this code path because is_64bit_mm(mm) evaluates to false.

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 923

Quoted 13 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +13 -0
+		 */
+		addr &= ~(mm_virt_space(mm) - 1);
+		return (addr / bd_entry_virt_space(mm)) * bd_entry_size;
+	} else {
+		int bd_entry_size = 4; /* 32-bit pointer */
+		return (addr / bd_entry_virt_space(mm)) * bd_entry_size;
+	}
+	/*
+	 * The two return calls above are exact copies.  If we
+	 * pull out a single copy and put it in here, gcc won't
+	 * realize that we're doing a power-of-2 divide and use
+	 * shifts.  It uses a real divide.  If we put them up
+	 * there, it manages to figure it out (gcc 4.8.3).

Can't we provide the shift values from bd_entry_virt_space() so we don't have to worry about gcc versions being more or less clever?

Thanks,

tglx

#37 lore raw

Re: [PATCH 16/19] x86, mpx: support 32-bit binaries on 64-bit kernel

Quoted 24 lines · Thomas Gleixner
Quoted 7 lines · Dave Hansen

From: Dave Hansen <[email protected]>

Changes from v20:

* Fix macro confusion between BD and BT
* Add accessor for bt_entry_size_bytes()

Forgot to say this earlier. Please put the changes after the changelog itself, i.e. after the '---'

Quoted 11 lines
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +6 -4
-#ifdef CONFIG_X86_64
-
-/* upper 28 bits [47:20] of the virtual address in 64-bit used to
- * index into bounds directory (BD).
+/*
+ * The upper 28 bits [47:20] of the virtual address in 64-bit
+ * are used to index into bounds directory (BD).
+ *
+ * The directory is 2G (2^31) in size, and with 8-byte entries
+ * it has 2^28 entries.
  */
Quoted 8 lines

arch/x86/include/asm/mpx.h line 24

Quoted 6 lines
arch/x86/include/asm/mpx.h renamed from arch/x86/include/asm/mpx.h~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +2 -4
-#define MPX_BD_ENTRY_OFFSET	28
-#define MPX_BD_ENTRY_SHIFT	3
-/* bits [19:3] of the virtual address in 64-bit used to index into
- * bounds table (BT).
+#define MPX_BD_SIZE_BYTES_64	(1UL<<31)
+/* An entry is a long, so 8 bytes and a shift of 3 */

I can see the 8 bytes, but where is the shift constant?

The comment is old. I'll zap it.

Quoted 70 lines
Quoted 27 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +27 -0
+static inline int bt_entry_size_bytes(struct mm_struct *mm)
+{
+	if (is_64bit_mm(mm))
+		return MPX_BT_ENTRY_BYTES_64;
+	else
+		return MPX_BT_ENTRY_BYTES_32;
+}
+
+/*
+ * Take a virtual address and turns it in to the offset in bytes
+ * inside of the bounds table where the bounds table entry
+ * controlling 'addr' can be found.
+ */
+static unsigned long mpx_get_bt_entry_offset_bytes(struct mm_struct *mm,
+		unsigned long addr)
+{
+	unsigned long bt_table_nr_entries;
+	unsigned long offset = addr;
+
+	if (is_64bit_mm(mm)) {
+		/* Bottom 3 bits are ignored on 64-bit */
+		offset >>= 3;
+		bt_table_nr_entries = MPX_BT_NR_ENTRIES_64;
+	} else {
+		/* Bottom 2 bits are ignored on 32-bit */
+		offset >>= 2;
+		bt_table_nr_entries = MPX_BT_NR_ENTRIES_32;

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 847

Quoted 10 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +10 -0
+	}
+	/*
+	 * We know the size of the table in to which we are
+	 * indexing, and we have eliminated all the low bits
+	 * which are ignored for indexing.
+	 *
+	 * Mask out all the high bits which we do not need
+	 * to index in to the table.
+	 */
+	offset &= (bt_table_nr_entries-1);

.... entries - 1);

And you might explain why nr_entries - 1 is a proper mask, i.e. nr_entries is always a power of 2.

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 852

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +5 -0
+	/*
+	 * We now have an entry offset in terms of *entries* in
+	 * the table.  We need to scale it back up to bytes.
+	 */
+	offset *= bt_entry_size_bytes(mm);

You could store the scale value out in the if () construct above, but I guess the compiler can figure that out as well :)

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +5 -0
+	return offset;
+}
+
+/*
+ * Total size of the process's virtual address space

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 867

Quoted 10 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +10 -0
+ * Use a u64 because 4GB (for 32-bit) won't fit in a long.
+ *
+ * __VIRTUAL_MASK does not work here.  It only covers the
+ * user address space and the tables cover the *entire*
+ * virtual address space supported on the CPU.
+ */
+static inline unsigned long long mm_virt_space(struct mm_struct *mm)
+{
+	if (is_64bit_mm(mm))
+		return 1ULL << 48;

cpu_info->x86_phys_bits will tell you the proper value

Quoted 5 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	else
+		return 1ULL << 32;

And for a 32bit kernel 32 might be wrong because with PAE you have 36 bits.

That's physical space. I really do need virtual space here.

But your comments stand for ->x86_virt_bits. I'll fix.

Quoted 14 lines
Quoted 14 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +14 -0
+static unsigned long mpx_get_bd_entry_offset(struct mm_struct *mm,
+		unsigned long addr)
+{
+	/*
+	 * There are several ways to derive the bd offsets.  We
+	 * use the following approach here:
+	 * 1. We know the size of the virtual address space
+	 * 2. We know the number of entries in a bounds table
+	 * 3. We know that each entry covers a fixed amount of
+	 *    virtual address space.
+	 * So, we can just divide the virtual address by the
+	 * virtual space used by one entry to determine which
+	 * entry "controls" the given virtual address.
+	 */
Quoted 8 lines

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 910

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +5 -0
+	if (is_64bit_mm(mm)) {
+		int bd_entry_size = 8; /* 64-bit pointer */
+		/*
+		 * Take the 64-bit addressing hole in to account.
+		 * This is a noop on 32-bit since it has no hole.

But a 32bit kernel will not take this code path because is_64bit_mm(mm) evaluates to false.

I meant that in case someone wondered why I didn't have that code in the 32-bit version. I'll move the comment.

Quoted 8 lines
Quoted 8 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +8 -0
+		 */
+		addr &= ~(mm_virt_space(mm) - 1);
+		return (addr / bd_entry_virt_space(mm)) * bd_entry_size;
+	} else {
+		int bd_entry_size = 4; /* 32-bit pointer */
+		return (addr / bd_entry_virt_space(mm)) * bd_entry_size;
+	}
+	/*
Quoted 8 lines

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 923

Quoted 5 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +5 -0
+	 * The two return calls above are exact copies.  If we
+	 * pull out a single copy and put it in here, gcc won't
+	 * realize that we're doing a power-of-2 divide and use
+	 * shifts.  It uses a real divide.  If we put them up
+	 * there, it manages to figure it out (gcc 4.8.3).

Can't we provide the shift values from bd_entry_virt_space() so we don't have to worry about gcc versions being more or less clever?

Yes, I could go back and rework all the math to be done with shifts instead of power-of-2 divides (which is what was done before). But, it's very clear the way that it stands, minus this wart. The code look a *lot* better this way.

This isn't super performance-sensitive code. It's basically in the munmap() path. I just really didn't like the idea of an actual integer divide in there.

So, if GCC breaks this, so be it. I don't think we'll ever notice. The optimization was just too obvious to completely ignore.

#38 lore raw

Re: [PATCH 16/19] x86, mpx: support 32-bit binaries on 64-bit kernel

Quoted 16 lines · Dave Hansen
Quoted 15 lines · Thomas Gleixner

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 867

Quoted 12 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +12 -0
+/*
+ * Total size of the process's virtual address space
+ * Use a u64 because 4GB (for 32-bit) won't fit in a long.
+ *
+ * __VIRTUAL_MASK does not work here.  It only covers the
+ * user address space and the tables cover the *entire*
+ * virtual address space supported on the CPU.
+ */
+static inline unsigned long long mm_virt_space(struct mm_struct *mm)
+{
+	if (is_64bit_mm(mm))
+		return 1ULL << 48;

cpu_info->x86_phys_bits will tell you the proper value

Quoted 7 lines
Quoted 5 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	else
+		return 1ULL << 32;

And for a 32bit kernel 32 might be wrong because with PAE you have 36 bits.

That's physical space. I really do need virtual space here.

Stupid me

Quoted 15 lines
Quoted 9 lines

arch/x86/mm/mpx.c static int unmap_single_bt(struct mm_str · line 923

Quoted 6 lines
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~0002-x86-mpx-support-32bit-binaries-on-64bit-kernel +6 -0
+	/*
+	 * The two return calls above are exact copies.  If we
+	 * pull out a single copy and put it in here, gcc won't
+	 * realize that we're doing a power-of-2 divide and use
+	 * shifts.  It uses a real divide.  If we put them up
+	 * there, it manages to figure it out (gcc 4.8.3).

Can't we provide the shift values from bd_entry_virt_space() so we don't have to worry about gcc versions being more or less clever?

Yes, I could go back and rework all the math to be done with shifts instead of power-of-2 divides (which is what was done before). But, it's very clear the way that it stands, minus this wart. The code look a *lot* better this way.

Quoted 6 lines

This isn't super performance-sensitive code. It's basically in the munmap() path. I just really didn't like the idea of an actual integer divide in there.

So, if GCC breaks this, so be it. I don't think we'll ever notice. The optimization was just too obvious to completely ignore.

Fair enough.

Thanks,

tglx

#39 lore raw

PATCH 17/19 x86, mpx: rewrite unmap code

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

The MPX code needs to clear out bounds tables for memory which is no longer in use. We do this when a userspace mapping is torn down (unmapped).

There are two modes:
1. An entire bounds table becomes unused, and can be freed and its pointer removed from the bounds directory. This happens either when a large mapping is torn down, or when a small mapping is torn down and it is the last mapping "covered" by a bounds table.
2. Only part of a bounds table becomes unused, in which case we free the backing memory as if MADV_DONTNEED was called.

The old code was a spaghetti mess of "edge" bounds tables where the edges were handled specially, even if we were unmapping an entire one. Non-edge bounds tables are always fully unmapped, but share a different code path from the edge ones. The old code had a bug where it was unmapping too much memory. I worked on fixing it for two days and gave up.

I didn't write the original code. I didn't particularly like it, but it worked, so I left it. After my debug session, I realized it was undebuggagle *and* buggy, so out it went.

I also wrote a new unmapping test program which uncovers bugs pretty nicely.

---

 b/arch/x86/mm/mpx.c |  413 +++++++++++++++++++++-------------------------------
 1 file changed, 169 insertions(+), 244 deletions(-)
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~rewrite-unmap-code +169 -244
diff -puN arch/x86/mm/mpx.c~rewrite-unmap-code arch/x86/mm/mpx.c
@@ -704,110 +704,6 @@ static int get_bt_addr(struct mm_struct
 	return 0;
 }
 
-/*
- * Free the backing physical pages of bounds table 'bt_addr'.
- * Assume start...end is within that bounds table.
- */
-static int zap_bt_entries(struct mm_struct *mm,
-		unsigned long bt_addr,
-		unsigned long start, unsigned long end)
-{
-	struct vm_area_struct *vma;
-	unsigned long addr, len;
-
-	/*
-	 * Find the first overlapping vma. If vma->vm_start > start, there
-	 * will be a hole in the bounds table. This -EINVAL return will
-	 * cause a SIGSEGV.
-	 */
-	vma = find_vma(mm, start);
-	if (!vma || vma->vm_start > start)
-		return -EINVAL;
-
-	/*
-	 * A NUMA policy on a VM_MPX VMA could cause this bouds table to
-	 * be split. So we need to look across the entire 'start -> end'
-	 * range of this bounds table, find all of the VM_MPX VMAs, and
-	 * zap only those.
-	 */
-	addr = start;
-	while (vma && vma->vm_start < end) {
-		/*
-		 * We followed a bounds directory entry down
-		 * here.  If we find a non-MPX VMA, that's bad,
-		 * so stop immediately and return an error.  This
-		 * probably results in a SIGSEGV.
-		 */
-		if (!is_mpx_vma(vma))
-			return -EINVAL;
-
-		len = min(vma->vm_end, end) - addr;
-		zap_page_range(vma, addr, len, NULL);
-		trace_mpx_unmap_zap(addr, addr+len);
-
-		vma = vma->vm_next;
-		addr = vma->vm_start;
-	}
-
-	return 0;
-}
-
-static int unmap_single_bt(struct mm_struct *mm,
-		long __user *bd_entry, unsigned long bt_addr)
-{
-	unsigned long expected_old_val = bt_addr | MPX_BD_ENTRY_VALID_FLAG;
-	unsigned long uninitialized_var(actual_old_val);
-	int ret;
-
-	while (1) {
-		int need_write = 1;
-		unsigned long cleared_bd_entry = 0;
-
-		pagefault_disable();
-		ret = mpx_cmpxchg_bd_entry(mm, &actual_old_val,
-				bd_entry, expected_old_val, cleared_bd_entry);
-		pagefault_enable();
-		if (!ret)
-			break;
-		if (ret == -EFAULT)
-			ret = mpx_resolve_fault(bd_entry, need_write);
-		/*
-		 * If we could not resolve the fault, consider it
-		 * userspace's fault and error out.
-		 */
-		if (ret)
-			return ret;
-	}
-	/*
-	 * The cmpxchg was performed, check the results.
-	 */
-	if (actual_old_val != expected_old_val) {
-		/*
-		 * Someone else raced with us to unmap the table.
-		 * There was no bounds table pointed to by the
-		 * directory, so declare success.  Somebody freed
-		 * it.
-		 */
-		if (!actual_old_val)
-			return 0;
-		/*
-		 * Something messed with the bounds directory
-		 * entry.  We hold mmap_sem for read or write
-		 * here, so it could not be a _new_ bounds table
-		 * that someone just allocated.  Something is
-		 * wrong, so pass up the error and SIGSEGV.
-		 */
-		return -EINVAL;
-	}
-
-	/*
-	 * Note, we are likely being called under do_munmap() already. To
-	 * avoid recursion, do_munmap() will check whether it comes
-	 * from one bounds table through VM_MPX flag.
-	 */
-	return do_munmap(mm, bt_addr, mpx_bt_size_bytes(mm));
-}
-
 static inline int bt_entry_size_bytes(struct mm_struct *mm)
 {
 	if (is_64bit_mm(mm))
@@ -882,13 +778,69 @@ static inline unsigned long bd_entry_vir
 }
 
 /*
- * Return an offset in terms of bytes in to the bounds
- * directory where the bounds directory entry for a given
- * virtual address resides.
- *
- * This has to be in bytes because the directory entries
- * are different sizes on 64/32 bit.
+ * Free the backing physical pages of bounds table 'bt_addr'.
+ * Assume start...end is within that bounds table.
  */
+static noinline int zap_bt_entries_mapping(struct mm_struct *mm,
+		unsigned long bt_addr,
+		unsigned long start_mapping, unsigned long end_mapping)
+{
+	struct vm_area_struct *vma;
+	unsigned long addr, len;
+	unsigned long start;
+	unsigned long end;
+
+	/*
+	 * if we 'end' on a boundary, the offset will be 0 which
+	 * is not what we want.  Back it up a byte to get the
+	 * last bt entry.  Then once we have the entry itself,
+	 * move 'end' back up by the table entry size.
+	 */
+	start = bt_addr + mpx_get_bt_entry_offset_bytes(mm, start_mapping);
+	end   = bt_addr + mpx_get_bt_entry_offset_bytes(mm, end_mapping - 1);
+	/*
+	 * Move end back up by one entry.  Among other things
+	 * this ensures that it remains page-aligned and does
+	 * not screw up zap_page_range()
+	 */
+	end += bt_entry_size_bytes(mm);
+
+	/*
+	 * Find the first overlapping vma. If vma->vm_start > start, there
+	 * will be a hole in the bounds table. This -EINVAL return will
+	 * cause a SIGSEGV.
+	 */
+	vma = find_vma(mm, start);
+	if (!vma || vma->vm_start > start)
+		return -EINVAL;
+
+	/*
+	 * A NUMA policy on a VM_MPX VMA could cause this bounds table to
+	 * be split. So we need to look across the entire 'start -> end'
+	 * range of this bounds table, find all of the VM_MPX VMAs, and
+	 * zap only those.
+	 */
+	addr = start;
+	while (vma && vma->vm_start < end) {
+		/*
+		 * We followed a bounds directory entry down
+		 * here.  If we find a non-MPX VMA, that's bad,
+		 * so stop immediately and return an error.  This
+		 * probably results in a SIGSEGV.
+		 */
+		if (!is_mpx_vma(vma))
+			return -EINVAL;
+
+		len = min(vma->vm_end, end) - addr;
+		zap_page_range(vma, addr, len, NULL);
+		trace_mpx_unmap_zap(addr, addr+len);
+
+		vma = vma->vm_next;
+		addr = vma->vm_start;
+	}
+	return 0;
+}
+
 static unsigned long mpx_get_bd_entry_offset(struct mm_struct *mm,
 		unsigned long addr)
 {
@@ -909,7 +861,7 @@ static unsigned long mpx_get_bd_entry_of
 		 * Take the 64-bit addressing hole in to account.
 		 * This is a noop on 32-bit since it has no hole.
 		 */
-		addr &= ~(mm_virt_space(mm) - 1);
+		addr &= (mm_virt_space(mm) - 1);
 		return (addr / bd_entry_virt_space(mm)) * bd_entry_size;
 	} else {
 		int bd_entry_size = 4; /* 32-bit pointer */
@@ -924,69 +876,80 @@ static unsigned long mpx_get_bd_entry_of
 	 */
 }
 
-/*
- * If the bounds table pointed by bounds directory 'bd_entry' is
- * not shared, unmap this whole bounds table. Otherwise, only free
- * those backing physical pages of bounds table entries covered
- * in this virtual address region start...end.
- */
-static int unmap_shared_bt(struct mm_struct *mm,
-		long __user *bd_entry, unsigned long start,
-		unsigned long end, bool prev_shared, bool next_shared)
+static int unmap_entire_bt(struct mm_struct *mm,
+		long __user *bd_entry, unsigned long bt_addr)
 {
-	unsigned long bt_addr;
-	unsigned long start_off, end_off;
+	unsigned long expected_old_val = bt_addr | MPX_BD_ENTRY_VALID_FLAG;
+	unsigned long uninitialized_var(actual_old_val);
 	int ret;
 
-	ret = get_bt_addr(mm, bd_entry, &bt_addr);
+	while (1) {
+		int need_write = 1;
+		unsigned long cleared_bd_entry = 0;
+
+		pagefault_disable();
+		ret = mpx_cmpxchg_bd_entry(mm, &actual_old_val,
+				bd_entry, expected_old_val, cleared_bd_entry);
+		pagefault_enable();
+		if (!ret)
+			break;
+		if (ret == -EFAULT)
+			ret = mpx_resolve_fault(bd_entry, need_write);
+		/*
+		 * If we could not resolve the fault, consider it
+		 * userspace's fault and error out.
+		 */
+		if (ret)
+			return ret;
+	}
+	/*
+	 * The cmpxchg was performed, check the results.
+	 */
+	if (actual_old_val != expected_old_val) {
+		/*
+		 * Someone else raced with us to unmap the table.
+		 * That is OK, since we were both trying to do
+		 * the same thing.  Declare success.
+		 */
+		if (!actual_old_val)
+			return 0;
+		/*
+		 * Something messed with the bounds directory
+		 * entry.  We hold mmap_sem for read or write
+		 * here, so it could not be a _new_ bounds table
+		 * that someone just allocated.  Something is
+		 * wrong, so pass up the error and SIGSEGV.
+		 */
+		return -EINVAL;
+	}
 	/*
-	 * We could see an "error" ret for not-present bounds
-	 * tables (not really an error), or actual errors, but
-	 * stop unmapping either way.
+	 * Note, we are likely being called under do_munmap() already. To
+	 * avoid recursion, do_munmap() will check whether it comes
+	 * from one bounds table through VM_MPX flag.
 	 */
-	if (ret)
-		return ret;
-
-	start_off = mpx_get_bt_entry_offset_bytes(mm, start);
-	end_off   = mpx_get_bt_entry_offset_bytes(mm, end);
-
-	if (prev_shared && next_shared)
-		ret = zap_bt_entries(mm, bt_addr,
-				bt_addr + start_off,
-				bt_addr + end_off);
-	else if (prev_shared)
-		ret = zap_bt_entries(mm, bt_addr,
-				bt_addr + start_off,
-				bt_addr + mpx_bt_size_bytes(mm));
-	else if (next_shared)
-		ret = zap_bt_entries(mm, bt_addr, bt_addr,
-				bt_addr + end_off);
-	else
-		ret = unmap_single_bt(mm, bd_entry, bt_addr);
-
-	return ret;
+	return do_munmap(mm, bt_addr, mpx_bt_size_bytes(mm));
 }
 
-/*
- * A virtual address region being munmap()ed might share bounds table
- * with adjacent VMAs. We only need to free the backing physical
- * memory of these shared bounds tables entries covered in this virtual
- * address region.
- */
-static int unmap_edge_bts(struct mm_struct *mm,
-		unsigned long start, unsigned long end)
+static int try_unmap_single_bt(struct mm_struct *mm,
+	       unsigned long start, unsigned long end)
 {
+	struct vm_area_struct *next;
+	struct vm_area_struct *prev;
+	/*
+	 * "bta" == Bounds Table Area: the area controlled by the
+	 * bounds table that we are unmapping.
+	 */
+	unsigned long bta_start_vaddr = start & ~(bd_entry_virt_space(mm)-1);
+	unsigned long bta_end_vaddr = bta_start_vaddr + bd_entry_virt_space(mm);
+	unsigned long uninitialized_var(bt_addr);
+	void __user *bde_vaddr;
 	int ret;
-	long __user *bde_start, *bde_end;
-	struct vm_area_struct *prev, *next;
-	bool prev_shared = false, next_shared = false;
-
-	bde_start = mm->bd_addr + mpx_get_bd_entry_offset(mm, start);
-	bde_end   = mm->bd_addr + mpx_get_bd_entry_offset(mm, end-1);
-
 	/*
-	 * Check whether bde_start and bde_end are shared with adjacent
-	 * VMAs.
+	 * We know 'start' and 'end' lie within an area controlled
+	 * by a single bounds table.  See if there are any other
+	 * VMAs controlled by that bounds table.  If there are not
+	 * then we can "expand" the are we are unmapping to possibly
+	 * cover the entire table.
 	 *
 	 * We already unliked the VMAs from the mm's rbtree so 'start'
 	 * is guaranteed to be in a hole. This gets us the first VMA
@@ -994,102 +957,64 @@ static int unmap_edge_bts(struct mm_stru
 	 * in to 'next'.
 	 */
 	next = find_vma_prev(mm, start, &prev);
-	if (prev && (mm->bd_addr + mpx_get_bd_entry_offset(mm, prev->vm_end-1))
-			== bde_start)
-		prev_shared = true;
-	if (next && (mm->bd_addr + mpx_get_bd_entry_offset(mm, next->vm_start))
-			== bde_end)
-		next_shared = true;
-
-	/*
-	 * This virtual address region being munmap()ed is only
-	 * covered by one bounds table.
-	 *
-	 * In this case, if this table is also shared with adjacent
-	 * VMAs, only part of the backing physical memory of the bounds
-	 * table need be freeed. Otherwise the whole bounds table need
-	 * be unmapped.
-	 */
-	if (bde_start == bde_end) {
-		return unmap_shared_bt(mm, bde_start, start, end,
-				prev_shared, next_shared);
+	if ((!prev || prev->vm_end <= bta_start_vaddr) &&
+	    (!next || next->vm_start >= bta_end_vaddr)) {
+		/*
+		 * No neighbor VMAs controlled by same bounds
+		 * table.  Try to unmap the whole thing
+		 */
+		start = bta_start_vaddr;
+		end = bta_end_vaddr;
 	}
 
+	bde_vaddr = mm->bd_addr + mpx_get_bd_entry_offset(mm, start);
+	ret = get_bt_addr(mm, bde_vaddr, &bt_addr);
 	/*
-	 * If more than one bounds tables are covered in this virtual
-	 * address region being munmap()ed, we need to separately check
-	 * whether bde_start and bde_end are shared with adjacent VMAs.
+	 * No bounds table there, so nothing to unmap.
 	 */
-	ret = unmap_shared_bt(mm, bde_start, start, end, prev_shared, false);
-	if (ret)
-		return ret;
-	ret = unmap_shared_bt(mm, bde_end, start, end, false, next_shared);
+	if (ret == -ENOENT) {
+		ret = 0;
+		return 0;
+	}
 	if (ret)
 		return ret;
-
-	return 0;
+	/*
+	 * We are unmapping an entire table.  Either because the
+	 * unmap that started this whole process was large enough
+	 * to cover an entire table, or that the unmap was small
+	 * but was the area covered by a bounds table.
+	 */
+	if ((start == bta_start_vaddr) &&
+	    (end == bta_end_vaddr))
+		return unmap_entire_bt(mm, bde_vaddr, bt_addr);
+	return zap_bt_entries_mapping(mm, bt_addr, start, end);
 }
 
 static int mpx_unmap_tables(struct mm_struct *mm,
 		unsigned long start, unsigned long end)
 {
-	int ret;
-	long __user *bd_entry, *bde_start, *bde_end;
-	unsigned long bt_addr;
-
+	unsigned long one_unmap_start;
 	trace_mpx_unmap_search(start, end);
-	/*
-	 * "Edge" bounds tables are those which are being used by the region
-	 * (start -> end), but that may be shared with adjacent areas.  If they
-	 * turn out to be completely unshared, they will be freed.  If they are
-	 * shared, we will free the backing store (like an MADV_DONTNEED) for
-	 * areas used by this region.
-	 */
-	ret = unmap_edge_bts(mm, start, end);
-	switch (ret) {
-		/* non-present tables are OK */
-		case 0:
-		case -ENOENT:
-			/* Success, or no tables to unmap */
-			break;
-		case -EINVAL:
-		case -EFAULT:
-		default:
-			return ret;
-	}
-
-	/*
-	 * Only unmap the bounds table that are
-	 *   1. fully covered
-	 *   2. not at the edges of the mapping, even if full aligned
-	 */
-	bde_start = mm->bd_addr + mpx_get_bd_entry_offset(mm, start);
-	bde_end   = mm->bd_addr + mpx_get_bd_entry_offset(mm, end-1);
-	for (bd_entry = bde_start + 1; bd_entry < bde_end; bd_entry++) {
-		ret = get_bt_addr(mm, bd_entry, &bt_addr);
-		switch (ret) {
-			case 0:
-				break;
-			case -ENOENT:
-				/* No table here, try the next one */
-				continue;
-			case -EINVAL:
-			case -EFAULT:
-			default:
-				/*
-				 * Note: we are being strict here.
-				 * Any time we run in to an issue
-				 * unmapping tables, we stop and
-				 * SIGSEGV.
-				 */
-				return ret;
-		}
 
-		ret = unmap_single_bt(mm, bd_entry, bt_addr);
+	one_unmap_start = start;
+	while (one_unmap_start < end) {
+		int ret;
+		unsigned long next_unmap_start = ALIGN(one_unmap_start+1,
+						       bd_entry_virt_space(mm));
+		unsigned long one_unmap_end = end;
+		/*
+		 * if the end is beyond the current bounds table,
+		 * move it back so we only deal with a single one
+		 * at a time
+		 */
+		if (one_unmap_end > next_unmap_start)
+			one_unmap_end = next_unmap_start;
+		ret = try_unmap_single_bt(mm, one_unmap_start, one_unmap_end);
 		if (ret)
 			return ret;
-	}
 
+		one_unmap_start = next_unmap_start;
+	}
 	return 0;
 }
 

_

Reviewed-by on v1 17/19
#40 lore raw

Re: [PATCH 17/19] x86, mpx: rewrite unmap code

Quoted 20 lines · Dave Hansen

The MPX code needs to clear out bounds tables for memory which is no longer in use. We do this when a userspace mapping is torn down (unmapped).

There are two modes:
1. An entire bounds table becomes unused, and can be freed and its pointer removed from the bounds directory. This happens either when a large mapping is torn down, or when a small mapping is torn down and it is the last mapping "covered" by a bounds table.
2. Only part of a bounds table becomes unused, in which case we free the backing memory as if MADV_DONTNEED was called.

The old code was a spaghetti mess of "edge" bounds tables where the edges were handled specially, even if we were unmapping an entire one. Non-edge bounds tables are always fully unmapped, but share a different code path from the edge ones. The old code had a bug where it was unmapping too much memory. I worked on fixing it for two days and gave up.

Quoted 6 lines

I didn't write the original code. I didn't particularly like it, but it worked, so I left it. After my debug session, I realized it was undebuggagle *and* buggy, so out it went.

I also wrote a new unmapping test program which uncovers bugs pretty nicely.

Very appreciated. This is really readable now.

#41 lore raw

PATCH 18/19 x86, mpx: do not count MPX VMAs as neighbors when unmapping

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

The comment pretty much says it all.

I wrote a test program that does lots of random allocations and forces bounds tables to be created. It came up with a layout like this:

  ....   | BOUNDS DIRECTORY ENTRY COVERS |  ....
         |    BOUNDS TABLE COVERS        |

| BOUNDS TABLE | REAL ALLOC | BOUNDS TABLE |

Unmapping "REAL ALLOC" should have been able to free the bounds table "covering" the "REAL ALLOC" because it was the last real user. But, the neighboring VMA bounds tables were found, considered as real neighbors, and we declined to free the bounds table covering the area.

Doing this over and over left a small but significant number of these orphans. Handling them is fairly straighforward. All we have to do is walk the VMAs and skip all of the MPX ones when looking for neighbors.

---

 b/arch/x86/mm/mpx.c |   24 +++++++++++++++++++-----
 1 file changed, 19 insertions(+), 5 deletions(-)
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~mpx-dont-count-mpx-vmas-as-neighbors +19 -5
diff -puN arch/x86/mm/mpx.c~mpx-dont-count-mpx-vmas-as-neighbors arch/x86/mm/mpx.c
@@ -945,16 +945,30 @@ static int try_unmap_single_bt(struct mm
 	void __user *bde_vaddr;
 	int ret;
 	/*
+	 * We already unliked the VMAs from the mm's rbtree so 'start'
+	 * is guaranteed to be in a hole. This gets us the first VMA
+	 * before the hole in to 'prev' and the next VMA after the hole
+	 * in to 'next'.
+	 */
+	next = find_vma_prev(mm, start, &prev);
+	/*
+	 * Do not count other MPX bounds table VMAs as neighbors.
+	 * Although theoretically possible, we do not allow bounds
+	 * tables for bounds tables so our heads do not explode.
+	 * If we count them as neighbors here, we may end up with
+	 * lots of tables even though we have no actual table
+	 * entries in use.
+	 */
+	while (next && is_mpx_vma(next))
+		next = next->vm_next;
+	while (prev && is_mpx_vma(prev))
+		prev = prev->vm_prev;
+	/*
 	 * We know 'start' and 'end' lie within an area controlled
 	 * by a single bounds table.  See if there are any other
 	 * VMAs controlled by that bounds table.  If there are not
 	 * then we can "expand" the are we are unmapping to possibly
 	 * cover the entire table.
-	 *
-	 * We already unliked the VMAs from the mm's rbtree so 'start'
-	 * is guaranteed to be in a hole. This gets us the first VMA
-	 * before the hole in to 'prev' and the next VMA after the hole
-	 * in to 'next'.
 	 */
 	next = find_vma_prev(mm, start, &prev);
 	if ((!prev || prev->vm_end <= bta_start_vaddr) &&

_

Reviewed-by on v1 18/19
#42 lore raw

Re: [PATCH 18/19] x86, mpx: do not count MPX VMAs as neighbors when unmapping

Quoted 16 lines · Dave Hansen

The comment pretty much says it all.

I wrote a test program that does lots of random allocations and forces bounds tables to be created. It came up with a layout like this:

  ....   | BOUNDS DIRECTORY ENTRY COVERS |  ....
         |    BOUNDS TABLE COVERS        |

| BOUNDS TABLE | REAL ALLOC | BOUNDS TABLE |

Unmapping "REAL ALLOC" should have been able to free the bounds table "covering" the "REAL ALLOC" because it was the last real user. But, the neighboring VMA bounds tables were found, considered as real neighbors, and we declined to free the bounds table covering the area.

Quoted 6 lines

Doing this over and over left a small but significant number of these orphans. Handling them is fairly straighforward. All we have to do is walk the VMAs and skip all of the MPX ones when looking for neighbors.

#43 lore raw

PATCH 19/19 x86, mpx: allow mixed binaries again

In reply to Dave Hansen

From: Dave Hansen <[email protected]>

We explicitly disable allowing 32-bit binaries to enable MPX on 64-bit kernels. Re-allow that.

---

 b/arch/x86/mm/mpx.c |    6 ------
 1 file changed, 6 deletions(-)
arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~x86-mpx-allow-mixed-binaries-again +0 -6
diff -puN arch/x86/mm/mpx.c~x86-mpx-allow-mixed-binaries-again arch/x86/mm/mpx.c
@@ -368,12 +368,6 @@ static __user void *mpx_get_bounds_dir(v
 		return MPX_INVALID_BOUNDS_DIR;
 
 	/*
-	 * 32-bit binaries on 64-bit kernels are currently
-	 * unsupported.
-	 */
-	if (IS_ENABLED(CONFIG_X86_64) && test_thread_flag(TIF_IA32))
-		return MPX_INVALID_BOUNDS_DIR;
-	/*
 	 * The bounds directory pointer is stored in a register
 	 * only accessible if we first do an xsave.
 	 */

_

Reviewed-by on v1 19/19
#44 lore raw

Re: [PATCH 19/19] x86, mpx: allow mixed binaries again

Quoted 4 lines · Dave Hansen

We explicitly disable allowing 32-bit binaries to enable MPX on 64-bit kernels. Re-allow that.

Quoted 23 lines

---

 b/arch/x86/mm/mpx.c |    6 ------
 1 file changed, 6 deletions(-)

diff -puN arch/x86/mm/mpx.c~x86-mpx-allow-mixed-binaries-again arch/x86/mm/mpx.c

arch/x86/mm/mpx.c renamed from arch/x86/mm/mpx.c~x86-mpx-allow-mixed-binaries-again +0 -6
@@ -368,12 +368,6 @@ static __user void *mpx_get_bounds_dir(v
 		return MPX_INVALID_BOUNDS_DIR;
 
 	/*
-	 * 32-bit binaries on 64-bit kernels are currently
-	 * unsupported.
-	 */
-	if (IS_ENABLED(CONFIG_X86_64) && test_thread_flag(TIF_IA32))
-		return MPX_INVALID_BOUNDS_DIR;
-	/*
 	 * The bounds directory pointer is stored in a register
 	 * only accessible if we first do an xsave.
 	 */

_