Skip to contentSkip to conversation

x86: Verify access_ok() context

RFC PATCH In review Peter Zijlstra 6 participants 18 messages Nov 22, 2016 – Jan 20, 2017

Subsystems (2)
lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

RFC PATCH 1/1 x86: Verify access_ok() context

I recently encountered wreckage because access_ok() was used where it should not be, add an explicit WARN when access_ok() is used wrongly.

---
 arch/x86/include/asm/uaccess.h |  7 +++++--
 include/linux/preempt.h        | 21 +++++++++++++--------
 2 files changed, 18 insertions(+), 10 deletions(-)
arch/x86/include/asm/uaccess.h +5 -2
index faf3687f1035..b139c46ba122 100644
@@ -88,8 +88,11 @@ static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un
  * checks that the pointer is in the user space range - after calling
  * this function, memory access functions may still return -EFAULT.
  */
-#define access_ok(type, addr, size) \
-	likely(!__range_not_ok(addr, size, user_addr_max()))
+#define access_ok(type, addr, size)					\
+({									\
+	WARN_ON_ONCE(!in_task());					\
+	likely(!__range_not_ok(addr, size, user_addr_max()));		\
+})
 
 /*
  * These are the main single-value transfer routines.  They automatically
include/linux/preempt.h +13 -8
index 75e4e30677f1..7eeceac52dea 100644
@@ -65,19 +65,24 @@
 
 /*
  * Are we doing bottom half or hardware interrupt processing?
- * Are we in a softirq context? Interrupt context?
- * in_softirq - Are we currently processing softirq or have bh disabled?
- * in_serving_softirq - Are we currently processing softirq?
+ *
+ * in_irq()       - We're in (hard) IRQ context
+ * in_softirq()   - We have BH disabled, or are processing softirqs
+ * in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled
+ * in_serving_softirq() - We're in softirq context
+ * in_nmi()       - We're in NMI context
+ * in_task()	  - We're in task context
+ *
+ * Note: due to the BH disabled confusion: in_softirq(),in_interrupt() really
+ *       should not be used in new code.
  */
 #define in_irq()		(hardirq_count())
 #define in_softirq()		(softirq_count())
 #define in_interrupt()		(irq_count())
 #define in_serving_softirq()	(softirq_count() & SOFTIRQ_OFFSET)
-
-/*
- * Are we in NMI context?
- */
-#define in_nmi()	(preempt_count() & NMI_MASK)
+#define in_nmi()		(preempt_count() & NMI_MASK)
+#define in_task()		(!(preempt_count() & \
+				   (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))
 
 /*
  * The preempt_count offset after preempt_disable();
#1 lore raw
Quoted 15 lines · Peter Zijlstra

I recently encountered wreckage because access_ok() was used where it should not be, add an explicit WARN when access_ok() is used wrongly.

---
 arch/x86/include/asm/uaccess.h |  7 +++++--
 include/linux/preempt.h        | 21 +++++++++++++--------
 2 files changed, 18 insertions(+), 10 deletions(-)
arch/x86/include/asm/uaccess.h +0 -0
index faf3687f1035..b139c46ba122 100644

arch/x86/include/asm/uaccess.h static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un · line 93

Quoted 8 lines
arch/x86/include/asm/uaccess.h +3 -2
@@ -88,8 +88,11 @@ static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un
  * checks that the pointer is in the user space range - after calling
  * this function, memory access functions may still return -EFAULT.
  */
-#define access_ok(type, addr, size) \
-       likely(!__range_not_ok(addr, size, user_addr_max()))
+#define access_ok(type, addr, size)                                    \
+({                                                                     \
+       WARN_ON_ONCE(!in_task());                                       \

Should this be guarded by some debug option? This may hurt performance on production systems quite a bit.

Quoted 24 lines
include/linux/preempt.h +10 -3
index 75e4e30677f1..7eeceac52dea 100644
@@ -65,19 +65,24 @@
 
 /*
  * Are we doing bottom half or hardware interrupt processing?
- * Are we in a softirq context? Interrupt context?
- * in_softirq - Are we currently processing softirq or have bh disabled?
- * in_serving_softirq - Are we currently processing softirq?
+ *
+ * in_irq()       - We're in (hard) IRQ context
+ * in_softirq()   - We have BH disabled, or are processing softirqs
+ * in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled
+ * in_serving_softirq() - We're in softirq context
+ * in_nmi()       - We're in NMI context
+ * in_task()     - We're in task context
+ *
+ * Note: due to the BH disabled confusion: in_softirq(),in_interrupt() really
+ *       should not be used in new code.
  */
 #define in_irq()               (hardirq_count())
 #define in_softirq()           (softirq_count())

include/linux/preempt.h line 85

Quoted 10 lines
include/linux/preempt.h +3 -5
 #define in_interrupt()         (irq_count())
 #define in_serving_softirq()   (softirq_count() & SOFTIRQ_OFFSET)
-
-/*
- * Are we in NMI context?
- */
-#define in_nmi()       (preempt_count() & NMI_MASK)
+#define in_nmi()               (preempt_count() & NMI_MASK)
+#define in_task()              (!(preempt_count() & \
+                                  (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))

LGTM.

For what it's worth, I think ARM recently started saving the address limit and resetting it to USER_DS on NMI entry.

--Andy

author maintainer
#2 lore raw
Quoted 8 lines · Andy Lutomirski

arch/x86/include/asm/uaccess.h static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un · line 93

Quoted 3 lines · Peter Zijlstra
arch/x86/include/asm/uaccess.h +3 -0
+#define access_ok(type, addr, size)                                    \
+({                                                                     \
+       WARN_ON_ONCE(!in_task());                                       \

Should this be guarded by some debug option? This may hurt performance on production systems quite a bit.

I suspected something like that; any suitable CONFIG come to mind? I'm somewhat reluctant to create yet another one for this.

CONFIG_DEBUG_VM seems somehow inappropriate.

Quoted 2 lines

For what it's worth, I think ARM recently started saving the address limit and resetting it to USER_DS on NMI entry.

Up to them of course, but doing less on interrupt entry/exit seems better.

#3 lore raw
Quoted 2 lines · Peter Zijlstra

CONFIG_DEBUG_VM seems somehow inappropriate.

The usual might_fault() logic? That uses

    defined(CONFIG_PROVE_LOCKING) || defined(CONFIG_DEBUG_ATOMIC_SLEEP)

(and "might_sleep()" uses just CONFIG_DEBUG_ATOMIC_SLEEP, maybe that's fine).

Linus

author maintainer
#4 lore raw
Quoted 10 lines · Linus Torvalds
Quoted 2 lines · Peter Zijlstra

CONFIG_DEBUG_VM seems somehow inappropriate.

The usual might_fault() logic? That uses

    defined(CONFIG_PROVE_LOCKING) || defined(CONFIG_DEBUG_ATOMIC_SLEEP)

(and "might_sleep()" uses just CONFIG_DEBUG_ATOMIC_SLEEP, maybe that's fine).

Fair enough; something like so then?

---
Subject: x86: Verify access_ok() context
From: Peter Zijlstra <[email protected]>
Date: Tue, 22 Nov 2016 10:57:15 +0100

I recently encountered wreckage because access_ok() was used where it should not be, add an explicit WARN when access_ok() is used wrongly.

Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 arch/x86/include/asm/uaccess.h |   13 +++++++++++--
 include/linux/preempt.h        |   21 +++++++++++++--------
 2 files changed, 24 insertions(+), 10 deletions(-)
arch/x86/include/asm/uaccess.h +11 -2
@@ -68,6 +68,12 @@ static inline bool __chk_range_not_ok(un
 	__chk_range_not_ok((unsigned long __force)(addr), size, limit); \
 })
 
+#ifdef CONFIG_DEBUG_ATOMIC_SLEEP
+#define ACCESS_OK_WARN()	WARN_ON_ONCE(!in_task())
+#else
+#define ACCESS_OK_WARN()
+#endif
+
 /**
  * access_ok: - Checks if a user space pointer is valid
  * @type: Type of access: %VERIFY_READ or %VERIFY_WRITE.  Note that
@@ -88,8 +94,11 @@ static inline bool __chk_range_not_ok(un
  * checks that the pointer is in the user space range - after calling
  * this function, memory access functions may still return -EFAULT.
  */
-#define access_ok(type, addr, size) \
-	likely(!__range_not_ok(addr, size, user_addr_max()))
+#define access_ok(type, addr, size)					\
+({									\
+	ACCESS_OK_WARN();						\
+	likely(!__range_not_ok(addr, size, user_addr_max()));		\
+})
 
 /*
  * These are the main single-value transfer routines.  They automatically
include/linux/preempt.h +13 -8
@@ -65,19 +65,24 @@
 
 /*
  * Are we doing bottom half or hardware interrupt processing?
- * Are we in a softirq context? Interrupt context?
- * in_softirq - Are we currently processing softirq or have bh disabled?
- * in_serving_softirq - Are we currently processing softirq?
+ *
+ * in_irq()       - We're in (hard) IRQ context
+ * in_softirq()   - We have BH disabled, or are processing softirqs
+ * in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled
+ * in_serving_softirq() - We're in softirq context
+ * in_nmi()       - We're in NMI context
+ * in_task()	  - We're in task context
+ *
+ * Note: due to the BH disabled confusion: in_softirq(),in_interrupt() really
+ *       should not be used in new code.
  */
 #define in_irq()		(hardirq_count())
 #define in_softirq()		(softirq_count())
 #define in_interrupt()		(irq_count())
 #define in_serving_softirq()	(softirq_count() & SOFTIRQ_OFFSET)
-
-/*
- * Are we in NMI context?
- */
-#define in_nmi()	(preempt_count() & NMI_MASK)
+#define in_nmi()		(preempt_count() & NMI_MASK)
+#define in_task()		(!(preempt_count() & \
+				   (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))
 
 /*
  * The preempt_count offset after preempt_disable();
#5 lore raw

If you call access_ok() with page faulting disabled, you'll still see this new warning. If you put that new access_ok() call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.

How about modifying it like this:

---
From: David Smith <[email protected]>
Date: Mon, 16 Jan 2017 14:07:31 -0600
Subject: [PATCH] Relax x86 new access_ok() warning a bit.

---
 arch/x86/include/asm/uaccess.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/uaccess.h +1 -1
index ea148313570f..0cbd3cca5e7b 100644
@@ -69,7 +69,7 @@ static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un
 })
 
 #ifdef CONFIG_DEBUG_ATOMIC_SLEEP
-# define WARN_ON_IN_IRQ()	WARN_ON_ONCE(!in_task())
+# define WARN_ON_IN_IRQ()	WARN_ON_ONCE(!in_task() && !pagefault_disabled())
 #else
 # define WARN_ON_IN_IRQ()
 #endif
Signature · 7 lines
-- 
David Smith
[email protected]
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)
#6 lore raw
Quoted 2 lines · David Smith

If you call access_ok() with page faulting disabled, you'll still see this new warning.

And how so? It's just checking for task context. page fault disable/enable has absolutely nothing to do with that.

Quoted 3 lines

If you put that new access_ok() call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.

Can you please elaborate where this access_ok() is placed in the module code?

Thanks,

tglx

#7 lore raw
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · David Smith

If you call access_ok() with page faulting disabled, you'll still see this new warning.

And how so? It's just checking for task context. page fault disable/enable has absolutely nothing to do with that.

True, task context and page fault disable/enable have nothing to do with each other. However, the access_ok() comment states:

* Context: User context only. This function may sleep if pagefaults are
* enabled.

That seems to indicate that the function won't sleep if pagefaults are disabled, and thus there is no need for a CONFIG_DEBUG_ATOMIC_SLEEP warning if pagefaults are disabled.

Quoted 6 lines
Quoted 3 lines

If you put that new access_ok() call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.

Can you please elaborate where this access_ok() is placed in the module code?

It doesn't really matter where you place the access_ok() call in the module code. If you call access_ok() in a module, then that module has its own WARN_ON_ONCE() static variable. If access_ok() was a function exported from the kernel, then there would be only one copy of the WARN_ON_ONCE() static variable.

Signature · 6 lines
David Smith
[email protected]
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)
#8 lore raw
Quoted 9 lines · David Smith
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · David Smith

If you call access_ok() with page faulting disabled, you'll still see this new warning.

And how so? It's just checking for task context. page fault disable/enable has absolutely nothing to do with that.

Quoted 6 lines

True, task context and page fault disable/enable have nothing to do with each other. However, the access_ok() comment states:

* Context: User context only. This function may sleep if pagefaults are
* enabled.

That seems to indicate that the function won't sleep if pagefaults are disabled, and thus there is no need for a CONFIG_DEBUG_ATOMIC_SLEEP warning if pagefaults are disabled.

ISTM even with pagefault_disable() in play, using access_ok() from, say, interrupt context is dangerous unless you've first checked that you're in a task. But I guess that in_task() would still return false, e.g. in perf.

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

If you put that new access_ok() call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.

Can you please elaborate where this access_ok() is placed in the module code?

It doesn't really matter where you place the access_ok() call in the module code. If you call access_ok() in a module, then that module has its own WARN_ON_ONCE() static variable. If access_ok() was a function exported from the kernel, then there would be only one copy of the WARN_ON_ONCE() static variable.

That doesn't seem like such a big deal to me.

#9 lore raw
Quoted 3 lines · Andy Lutomirski
Quoted 2 lines · David Smith
Quoted 1 line · Thomas Gleixner

On Mon, 16 Jan 2017, David Smith wrote:

... stuff deleted ...

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

If you put that new access_ok() call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.

Can you please elaborate where this access_ok() is placed in the module code?

Quoted 7 lines
Quoted 5 lines

It doesn't really matter where you place the access_ok() call in the module code. If you call access_ok() in a module, then that module has its own WARN_ON_ONCE() static variable. If access_ok() was a function exported from the kernel, then there would be only one copy of the WARN_ON_ONCE() static variable.

That doesn't seem like such a big deal to me.

To be clear here, I'm not suggesting we replace the access_ok() macro with an exported kernel function. I'm just stating the fact that if you have several modules that call the access_ok() macro (or one module that gets loaded/unloaded/reloaded multiple times), each one can produce the new warning. I'm not seeing a "flood" of these new warnings, but a steady enough stream of them to be annoying.

Signature · 6 lines
David Smith
[email protected]
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)
author maintainer
#10 lore raw
In reply to Andy Lutomirski
Quoted 4 lines · Andy Lutomirski

ISTM even with pagefault_disable() in play, using access_ok() from, say, interrupt context is dangerous unless you've first checked that you're in a task. But I guess that in_task() would still return false, e.g. in perf.

The test was created exactly because perf was using access_ok() _wrongly_. See commit: ae31fe51a3cc ("perf/x86: Restore TASK_SIZE check on frame pointer").

#11 lore raw
Quoted 9 lines · Peter Zijlstra
Quoted 4 lines · Andy Lutomirski

ISTM even with pagefault_disable() in play, using access_ok() from, say, interrupt context is dangerous unless you've first checked that you're in a task. But I guess that in_task() would still return false, e.g. in perf.

The test was created exactly because perf was using access_ok() _wrongly_. See commit: ae31fe51a3cc ("perf/x86: Restore TASK_SIZE check on frame pointer").

If you validate a user space address against current outside the task context, then what guarantees that this user space address belongs to current? Nothing!

Sure, there are interrupts like breakpoints, etc. where we exactly know that the address which we are looking at belongs to current, because the code accesses soemthing which belongs exactly to that breakpoint. And in these cases we need a check which is designed specifically for that case.

Thanks,

tglx

#12 lore raw
In reply to David Smith
Quoted 10 lines · David Smith
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines

If you put that new access_ok() call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.

Can you please elaborate where this access_ok() is placed in the module code?

It doesn't really matter where you place the access_ok() call in the module code.

It does matter very much, because the fact that the warning triggers tells me that it's placed in code which is NOT executed in task context.

Quoted 4 lines

If you call access_ok() in a module, then that module has its own WARN_ON_ONCE() static variable. If access_ok() was a function exported from the kernel, then there would be only one copy of the WARN_ON_ONCE() static variable.

Not a big deal. If access_ok() is called from the wrong context in that module then this should be fixed and not the warning supressed.

We are not papering over problems.

Thanks,

tglx

#13 lore raw

Hi, Thomas -

Quoted 5 lines · Thomas Gleixner

[...]
It does matter very much, because the fact that the warning triggers tells me that it's placed in code which is NOT executed in task context. [...]
We are not papering over problems.

Understood. We were interpreting the comments around access_ok to mean that the underlying hazard condition was different (stricter) than in_task(). If the warning could be made to match that hazard condition more precisely, then safe but non-in_task() callers can use access_ok() without the warning.

- FChE

#14 lore raw
Quoted 9 lines · Frank Ch. Eigler

Hi, Thomas -

Quoted 5 lines · Thomas Gleixner

[...]
It does matter very much, because the fact that the warning triggers tells me that it's placed in code which is NOT executed in task context. [...]
We are not papering over problems.

Quoted 5 lines

Understood. We were interpreting the comments around access_ok to mean that the underlying hazard condition was different (stricter) than in_task(). If the warning could be made to match that hazard condition more precisely, then safe but non-in_task() callers can use access_ok() without the warning.

Well, if you are not in thread context then the check is pointless:

	__range_not_ok(addr, size, user_addr_max())

and:

#define user_addr_max() (current->thread.addr_limit.seg)

So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?

I assume this is about systemtap modules. Can you please explain what you are trying to achieve? I guess you know that you actually access current, but then we need a seperate special function and not relaxing of the checks.

Thanks,

tglx

#15 lore raw

Hi, Thomas -

Quoted 8 lines

Well, if you are not in thread context then the check is pointless:

	__range_not_ok(addr, size, user_addr_max())

and:
#define user_addr_max() (current->thread.addr_limit.seg)

So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?

We're probably in task context in that there is a valid current(), but running with preemption and/or interrupts and/or pagefaults disabled at that point, so in_task() objects. Think of it like from a kprobes handler callback, except maybe more temporary preemption blocking.

Quoted 4 lines

I assume this is about systemtap modules. Can you please explain what you are trying to achieve? I guess you know that you actually access current, but then we need a seperate special function and not relaxing of the checks.

This part is used in a part of the runtime that is a userspace analogue of probe_kernel_address(), where we're given a potential userspace address. We would like to quickly test whether it's even plausible as a userspace address, before doing a (pagefault-disabled) trial fetch/store to it.

- FChE

author maintainer
#16 lore raw
Quoted 7 lines · Frank Ch. Eigler

Hi, Thomas -

Quoted 5 lines

Well, if you are not in thread context then the check is pointless:

	__range_not_ok(addr, size, user_addr_max())

and:
#define user_addr_max() (current->thread.addr_limit.seg)

Quoted 8 lines
Quoted 3 lines

So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?

We're probably in task context in that there is a valid current(), but running with preemption and/or interrupts and/or pagefaults disabled at that point, so in_task() objects. Think of it like from a kprobes handler callback, except maybe more temporary preemption blocking.

#define in_task() (!(preempt_count() & \
(NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))

So it doesn't care about preempt_disable(), and it doesn't care about local_irq_disable(), it also doesn't care about local_bh_disable().

What it does care about are nmi_enter(), irq_enter() and __do_softirq().

#17 lore raw
In reply to Frank Ch. Eigler

Frank.

Quoted 10 lines · Frank Ch. Eigler
Quoted 8 lines

Well, if you are not in thread context then the check is pointless:

	__range_not_ok(addr, size, user_addr_max())

and:
#define user_addr_max() (current->thread.addr_limit.seg)

So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?

We're probably in task context in that there is a valid current(), but

current is always accessible no matter in which context you are - task, softirq, hardirq, nmi ...

Quoted 2 lines

running with preemption and/or interrupts and/or pagefaults disabled at that point, so in_task() objects.

As Peter explained, neither preempt disable nor interrupt disable not pagefault disabled have any influence on in_task(). It merily checks the
context: !in_softirq() && !in_hardirq() && !in_nmi().

So that warning happens definitely not from task context.

Care to share the code?

Thanks,

tglx