x86: Verify access_ok() context
Peter Zijlstra
maintainerRFC PATCH 1/1 x86: Verify access_ok() context
I recently encountered wreckage because access_ok() was used where it should not be, add an explicit WARN when access_ok() is used wrongly.
--- arch/x86/include/asm/uaccess.h | 7 +++++-- include/linux/preempt.h | 21 +++++++++++++-------- 2 files changed, 18 insertions(+), 10 deletions(-)
arch/x86/include/asm/uaccess.h +5 -2
@@ -88,8 +88,11 @@ static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un* checks that the pointer is in the user space range - after calling * this function, memory access functions may still return -EFAULT. */ -#define access_ok (type, addr, size) \ - likely(!__range_not_ok(addr, size, user_addr_max())) +#define access_ok (type, addr, size) \ +({ \ + WARN_ON_ONCE(!in_task()); \ + likely(!__range_not_ok(addr, size, user_addr_max())); \ +})/* * These are the main single-value transfer routines. They automatically
include/linux/preempt.h +13 -8
@@ -65,19 +65,24 @@/* * Are we doing bottom half or hardware interrupt processing? -* Are we in a softirq context? Interrupt context? -* in_softirq - Are we currently processing softirq or have bh disabled? -* in_serving_softirq - Are we currently processing softirq? +* +* in_irq() - We're in (hard) IRQ context +* in_softirq() - We have BH disabled, or are processing softirqs +* in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled +* in_serving_softirq() - We're in softirq context +* in_nmi() - We're in NMI context +* in_task() - We're in task context +* +* Note: due to the BH disabled confusion: in_softirq(),in_interrupt() really +* should not be used in new code. */ #define in_irq () (hardirq_count())#define in_softirq () (softirq_count())#define in_interrupt () (irq_count())#define in_serving_softirq () (softirq_count() & SOFTIRQ_OFFSET) - -/* -* Are we in NMI context? -*/ -#define in_nmi () (preempt_count() & NMI_MASK) +#define in_nmi () (preempt_count() & NMI_MASK) +#define in_task () (!(preempt_count() & \ + (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))/* * The preempt_count offset after preempt_disable();
Quoted 15 lines · Peter Zijlstra
I recently encountered wreckage because
access_ok()was used where it should not be, add an explicit WARN whenaccess_ok()is used wrongly.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- arch/x86/include/asm/uaccess.h | 7 +++++-- include/linux/preempt.h | 21 +++++++++++++-------- 2 files changed, 18 insertions(+), 10 deletions(-)arch/x86/include/asm/uaccess.h +0
-0
arch/x86/include/asm/uaccess.h static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un · line 93
Quoted 8 lines
arch/x86/include/asm/uaccess.h +3
-2@@ -88,8 +88,11 @@ static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un* checks that the pointer is in the user space range - after calling * this function, memory access functions may still return -EFAULT. */ -#define access_ok (type, addr, size) \ - likely(!__range_not_ok(addr, size, user_addr_max())) +#define access_ok (type, addr, size) \ +({ \ + WARN_ON_ONCE(!in_task()); \
Should this be guarded by some debug option? This may hurt performance on production systems quite a bit.
Quoted 24 lines
include/linux/preempt.h +10
-3@@ -65,19 +65,24 @@/* * Are we doing bottom half or hardware interrupt processing? -* Are we in a softirq context? Interrupt context? -* in_softirq - Are we currently processing softirq or have bh disabled? -* in_serving_softirq - Are we currently processing softirq? +* +* in_irq() - We're in (hard) IRQ context +* in_softirq() - We have BH disabled, or are processing softirqs +* in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled +* in_serving_softirq() - We're in softirq context +* in_nmi() - We're in NMI context +* in_task() - We're in task context +* +* Note: due to the BH disabled confusion: in_softirq(),in_interrupt() really +* should not be used in new code. */ #define in_irq () (hardirq_count())#define in_softirq () (softirq_count())
include/linux/preempt.h line 85
Quoted 10 lines
include/linux/preempt.h +3
-5#define in_interrupt () (irq_count())#define in_serving_softirq () (softirq_count() & SOFTIRQ_OFFSET) - -/* -* Are we in NMI context? -*/ -#define in_nmi () (preempt_count() & NMI_MASK) +#define in_nmi () (preempt_count() & NMI_MASK) +#define in_task () (!(preempt_count() & \ + (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))
LGTM.
For what it's worth, I think ARM recently started saving the address limit and resetting it to USER_DS on NMI entry.
--Andy
Peter Zijlstra
maintainerQuoted 8 lines · Andy Lutomirski
arch/x86/include/asm/uaccess.h static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un · line 93
Quoted 3 lines · Peter Zijlstra
arch/x86/include/asm/uaccess.h +3
-0+#define access_ok (type, addr, size) \ +({ \ + WARN_ON_ONCE(!in_task()); \Should this be guarded by some debug option? This may hurt performance on production systems quite a bit.
I suspected something like that; any suitable CONFIG come to mind? I'm somewhat reluctant to create yet another one for this.
CONFIG_DEBUG_VM seems somehow inappropriate.
Quoted 2 lines
For what it's worth, I think ARM recently started saving the address limit and resetting it to
USER_DSon NMI entry.
Up to them of course, but doing less on interrupt entry/exit seems better.
Quoted 2 lines · Peter Zijlstra
CONFIG_DEBUG_VMseems somehow inappropriate.
The usual might_fault() logic? That uses
defined(CONFIG_PROVE_LOCKING) || defined(CONFIG_DEBUG_ATOMIC_SLEEP)
(and "might_sleep()" uses just CONFIG_DEBUG_ATOMIC_SLEEP, maybe that's fine).
Linus
Peter Zijlstra
maintainerQuoted 10 lines · Linus Torvalds
Quoted 2 lines · Peter Zijlstra
CONFIG_DEBUG_VMseems somehow inappropriate.The usual
might_fault()logic? That usesdefined(CONFIG_PROVE_LOCKING) || defined(CONFIG_DEBUG_ATOMIC_SLEEP)(and "
might_sleep()" uses justCONFIG_DEBUG_ATOMIC_SLEEP, maybe that's fine).
Fair enough; something like so then?
---
Subject: x86: Verify access_ok() context
From: Peter Zijlstra <[email protected]>
Date: Tue, 22 Nov 2016 10:57:15 +0100
I recently encountered wreckage because access_ok() was used where it should not be, add an explicit WARN when access_ok() is used wrongly.
--- arch/x86/include/asm/uaccess.h | 13 +++++++++++-- include/linux/preempt.h | 21 +++++++++++++-------- 2 files changed, 24 insertions(+), 10 deletions(-)
arch/x86/include/asm/uaccess.h +11 -2
@@ -68,6 +68,12 @@ static inline bool __chk_range_not_ok(un__chk_range_not_ok ((unsigned long __force )(addr), size, limit); \ }) +#ifdef CONFIG_DEBUG_ATOMIC_SLEEP +#define ACCESS_OK_WARN () WARN_ON_ONCE(!in_task()) +#else +#define ACCESS_OK_WARN () +#endif +/** * access_ok: - Checks if a user space pointer is valid * @type: Type of access: %VERIFY_READ or %VERIFY_WRITE. Note that
@@ -88,8 +94,11 @@ static inline bool __chk_range_not_ok(un* checks that the pointer is in the user space range - after calling * this function, memory access functions may still return -EFAULT. */ -#define access_ok (type, addr, size) \ - likely(!__range_not_ok(addr, size, user_addr_max())) +#define access_ok (type, addr, size) \ +({ \ + ACCESS_OK_WARN(); \ + likely(!__range_not_ok(addr, size, user_addr_max())); \ +})/* * These are the main single-value transfer routines. They automatically
include/linux/preempt.h +13 -8
@@ -65,19 +65,24 @@/* * Are we doing bottom half or hardware interrupt processing? -* Are we in a softirq context? Interrupt context? -* in_softirq - Are we currently processing softirq or have bh disabled? -* in_serving_softirq - Are we currently processing softirq? +* +* in_irq() - We're in (hard) IRQ context +* in_softirq() - We have BH disabled, or are processing softirqs +* in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled +* in_serving_softirq() - We're in softirq context +* in_nmi() - We're in NMI context +* in_task() - We're in task context +* +* Note: due to the BH disabled confusion: in_softirq(),in_interrupt() really +* should not be used in new code. */ #define in_irq () (hardirq_count())#define in_softirq () (softirq_count())#define in_interrupt () (irq_count())#define in_serving_softirq () (softirq_count() & SOFTIRQ_OFFSET) - -/* -* Are we in NMI context? -*/ -#define in_nmi () (preempt_count() & NMI_MASK) +#define in_nmi () (preempt_count() & NMI_MASK) +#define in_task () (!(preempt_count() & \ + (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))/* * The preempt_count offset after preempt_disable();
If you call access_ok() with page faulting disabled, you'll still see this new warning. If you put that new access_ok() call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.
How about modifying it like this:
---
From: David Smith <[email protected]>
Date: Mon, 16 Jan 2017 14:07:31 -0600
Subject: [PATCH] Relax x86 new access_ok() warning a bit.
--- arch/x86/include/asm/uaccess.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/uaccess.h +1 -1
@@ -69,7 +69,7 @@ static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un })#ifdef CONFIG_DEBUG_ATOMIC_SLEEP -# define WARN_ON_IN_IRQ () WARN_ON_ONCE(!in_task()) +# define WARN_ON_IN_IRQ () WARN_ON_ONCE(!in_task() && !pagefault_disabled())#else # define WARN_ON_IN_IRQ ()#endif
Signature · 7 lines
-- David Smith [email protected] Red Hat http://www.redhat.com 256.217.0141 (direct) 256.837.0057 (fax)
Quoted 2 lines · David Smith
If you call
access_ok()with page faulting disabled, you'll still see this new warning.
And how so? It's just checking for task context. page fault disable/enable has absolutely nothing to do with that.
Quoted 3 lines
If you put that new
access_ok()call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.
Can you please elaborate where this access_ok() is placed in the module code?
Thanks,
tglx
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · David Smith
If you call
access_ok()with page faulting disabled, you'll still see this new warning.And how so? It's just checking for task context. page fault disable/enable has absolutely nothing to do with that.
True, task context and page fault disable/enable have nothing to do with each other. However, the access_ok() comment states:
* Context: User context only. This function may sleep if pagefaults are
* enabled.
That seems to indicate that the function won't sleep if pagefaults are disabled, and thus there is no need for a CONFIG_DEBUG_ATOMIC_SLEEP warning if pagefaults are disabled.
Quoted 6 lines
Quoted 3 lines
If you put that new
access_ok()call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.Can you please elaborate where this
access_ok()is placed in the module code?
It doesn't really matter where you place the access_ok() call in the module code. If you call access_ok() in a module, then that module has its own WARN_ON_ONCE() static variable. If access_ok() was a function exported from the kernel, then there would be only one copy of the WARN_ON_ONCE() static variable.
Signature · 6 lines
David Smith [email protected] Red Hat http://www.redhat.com 256.217.0141 (direct) 256.837.0057 (fax)
Quoted 9 lines · David Smith
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · David Smith
If you call
access_ok()with page faulting disabled, you'll still see this new warning.And how so? It's just checking for task context. page fault disable/enable has absolutely nothing to do with that.
Quoted 6 lines
True, task context and page fault disable/enable have nothing to do with each other. However, the
access_ok()comment states:* Context: User context only. This function may sleep if pagefaults are
* enabled.That seems to indicate that the function won't sleep if pagefaults are disabled, and thus there is no need for a
CONFIG_DEBUG_ATOMIC_SLEEPwarning if pagefaults are disabled.
ISTM even with pagefault_disable() in play, using access_ok() from, say, interrupt context is dangerous unless you've first checked that you're in a task. But I guess that in_task() would still return false, e.g. in perf.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
If you put that new
access_ok()call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.Can you please elaborate where this
access_ok()is placed in the module code?It doesn't really matter where you place the
access_ok()call in the module code. If you callaccess_ok()in a module, then that module has its ownWARN_ON_ONCE()static variable. Ifaccess_ok()was a function exported from the kernel, then there would be only one copy of theWARN_ON_ONCE()static variable.
That doesn't seem like such a big deal to me.
Quoted 3 lines · Andy Lutomirski
Quoted 2 lines · David Smith
Quoted 1 line · Thomas Gleixner
On Mon, 16 Jan 2017, David Smith wrote:
... stuff deleted ...
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
If you put that new
access_ok()call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.Can you please elaborate where this
access_ok()is placed in the module code?
Quoted 7 lines
Quoted 5 lines
It doesn't really matter where you place the
access_ok()call in the module code. If you callaccess_ok()in a module, then that module has its ownWARN_ON_ONCE()static variable. Ifaccess_ok()was a function exported from the kernel, then there would be only one copy of theWARN_ON_ONCE()static variable.That doesn't seem like such a big deal to me.
To be clear here, I'm not suggesting we replace the access_ok() macro with an exported kernel function. I'm just stating the fact that if you have several modules that call the access_ok() macro (or one module that gets loaded/unloaded/reloaded multiple times), each one can produce the new warning. I'm not seeing a "flood" of these new warnings, but a steady enough stream of them to be annoying.
Signature · 6 lines
David Smith [email protected] Red Hat http://www.redhat.com 256.217.0141 (direct) 256.837.0057 (fax)
Peter Zijlstra
maintainerQuoted 4 lines · Andy Lutomirski
ISTM even with
pagefault_disable()in play, usingaccess_ok()from, say, interrupt context is dangerous unless you've first checked that you're in a task. But I guess thatin_task()would still return false, e.g. in perf.
The test was created exactly because perf was using access_ok() _wrongly_. See commit: ae31fe51a3cc ("perf/x86: Restore TASK_SIZE check on frame pointer").
Quoted 9 lines · Peter Zijlstra
Quoted 4 lines · Andy Lutomirski
ISTM even with
pagefault_disable()in play, usingaccess_ok()from, say, interrupt context is dangerous unless you've first checked that you're in a task. But I guess thatin_task()would still return false, e.g. in perf.The test was created exactly because perf was using
access_ok()_wrongly_. See commit:ae31fe51a3cc("perf/x86: RestoreTASK_SIZEcheck on frame pointer").
If you validate a user space address against current outside the task context, then what guarantees that this user space address belongs to current? Nothing!
Sure, there are interrupts like breakpoints, etc. where we exactly know that the address which we are looking at belongs to current, because the code accesses soemthing which belongs exactly to that breakpoint. And in these cases we need a check which is designed specifically for that case.
Thanks,
tglx
Quoted 10 lines · David Smith
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines
If you put that new
access_ok()call in a module that gets loaded/unloaded, you see one warning for every module load, which gets a bit annoying.Can you please elaborate where this
access_ok()is placed in the module code?It doesn't really matter where you place the
access_ok()call in the module code.
It does matter very much, because the fact that the warning triggers tells me that it's placed in code which is NOT executed in task context.
Quoted 4 lines
If you call
access_ok()in a module, then that module has its ownWARN_ON_ONCE()static variable. Ifaccess_ok()was a function exported from the kernel, then there would be only one copy of theWARN_ON_ONCE()static variable.
Not a big deal. If access_ok() is called from the wrong context in that module then this should be fixed and not the warning supressed.
We are not papering over problems.
Thanks,
tglx
Hi, Thomas -
Quoted 5 lines · Thomas Gleixner
[...]
It does matter very much, because the fact that the warning triggers tells me that it's placed in code which is NOT executed in task context. [...]
We are not papering over problems.
Understood. We were interpreting the comments around access_ok to mean that the underlying hazard condition was different (stricter) than in_task(). If the warning could be made to match that hazard condition more precisely, then safe but non-in_task() callers can use access_ok() without the warning.
- FChE
Quoted 9 lines · Frank Ch. Eigler
Hi, Thomas -
Quoted 5 lines · Thomas Gleixner
[...]
It does matter very much, because the fact that the warning triggers tells me that it's placed in code which is NOT executed in task context. [...]
We are not papering over problems.
Quoted 5 lines
Understood. We were interpreting the comments around
access_okto mean that the underlying hazard condition was different (stricter) thanin_task(). If the warning could be made to match that hazard condition more precisely, then safe but non-in_task() callers can useaccess_ok()without the warning.
Well, if you are not in thread context then the check is pointless:
__range_not_ok(addr, size, user_addr_max())
and:
#define user_addr_max() (current->thread.addr_limit.seg)
So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?
I assume this is about systemtap modules. Can you please explain what you are trying to achieve? I guess you know that you actually access current, but then we need a seperate special function and not relaxing of the checks.
Thanks,
tglx
Hi, Thomas -
Quoted 8 lines
Well, if you are not in thread context then the check is pointless:
__range_not_ok(addr, size, user_addr_max())and:
#defineuser_addr_max()(current->thread.addr_limit.seg)So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?
We're probably in task context in that there is a valid current(), but running with preemption and/or interrupts and/or pagefaults disabled at that point, so in_task() objects. Think of it like from a kprobes handler callback, except maybe more temporary preemption blocking.
Quoted 4 lines
I assume this is about systemtap modules. Can you please explain what you are trying to achieve? I guess you know that you actually access current, but then we need a seperate special function and not relaxing of the checks.
This part is used in a part of the runtime that is a userspace analogue of probe_kernel_address(), where we're given a potential userspace address. We would like to quickly test whether it's even plausible as a userspace address, before doing a (pagefault-disabled) trial fetch/store to it.
- FChE
Peter Zijlstra
maintainerQuoted 7 lines · Frank Ch. Eigler
Hi, Thomas -
Quoted 5 lines
Well, if you are not in thread context then the check is pointless:
__range_not_ok(addr, size, user_addr_max())and:
#defineuser_addr_max()(current->thread.addr_limit.seg)
Quoted 8 lines
Quoted 3 lines
So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?
We're probably in task context in that there is a valid
current(), but running with preemption and/or interrupts and/or pagefaults disabled at that point, soin_task()objects. Think of it like from a kprobes handler callback, except maybe more temporary preemption blocking.
#define in_task() (!(preempt_count() & \
(NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))
So it doesn't care about preempt_disable(), and it doesn't care about local_irq_disable(), it also doesn't care about local_bh_disable().
What it does care about are nmi_enter(), irq_enter() and __do_softirq().
Frank.
Quoted 10 lines · Frank Ch. Eigler
Quoted 8 lines
Well, if you are not in thread context then the check is pointless:
__range_not_ok(addr, size, user_addr_max())and:
#defineuser_addr_max()(current->thread.addr_limit.seg)So what guarantees when you are not in context of current, i.e. in thread context, that the addr/size which is checked against the limits of current actually belongs to current?
We're probably in task context in that there is a valid
current(), but
current is always accessible no matter in which context you are - task, softirq, hardirq, nmi ...
Quoted 2 lines
running with preemption and/or interrupts and/or pagefaults disabled at that point, so
in_task()objects.
As Peter explained, neither preempt disable nor interrupt disable not pagefault disabled have any influence on in_task(). It merily checks the
context: !in_softirq() && !in_hardirq() && !in_nmi().
So that warning happens definitely not from task context.
Care to share the code?
Thanks,
tglx