handle worst-case heap randomization in mmap_base
Subsystems (5)
cover handle worst-case heap randomization in mmap_base
Increase mmap_base by the worst-case brk randomization so that the stack and heap remain apart.
In Linux 4.13 a change was committed that special cased the kernel ELF loader when the loader is invoked directly (eab09532d400; binfmt_elf: use ELF_ET_DYN_BASE only for PIE). Generally, the loader isn’t invoked directly and this issue is limited to cases where it is, (e.g to set a non-inheritable LD_LIBRARY_PATH, testing new versions of the loader). In those rare cases, the loader doesn't take into account the amount of brk randomization that will be applied by arch_randomize_brk(). This can lead to the stack and heap being arbitrarily close to each other.
Ali Saidi (2):
arm64/mmap: handle worst-case heap randomization in mmap_base x86/mmap: handle worst-case heap randomization in mmap_base
arch/arm64/mm/mmap.c | 8 ++++++++ arch/x86/mm/mmap.c | 4 ++++ 2 files changed, 12 insertions(+)
Signature · 1 line
2.15.3.AMZN
PATCH 1/2 arm64/mmap: handle worst-case heap randomization in mmap_base
Increase mmap_base by the worst-case brk randomization so that the stack and heap remain apart.
In Linux 4.13 a change was committed that special cased the kernel ELF loader when the loader is invoked directly (eab09532d400; binfmt_elf: use ELF_ET_DYN_BASE only for PIE). Generally, the loader isn’t invoked directly and this issue is limited to cases where it is, (e.g to set a non-inheritable LD_LIBRARY_PATH, testing new versions of the loader). In those rare cases, the loader doesn't take into account the amount of brk randomization that will be applied by arch_randomize_brk(). This can lead to the stack and heap being arbitrarily close to each other.
--- arch/arm64/mm/mmap.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/arm64/mm/mmap.c +8 -0
@@ -67,6 +67,14 @@ static unsigned long mmap_base(unsigned long rnd, struct rlimit *rlim_stack)unsigned long gap = rlim_stack->rlim_cur;unsigned long pad = (STACK_RND_MASK <<PAGE_SHIFT ) + stack_guard_gap; +/* Provide space for randomization when randomize_va_space == 2 and +* ld-linux.so is called directly. Values from arch_randomize_brk() +*/ +if (test_thread_flag (TIF_32BIT )) + pad +=SZ_32M ; +else + pad +=SZ_1G ; +/* Values close to RLIM_INFINITY can overflow. */ if (gap + pad > gap) gap += pad;
Signature · 1 line
2.15.3.AMZN
PATCH 2/2 x86/mmap: handle worst-case heap randomization in mmap_base
In reply to Ali SaidiIncrease mmap_base by the worst-case brk randomization so that the stack and heap remain apart.
In Linux 4.13 a change was committed that special cased the kernel ELF loader when the loader is invoked directly (eab09532d400; binfmt_elf: use ELF_ET_DYN_BASE only for PIE). Generally, the loader isn’t invoked directly and this issue is limited to cases where it is, (e.g to set a non-inheritable LD_LIBRARY_PATH, testing new versions of the loader). In those rare cases, the loader doesn't take into account the amount of brk randomization that will be applied by arch_randomize_brk(). This can lead to the stack and heap being arbitrarily close to each other.
--- arch/x86/mm/mmap.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/mm/mmap.c +4 -0
@@ -31,6 +31,7 @@#include <linux/sched/signal.h> #include <linux/sched/mm.h> #include <linux/compat.h> +#include <linux/sizes.h> #include <asm/elf.h> #include "physaddr.h"
@@ -97,6 +98,9 @@ static unsigned long mmap_base(unsigned long rnd, unsigned long task_size,unsigned long pad =stack_maxrandom_size (task_size) + stack_guard_gap;unsigned long gap_min, gap_max; +/* Provide space for brk randomization */ + pad +=SZ_32M ; +/* Values close to RLIM_INFINITY can overflow. */ if (gap + pad > gap) gap += pad;
Signature · 1 line
2.15.3.AMZN
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
arch/x86/mm/mmap.c static unsigned long mmap_base(unsigned long rnd, unsigned long task_size, · line 102
Quoted 2 lines · Ali Saidi
arch/x86/mm/mmap.c +2
-0+/* Provide space for brk randomization */ + pad +=SZ_32M ;
Just curious: Why is the padding in your other patch conditional on the 32-bit vs. 64-bit apps, but here it's always 32M?
Also, did you hit this problem in practice somehow?
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
Quoted 10 lines · Dave Hansen
arch/x86/mm/mmap.c static unsigned long mmap_base(unsigned long rnd, unsigned long task_size, · line 102
Quoted 4 lines · Ali Saidi
arch/x86/mm/mmap.c +2
-0+/* Provide space for brk randomization */ + pad +=SZ_32M ;Just curious: Why is the padding in your other patch conditional on the 32-bit vs. 64-bit apps, but here it's always 32M?
Arm changes the amount of brk based on the process being 32 vs 64 bit. X86 doesn’t appear to do this.
Quoted 2 lines
Also, did you hit this problem in practice somehow?
Just debugging a crash when testing a version of a library I compiled.
Ali
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
In reply to Ali SaidiQuoted 4 lines · Ali Saidi
Increase
mmap_baseby the worst-case brk randomization so that the stack and heap remain apart.
Quoted 8 lines
In Linux 4.13 a change was committed that special cased the kernel ELF loader when the loader is invoked directly (
eab09532d400;binfmt_elf: useELF_ET_DYN_BASEonly for PIE). Generally, the loader isn’t invoked directly and this issue is limited to cases where it is, (e.g to set a non-inheritableLD_LIBRARY_PATH, testing new versions of the loader). In those rare cases, the loader doesn't take into account the amount of brk randomization that will be applied byarch_randomize_brk(). This can lead to the stack and heap being arbitrarily close to each other.
In the case of using the loader directly, brk (so helpfully identified as "[heap]") is allocated with the _loader_ not the binary. For example, with ASLR entirely disabled, you can see this more clearly:
$ /bin/cat /proc/self/maps 555555554000-55555555c000 r-xp 00000000 fd:02 34603015 /bin/cat 55555575b000-55555575c000 r--p 00007000 fd:02 34603015 /bin/cat 55555575c000-55555575d000 rw-p 00008000 fd:02 34603015 /bin/cat 55555575d000-55555577e000 rw-p 00000000 00:00 0 [heap] ... 7ffff7ff7000-7ffff7ffa000 r--p 00000000 00:00 0 [vvar] 7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0 [vdso] 7ffff7ffc000-7ffff7ffd000 r--p 00027000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffd000-7ffff7ffe000 rw-p 00028000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0 7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]
$ /lib/x86_64-linux-gnu/ld-2.27.so /bin/cat /proc/self/maps ... 7ffff7bcc000-7ffff7bd4000 r-xp 00000000 fd:02 34603015 /bin/cat 7ffff7bd4000-7ffff7dd3000 ---p 00008000 fd:02 34603015 /bin/cat 7ffff7dd3000-7ffff7dd4000 r--p 00007000 fd:02 34603015 /bin/cat 7ffff7dd4000-7ffff7dd5000 rw-p 00008000 fd:02 34603015 /bin/cat 7ffff7dd5000-7ffff7dfc000 r-xp 00000000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7fb2000-7ffff7fd6000 rw-p 00000000 00:00 0 7ffff7ff7000-7ffff7ffa000 r--p 00000000 00:00 0 [vvar] 7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0 [vdso] 7ffff7ffc000-7ffff7ffd000 r--p 00027000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffd000-7ffff7ffe000 rw-p 00028000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffe000-7ffff8020000 rw-p 00000000 00:00 0 [heap] 7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]
So I think changing this globally isn't the right solution (normally brk is between text and mmap). Adjusting the mmap base padding means we lose even more memory space. Perhaps it would be better if brk allocation would be placed before the mmap region (i.e. use ELF_ET_DYN_BASE). This seems to work for me:
fs/binfmt_elf.c +9 -0
@@ -1131,6 +1131,15 @@ static int load_elf_binary(struct linux_binprm *bprm) current->mm->end_data = end_data; current->mm->start_stack = bprm->p; +/* +* When executing a loader directly (ET_DYN without Interp), move +* the brk area out of the mmap region (since it grows up, and may +* collide early with the stack growing down), and into the unused +* ELF_ET_DYN_BASE region. +*/ +if (!elf_interpreter) + current->mm->brk = current->mm->start_brk =ELF_ET_DYN_BASE ; +if ((current->flags &PF_RANDOMIZE ) && (randomize_va_space >1 )) { current->mm->brk = current->mm->start_brk =arch_randomize_brk (current->mm);
$ /lib/x86_64-linux-gnu/ld-2.27.so /bin/cat /proc/self/maps 555556de3000-555556e04000 rw-p 00000000 00:00 0 [heap] 7f8467da9000-7f8467f90000 r-xp 00000000 fd:01 399295 /lib/x86_64-linux-gnu/libc-2.27.so ... 7f846819a000-7f84681a2000 r-xp 00000000 fd:01 263229 /bin/cat ... 7f84685cb000-7f84685cc000 rw-p 00028000 fd:01 399286 /lib/x86_64-linux-gnu/ld-2.27.so 7f84685cc000-7f84685cd000 rw-p 00000000 00:00 0 7ffce68f8000-7ffce6919000 rw-p 00000000 00:00 0 [stack] 7ffce69f0000-7ffce69f3000 r--p 00000000 00:00 0 [vvar] 7ffce69f3000-7ffce69f4000 r-xp 00000000 00:00 0 [vdso]
Does anyone see problems with this? (Note that ET_EXEC base is 0x400000, so no collision there...)
Signature · 1 line
Kees Cook
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
Quoted 101 lines · Kees Cook
Quoted 4 lines · Ali Saidi
Increase
mmap_baseby the worst-case brk randomization so that the stack and heap remain apart.Quoted 8 lines
In Linux 4.13 a change was committed that special cased the kernel ELF loader when the loader is invoked directly (
eab09532d400;binfmt_elf: useELF_ET_DYN_BASEonly for PIE). Generally, the loader isn’t invoked directly and this issue is limited to cases where it is, (e.g to set a non-inheritableLD_LIBRARY_PATH, testing new versions of the loader). In those rare cases, the loader doesn't take into account the amount of brk randomization that will be applied byarch_randomize_brk(). This can lead to the stack and heap being arbitrarily close to each other.In the case of using the loader directly, brk (so helpfully identified as "[heap]") is allocated with the _loader_ not the binary. For example, with ASLR entirely disabled, you can see this more clearly:
$ /bin/cat /proc/self/maps 555555554000-55555555c000 r-xp 00000000 fd:02 34603015 /bin/cat 55555575b000-55555575c000 r--p 00007000 fd:02 34603015 /bin/cat 55555575c000-55555575d000 rw-p 00008000 fd:02 34603015 /bin/cat 55555575d000-55555577e000 rw-p 00000000 00:00 0 [heap] ... 7ffff7ff7000-7ffff7ffa000 r--p 00000000 00:00 0 [vvar] 7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0 [vdso] 7ffff7ffc000-7ffff7ffd000 r--p 00027000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffd000-7ffff7ffe000 rw-p 00028000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0 7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]$ /lib/x86_64-linux-gnu/ld-2.27.so /bin/cat /proc/self/maps ... 7ffff7bcc000-7ffff7bd4000 r-xp 00000000 fd:02 34603015 /bin/cat 7ffff7bd4000-7ffff7dd3000 ---p 00008000 fd:02 34603015 /bin/cat 7ffff7dd3000-7ffff7dd4000 r--p 00007000 fd:02 34603015 /bin/cat 7ffff7dd4000-7ffff7dd5000 rw-p 00008000 fd:02 34603015 /bin/cat 7ffff7dd5000-7ffff7dfc000 r-xp 00000000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7fb2000-7ffff7fd6000 rw-p 00000000 00:00 0 7ffff7ff7000-7ffff7ffa000 r--p 00000000 00:00 0 [vvar] 7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0 [vdso] 7ffff7ffc000-7ffff7ffd000 r--p 00027000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffd000-7ffff7ffe000 rw-p 00028000 fd:02 49287483 /lib/x86_64-linux-gnu/ld-2.27.so 7ffff7ffe000-7ffff8020000 rw-p 00000000 00:00 0 [heap] 7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]So I think changing this globally isn't the right solution (normally brk is between text and mmap). Adjusting the mmap base padding means we lose even more memory space. Perhaps it would be better if brk allocation would be placed before the mmap region (i.e. use
ELF_ET_DYN_BASE). This seems to work for me:fs/binfmt_elf.c +9
-0@@ -1131,6 +1131,15 @@ static int load_elf_binary(struct linux_binprm *bprm) current->mm->end_data = end_data; current->mm->start_stack = bprm->p; +/* +* When executing a loader directly (ET_DYN without Interp), move +* the brk area out of the mmap region (since it grows up, and may +* collide early with the stack growing down), and into the unused +* ELF_ET_DYN_BASE region. +*/ +if (!elf_interpreter) + current->mm->brk = current->mm->start_brk =ELF_ET_DYN_BASE ; +if ((current->flags &PF_RANDOMIZE ) && (randomize_va_space >1 )) { current->mm->brk = current->mm->start_brk =arch_randomize_brk (current->mm);$ /lib/x86_64-linux-gnu/ld-2.27.so /bin/cat /proc/self/maps 555556de3000-555556e04000 rw-p 00000000 00:00 0 [heap] 7f8467da9000-7f8467f90000 r-xp 00000000 fd:01 399295 /lib/x86_64-linux-gnu/libc-2.27.so ... 7f846819a000-7f84681a2000 r-xp 00000000 fd:01 263229 /bin/cat ... 7f84685cb000-7f84685cc000 rw-p 00028000 fd:01 399286 /lib/x86_64-linux-gnu/ld-2.27.so 7f84685cc000-7f84685cd000 rw-p 00000000 00:00 0 7ffce68f8000-7ffce6919000 rw-p 00000000 00:00 0 [stack] 7ffce69f0000-7ffce69f3000 r--p 00000000 00:00 0 [vvar] 7ffce69f3000-7ffce69f4000 r-xp 00000000 00:00 0 [vdso]
Quoted 2 lines
Does anyone see problems with this? (Note that
ET_EXECbase is0x400000, so no collision there...)
Adding some more people to CC... what do people think about this moving of the brk to ELF_ET_DYN_BASE in this corner-case? Anything that worked before should still work (i.e. the ultimately-launched binary already had the brk very far from its text, so this should be no different from a COMPAT_BRK standpoint). The only risk I see here is that if someone started to suddenly depend on the entire memory space above the mmap region being available when launching binaries via a direct loader execs... which seems highly unlikely, I'd hope:
this would mean a binary would not work when execed normally.
Signature · 1 line
Kees Cook
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
On 3/27/19, 2:52 PM, "linux-arm-kernel on behalf of Kees Cook" <[email protected] on behalf of [email protected]> wrote:
Adding some more people to CC... what do people think about this moving of the brk to ELF_ET_DYN_BASE in this corner-case? Anything that worked before should still work (i.e. the ultimately-launched binary already had the brk very far from its text, so this should be no different from a COMPAT_BRK standpoint). The only risk I see here is that if someone started to suddenly depend on the entire memory space above the mmap region being available when launching binaries via a direct loader execs... which seems highly unlikely, I'd hope:
this would mean a binary would not work when execed normally.
Kees' proposal addresses the issue for me. Anyone have concerns on this proposed solution?
Ali
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
Quoted 12 lines · Saidi, Ali
On 3/27/19, 2:52 PM, "linux-arm-kernel on behalf of Kees Cook" <[email protected] on behalf of [email protected]> wrote:
Adding some more people to CC... what do people think about this moving of the brk to
ELF_ET_DYN_BASEin this corner-case? Anything that worked before should still work (i.e. the ultimately-launched binary already had the brk very far from its text, so this should be no different from aCOMPAT_BRKstandpoint). The only risk I see here is that if someone started to suddenly depend on the entire memory space above the mmap region being available when launching binaries via a direct loader execs... which seems highly unlikely, I'd hope:
this would mean a binary would not work when execed normally.
Quoted 1 line
Kees' proposal addresses the issue for me. Anyone have concerns on this proposed solution?
I'd suggest incorporating all feedback and sending a v2 series - it's much easier to get people's attention via code submitted. ;-)
Thanks,
Ingo
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
Quoted 2 lines · Ingo Molnar
I'd suggest incorporating all feedback and sending a v2 series - it's much easier to get people's attention via code submitted. ;-)
I sent my patch out, akpm added it to -mm and then xtensa broke, and it got removed. Soooo... I'm looking at it again now. I think I might combine the ideas and in the interpreter case bump the entire mmap base by the brk randomization size. That should make things less strange and solve the corner case without reducing available address space in the general case.
Signature · 6 lines
Kees Cook _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
In reply to Ali SaidiQuoted 11 lines · Ali Saidi
Increase
mmap_baseby the worst-case brk randomization so that the stack and heap remain apart.In Linux 4.13 a change was committed that special cased the kernel ELF loader when the loader is invoked directly (
eab09532d400;binfmt_elf: useELF_ET_DYN_BASEonly for PIE). Generally, the loader isn’t invoked directly and this issue is limited to cases where it is, (e.g to set a non-inheritableLD_LIBRARY_PATH, testing new versions of the loader). In those rare cases, the loader doesn't take into account the amount of brk randomization that will be applied byarch_randomize_brk(). This can lead to the stack and heap being arbitrarily close to each other.
That explains not why you need this change. What's the consequence of them being close to each other?
Thanks,
tglx
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
On 3/21/19, 9:11 AM, "linux-arm-kernel on behalf of Thomas Gleixner" <[email protected] on behalf of [email protected]> wrote:
On Tue, 12 Mar 2019, Ali Saidi wrote:
> Increase mmap_base by the worst-case brk randomization so that > the stack and heap remain apart.
>
> In Linux 4.13 a change was committed that special cased the kernel ELF > loader when the loader is invoked directly (eab09532d400; binfmt_elf: use > ELF_ET_DYN_BASE only for PIE). Generally, the loader isn’t invoked > directly and this issue is limited to cases where it is, (e.g to set a > non-inheritable LD_LIBRARY_PATH, testing new versions of the loader). In > those rare cases, the loader doesn't take into account the amount of brk > randomization that will be applied by arch_randomize_brk(). This can > lead to the stack and heap being arbitrarily close to each other.
That explains not why you need this change. What's the consequence of them being close to each other?
The process doesn't get it's requested stack size and stack allocations could end up scribbling on the heap.
Thanks,
Ali
Re: [PATCH 2/2] x86/mmap: handle worst-case heap randomization in mmap_base
Quoted 16 lines · Ali
On 3/21/19, 9:11 AM, "linux-arm-kernel on behalf of Thomas Gleixner" <[email protected] on behalf of [email protected]> wrote:
On Tue, 12 Mar 2019, Ali Saidi wrote:
> Increase
mmap_baseby the worst-case brk randomization so that > the stack and heap remain apart.
>
> In Linux 4.13 a change was committed that special cased the kernel ELF > loader when the loader is invoked directly (eab09532d400;binfmt_elf: use >ELF_ET_DYN_BASEonly for PIE). Generally, the loader isn’t invoked > directly and this issue is limited to cases where it is, (e.g to set a > non-inheritableLD_LIBRARY_PATH, testing new versions of the loader). In > those rare cases, the loader doesn't take into account the amount of brk > randomization that will be applied byarch_randomize_brk(). This can > lead to the stack and heap being arbitrarily close to each other.
Quoted 5 lines
That explains not why you need this change. What's the consequence of them being close to each other?
The process doesn't get it's requested stack size and stack allocations could end up scribbling on the heap.
And exactly that information wants to be in the changelog.
Thanks,
tglx