MIPS: Ensure VDSO pages mapped above STACK_TOP
Subsystems (2)
PATCH RESEND 1/1 MIPS: Ensure VDSO pages mapped above STACK_TOP
Unlimited stack size (ulimit -s unlimited) causes kernel to use legacy layout for applications. Thus, if VDSO isn't mapped above STACK_TOP, it will be mapped at a very low address. This will probably cause an early brk() failure, because the application's initial mm->brk is usually below VDSO (especially when COMPAT_BRK is enabled) and there is no more room to expand its heap.
This patch reserve 4 MB space above STACK_TOP, and use the low 2 MB for VDSO randomization (as a result, VDSO pages can use as much as 2 MB).
--- arch/mips/include/asm/processor.h | 5 +++-- arch/mips/kernel/vdso.c | 16 +++++++++++++++- 2 files changed, 18 insertions(+), 3 deletions(-)
arch/mips/include/asm/processor.h +3 -2
@@ -13,6 +13,7 @@#include <linux/atomic.h> #include <linux/cpumask.h> +#include <linux/sizes.h> #include <linux/threads.h> #include <asm/cachectl.h>
@@ -82,9 +83,9 @@ extern unsigned int vced_count, vcei_count;/* * One page above the stack is used for branch delay slot "emulation". -* See dsemul.c for details. +* See dsemul.c for details, other pages are for VDSO. */ -#define STACK_TOP ((TASK_SIZE & PAGE_MASK) - PAGE_SIZE) +#define STACK_TOP ((TASK_SIZE & PAGE_MASK) - SZ_4M)/* * This decides where the kernel will search for a free chunk of vm
arch/mips/kernel/vdso.c +15 -1
@@ -15,6 +15,7 @@#include <linux/ioport.h> #include <linux/kernel.h> #include <linux/mm.h> +#include <linux/random.h> #include <linux/sched.h> #include <linux/slab.h> #include <linux/timekeeper_internal.h>
@@ -97,6 +98,19 @@ void update_vsyscall_tz(void) } } +static unsigned long vdso_base (void ) +{ +unsigned long offset =0UL ; + +if (current->flags &PF_RANDOMIZE ) { + offset =get_random_int (); + offset <<=PAGE_SHIFT ; + offset &=0x1ffffful ;/* 2 MB */ + } + +return STACK_TOP +PAGE_SIZE + offset; +} +int arch_setup_additional_pages (struct linux_binprm *bprm,int uses_interp) {struct mips_vdso_image *image = current->thread.abi->vdso;
@@ -137,7 +151,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (cpu_has_dc_aliases) size += shm_align_mask +1 ; - base =get_unmapped_area (NULL ,0 , size,0 ,0 ); + base =get_unmapped_area (NULL ,vdso_base (), size,0 ,0 );if (IS_ERR_VALUE (base)) { ret = base; goto out;
Signature · 1 line
2.7.0
Hi Huacai,
Quoted 7 lines · Huacai Chen
Unlimited stack size (ulimit -s unlimited) causes kernel to use legacy layout for applications. Thus, if VDSO isn't mapped above
STACK_TOP, it will be mapped at a very low address. This will probably cause an earlybrk()failure, because the application's initialmm->brkis usually below VDSO (especially whenCOMPAT_BRKis enabled) and there is no more room to expand its heap.
Quoted 8 lines
This patch reserve 4 MB space above
STACK_TOP, and use the low 2 MB for VDSO randomization (as a result, VDSO pages can use as much as 2 MB).Signed-off-by Huacai Chen <[email protected]>--- arch/mips/include/asm/processor.h | 5 +++-- arch/mips/kernel/vdso.c | 16 +++++++++++++++- 2 files changed, 18 insertions(+), 3 deletions(-)
Could you give an example of a program that fails due to this?
I'm able to reproduce that with ulimit -s unlimited the VDSO gets placed just above the heap, but the ELF interpreter & shared libraries get mapped nearby too so even with the VDSO moved a brk syscall would still presumably fail at around the same point. For example:
# ulimit -s unlimited; cat /proc/self/maps 00400000-004ec000 r-xp 00000000 08:00 71436 /usr/bin/coreutils 004fc000-004fd000 rwxp 000ec000 08:00 71436 /usr/bin/coreutils 004fd000-0050f000 rwxp 00000000 00:00 0 00cc3000-00ce4000 rwxp 00000000 00:00 0 [heap] 2ab75000-2ab96000 r-xp 00000000 08:00 44641 /lib/ld-linux-mipsn8.so.1 2ab96000-2ab98000 r--p 00000000 00:00 0 [vvar] 2ab98000-2ab99000 r-xp 00000000 00:00 0 [vdso] 2ab99000-2ab9d000 rwxp 00000000 00:00 0 2aba5000-2aba6000 r-xp 00020000 08:00 44641 /lib/ld-linux-mipsn8.so.1 2aba6000-2aba7000 rwxp 00021000 08:00 44641 /lib/ld-linux-mipsn8.so.1 2aba7000-2ad18000 r-xp 00000000 08:00 477163 /usr/lib/libcrypto.so.1.0.0 2ad18000-2ad27000 ---p 00171000 08:00 477163 /usr/lib/libcrypto.so.1.0.0 ...
Are you running something statically linked to avoid that?
Apart from that I'm not keen on unconditionally taking away 4MB of address space. For o32 programs we already only have 2GB of user address space & it's not unheard of for programs to bump up against that limit, so reducing it further would need to be a last resort. If we did need to move VDSO to the top of the user address space we can do that with a hint to get_unmapped_area() without changing STACK_TOP.
Thanks,
Paul
Quoted 41 lines · Paul Burton
Hi Huacai,
Quoted 7 lines · Huacai Chen
Unlimited stack size (ulimit -s unlimited) causes kernel to use legacy layout for applications. Thus, if VDSO isn't mapped above
STACK_TOP, it will be mapped at a very low address. This will probably cause an earlybrk()failure, because the application's initialmm->brkis usually below VDSO (especially whenCOMPAT_BRKis enabled) and there is no more room to expand its heap.Quoted 8 lines
This patch reserve 4 MB space above
STACK_TOP, and use the low 2 MB for VDSO randomization (as a result, VDSO pages can use as much as 2 MB).Signed-off-by Huacai Chen <[email protected]>--- arch/mips/include/asm/processor.h | 5 +++-- arch/mips/kernel/vdso.c | 16 +++++++++++++++- 2 files changed, 18 insertions(+), 3 deletions(-)Could you give an example of a program that fails due to this?
I'm able to reproduce that with ulimit -s unlimited the VDSO gets placed just above the heap, but the ELF interpreter & shared libraries get mapped nearby too so even with the VDSO moved a brk syscall would still presumably fail at around the same point. For example:
# ulimit -s unlimited; cat /proc/self/maps 00400000-004ec000 r-xp 00000000 08:00 71436 /usr/bin/coreutils 004fc000-004fd000 rwxp 000ec000 08:00 71436 /usr/bin/coreutils 004fd000-0050f000 rwxp 00000000 00:00 0 00cc3000-00ce4000 rwxp 00000000 00:00 0 [heap] 2ab75000-2ab96000 r-xp 00000000 08:00 44641 /lib/ld-linux-mipsn8.so.1 2ab96000-2ab98000 r--p 00000000 00:00 0 [vvar] 2ab98000-2ab99000 r-xp 00000000 00:00 0 [vdso] 2ab99000-2ab9d000 rwxp 00000000 00:00 0 2aba5000-2aba6000 r-xp 00020000 08:00 44641 /lib/ld-linux-mipsn8.so.1 2aba6000-2aba7000 rwxp 00021000 08:00 44641 /lib/ld-linux-mipsn8.so.1 2aba7000-2ad18000 r-xp 00000000 08:00 477163 /usr/lib/libcrypto.so.1.0.0 2ad18000-2ad27000 ---p 00171000 08:00 477163 /usr/lib/libcrypto.so.1.0.0 ...
Quoted 1 line
Are you running something statically linked to avoid that?
Yes, I use a statically linked binary, so heap expanding is only affected by VDSO, but not by ld.so. However, in the legacy layout case, both VDSO and ld.so are mapped begin with TASK_UNMAPPED_BASE (0x2AAAB000), so a dynamically linked binary's heap will not probablly hit VDSO/ld.so (because dynamically linked binaries are small).
Quoted 7 lines
Apart from that I'm not keen on unconditionally taking away 4MB of address space. For o32 programs we already only have 2GB of user address space & it's not unheard of for programs to bump up against that limit, so reducing it further would need to be a last resort. If we did need to move VDSO to the top of the user address space we can do that with a hint to
get_unmapped_area()without changingSTACK_TOP.
The task stack is allocated very early, if without changing STACK_TOP, get_unmapped_area() cannot assure that VDSO is above stack.
Huacai
Quoted 3 lines
Thanks,
Paul
PATCH 1/1 MIPS: VDSO: Always map near top of user memory
In reply to Huacai ChenWhen using the legacy mmap layout, for example triggered using ulimit -s unlimited, get_unmapped_area() fills memory from bottom to top starting from a fairly low address near TASK_UNMAPPED_BASE.
This placement is suboptimal if the user application wishes to allocate large amounts of heap memory using the brk syscall. With the VDSO being located low in the user's virtual address space, the amount of space available for access using brk is limited much more than it was prior to the introduction of the VDSO.
For example:
# ulimit -s unlimited; cat /proc/self/maps 00400000-004ec000 r-xp 00000000 08:00 71436 /usr/bin/coreutils 004fc000-004fd000 rwxp 000ec000 08:00 71436 /usr/bin/coreutils 004fd000-0050f000 rwxp 00000000 00:00 0 00cc3000-00ce4000 rwxp 00000000 00:00 0 [heap] 2ab96000-2ab98000 r--p 00000000 00:00 0 [vvar] 2ab98000-2ab99000 r-xp 00000000 00:00 0 [vdso] 2ab99000-2ab9d000 rwxp 00000000 00:00 0 ...
Resolve this by adjusting STACK_TOP to reserve space for the VDSO & providing an address hint to get_unmapped_area() causing it to use this space even when using the legacy mmap layout.
We reserve enough space for the VDSO, plus 1MB or 8MB for 32 bit & 64 bit systems respectively within which we randomize the VDSO base address. Previously this randomization was taken care of by the mmap base address randomization performed by arch_mmap_rnd(). The 1MB & 8MB sizes are somewhat arbitrary but chosen such that we have some randomization without taking up too much of the user's virtual address space, which is often in short supply for 32 bit systems.
With this the VDSO is always mapped at a high address, leaving lots of space for statically linked programs to make use of brk:
# ulimit -s unlimited; cat /proc/self/maps 00400000-004ec000 r-xp 00000000 08:00 71436 /usr/bin/coreutils 004fc000-004fd000 rwxp 000ec000 08:00 71436 /usr/bin/coreutils 004fd000-0050f000 rwxp 00000000 00:00 0 00c28000-00c49000 rwxp 00000000 00:00 0 [heap] ... 7f67c000-7f69d000 rwxp 00000000 00:00 0 [stack] 7f7fc000-7f7fd000 rwxp 00000000 00:00 0 7fcf1000-7fcf3000 r--p 00000000 00:00 0 [vvar] 7fcf3000-7fcf4000 r-xp 00000000 00:00 0 [vdso]
---
Huacai, could you test if this fixes your problem?
It takes up less of the user's virtual address space (only 1MB for MIPS32), takes none at all when randomization is disabled, and doesn't make assumptions about the size of the VDSO.
--- arch/mips/include/asm/processor.h | 10 +++++----- arch/mips/kernel/process.c | 25 +++++++++++++++++++++++++ arch/mips/kernel/vdso.c | 18 +++++++++++++++++- 3 files changed, 47 insertions(+), 6 deletions(-)
arch/mips/include/asm/processor.h +5 -5
@@ -13,6 +13,7 @@#include <linux/atomic.h> #include <linux/cpumask.h> +#include <linux/sizes.h> #include <linux/threads.h> #include <asm/cachectl.h>
@@ -80,11 +81,10 @@ extern unsigned int vced_count, vcei_count;#endif -/* -* One page above the stack is used for branch delay slot "emulation". -* See dsemul.c for details. -*/ -#define STACK_TOP ((TASK_SIZE & PAGE_MASK) - PAGE_SIZE) +#define VDSO_RANDOMIZE_SIZE (test_thread_flag(TIF_32BIT_ADDR) ? SZ_1M : SZ_8M) + +extern unsigned long mips_stack_top (void ); +#define STACK_TOP mips_stack_top()/* * This decides where the kernel will search for a free chunk of vm
arch/mips/kernel/process.c +25 -0
@@ -32,6 +32,7 @@#include <linux/nmi.h> #include <linux/cpu.h> +#include <asm/abi.h> #include <asm/asm.h> #include <asm/bootinfo.h> #include <asm/cpu.h>
@@ -39,6 +40,7 @@#include <asm/dsp.h> #include <asm/fpu.h> #include <asm/irq.h> +#include <asm/mips-gic.h> #include <asm/msa.h> #include <asm/pgtable.h> #include <asm/mipsregs.h>
@@ -645,6 +647,29 @@ unsigned long get_wchan(struct task_struct *task)return pc; } +unsigned long mips_stack_top (void ) +{ +unsigned long top =TASK_SIZE &PAGE_MASK ; + +/* One page for branch delay slot "emulation" */ + top -=PAGE_SIZE ; + +/* Space for the VDSO, data page & GIC user page */ + top -=PAGE_ALIGN (current->thread.abi->vdso->size); + top -=PAGE_SIZE ; + top -=mips_gic_present () ?PAGE_SIZE :0 ; + +/* Space for cache colour alignment */ +if (cpu_has_dc_aliases) + top -= shm_align_mask +1 ; + +/* Space to randomize the VDSO base */ +if (current->flags &PF_RANDOMIZE ) + top -=VDSO_RANDOMIZE_SIZE ; + +return top; +} +/* * Don't forget that the stack pointer must be aligned on a 8 bytes * boundary for 32-bits ABI and 16 bytes for 64-bits ABI.
arch/mips/kernel/vdso.c +17 -1
@@ -15,6 +15,7 @@#include <linux/ioport.h> #include <linux/kernel.h> #include <linux/mm.h> +#include <linux/random.h> #include <linux/sched.h> #include <linux/slab.h> #include <linux/timekeeper_internal.h>
@@ -97,6 +98,21 @@ void update_vsyscall_tz(void) } } +static unsigned long vdso_base (void ) +{ +unsigned long base; + +/* Skip the delay slot emulation page */ + base =STACK_TOP +PAGE_SIZE ; + +if (current->flags &PF_RANDOMIZE ) { + base +=get_random_int () & (VDSO_RANDOMIZE_SIZE -1 ); + base =PAGE_ALIGN (base); + } + +return base; +} +int arch_setup_additional_pages (struct linux_binprm *bprm,int uses_interp) {struct mips_vdso_image *image = current->thread.abi->vdso;
@@ -137,7 +153,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (cpu_has_dc_aliases) size += shm_align_mask +1 ; - base =get_unmapped_area (NULL ,0 , size,0 ,0 ); + base =get_unmapped_area (NULL ,vdso_base (), size,0 ,0 );if (IS_ERR_VALUE (base)) { ret = base; goto out;
Signature · 1 line
2.18.0
Re:[PATCH] MIPS: VDSO: Always map near top of user memory
Hi, Paul,
I think this patch can solve my problem, but I have something to say:
1, VDSO randomization is introduced in commit ccd3988086364837d0c0fb4563d7 in 3.19. So this patch should backported as early as 3.19. 2, vdso_base() may need to be modified for 4.4, because 4.4 has no delay slot emulation.
3, Maybe it is better to set VDSO_RANDOMIZE_SIZE to 16MB for 64bit kernel, because commit ccd3988086364837d0c0fb4563d7 use 16MB.
Huacai
------------------ Original ------------------
From: "Paul Burton"<[email protected]>; Date: Wed, Sep 26, 2018 07:23 AM To: "Huacai Chen"<[email protected]>; "Huacai Chen"<[email protected]>; "linux-mips"<[email protected]>;
Cc: "stable"<[email protected]>; "Paul Burton"<[email protected]>;
Subject: [PATCH] MIPS: VDSO: Always map near top of user memory
When using the legacy mmap layout, for example triggered using ulimit -s unlimited, get_unmapped_area() fills memory from bottom to top starting from a fairly low address near TASK_UNMAPPED_BASE.
This placement is suboptimal if the user application wishes to allocate large amounts of heap memory using the brk syscall. With the VDSO being located low in the user's virtual address space, the amount of space available for access using brk is limited much more than it was prior to the introduction of the VDSO.
For example:
# ulimit -s unlimited; cat /proc/self/maps 00400000-004ec000 r-xp 00000000 08:00 71436 /usr/bin/coreutils 004fc000-004fd000 rwxp 000ec000 08:00 71436 /usr/bin/coreutils 004fd000-0050f000 rwxp 00000000 00:00 0 00cc3000-00ce4000 rwxp 00000000 00:00 0 [heap] 2ab96000-2ab98000 r--p 00000000 00:00 0 [vvar] 2ab98000-2ab99000 r-xp 00000000 00:00 0 [vdso] 2ab99000-2ab9d000 rwxp 00000000 00:00 0 ...
Resolve this by adjusting STACK_TOP to reserve space for the VDSO & providing an address hint to get_unmapped_area() causing it to use this space even when using the legacy mmap layout.
We reserve enough space for the VDSO, plus 1MB or 8MB for 32 bit & 64 bit systems respectively within which we randomize the VDSO base address. Previously this randomization was taken care of by the mmap base address randomization performed by arch_mmap_rnd(). The 1MB & 8MB sizes are somewhat arbitrary but chosen such that we have some randomization without taking up too much of the user's virtual address space, which is often in short supply for 32 bit systems.
With this the VDSO is always mapped at a high address, leaving lots of space for statically linked programs to make use of brk:
# ulimit -s unlimited; cat /proc/self/maps 00400000-004ec000 r-xp 00000000 08:00 71436 /usr/bin/coreutils 004fc000-004fd000 rwxp 000ec000 08:00 71436 /usr/bin/coreutils 004fd000-0050f000 rwxp 00000000 00:00 0 00c28000-00c49000 rwxp 00000000 00:00 0 [heap] ... 7f67c000-7f69d000 rwxp 00000000 00:00 0 [stack] 7f7fc000-7f7fd000 rwxp 00000000 00:00 0 7fcf1000-7fcf3000 r--p 00000000 00:00 0 [vvar] 7fcf3000-7fcf4000 r-xp 00000000 00:00 0 [vdso]
---
Huacai, could you test if this fixes your problem?
It takes up less of the user's virtual address space (only 1MB for MIPS32), takes none at all when randomization is disabled, and doesn't make assumptions about the size of the VDSO.
--- arch/mips/include/asm/processor.h | 10 +++++----- arch/mips/kernel/process.c | 25 +++++++++++++++++++++++++ arch/mips/kernel/vdso.c | 18 +++++++++++++++++- 3 files changed, 47 insertions(+), 6 deletions(-)
arch/mips/include/asm/processor.h +5 -5
@@ -13,6 +13,7 @@#include <linux/atomic.h> #include <linux/cpumask.h> +#include <linux/sizes.h> #include <linux/threads.h> #include <asm/cachectl.h>
@@ -80,11 +81,10 @@ extern unsigned int vced_count, vcei_count;#endif -/* -* One page above the stack is used for branch delay slot "emulation". -* See dsemul.c for details. -*/ -#define STACK_TOP ((TASK_SIZE & PAGE_MASK) - PAGE_SIZE) +#define VDSO_RANDOMIZE_SIZE (test_thread_flag(TIF_32BIT_ADDR) ? SZ_1M : SZ_8M) + +extern unsigned long mips_stack_top (void ); +#define STACK_TOP mips_stack_top()/* * This decides where the kernel will search for a free chunk of vm
arch/mips/kernel/process.c +25 -0
@@ -32,6 +32,7 @@#include <linux/nmi.h> #include <linux/cpu.h> +#include <asm/abi.h> #include <asm/asm.h> #include <asm/bootinfo.h> #include <asm/cpu.h>
@@ -39,6 +40,7 @@#include <asm/dsp.h> #include <asm/fpu.h> #include <asm/irq.h> +#include <asm/mips-gic.h> #include <asm/msa.h> #include <asm/pgtable.h> #include <asm/mipsregs.h>
@@ -645,6 +647,29 @@ unsigned long get_wchan(struct task_struct *task)return pc; } +unsigned long mips_stack_top (void ) +{ +unsigned long top =TASK_SIZE &PAGE_MASK ; + +/* One page for branch delay slot "emulation" */ + top -=PAGE_SIZE ; + +/* Space for the VDSO, data page & GIC user page */ + top -=PAGE_ALIGN (current->thread.abi->vdso->size); + top -=PAGE_SIZE ; + top -=mips_gic_present () ?PAGE_SIZE :0 ; + +/* Space for cache colour alignment */ +if (cpu_has_dc_aliases) + top -= shm_align_mask +1 ; + +/* Space to randomize the VDSO base */ +if (current->flags &PF_RANDOMIZE ) + top -=VDSO_RANDOMIZE_SIZE ; + +return top; +} +/* * Don't forget that the stack pointer must be aligned on a 8 bytes * boundary for 32-bits ABI and 16 bytes for 64-bits ABI.
arch/mips/kernel/vdso.c +17 -1
@@ -15,6 +15,7 @@#include <linux/ioport.h> #include <linux/kernel.h> #include <linux/mm.h> +#include <linux/random.h> #include <linux/sched.h> #include <linux/slab.h> #include <linux/timekeeper_internal.h>
@@ -97,6 +98,21 @@ void update_vsyscall_tz(void) } } +static unsigned long vdso_base (void ) +{ +unsigned long base; + +/* Skip the delay slot emulation page */ + base =STACK_TOP +PAGE_SIZE ; + +if (current->flags &PF_RANDOMIZE ) { + base +=get_random_int () & (VDSO_RANDOMIZE_SIZE -1 ); + base =PAGE_ALIGN (base); + } + +return base; +} +int arch_setup_additional_pages (struct linux_binprm *bprm,int uses_interp) {struct mips_vdso_image *image = current->thread.abi->vdso;
@@ -137,7 +153,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (cpu_has_dc_aliases) size += shm_align_mask +1 ; - base =get_unmapped_area (NULL ,0 , size,0 ,0 ); + base =get_unmapped_area (NULL ,vdso_base (), size,0 ,0 );if (IS_ERR_VALUE (base)) { ret = base; goto out;
Signature · 1 line
2.18.0
Re: [PATCH] MIPS: VDSO: Always map near top of user memory
Hi Huacai,
Quoted 5 lines · 陈华才
I think this patch can solve my problem, but I have something to say:
1, VDSO randomization is introduced in commit
ccd3988086364837d0c0fb4563d7in 3.19. So this patch should backported as early as 3.19.
Ugh, OK so the not-really-a-VDSO page that we called VDSO before we really had a VDSO was randomly located too. A backport to that would need to be significantly different.
However the only currently supported LTS kernel older than 4.4 (which introduced the real VDSO) is 3.16. 3.16 didn't support the randomization anyway, I don't think it's worth the effort to backport any further than 4.4.
Quoted 2 lines
2,
vdso_base()may need to be modified for 4.4, because 4.4 has no delay slot emulation.
Yes I know. The GIC code changed in 4.14 too so backports will need to take that into account too.
Quoted 2 lines
3, Maybe it is better to set
VDSO_RANDOMIZE_SIZEto 16MB for 64bit kernel, because commitccd3988086364837d0c0fb4563d7use 16MB.
It actually uses 256MB for MIPS64. Since address space isn't scarce for MIPS64 I'll go with that.
Thanks,
Paul