x86: Don't randomize stack unless current->personality permits it
Quoted 26 lines · Frank van Maarseveen
Do not randomize stack location unless
current->personalitypermits it.Signed-off-by Frank van Maarseveen <[email protected]>---
The problem seems also present in
arch/um/kernel/process_kern.c arch/x86_64/kernel/process.carch/i386/kernel/process.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletion(-)diff -rup
a/arch/i386/kernel/process.cb/arch/i386/kernel/process.c
---a/arch/i386/kernel/process.c2006-06-23 16:08:13.000000000
+0200 +++b/arch/i386/kernel/process.c2006-07-11
14:39:20.000000000 +0200 @@ -38,6 +38,7 @@
#include <linux/kallsyms.h>
#include <linux/ptrace.h>
#include <linux/random.h>(quoted diff) +1
-0+#include <linux/personality.h> #include <asm/uaccess.h> #include <asm/pgtable.h>
Review asmlinkage int sys_get_thread_area(struc · line 905
Quoted 8 lines
(quoted diff) +1
-1@@ -898,7 +899,7 @@ asmlinkage int sys_get_thread_area(struc unsigned long arch_align_stack(unsigned long sp) { - if (randomize_va_space) + if (!(current->personality & ADDR_NO_RANDOMIZE) && randomize_va_space) sp -= get_random_int() % 8192; return sp & ~0xf; }
It still blips on my system.
echo 0 > /proc/sys/kernel/randomize_va_space makes the blips go away.
???
Thanks!
--
Al
Quoted 38 lines · Al Boldi
Quoted 26 lines · Frank van Maarseveen
Do not randomize stack location unless
current->personalitypermits it.Signed-off-by Frank van Maarseveen <[email protected]>---
The problem seems also present in
arch/um/kernel/process_kern.c arch/x86_64/kernel/process.carch/i386/kernel/process.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletion(-)diff -rup
a/arch/i386/kernel/process.cb/arch/i386/kernel/process.c
---a/arch/i386/kernel/process.c2006-06-23 16:08:13.000000000
+0200 +++b/arch/i386/kernel/process.c2006-07-11
14:39:20.000000000 +0200 @@ -38,6 +38,7 @@
#include <linux/kallsyms.h>
#include <linux/ptrace.h>
#include <linux/random.h>(quoted diff) +1
-0+#include <linux/personality.h> #include <asm/uaccess.h> #include <asm/pgtable.h>Review asmlinkage int sys_get_thread_area(struc · line 905
Quoted 8 lines
(quoted diff) +1
-1@@ -898,7 +899,7 @@ asmlinkage int sys_get_thread_area(struc unsigned long arch_align_stack(unsigned long sp) { - if (randomize_va_space) + if (!(current->personality & ADDR_NO_RANDOMIZE) && randomize_va_space) sp -= get_random_int() % 8192; return sp & ~0xf; }It still blips on my system.
Quoted 3 lines
echo 0 >
/proc/sys/kernel/randomize_va_spacemakes the blips go away.???
fs/binfmt_elf.c:randomize_stack_top would need the same check -Andi
Quoted 39 lines · Andi Kleen
Quoted 38 lines · Al Boldi
Quoted 26 lines · Frank van Maarseveen
Do not randomize stack location unless
current->personalitypermits it.Signed-off-by Frank van Maarseveen <[email protected]>---
The problem seems also present in
arch/um/kernel/process_kern.c arch/x86_64/kernel/process.carch/i386/kernel/process.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletion(-)diff -rup
a/arch/i386/kernel/process.cb/arch/i386/kernel/process.c
---a/arch/i386/kernel/process.c2006-06-23 16:08:13.000000000
+0200 +++b/arch/i386/kernel/process.c2006-07-11
14:39:20.000000000 +0200 @@ -38,6 +38,7 @@
#include <linux/kallsyms.h>
#include <linux/ptrace.h>
#include <linux/random.h>(quoted diff) +1
-0+#include <linux/personality.h> #include <asm/uaccess.h> #include <asm/pgtable.h>Review asmlinkage int sys_get_thread_area(struc · line 905
Quoted 8 lines
(quoted diff) +1
-1@@ -898,7 +899,7 @@ asmlinkage int sys_get_thread_area(struc unsigned long arch_align_stack(unsigned long sp) { - if (randomize_va_space) + if (!(current->personality & ADDR_NO_RANDOMIZE) && randomize_va_space) sp -= get_random_int() % 8192; return sp & ~0xf; }It still blips on my system.
Quoted 5 lines
Quoted 3 lines
echo 0 >
/proc/sys/kernel/randomize_va_spacemakes the blips go away.???
fs/binfmt_elf.c:randomize_stack_top would need the same check
It already checks for it, but makes no difference.
Thanks for looking into this!
--
Al
Quoted 4 lines · Al Boldi
It already checks for it, but makes no difference.
Thanks for looking into this!
well glibc also randomizes things a bit (for better cache coloring) ...
Quoted 6 lines · Arjan van de Ven
Quoted 3 lines · Al Boldi
It already checks for it, but makes no difference.
Thanks for looking into this!
well glibc also randomizes things a bit (for better cache coloring) ...
Any way to turn it off?
Thanks!
--
Al
Quoted 5 lines · Andi Kleen
Quoted 3 lines · Al Boldi
Quoted 2 lines · Frank van Maarseveen
Do not randomize stack location unless
current->personalitypermits it.
[...]
Quoted 8 lines
Quoted 6 lines
It still blips on my system.
echo 0 >
/proc/sys/kernel/randomize_va_spacemakes the blips go away.???
fs/binfmt_elf.c:randomize_stack_top would need the same check
Actually, randomize_stack_top() checks
if (current->flags & PF_RANDOMIZE) {and it's only called from load_elf_binary() right after this:
if (!(current->personality & ADDR_NO_RANDOMIZE) && randomize_va_space) current->flags |= PF_RANDOMIZE;
Further on create_elf_tables() is called and that one calls arch_align_stack() so maybe it is more appropriate to test (current->flags & PF_RANDOMIZE) in arch_align_stack() instead of recomputing it.
Signature · 1 line
Frank
Quoted 20 lines · Frank van Maarseveen
Quoted 4 lines · Andi Kleen
Quoted 3 lines · Al Boldi
Quoted 2 lines · Frank van Maarseveen
Do not randomize stack location unless
current->personalitypermits it.[...]
Quoted 7 lines
Quoted 5 lines
It still blips on my system.
echo 0 >
/proc/sys/kernel/randomize_va_spacemakes the blips go away.???
fs/binfmt_elf.c:randomize_stack_top would need the same check
Actually,
randomize_stack_top()checksif (current->flags & PF_RANDOMIZE) {
Quoted 8 lines
and it's only called from
load_elf_binary()right after this:if (!(current->personality & ADDR_NO_RANDOMIZE) && randomize_va_space) current->flags |= PF_RANDOMIZE;Further on
create_elf_tables()is called and that one callsarch_align_stack()so maybe it is more appropriate to test (current->flags&PF_RANDOMIZE) inarch_align_stack()instead of recomputing it.
exec.c uses arch_align_stack() also for non-elf.
BTW, why does randomize_stack_top() mod against (8192*1024) instead of (8192) like arch_align_stack()?
Thanks!
--
Al
Quoted 4 lines
BTW, why does
randomize_stack_top()mod against (8192*1024) instead of (8192) likearch_align_stack()?
because it wants to randomize for 8Mb, unlike arch_align_stack which
wants to randomize the last 8Kb within this 8Mb ;)
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines
BTW, why does
randomize_stack_top()mod against (8192*1024) instead of (8192) likearch_align_stack()?because it wants to randomize for 8Mb, unlike
arch_align_stackwhich
wants to randomize the last 8Kb within this 8Mb ;)
Randomizing twice?
Anyway, I tried different combinations of turning off randomization in both functions and got mixed results, so it looks like there is some interaction here.
Trying different compiler versions and switches also show different results.
Calling these slowdowns blips is really an understatement, as there are cases which lock into 800% hits. i.e: processes that, when repeatedly called, lock into a continuous 8x slowdown on i686P4.
There is even a case where a mere rename or running through an extra shell causes a slowdown. And that's with randomization turned off.
2.4.31 doesn't show these slowdowns.
What is 2.6 doing?
Thanks!
--
Al
Quoted 8 lines · Al Boldi
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines
BTW, why does
randomize_stack_top()mod against (8192*1024) instead of (8192) likearch_align_stack()?because it wants to randomize for 8Mb, unlike
arch_align_stackwhich
wants to randomize the last 8Kb within this 8Mb ;)Randomizing twice?
a VMA can only be randomized in 4Kb (well page size) granularity, so the 8Mb randomization can only work in that 4Kb unit, the "second" randomization can work in 16 byte granularity.
Quoted 2 lines
There is even a case where a mere rename or running through an extra shell causes a slowdown. And that's with randomization turned off.
randomization off will slow stuff down yes... you get cache alias contention that way.
Quoted 1 line
2.4.31 doesn't show these slowdowns.
2.4.31 randomizes the stack with 8Kb.
Quoted 1 line
What is 2.6 doing?
you're not providing a lot of info ;)
why do you suspect randomization as cause for whatever slowdown you are seeing? What kind of slowdown are you seeing?
Quoted 14 lines · Arjan van de Ven
Quoted 8 lines · Al Boldi
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines
BTW, why does
randomize_stack_top()mod against (8192*1024) instead of (8192) likearch_align_stack()?because it wants to randomize for 8Mb, unlike
arch_align_stackwhich
wants to randomize the last 8Kb within this 8Mb ;)Randomizing twice?
a VMA can only be randomized in 4Kb (well page size) granularity, so the 8Mb randomization can only work in that 4Kb unit, the "second" randomization can work in 16 byte granularity.
Quoted 5 lines
Quoted 2 lines
There is even a case where a mere rename or running through an extra shell causes a slowdown. And that's with randomization turned off.
randomization off will slow stuff down yes... you get cache alias contention that way.
Randomization on. Executable runs with 8x blips/hits. Randomization off. Executable runs without blips/hits.
With randomization off, a mere rename causes an 8x-slowdown to occur. Run this renamed executable through sh -c ./tstExec, and the slowdown disappears. Really weired :)
Quoted 10 lines
Quoted 1 line
2.4.31 doesn't show these slowdowns.
2.4.31 randomizes the stack with 8Kb.
Quoted 1 line
What is 2.6 doing?
you're not providing a lot of info ;)
why do you suspect randomization as cause for whatever slowdown you are seeing?
echo 0 > /proc/sys/kernel/randomize_va_space makes the blips/hits to go away, most of the time.
Quoted 1 line
What kind of slowdown are you seeing?
see below.
Beware, this is highly compiler/glibc/distribution dependent.
My test environment is mdk9.1, gcc-3.2.2-3, 2.6.17.4, compile with -Os switch Confirmed on rhel4, gcc-4.0.1, 2.6.9-5.EL, compile with no switches
Thanks!
--
Al
---
#include <stdio.h>
#include <sys/time.h>
unsigned long elapsed(int start) {
static struct timeval s,e; if (start) return gettimeofday(&s, NULL); gettimeofday(&e, NULL); return ((e.tv_sec - s.tv_sec) * 1000 + (e.tv_usec - s.tv_usec) / 1000); }
void fn2(double *x, double *y) {
*x = *y; }
void fn() {
long i = 9999999;
double x,y;
elapsed(1);
while (i--) fn2(&x,&y);
printf("%4lu ",elapsed(0));
}int main(int argc, char **argv) {
fn();
fn();
fn();
fn();
fn();
fn();
fn();
fn();
fn();
fn();
printf("msec\n");
return 0;
}Quoted 15 lines · Al Boldi
Quoted 14 lines · Arjan van de Ven
Quoted 8 lines · Al Boldi
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines
BTW, why does
randomize_stack_top()mod against (8192*1024) instead of (8192) likearch_align_stack()?because it wants to randomize for 8Mb, unlike
arch_align_stackwhich
wants to randomize the last 8Kb within this 8Mb ;)Randomizing twice?
a VMA can only be randomized in 4Kb (well page size) granularity, so the 8Mb randomization can only work in that 4Kb unit, the "second" randomization can work in 16 byte granularity.
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
There is even a case where a mere rename or running through an extra shell causes a slowdown. And that's with randomization turned off.
randomization off will slow stuff down yes... you get cache alias contention that way.
a question.. do you have prelink installed/active on your system? that may very well mess with timings...
Quoted 16 lines · Arjan van de Ven
Quoted 15 lines · Al Boldi
Quoted 14 lines · Arjan van de Ven
Quoted 8 lines · Al Boldi
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines
BTW, why does
randomize_stack_top()mod against (8192*1024) instead of (8192) likearch_align_stack()?because it wants to randomize for 8Mb, unlike
arch_align_stack
which wants to randomize the last 8Kb within this 8Mb ;)Randomizing twice?
a VMA can only be randomized in 4Kb (well page size) granularity, so the 8Mb randomization can only work in that 4Kb unit, the "second" randomization can work in 16 byte granularity.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
There is even a case where a mere rename or running through an extra shell causes a slowdown. And that's with randomization turned off.
randomization off will slow stuff down yes... you get cache alias contention that way.
a question.. do you have prelink installed/active on your system? that may very well mess with timings...
No, not that I am aware of. How can I find out?
I confirmed this on a standard rhel4 virgin-install. Does rhel4 have prelink enabled by default?
But even so, I don't think this is a timing issue, as the slowdowns are so huge it is obvious even without timing.
And 2.4.31 doesn't have this problem.
Did you try it?
Thanks!
--
Al
Quoted 9 lines · Al Boldi
[...] > void
fn(){long i = 9999999; double x,y; elapsed(1); while (i--) fn2(&x,&y); printf("%4lu ",elapsed(0)); }
You are not initializing x and y and with -Os at least my gcc really uses floating point load/store operations to handle that code.
Maybe the coprocessor has a hard time normalizing certain garbage on the stack, but without/with randomization the data comes from other addresses and you're just lucky with the contents.
Does this also happens if you add a "x=0, y=0;" line to that function?
Signature · 4 lines
Paulo Marques - www.grupopie.com
Pointy-Haired Boss: I don't see anything that could stand in our way.
Dilbert: Sanity? Reality? The laws of physics?Quoted 6 lines · Paulo Marques
Quoted 5 lines · Al Boldi
[...] > void
fn(){long i = 9999999; double x,y;
Quoted 7 lines
Quoted 4 lines
elapsed(1); while (i--) fn2(&x,&y); printf("%4lu ",elapsed(0)); }You are not initializing x and y and with -Os at least my gcc really uses floating point load/store operations to handle that code.
Thanks for pointing that out.
I was really waiting for someone to critique this, but keep in mind this code tries to surface a performance problem, and any modification changes the semantics of the compiled code, which then may yield different results.
Quoted 3 lines
Maybe the coprocessor has a hard time normalizing certain garbage on the stack, but without/with randomization the data comes from other addresses and you're just lucky with the contents.
Good point, but this random garbage makes the test even more realistic, as this code would thus cover more variations without actually coding for it.
Quoted 1 line
Does this also happens if you add a "x=0, y=0;" line to that function?
with arch_stack_align using 0xf
gcc -Os tstExec.c
randomization on
causes 2x blips/hits
randomization off
causes no blips/hits
mv a.out tstExec
causes continuous 2x slowdown
sh -c ./tstExec
causes slowdown to disappear (can somebody explain this weirdness?)
with arch_stack_align using 0x7f
all weirdness is gone
gcc -O3 tstExec.c
randomization on
causes some minor blips/hits
randomization off
causes even less blips/hits
Going one step further,
with #define arch_stack_align(x) (x)
all blips/hits/weirdness are gone
Which means that either arch_stack_align isn't necessary at all, or randomization isn't working as intended.
Can somebody prove me wrong here?
Thanks!
--
Al