MAX_ARG_PAGES has no effect?
The other day I was running a grep on a big directory tree and got a "Argument list too long" error. Since I'd like to have this work without messing with find and xargs each time, I went into include/linux/binfmts.h and changed
#define MAX_ARG_PAGES 32
to
#define MAX_ARG_PAGES 64
I recompiled and installed the kernel, but there's no change (getconf ARG_MAX still gives 131072.) What am I missing?
I am running 2.6.13 on amd64.
Thanks,
Nick Matteo
Quoted 5 lines · Nick Matteo
The other day I was running a grep on a big directory tree and got a "Argument list too long" error. Since I'd like to have this work without messing with find and xargs each time, I went into
include/linux/binfmts.hand changed
Quoted 8 lines
#define
MAX_ARG_PAGES32to
#define
MAX_ARG_PAGES64I recompiled and installed the kernel, but there's no change (getconf
ARG_MAXstill gives 131072.) What am I missing?
MAX_ARG_PAGES should work just fine. I think the 'getconf ARG_MAX' output is hardcoded. (because the kernel does not provide the information dynamically)
Ingo
Quoted 6 lines · Ingo Molnar
Quoted 2 lines
I recompiled and installed the kernel, but there's no change (getconf
ARG_MAXstill gives 131072.) What am I missing?
MAX_ARG_PAGESshould work just fine. I think the 'getconfARG_MAX' output is hardcoded. (because the kernel does not provide the information dynamically)
Yeah, you get the value of ARG_MAX from <linux/limits.h> that was compiled in when you compiled glibc.
Jakub
Quoted 4 lines · Ingo Molnar
MAX_ARG_PAGESshould work just fine. I think the 'getconfARG_MAX' output is hardcoded. (because the kernel does not provide the information dynamically)
Perhaps it would be a good idea to make it a sysctl. Is there any reason it should be hardcoded? I cannot think of any.
Ok if someone lowers the sysctl then execve has to handle the case of the args/environment possibly not fitting anymore, but that should be easy.
-Andi
Quoted 6 lines · Andi Kleen
Quoted 4 lines · Ingo Molnar
MAX_ARG_PAGESshould work just fine. I think the 'getconfARG_MAX' output is hardcoded. (because the kernel does not provide the information dynamically)
Quoted 6 lines
Perhaps it would be a good idea to make it a sysctl. Is there any reason it should be hardcoded? I cannot think of any.
Ok if someone lowers the sysctl then execve has to handle the case of the args/environment possibly not fitting anymore, but that should be easy.
the whole thing should be reworked, so that there is no artificial limit like MAX_ARG_PAGES. (it is after all just another piece of memory, in theory)
I have tried this a couple of times but failed - it's a hard problem. Linus had the idea years ago to page-flip the argument data into the new process's address space, but that doesnt work out in practice due to the way glibc has to extend the environment space. (glibc extends it by modifying the environment array, or relocating it if it has to be grown. execve() currently automatically 'linearizes' the environment by copying both the array and the old and new environment strings to a linear piece of memory.)
If we do unconditional page-flipping then we fragment the argument space, if we do both page-flipping if things are unfragmented and well-aligned, and 'compact' the layout otherwise, we havent solved the problem and have introduced a significant extra layer of complexity to an already security-sensitive and fragile piece of code.
The best method i found was to get rid of bprm->pages[] and to directly copy strings into the new mm via kmap (and to follow whatever RAM allocation policies/limits there are for the new mm), but that's quite ugly.
Ingo
Quoted 3 lines · Ingo Molnar
the whole thing should be reworked, so that there is no artificial limit like
MAX_ARG_PAGES. (it is after all just another piece of memory, in theory)
Yes, a sysctl would probably lead to fragmentation problems and then people would do ugly linked lists of buffers like poll.
Quoted 5 lines
If we do unconditional page-flipping then we fragment the argument space, if we do both page-flipping if things are unfragmented and well-aligned, and 'compact' the layout otherwise, we havent solved the problem and have introduced a significant extra layer of complexity to an already security-sensitive and fragile piece of code.
Page flipping = COW like fork would do?
Not sure how this would work - the arguments of execve can be anywhere in the address space and would presumably be often be in a inconvenient place like in the middle of the stack of the new executable.
Quoted 4 lines
The best method i found was to get rid of bprm->pages[] and to directly copy strings into the new mm via kmap (and to follow whatever RAM allocation policies/limits there are for the new mm), but that's quite ugly.
That sounds better.
-Andi
Quoted 8 lines · Andi Kleen
Quoted 3 lines · Ingo Molnar
the whole thing should be reworked, so that there is no artificial limit like
MAX_ARG_PAGES. (it is after all just another piece of memory, in theory)Yes, a sysctl would probably lead to fragmentation problems and then people would do ugly linked lists of buffers like poll.
not really fragmentation problems (the unit of allocation of argument pages is already a single page, and we do an array of pages), the real problem is the DoS - right now the array pages are unswappable while an exec() is ongoing.
Quoted 7 lines
Quoted 5 lines
If we do unconditional page-flipping then we fragment the argument space, if we do both page-flipping if things are unfragmented and well-aligned, and 'compact' the layout otherwise, we havent solved the problem and have introduced a significant extra layer of complexity to an already security-sensitive and fragile piece of code.
Page flipping = COW like fork would do?
i dont think we need COW. During execve() we are destroying the old context and are creating a completely new context, so in theory we could just 'flip over' the argument/environment pages (which are a parameter to sys_execve()) from the old mm into the newly created mm, without caring about the old mm.
Quoted 4 lines
Not sure how this would work - the arguments of execve can be anywhere in the address space and would presumably be often be in a inconvenient place like in the middle of the stack of the new executable.
yes, that's one of the issues. I've done some instrumentation some time ago and it seemed that the arguments are typically page-aligned, so the only issue would be to clear the partial page at the end of the arguments. But i still think the concept is volatile.
Quoted 6 lines
Quoted 4 lines
The best method i found was to get rid of bprm->pages[] and to directly copy strings into the new mm via kmap (and to follow whatever RAM allocation policies/limits there are for the new mm), but that's quite ugly.
That sounds better.
yeah. It's also pretty laborous though.
Ingo