File Limit in Kernel?
A message that is not in the archive (sent off-list or not yet received): 1037115535.1439.5.camel@beowulf.cryptocomm.com.suse.lists.linux.kernel
Quoted 10 lines · Adam Voigt
--=-gjU1b+qYiuNy2hSLpQYr
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printableI have a directory with 39,000 files in it, and I'm trying to use the cp command to copy them into another directory, and neither the cp or the mv command will work, they both same "argument list too long" when I use:
cp -f *
/usr/local/www/images
Kind of. The * is expanded by the shell. The kernel limits the max length of program arguments, which is biting you here. In theory you could increase the MAX_ARG_PAGES #define in linux/binfmts.h and recompile. No guarantee that it won't have any bad side effects though. The default is rather low, it should be probably increased (I also regularly run into this)
The actual limit of files per directory is usually around 65000 in most fs.
For your immediate problem you can use
find -type f | xargs cp -iX -f X /usr/local/www/images
-Andi
Quoted 8 lines · Andi Kleen
Quoted 1 line
cp -f *
/usr/local/www/imagesKind of. The * is expanded by the shell. The kernel limits the max length of program arguments, which is biting you here. In theory you could increase the
MAX_ARG_PAGES#define inlinux/binfmts.hand recompile. No guarantee that it won't have any bad side effects though. The default is rather low, it should be probably increased (I also regularly run into this)
Yes, you can do this. I used to do it with 2.0 kernels, because our "make" command lines were very lock, and you couldn't use files to hold the list of make-generated names because even "echo $(LIST_OF_FILES) > list" hit this limit.
Ages ago somebody promised to fix this limitation :)
-- Jamie
Hi!
Quoted 5 lines
Quoted 5 lines
I have a directory with 39,000 files in it, and I'm trying to use the cp command to copy them into another directory, and neither the cp or the mv command will work, they both same "argument list too long" when I use:
Quoted 8 lines
Quoted 1 line
cp -f *
/usr/local/www/imagesKind of. The * is expanded by the shell. The kernel limits the max length of program arguments, which is biting you here. In theory you could increase the
MAX_ARG_PAGES#define inlinux/binfmts.hand recompile. No guarantee that it won't have any bad side effects though. The default is rather low, it should be probably increased (I also regularly run into this)
I have been making that limit higher 5 years ago. Perhaps its time to up it for everyone?
Pavel
Signature · 1 line
When do you have heart between your knees?
Quoted 16 lines · Pavel Machek
Hi!
Quoted 5 lines
Quoted 5 lines
I have a directory with 39,000 files in it, and I'm trying to use the cp command to copy them into another directory, and neither the cp or the mv command will work, they both same "argument list too long" when I use:
Quoted 8 lines
Quoted 1 line
cp -f *
/usr/local/www/imagesKind of. The * is expanded by the shell. The kernel limits the max length of program arguments, which is biting you here. In theory you could increase the
MAX_ARG_PAGES#define inlinux/binfmts.hand recompile. No guarantee that it won't have any bad side effects though. The default is rather low, it should be probably increased (I also regularly run into this)
Quoted 2 lines
I have been making that limit higher 5 years ago. Perhaps its time to up it for everyone?
Is this something that _must_ be set at compile time, or could it be made tuneable via /proc?
Shalon Wood
Hi!
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
Quoted 7 lines
I have a directory with 39,000 files in it, and I'm trying to use the cp command to copy them into another directory, and neither the cp or the mv command will work, they both same "argument list too long" when I use:
cp -f *
/usr/local/www/imagesKind of. The * is expanded by the shell. The kernel limits the max length of program arguments, which is biting you here. In theory you could increase the
MAX_ARG_PAGES#define inlinux/binfmts.hand recompile. No guarantee that it won't have any bad side effects though. The default is rather low, it should be probably increased (I also regularly run into this)
Quoted 5 lines
Quoted 2 lines
I have been making that limit higher 5 years ago. Perhaps its time to up it for everyone?
Is this something that _must_ be set at compile time, or could it be made tuneable via /proc?
Probably can be made tuneable, but I've nonot seen a patch...
Pavel
Signature · 2 lines
Casualities in World Trade Center: ~3k dead inside the building, cryptography in U.S.A. and free speech in Czech Republic.