PATCH: report NGROUPS_MAX via a sysctl (read-only)
Attached is a simple patch to expose NGROUPS_MAX via sysctl. Nothing fancy, just a read-only variable. glibc can use this to sysconf() the value properly, so apps will stop relying on NGROUPS_MAX as a real constant.
Is this the right path? Or should there be a sysconf-specific mechanism?
Signature · 4 lines
Tim Hockin Sun Microsystems, Linux Software Engineering [email protected] All opinions are my own, not Sun's
===== include/linux/sysctl.h 1.64 vs edited =====
include/linux/sysctl.h +1 -0
@@ -129,6 +129,7 @@KERN_HPPA_UNALIGNED =59 ,/* int: hppa unaligned-trap enable */ KERN_PRINTK_RATELIMIT =60 ,/* int: tune printk ratelimiting */ KERN_PRINTK_RATELIMIT_BURST =61 ,/* int: tune printk ratelimiting */ + KERN_NGROUPS_MAX=62, /* int: NGROUPS_MAX */ };
===== kernel/sysctl.c 1.61 vs edited =====
kernel/sysctl.c +11 -0
@@ -38,6 +38,7 @@#include <linux/security.h> #include <linux/initrd.h> #include <linux/times.h> +#include <linux/limits.h> #include <asm/uaccess.h> #ifdef CONFIG_ROOT_NFS
@@ -68,6 +69,8 @@static int maxolduid =65535 ;static int minolduid; +static int ngroups_max =NGROUPS_MAX ; +#ifdef CONFIG_KMOD extern char modprobe_path[];#endif
@@ -591,6 +594,14 @@ .data = &printk_ratelimit_burst, .maxlen = sizeof(int), .mode = 0644, + .proc_handler = &proc_dointvec, + }, + { + .ctl_name = KERN_NGROUPS_MAX, + .procname = "ngroups_max", + .data = &ngroups_max, + .maxlen = sizeof (int), + .mode = 0444, .proc_handler = &proc_dointvec, }, { .ctl_name = 0 }
Quoted 2 lines · Tim Hockin
Attached is a simple patch to expose
NGROUPS_MAXvia sysctl.
Why does userspace actually care? You try to do an oversized setgroups(), so you get an error?
And why does NGROUPS_MAX still exist, come to that? AFAICT the only thing it does is to prevent users from being able to allocate too much kernel memory??
Quoted 4 lines · Andrew Morton
Quoted 1 line
Attached is a simple patch to expose
NGROUPS_MAXvia sysctl.Why does userspace actually care? You try to do an oversized
setgroups(), so you get an error?
I am systematically tracking down apps that use it. glibc is almost free of it. sysconf() still uses it, but as long as the value compiled into glibc as NGROUPS_MAX is less-than-or-equal-to the current kernel's idea, it meets POSIX, right? If any one goes into their kernel source and lowers NGROUPS_MAX they might break things, but I guess that isn't too big of a worry. Some apps are still assuming that the value they get from sysconf() is the absolute max number of groups. Anyone with libc compiled against an older kernel will see 32, when they could have 64k.
Quoted 1 line
And why does
NGROUPS_MAXstill exist, come to that? AFAICT the only thing
Because Linus would not let me set it to INT_MAX. Something about "insanity" ;)
As long as we can convince userspace apps to not trust sysconf() or NGROUPS_MAX constants, I guess we don't *need* it exposed. I guess that's a veto, then?
Signature · 4 lines
Tim Hockin Sun Microsystems, Linux Software Engineering [email protected] All opinions are my own, not Sun's
Quoted 7 lines · Tim Hockin
Quoted 4 lines · Andrew Morton
Quoted 1 line
Attached is a simple patch to expose
NGROUPS_MAXvia sysctl.Why does userspace actually care? You try to do an oversized
setgroups(), so you get an error?
Quoted 8 lines
I am systematically tracking down apps that use it. glibc is almost free of it.
sysconf()still uses it, but as long as the value compiled into glibc asNGROUPS_MAXis less-than-or-equal-to the current kernel's idea, it meets POSIX, right? If any one goes into their kernel source and lowersNGROUPS_MAXthey might break things, but I guess that isn't too big of a worry. Some apps are still assuming that the value they get fromsysconf()is the absolute max number of groups. Anyone with libc compiled against an older kernel will see 32, when they could have 64k.
OK, well certainly fishing the number out of the currently-running kernel is the one sure way of getting it right.
Quoted 4 lines
Quoted 1 line
And why does
NGROUPS_MAXstill exist, come to that? AFAICT the only thingBecause Linus would not let me set it to
INT_MAX. Something about "insanity" ;)
Is 64k enough?
Quoted 12 lines · Andrew Morton
Quoted 11 lines
Quoted 2 lines
Why does userspace actually care? You try to do an oversized
setgroups(), so you get an error?I am systematically tracking down apps that use it. glibc is almost free of it.
sysconf()still uses it, but as long as the value compiled into glibc asNGROUPS_MAXis less-than-or-equal-to the current kernel's idea, it meets POSIX, right? If any one goes into their kernel source and lowersNGROUPS_MAXthey might break things, but I guess that isn't too big of a worry. Some apps are still assuming that the value they get fromsysconf()is the absolute max number of groups. Anyone with libc compiled against an older kernel will see 32, when they could have 64k.
Quoted 2 lines
OK, well certainly fishing the number out of the currently-running kernel is the one sure way of getting it right.
Well, really I don't see how apps would want to use it in any way that was correct. You can't use it as the size of an array. If it WERE defined as INT_MAX... :)
On the other hand, some obsessive-compulsive part of me says that a constant like that SHOULD be exposed. Which is why I asked about doing either a sys_sysconf() call to expose those, or adding something to sysfs/procfs/somethingfs to gather all the sysconf-mandated constants and maybe other kernel constants.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
And why does
NGROUPS_MAXstill exist, come to that? AFAICT the only thingBecause Linus would not let me set it to
INT_MAX. Something about "insanity" ;)Is 64k enough?
64k makes our users happy. It was a concession that had to be made. Now that the infrastructure for allocating, sorting, and searching is in place, any user can change NGROUPS_MAX to 128k or 256k or INT_MAX and recompile - no other work needed. That's far better than where we were.
Of course, if we DON'T expose things like NGROUPS_MAX, then userspace won't know if the user changes it to 256k. But then again, what would they be using it for that would not be wrong? I'm not sure it is for the kernel to say "you don't need to know this".
Quoted 2 lines · Tim Hockin
Well, really I don't see how apps would want to use it in any way that was correct.
The mere existence of the value means it can be used correctly in application code for sanity checking.
eg:
assert(list_length(group_list) < ngroups_max()); list_append(&group_list, group);
An application might also use it to automatically size data structure details, such as the parameters of a hash function.
h = hash_create(ngroups_max());
Returning INT_MAX for NGROUPS_MAX isn't wrong, but you then can't blame user space for making inefficient choices if the kernel limit is actually smaller.
Cheers,
Glen
Quoted 2 lines · Glen Turner
The mere existence of the value means it can be used correctly in application code for sanity checking.
Andrew seems to have picked it up for -mm. I sent a patch to glibc-bugs (is that the right place? any libc folk?) to use the new sysctl in sysconf(_SC_NGROUPS_MAX). So if/when all those releases converge, you should be able to get the kernel's idea of NGROUPS_MAX from sysconf(_SC_NGROUPS_MAX);
Quoted 3 lines
Returning
INT_MAXforNGROUPS_MAXisn't wrong, but you then can't blame user space for making inefficient choices if the kernel limit is actually smaller.
Well, if the kernel HAS no limit, then NGROUPS_MAX really is INT_MAX. There are not many legit uses I can think of for userspace to actually care about NGROUPS_MAX. Just the current number of groups, which is easily gotten via setgroups().
Now that the sysctl is in, it's a very tiny patch to make ngroups_max actually raisable/ by sysctl, though it is debatable whether it is useful AT ALL, and dubious whether it is safe to lower that value after any apps are running.
Signature · 4 lines
Tim Hockin [email protected] Soon anyone who's not on the World Wide Web will qualify for a government subsidy for the home-pageless.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 1 line · Tim Hockin
I sent a patch to glibc-bugs (is that the right place?
No. libc-alpha (at) sources.redhat.com.
- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
iD8DBQFAPE+k2ijCOnn/RHQRAsNoAJ9rzW4nh8e5t1b8pDu/Ad7coTl71QCgiLhT qYs6dwTrImTJOZc0JK88z/Q=
=8CR9
-----END PGP SIGNATURE-----