making kmalloc BUG() might not be a good idea
Subsystems (2)
I'm sort-of concerned about this change:
[PATCH] __kmalloc: Generate BUG if size requested is too large.
it opens a can of worms, and stuff that used to generate -ENOMEM kinds of failures will now BUG() the kernel.
Unless you're going to audit every user triggerable path for proper size limiting, I think we should revert this change.
Thanks.
Quoted 4 lines · David S. Miller
I'm sort-of concerned about this change:
[PATCH]
__kmalloc: Generate BUG if size requested is too large.
Quoted 10 lines
it opens a can of worms, and stuff that used to generate
-ENOMEMkinds of failures will nowBUG()the kernel.Unless you're going to audit every user triggerable path for proper size limiting, I think we should revert this change.
Thanks.
Making it WARN might be a good compromise.
Send instant messages to your online friends http://au.messenger.yahoo.com
From: Nick Piggin <[email protected]>
Date: Fri, 23 Sep 2005 16:30:33 +1000
Quoted 6 lines
Quoted 4 lines · David S. Miller
I'm sort-of concerned about this change:
[PATCH]
__kmalloc: Generate BUG if size requested is too large.
Quoted 8 lines
Quoted 6 lines
it opens a can of worms, and stuff that used to generate
-ENOMEMkinds of failures will nowBUG()the kernel.Unless you're going to audit every user triggerable path for proper size limiting, I think we should revert this change.
Making it WARN might be a good compromise.
It's a better, but it's still turning a harmless -ENOMEM into a DoS log file filler for the cases where a size limit check is missing and is user triggerable.
Another idea is to put it under CONFIG_DEBUG or something.
Hi,
Quoted 8 lines · Nick Piggin
Quoted 6 lines · David S. Miller
I'm sort-of concerned about this change:
[PATCH]
__kmalloc: Generate BUG if size requested is too large.it opens a can of worms, and stuff that used to generate
-ENOMEMkinds of failures will nowBUG()the kernel.Making it WARN might be a good compromise.
Which has the potential to spam the logs with a user triggerable event without even killing the responsible process.
Same problem, just worse.
I could live with a solution that enables it based on a config.
KERNEL_HACKING is no such config. That feature is almost always enabled, because MAGIC_SYSRQ depends on it and a significant amount of Linux-Admins like it for a "sync, remount ro and reboot" sequence. So you need a new one.
Regards
Ingo Oeser
Quoted 13 lines · Ingo Oeser
Hi,
Quoted 9 lines · Nick Piggin
Quoted 6 lines · David S. Miller
I'm sort-of concerned about this change:
[PATCH]
__kmalloc: Generate BUG if size requested is too large.it opens a can of worms, and stuff that used to generate
-ENOMEMkinds of failures will nowBUG()the kernel.
Quoted 7 lines
Quoted 1 line
Making it WARN might be a good compromise.
Which has the potential to spam the logs with a user triggerable event without even killing the responsible process.
Same problem, just worse.
As opposed to potentially taking the system down? I don't think so.
Quoted 2 lines
I could live with a solution that enables it based on a config.
Then you'll get people not enabling it on real workloads, or tuning it off if it bugs them. No, the point of having a WARN there is really for people like SGI to detect a few rare failure cases when they first boot up their 1024+ CPU systems. It is not going to spam anyone's logs (and if it does it *needs* fixing).
What you don't want is to kill the responsible process, because at that point they're deep in the kernel, probably holding other locks and resources.
Signature · 3 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
From: Nick Piggin <[email protected]>
Date: Fri, 23 Sep 2005 17:58:00 +1000
Quoted 5 lines
Then you'll get people not enabling it on real workloads, or tuning it off if it bugs them. No, the point of having a WARN there is really for people like SGI to detect a few rare failure cases when they first boot up their 1024+ CPU systems. It is not going to spam anyone's logs (and if it does it *needs* fixing).
SGI (and people "like" them) can't enable a debug option when bringing up new changes for the first time on that huge system? Why is this?
What in the world are all these CONFIG_*DEBUG* options for then? They are there for "I'm doing something radically new, or my new change isn't working, therefore I need more debugging than usual."
We want it to spam the logs, sure, during _development_. We don't want it on production systems where any kind of downtime is a very serious problem. Rate limited, maybe, but not for every call as that's simply asking for trouble.
This is why we have things like net_ratelimit() in the networking btw. It's there so you can't remotely spam someone's logs just becuase you figured out the "bug of the week" magic packet that erroneously generates a huge number of log messages.
If we know how to make certain classes of bugs non-lethal, we should do so because there will always be bugs. :-) This change makes previously non-lethal bugs potentially kill the machine.
Quoted 8 lines · David S. Miller
From: Nick Piggin <[email protected]>
Date: Fri, 23 Sep 2005 17:58:00 +1000If we know how to make certain classes of bugs non-lethal, we should do so because there will always be bugs. :-) This change makes previously non-lethal bugs potentially kill the machine.
Oh the BUG is bad, sure. I just thought WARN would be a better _compromise_ than BUG in that it will achieve the same result without takeing the machine down.
I think the CONFIG_DEBUG options are there for some major types of debugging that require significant infrastructure or can slow down the kernel quite a lot. With that said, I think there is an option somewhere to turn off all WARNs and remove strings from all BUGs.
Regarding proliferation of assertions and warnings everywhere - without any official standard, I think we're mostly being sensible with them (at least in the core code that I look at). A warn in kmalloc for this wouldn't be anything radical.
I don't much care for it, but I agree the BUG has to go.
Signature · 3 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Quoted 19 lines · Nick Piggin
Quoted 8 lines · David S. Miller
From: Nick Piggin <[email protected]>
Date: Fri, 23 Sep 2005 17:58:00 +1000If we know how to make certain classes of bugs non-lethal, we should do so because there will always be bugs. :-) This change makes previously non-lethal bugs potentially kill the machine.
Oh the BUG is bad, sure. I just thought WARN would be a better _compromise_ than BUG in that it will achieve the same result without takeing the machine down.
I think the
CONFIG_DEBUGoptions are there for some major types of debugging that require significant infrastructure or can slow down the kernel quite a lot. With that said, I think there is an option somewhere to turn off all WARNs and remove strings from all BUGs.
Quoted 7 lines
Regarding proliferation of assertions and warnings everywhere - without any official standard, I think we're mostly being sensible with them (at least in the core code that I look at). A warn in kmalloc for this wouldn't be anything radical.
I don't much care for it, but I agree the BUG has to go.
Nice to see: + revert-oversized-kmalloc-check.patch added to -mm tree --
Coywolf Qi Hunt
http://sosdg.org/~coywolf/
Make kzalloc a macro
How about this patch making kzalloc a macro?
---
Make kzalloc a macro and use __GFP_ZERO for zeroed slab allocations
kzalloc is right now a function call. The optimization that the kmalloc macro provides does not work for kzalloc invocations. kmalloc also determines the slab to use at compile time and fails the compilation if the size is too big. kzalloc cannot do not.
Moreover there is no kzalloc_node.
This patch adds the ability to the slab allocator to indicate that an entity should be zeroed by using __GFP_ZERO in the same way to the page allocator.
__GFP_ZERO may be specified when using any slab allocation operation.
Then kzalloc can be defined as a simple macro which will then perform all the compile time checks of kmalloc and also check the sizes.
include/linux/slab.h +1 -1
@@ -99,7 +99,7 @@ found:return __kmalloc (size, flags); } -extern void *kzalloc (size_t ,unsigned int __nocast ); +#define kzalloc (__size, __flags) kmalloc(__size, (__flags) | __GFP_ZERO)/** * kcalloc - allocate memory for an array. The memory is set to zero.
mm/slab.c +27 -34
107 lines not on this page · raw message
Re: Make kzalloc a macro
Next rev. There was an issue with kmalloc_node:
---
Make kzalloc a macro and use __GFP_ZERO for zeroed slab allocations
kzalloc is right now a function call. The optimization that the kmalloc macro provides does not work for kzalloc invocations. kmalloc also determines the slab to use at compile time and fails the compilation if the size is too big. kzalloc cannot do not.
Moreover there is no kzalloc_node.
This patch adds the ability to the slab allocator to indicate that an entity should be zeroed by using __GFP_ZERO in the same way to the page allocator.
__GFP_ZERO may be specified when using any slab allocation operation.
Then kzalloc can be defined as a simple macro which will then perform all the compile time checks of kmalloc and also check the sizes.
include/linux/slab.h +1 -1
@@ -99,7 +99,7 @@ found:return __kmalloc (size, flags); } -extern void *kzalloc (size_t ,unsigned int __nocast ); +#define kzalloc (__size, __flags) kmalloc(__size, (__flags) | __GFP_ZERO)/** * kcalloc - allocate memory for an array. The memory is set to zero.
mm/slab.c +23 -32
95 lines not on this page · raw message
Re: Make kzalloc a macro
Quoted 3 lines · Christoph Lameter
Make kzalloc a macro and use
__GFP_ZEROfor zeroed slab allocations
I'd question the usefulness of this. It adds more code to a fastpath (kmem_cache_alloc) so as to speed up a slowpath (kzalloc()) which is already slow due to its memset.
It makes my kernel a bit fatter too - 150-odd bytes of text for some reason.
Re: Make kzalloc a macro
Quoted 3 lines · Andrew Morton
I'd question the usefulness of this. It adds more code to a fastpath (
kmem_cache_alloc) so as to speed up a slowpath (kzalloc()) which is already slow due to its memset.
It tries to unify both and make usage consistent over the allocator functions in slab.c. kzalloc essentially vanishes.
Quoted 2 lines
It makes my kernel a bit fatter too - 150-odd bytes of text for some reason.
Yes the inline function doubles the code generated for obj_checkout because it occurs in __cache_alloc and __cache_alloc_node. __cache_alloc is also an inline and is expanded three times.
Removing the inline from __cache_alloc could reduced code size.
Re: Make kzalloc a macro
In reply to Christoph LameterQuoted 12 lines · Christoph Lameter
How about this patch making kzalloc a macro?
---
Make kzalloc a macro and use
__GFP_ZEROfor zeroed slab allocationskzalloc is right now a function call. The optimization that the kmalloc macro provides does not work for kzalloc invocations. kmalloc also determines the slab to use at compile time and fails the compilation if the size is too big. kzalloc cannot do not.
include/linux/slab.h found: · line 102
Quoted 2 lines
include/linux/slab.h +1
-1-extern void *kzalloc (size_t ,unsigned int __nocast ); +#define kzalloc (__size, __flags) kmalloc(__size, (__flags) | __GFP_ZERO)
Why macro and not an inline function?
mm/slab.c cache_alloc_debugcheck_after(kmem_cache_ · line 2516
Quoted 6 lines
mm/slab.c +6
-0+static inline void *obj_checkout (kmem_cache_t *cachep,unsigned int __nocast flags,void *objp) +{ +if (likely (objp)) { + objp =cache_alloc_debugcheck_after (cachep, flags, objp, +__builtin_return_address (0 )); +if (unlikely (flags & __GFP_ZERO))
Why unlikely?
mm/slab.c cache_alloc_debugcheck_after(kmem_cache_ · line 2522
Quoted 6 lines
mm/slab.c +6
-0+memset (objp,0 ,obj_reallen (cachep)); + else +prefetchw (objp ); + } +return objp; +}
--
vda