kmemleak not play well with low memory situation
Subsystems (3)
When in low memory situation with swapping, kmemleak quickly disable itself due to cannot allocate a kmemleak_object structure. If kmemleak could hold a bit longer, the system will trigger OOM soon to free up the memory. This is a bit tricky to solve because in __alloc_pages_slowpath() even though it has GFP_NOFAIL, it would fail allocation due to no GFP_DIRECT_RECLAIM,
/* Caller is not willing to reclaim, we can't balance anything */ if (!can_direct_reclaim)
goto nopage;
if (WARN_ON_ONCE(!can_direct_reclaim))
goto fail;
If adding GFP_DIRECT_RECLAIM to kmemleak_alloc(), it will trigger endless warnings in slab_pre_alloc_hook(),
might_sleep_if(gfpflags_allow_blocking(flags));
PATCH 1/1 kmemleak: survive in a low-memory situation
Kmemleak could quickly fail to allocate an object structure and then disable itself in a low-memory situation. For example, running a mmap() workload triggering swapping and OOM [1].
First, it unnecessarily attempt to allocate even though the tracking object is NULL in kmem_cache_alloc(). For example,
alloc_io
bio_alloc_bioset
mempool_alloc
mempool_alloc_slab
kmem_cache_alloc
slab_alloc_node
__slab_alloc <-- could return NULL
slab_post_alloc_hook
kmemleak_alloc_recursive
Second, kmemleak allocation could fail even though the trackig object is succeeded. Hence, it could still try to start a direct reclaim if it is not executed in an atomic context (spinlock, irq-handler etc), or a high-priority allocation in an atomic context as a last-ditch effort.
[1]
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/oom/oom01.c
--- mm/kmemleak.c | 10 ++++++++++ mm/slab.h | 17 +++++++++-------- 2 files changed, 19 insertions(+), 8 deletions(-)
mm/kmemleak.c +10 -0
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent; object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp)); +#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif if (!object) {pr_warn ("Cannot allocate a kmemleak_object structure\n" );kmemleak_disable ();
mm/slab.h +9 -8
@@ -435,15 +435,16 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, {size_t i; - flags &= gfp_allowed_mask; -for (i =0 ; i < size; i++) { -void *object = p[i]; - -kmemleak_alloc_recursive (object, s->object_size,1 , - s->flags, flags); - p[i] =kasan_slab_alloc (s, object, flags); +if (*p) { + flags &= gfp_allowed_mask; +for (i =0 ; i < size; i++) { +void *object = p[i]; + +kmemleak_alloc_recursive (object, s->object_size,1 , + s->flags, flags); + p[i] =kasan_slab_alloc (s, object, flags); + } } -if (memcg_kmem_enabled ())memcg_kmem_put_cache (s); }
Signature · 1 line
2.17.2 (Apple Git-113)
Catalin Marinas
maintainerRe: [PATCH] kmemleak: survive in a low-memory situation
Hi Qian,
Quoted 7 lines · Qian Cai
Kmemleak could quickly fail to allocate an object structure and then disable itself in a low-memory situation. For example, running a
mmap()workload triggering swapping and OOM [1].First, it unnecessarily attempt to allocate even though the tracking object is NULL in
kmem_cache_alloc(). For example,
Quoted 9 lines
alloc_io
bio_alloc_bioset
mempool_alloc
mempool_alloc_slab
kmem_cache_alloc
slab_alloc_node
__slab_alloc<-- could return NULL
slab_post_alloc_hook
kmemleak_alloc_recursive
kmemleak_alloc() only continues with the kmemleak_object allocation if the given pointer is not NULL.
Quoted 15 lines
mm/slab.h +0
-7@@ -435,15 +435,16 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, {size_t i; - flags &= gfp_allowed_mask; -for (i =0 ; i < size; i++) { -void *object = p[i]; - -kmemleak_alloc_recursive (object, s->object_size,1 , - s->flags, flags); - p[i] =kasan_slab_alloc (s, object, flags);
mm/slab.h static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, · line 447
Quoted 10 lines
mm/slab.h +9
-0+if (*p) { + flags &= gfp_allowed_mask; +for (i =0 ; i < size; i++) { +void *object = p[i]; + +kmemleak_alloc_recursive (object, s->object_size,1 , + s->flags, flags); + p[i] =kasan_slab_alloc (s, object, flags); + } }
This is not necessary for kmemleak.
Signature · 1 line
Catalin
PATCH v2 1/1 kmemleak: survive in a low-memory situation
Kmemleak could quickly fail to allocate an object structure and then disable itself in a low-memory situation. For example, running a mmap() workload triggering swapping and OOM [1].
Kmemleak allocation could fail even though the trackig object is succeeded. Hence, it could still try to start a direct reclaim if it is not executed in an atomic context (spinlock, irq-handler etc), or a high-priority allocation in an atomic context as a last-ditch effort.
[1]
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/oom/oom01.c
---
v2: remove the needless checking for NULL objects in slab_post_alloc_hook()
pointed out by Catalin.
mm/kmemleak.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
mm/kmemleak.c +10 -0
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent; object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp)); +#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif if (!object) {pr_warn ("Cannot allocate a kmemleak_object structure\n" );kmemleak_disable ();
Signature · 1 line
2.17.2 (Apple Git-113)
Re: [PATCH v2] kmemleak: survive in a low-memory situation
On Wed 02-01-19 13:06:19, Qian Cai wrote:
[...]
Quoted 8 lines
mm/kmemleak.c +0
-0@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent; object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp));
mm/kmemleak.c static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · line 588
Quoted 10 lines
mm/kmemleak.c +10
-0+#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif
I do not get it. How can this possibly help when gfp_kmemleak_mask() adds __GFP_NOFAIL modifier to the given gfp mask? Or is this not the case anymore in some tree?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2] kmemleak: survive in a low-memory situation
Quoted 9 lines · Michal Hocko
On Wed 02-01-19 13:06:19, Qian Cai wrote:
[...]Quoted 7 lines
mm/kmemleak.c +0
-0@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent;
Quoted 15 lines
mm/kmemleak.c static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · line 588
Quoted 11 lines
mm/kmemleak.c +10
-0object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp)); +#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif I do not get it. How can this possibly help when
gfp_kmemleak_mask()adds__GFP_NOFAILmodifier to the given gfp mask? Or is this not the case anymore in some tree?
Well, __GFP_NOFAIL can still fail easily without __GFP_DIRECT_RECLAIM in a low-memory situation.
__alloc_pages_slowpath():
/* Caller is not willing to reclaim, we can't balance anything */ if (!can_direct_reclaim)
goto nopage;
nopage:
/*
* All existing users of the __GFP_NOFAIL are blockable, so
* warn of any new users that actually require GFP_NOWAIT */
if (WARN_ON_ONCE(!can_direct_reclaim))
goto fail;
Re: [PATCH v2] kmemleak: survive in a low-memory situation
Quoted 26 lines · Qian Cai
Quoted 9 lines · Michal Hocko
On Wed 02-01-19 13:06:19, Qian Cai wrote:
[...]Quoted 7 lines
mm/kmemleak.c +0
-0@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent;Quoted 15 lines
mm/kmemleak.c static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · line 588
Quoted 11 lines
mm/kmemleak.c +10
-0object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp)); +#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif I do not get it. How can this possibly help when
gfp_kmemleak_mask()adds__GFP_NOFAILmodifier to the given gfp mask? Or is this not the case anymore in some tree?
Quoted 2 lines
Well,
__GFP_NOFAILcan still fail easily without__GFP_DIRECT_RECLAIMin a low-memory situation.
OK, I guess I understand now. So the issue is that a (general) atomic allocation will provide its gfp mask down to kmemleak and you are trying/hoping that if the allocation is no from an atomic context then you can fortify it by using a sleepable allocation for the kmemleak metadata or giving it access to memory reserves for atomic allocations.
I think this is still fragile because most atomic allocations are for a good reason. As I've said earlier the current implementation which abuses __GFP_NOFAIL is fra from great and we have discussed some alternatives. Not sure whan came out of it.
I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.
Signature · 2 lines
Michal Hocko SUSE Labs
Catalin Marinas
maintainerRe: [PATCH v2] kmemleak: survive in a low-memory situation
Quoted 20 lines · Michal Hocko
Quoted 20 lines
Quoted 20 lines · Qian Cai
[...]
Quoted 18 lines
mm/kmemleak.c +10
-0@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent; object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp)); +#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif
[...]
Quoted 3 lines
I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.
This would be nice indeed and it was discussed last year. I just haven't got around to trying anything yet:
https://marc.info/?l=linux-mm&m=152812489819532
Signature · 1 line
Catalin
Re: [PATCH v2] kmemleak: survive in a low-memory situation
Quoted 22 lines · Catalin Marinas
Quoted 20 lines · Michal Hocko
Quoted 20 lines
Quoted 20 lines · Qian Cai
[...]
Quoted 18 lines
mm/kmemleak.c +10
-0@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent; object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp)); +#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif [...]
Quoted 9 lines
Quoted 3 lines
I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.
This would be nice indeed and it was discussed last year. I just haven't got around to trying anything yet:
It could be helpful to apply this 10-line patch first if has no fundamental issue, as it survives probably 50 times running LTP oom* workloads without a single kmemleak allocation failure.
Of course, if someone is going to embed kmemleak metadata into slab objects itself soon, this workaround is not needed.
Re: [PATCH v2] kmemleak: survive in a low-memory situation
Quoted 35 lines · Qian Cai
Quoted 22 lines · Catalin Marinas
Quoted 20 lines · Michal Hocko
Quoted 20 lines
Quoted 20 lines · Qian Cai
[...]
Quoted 18 lines
mm/kmemleak.c +10
-0@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,struct rb_node **link, *rb_parent; object =kmem_cache_alloc (object_cache,gfp_kmemleak_mask (gfp)); +#ifdef CONFIG_PREEMPT_COUNT +if (!object) { +/* last-ditch effort in a low-memory situation */ +if (irqs_disabled () ||is_idle_task (current) ||in_atomic ()) + gfp =GFP_ATOMIC ; +else + gfp =gfp_kmemleak_mask (gfp) | __GFP_DIRECT_RECLAIM; + object =kmem_cache_alloc (object_cache, gfp); + } +#endif [...]
Quoted 9 lines
Quoted 3 lines
I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.
This would be nice indeed and it was discussed last year. I just haven't got around to trying anything yet:
Quoted 7 lines
It could be helpful to apply this 10-line patch first if has no fundamental issue, as it survives probably 50 times running LTP oom* workloads without a single kmemleak allocation failure.
Of course, if someone is going to embed kmemleak metadata into slab objects itself soon, this workaround is not needed.
Well, it is really hard to tell even if someone get eventually redesign kmemleak to embed the metadata into slab objects alone would survive LTP oom* workloads, because it seems still use separate metadata for non-slab objects where kmemleak allocation could fail like it right now and disable itself again.