Skip to contentSkip to conversation

kmemleak not play well with low memory situation

PATCH Superseded Qian Cai 3 participants 10 messages Dec 30, 2018 – Jan 8, 2019

Subsystems (3)

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Qian Cai

author
#0 lore raw

When in low memory situation with swapping, kmemleak quickly disable itself due to cannot allocate a kmemleak_object structure. If kmemleak could hold a bit longer, the system will trigger OOM soon to free up the memory. This is a bit tricky to solve because in __alloc_pages_slowpath() even though it has GFP_NOFAIL, it would fail allocation due to no GFP_DIRECT_RECLAIM,

/* Caller is not willing to reclaim, we can't balance anything */ if (!can_direct_reclaim)

	goto nopage;

if (WARN_ON_ONCE(!can_direct_reclaim))

	goto fail;

If adding GFP_DIRECT_RECLAIM to kmemleak_alloc(), it will trigger endless warnings in slab_pre_alloc_hook(),

might_sleep_if(gfpflags_allow_blocking(flags));

author
#1 lore raw

PATCH 1/1 kmemleak: survive in a low-memory situation

Kmemleak could quickly fail to allocate an object structure and then disable itself in a low-memory situation. For example, running a mmap() workload triggering swapping and OOM [1].

First, it unnecessarily attempt to allocate even though the tracking object is NULL in kmem_cache_alloc(). For example,

alloc_io
bio_alloc_bioset
mempool_alloc
mempool_alloc_slab
kmem_cache_alloc
slab_alloc_node
__slab_alloc <-- could return NULL
slab_post_alloc_hook
kmemleak_alloc_recursive

Second, kmemleak allocation could fail even though the trackig object is succeeded. Hence, it could still try to start a direct reclaim if it is not executed in an atomic context (spinlock, irq-handler etc), or a high-priority allocation in an atomic context as a last-ditch effort.

[1]
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/oom/oom01.c

---
 mm/kmemleak.c | 10 ++++++++++
 mm/slab.h     | 17 +++++++++--------
 2 files changed, 19 insertions(+), 8 deletions(-)
mm/kmemleak.c +10 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif
 	if (!object) {
 		pr_warn("Cannot allocate a kmemleak_object structure\n");
 		kmemleak_disable();
mm/slab.h +9 -8
index 4190c24ef0e9..51a9a942cc56 100644
@@ -435,15 +435,16 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
 {
 	size_t i;
 
-	flags &= gfp_allowed_mask;
-	for (i = 0; i < size; i++) {
-		void *object = p[i];
-
-		kmemleak_alloc_recursive(object, s->object_size, 1,
-					 s->flags, flags);
-		p[i] = kasan_slab_alloc(s, object, flags);
+	if (*p) {
+		flags &= gfp_allowed_mask;
+		for (i = 0; i < size; i++) {
+			void *object = p[i];
+
+			kmemleak_alloc_recursive(object, s->object_size, 1,
+						 s->flags, flags);
+			p[i] = kasan_slab_alloc(s, object, flags);
+		}
 	}
-
 	if (memcg_kmem_enabled())
 		memcg_kmem_put_cache(s);
 }
Signature · 1 line
2.17.2 (Apple Git-113)
maintainer
#2 lore raw

Re: [PATCH] kmemleak: survive in a low-memory situation

Hi Qian,

Quoted 7 lines · Qian Cai

Kmemleak could quickly fail to allocate an object structure and then disable itself in a low-memory situation. For example, running a mmap() workload triggering swapping and OOM [1].

First, it unnecessarily attempt to allocate even though the tracking object is NULL in kmem_cache_alloc(). For example,

Quoted 9 lines

alloc_io
bio_alloc_bioset
mempool_alloc
mempool_alloc_slab
kmem_cache_alloc
slab_alloc_node
__slab_alloc <-- could return NULL
slab_post_alloc_hook
kmemleak_alloc_recursive

kmemleak_alloc() only continues with the kmemleak_object allocation if the given pointer is not NULL.

Quoted 15 lines
mm/slab.h +0 -7
index 4190c24ef0e9..51a9a942cc56 100644
@@ -435,15 +435,16 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
 {
 	size_t i;
 
-	flags &= gfp_allowed_mask;
-	for (i = 0; i < size; i++) {
-		void *object = p[i];
-
-		kmemleak_alloc_recursive(object, s->object_size, 1,
-					 s->flags, flags);
-		p[i] = kasan_slab_alloc(s, object, flags);

mm/slab.h static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, · line 447

Quoted 10 lines
mm/slab.h +9 -0
+	if (*p) {
+		flags &= gfp_allowed_mask;
+		for (i = 0; i < size; i++) {
+			void *object = p[i];
+
+			kmemleak_alloc_recursive(object, s->object_size, 1,
+						 s->flags, flags);
+			p[i] = kasan_slab_alloc(s, object, flags);
+		}
 	}

This is not necessary for kmemleak.

Signature · 1 line
Catalin
author
#3 lore raw

PATCH v2 1/1 kmemleak: survive in a low-memory situation

Kmemleak could quickly fail to allocate an object structure and then disable itself in a low-memory situation. For example, running a mmap() workload triggering swapping and OOM [1].

Kmemleak allocation could fail even though the trackig object is succeeded. Hence, it could still try to start a direct reclaim if it is not executed in an atomic context (spinlock, irq-handler etc), or a high-priority allocation in an atomic context as a last-ditch effort.

[1]
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/oom/oom01.c

---

v2: remove the needless checking for NULL objects in slab_post_alloc_hook()
pointed out by Catalin.

 mm/kmemleak.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
mm/kmemleak.c +10 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif
 	if (!object) {
 		pr_warn("Cannot allocate a kmemleak_object structure\n");
 		kmemleak_disable();
Signature · 1 line
2.17.2 (Apple Git-113)
#4 lore raw

Re: [PATCH v2] kmemleak: survive in a low-memory situation

On Wed 02-01-19 13:06:19, Qian Cai wrote:
[...]

Quoted 8 lines
mm/kmemleak.c +0 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));

mm/kmemleak.c static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · line 588

Quoted 10 lines
mm/kmemleak.c +10 -0
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif

I do not get it. How can this possibly help when gfp_kmemleak_mask() adds __GFP_NOFAIL modifier to the given gfp mask? Or is this not the case anymore in some tree?

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#5 lore raw

Re: [PATCH v2] kmemleak: survive in a low-memory situation

Quoted 9 lines · Michal Hocko

On Wed 02-01-19 13:06:19, Qian Cai wrote:
[...]

Quoted 7 lines
mm/kmemleak.c +0 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
Quoted 15 lines

mm/kmemleak.c static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · line 588

Quoted 11 lines
mm/kmemleak.c +10 -0
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif

I do not get it. How can this possibly help when gfp_kmemleak_mask() adds __GFP_NOFAIL modifier to the given gfp mask? Or is this not the case anymore in some tree?

Well, __GFP_NOFAIL can still fail easily without __GFP_DIRECT_RECLAIM in a low-memory situation.

__alloc_pages_slowpath():

/* Caller is not willing to reclaim, we can't balance anything */ if (!can_direct_reclaim)

	goto nopage;

nopage:

/*
* All existing users of the __GFP_NOFAIL are blockable, so
* warn of any new users that actually require GFP_NOWAIT */
if (WARN_ON_ONCE(!can_direct_reclaim))

	goto fail;
#6 lore raw

Re: [PATCH v2] kmemleak: survive in a low-memory situation

Quoted 26 lines · Qian Cai
Quoted 9 lines · Michal Hocko

On Wed 02-01-19 13:06:19, Qian Cai wrote:
[...]

Quoted 7 lines
mm/kmemleak.c +0 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
Quoted 15 lines

mm/kmemleak.c static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · line 588

Quoted 11 lines
mm/kmemleak.c +10 -0
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif

I do not get it. How can this possibly help when gfp_kmemleak_mask() adds __GFP_NOFAIL modifier to the given gfp mask? Or is this not the case anymore in some tree?

Quoted 2 lines

Well, __GFP_NOFAIL can still fail easily without __GFP_DIRECT_RECLAIM in a low-memory situation.

OK, I guess I understand now. So the issue is that a (general) atomic allocation will provide its gfp mask down to kmemleak and you are trying/hoping that if the allocation is no from an atomic context then you can fortify it by using a sleepable allocation for the kmemleak metadata or giving it access to memory reserves for atomic allocations.

I think this is still fragile because most atomic allocations are for a good reason. As I've said earlier the current implementation which abuses __GFP_NOFAIL is fra from great and we have discussed some alternatives. Not sure whan came out of it.

I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.

Signature · 2 lines
Michal Hocko
SUSE Labs
maintainer
#7 lore raw

Re: [PATCH v2] kmemleak: survive in a low-memory situation

Quoted 20 lines · Michal Hocko
Quoted 20 lines
Quoted 20 lines · Qian Cai

[...]

Quoted 18 lines
mm/kmemleak.c +10 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif

[...]

Quoted 3 lines

I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.

This would be nice indeed and it was discussed last year. I just haven't got around to trying anything yet:

https://marc.info/?l=linux-mm&m=152812489819532

Signature · 1 line
Catalin
author
#8 lore raw

Re: [PATCH v2] kmemleak: survive in a low-memory situation

Quoted 22 lines · Catalin Marinas
Quoted 20 lines · Michal Hocko
Quoted 20 lines
Quoted 20 lines · Qian Cai

[...]

Quoted 18 lines
mm/kmemleak.c +10 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif

[...]

Quoted 9 lines
Quoted 3 lines

I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.

This would be nice indeed and it was discussed last year. I just haven't got around to trying anything yet:

https://marc.info/?l=linux-mm&m=152812489819532

It could be helpful to apply this 10-line patch first if has no fundamental issue, as it survives probably 50 times running LTP oom* workloads without a single kmemleak allocation failure.

Of course, if someone is going to embed kmemleak metadata into slab objects itself soon, this workaround is not needed.

author
#9 lore raw

Re: [PATCH v2] kmemleak: survive in a low-memory situation

Quoted 35 lines · Qian Cai
Quoted 22 lines · Catalin Marinas
Quoted 20 lines · Michal Hocko
Quoted 20 lines
Quoted 20 lines · Qian Cai

[...]

Quoted 18 lines
mm/kmemleak.c +10 -0
index f9d9dc250428..9e1aa3b7df75 100644
@@ -576,6 +576,16 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size,
 	struct rb_node **link, *rb_parent;
 
 	object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp));
+#ifdef CONFIG_PREEMPT_COUNT
+	if (!object) {
+		/* last-ditch effort in a low-memory situation */
+		if (irqs_disabled() || is_idle_task(current) || in_atomic())
+			gfp = GFP_ATOMIC;
+		else
+			gfp = gfp_kmemleak_mask(gfp) | __GFP_DIRECT_RECLAIM;
+		object = kmem_cache_alloc(object_cache, gfp);
+	}
+#endif

[...]

Quoted 9 lines
Quoted 3 lines

I will not object to this workaround but I strongly believe that kmemleak should rethink the metadata allocation strategy to be really robust.

This would be nice indeed and it was discussed last year. I just haven't got around to trying anything yet:

https://marc.info/?l=linux-mm&m=152812489819532

Quoted 7 lines

It could be helpful to apply this 10-line patch first if has no fundamental issue, as it survives probably 50 times running LTP oom* workloads without a single kmemleak allocation failure.

Of course, if someone is going to embed kmemleak metadata into slab objects itself soon, this workaround is not needed.

Well, it is really hard to tell even if someone get eventually redesign kmemleak to embed the metadata into slab objects alone would survive LTP oom* workloads, because it seems still use separate metadata for non-slab objects where kmemleak allocation could fail like it right now and disable itself again.