Skip to contentSkip to conversation

mm: handle potential NULL return from anon_vma_name_reuse()

PATCH NAK Ye Liu 3 participants 5 messages Apr 21

Subsystems (4)
lore
Share & tools
lore

Ye Liu

author
#0 lore raw

PATCH 1/1 mm: handle potential NULL return from anon_vma_name_reuse()

From: Ye Liu <[email protected]>

The anon_vma_name_reuse() function may return NULL if memory allocation fails in anon_vma_name_alloc(). Currently, callers dup_anon_vma_name() and replace_anon_vma_name() do not check the return value, which could lead to NULL pointer dereferences.

This patch adds proper error handling:
- In dup_anon_vma_name(), if anon_vma_name_reuse() returns NULL, emit a warning via WARN_ON_ONCE(1) since this is an unexpected condition.
- In replace_anon_vma_name(), return -ENOMEM to propagate the allocation failure to the caller.

These changes improve robustness against memory allocation failures.

---
 include/linux/mm_inline.h | 12 +++++++++---
 mm/madvise.c              |  7 ++++++-
 2 files changed, 15 insertions(+), 4 deletions(-)
include/linux/mm_inline.h +9 -3
index a171070e15f0..9bbaf8287806 100644
@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma,
 				     struct vm_area_struct *new_vma)
 {
 	struct anon_vma_name *anon_name = anon_vma_name(orig_vma);
-
-	if (anon_name)
-		new_vma->anon_name = anon_vma_name_reuse(anon_name);
+	struct anon_vma_name *new_name;
+
+	if (anon_name) {
+		new_name = anon_vma_name_reuse(anon_name);
+		if (new_name)
+			new_vma->anon_name = new_name;
+		else
+			WARN_ON_ONCE(1);
+	}
 }
 
 static inline void free_anon_vma_name(struct vm_area_struct *vma)
mm/madvise.c +6 -1
index 69708e953cf5..ccb937a37e70 100644
@@ -118,6 +118,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,
 				 struct anon_vma_name *anon_name)
 {
 	struct anon_vma_name *orig_name = anon_vma_name(vma);
+	struct anon_vma_name *new_name;
 
 	if (!anon_name) {
 		vma->anon_name = NULL;
@@ -128,7 +129,11 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,
 	if (anon_vma_name_eq(orig_name, anon_name))
 		return 0;
 
-	vma->anon_name = anon_vma_name_reuse(anon_name);
+	new_name = anon_vma_name_reuse(anon_name);
+	if (!new_name)
+		return -ENOMEM;
+
+	vma->anon_name = new_name;
 	anon_vma_name_put(orig_name);
 
 	return 0;
Signature · 1 line
2.43.0
#1 lore raw
Quoted 6 lines · Ye Liu

From: Ye Liu <[email protected]>

The anon_vma_name_reuse() function may return NULL if memory allocation fails in anon_vma_name_alloc(). Currently, callers dup_anon_vma_name() and replace_anon_vma_name() do not check the return value, which could lead to NULL pointer dereferences.

When, where?

If we set vma->anon_name = NULL, it just looks like there is no anon VMA, or what am I missing?

Quoted 24 lines

This patch adds proper error handling:
- In dup_anon_vma_name(), if anon_vma_name_reuse() returns NULL, emit a warning via WARN_ON_ONCE(1) since this is an unexpected condition.
- In replace_anon_vma_name(), return -ENOMEM to propagate the allocation failure to the caller.

These changes improve robustness against memory allocation failures.

---
 include/linux/mm_inline.h | 12 +++++++++---
 mm/madvise.c              |  7 ++++++-
 2 files changed, 15 insertions(+), 4 deletions(-)
include/linux/mm_inline.h +0 -1
index a171070e15f0..9bbaf8287806 100644
@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma,
 				     struct vm_area_struct *new_vma)
 {
 	struct anon_vma_name *anon_name = anon_vma_name(orig_vma);
-

include/linux/mm_inline.h static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, · line 431

Quoted 10 lines
include/linux/mm_inline.h +8 -2
-	if (anon_name)
-		new_vma->anon_name = anon_vma_name_reuse(anon_name);
+	struct anon_vma_name *new_name;
+
+	if (anon_name) {
+		new_name = anon_vma_name_reuse(anon_name);
+		if (new_name)
+			new_vma->anon_name = new_name;
+		else
+			WARN_ON_ONCE(1);

Triggering a warning when an allocation fails? Certainly not.

Signature · 3 lines
Cheers,

David
maintainer
#2 lore raw
Quoted 41 lines · David Hildenbrand (Arm)
Quoted 6 lines · Ye Liu

From: Ye Liu <[email protected]>

The anon_vma_name_reuse() function may return NULL if memory allocation fails in anon_vma_name_alloc(). Currently, callers dup_anon_vma_name() and replace_anon_vma_name() do not check the return value, which could lead to NULL pointer dereferences.

When, where?

If we set vma->anon_name = NULL, it just looks like there is no anon VMA, or what am I missing?

Quoted 28 lines

This patch adds proper error handling:
- In dup_anon_vma_name(), if anon_vma_name_reuse() returns NULL, emit a warning via WARN_ON_ONCE(1) since this is an unexpected condition.
- In replace_anon_vma_name(), return -ENOMEM to propagate the allocation failure to the caller.

These changes improve robustness against memory allocation failures.

---
 include/linux/mm_inline.h | 12 +++++++++---
 mm/madvise.c              |  7 ++++++-
 2 files changed, 15 insertions(+), 4 deletions(-)
include/linux/mm_inline.h +2 -3
index a171070e15f0..9bbaf8287806 100644
@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma,
 				     struct vm_area_struct *new_vma)
 {
 	struct anon_vma_name *anon_name = anon_vma_name(orig_vma);
-
-	if (anon_name)
-		new_vma->anon_name = anon_vma_name_reuse(anon_name);
+	struct anon_vma_name *new_name;
+
Quoted 8 lines

include/linux/mm_inline.h static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, · line 431

Quoted 6 lines
include/linux/mm_inline.h +6 -0
+	if (anon_name) {
+		new_name = anon_vma_name_reuse(anon_name);
+		if (new_name)
+			new_vma->anon_name = new_name;
+		else
+			WARN_ON_ONCE(1);

Triggering a warning when an allocation fails? Certainly not.

It's pleasing to reply independently and see that we've reached the same conclusions here :P

I mean, yeah agreed obviously!

Quoted 5 lines

--
Cheers,

David

Cheers, Lorenzo

maintainer NAK
#3 lore raw
In reply to Ye Liu

NAK, expected allocation failures (even if practically impossible) should not cause arbitrary kernel warnings.

But also this is very silly, you would need INT_MAX references on the anon_name to happen first...

Quoted 6 lines · Ye Liu

From: Ye Liu <[email protected]>

The anon_vma_name_reuse() function may return NULL if memory allocation fails in anon_vma_name_alloc(). Currently, callers dup_anon_vma_name() and replace_anon_vma_name() do not check the return value, which could lead to NULL pointer dereferences.

We assign NULL if the allocation failed. And every code path understands vma->anon_vma_name being NULL to be a valid situation of that VMA not having a name.

In the case you're failing an allocation that small, the system is under extreme memory pressure.

Not propagating an anon_vma_name, which is cosmetic, is totally fine under those circumstances.

But also, you'd require the anon_vma_name to be saturated at REFCOUNT_MAX == INT_MAX.

So this is just silly.

Quoted 4 lines

This patch adds proper error handling:
- In dup_anon_vma_name(), if anon_vma_name_reuse() returns NULL, emit a warning via WARN_ON_ONCE(1) since this is an unexpected condition.

It's not? Your whole thesis is that allocation failures can happen here and cause a problem, so you can't simultaneously say it's 'unexpected'?

Quoted 4 lines

- In replace_anon_vma_name(), return -ENOMEM to propagate the allocation failure to the caller.

These changes improve robustness against memory allocation failures.

No, they add a bunch of noise, then make an allocation failure into an arbitrary, new kernel warning.

Quoted 17 lines
---
 include/linux/mm_inline.h | 12 +++++++++---
 mm/madvise.c              |  7 ++++++-
 2 files changed, 15 insertions(+), 4 deletions(-)
include/linux/mm_inline.h +0 -2
index a171070e15f0..9bbaf8287806 100644
@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma,
 				     struct vm_area_struct *new_vma)
 {
 	struct anon_vma_name *anon_name = anon_vma_name(orig_vma);
-
-	if (anon_name)

include/linux/mm_inline.h static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, · line 432

Quoted 10 lines
include/linux/mm_inline.h +9 -1
-		new_vma->anon_name = anon_vma_name_reuse(anon_name);
+	struct anon_vma_name *new_name;
+
+	if (anon_name) {
+		new_name = anon_vma_name_reuse(anon_name);
+		if (new_name)
+			new_vma->anon_name = new_name;
+		else
+			WARN_ON_ONCE(1);
+	}

This is horrible code, but as I said above, this is not only wrong it's actively bad - you're causing an allocation failure to result in a kernel warning.

We don't mind anon_vma_name not being propagated after billions of references and under maximally extreme memory pressure/fatal signal propagation.

Quoted 16 lines
include/linux/mm_inline.h +0 -0
 }
 
 static inline void free_anon_vma_name(struct vm_area_struct *vma)
mm/madvise.c +1 -0
index 69708e953cf5..ccb937a37e70 100644
@@ -118,6 +118,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,
 				 struct anon_vma_name *anon_name)
 {
 	struct anon_vma_name *orig_name = anon_vma_name(vma);
+	struct anon_vma_name *new_name;
 
 	if (!anon_name) {
 		vma->anon_name = NULL;

mm/madvise.c static int replace_anon_vma_name(struct vm_area_struct *vma, · line 136

Quoted 9 lines
mm/madvise.c +5 -1
@@ -128,7 +129,11 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,
 	if (anon_vma_name_eq(orig_name, anon_name))
 		return 0;
 
-	vma->anon_name = anon_vma_name_reuse(anon_name);
+	new_name = anon_vma_name_reuse(anon_name);
+	if (!new_name)
+		return -ENOMEM;
+
+	vma->anon_name = new_name;

This is pointless noise. Billions of references, extreme memory pressure (practically impossible anyway).

Quoted 6 lines
mm/madvise.c +0 -1
 	anon_vma_name_put(orig_name);
 
 	return 0;
--

2.43.0

Thanks, Lorenzo

author
#4 lore raw
Quoted 2 lines · Lorenzo Stoakes

NAK, expected allocation failures (even if practically impossible) should not cause arbitrary kernel warnings.

Thank you both for the thorough review and explanation.

You are absolutely right. I mistakenly assumed that leaving vma->anon_name as NULL could lead to a NULL pointer dereference in some path, but as you pointed out, the code is designed to treat NULL as "no name", which is perfectly valid.

I also acknowledge that adding a WARN_ON_ONCE in an allocation failure path is incorrect and harmful under memory pressure.

Please disregard this patch. I've learned a valuable lesson about error handling in cosmetic features.

Thank you again for the guidance.

Signature · 2 lines
Thanks,
Ye Liu