mm: handle potential NULL return from anon_vma_name_reuse()
Subsystems (4)
PATCH 1/1 mm: handle potential NULL return from anon_vma_name_reuse()
From: Ye Liu <[email protected]>
The anon_vma_name_reuse() function may return NULL if memory allocation fails in anon_vma_name_alloc(). Currently, callers dup_anon_vma_name() and replace_anon_vma_name() do not check the return value, which could lead to NULL pointer dereferences.
This patch adds proper error handling:
- In dup_anon_vma_name(), if anon_vma_name_reuse() returns NULL, emit a warning via WARN_ON_ONCE(1) since this is an unexpected condition.
- In replace_anon_vma_name(), return -ENOMEM to propagate the allocation failure to the caller.
These changes improve robustness against memory allocation failures.
--- include/linux/mm_inline.h | 12 +++++++++--- mm/madvise.c | 7 ++++++- 2 files changed, 15 insertions(+), 4 deletions(-)
include/linux/mm_inline.h +9 -3
@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, struct vm_area_struct *new_vma) {struct anon_vma_name *anon_name =anon_vma_name (orig_vma); - -if (anon_name) - new_vma->anon_name =anon_vma_name_reuse (anon_name); +struct anon_vma_name *new_name; + +if (anon_name) { + new_name =anon_vma_name_reuse (anon_name); +if (new_name) + new_vma->anon_name = new_name; +else +WARN_ON_ONCE (1 ); + } }static inline void free_anon_vma_name (struct vm_area_struct *vma)
mm/madvise.c +6 -1
@@ -118,6 +118,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *anon_name) {struct anon_vma_name *orig_name =anon_vma_name (vma); +struct anon_vma_name *new_name;if (!anon_name) { vma->anon_name =NULL ;
@@ -128,7 +129,11 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,if (anon_vma_name_eq (orig_name, anon_name))return 0 ; - vma->anon_name =anon_vma_name_reuse (anon_name); + new_name =anon_vma_name_reuse (anon_name); +if (!new_name) +return -ENOMEM ; + + vma->anon_name = new_name;anon_vma_name_put (orig_name);return 0 ;
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerQuoted 6 lines · Ye Liu
From: Ye Liu <[email protected]>
The
anon_vma_name_reuse()function may return NULL if memory allocation fails inanon_vma_name_alloc(). Currently, callersdup_anon_vma_name()andreplace_anon_vma_name()do not check the return value, which could lead to NULL pointer dereferences.
When, where?
If we set vma->anon_name = NULL, it just looks like there is no anon VMA, or what am I missing?
Quoted 24 lines
This patch adds proper error handling:
- Indup_anon_vma_name(), ifanon_vma_name_reuse()returns NULL, emit a warning viaWARN_ON_ONCE(1)since this is an unexpected condition.
- Inreplace_anon_vma_name(), return-ENOMEMto propagate the allocation failure to the caller.These changes improve robustness against memory allocation failures.
Signed-off-by Ye Liu <[email protected]>--- include/linux/mm_inline.h | 12 +++++++++--- mm/madvise.c | 7 ++++++- 2 files changed, 15 insertions(+), 4 deletions(-)include/linux/mm_inline.h +0
-1@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, struct vm_area_struct *new_vma) {struct anon_vma_name *anon_name =anon_vma_name (orig_vma); -
include/linux/mm_inline.h static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, · line 431
Quoted 10 lines
include/linux/mm_inline.h +8
-2-if (anon_name) - new_vma->anon_name =anon_vma_name_reuse (anon_name); +struct anon_vma_name *new_name; + +if (anon_name) { + new_name =anon_vma_name_reuse (anon_name); +if (new_name) + new_vma->anon_name = new_name; +else +WARN_ON_ONCE (1 );
Triggering a warning when an allocation fails? Certainly not.
Signature · 3 lines
Cheers, David
Lorenzo Stoakes
maintainerQuoted 41 lines · David Hildenbrand (Arm)
Quoted 6 lines · Ye Liu
From: Ye Liu <[email protected]>
The
anon_vma_name_reuse()function may return NULL if memory allocation fails inanon_vma_name_alloc(). Currently, callersdup_anon_vma_name()andreplace_anon_vma_name()do not check the return value, which could lead to NULL pointer dereferences.When, where?
If we set
vma->anon_name= NULL, it just looks like there is no anon VMA, or what am I missing?Quoted 28 lines
This patch adds proper error handling:
- Indup_anon_vma_name(), ifanon_vma_name_reuse()returns NULL, emit a warning viaWARN_ON_ONCE(1)since this is an unexpected condition.
- Inreplace_anon_vma_name(), return-ENOMEMto propagate the allocation failure to the caller.These changes improve robustness against memory allocation failures.
Signed-off-by Ye Liu <[email protected]>--- include/linux/mm_inline.h | 12 +++++++++--- mm/madvise.c | 7 ++++++- 2 files changed, 15 insertions(+), 4 deletions(-)include/linux/mm_inline.h +2
-3@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, struct vm_area_struct *new_vma) {struct anon_vma_name *anon_name =anon_vma_name (orig_vma); - -if (anon_name) - new_vma->anon_name =anon_vma_name_reuse (anon_name); +struct anon_vma_name *new_name; +
Quoted 8 lines
include/linux/mm_inline.h static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, · line 431
Quoted 6 lines
include/linux/mm_inline.h +6
-0+if (anon_name) { + new_name =anon_vma_name_reuse (anon_name); +if (new_name) + new_vma->anon_name = new_name; +else +WARN_ON_ONCE (1 );Triggering a warning when an allocation fails? Certainly not.
It's pleasing to reply independently and see that we've reached the same conclusions here :P
I mean, yeah agreed obviously!
Quoted 5 lines
--
Cheers,David
Cheers, Lorenzo
NAK, expected allocation failures (even if practically impossible) should not cause arbitrary kernel warnings.
But also this is very silly, you would need INT_MAX references on the anon_name to happen first...
Quoted 6 lines · Ye Liu
From: Ye Liu <[email protected]>
The
anon_vma_name_reuse()function may return NULL if memory allocation fails inanon_vma_name_alloc(). Currently, callersdup_anon_vma_name()andreplace_anon_vma_name()do not check the return value, which could lead to NULL pointer dereferences.
We assign NULL if the allocation failed. And every code path understands vma->anon_vma_name being NULL to be a valid situation of that VMA not having a name.
In the case you're failing an allocation that small, the system is under extreme memory pressure.
Not propagating an anon_vma_name, which is cosmetic, is totally fine under those circumstances.
But also, you'd require the anon_vma_name to be saturated at REFCOUNT_MAX == INT_MAX.
So this is just silly.
Quoted 4 lines
This patch adds proper error handling:
- Indup_anon_vma_name(), ifanon_vma_name_reuse()returns NULL, emit a warning viaWARN_ON_ONCE(1)since this is an unexpected condition.
It's not? Your whole thesis is that allocation failures can happen here and cause a problem, so you can't simultaneously say it's 'unexpected'?
Quoted 4 lines
- In
replace_anon_vma_name(), return-ENOMEMto propagate the allocation failure to the caller.These changes improve robustness against memory allocation failures.
No, they add a bunch of noise, then make an allocation failure into an arbitrary, new kernel warning.
Quoted 17 lines
Signed-off-by Ye Liu <[email protected]>--- include/linux/mm_inline.h | 12 +++++++++--- mm/madvise.c | 7 ++++++- 2 files changed, 15 insertions(+), 4 deletions(-)include/linux/mm_inline.h +0
-2@@ -421,9 +421,15 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, struct vm_area_struct *new_vma) {struct anon_vma_name *anon_name =anon_vma_name (orig_vma); - -if (anon_name)
include/linux/mm_inline.h static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, · line 432
Quoted 10 lines
include/linux/mm_inline.h +9
-1- new_vma->anon_name =anon_vma_name_reuse (anon_name); +struct anon_vma_name *new_name; + +if (anon_name) { + new_name =anon_vma_name_reuse (anon_name); +if (new_name) + new_vma->anon_name = new_name; +else +WARN_ON_ONCE (1 ); + }
This is horrible code, but as I said above, this is not only wrong it's actively bad - you're causing an allocation failure to result in a kernel warning.
We don't mind anon_vma_name not being propagated after billions of references and under maximally extreme memory pressure/fatal signal propagation.
Quoted 16 lines
include/linux/mm_inline.h +0
-0}static inline void free_anon_vma_name (struct vm_area_struct *vma)mm/madvise.c +1
-0@@ -118,6 +118,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *anon_name) {struct anon_vma_name *orig_name =anon_vma_name (vma); +struct anon_vma_name *new_name;if (!anon_name) { vma->anon_name =NULL ;
mm/madvise.c static int replace_anon_vma_name(struct vm_area_struct *vma, · line 136
Quoted 9 lines
mm/madvise.c +5
-1@@ -128,7 +129,11 @@ static int replace_anon_vma_name(struct vm_area_struct *vma,if (anon_vma_name_eq (orig_name, anon_name))return 0 ; - vma->anon_name =anon_vma_name_reuse (anon_name); + new_name =anon_vma_name_reuse (anon_name); +if (!new_name) +return -ENOMEM ; + + vma->anon_name = new_name;
This is pointless noise. Billions of references, extreme memory pressure (practically impossible anyway).
Quoted 6 lines
mm/madvise.c +0
-1anon_vma_name_put (orig_name);return 0 ; --2.43.0
Thanks, Lorenzo
Quoted 2 lines · Lorenzo Stoakes
NAK, expected allocation failures (even if practically impossible) should not cause arbitrary kernel warnings.
Thank you both for the thorough review and explanation.
You are absolutely right. I mistakenly assumed that leaving vma->anon_name as NULL could lead to a NULL pointer dereference in some path, but as you pointed out, the code is designed to treat NULL as "no name", which is perfectly valid.
I also acknowledge that adding a WARN_ON_ONCE in an allocation failure path is incorrect and harmful under memory pressure.
Please disregard this patch. I've learned a valuable lesson about error handling in cosmetic features.
Thank you again for the guidance.
Signature · 2 lines
Thanks, Ye Liu