Skip to contentSkip to conversation

mm/page_alloc,slab: return NULL early from *_nolock() memory allocation APIs in NMI on UP

PATCH In review Harry Yoo (Oracle) 2 participants 5 messages Apr 27

Subsystems (4)
lore
Share & tools
lore

Harry Yoo (Oracle)

author maintainer
#0 lore raw

v2 cover mm/page_alloc,slab: return NULL early from *_nolock() memory allocation APIs in NMI on UP

Due to my mistake, V1 was sent twice w/o proper cover letter and
Cc: stable. Please ignore V1. Apologies for the noise.

Changes since V1:
- used b4 to send patch series (w/ a proper cover letter) instead of my broken git send-email script (Thanks Vlastimil)
- added Cc: stable to patches 1 and 2

On UP kernels (!CONFIG_SMP), spin_trylock() is a no-op that unconditionally succeeds even when the lock is already held. As a result, alloc_frozen_pages_nolock() and kmalloc_nolock() called from an NMI context can successfully re-acquire the lock that the page/slab allocators are already holding (no deadlock because it's trylock, but leads to e.g., allocating the same page/object twice and causing use-after-free).

It was discovered while testing the new kmalloc/kfree_nolock() test case in the slub_kunit test module with CONFIG_DEBUG_SPINLOCK=y on a UP kernel.

Patch 1 fixes alloc_frozen_pages_nolock() and
patch 2 fixes kmalloc_nolock().

Note: As pointed out by Vlastimil Babka [1], in theory a kprobe in a locked section could trigger the same issue. However, fixing that involves a non-trivial rework (e.g., inventing a new spinlock type) or introduces unnecessary overhead for all spinlocks on UP (e.g., let all spinlocks check locked status on UP).

Given that BPF tracing on UP is rare, and it's even more unlikely to trace a function called from the memory allocator within the locked section, this patch series addresses the issue only on NMI contexts (which is rare as well but now covered by the new test case).

[1] https://lore.kernel.org/linux-mm/[email protected]

Cc stable.vger.kernel.org

---
Harry Yoo (Oracle) (2):
mm/page_alloc: return NULL early from alloc_frozen_pages_nolock() in NMI on UP mm/slab: return NULL early from kmalloc_nolock() in NMI on UP

 mm/page_alloc.c | 5 +++++
 mm/slub.c       | 4 ++++
 2 files changed, 9 insertions(+)

---
base-commit: ba24da38a519dfcff8cce3f3f2726d7b159a4d75
change-id: 20260427-nolock-api-fix-bd056911e68e

Best regards,

Signature · 2 lines
Cheers,
Harry / Hyeonggon
author maintainer
#1 lore raw

PATCH v2 1/2 mm/page_alloc: return NULL early from alloc_frozen_pages_nolock() in NMI on UP

On UP kernels (!CONFIG_SMP), spin_trylock() is a no-op that unconditionally succeeds even when the lock is already held. As a result, alloc_frozen_pages_nolock() called from NMI context can re-enter rmqueue() and acquire the zone lock that the interrupted context is already holding, corrupting the freelists.

With CONFIG_DEBUG_SPINLOCK on UP, the following BUG is triggered with the slub_kunit test module:

  BUG: spinlock trylock failure on UP on CPU#0, kunit_try_catch/243
  [...]
  Call Trace:
   <NMI>
   dump_stack_lvl+0x3f/0x60
   do_raw_spin_trylock+0x41/0x50
   _raw_spin_trylock+0x24/0x50
   rmqueue.isra.0+0x2a9/0xa70
   get_page_from_freelist+0xeb/0x450
   alloc_frozen_pages_nolock_noprof+0x111/0x1e0
   allocate_slab+0x42a/0x500
   ___slab_alloc+0xa7/0x4c0
   kmalloc_nolock_noprof+0x164/0x310
   [...]
   </NMI>

Fix this by returning NULL early when invoked from NMI on a UP kernel.

Fixes d7242af86434 ("mm: Introduce alloc_frozen_pages_nolock()")
---
 mm/page_alloc.c | 5 +++++
 1 file changed, 5 insertions(+)
mm/page_alloc.c +5 -0
index 71859993dd54..23c7298d3be2 100644
@@ -7737,6 +7737,11 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned
 	 */
 	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
 		return NULL;
+
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return NULL;
+
 	if (!pcp_allowed_order(order))
 		return NULL;
 
Signature · 1 line
2.43.0
author maintainer
#2 lore raw

PATCH v2 2/2 mm/slab: return NULL early from kmalloc_nolock() in NMI on UP

In reply to Harry Yoo (Oracle)

On UP kernels (!CONFIG_SMP), spin_trylock() is a no-op that unconditionally succeeds even when the lock is already held. As a result, kmalloc_nolock() called from NMI context can re-enter the slab allocator and acquire n->list_lock that the interrupted context is already holding, corrupting slab state.

With CONFIG_DEBUG_SPINLOCK on UP, the following BUG is triggered with the slub_kunit test module:

  BUG: spinlock trylock failure on UP on CPU#0, kunit_try_catch/243
  [...]
  Call Trace:
   <NMI>
   dump_stack_lvl+0x3f/0x60
   do_raw_spin_trylock+0x41/0x50
   _raw_spin_trylock+0x24/0x50
   get_from_partial_node+0x120/0x4d0
   ___slab_alloc+0x8a/0x4c0
   kmalloc_nolock_noprof+0x164/0x310
   [...]
   </NMI>

Fix this by returning NULL early when invoked from NMI on a UP kernel.

Fixes af92793e52c3 ("slab: Introduce kmalloc_nolock() and kfree_nolock().")
---
 mm/slub.c | 4 ++++
 1 file changed, 4 insertions(+)
mm/slub.c +4 -0
index 92362eeb13e5..b4ec15df92f6 100644
@@ -5339,6 +5339,10 @@ void *kmalloc_nolock_noprof(size_t size, gfp_t gfp_flags, int node)
 	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
 		return NULL;
 
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return NULL;
+
 retry:
 	if (unlikely(size > KMALLOC_MAX_CACHE_SIZE))
 		return NULL;
Signature · 1 line
2.43.0
#3 lore raw
In reply to Harry Yoo (Oracle)
Quoted 16 lines · Harry Yoo (Oracle)

Due to my mistake, V1 was sent twice w/o proper cover letter and
Cc: stable. Please ignore V1. Apologies for the noise.

Changes since V1:
- used b4 to send patch series (w/ a proper cover letter) instead of my broken git send-email script (Thanks Vlastimil)
- added Cc: stable to patches 1 and 2

On UP kernels (!CONFIG_SMP), spin_trylock() is a no-op that unconditionally succeeds even when the lock is already held. As a result, alloc_frozen_pages_nolock() and kmalloc_nolock() called from an NMI context can successfully re-acquire the lock that the page/slab allocators are already holding (no deadlock because it's trylock, but leads to e.g., allocating the same page/object twice and causing use-after-free).

Quoted 6 lines

It was discovered while testing the new kmalloc/kfree_nolock() test case in the slub_kunit test module with CONFIG_DEBUG_SPINLOCK=y on a UP kernel.

Patch 1 fixes alloc_frozen_pages_nolock() and
patch 2 fixes kmalloc_nolock().

Thanks. Given the problem exposed is in a slab kunit test I think it's better to handle this in the slab tree. The page_alloc change is small and should not cause conflicts. So I've merged both in slab/for-next.

Quoted 31 lines

Note: As pointed out by Vlastimil Babka [1], in theory a kprobe in a locked section could trigger the same issue. However, fixing that involves a non-trivial rework (e.g., inventing a new spinlock type) or introduces unnecessary overhead for all spinlocks on UP (e.g., let all spinlocks check locked status on UP).

Given that BPF tracing on UP is rare, and it's even more unlikely to trace a function called from the memory allocator within the locked section, this patch series addresses the issue only on NMI contexts (which is rare as well but now covered by the new test case).

[1] https://lore.kernel.org/linux-mm/[email protected]

Cc stable.vger.kernel.org

---
Harry Yoo (Oracle) (2):
mm/page_alloc: return NULL early from alloc_frozen_pages_nolock() in NMI on UP mm/slab: return NULL early from kmalloc_nolock() in NMI on UP

 mm/page_alloc.c | 5 +++++
 mm/slub.c       | 4 ++++
 2 files changed, 9 insertions(+)

---
base-commit: ba24da38a519dfcff8cce3f3f2726d7b159a4d75
change-id: 20260427-nolock-api-fix-bd056911e68e

Best regards,

Signature · 2 lines
Cheers,
Harry / Hyeonggon
author maintainer
#4 lore raw
Quoted 21 lines · Vlastimil Babka (SUSE)
Quoted 20 lines · Harry Yoo (Oracle)

Due to my mistake, V1 was sent twice w/o proper cover letter and
Cc: stable. Please ignore V1. Apologies for the noise.

Changes since V1:
- used b4 to send patch series (w/ a proper cover letter) instead of my broken git send-email script (Thanks Vlastimil)
- added Cc: stable to patches 1 and 2

On UP kernels (!CONFIG_SMP), spin_trylock() is a no-op that unconditionally succeeds even when the lock is already held. As a result, alloc_frozen_pages_nolock() and kmalloc_nolock() called from an NMI context can successfully re-acquire the lock that the page/slab allocators are already holding (no deadlock because it's trylock, but leads to e.g., allocating the same page/object twice and causing use-after-free).

It was discovered while testing the new kmalloc/kfree_nolock() test case in the slub_kunit test module with CONFIG_DEBUG_SPINLOCK=y on a UP kernel.

Quoted 5 lines
Quoted 2 lines

Patch 1 fixes alloc_frozen_pages_nolock() and
patch 2 fixes kmalloc_nolock().

Thanks. Given the problem exposed is in a slab kunit test I think it's better to handle this in the slab tree.

Ack, I'm fine either way.

Quoted 2 lines

The page_alloc change is small and
should not cause conflicts.

Right.

Quoted 1 line

So I've merged both in slab/for-next.

Thanks! (I see it's been added to slab/for-next-fixes)

Signature · 2 lines
Cheers,
Harry / Hyeonggon