memcg: skip cgroup_file_notify if spinning is not allowed

PATCH Applied Shakeel Butt 2 participants 9 messages Sep 22, 2025 – Sep 23, 2025

Subsystems (4)

Memory management BPF cgroup memcg

lore

Share & tools

lore

Shakeel Butt

author maintainer

2025-09-22 22:02 UTC #0 lore raw

PATCH v2 1/1 memcg: skip cgroup_file_notify if spinning is not allowed

Generally memcg charging is allowed from all the contexts including NMI where even spinning on spinlock can cause locking issues. However one call chain was missed during the addition of memcg charging from any context support. That is try_charge_memcg() -> memcg_memory_event() -> cgroup_file_notify().

The possible function call tree under cgroup_file_notify() can acquire many different spin locks in spinning mode. Some of them are cgroup_file_kn_lock, kernfs_notify_lock, pool_workqeue's lock. So, let's just skip cgroup_file_notify() from memcg charging if the context does not allow spinning.

Alternative approach was also explored where instead of skipping cgroup_file_notify(), we defer the memcg event processing to irq_work [1]. However it adds complexity and it was decided to keep things simple until we need more memcg events with !allow_spinning requirement.

Link https://lore.kernel.org/all/5qi2llyzf7gklncflo6gxoozljbm4h3tpnuv4u4ej4ztysvi6f@x44v7nz2wdzd/ [1] open here

Signed-off-by Shakeel Butt <[email protected]>

Acked-by Michal Hocko <[email protected]>

---
Changes since v1:
- Add warning if !allow_spinning is used with memcg events other than MEMCG_MAX (requested by Roman)

 include/linux/memcontrol.h | 26 +++++++++++++++++++-------
 mm/memcontrol.c            |  7 ++++---
 2 files changed, 23 insertions(+), 10 deletions(-)

include/linux/memcontrol.h +19 -7

index 16fe0306e50e..873e510d6f8d 100644
@@ -1001,22 +1001,28 @@ static inline void count_memcg_event_mm(struct mm_struct *mm,
 	count_memcg_events_mm(mm, idx, 1);
 }
 
-static inline void memcg_memory_event(struct mem_cgroup *memcg,
-				      enum memcg_memory_event event)
+static inline void __memcg_memory_event(struct mem_cgroup *memcg,
+					enum memcg_memory_event event,
+					bool allow_spinning)
 {
 	bool swap_event = event == MEMCG_SWAP_HIGH || event == MEMCG_SWAP_MAX ||
 			  event == MEMCG_SWAP_FAIL;
 
+	/* For now only MEMCG_MAX can happen with !allow_spinning context. */
+	VM_WARN_ON_ONCE(!allow_spinning && event != MEMCG_MAX);
+
 	atomic_long_inc(&memcg->memory_events_local[event]);
-	if (!swap_event)
+	if (!swap_event && allow_spinning)
 		cgroup_file_notify(&memcg->events_local_file);
 
 	do {
 		atomic_long_inc(&memcg->memory_events[event]);
-		if (swap_event)
-			cgroup_file_notify(&memcg->swap_events_file);
-		else
-			cgroup_file_notify(&memcg->events_file);
+		if (allow_spinning) {
+			if (swap_event)
+				cgroup_file_notify(&memcg->swap_events_file);
+			else
+				cgroup_file_notify(&memcg->events_file);
+		}
 
 		if (!cgroup_subsys_on_dfl(memory_cgrp_subsys))
 			break;

@@ -1026,6 +1032,12 @@ static inline void memcg_memory_event(struct mem_cgroup *memcg,
 		 !mem_cgroup_is_root(memcg));
 }
 
+static inline void memcg_memory_event(struct mem_cgroup *memcg,
+				      enum memcg_memory_event event)
+{
+	__memcg_memory_event(memcg, event, true);
+}
+
 static inline void memcg_memory_event_mm(struct mm_struct *mm,
 					 enum memcg_memory_event event)
 {

mm/memcontrol.c +4 -3

index e090f29eb03b..4deda33625f4 100644
@@ -2307,12 +2307,13 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
 	bool drained = false;
 	bool raised_max_event = false;
 	unsigned long pflags;
+	bool allow_spinning = gfpflags_allow_spinning(gfp_mask);
 
 retry:
 	if (consume_stock(memcg, nr_pages))
 		return 0;
 
-	if (!gfpflags_allow_spinning(gfp_mask))
+	if (!allow_spinning)
 		/* Avoid the refill and flush of the older stock */
 		batch = nr_pages;

@@ -2348,7 +2349,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
 	if (!gfpflags_allow_blocking(gfp_mask))
 		goto nomem;
 
-	memcg_memory_event(mem_over_limit, MEMCG_MAX);
+	__memcg_memory_event(mem_over_limit, MEMCG_MAX, allow_spinning);
 	raised_max_event = true;
 
 	psi_memstall_enter(&pflags);

@@ -2415,7 +2416,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
 	 * a MEMCG_MAX event.
 	 */
 	if (!raised_max_event)
-		memcg_memory_event(mem_over_limit, MEMCG_MAX);
+		__memcg_memory_event(mem_over_limit, MEMCG_MAX, allow_spinning);
 
 	/*
 	 * The allocation either can't fail or will lead to more memory

Signature · 1 line

2.47.3

Andrew Morton

maintainer

2025-09-22 23:03 UTC #1 lore raw

Quoted 12 lines · On Mon, 22 Sep 2025 15:02:03 -0700 Shakeel Butt <[email protected]> wrote:

Generally memcg charging is allowed from all the contexts including NMI where even spinning on spinlock can cause locking issues. However one call chain was missed during the addition of memcg charging from any context support. That is try_charge_memcg() -> memcg_memory_event() -> cgroup_file_notify().
The possible function call tree under cgroup_file_notify() can acquire many different spin locks in spinning mode. Some of them are cgroup_file_kn_lock, kernfs_notify_lock, pool_workqeue's lock. So, let's just skip cgroup_file_notify() from memcg charging if the context does not allow spinning.

Quoted 4 lines

Alternative approach was also explored where instead of skipping cgroup_file_notify(), we defer the memcg event processing to irq_work [1]. However it adds complexity and it was decided to keep things simple until we need more memcg events with !allow_spinning requirement.

What are the downsides here? Inaccurate charging obviously, but how might this affect users?

mm/memcontrol.c static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, · line 2311

Quoted 8 lines

mm/memcontrol.c +1 -0

@@ -2307,12 +2307,13 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
 	bool drained = false;
 	bool raised_max_event = false;
 	unsigned long pflags;
+	bool allow_spinning = gfpflags_allow_spinning(gfp_mask);

Does this affect only the problematic call chain which you have identified, or might other callers be undesirably affected?

Shakeel Butt

author maintainer

2025-09-22 23:23 UTC #2 lore raw

Quoted 14 lines · Andrew Morton

Quoted 12 lines · On Mon, 22 Sep 2025 15:02:03 -0700 Shakeel Butt <[email protected]> wrote:
Generally memcg charging is allowed from all the contexts including NMI where even spinning on spinlock can cause locking issues. However one call chain was missed during the addition of memcg charging from any context support. That is try_charge_memcg() -> memcg_memory_event() -> cgroup_file_notify().
The possible function call tree under cgroup_file_notify() can acquire many different spin locks in spinning mode. Some of them are cgroup_file_kn_lock, kernfs_notify_lock, pool_workqeue's lock. So, let's just skip cgroup_file_notify() from memcg charging if the context does not allow spinning.

Quoted 7 lines

Quoted 4 lines
Alternative approach was also explored where instead of skipping cgroup_file_notify(), we defer the memcg event processing to irq_work [1]. However it adds complexity and it was decided to keep things simple until we need more memcg events with !allow_spinning requirement.
What are the downsides here? Inaccurate charging obviously, but how might this affect users?

Charging will still be accurate. The only thing we will miss is the possible notifications to the userspace for memory.events[.local] files.

Quoted 12 lines

mm/memcontrol.c static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, · line 2311
Quoted 9 lines
mm/memcontrol.c +1 -0
@@ -2307,12 +2307,13 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
 	bool drained = false;
 	bool raised_max_event = false;
 	unsigned long pflags;
+	bool allow_spinning = gfpflags_allow_spinning(gfp_mask);
 
Does this affect only the problematic call chain which you have identified, or might other callers be undesirably affected?

It will only affect the call chain which can not spin due to possibly NMI context and at the moment only bpf programs can cause that.

Andrew Morton

maintainer

2025-09-22 23:43 UTC #3 lore raw

Quoted 13 lines · On Mon, 22 Sep 2025 16:22:57 -0700 Shakeel Butt <[email protected]> wrote:

Quoted 12 lines
mm/memcontrol.c static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, · line 2311
Quoted 9 lines
mm/memcontrol.c +1 -0
@@ -2307,12 +2307,13 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
 	bool drained = false;
 	bool raised_max_event = false;
 	unsigned long pflags;
+	bool allow_spinning = gfpflags_allow_spinning(gfp_mask);
 
Does this affect only the problematic call chain which you have identified, or might other callers be undesirably affected?

Quoted 2 lines

It will only affect the call chain which can not spin due to possibly NMI context and at the moment only bpf programs can cause that.

"possibly" NMI context? Is it possible that a bpf caller which could have taken locks will now skip the notifications? Or do the gfp_flags get propagated all the way through?

Shakeel Butt

author maintainer

2025-09-22 23:58 UTC #4 lore raw

Quoted 15 lines · Andrew Morton

Quoted 13 lines · On Mon, 22 Sep 2025 16:22:57 -0700 Shakeel Butt <[email protected]> wrote:
Quoted 12 lines
mm/memcontrol.c static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, · line 2311
Quoted 9 lines
mm/memcontrol.c +1 -0
@@ -2307,12 +2307,13 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
 	bool drained = false;
 	bool raised_max_event = false;
 	unsigned long pflags;
+	bool allow_spinning = gfpflags_allow_spinning(gfp_mask);
 
Does this affect only the problematic call chain which you have identified, or might other callers be undesirably affected?

Quoted 4 lines

Quoted 2 lines
It will only affect the call chain which can not spin due to possibly NMI context and at the moment only bpf programs can cause that.
"possibly" NMI context?

NMI is one source which can cause recursive context but bpf programs attached to specific call chains can also cause this recursion. For example in [1], a bpf program related to sched_ext was called with scheduler locks held and then that program made a memcg charged allocation which can potentially call cgroup_file_notify(). The notification call chain again tries to grab scheduler locks and potentially causing deadlocks.

[1] https://lore.kernel.org/all/[email protected]/

Quoted 3 lines

Is it possible that a bpf caller which could
have taken locks will now skip the notifications? Or do the gfp_flags get propagated all the way through?

The bpf programs which might be called/triggerd in a context where spinning on a lock might not be safe, will skip the notifications. The gfp_flags are plugged through. Basically we have kmalloc_nolock() interfaces coming up which will make sure the correct gfp flags are passed through.

Andrew Morton

maintainer

2025-09-22 23:04 UTC #5 lore raw

In reply to Shakeel Butt

Quoted 12 lines · On Mon, 22 Sep 2025 15:02:03 -0700 Shakeel Butt <[email protected]> wrote:

Generally memcg charging is allowed from all the contexts including NMI where even spinning on spinlock can cause locking issues. However one call chain was missed during the addition of memcg charging from any context support. That is try_charge_memcg() -> memcg_memory_event() -> cgroup_file_notify().
The possible function call tree under cgroup_file_notify() can acquire many different spin locks in spinning mode. Some of them are cgroup_file_kn_lock, kernfs_notify_lock, pool_workqeue's lock. So, let's just skip cgroup_file_notify() from memcg charging if the context does not allow spinning.

Quoted 8 lines

Alternative approach was also explored where instead of skipping cgroup_file_notify(), we defer the memcg event processing to irq_work [1]. However it adds complexity and it was decided to keep things simple until we need more memcg events with !allow_spinning requirement.
Link https://lore.kernel.org/all/5qi2llyzf7gklncflo6gxoozljbm4h3tpnuv4u4ej4ztysvi6f@x44v7nz2wdzd/ [1] open here
Signed-off-by Shakeel Butt <[email protected]>
Acked-by Michal Hocko <[email protected]>

Fixes a possible kernel deadlock, yes?

Is a cc:stable appropriate and can we identify a Fixes: target?

Thanks.

(Did it ever generate lockdep warnings?)

Shakeel Butt

author maintainer

2025-09-22 23:40 UTC #6 lore raw

Quoted 23 lines · Andrew Morton

Quoted 12 lines · On Mon, 22 Sep 2025 15:02:03 -0700 Shakeel Butt <[email protected]> wrote:
Generally memcg charging is allowed from all the contexts including NMI where even spinning on spinlock can cause locking issues. However one call chain was missed during the addition of memcg charging from any context support. That is try_charge_memcg() -> memcg_memory_event() -> cgroup_file_notify().
The possible function call tree under cgroup_file_notify() can acquire many different spin locks in spinning mode. Some of them are cgroup_file_kn_lock, kernfs_notify_lock, pool_workqeue's lock. So, let's just skip cgroup_file_notify() from memcg charging if the context does not allow spinning.
Quoted 8 lines
Alternative approach was also explored where instead of skipping cgroup_file_notify(), we defer the memcg event processing to irq_work [1]. However it adds complexity and it was decided to keep things simple until we need more memcg events with !allow_spinning requirement.
Link https://lore.kernel.org/all/5qi2llyzf7gklncflo6gxoozljbm4h3tpnuv4u4ej4ztysvi6f@x44v7nz2wdzd/ [1] open here
Signed-off-by Shakeel Butt <[email protected]>
Acked-by Michal Hocko <[email protected]>

Quoted 7 lines

Fixes a possible kernel deadlock, yes?
Is a cc:stable appropriate and can we identify a Fixes: target?
Thanks.
(Did it ever generate lockdep warnings?)

The report is here:
https://lore.kernel.org/all/[email protected]/

I am not sure about the Fixes tag though or more like which one to put in the Fixes as we recently started supporting memcg charging for NMI context or allowing bpf programs to do memcg charged allocations in recursive context (see the above report for this recursive call chain). There is no single commit which can be blamed here.

Alexei, what do you suggest?

Andrew Morton

maintainer

2025-09-22 23:55 UTC #7 lore raw

Quoted 32 lines · On Mon, 22 Sep 2025 16:39:53 -0700 Shakeel Butt <[email protected]> wrote:

Quoted 23 lines · Andrew Morton
Quoted 12 lines · On Mon, 22 Sep 2025 15:02:03 -0700 Shakeel Butt <[email protected]> wrote:
Generally memcg charging is allowed from all the contexts including NMI where even spinning on spinlock can cause locking issues. However one call chain was missed during the addition of memcg charging from any context support. That is try_charge_memcg() -> memcg_memory_event() -> cgroup_file_notify().
The possible function call tree under cgroup_file_notify() can acquire many different spin locks in spinning mode. Some of them are cgroup_file_kn_lock, kernfs_notify_lock, pool_workqeue's lock. So, let's just skip cgroup_file_notify() from memcg charging if the context does not allow spinning.
Quoted 8 lines
Alternative approach was also explored where instead of skipping cgroup_file_notify(), we defer the memcg event processing to irq_work [1]. However it adds complexity and it was decided to keep things simple until we need more memcg events with !allow_spinning requirement.
Link https://lore.kernel.org/all/5qi2llyzf7gklncflo6gxoozljbm4h3tpnuv4u4ej4ztysvi6f@x44v7nz2wdzd/ [1] open here
Signed-off-by Shakeel Butt <[email protected]>
Acked-by Michal Hocko <[email protected]>
Quoted 7 lines
Fixes a possible kernel deadlock, yes?
Is a cc:stable appropriate and can we identify a Fixes: target?
Thanks.
(Did it ever generate lockdep warnings?)

Quoted 8 lines

The report is here:
https://lore.kernel.org/all/[email protected]/
I am not sure about the Fixes tag though or more like which one to put in the Fixes as we recently started supporting memcg charging for NMI context or allowing bpf programs to do memcg charged allocations in recursive context (see the above report for this recursive call chain). There is no single commit which can be blamed here.

I tend to view the Fixes: as us suggesting which kernel versions should be patched. I'm suspecting that's 6.16+, so using the final relevant patch in that release as a Fixes: target would work.

Shakeel Butt

author maintainer

2025-09-23 00:00 UTC #8 lore raw

Quoted 43 lines · Andrew Morton

Quoted 32 lines · On Mon, 22 Sep 2025 16:39:53 -0700 Shakeel Butt <[email protected]> wrote:
Quoted 23 lines · Andrew Morton
Quoted 12 lines · On Mon, 22 Sep 2025 15:02:03 -0700 Shakeel Butt <[email protected]> wrote:
Generally memcg charging is allowed from all the contexts including NMI where even spinning on spinlock can cause locking issues. However one call chain was missed during the addition of memcg charging from any context support. That is try_charge_memcg() -> memcg_memory_event() -> cgroup_file_notify().
The possible function call tree under cgroup_file_notify() can acquire many different spin locks in spinning mode. Some of them are cgroup_file_kn_lock, kernfs_notify_lock, pool_workqeue's lock. So, let's just skip cgroup_file_notify() from memcg charging if the context does not allow spinning.
Quoted 8 lines
Alternative approach was also explored where instead of skipping cgroup_file_notify(), we defer the memcg event processing to irq_work [1]. However it adds complexity and it was decided to keep things simple until we need more memcg events with !allow_spinning requirement.
Link https://lore.kernel.org/all/5qi2llyzf7gklncflo6gxoozljbm4h3tpnuv4u4ej4ztysvi6f@x44v7nz2wdzd/ [1] open here
Signed-off-by Shakeel Butt <[email protected]>
Acked-by Michal Hocko <[email protected]>
Quoted 7 lines
Fixes a possible kernel deadlock, yes?
Is a cc:stable appropriate and can we identify a Fixes: target?
Thanks.
(Did it ever generate lockdep warnings?)
Quoted 8 lines
The report is here:
https://lore.kernel.org/all/[email protected]/
I am not sure about the Fixes tag though or more like which one to put in the Fixes as we recently started supporting memcg charging for NMI context or allowing bpf programs to do memcg charged allocations in recursive context (see the above report for this recursive call chain). There is no single commit which can be blamed here.

Quoted 4 lines

I tend to view the Fixes: as us suggesting which kernel versions should be patched. I'm suspecting that's 6.16+, so using the final relevant patch in that release as a Fixes: target would work.

Sounds good. Let use the following.

Fixes 3ac4638a734a ("memcg: make memcg_rstat_updated nmi safe")