Skip to contentSkip to conversation

fs/proc: Expose mm_cpumask in /proc/[pid]/status

PATCH In review Aaron Tomlin 5 participants 10 messages Jan 15 – Jan 16

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v3 cover fs/proc: Expose mm_cpumask in /proc/[pid]/status

Hi Oleg, David, Greg, Andrew,

This patch introduces a mechanism to expose the mm_cpumask of a process via the /proc/[pid]/status interface.

In high-performance and large-scale NUMA environments, diagnosing latency spikes attributed to Inter-Processor Interrupts (IPIs) can be particularly challenging. While cpus_allowed describes where a thread may execute, it does not describe the "memory footprint" - specifically, the set of CPUs that may hold stale Translation Lookaside Buffer (TLB) entries for the process.

It is this footprint (mm_cpumask) that dictates the target destination for TLB flush IPIs. Discrepancies between a process's scheduling affinity and its memory footprint are a common source of system noise and performance degradation. By exposing this mask, we provide userspace with the visibility required to debug these "invisible" sources of latency.

These fields are exposed only on architectures that explicitly opt-in via CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary because mm_cpumask semantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.

For example, outside x86:

    # make fs/proc/array.i
    # grep task_cpus_active_mm -B 1 -A 3 --max-count 1 fs/proc/array.i
    # 430 "fs/proc/array.c"
    static inline __attribute__((__gnu_inline__)) __attribute__((__unused__)) __attribute__((no_instrument_function)) void task_cpus_active_mm(struct seq_file *m, struct mm_struct *mm)
    {
    }

The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.

Changes since v2 [1]:
- Introduce new configuration ARCH_WANT_PROC_CPUS_ACTIVE_MM. The x86 architecture now explicitly selects this feature, ensuring that the field is only exposed where the mm_cpumask semantics are meaningful for TLB coherency (David Hildenbrand)

Changes since v1 [2]:
- Document new Cpus_active_mm and Cpus_active_mm_list entries in /proc/[pid]/status (Oleg Nesterov)

[1]: https://lore.kernel.org/lkml/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/

Aaron Tomlin (1):
fs/proc: Expose mm_cpumask in /proc/[pid]/status

 Documentation/filesystems/proc.rst |  7 +++++++
 arch/x86/Kconfig                   |  1 +
 fs/proc/Kconfig                    | 14 ++++++++++++++
 fs/proc/array.c                    | 28 +++++++++++++++++++++++++++-
 4 files changed, 49 insertions(+), 1 deletion(-)
Signature · 1 line
2.51.0
#1 lore raw

PATCH v3 1/1 fs/proc: Expose mm_cpumask in /proc/[pid]/status

This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "Cpus_active_mm" and "Cpus_active_mm_list". The mm_cpumask is primarily used for TLB and cache synchronisation.

Exposing this information allows userspace to easily describe the relationship between CPUs where a memory descriptor is "active" and the CPUs where the thread is allowed to execute. The primary intent is to provide visibility into the "memory footprint" across CPUs, which is invaluable for debugging performance issues related to IPI storms and TLB shootdowns in large-scale NUMA systems. The CPU-affinity sets the boundary; the mm_cpumask records the arrival; they complement each other.

Frequent mm_cpumask changes may indicate instability in placement policies or excessive task migration overhead.

These fields are exposed only on architectures that explicitly opt-in via CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary because mm_cpumask semantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.

The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.

---
 Documentation/filesystems/proc.rst |  7 +++++++
 arch/x86/Kconfig                   |  1 +
 fs/proc/Kconfig                    | 14 ++++++++++++++
 fs/proc/array.c                    | 28 +++++++++++++++++++++++++++-
 4 files changed, 49 insertions(+), 1 deletion(-)
Documentation/filesystems/proc.rst +7 -0
index 8256e857e2d7..c6ced84c5c68 100644
@@ -291,12 +291,19 @@ It's slow but very precise.
  SpeculationIndirectBranch   indirect branch speculation mode
  Cpus_allowed                mask of CPUs on which this process may run
  Cpus_allowed_list           Same as previous, but in "list format"
+ Cpus_active_mm              mask of CPUs on which this process has an active
+                             memory context
+ Cpus_active_mm_list         Same as previous, but in "list format"
  Mems_allowed                mask of memory nodes allowed to this process
  Mems_allowed_list           Same as previous, but in "list format"
  voluntary_ctxt_switches     number of voluntary context switches
  nonvoluntary_ctxt_switches  number of non voluntary context switches
  ==========================  ===================================================
 
+Note "Cpus_active_mm" is currently only supported on x86. Its semantics are
+architecture-dependent; on x86, it represents the set of CPUs that may hold
+stale TLB entries for the process and thus require IPI-based TLB shootdowns to
+maintain coherency.
 
 .. table:: Table 1-3: Contents of the statm fields (as of 2.6.8-rc3)
 
arch/x86/Kconfig +1 -0
index 80527299f859..f0997791dbdb 100644
@@ -152,6 +152,7 @@ config X86
 	select ARCH_WANTS_THP_SWAP		if X86_64
 	select ARCH_HAS_PARANOID_L1D_FLUSH
 	select ARCH_WANT_IRQS_OFF_ACTIVATE_MM
+	select ARCH_WANT_PROC_CPUS_ACTIVE_MM
 	select BUILDTIME_TABLE_SORT
 	select CLKEVT_I8253
 	select CLOCKSOURCE_WATCHDOG
fs/proc/Kconfig +14 -0
index 6ae966c561e7..952c40cf3baa 100644
@@ -127,3 +127,17 @@ config PROC_PID_ARCH_STATUS
 config PROC_CPU_RESCTRL
 	def_bool n
 	depends on PROC_FS
+
+config ARCH_WANT_PROC_CPUS_ACTIVE_MM
+	bool
+	depends on PROC_FS
+	help
+	  Selected by architectures that reliably maintain mm_cpumask for TLB
+	  and cache synchronisation and wish to expose it in
+	  /proc/[pid]/status. Exposing this information allows userspace to
+	  easily describe the relationship between CPUs where a memory
+	  descriptor is "active" and the CPUs where the thread is allowed to
+	  execute. The primary intent is to provide visibility into the
+	  "memory footprint" across CPUs, which is invaluable for debugging
+	  performance issues related to IPI storms and TLB shootdowns in
+	  large-scale NUMA systems.
fs/proc/array.c +27 -1
index 42932f88141a..c16aad59e0a7 100644
@@ -409,6 +409,29 @@ static void task_cpus_allowed(struct seq_file *m, struct task_struct *task)
 		   cpumask_pr_args(&task->cpus_mask));
 }
 
+/**
+ * task_cpus_active_mm - Show the mm_cpumask for a process
+ * @m: The seq_file structure for the /proc/PID/status output
+ * @mm: The memory descriptor of the process
+ *
+ * Prints the set of CPUs, representing the CPU affinity of the process's
+ * active memory context, in both mask and list format. This mask is
+ * primarily used for TLB and cache synchronisation.
+ */
+#ifdef CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM
+static void task_cpus_active_mm(struct seq_file *m, struct mm_struct *mm)
+{
+	seq_printf(m, "Cpus_active_mm:\t%*pb\n",
+		   cpumask_pr_args(mm_cpumask(mm)));
+	seq_printf(m, "Cpus_active_mm_list:\t%*pbl\n",
+		   cpumask_pr_args(mm_cpumask(mm)));
+}
+#else
+static inline void task_cpus_active_mm(struct seq_file *m, struct mm_struct *mm)
+{
+}
+#endif
+
 static inline void task_core_dumping(struct seq_file *m, struct task_struct *task)
 {
 	seq_put_decimal_ull(m, "CoreDumping:\t", !!task->signal->core_state);
@@ -450,12 +473,15 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,
 		task_core_dumping(m, task);
 		task_thp_status(m, mm);
 		task_untag_mask(m, mm);
-		mmput(mm);
 	}
 	task_sig(m, task);
 	task_cap(m, task);
 	task_seccomp(m, task);
 	task_cpus_allowed(m, task);
+	if (mm) {
+		task_cpus_active_mm(m, mm);
+		mmput(mm);
+	}
 	cpuset_task_status_allowed(m, task);
 	task_context_switch_counts(m, task);
 	arch_proc_pid_thread_features(m, task);
Signature · 1 line
2.51.0
#2 lore raw
Quoted 25 lines · Aaron Tomlin

This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "Cpus_active_mm" and "Cpus_active_mm_list". The mm_cpumask is primarily used for TLB and cache synchronisation.

Exposing this information allows userspace to easily describe the relationship between CPUs where a memory descriptor is "active" and the CPUs where the thread is allowed to execute. The primary intent is to provide visibility into the "memory footprint" across CPUs, which is invaluable for debugging performance issues related to IPI storms and TLB shootdowns in large-scale NUMA systems. The CPU-affinity sets the boundary; the mm_cpumask records the arrival; they complement each other.

Frequent mm_cpumask changes may indicate instability in placement policies or excessive task migration overhead.

These fields are exposed only on architectures that explicitly opt-in via CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary because mm_cpumask semantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.

Quoted 6 lines

The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.

Yes, restricting to architectures that have the expected semantics is better.

... but we better get the blessing from x86 folks :)

(CCing the x86 MM folks)

Quoted 118 lines
---
  Documentation/filesystems/proc.rst |  7 +++++++
  arch/x86/Kconfig                   |  1 +
  fs/proc/Kconfig                    | 14 ++++++++++++++
  fs/proc/array.c                    | 28 +++++++++++++++++++++++++++-
  4 files changed, 49 insertions(+), 1 deletion(-)
Documentation/filesystems/proc.rst +7 -0
index 8256e857e2d7..c6ced84c5c68 100644
@@ -291,12 +291,19 @@ It's slow but very precise.
   SpeculationIndirectBranch   indirect branch speculation mode
   Cpus_allowed                mask of CPUs on which this process may run
   Cpus_allowed_list           Same as previous, but in "list format"
+ Cpus_active_mm              mask of CPUs on which this process has an active
+                             memory context
+ Cpus_active_mm_list         Same as previous, but in "list format"
   Mems_allowed                mask of memory nodes allowed to this process
   Mems_allowed_list           Same as previous, but in "list format"
   voluntary_ctxt_switches     number of voluntary context switches
   nonvoluntary_ctxt_switches  number of non voluntary context switches
   ==========================  ===================================================
 
+Note "Cpus_active_mm" is currently only supported on x86. Its semantics are
+architecture-dependent; on x86, it represents the set of CPUs that may hold
+stale TLB entries for the process and thus require IPI-based TLB shootdowns to
+maintain coherency.
 
  .. table:: Table 1-3: Contents of the statm fields (as of 2.6.8-rc3)
 
arch/x86/Kconfig +1 -0
index 80527299f859..f0997791dbdb 100644
@@ -152,6 +152,7 @@ config X86
  	select ARCH_WANTS_THP_SWAP		if X86_64
  	select ARCH_HAS_PARANOID_L1D_FLUSH
  	select ARCH_WANT_IRQS_OFF_ACTIVATE_MM
+	select ARCH_WANT_PROC_CPUS_ACTIVE_MM
  	select BUILDTIME_TABLE_SORT
  	select CLKEVT_I8253
  	select CLOCKSOURCE_WATCHDOG
fs/proc/Kconfig +14 -0
index 6ae966c561e7..952c40cf3baa 100644
@@ -127,3 +127,17 @@ config PROC_PID_ARCH_STATUS
  config PROC_CPU_RESCTRL
  	def_bool n
  	depends on PROC_FS
+
+config ARCH_WANT_PROC_CPUS_ACTIVE_MM
+	bool
+	depends on PROC_FS
+	help
+	  Selected by architectures that reliably maintain mm_cpumask for TLB
+	  and cache synchronisation and wish to expose it in
+	  /proc/[pid]/status. Exposing this information allows userspace to
+	  easily describe the relationship between CPUs where a memory
+	  descriptor is "active" and the CPUs where the thread is allowed to
+	  execute. The primary intent is to provide visibility into the
+	  "memory footprint" across CPUs, which is invaluable for debugging
+	  performance issues related to IPI storms and TLB shootdowns in
+	  large-scale NUMA systems.
fs/proc/array.c +27 -1
index 42932f88141a..c16aad59e0a7 100644
@@ -409,6 +409,29 @@ static void task_cpus_allowed(struct seq_file *m, struct task_struct *task)
  		   cpumask_pr_args(&task->cpus_mask));
  }
 
+/**
+ * task_cpus_active_mm - Show the mm_cpumask for a process
+ * @m: The seq_file structure for the /proc/PID/status output
+ * @mm: The memory descriptor of the process
+ *
+ * Prints the set of CPUs, representing the CPU affinity of the process's
+ * active memory context, in both mask and list format. This mask is
+ * primarily used for TLB and cache synchronisation.
+ */
+#ifdef CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM
+static void task_cpus_active_mm(struct seq_file *m, struct mm_struct *mm)
+{
+	seq_printf(m, "Cpus_active_mm:\t%*pb\n",
+		   cpumask_pr_args(mm_cpumask(mm)));
+	seq_printf(m, "Cpus_active_mm_list:\t%*pbl\n",
+		   cpumask_pr_args(mm_cpumask(mm)));
+}
+#else
+static inline void task_cpus_active_mm(struct seq_file *m, struct mm_struct *mm)
+{
+}
+#endif
+
  static inline void task_core_dumping(struct seq_file *m, struct task_struct *task)
  {
  	seq_put_decimal_ull(m, "CoreDumping:\t", !!task->signal->core_state);
@@ -450,12 +473,15 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,
  		task_core_dumping(m, task);
  		task_thp_status(m, mm);
  		task_untag_mask(m, mm);
-		mmput(mm);
  	}
  	task_sig(m, task);
  	task_cap(m, task);
  	task_seccomp(m, task);
  	task_cpus_allowed(m, task);
+	if (mm) {
+		task_cpus_active_mm(m, mm);
+		mmput(mm);
+	}
  	cpuset_task_status_allowed(m, task);
  	task_context_switch_counts(m, task);
  	arch_proc_pid_thread_features(m, task);
Signature · 3 lines
Cheers

David
#3 lore raw
Quoted 34 lines · David Hildenbrand (Red Hat)
Quoted 25 lines · Aaron Tomlin

This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "Cpus_active_mm" and "Cpus_active_mm_list". The mm_cpumask is primarily used for TLB and cache synchronisation.

Exposing this information allows userspace to easily describe the relationship between CPUs where a memory descriptor is "active" and the CPUs where the thread is allowed to execute. The primary intent is to provide visibility into the "memory footprint" across CPUs, which is invaluable for debugging performance issues related to IPI storms and TLB shootdowns in large-scale NUMA systems. The CPU-affinity sets the boundary; the mm_cpumask records the arrival; they complement each other.

Frequent mm_cpumask changes may indicate instability in placement policies or excessive task migration overhead.

These fields are exposed only on architectures that explicitly opt-in via CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary because mm_cpumask semantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.

Quoted 6 lines

The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.

Quoted 6 lines

Yes, restricting to architectures that have the expected semantics is better.

... but we better get the blessing from x86 folks :)

(CCing the x86 MM folks)

Yeah, seems like a very bad idea this. mm_cpumask really is an arch detail you cannot rely on very much. Exposing this to userspace is terrible.

#4 lore raw
In reply to David Hildenbrand (Red Hat)
Quoted 6 lines · David Hildenbrand (Red Hat)
Quoted 5 lines · Aaron Tomlin

This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "Cpus_active_mm" and "Cpus_active_mm_list". The mm_cpumask is primarily used for TLB and cache synchronisation.

I don't think this is the kind of thing we want to expose as ABI. It's too deep of an implementation detail. Any meaning derived from it could also change on a whim.

For instance, we've changed the rules about when CPUs are put in or taken out of mm_cpumask() over time. I think the rules might have even depended on the idle driver that your system was using at one time. I think Rik also just changed some rules around it in his INVLPGB patches.

I'm not denying how valuable this kind of information might be. I just don't think it's generally useful enough to justify an ABI that we need to maintain forever. Tracing seems like a much more appropriate way to get the data you are after than new ABI.

Can you get the info that you're after with kprobes? Or new tracepoints?

#5 lore raw
Quoted 9 lines · Dave Hansen

I don't think this is the kind of thing we want to expose as ABI. It's too deep of an implementation detail. Any meaning derived from it could also change on a whim.

For instance, we've changed the rules about when CPUs are put in or taken out of mm_cpumask() over time. I think the rules might have even depended on the idle driver that your system was using at one time. I think Rik also just changed some rules around it in his INVLPGB patches.

Quoted 6 lines

I'm not denying how valuable this kind of information might be. I just don't think it's generally useful enough to justify an ABI that we need to maintain forever. Tracing seems like a much more appropriate way to get the data you are after than new ABI.

Can you get the info that you're after with kprobes? Or new tracepoints?

Hi Dave and Peter,

I fully appreciate your concern regarding the exposure of deep implementation details as stable ABI. I understand that the semantics of mm_cpumask are fluid. I certainly do not wish to ossify internal logic.

While the static tracepoint trace_tlb_flush is available, the primary argument for exposing this via /proc/[pid]/status is one of immediacy and the lack of external dependencies. Having an instantaneous snapshot available without requiring e.g., Ftrace or eBPF, is invaluable for quick diagnostic checks in production environments.

Based on my reading of arch/x86/mm/tlb.c, the lifecycle of each bit in mm_cpumask appears to follow this logic:

1. Schedule on (switch_mm): Bit set.
2. Schedule off: Bit remains set (CPU enters "Lazy" mode).
3. Remote TLB Flush (IPI):
- If Running: Flush TLB, bit remains set.
- If lazy (leave_mm): Switch to init_mm, bit clearing is deferred.
- If stale (mm != loaded_mm): bit is cleared immediately (effectively the second IPI for a CPU that was previously lazy).

Would you be amenable to this exposure if it were guarded behind a specific CONFIG_DEBUG option (e.g., CONFIG_DEBUG_MM_CPUMASK_INFO)? This would clearly mark it as a diagnostic aid for debugging, allowing educated users to opt-in to the visibility without implying a permanent guarantee of semantic stability for general userspace applications.

Please let me know your thoughts. Thank you.

Kind regards,

Signature · 1 line
Aaron Tomlin
#6 lore raw
Quoted 5 lines · Aaron Tomlin

Based on my reading of arch/x86/mm/tlb.c, the lifecycle of each bit in
mm_cpumask appears to follow this logic:

Quoted 10 lines

    1. Schedule on (switch_mm): Bit set.
    2. Schedule off: Bit remains set (CPU enters "Lazy" mode).
    3. Remote TLB Flush (IPI):
       - If Running: Flush TLB, bit remains set.
       - If lazy (leave_mm): Switch to init_mm, bit clearing is deferred.
       - If stale (mm != loaded_mm): bit is cleared immediately (effectively the second IPI for a CPU that was previously lazy).

You're close. When a process uses INVLPGB, no remote TLB flushing IPIs will get sent, and CPUs never get cleared from the mm_cpumask.

Signature · 1 line
All Rights Reversed.
#8 lore raw
In reply to Aaron Tomlin
Quoted 2 lines · Aaron Tomlin

Would you be amenable to this exposure if it were guarded behind a specific CONFIG_DEBUG option (e.g., CONFIG_DEBUG_MM_CPUMASK_INFO)?

Not really.

ABI behind a config option is still ABI.

#7 lore raw
In reply to Rik van Riel
Quoted 17 lines · Rik van Riel
Quoted 5 lines · Aaron Tomlin

Based on my reading of arch/x86/mm/tlb.c, the lifecycle of each bit in
mm_cpumask appears to follow this logic:

Quoted 10 lines

    1. Schedule on (switch_mm): Bit set.
    2. Schedule off: Bit remains set (CPU enters "Lazy" mode).
    3. Remote TLB Flush (IPI):
       - If Running: Flush TLB, bit remains set.
       - If lazy (leave_mm): Switch to init_mm, bit clearing is deferred.
       - If stale (mm != loaded_mm): bit is cleared immediately (effectively the second IPI for a CPU that was previously lazy).

Quoted 3 lines

You're close. When a process uses INVLPGB, no remote TLB flushing IPIs will get sent, and CPUs never get cleared from the mm_cpumask.

Hi Rik,

Not close enough :)

It is good to hear from you, and thank you for the clarification regarding X86_FEATURE_INVLPGB.

You are quite right; as flush_tlb_func() serves as the sole mechanism for clearing bits from the mm_cpumask, bypassing IPIs inherently bypasses the cleanup logic. Consequently, in this scenario, the bit is set upon scheduling but never cleared, as the hardware-broadcast invalidations circumvent the software handler responsible for maintaining the mask.

Kind regards,

Signature · 1 line
Aaron Tomlin
#9 lore raw
In reply to Dave Hansen
Quoted 7 lines · Dave Hansen
Quoted 2 lines · Aaron Tomlin

Would you be amenable to this exposure if it were guarded behind a specific CONFIG_DEBUG option (e.g., CONFIG_DEBUG_MM_CPUMASK_INFO)?

Not really.

ABI behind a config option is still ABI.

Hi Dave,

I understand.

Kind regards,

Signature · 1 line
Aaron Tomlin