fs/proc: Expose mm_cpumask in /proc/[pid]/status
Subsystems (4)
v3 cover fs/proc: Expose mm_cpumask in /proc/[pid]/status
Hi Oleg, David, Greg, Andrew,
This patch introduces a mechanism to expose the mm_cpumask of a process via the /proc/[pid]/status interface.
In high-performance and large-scale NUMA environments, diagnosing latency spikes attributed to Inter-Processor Interrupts (IPIs) can be particularly challenging. While cpus_allowed describes where a thread may execute, it does not describe the "memory footprint" - specifically, the set of CPUs that may hold stale Translation Lookaside Buffer (TLB) entries for the process.
It is this footprint (mm_cpumask) that dictates the target destination for TLB flush IPIs. Discrepancies between a process's scheduling affinity and its memory footprint are a common source of system noise and performance degradation. By exposing this mask, we provide userspace with the visibility required to debug these "invisible" sources of latency.
These fields are exposed only on architectures that explicitly opt-in via CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary because mm_cpumask semantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.
For example, outside x86:
# make fs/proc/array.i
# grep task_cpus_active_mm -B 1 -A 3 --max-count 1 fs/proc/array.i
# 430 "fs/proc/array.c"
static inline __attribute__((__gnu_inline__)) __attribute__((__unused__)) __attribute__((no_instrument_function)) void task_cpus_active_mm(struct seq_file *m, struct mm_struct *mm)
{
}The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.
Changes since v2 [1]:
- Introduce new configuration ARCH_WANT_PROC_CPUS_ACTIVE_MM. The x86 architecture now explicitly selects this feature, ensuring that the field is only exposed where the mm_cpumask semantics are meaningful for TLB coherency (David Hildenbrand)
Changes since v1 [2]:
- Document new Cpus_active_mm and Cpus_active_mm_list entries in /proc/[pid]/status (Oleg Nesterov)
[1]: https://lore.kernel.org/lkml/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
Aaron Tomlin (1):
fs/proc: Expose mm_cpumask in /proc/[pid]/status
Documentation/filesystems/proc.rst | 7 +++++++ arch/x86/Kconfig | 1 + fs/proc/Kconfig | 14 ++++++++++++++ fs/proc/array.c | 28 +++++++++++++++++++++++++++- 4 files changed, 49 insertions(+), 1 deletion(-)
Signature · 1 line
2.51.0
PATCH v3 1/1 fs/proc: Expose mm_cpumask in /proc/[pid]/status
This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "Cpus_active_mm" and "Cpus_active_mm_list". The mm_cpumask is primarily used for TLB and cache synchronisation.
Exposing this information allows userspace to easily describe the relationship between CPUs where a memory descriptor is "active" and the CPUs where the thread is allowed to execute. The primary intent is to provide visibility into the "memory footprint" across CPUs, which is invaluable for debugging performance issues related to IPI storms and TLB shootdowns in large-scale NUMA systems. The CPU-affinity sets the boundary; the mm_cpumask records the arrival; they complement each other.
Frequent mm_cpumask changes may indicate instability in placement policies or excessive task migration overhead.
These fields are exposed only on architectures that explicitly opt-in via CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary because mm_cpumask semantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.
The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.
--- Documentation/filesystems/proc.rst | 7 +++++++ arch/x86/Kconfig | 1 + fs/proc/Kconfig | 14 ++++++++++++++ fs/proc/array.c | 28 +++++++++++++++++++++++++++- 4 files changed, 49 insertions(+), 1 deletion(-)
Documentation/filesystems/proc.rst +7 -0
@@ -291,12 +291,19 @@ It's slow but very precise. SpeculationIndirectBranch indirect branch speculation mode Cpus_allowed mask of CPUs on which this process may run Cpus_allowed_list Same as previous, but in "list format" + Cpus_active_mm mask of CPUs on which this process has an active + memory context + Cpus_active_mm_list Same as previous, but in "list format" Mems_allowed mask of memory nodes allowed to this process Mems_allowed_list Same as previous, but in "list format" voluntary_ctxt_switches number of voluntary context switches nonvoluntary_ctxt_switches number of non voluntary context switches ========================== =================================================== +Note "Cpus_active_mm" is currently only supported on x86. Its semantics are +architecture-dependent; on x86, it represents the set of CPUs that may hold +stale TLB entries for the process and thus require IPI-based TLB shootdowns to +maintain coherency. .. table:: Table 1-3: Contents of the statm fields (as of 2.6.8-rc3)
arch/x86/Kconfig +1 -0
@@ -152,6 +152,7 @@ config X86 select ARCH_WANTS_THP_SWAP if X86_64 select ARCH_HAS_PARANOID_L1D_FLUSH select ARCH_WANT_IRQS_OFF_ACTIVATE_MM + select ARCH_WANT_PROC_CPUS_ACTIVE_MM select BUILDTIME_TABLE_SORT select CLKEVT_I8253 select CLOCKSOURCE_WATCHDOG
fs/proc/Kconfig +14 -0
@@ -127,3 +127,17 @@ config PROC_PID_ARCH_STATUS config PROC_CPU_RESCTRL def_bool n depends on PROC_FS + +config ARCH_WANT_PROC_CPUS_ACTIVE_MM + bool + depends on PROC_FS + help + Selected by architectures that reliably maintain mm_cpumask for TLB + and cache synchronisation and wish to expose it in + /proc/[pid]/status. Exposing this information allows userspace to + easily describe the relationship between CPUs where a memory + descriptor is "active" and the CPUs where the thread is allowed to + execute. The primary intent is to provide visibility into the + "memory footprint" across CPUs, which is invaluable for debugging + performance issues related to IPI storms and TLB shootdowns in + large-scale NUMA systems.
fs/proc/array.c +27 -1
@@ -409,6 +409,29 @@ static void task_cpus_allowed(struct seq_file *m, struct task_struct *task) cpumask_pr_args(&task->cpus_mask)); } +/** +* task_cpus_active_mm - Show the mm_cpumask for a process +* @m: The seq_file structure for the /proc/PID/status output +* @mm: The memory descriptor of the process +* +* Prints the set of CPUs, representing the CPU affinity of the process's +* active memory context, in both mask and list format. This mask is +* primarily used for TLB and cache synchronisation. +*/ +#ifdef CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM +static void task_cpus_active_mm (struct seq_file *m,struct mm_struct *mm) +{ +seq_printf (m,"Cpus_active_mm:\t%*pb\n" , +cpumask_pr_args (mm_cpumask (mm))); +seq_printf (m,"Cpus_active_mm_list:\t%*pbl\n" , +cpumask_pr_args (mm_cpumask (mm))); +} +#else +static inline void task_cpus_active_mm (struct seq_file *m,struct mm_struct *mm) +{ +} +#endif +static inline void task_core_dumping (struct seq_file *m,struct task_struct *task) {seq_put_decimal_ull (m,"CoreDumping:\t" , !!task->signal->core_state);
@@ -450,12 +473,15 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,task_core_dumping (m, task);task_thp_status (m, mm);task_untag_mask (m, mm); -mmput (mm); }task_sig (m, task);task_cap (m, task);task_seccomp (m, task);task_cpus_allowed (m, task); +if (mm) { +task_cpus_active_mm (m, mm); +mmput (mm); + }cpuset_task_status_allowed (m, task);task_context_switch_counts (m, task);arch_proc_pid_thread_features (m, task);
Signature · 1 line
2.51.0
Quoted 25 lines · Aaron Tomlin
This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "
Cpus_active_mm" and "Cpus_active_mm_list". Themm_cpumaskis primarily used for TLB and cache synchronisation.Exposing this information allows userspace to easily describe the relationship between CPUs where a memory descriptor is "active" and the CPUs where the thread is allowed to execute. The primary intent is to provide visibility into the "memory footprint" across CPUs, which is invaluable for debugging performance issues related to IPI storms and TLB shootdowns in large-scale NUMA systems. The CPU-affinity sets the boundary; the
mm_cpumaskrecords the arrival; they complement each other.Frequent
mm_cpumaskchanges may indicate instability in placement policies or excessive task migration overhead.These fields are exposed only on architectures that explicitly opt-in via
CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary becausemm_cpumasksemantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.
Quoted 6 lines
The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.
Yes, restricting to architectures that have the expected semantics is better.
... but we better get the blessing from x86 folks :)
(CCing the x86 MM folks)
Quoted 118 lines
Signed-off-by Aaron Tomlin <[email protected]>--- Documentation/filesystems/proc.rst | 7 +++++++ arch/x86/Kconfig | 1 + fs/proc/Kconfig | 14 ++++++++++++++ fs/proc/array.c | 28 +++++++++++++++++++++++++++- 4 files changed, 49 insertions(+), 1 deletion(-)Documentation/filesystems/proc.rst +7
-0@@ -291,12 +291,19 @@ It's slow but very precise. SpeculationIndirectBranch indirect branch speculation mode Cpus_allowed mask of CPUs on which this process may run Cpus_allowed_list Same as previous, but in "list format" + Cpus_active_mm mask of CPUs on which this process has an active + memory context + Cpus_active_mm_list Same as previous, but in "list format" Mems_allowed mask of memory nodes allowed to this process Mems_allowed_list Same as previous, but in "list format" voluntary_ctxt_switches number of voluntary context switches nonvoluntary_ctxt_switches number of non voluntary context switches ========================== =================================================== +Note "Cpus_active_mm" is currently only supported on x86. Its semantics are +architecture-dependent; on x86, it represents the set of CPUs that may hold +stale TLB entries for the process and thus require IPI-based TLB shootdowns to +maintain coherency. .. table:: Table 1-3: Contents of the statm fields (as of 2.6.8-rc3)arch/x86/Kconfig +1
-0@@ -152,6 +152,7 @@ config X86 select ARCH_WANTS_THP_SWAP if X86_64 select ARCH_HAS_PARANOID_L1D_FLUSH select ARCH_WANT_IRQS_OFF_ACTIVATE_MM + select ARCH_WANT_PROC_CPUS_ACTIVE_MM select BUILDTIME_TABLE_SORT select CLKEVT_I8253 select CLOCKSOURCE_WATCHDOGfs/proc/Kconfig +14
-0@@ -127,3 +127,17 @@ config PROC_PID_ARCH_STATUS config PROC_CPU_RESCTRL def_bool n depends on PROC_FS + +config ARCH_WANT_PROC_CPUS_ACTIVE_MM + bool + depends on PROC_FS + help + Selected by architectures that reliably maintain mm_cpumask for TLB + and cache synchronisation and wish to expose it in + /proc/[pid]/status. Exposing this information allows userspace to + easily describe the relationship between CPUs where a memory + descriptor is "active" and the CPUs where the thread is allowed to + execute. The primary intent is to provide visibility into the + "memory footprint" across CPUs, which is invaluable for debugging + performance issues related to IPI storms and TLB shootdowns in + large-scale NUMA systems.fs/proc/array.c +27
-1@@ -409,6 +409,29 @@ static void task_cpus_allowed(struct seq_file *m, struct task_struct *task) cpumask_pr_args(&task->cpus_mask)); } +/** +* task_cpus_active_mm - Show the mm_cpumask for a process +* @m: The seq_file structure for the /proc/PID/status output +* @mm: The memory descriptor of the process +* +* Prints the set of CPUs, representing the CPU affinity of the process's +* active memory context, in both mask and list format. This mask is +* primarily used for TLB and cache synchronisation. +*/ +#ifdef CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM +static void task_cpus_active_mm (struct seq_file *m,struct mm_struct *mm) +{ +seq_printf (m,"Cpus_active_mm:\t%*pb\n" , +cpumask_pr_args (mm_cpumask (mm))); +seq_printf (m,"Cpus_active_mm_list:\t%*pbl\n" , +cpumask_pr_args (mm_cpumask (mm))); +} +#else +static inline void task_cpus_active_mm (struct seq_file *m,struct mm_struct *mm) +{ +} +#endif +static inline void task_core_dumping (struct seq_file *m,struct task_struct *task) {seq_put_decimal_ull (m,"CoreDumping:\t" , !!task->signal->core_state);@@ -450,12 +473,15 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,task_core_dumping (m, task);task_thp_status (m, mm);task_untag_mask (m, mm); -mmput (mm); }task_sig (m, task);task_cap (m, task);task_seccomp (m, task);task_cpus_allowed (m, task); +if (mm) { +task_cpus_active_mm (m, mm); +mmput (mm); + }cpuset_task_status_allowed (m, task);task_context_switch_counts (m, task);arch_proc_pid_thread_features (m, task);
Signature · 3 lines
Cheers David
Quoted 34 lines · David Hildenbrand (Red Hat)
Quoted 25 lines · Aaron Tomlin
This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "
Cpus_active_mm" and "Cpus_active_mm_list". Themm_cpumaskis primarily used for TLB and cache synchronisation.Exposing this information allows userspace to easily describe the relationship between CPUs where a memory descriptor is "active" and the CPUs where the thread is allowed to execute. The primary intent is to provide visibility into the "memory footprint" across CPUs, which is invaluable for debugging performance issues related to IPI storms and TLB shootdowns in large-scale NUMA systems. The CPU-affinity sets the boundary; the
mm_cpumaskrecords the arrival; they complement each other.Frequent
mm_cpumaskchanges may indicate instability in placement policies or excessive task migration overhead.These fields are exposed only on architectures that explicitly opt-in via
CONFIG_ARCH_WANT_PROC_CPUS_ACTIVE_MM. This is necessary becausemm_cpumasksemantics vary significantly across architectures; some (e.g., x86) actively maintain the mask for coherency, while others may never clear bits, rendering the data misleading for this specific use case. x86 is updated to select this feature by default.Quoted 6 lines
The implementation reads the mask directly without introducing additional locks or snapshots. While this implies that the hex mask and list format could theoretically observe slightly different states on a rapidly changing system, this "best-effort" approach aligns with the standard design philosophy of /proc and avoids imposing locking overhead on critical memory management paths.
Quoted 6 lines
Yes, restricting to architectures that have the expected semantics is better.
... but we better get the blessing from x86 folks :)
(CCing the x86 MM folks)
Yeah, seems like a very bad idea this. mm_cpumask really is an arch detail you cannot rely on very much. Exposing this to userspace is terrible.
Quoted 6 lines · David Hildenbrand (Red Hat)
Quoted 5 lines · Aaron Tomlin
This patch introduces two new fields to /proc/[pid]/status to display the set of CPUs, representing the CPU affinity of the process's active memory context, in both mask and list format: "
Cpus_active_mm" and "Cpus_active_mm_list". Themm_cpumaskis primarily used for TLB and cache synchronisation.
I don't think this is the kind of thing we want to expose as ABI. It's too deep of an implementation detail. Any meaning derived from it could also change on a whim.
For instance, we've changed the rules about when CPUs are put in or taken out of mm_cpumask() over time. I think the rules might have even depended on the idle driver that your system was using at one time. I think Rik also just changed some rules around it in his INVLPGB patches.
I'm not denying how valuable this kind of information might be. I just don't think it's generally useful enough to justify an ABI that we need to maintain forever. Tracing seems like a much more appropriate way to get the data you are after than new ABI.
Can you get the info that you're after with kprobes? Or new tracepoints?
Quoted 9 lines · Dave Hansen
I don't think this is the kind of thing we want to expose as ABI. It's too deep of an implementation detail. Any meaning derived from it could also change on a whim.
For instance, we've changed the rules about when CPUs are put in or taken out of
mm_cpumask()over time. I think the rules might have even depended on the idle driver that your system was using at one time. I think Rik also just changed some rules around it in his INVLPGB patches.
Quoted 6 lines
I'm not denying how valuable this kind of information might be. I just don't think it's generally useful enough to justify an ABI that we need to maintain forever. Tracing seems like a much more appropriate way to get the data you are after than new ABI.
Can you get the info that you're after with kprobes? Or new tracepoints?
Hi Dave and Peter,
I fully appreciate your concern regarding the exposure of deep implementation details as stable ABI. I understand that the semantics of mm_cpumask are fluid. I certainly do not wish to ossify internal logic.
While the static tracepoint trace_tlb_flush is available, the primary argument for exposing this via /proc/[pid]/status is one of immediacy and the lack of external dependencies. Having an instantaneous snapshot available without requiring e.g., Ftrace or eBPF, is invaluable for quick diagnostic checks in production environments.
Based on my reading of arch/x86/mm/tlb.c, the lifecycle of each bit in mm_cpumask appears to follow this logic:
1. Schedule on (switch_mm): Bit set.
2. Schedule off: Bit remains set (CPU enters "Lazy" mode).
3. Remote TLB Flush (IPI):
- If Running: Flush TLB, bit remains set.
- If lazy (leave_mm): Switch to init_mm, bit clearing is deferred.
- If stale (mm != loaded_mm): bit is cleared immediately (effectively the second IPI for a CPU that was previously lazy).
Would you be amenable to this exposure if it were guarded behind a specific CONFIG_DEBUG option (e.g., CONFIG_DEBUG_MM_CPUMASK_INFO)? This would clearly mark it as a diagnostic aid for debugging, allowing educated users to opt-in to the visibility without implying a permanent guarantee of semantic stability for general userspace applications.
Please let me know your thoughts. Thank you.
Kind regards,
Signature · 1 line
Aaron Tomlin
Quoted 5 lines · Aaron Tomlin
Based on my reading of
arch/x86/mm/tlb.c, the lifecycle of each bit inmm_cpumaskappears to follow this logic:
Quoted 10 lines
1. Schedule on (
switch_mm): Bit set.
2. Schedule off: Bit remains set (CPU enters "Lazy" mode).
3. Remote TLB Flush (IPI):
- If Running: Flush TLB, bit remains set.
- If lazy (leave_mm): Switch toinit_mm, bit clearing is deferred.
- If stale (mm !=loaded_mm): bit is cleared immediately (effectively the second IPI for a CPU that was previously lazy).
You're close. When a process uses INVLPGB, no remote TLB flushing IPIs will get sent, and CPUs never get cleared from the mm_cpumask.
Signature · 1 line
All Rights Reversed.
Quoted 17 lines · Rik van Riel
Quoted 5 lines · Aaron Tomlin
Based on my reading of
arch/x86/mm/tlb.c, the lifecycle of each bit inmm_cpumaskappears to follow this logic:Quoted 10 lines
1. Schedule on (
switch_mm): Bit set.
2. Schedule off: Bit remains set (CPU enters "Lazy" mode).
3. Remote TLB Flush (IPI):
- If Running: Flush TLB, bit remains set.
- If lazy (leave_mm): Switch toinit_mm, bit clearing is deferred.
- If stale (mm !=loaded_mm): bit is cleared immediately (effectively the second IPI for a CPU that was previously lazy).
Quoted 3 lines
You're close. When a process uses INVLPGB, no remote TLB flushing IPIs will get sent, and CPUs never get cleared from the
mm_cpumask.
Hi Rik,
Not close enough :)
It is good to hear from you, and thank you for the clarification regarding X86_FEATURE_INVLPGB.
You are quite right; as flush_tlb_func() serves as the sole mechanism for clearing bits from the mm_cpumask, bypassing IPIs inherently bypasses the cleanup logic. Consequently, in this scenario, the bit is set upon scheduling but never cleared, as the hardware-broadcast invalidations circumvent the software handler responsible for maintaining the mask.
Kind regards,
Signature · 1 line
Aaron Tomlin
Quoted 2 lines · Aaron Tomlin
Would you be amenable to this exposure if it were guarded behind a specific
CONFIG_DEBUGoption (e.g.,CONFIG_DEBUG_MM_CPUMASK_INFO)?
Not really.
ABI behind a config option is still ABI.
Quoted 7 lines · Dave Hansen
Quoted 2 lines · Aaron Tomlin
Would you be amenable to this exposure if it were guarded behind a specific
CONFIG_DEBUGoption (e.g.,CONFIG_DEBUG_MM_CPUMASK_INFO)?Not really.
ABI behind a config option is still ABI.
Hi Dave,
I understand.
Kind regards,
Signature · 1 line
Aaron Tomlin