Skip to contentSkip to conversation

proc: add /proc/<pid>/arch_status

PATCH Superseded Aubrey Li 2 participants 5 messages Apr 25, 2019 – May 27, 2019

Subsystems (4)

v18 of 19 Superseded Latest is v19 →

lore
Share & tools
lore

Aubrey Li

author
#0 lore raw

PATCH v18 1/3 proc: add /proc/<pid>/arch_status

The architecture specific information of the running processes could be useful to the userland. Add /proc/<pid>/arch_status interface support to examine process architecture specific information externally.

Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Andi Kleen <[email protected]>
Cc Tim Chen <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Arjan van de Ven <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Linux API <[email protected]>
---
 fs/proc/Kconfig         | 4 ++++
 fs/proc/base.c          | 6 ++++++
 include/linux/proc_fs.h | 9 +++++++++
 3 files changed, 19 insertions(+)
fs/proc/Kconfig +4 -0
index 817c02b13b1d..d80ebf19d5f1 100644
@@ -97,3 +97,7 @@ config PROC_CHILDREN
 
 	  Say Y if you are running any user-space software which takes benefit from
 	  this interface. For example, rkt is such a piece of software.
+
+config PROC_PID_ARCH_STATUS
+	def_bool n
+	depends on PROC_FS
fs/proc/base.c +6 -0
index 6a803a0b75df..8c71eba47031 100644
@@ -3061,6 +3061,9 @@ static const struct pid_entry tgid_base_stuff[] = {
 #ifdef CONFIG_STACKLEAK_METRICS
 	ONE("stack_depth", S_IRUGO, proc_stack_depth),
 #endif
+#ifdef CONFIG_PROC_PID_ARCH_STATUS
+	ONE("arch_status", S_IRUGO, proc_pid_arch_status),
+#endif
 };
 
 static int proc_tgid_base_readdir(struct file *file, struct dir_context *ctx)
@@ -3449,6 +3452,9 @@ static const struct pid_entry tid_base_stuff[] = {
 #ifdef CONFIG_LIVEPATCH
 	ONE("patch_state",  S_IRUSR, proc_pid_patch_state),
 #endif
+#ifdef CONFIG_PROC_PID_ARCH_STATUS
+	ONE("arch_status", S_IRUGO, proc_pid_arch_status),
+#endif
 };
 
 static int proc_tid_base_readdir(struct file *file, struct dir_context *ctx)
include/linux/proc_fs.h +9 -0
index 52a283ba0465..a705aa2d03f9 100644
@@ -75,6 +75,15 @@ struct proc_dir_entry *proc_create_net_single_write(const char *name, umode_t mo
 						    void *data);
 extern struct pid *tgid_pidfd_to_pid(const struct file *file);
 
+#ifdef CONFIG_PROC_PID_ARCH_STATUS
+/*
+ * The architecture which selects CONFIG_PROC_PID_ARCH_STATUS must
+ * provide proc_pid_arch_status() definition.
+ */
+int proc_pid_arch_status(struct seq_file *m, struct pid_namespace *ns,
+			struct pid *pid, struct task_struct *task);
+#endif /* CONFIG_PROC_PID_ARCH_STATUS */
+
 #else /* CONFIG_PROC_FS */
 
 static inline void proc_root_init(void)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH v18 2/3 x86,/proc/pid/arch_status: Add AVX-512 usage elapsed time

AVX-512 components use could cause core turbo frequency drop. So it's useful to expose AVX-512 usage elapsed time as a heuristic hint for the user space job scheduler to cluster the AVX-512 using tasks together.

Tensorflow example:

$ while [ 1 ]; do cat /proc/tid/arch_status | grep AVX512; sleep 1; done
AVX512_elapsed_ms:      4
AVX512_elapsed_ms:      8
AVX512_elapsed_ms:      4

This means that 4 milliseconds have elapsed since the AVX512 usage of tensorflow task was detected when the task was scheduled out.

Or:

$ cat /proc/tid/arch_status | grep AVX512
AVX512_elapsed_ms:      -1

The number '-1' indicates that no AVX512 usage recorded before thus the task unlikely has frequency drop issue.

User space tools may want to further check by:

$ perf stat --pid <pid> -e core_power.lvl2_turbo_license -- sleep 1

Performance counter stats for process id '3558':

     3,251,565,961      core_power.lvl2_turbo_license

       1.004031387 seconds time elapsed

Non-zero counter value confirms that the task causes frequency drop.

Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Andi Kleen <[email protected]>
Cc Tim Chen <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Arjan van de Ven <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Linux API <[email protected]>
---
 arch/x86/Kconfig             |  1 +
 arch/x86/kernel/fpu/xstate.c | 47 ++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)
arch/x86/Kconfig +1 -0
index 5ad92419be19..d5a9c5ddd453 100644
@@ -208,6 +208,7 @@ config X86
 	select USER_STACKTRACE_SUPPORT
 	select VIRT_TO_BUS
 	select X86_FEATURE_NAMES		if PROC_FS
+	select PROC_PID_ARCH_STATUS		if PROC_FS
 
 config INSTRUCTION_DECODER
 	def_bool y
arch/x86/kernel/fpu/xstate.c +47 -0

56 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#2 lore raw

PATCH v18 3/3 Documentation/filesystems/proc.txt: add arch_status file

In reply to Aubrey Li

Added /proc/<pid>/arch_status file, and added AVX512_elapsed_ms in /proc/<pid>/arch_status. Report it in Documentation/filesystems/proc.txt

Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Andi Kleen <[email protected]>
Cc Tim Chen <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Arjan van de Ven <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Linux API <[email protected]>
---
 Documentation/filesystems/proc.txt | 39 ++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
Documentation/filesystems/proc.txt +39 -0
index 66cad5c86171..e8bc403d15df 100644
@@ -45,6 +45,7 @@ Table of Contents
   3.9   /proc/<pid>/map_files - Information about memory mapped files
   3.10  /proc/<pid>/timerslack_ns - Task timerslack value
   3.11	/proc/<pid>/patch_state - Livepatch patch operation state
+  3.12	/proc/<pid>/arch_status - Task architecture specific information
 
   4	Configuring procfs
   4.1	Mount options
@@ -1948,6 +1949,44 @@ patched.  If the patch is being enabled, then the task has already been
 patched.  If the patch is being disabled, then the task hasn't been
 unpatched yet.
 
+3.12 /proc/<pid>/arch_status - task architecture specific status
+-------------------------------------------------------------------
+When CONFIG_PROC_PID_ARCH_STATUS is enabled, this file displays the
+architecture specific status of the task.
+
+Example
+-------
+ $ cat /proc/6753/arch_status
+ AVX512_elapsed_ms:      8
+
+Description
+-----------
+
+x86 specific entries:
+---------------------
+ AVX512_elapsed_ms:
+ ------------------
+  If AVX512 is supported on the machine, this entry shows the milliseconds
+  elapsed since the last time AVX512 usage was recorded. The recording
+  happens on a best effort basis when a task is scheduled out. This means
+  that the value depends on two factors:
+
+    1) The time which the task spent on the CPU without being scheduled
+       out. With CPU isolation and a single runnable task this can take
+       several seconds.
+
+    2) The time since the task was scheduled out last. Depending on the
+       reason for being scheduled out (time slice exhausted, syscall ...)
+       this can be arbitrary long time.
+
+  As a consequence the value cannot be considered precise and authoritative
+  information. The application which uses this information has to be aware
+  of the overall scenario on the system in order to determine whether a
+  task is a real AVX512 user or not.
+
+  A special value of '-1' indicates that no AVX512 usage was recorded, thus
+  the task is unlikely an AVX512 user, but depends on the workload and the
+  scheduling scenario, it also could be a false negative mentioned above.
 
 ------------------------------------------------------------------------------
 Configuring procfs
Signature · 1 line
2.17.1
Acked-by on v18 1/3
#3 lore raw
In reply to Aubrey Li
Quoted 4 lines · On Thu, 25 Apr 2019 22:32:17 +0800 Aubrey Li <[email protected]> wrote:

The architecture specific information of the running processes could be useful to the userland. Add /proc/<pid>/arch_status interface support to examine process architecture specific information externally.

I'll give this an

from a procfs POV and shall let the x86 maintainers worry about it.

I must say I'm a bit surprised that we don't already provide some form of per-process CPU-specific info anywhere in procfs. Something to piggy-back this onto. But I can't find such a thing.

I assume we've already discussed why this is a new procfs file rather than merely a new line in /proc/<pid>/status. If so, please add the reasoning to the changelog. If not, please discuss now ;)

author
#4 lore raw
Quoted 10 lines · Andrew Morton
Quoted 4 lines · On Thu, 25 Apr 2019 22:32:17 +0800 Aubrey Li <[email protected]> wrote:

The architecture specific information of the running processes could be useful to the userland. Add /proc/<pid>/arch_status interface support to examine process architecture specific information externally.

I'll give this an

Thanks!

Quoted 11 lines

from a procfs POV and shall let the x86 maintainers worry about it.

I must say I'm a bit surprised that we don't already provide some form of per-process CPU-specific info anywhere in procfs. Something to piggy-back this onto. But I can't find such a thing.

I assume we've already discussed why this is a new procfs file rather than merely a new line in /proc/<pid>/status. If so, please add the reasoning to the changelog. If not, please discuss now ;)

Andy and Thomas may want to give more comments. The discussion was that we don't want /proc/PID/status to be different on different architectures. It would be better to separate the arch staff into its own file /proc/PID/ arch_status and make sure that everything in it is namespaced.

Thanks,
-Aubrey