/proc/pid/status: Add support for architecture specific output
Subsystems (4)
PATCH v15 1/3 /proc/pid/status: Add support for architecture specific output
The architecture specific information of the running processes could be useful to the userland. Add support to examine process architecture specific information externally.
--- fs/proc/array.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/proc/array.c +6 -0
@@ -96,6 +96,11 @@#include <asm/processor.h> #include "internal.h" +/* Add support for architecture specific output in /proc/pid/status */ +#ifndef arch_proc_pid_status +#define arch_proc_pid_status (m, task) +#endif +void proc_task_name (struct seq_file *m,struct task_struct *p,bool escape) {char *buf;
@@ -424,6 +429,7 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,task_cpus_allowed (m, task);cpuset_task_status_allowed (m, task);task_context_switch_counts (m, task); +arch_proc_pid_status (m, task);return 0 ; }
Signature · 1 line
2.21.0
PATCH v15 2/3 x86,/proc/pid/status: Add AVX-512 usage elapsed time
AVX-512 components use could cause core turbo frequency drop. So it's useful to expose AVX-512 usage elapsed time as a heuristic hint for the user space job scheduler to cluster the AVX-512 using tasks together.
Tensorflow example:
$ while [ 1 ]; do cat /proc/tid/status | grep AVX; sleep 1; done AVX512_elapsed_ms: 4 AVX512_elapsed_ms: 8 AVX512_elapsed_ms: 4
This means that 4 milliseconds have elapsed since the AVX512 usage of tensorflow task was detected when the task was scheduled out.
Or:
$ cat /proc/tid/status | grep AVX512_elapsed_ms AVX512_elapsed_ms: -1
The number '-1' indicates that no AVX512 usage recorded before thus the task unlikely has frequency drop issue.
User space tools may want to further check by:
$ perf stat --pid <pid> -e core_power.lvl2_turbo_license -- sleep 1
Performance counter stats for process id '3558':
3,251,565,961 core_power.lvl2_turbo_license
1.004031387 seconds time elapsedNon-zero counter value confirms that the task causes frequency drop.
--- arch/x86/include/asm/processor.h | 4 +++ arch/x86/kernel/fpu/xstate.c | 42 ++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+)
arch/x86/include/asm/processor.h +4 -0
@@ -991,4 +991,8 @@ enum l1tf_mitigations {extern enum l1tf_mitigations l1tf_mitigation; +/* Add support for architecture specific output in /proc/pid/status */ +void arch_proc_pid_status (struct seq_file *m,struct task_struct *task); +#define arch_proc_pid_status arch_proc_pid_status +#endif /* _ASM_X86_PROCESSOR_H */
arch/x86/kernel/fpu/xstate.c +42 -0
51 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH v15 3/3 Documentation/filesystems/proc.txt: add AVX512_elapsed_ms
In reply to Aubrey LiAdded AVX512_elapsed_ms in /proc/<pid>/status. Report it in Documentation/filesystems/proc.txt
--- Documentation/filesystems/proc.txt | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-)
Documentation/filesystems/proc.txt +28 -1
@@ -207,6 +207,7 @@ read the file /proc/PID/status: Speculation_Store_Bypass: thread vulnerable voluntary_ctxt_switches: 0 nonvoluntary_ctxt_switches: 1 + AVX512_elapsed_ms: 8 This shows you nearly the same information you would get if you viewed it with the ps command. In fact, ps uses the proc file system to obtain its
@@ -224,7 +225,7 @@ asynchronous manner and the value may not be very precise. To see a precise snapshot of a moment, you can see /proc/<pid>/smaps file and scan page table. It's slow but very precise. -Table 1-2: Contents of the status files (as of 4.19) +Table 1-2: Contents of the status files (as of 5.1) .............................................................................. Field Content Name filename of the executable
@@ -289,6 +290,32 @@ Table 1-2: Contents of the status files (as of 4.19) Mems_allowed_list Same as previous, but in "list format" voluntary_ctxt_switches number of voluntary context switches nonvoluntary_ctxt_switches number of non voluntary context switches + AVX512_elapsed_ms time elapsed since last AVX512 usage recorded + + AVX512_elapsed_ms: + ------------------ + If AVX512 is supported on the machine, this entry shows the milliseconds + elapsed since the last time AVX512 usage was recorded. The recording + happens on a best effort basis when a task is scheduled out. This means + that the value depends on two factors: + + 1) The time which the task spent on the CPU without being scheduled + out. With CPU isolation and a single runnable task this can take + several seconds. + + 2) The time since the task was scheduled out last. Depending on the + reason for being scheduled out (time slice exhausted, syscall ...) + this can be arbitrary long time. + + As a consequence the value cannot be considered precise and authoritative + information. The application which uses this information has to be aware + of the overall scenario on the system in order to determine whether a + task is a real AVX512 user or not. + + A special value of '-1' indicates that no AVX512 usage was recorded, thus + the task is unlikely an AVX512 user, but depends on the workload and the + scheduling scenario, it also could be a false negative mentioned above. + .............................................................................. Table 1-3: Contents of the statm files (as of 2.6.8-rc3)
Signature · 1 line
2.21.0
Quoted 3 lines · On Tue, 16 Apr 2019 14:32:48 +0800 Aubrey Li <[email protected]> wrote:
The architecture specific information of the running processes could be useful to the userland. Add support to examine process architecture specific information externally.
The implementation looks just fine to me. Have you had any feedback on the overall desirability of adding this feature?
fs/proc/array.c line 102
Quoted 10 lines
fs/proc/array.c +4
-0@@ -96,6 +96,11 @@#include <asm/processor.h> #include "internal.h" +/* Add support for architecture specific output in /proc/pid/status */ +#ifndef arch_proc_pid_status +#define arch_proc_pid_status (m, task) +#endif
To this I suggest adding
/* arch_proc_pid_status() must be defined in asm/processor.h */
Because we've regularly had different architectures defining such things in different headers, resulting in a mess.
Quoted 9 lines · Andrew Morton
Quoted 3 lines · On Tue, 16 Apr 2019 14:32:48 +0800 Aubrey Li <[email protected]> wrote:
The architecture specific information of the running processes could be useful to the userland. Add support to examine process architecture specific information externally.
The implementation looks just fine to me. Have you had any feedback on the overall desirability of adding this feature?
This is orientated by the customer's complain. When the customer colocated their latency critical tasks with AVX512 tasks, the latency critical tasks were affected due to core frequency slowing down by AVX512 use. So we propose this interface for the user space tools to identify AVX512 using task and apply user space dispatching policy. We may have subsequent effort based on this proposal.
Quoted 13 lines
fs/proc/array.c line 102
Quoted 10 lines
fs/proc/array.c +4
-0@@ -96,6 +96,11 @@#include <asm/processor.h> #include "internal.h" +/* Add support for architecture specific output in /proc/pid/status */ +#ifndef arch_proc_pid_status +#define arch_proc_pid_status (m, task) +#endif To this I suggest adding
Quoted 4 lines
/*
arch_proc_pid_status()must be defined inasm/processor.h*/Because we've regularly had different architectures defining such things in different headers, resulting in a mess.
Thanks, I'll add it in the next version.
-Aubrey
Quoted 9 lines · Andrew Morton
Quoted 3 lines · On Tue, 16 Apr 2019 14:32:48 +0800 Aubrey Li <[email protected]> wrote:
The architecture specific information of the running processes could be useful to the userland. Add support to examine process architecture specific information externally.
The implementation looks just fine to me. Have you had any feedback on the overall desirability of adding this feature?
I think I've been the most outspoken, and my not-all-that-strong opinion is that I don't really like it. /proc/PID/status is already a bit of a mess, and I don't think we really want it to be a mess that is different on different architectures. Hence my suggestion of /proc/PID/x86_status instead. Or we could do /proc/PID/arch_status, I suppose, and make sure that everything in it is namespaced. (We could easily end up with a situation where status fields from more than one architecture are present. Think i386 + x86_64. Thata's also the case where qemu userspace emulation is used to run binaries meant for one architecture on a different architecture's kernel.)
--Andy
Quoted 8 lines · Andy Lutomirski
Quoted 7 lines · Andrew Morton
Quoted 3 lines · On Tue, 16 Apr 2019 14:32:48 +0800 Aubrey Li <[email protected]> wrote:
The architecture specific information of the running processes could be useful to the userland. Add support to examine process architecture specific information externally.
Quoted 8 lines
Quoted 2 lines
The implementation looks just fine to me. Have you had any feedback on the overall desirability of adding this feature?
I think I've been the most outspoken, and my not-all-that-strong opinion is that I don't really like it.
/proc/PID/statusis already a bit of a mess, and I don't think we really want it to be a mess that is different on different architectures. Hence my suggestion of/proc/PID/x86_statusinstead. Or we could do/proc/PID/arch_status, I
arch_status looks like the right thing to do.
Thanks,
tglx
Quoted 13 lines · Thomas Gleixner
Quoted 11 lines · Andy Lutomirski
Quoted 9 lines · Andrew Morton
Quoted 3 lines · On Tue, 16 Apr 2019 14:32:48 +0800 Aubrey Li <[email protected]> wrote:
The architecture specific information of the running processes could be useful to the userland. Add support to examine process architecture specific information externally.
The implementation looks just fine to me. Have you had any feedback on the overall desirability of adding this feature?
Quoted 7 lines
Quoted 5 lines
I think I've been the most outspoken, and my not-all-that-strong opinion is that I don't really like it.
/proc/PID/statusis already a bit of a mess, and I don't think we really want it to be a mess that is different on different architectures. Hence my suggestion of/proc/PID/x86_statusinstead. Or we could do/proc/PID/arch_status, I
arch_statuslooks like the right thing to do.
Thanks Andy and Thomas, let me change the patch to use arch_status instead.