sched/cputime: Ensure accurate utime and stime ratio in cputime_adjust()
PATCH v2 1/1 sched/cputime: Ensure accurate utime and stime ratio in cputime_adjust()
If users access "/proc/pid/stat", the utime and stime ratio in the current SAMPLE period are excepted, but currently cputime_adjust() always calculates with the ratio of the WHOLE lifetime of the process.
This results in inaccurate utime and stime in "/proc/pid/stat". For example, a process runs for a while with "50% usr, 0% sys", then followed by "100% sys". For later while, the following is excepted:
0.0 usr, 100.0 sys
but we got:
10.0 usr, 90.0 sys
This patch uses the accurate ratio in cputime_adjust() to address the issue. A new task_cputime type field is added in prev_cputime to record previous task_cputime so that we can get the elapsed times as the accurate ratio.
---v1->v2:
- Rewrite the changelog.
include/linux/sched.h | 34 ++++++++++++------------ include/linux/sched/cputime.h | 12 ++++++++- kernel/sched/cputime.c | 61 ++++++++++++++++--------------------------- 3 files changed, 52 insertions(+), 55 deletions(-)
include/linux/sched.h +18 -16
@@ -223,10 +223,27 @@ extern void io_schedule_finish(int token);extern long io_schedule_timeout (long timeout);extern void io_schedule (void ); +/** +* struct task_cputime - collected CPU time counts +* @utime: time spent in user mode, in nanoseconds +* @stime: time spent in kernel mode, in nanoseconds +* @sum_exec_runtime: total time spent on the CPU, in nanoseconds +* +* This structure groups together three kinds of CPU time that are tracked for +* threads and thread groups. Most things considering CPU time want to group +* these counts together and treat all three of them in parallel. +*/ +struct task_cputime { +u64 utime; +u64 stime; +unsigned long long sum_exec_runtime; +}; +/** * struct prev_cputime - snapshot of system and user cputime * @utime: time spent in user mode * @stime: time spent in system mode +* @cputime: previous task_cputime to calculate utime/stime * @lock: protects the above two fields * * Stores previous user/system time values such that we can guarantee
@@ -236,26 +253,11 @@ struct prev_cputime {#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE u64 utime;u64 stime; +struct task_cputime cputime;raw_spinlock_t lock;#endif }; -/** -* struct task_cputime - collected CPU time counts -* @utime: time spent in user mode, in nanoseconds -* @stime: time spent in kernel mode, in nanoseconds -* @sum_exec_runtime: total time spent on the CPU, in nanoseconds -* -* This structure groups together three kinds of CPU time that are tracked for -* threads and thread groups. Most things considering CPU time want to group -* these counts together and treat all three of them in parallel. -*/ -struct task_cputime { -u64 utime; -u64 stime; -unsigned long long sum_exec_runtime; -}; -/* Alternate field names when used on cache expirations: */ #define virt_exp utime#define prof_exp stime
include/linux/sched/cputime.h +11 -1
@@ -175,10 +175,20 @@ static inline void account_group_exec_runtime(struct task_struct *tsk,atomic64_add (ns, &cputimer->cputime_atomic.sum_exec_runtime); } -static inline void prev_cputime_init (struct prev_cputime *prev) +static inline void prev_cputime_clear (struct prev_cputime *prev) {#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE prev->utime = prev->stime =0 ; + prev->cputime.utime =0 ; + prev->cputime.stime =0 ; + prev->cputime.sum_exec_runtime =0 ; +#endif +} + +static inline void prev_cputime_init (struct prev_cputime *prev) +{ +#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE +prev_cputime_clear (prev);raw_spin_lock_init (&prev->lock);#endif }
kernel/sched/cputime.c +23 -38
92 lines not on this page · raw message
Signature · 1 line
2.14.1.40.g8e62ba1
tip-bot for Xunlei Pang
merge botsched/cputime: Ensure accurate utime and stime ratio in cputime_adjust()
Commit-ID: 8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Gitweb: https://git.kernel.org/tip/8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Author: Xunlei Pang <[email protected]> AuthorDate: Mon, 9 Jul 2018 22:58:43 +0800 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 16 Jul 2018 00:28:31 +0200
sched/cputime: Ensure accurate utime and stime ratio in cputime_adjust()
If users access "/proc/pid/stat", the utime and stime ratio in the current SAMPLE period are excepted, but currently cputime_adjust() always calculates with the ratio of the WHOLE lifetime of the process.
This results in inaccurate utime and stime in "/proc/pid/stat". For example, a process runs for a while with "50% usr, 0% sys", then followed by "100% sys". For later while, the following is excepted:
0.0 usr, 100.0 sys
but we get:
10.0 usr, 90.0 sys
This patch uses the accurate ratio in cputime_adjust() to address the issue. A new 'task_cputime' type field is added in prev_cputime to record previous 'task_cputime' so that we can get the elapsed times as the accurate ratio.
--- include/linux/sched.h | 34 ++++++++++++------------ include/linux/sched/cputime.h | 12 ++++++++- kernel/sched/cputime.c | 61 ++++++++++++++++--------------------------- 3 files changed, 52 insertions(+), 55 deletions(-)
include/linux/sched.h +18 -16
@@ -223,10 +223,27 @@ extern void io_schedule_finish(int token);extern long io_schedule_timeout (long timeout);extern void io_schedule (void ); +/** +* struct task_cputime - collected CPU time counts +* @utime: time spent in user mode, in nanoseconds +* @stime: time spent in kernel mode, in nanoseconds +* @sum_exec_runtime: total time spent on the CPU, in nanoseconds +* +* This structure groups together three kinds of CPU time that are tracked for +* threads and thread groups. Most things considering CPU time want to group +* these counts together and treat all three of them in parallel. +*/ +struct task_cputime { +u64 utime; +u64 stime; +unsigned long long sum_exec_runtime; +}; +/** * struct prev_cputime - snapshot of system and user cputime * @utime: time spent in user mode * @stime: time spent in system mode +* @cputime: previous task_cputime to calculate utime/stime * @lock: protects the above two fields * * Stores previous user/system time values such that we can guarantee
@@ -236,26 +253,11 @@ struct prev_cputime {#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE u64 utime;u64 stime; +struct task_cputime cputime;raw_spinlock_t lock;#endif }; -/** -* struct task_cputime - collected CPU time counts -* @utime: time spent in user mode, in nanoseconds -* @stime: time spent in kernel mode, in nanoseconds -* @sum_exec_runtime: total time spent on the CPU, in nanoseconds -* -* This structure groups together three kinds of CPU time that are tracked for -* threads and thread groups. Most things considering CPU time want to group -* these counts together and treat all three of them in parallel. -*/ -struct task_cputime { -u64 utime; -u64 stime; -unsigned long long sum_exec_runtime; -}; -/* Alternate field names when used on cache expirations: */ #define virt_exp utime#define prof_exp stime
include/linux/sched/cputime.h +11 -1
@@ -175,10 +175,20 @@ static inline void account_group_exec_runtime(struct task_struct *tsk,atomic64_add (ns, &cputimer->cputime_atomic.sum_exec_runtime); } -static inline void prev_cputime_init (struct prev_cputime *prev) +static inline void prev_cputime_clear (struct prev_cputime *prev) {#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE prev->utime = prev->stime =0 ; + prev->cputime.utime =0 ; + prev->cputime.stime =0 ; + prev->cputime.sum_exec_runtime =0 ; +#endif +} + +static inline void prev_cputime_init (struct prev_cputime *prev) +{ +#ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE +prev_cputime_clear (prev);raw_spin_lock_init (&prev->lock);#endif }
kernel/sched/cputime.c +23 -38
92 lines not on this page · raw message
Peter Zijlstra
maintainerQuoted 19 lines · tip-bot for Xunlei Pang
Commit-ID: 8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Gitweb: https://git.kernel.org/tip/8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Author: Xunlei Pang <[email protected]> AuthorDate: Mon, 9 Jul 2018 22:58:43 +0800 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 16 Jul 2018 00:28:31 +0200sched/cputime: Ensure accurate utime and stime ratio in
cputime_adjust()If users access "
/proc/pid/stat", the utime and stime ratio in the current SAMPLE period are excepted, but currentlycputime_adjust()always calculates with the ratio of the WHOLE lifetime of the process.This results in inaccurate utime and stime in "
/proc/pid/stat". For example, a process runs for a while with "50% usr, 0% sys", then followed by "100% sys". For later while, the following is excepted:0.0 usr, 100.0 sys
Quoted 8 lines
but we get:
10.0 usr, 90.0 sys
This patch uses the accurate ratio in
cputime_adjust()to address the issue. A new 'task_cputime' type field is added inprev_cputimeto record previous 'task_cputime' so that we can get the elapsed times as the accurate ratio.
Ingo, please make this one go away. I still have no idae what the problem is and I've not had time to reverse engineer the patch.
The previous (v1) Changelog was a pile of incoherent rambling and the above doesn't explain anything much.
I want a clear description of the problem and a coherent explanation of the proposed solution without having to reverse engineer the actual patch.
Quoted 32 lines · Peter Zijlstra
Quoted 19 lines · tip-bot for Xunlei Pang
Commit-ID: 8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Gitweb: https://git.kernel.org/tip/8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Author: Xunlei Pang <[email protected]> AuthorDate: Mon, 9 Jul 2018 22:58:43 +0800 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 16 Jul 2018 00:28:31 +0200sched/cputime: Ensure accurate utime and stime ratio in
cputime_adjust()If users access "
/proc/pid/stat", the utime and stime ratio in the current SAMPLE period are excepted, but currentlycputime_adjust()always calculates with the ratio of the WHOLE lifetime of the process.This results in inaccurate utime and stime in "
/proc/pid/stat". For example, a process runs for a while with "50% usr, 0% sys", then followed by "100% sys". For later while, the following is excepted:0.0 usr, 100.0 sys
Quoted 8 lines
but we get:
10.0 usr, 90.0 sys
This patch uses the accurate ratio in
cputime_adjust()to address the issue. A new 'task_cputime' type field is added inprev_cputimeto record previous 'task_cputime' so that we can get the elapsed times as the accurate ratio.Ingo, please make this one go away. I still have no idae what the problem is and I've not had time to reverse engineer the patch.
Quoted 6 lines
The previous (v1) Changelog was a pile of incoherent rambling and the above doesn't explain anything much.
I want a clear description of the problem and a coherent explanation of the proposed solution without having to reverse engineer the actual patch.
I must confess I'm having trouble to understand what the real problem is.
Quoted 22 lines · Peter Zijlstra
Quoted 21 lines · tip-bot for Xunlei Pang
Commit-ID: 8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Gitweb: https://git.kernel.org/tip/8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Author: Xunlei Pang <[email protected]> AuthorDate: Mon, 9 Jul 2018 22:58:43 +0800 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 16 Jul 2018 00:28:31 +0200sched/cputime: Ensure accurate utime and stime ratio in
cputime_adjust()If users access "
/proc/pid/stat", the utime and stime ratio in the current SAMPLE period are excepted, but currentlycputime_adjust()always calculates with the ratio of the WHOLE lifetime of the process.This results in inaccurate utime and stime in "
/proc/pid/stat". For example, a process runs for a while with "50% usr, 0% sys", then followed by "100% sys". For later while, the following is excepted:0.0 usr, 100.0 sys
but we get:
Quoted 8 lines
Quoted 6 lines
10.0 usr, 90.0 sys
This patch uses the accurate ratio in
cputime_adjust()to address the issue. A new 'task_cputime' type field is added inprev_cputimeto record previous 'task_cputime' so that we can get the elapsed times as the accurate ratio.Ingo, please make this one go away.
Sure, I've removed it from sched/core.
Thanks,
Ingo
Quoted 27 lines · Ingo Molnar
Quoted 24 lines · Peter Zijlstra
Quoted 23 lines · tip-bot for Xunlei Pang
Commit-ID: 8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Gitweb: https://git.kernel.org/tip/8d4c00dc38a8aa30dae8402955e55e7b34e74bc8 Author: Xunlei Pang <[email protected]> AuthorDate: Mon, 9 Jul 2018 22:58:43 +0800 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 16 Jul 2018 00:28:31 +0200sched/cputime: Ensure accurate utime and stime ratio in
cputime_adjust()If users access "
/proc/pid/stat", the utime and stime ratio in the current SAMPLE period are excepted, but currentlycputime_adjust()always calculates with the ratio of the WHOLE lifetime of the process.This results in inaccurate utime and stime in "
/proc/pid/stat". For example, a process runs for a while with "50% usr, 0% sys", then followed by "100% sys". For later while, the following is excepted:0.0 usr, 100.0 sys
but we get:
10.0 usr, 90.0 sys
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
This patch uses the accurate ratio in
cputime_adjust()to address the issue. A new 'task_cputime' type field is added inprev_cputimeto record previous 'task_cputime' so that we can get the elapsed times as the accurate ratio.Ingo, please make this one go away.
Sure, I've removed it from sched/core.
Hi Ingo, Peter, Frederic,
I captured some runtime data using trace to explain it, hope this can illustrate the motive behind my patch. Anyone could help improve my changelog is appreciated if you think so after reading.
Here are the simple trace_printk I added to capture the real data:
kernel/sched/cputime.c +0 -0
@@ -611,6 +611,9 @@ void cputime_adjust(struct task_cputime *curr,
struct prev_cputime *prev,
stime = curr->stime;
utime = curr->utime;+ if (!strncmp(current->comm, "cat", 3))
+ trace_printk("task tick-based utime %lld stime %lld,
scheduler rtime %lld\n", utime, stime, rtime);
+
/*
* If either stime or utime are 0, assume all runtime is userspace.
* Once a task gets some ticks, the monotonicy code at 'update:'
@@ -651,9 +654,14 @@ void cputime_adjust(struct task_cputime *curr, struct prev_cputime *prev,
stime = rtime - utime;
}+ if (!strncmp(current->comm, "cat", 3))
+ trace_printk("result: old utime %lld stime %lld, new
utime %lld stime %lld\n",
+ prev->utime, prev->stime, utime, stime);
+
prev->stime = stime;
prev->utime = utime;
out:
*ut = prev->utime;
*st = prev->stime;
raw_spin_unlock_irqrestore(&prev->lock, flags);Using the reproducer I described in the changelog, it runs for a while with "50% usr, 0% sys", then followed by "100% sys". A shell script accesses its /proc/pid/stat at the interval of one second, and got:
50.0 usr, 0.0 sys 51.0 usr, 0.0 sys 50.0 usr, 0.0 sys
...
9.0 usr, 91.0 sys
9.0 usr, 91.0 sys
The trace data corresponds to the last sample period:
trace entry 1:
cat-20755 [022] d... 1370.106496: cputime_adjust: task
tick-based utime 362560000000 stime 2551000000, scheduler rtime 333060702626
cat-20755 [022] d... 1370.106497: cputime_adjust: result:old utime 330729718142 stime 2306983867, new utime 330733635372 stime 2327067254
trace entry 2:
cat-20773 [005] d... 1371.109825: cputime_adjust: task
tick-based utime 362567000000 stime 3547000000, scheduler rtime 334063718912
cat-20773 [005] d... 1371.109826: cputime_adjust: result:old utime 330733635372 stime 2327067254, new utime 330827229702 stime 3236489210
1) expected behaviour
Let's compare the last two trace entries(all the data below is in ns):
task tick-based utime: 362560000000->362567000000 increased 7000000 task tick-based stime: 2551000000 ->3547000000 increased 996000000 scheduler rtime: 333060702626->334063718912 increased 1003016286
The application actually runs almost 100%sys at the moment, we can use the task tick-based utime and stime increased to double check:
996000000/(7000000+996000000) > 99%sys
2) the current cputime_adjust() inaccurate result
But for the current cputime_adjust(), we get the following adjusted utime and stime increase in this sample period:
adjusted utime: 330733635372->330827229702 increased 93594330
adjusted stime: 2327067254 ->3236489210 increased 909421956
so 909421956/(93594330+909421956)=91%sys as the shell script shows above.
3) root cause
The root cause of the issue is that the current cputime_adjust() always passes the whole times to scale_stime() to split the whole utime and stime. In this patch, we pass all the increased deltas in 1) within user's sample period to scale_stime() instead and accumulate the corresponding results to the previous saved adjusted utime and stime, so guarantee the accurate usr and sys increase within the user sample period.
Thanks,
Xunlei
Peter Zijlstra
maintainerQuoted 33 lines · Xunlei Pang
The trace data corresponds to the last sample period:
trace entry 1:cat-20755 [022] d... 1370.106496: cputime_adjust: task tick-based utime 362560000000 stime 2551000000, scheduler rtime 333060702626 cat-20755 [022] d... 1370.106497: cputime_adjust: result:old utime 330729718142 stime 2306983867, new utime 330733635372 stime 2327067254
trace entry 2:
cat-20773 [005] d... 1371.109825: cputime_adjust: task tick-based utime 362567000000 stime 3547000000, scheduler rtime 334063718912 cat-20773 [005] d... 1371.109826: cputime_adjust: result:old utime 330733635372 stime 2327067254, new utime 330827229702 stime 3236489210
1) expected behaviour
Let's compare the last two trace entries(all the data below is in ns):task tick-based utime: 362560000000->362567000000 increased 7000000 task tick-based stime: 2551000000 ->3547000000 increased 996000000 scheduler rtime: 333060702626->334063718912 increased 1003016286The application actually runs almost 100%sys at the moment, we can use the task tick-based utime and stime increased to double check:
996000000/(7000000+996000000) > 99%sys2) the current
cputime_adjust()inaccurate result
But for the currentcputime_adjust(), we get the following adjusted utime and stime increase in this sample period:
adjusted utime: 330733635372->330827229702 increased 93594330
adjusted stime: 2327067254 ->3236489210 increased 909421956so 909421956/(93594330+909421956)=91%sys as the shell script shows above.
Quoted 8 lines
3) root cause
The root cause of the issue is that the currentcputime_adjust()always passes the whole times toscale_stime()to split the whole utime and stime. In this patch, we pass all the increased deltas in 1) within user's sample period toscale_stime()instead and accumulate the corresponding results to the previous saved adjusted utime and stime, so guarantee the accurate usr and sys increase within the user sample period.
But why it this a problem?
Since its sample based there's really nothing much you can guarantee. What if your test program were to run in userspace for 50% of the time but is so constructed to always be in kernel space when the tick happens?
Then you would 'expect' it to be 50% user and 50% sys, but you're also not getting that.
This stuff cannot be perfect, and the current code provides 'sensible' numbers over the long run for most programs. Why muck with it?
Quoted 50 lines · Peter Zijlstra
Quoted 33 lines · Xunlei Pang
The trace data corresponds to the last sample period:
trace entry 1:cat-20755 [022] d... 1370.106496: cputime_adjust: task tick-based utime 362560000000 stime 2551000000, scheduler rtime 333060702626 cat-20755 [022] d... 1370.106497: cputime_adjust: result:old utime 330729718142 stime 2306983867, new utime 330733635372 stime 2327067254
trace entry 2:
cat-20773 [005] d... 1371.109825: cputime_adjust: task tick-based utime 362567000000 stime 3547000000, scheduler rtime 334063718912 cat-20773 [005] d... 1371.109826: cputime_adjust: result:old utime 330733635372 stime 2327067254, new utime 330827229702 stime 3236489210
1) expected behaviour
Let's compare the last two trace entries(all the data below is in ns):task tick-based utime: 362560000000->362567000000 increased 7000000 task tick-based stime: 2551000000 ->3547000000 increased 996000000 scheduler rtime: 333060702626->334063718912 increased 1003016286The application actually runs almost 100%sys at the moment, we can use the task tick-based utime and stime increased to double check:
996000000/(7000000+996000000) > 99%sys2) the current
cputime_adjust()inaccurate result
But for the currentcputime_adjust(), we get the following adjusted utime and stime increase in this sample period:
adjusted utime: 330733635372->330827229702 increased 93594330
adjusted stime: 2327067254 ->3236489210 increased 909421956so 909421956/(93594330+909421956)=91%sys as the shell script shows above.
Quoted 8 lines
3) root cause
The root cause of the issue is that the currentcputime_adjust()always passes the whole times toscale_stime()to split the whole utime and stime. In this patch, we pass all the increased deltas in 1) within user's sample period toscale_stime()instead and accumulate the corresponding results to the previous saved adjusted utime and stime, so guarantee the accurate usr and sys increase within the user sample period.But why it this a problem?
Since its sample based there's really nothing much you can guarantee. What if your test program were to run in userspace for 50% of the time but is so constructed to always be in kernel space when the tick happens?
Quoted 6 lines
Then you would 'expect' it to be 50% user and 50% sys, but you're also not getting that.
This stuff cannot be perfect, and the current code provides 'sensible' numbers over the long run for most programs. Why muck with it?
Basically I am ok with the current implementation, except for one scenario we've met: when kernel went wrong for some reason with 100% sys suddenly for seconds(even trigger softlockup), the statistics monitor didn't reflect the fact, which confused people. One example with our per-cgroup top, we ever noticed "20% usr, 80% sys" displayed while in fact the kernel was in some busy loop(100% sys) at that moment, and the tick based time are of course all sys samples in such case.
Peter Zijlstra
maintainerQuoted 11 lines · Xunlei Pang
If users access "
/proc/pid/stat", the utime and stime ratio in the current SAMPLE period are excepted, but currentlycputime_adjust()always calculates with the ratio of the WHOLE lifetime of the process.This results in inaccurate utime and stime in "
/proc/pid/stat". For example, a process runs for a while with "50% usr, 0% sys", then followed by "100% sys". For later while, the following is excepted:
0.0 usr, 100.0 sys
but we got:
10.0 usr, 90.0 sys
Quoted 4 lines
This patch uses the accurate ratio in
cputime_adjust()to address the issue. A newtask_cputimetype field is added inprev_cputimeto record previoustask_cputimeso that we can get the elapsed times as the accurate ratio.
This still does not explain anything...