Incorrect CPU process accounting using CONFIG_HZ=100
Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for sched.c?
Thanks!
--
Al
Quoted 8 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for
sched.c?
Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)
Quoted 1 line
Thanks!
Jan Engelhardt
Quoted 10 lines · Jan Engelhardt
Quoted 6 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for
sched.c?Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)
That's what I thought for a long time. But at closer inspection, top d.1 slows down other apps by about the same amount of time at 1000Hz and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
Thanks!
--
Al
Quoted 8 lines · Al Boldi
Quoted 7 lines · Jan Engelhardt
Quoted 6 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for
sched.c?
Quoted 7 lines
Quoted 3 lines
Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)That's what I thought for a long time. But at closer inspection, top d.1 slows down other apps by about the same amount of time at 1000Hz and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
It is not a bug... it is design decision. If you eat "too little" cpu time, you'll be accouted 0 msec. That's what happens at 100Hz...
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 13 lines · Pavel Machek
Quoted 12 lines · Al Boldi
Quoted 10 lines · Jan Engelhardt
Quoted 6 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for
sched.c?Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)
Quoted 6 lines
Quoted 3 lines
That's what I thought for a long time. But at closer inspection, top d.1 slows down other apps by about the same amount of time at 1000Hz and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
It is not a bug... it is design decision. If you eat "too little" cpu time, you'll be accouted 0 msec. That's what happens at 100Hz...
Bummer!
Meanwhile, can't "too little" cpu time be made relative to CONFIG_HZ?
Thanks!
--
Al
Quoted 19 lines · Al Boldi
Quoted 18 lines · Pavel Machek
Quoted 16 lines · Al Boldi
Quoted 10 lines · Jan Engelhardt
Quoted 6 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for
sched.c?Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)That's what I thought for a long time. But at closer inspection, top d.1 slows down other apps by about the same amount of time at 1000Hz and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
Quoted 6 lines
Quoted 2 lines
It is not a bug... it is design decision. If you eat "too little" cpu time, you'll be accouted 0 msec. That's what happens at 100Hz...
Bummer!
Meanwhile, can't "too little" cpu time be made relative to
CONFIG_HZ?
It is and that's what you're perceiving as the problem. We only charge tasks in ticks and it's the tick size they get charged with. So at 100HZ if a task is running when a tick fires it gets charged 1% cpu. If it runs for 100 ticks it gets charged with 100% cpu. At 1000HZ it gets charged .1% cpu per tick and so on. The actual problem is that tasks only get charged if they happen to be running at the precise moment the tick fires. Now you could increase the accuracy of this timekeeping but it is expensive and this is exactly the sort of thing that we're saving cpu resources on by running at 100HZ (one of many).
Signature · 1 line
-ck
Quoted 17 lines · Con Kolivas
Quoted 16 lines · Al Boldi
Quoted 15 lines · Pavel Machek
Quoted 14 lines · Al Boldi
Quoted 9 lines · Jan Engelhardt
Quoted 6 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for
sched.c?Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)That's what I thought for a long time. But at closer inspection, top d.1 slows down other apps by about the same amount of time at 1000Hz and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
It is not a bug... it is design decision. If you eat "too little" cpu time, you'll be accouted 0 msec. That's what happens at 100Hz...
Bummer!
Meanwhile, can't "too little" cpu time be made relative to
CONFIG_HZ?It is and that's what you're perceiving as the problem. We only charge tasks in ticks and it's the tick size they get charged with. So at 100HZ if a task is running when a tick fires it gets charged 1% cpu. If it runs for 100 ticks it gets charged with 100% cpu. At 1000HZ it gets charged .1% cpu per tick and so on. The actual problem is that tasks only get charged if they happen to be running at the precise moment the tick fires. Now you could increase the accuracy of this timekeeping but it is expensive and this is exactly the sort of thing that we're saving cpu resources on by running at 100HZ (one of many).
It could be (partly) done fairly cheaply in nanoseconds if sched_clock() was reliable enough (but comments on this mail list indicate that it currently isn't) as it is already called in all the right places for getting the total cpu time used (so just a subtraction, addition and assignment). The reason that I say partly is that splitting the time into "system" and "user" would be a more complex problem.
Peter
PS It's also called already at appropriate places for measuring total sleep and total cpu delay time (i.e. time spent on a run queue waiting to get on to a CPU). PPS The task time stamp is also already updated at all the appropriate places for use in this mechanism.
Signature · 4 lines
Peter Williams [email protected] "Learning, n. The kind of ignorance distinguishing the studious." -- Ambrose Bierce
Quoted 32 lines · Peter Williams
Quoted 23 lines · Con Kolivas
Quoted 22 lines · Al Boldi
Quoted 20 lines · Pavel Machek
Quoted 8 lines · Al Boldi
Quoted 7 lines · Jan Engelhardt
Quoted 6 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Will this bug have consequences for
sched.c?Quoted 8 lines
Quoted 3 lines
Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)That's what I thought for a long time. But at closer inspection, top d.1 slows down other apps by about the same amount of time at 1000Hz and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
It is not a bug... it is design decision. If you eat "too little" cpu time, you'll be accouted 0 msec. That's what happens at 100Hz...
Quoted 7 lines
Quoted 2 lines
Bummer!
The actual problem is that tasks
only get charged if they happen to be running at the precise moment the tick fires. Now you could increase the accuracy of this timekeeping but it is expensive and this is exactly the sort of thing that we're saving cpu resources on by running at 100HZ (one of many).
Quoted 6 lines
It could be (partly) done fairly cheaply in nanoseconds if
sched_clock()was reliable enough (but comments on this mail list indicate that it currently isn't) as it is already called in all the right places for getting the total cpu time used (so just a subtraction, addition and assignment). The reason that I say partly is that splitting the time into "system" and "user" would be a more complex problem.
If I am reading this correctly, then the kernel is accounting process times twice:
1. for external proc monitoring, using a probed approach
2. for scheduling, using an inlined approach
Wouldn't merging the two approaches be in the interest of conserving cpu resources, while at the same time reflecting an accurate view of cpu utilization?
Thanks!
--
Al
Quoted 35 lines · Al Boldi
Quoted 27 lines · Peter Williams
Quoted 19 lines · Con Kolivas
Quoted 18 lines · Al Boldi
Quoted 17 lines · Pavel Machek
Quoted 6 lines · Al Boldi
Quoted 5 lines · Jan Engelhardt
Quoted 5 lines
Setting
CONFIG_HZ=100results in incorrect CPU process accounting.This can be seen running top d.1, that shows top, itself, consuming 0ms CPUtime.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Will this bug have consequences for
sched.c?Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)That's what I thought for a long time. But at closer inspection, top d.1 slows down other apps by about the same amount of time at 1000Hz and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
It is not a bug... it is design decision. If you eat "too little" cpu time, you'll be accouted 0 msec. That's what happens at 100Hz...
Quoted 7 lines
Quoted 2 lines
Bummer!
The actual problem is that tasks
only get charged if they happen to be running at the precise moment the tick fires. Now you could increase the accuracy of this timekeeping but it is expensive and this is exactly the sort of thing that we're saving cpu resources on by running at 100HZ (one of many).Quoted 6 lines
It could be (partly) done fairly cheaply in nanoseconds if
sched_clock()was reliable enough (but comments on this mail list indicate that it currently isn't) as it is already called in all the right places for getting the total cpu time used (so just a subtraction, addition and assignment). The reason that I say partly is that splitting the time into "system" and "user" would be a more complex problem.
Quoted 3 lines
If I am reading this correctly, then the kernel is accounting process times twice:
1. for external proc monitoring, using a probed approach
2. for scheduling, using an inlined approach
Not exactly (e.g. there's no separation between user and sys time available in line) but the possibilities are there.
Quoted 2 lines
Wouldn't merging the two approaches be in the interest of conserving cpu resources, while at the same time reflecting an accurate view of cpu utilization?
I think that this would be a worthwhile endeavour once/if sched_clock() is fixed. This is especially the case as CPUs get faster as many tasks may run to completion in less than a tick.
Peter
Signature · 4 lines
Peter Williams [email protected] "Learning, n. The kind of ignorance distinguishing the studious." -- Ambrose Bierce
Quoted 7 lines · Peter Williams
Quoted 5 lines · Al Boldi
Peter Williams wrote:
twice:
1. for external proc monitoring, using a probed approach
2. for scheduling, using an inlined approach
Quoted 6 lines
Not exactly (e.g. there's no separation between user and sys time available in line) but the possibilities are there.
Quoted 2 lines
Wouldn't merging the two approaches be in the interest of conserving cpu resources, while at the same time reflecting an accurate view of cpu utilization?
I think that this would be a worthwhile endeavour once/if
sched_clock()is fixed. This is especially the case as CPUs get faster as many tasks may run to completion in less than a tick.
That may not be as simple as it seems. To properly account system v user time using the sched_clock we'd have to hook into arch dependant asm code to know when entering and exiting kernel context. That is far more invasive than the simple on/off runqueue timing we currently do for scheduling accounting.
--
-ck
Quoted 9 lines · Con Kolivas
Quoted 7 lines · Peter Williams
Quoted 5 lines · Al Boldi
Peter Williams wrote:
twice:
1. for external proc monitoring, using a probed approach
2. for scheduling, using an inlined approach
Quoted 8 lines
Quoted 6 lines
Not exactly (e.g. there's no separation between user and sys time available in line) but the possibilities are there.
Quoted 2 lines
Wouldn't merging the two approaches be in the interest of conserving cpu resources, while at the same time reflecting an accurate view of cpu utilization?
I think that this would be a worthwhile endeavour once/if
sched_clock()is fixed. This is especially the case as CPUs get faster as many tasks may run to completion in less than a tick.That may not be as simple as it seems. To properly account system v user time using the
sched_clockwe'd have to hook into arch dependant asm code to know when entering and exiting kernel context. That is far more invasive than the simple on/off runqueue timing we currently do for scheduling accounting.
Yes, it is a problem and we may have to do something approximate like counting ticks for sys time and subtracting that from the total to get user time when reporting the times to user space (only a bit more complex to make sure we don't end up with negative times).
How is it intended to handle this problem in the tickless kernel?
Peter
PS It's all moot until sched_clock() is fixed anyway.
PPS Recent kernels (-mm ones at least) keep a sched_time in nsecs for each task but I've no idea what it's used for.
Signature · 4 lines
Peter Williams [email protected] "Learning, n. The kind of ignorance distinguishing the studious." -- Ambrose Bierce
Quoted 15 lines · Peter Williams
Quoted 14 lines · Con Kolivas
Quoted 12 lines · Peter Williams
Quoted 5 lines · Al Boldi
Peter Williams wrote:
twice:
1. for external proc monitoring, using a probed approach
2. for scheduling, using an inlined approachNot exactly (e.g. there's no separation between user and sys time available in line) but the possibilities are there.
Quoted 2 lines
Wouldn't merging the two approaches be in the interest of conserving cpu resources, while at the same time reflecting an accurate view of cpu utilization?
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
I think that this would be a worthwhile endeavour once/if
sched_clock()is fixed. This is especially the case as CPUs get faster as many tasks may run to completion in less than a tick.That may not be as simple as it seems. To properly account system v user time using the
sched_clockwe'd have to hook into arch dependant asm code to know when entering and exiting kernel context. That is far more invasive than the simple on/off runqueue timing we currently do for scheduling accounting.Yes, it is a problem and we may have to do something approximate like counting ticks for sys time and subtracting that from the total to get user time when reporting the times to user space (only a bit more complex to make sure we don't end up with negative times).
How is it intended to handle this problem in the tickless kernel?
The tickless name is a misnomer. The implemenation I had and the one that tglx/mingo are maintaining still tick but they skip idle ticks. This means that all we need to do is add the number of skipped idle ticks whenever we tick to the idle count.
--
-ck
Hi!
Quoted 14 lines
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
Wouldn't merging the two approaches be in the interest of conserving cpu resources, while at the same time reflecting an accurate view of cpu utilization?
I think that this would be a worthwhile endeavour once/if
sched_clock()is fixed. This is especially the case as CPUs get faster as many tasks may run to completion in less than a tick.Quoted 5 lines
That may not be as simple as it seems. To properly account system v user time using the
sched_clockwe'd have to hook into arch dependant asm code to know when entering and exiting kernel context. That is far more invasive than the simple on/off runqueue timing we currently do for scheduling accounting.
Quoted 4 lines
Yes, it is a problem and we may have to do something approximate like counting ticks for sys time and subtracting that from the total to get user time when reporting the times to user space (only a bit more complex to make sure we don't end up with negative times).
Yes, please.
Quoted 1 line
How is it intended to handle this problem in the tickless kernel?
Even our "tickless" kernels do tick while CPU is busy.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html