hrtimers: Special-case zero length sleeps
PATCH 1/1 hrtimers: Special-case zero length sleeps
sleep(0) is a common construct used by applications that want to trigger the scheduler. sched_yield() might make more sense, but only appeared in POSIX.1-2001 and so plenty of example code still uses the sleep(0) form. This wouldn't normally be a problem, but it means that event-driven applications that are merely trying to avoid starving other processes may actually end up sleeping due to having large timer_slack values. Special- casing this seems reasonable.
--- kernel/hrtimer.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-)
kernel/hrtimer.c +8 -0
@@ -1566,6 +1566,14 @@ long hrtimer_nanosleep(struct timespec *rqtp, struct timespec __user *rmtp,if (rt_task (current)) slack =0 ; +/* +* Applications will often sleep(0) to indicate that they wish to +* be scheduled. Special case that to avoid actually putting them +* to sleep for the duration of the slack. +*/ +if (rqtp->tv_sec ==0 && rqtp->tv_nsec ==0 ) + slack =0 ; +hrtimer_init_on_stack (&t.timer, clockid, mode);hrtimer_set_expires_range_ns (&t.timer,timespec_to_ktime (*rqtp), slack);if (do_nanosleep (&t, mode))
Signature · 1 line
1.7.6.4
Quoted 18 lines · Matthew Garrett
sleep(0) is a common construct used by applications that want to trigger the scheduler.
sched_yield()might make more sense, but only appeared in POSIX.1-2001 and so plenty of example code still uses the sleep(0) form. This wouldn't normally be a problem, but it means that event-driven applications that are merely trying to avoid starving other processes may actually end up sleeping due to having largetimer_slackvalues. Special- casing this seems reasonable.Signed-off-by Matthew Garrett <[email protected]>--- kernel/hrtimer.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-)kernel/hrtimer.c +0
-0
kernel/hrtimer.c long hrtimer_nanosleep(struct timespec *rqtp, struct timespec __user *rmtp, · line 1575
Quoted 10 lines
kernel/hrtimer.c +7
-0@@ -1566,6 +1566,14 @@ long hrtimer_nanosleep(struct timespec *rqtp, struct timespec __user *rmtp,if (rt_task (current)) slack =0 ; +/* +* Applications will often sleep(0) to indicate that they wish to +* be scheduled. Special case that to avoid actually putting them +* to sleep for the duration of the slack. +*/ +if (rqtp->tv_sec ==0 && rqtp->tv_nsec ==0 ) + slack =0 ;
That's pretty pointless. You can simply return 0 here as do_nanosleep() will not call the scheduler on an already expired timer, which is always true for a relative timer with delta 0.
Thanks,
tglx
Quoted 9 lines · Thomas Gleixner
kernel/hrtimer.c long hrtimer_nanosleep(struct timespec *rqtp, struct timespec __user *rmtp, · line 1575
Quoted 5 lines
kernel/hrtimer.c +5
-0+* be scheduled. Special case that to avoid actually putting them +* to sleep for the duration of the slack. +*/ +if (rqtp->tv_sec ==0 && rqtp->tv_nsec ==0 ) + slack =0 ;That's pretty pointless. You can simply return 0 here as
do_nanosleep()will not call the scheduler on an already expired timer, which is always true for a relative timer with delta 0.
I'm actually starting to wonder about the applications doing this. We default to adding a small amount of slack even if the application has done sleep(0), which will mean that the timer hasn't expired at this point. Do we then go through the scheduler differently? Are these applications actually relying on an invalid assumption?
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 11 lines · Matthew Garrett
Quoted 9 lines · Thomas Gleixner
kernel/hrtimer.c long hrtimer_nanosleep(struct timespec *rqtp, struct timespec __user *rmtp, · line 1575
Quoted 5 lines
kernel/hrtimer.c +5
-0+* be scheduled. Special case that to avoid actually putting them +* to sleep for the duration of the slack. +*/ +if (rqtp->tv_sec ==0 && rqtp->tv_nsec ==0 ) + slack =0 ;That's pretty pointless. You can simply return 0 here as
do_nanosleep()will not call the scheduler on an already expired timer, which is always true for a relative timer with delta 0.
Quoted 4 lines
I'm actually starting to wonder about the applications doing this. We default to adding a small amount of slack even if the application has done sleep(0), which will mean that the timer hasn't expired at this point. Do we then go through the scheduler differently? Are these
When the slack is large enough that the timer is actually not expired right away, which is usually the case, then we end up in schedule() and the task gets scheduled out until the timer fires. With your approach of making the slack 0 for sleep(0) calls the code does not call schedule() because the timer is definitely expired.
Quoted 1 line
applications actually relying on an invalid assumption?
Oh yes. sleep(0) has no guarantee about its behaviour at all. The only guarantee of sleep() is that it wont return before the requested time has elapsed, but there is no upper bound when it returns after the sleep time is over. So it's perfectly fine from the standards POV that sleep(0) actually sleeps and puts the tasks for some random time away. It's also correct when it returns right away w/o going through schedule(). The fact that sleep(0) ended up in schedule() even when the timer was already and the task state therefor was RUNNING on some unix implementations does not change that at all.
Just for the extended fun of it: The pre hrtimer implementation in Linux put the task on sleep as well up to the next jiffies boundary, so anything which used sleep(0) on a pre hrtimer kernel was going to sleep. That's also the case today when high resolution timers are disabled (compile or runtime).
So anything which relies on sleep(0) as a fast scheduling point is and has been broken forever.
Thanks,
tglx
Quoted 8 lines · Thomas Gleixner
Just for the extended fun of it: The pre hrtimer implementation in Linux put the task on sleep as well up to the next jiffies boundary, so anything which used sleep(0) on a pre hrtimer kernel was going to sleep. That's also the case today when high resolution timers are disabled (compile or runtime).
So anything which relies on sleep(0) as a fast scheduling point is and has been broken forever.
Excellent. So the real question is what /should/ sleep(0) do - nothing, schedule or sleep for an arbitrary period of time that could be years?
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 8 lines · Matthew Garrett
Quoted 6 lines · Thomas Gleixner
Just for the extended fun of it: The pre hrtimer implementation in Linux put the task on sleep as well up to the next jiffies boundary, so anything which used sleep(0) on a pre hrtimer kernel was going to sleep. That's also the case today when high resolution timers are disabled (compile or runtime).
Quoted 5 lines
Quoted 2 lines
So anything which relies on sleep(0) as a fast scheduling point is and has been broken forever.
Excellent. So the real question is what /should/ sleep(0) do - nothing, schedule or sleep for an arbitrary period of time that could be years?
Well, I don't expect slack to be set to years and I really don't want to special case sleep(0), because then we might end up discussing special casing usleep(1) or nanosleep(1ns) as well.
slack gets added to all timers, period.
You can tune the slack for your app if you think that it's necessary, but I bet that in 99,9% of the cases the programmer does not even know why the hell he put sleep(0), usleep(1) or nanosleep(1ns) into his code.
Thanks,
tglx
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · Matthew Garrett
Excellent. So the real question is what /should/ sleep(0) do - nothing, schedule or sleep for an arbitrary period of time that could be years?
Well, I don't expect slack to be set to years and I really don't want to special case sleep(0), because then we might end up discussing special casing usleep(1) or nanosleep(1ns) as well.
Increasing slack to the seconds range has measureable power management benefits, but there's some code that ends up broken as a result even when they're nominally event driven. I've no problem with us just declaring that code as broken, but it would be less effort to special case it. Application authors do seem to have ended up under the belief that sleep(0) is a meaningful thing to do, and the internet seems to be full of suggestions to use it rather than sched_yield().
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 3 lines · Matthew Garrett
Application authors do seem to have ended up under the belief that sleep(0) is a meaningful thing to do, and the internet seems to be full of suggestions to use it rather than
sched_yield().
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Matthew Garrett
Application authors do seem to have ended up under the belief that sleep(0) is a meaningful thing to do, and the internet seems to be full of suggestions to use it rather than
sched_yield().
I'm not even going to pretend to disagree. The question is whether we want to support their breakage anyway.
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 9 lines · Matthew Garrett
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · Matthew Garrett
Excellent. So the real question is what /should/ sleep(0) do - nothing, schedule or sleep for an arbitrary period of time that could be years?
Well, I don't expect slack to be set to years and I really don't want to special case sleep(0), because then we might end up discussing special casing usleep(1) or nanosleep(1ns) as well.
Quoted 7 lines
Increasing slack to the seconds range has measureable power management benefits, but there's some code that ends up broken as a result even when they're nominally event driven. I've no problem with us just declaring that code as broken, but it would be less effort to special case it. Application authors do seem to have ended up under the belief that sleep(0) is a meaningful thing to do, and the internet seems to be full of suggestions to use it rather than
sched_yield().
The internet is full of crappy suggestions written by absolutely clueless and advisory resistant morons.
Dammit, we cannot come up with a reasonable definition for special casing that stuff simply because you cannot draw a clear boundary what to special case and what not. And there is no sensible definition for what to do - return right away or go through schedule() or what ever.
sleep(0) is as pointless as sched_yield() and it's about time that we stop to create a fucking mess in the kernel just because user space programmers refuse to understand how an operating system works and how proper programming should be done.
Thanks,
tglx
Quoted 9 lines · Thomas Gleixner
Dammit, we cannot come up with a reasonable definition for special casing that stuff simply because you cannot draw a clear boundary what to special case and what not. And there is no sensible definition for what to do - return right away or go through
schedule()or what ever.sleep(0) is as pointless as
sched_yield()and it's about time that we stop to create a fucking mess in the kernel just because user space programmers refuse to understand how an operating system works and how proper programming should be done.
Ok. I'll work on finding how prevelant it is and see if we can kill it off in userspace.
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 2 lines · Matthew Garrett
Ok. I'll work on finding how prevelant it is and see if we can kill it off in userspace.
Checking through an exploded Fedora kernel tree suggests around 125 packages out of 11000 or so, so around 1% of userspace seems to use sleep(0) under certain circumstances. We can probably fix everything in the distribution, but that suggests that there's also going to be a significant amount of code in the outside world that's also broken.
Userspace clearly has an expectation that sleep(0) is magic in some ill-defined way. We'd be well within our rights to break that expectation, but I think it's common enough to warrant special casing.
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 3 lines
Userspace clearly has an expectation that sleep(0) is magic in some ill-defined way. We'd be well within our rights to break that expectation, but I think it's common enough to warrant special casing.
In historical Unix sleep(0) ends up the nearest equivalent it had to triggering a reschedule and giving up the rest of the timeslice.
I suspect special casing it as yield() isn't far from the right result ?
Alan
Quoted 8 lines · Alan Cox
Quoted 3 lines
Userspace clearly has an expectation that sleep(0) is magic in some ill-defined way. We'd be well within our rights to break that expectation, but I think it's common enough to warrant special casing.
In historical Unix sleep(0) ends up the nearest equivalent it had to triggering a reschedule and giving up the rest of the timeslice.
I suspect special casing it as
yield()isn't far from the right result ?
But why go that way? Using sleep(0) or yield() is pretty much always the wrong thing to do anyway, this is a great opportunity for all folks to find these sites and fix them.
Wasn't that what open-source is all about, doing the right thing?
Why should we care about obviously broken crap?
Furthermore, pushing slack to several seconds will also break stuff that needed those timers to expire sooner, who is going to fix that?
So we've got a stacking of two ill-considered things:
- applications using yield()/sleep(0)
- weirdos pushing timer slack to the seconds range
Individually both cause/are borkage, and now you want to add code to the kernel to mitigate some, but nowhere near all, of it?
What's next, we're actually going to give people their O_PONIES?
Quoted 11 lines · Peter Zijlstra
Quoted 4 lines · Alan Cox
In historical Unix sleep(0) ends up the nearest equivalent it had to triggering a reschedule and giving up the rest of the timeslice.
I suspect special casing it as
yield()isn't far from the right result ?But why go that way? Using sleep(0) or
yield()is pretty much always the wrong thing to do anyway, this is a great opportunity for all folks to find these sites and fix them.Wasn't that what open-source is all about, doing the right thing?
Doing the right thing if practical. How about we special-case for now with a once-per-process printk and kill it further down the line?
Quoted 4 lines
Why should we care about obviously broken crap?
Furthermore, pushing slack to several seconds will also break stuff that needed those timers to expire sooner, who is going to fix that?
The reason to change timer slack is because we're willing to break polling applications in order to gain power savings. The problem is that there are event-driven applications that are also going to be broken because some ridiculous proportion of userspace believes that sleep(0) is a thing that they can do in an event-driven application. The question is whether the cost of special-casing that in the kernel is more than fixing all of them.
Quoted 8 lines
So we've got a stacking of two ill-considered things:
- applications using yield()/sleep(0)
- weirdos pushing timer slack to the seconds rangeIndividually both cause/are borkage, and now you want to add code to the kernel to mitigate some, but nowhere near all, of it?
What's next, we're actually going to give people their
O_PONIES?
If we could give people O_PONIES then why wouldn't we? The only reason not to is because it costs too much elsewhere.
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 7 lines · Matthew Garrett
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Alan Cox
In historical Unix sleep(0) ends up the nearest equivalent it had to triggering a reschedule and giving up the rest of the timeslice.
I suspect special casing it as
yield()isn't far from the right result ?
Quoted 8 lines
Quoted 5 lines
But why go that way? Using sleep(0) or
yield()is pretty much always the wrong thing to do anyway, this is a great opportunity for all folks to find these sites and fix them.Wasn't that what open-source is all about, doing the right thing?
Doing the right thing if practical. How about we special-case for now with a once-per-process printk and kill it further down the line?
We've done this for stuff which was once kinda guaranteed (intentionally or not), but sleep(0) has never had any guarantee whatsoever. Quite the contrary it changed its behaviour several times over the last decades and there is no reason to manifest some weird ass semantics now.
Quoted 10 lines
Quoted 4 lines
Why should we care about obviously broken crap?
Furthermore, pushing slack to several seconds will also break stuff that needed those timers to expire sooner, who is going to fix that?
The reason to change timer slack is because we're willing to break polling applications in order to gain power savings. The problem is that there are event-driven applications that are also going to be broken because some ridiculous proportion of userspace believes that sleep(0) is a thing that they can do in an event-driven application. The question
If we are willing to break polling crap, then there is no fricking reason not to break lousy programmed "event driven" crap while we are at it.
Quoted 2 lines
is whether the cost of special-casing that in the kernel is more than fixing all of them.
The cost of special casing stuff is maintainence, inconsistance of interfaces and extended confusion. And each of that is worse than breaking some (i.e. 1% of the total app stack out there) weirdo apps.
Quoted 4 lines
Quoted 1 line
What's next, we're actually going to give people their
O_PONIES?If we could give people
O_PONIESthen why wouldn't we? The only reason not to is because it costs too much elsewhere.
Wrong. The reason is that O_PONIES is impossible to define and that's equally true for any exception to the (nano)sleep interface.
Thanks,
tglx
Quoted 5 lines · Thomas Gleixner
Quoted 3 lines · Matthew Garrett
sleep(0) is a common construct used by applications that want to trigger the scheduler.
sched_yield()might make more sense, but only appeared in POSIX.1-2001 and so plenty of example code still uses the sleep(0) form.
Note that sched_yield() is unspecified for SCHED_OTHER, so any application using it that is not a real-time application is outside spec anyway.
Furthermore, both sched_yield() and sleep(0) are fair indications the application is broken, wait for proper events, not random delays.
Quoted 4 lines
Quoted 4 lines
This wouldn't normally be a problem, but it means that event-driven applications that are merely trying to avoid starving other processes may actually end up sleeping due to having large
timer_slackvalues. Special- casing this seems reasonable.
Again, sleep granularity isn't specified and thus this behaviour isn't actually buggy, all the spec promises is we won't sleep shorter than asked.
Also, starvation avoidance isn't something that userspace should concern itself with.
Regardless of whether userspace should be concerning itself about this kind of thing or not, there's plenty of userspace that calls sleep(0) on the assumption that it'll get rescheduled. This makes using whole-system timer slack difficult, because there are some applications that do this even if they're event-driven and sleeping for significant lengths of time here breaks them. I'd certainly understand the argument for fixing userspace instead, but that's a massive task for something that's easily special-cased in the kernel.
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 3 lines · Matthew Garrett
Regardless of whether userspace should be concerning itself about this kind of thing or not, there's plenty of userspace that calls sleep(0) on the assumption that it'll get rescheduled. This makes using whole-system
Right. And it got always rescheduled because the old pre hrtimer implementation put it to sleep unconditionally. The hrtimer code before we introduced the slack stuff returned right away when the relative delta was 0. Slack changed that back to the pre hrtimer behaviour. Also if you disable high res timers (compile or runtime) you go to sleep unconditionally until the next jiffies boundary.
Quoted 3 lines
timer slack difficult, because there are some applications that do this even if they're event-driven and sleeping for significant lengths of time here breaks them. I'd certainly understand the argument for fixing
What is significant? If the app relies on sleep(0) returning instantly, then it's going to malfunction on a highres=n kernel with HZ=100 as well. Also when there are actually other runnable tasks and it gets scheduled away, then there is no guarantee that it comes back to the cpu within a defined boundary.
Quoted 2 lines
userspace instead, but that's a massive task for something that's easily special-cased in the kernel.
So what's the correct special case solution? Return right away, call schedule() with state RUNNING or some other magic crap?
Again, if a SCHED_OTHER task cannot cope with the fact that it gets scheduled away for unbound amount of time, then changing the behaviour of sleep(0) to some magic yield() variant does not help at all. It's still broken and no special case in the kernel can fix that.
Thanks,
tglx