sched, wait: Collapse __wait_event macros -v4
Subsystems (6)
Peter Zijlstra
maintainercover sched, wait: Collapse __wait_event macros -v4
While poking at the cpu hotplug implementation I had a desire to use wait_event() with schedule_preempt_disabled() and found the rediculous amount of duplication in linux/wait.h.
These patches attempt to cure some of this while also making it 'trivial' to generate new variants.
Compile and boot tested on x86_64.
Changes since -v3:
- added LKML to the CC (D'0h!!)
- introduced ___wait_cond_timeout() to avoid double evalutation of 'condition'
Changes since -v2:
- removed ___wait_cmd_lock_irq(), ___wait_cmd_lock_irq_timo()
- renamed ___wait_cmd_test_timeout() to ___wait_test_timeout()
- renamed ___wait_cmd_schedule_timeout() to ___wait_sched_timeout()
- fixed schedule_timeout() invocations in patch 6; don't restart a schedule_timeout() at the total timeout; rather continue where we left off for a total fixed timeout.
- use 'long' return type for ___wait_event() so that timeout values for schedule_timeout() fit.
- fix wait_event_interruptible_lock_irq_timeout() and use long return type.
--- arch/mips/kernel/rtlx.c | 19 +- include/linux/tty.h | 28 +--- include/linux/wait.h | 268 +++++++++++++--------------------------- net/irda/af_irda.c | 5 net/netfilter/ipvs/ip_vs_sync.c | 7 - 5 files changed, 108 insertions(+), 219 deletions(-)
Peter Zijlstra
maintainerPATCH 1/6 sched, wait: Make the signal_pending() checks consistent
There's two patterns to check signals in the __wait_event*() macros:
if (!signal_pending(current)) {
schedule();
continue; }
ret = -ERESTARTSYS;
break;
And the more natural:
if (signal_pending(current)) {
ret = -ERESTARTSYS;
break; }
schedule();
Change them all into the latter form.
--- include/linux/tty.h | 13 ++++++------- include/linux/wait.h | 35 ++++++++++++++++------------------- 2 files changed, 22 insertions(+), 26 deletions(-)
include/linux/tty.h +6 -7
@@ -686,14 +686,13 @@ do { \prepare_to_wait (&wq, &__wait,TASK_INTERRUPTIBLE ); \if (condition) \break ; \ -if (!signal_pending (current)) { \ -tty_unlock (tty); \ -schedule (); \ -tty_lock (tty); \ -continue ; \ +if (signal_pending (current)) { \ + ret = -ERESTARTSYS ; \ +break ; \ } \ - ret = -ERESTARTSYS ; \ -break ; \ +tty_unlock (tty); \ +schedule (); \ +tty_lock (tty); \ } \finish_wait (&wq, &__wait); \ }while (0 )
include/linux/wait.h +16 -19
@@ -261,12 +261,11 @@ do { \prepare_to_wait (&wq, &__wait,TASK_INTERRUPTIBLE ); \if (condition) \break ; \ -if (!signal_pending (current)) { \ -schedule (); \ -continue ; \ +if (signal_pending (current)) { \ + ret = -ERESTARTSYS ; \ +break ; \ } \ - ret = -ERESTARTSYS ; \ -break ; \ +schedule (); \ } \finish_wait (&wq, &__wait); \ }while (0 )
@@ -302,14 +301,13 @@ do { \prepare_to_wait (&wq, &__wait,TASK_INTERRUPTIBLE ); \if (condition) \break ; \ -if (!signal_pending (current)) { \ - ret =schedule_timeout (ret); \ -if (!ret) \ -break ; \ -continue ; \ +if (signal_pending (current)) { \ + ret = -ERESTARTSYS ; \ +break ; \ } \ - ret = -ERESTARTSYS ; \ -break ; \ + ret =schedule_timeout (ret); \ +if (!ret) \ +break ; \ } \if (!ret && (condition)) \ ret =1 ; \
@@ -439,14 +437,13 @@ do { \finish_wait (&wq, &__wait); \break ; \ } \ -if (!signal_pending (current)) { \ -schedule (); \ -continue ; \ - } \ - ret = -ERESTARTSYS ; \ -abort_exclusive_wait (&wq, &__wait, \ +if (signal_pending (current)) { \ + ret = -ERESTARTSYS ; \ +abort_exclusive_wait (&wq, &__wait, \TASK_INTERRUPTIBLE ,NULL ); \ -break ; \ +break ; \ + } \ +schedule (); \ } \ }while (0 )
Peter Zijlstra
maintainerPATCH 2/6 sched, wait: Change timeout logic
In reply to Peter ZijlstraCommit 4c663cf ("wait: fix false timeouts when using wait_event_timeout()") introduced an additional condition check after a timeout but there's a few issues;
- it forgot one site
- it put the check after the main loop; not at the actual timeout check.
Cure both; by wrapping the condition (as suggested by Oleg), this avoids double evaluation of 'condition' which could be quite big.
--- include/linux/wait.h | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
include/linux/wait.h +11 -13
@@ -179,6 +179,14 @@ wait_queue_head_t *bit_waitqueue(void *,#define wake_up_interruptible_sync_poll (x, m) \ __wake_up_sync_key((x), TASK_INTERRUPTIBLE, 1, (void *) (m)) +#define ___wait_cond_timeout (condition, ret) \ +({ \ + bool __cond = (condition); \ + if (__cond && !ret) \ + ret = 1; \ + __cond || !ret; \ +}) +#define __wait_event (wq, condition) \ do { \ DEFINE_WAIT(__wait); \
@@ -217,14 +225,10 @@ do { \ \for (;;) { \prepare_to_wait (&wq, &__wait,TASK_UNINTERRUPTIBLE ); \ -if (condition) \ +if (___wait_cond_timeout (condition, ret)) \break ; \ ret =schedule_timeout (ret); \ -if (!ret) \ -break ; \ } \ -if (!ret && (condition)) \ - ret =1 ; \finish_wait (&wq, &__wait); \ }while (0 )
@@ -299,18 +303,14 @@ do { \ \for (;;) { \prepare_to_wait (&wq, &__wait,TASK_INTERRUPTIBLE ); \ -if (condition) \ +if (___wait_cond_timeout (condition, ret)) \break ; \if (signal_pending (current)) { \ ret = -ERESTARTSYS ; \break ; \ } \ ret =schedule_timeout (ret); \ -if (!ret) \ -break ; \ } \ -if (!ret && (condition)) \ - ret =1 ; \finish_wait (&wq, &__wait); \ }while (0 )
@@ -815,7 +815,7 @@ do { \ \for (;;) { \prepare_to_wait (&wq, &__wait,TASK_INTERRUPTIBLE ); \ -if (condition) \ +if (___wait_cond_timeout (condition, ret)) \break ; \if (signal_pending (current)) { \ ret = -ERESTARTSYS ; \
@@ -824,8 +824,6 @@ do { \spin_unlock_irq (&lock); \ ret =schedule_timeout (ret); \spin_lock_irq (&lock); \ -if (!ret) \ -break ; \ } \finish_wait (&wq, &__wait); \ }while (0 )
Re: [PATCH 2/6] sched, wait: Change timeout logic
Quoted 2 lines · Peter Zijlstra
Commit 4c663cf ("wait: fix false timeouts when using
wait_event_timeout()")
Btw, unrelated to the patch itselt: you seem to use the old broken 7-character short SHA1 format.
Yes, 7 hex characters is generally still unique. But no, it won't stay so. The birthday paradox means that you start to get collisions more quickly than you'd think, and the original 7-character hex string default was done early in git development, when the BK repository was considered "big", and that one was approaching having 16-bit commit numbers.
Oh, how naive. In the eight+ years we've used git, we've flown past that old 16-bit limit in commit numbers (we're now around 400k commits), and we have over 3M objects. So we already have lots of collisions in 7 characters. Now, git is smart enough that when it picks a short form of the SHA1, it will never pick a collision, but since the tree keeps growing, they can happen later.
As a result, I currently strongly suggest kernel-developers do
git config --global core.abbrev 12
to set the default abbreviated SHA1 count to 12 characters, not the old default of 7. That pushes out the collision worry to the point where we're talking "later generations" rather than "forseeable future".
Linus
Peter Zijlstra
maintainerRe: [PATCH 2/6] sched, wait: Change timeout logic
Quoted 4 lines · Linus Torvalds
As a result, I currently strongly suggest kernel-developers do
git config --global core.abbrev 12
Done!
Peter Zijlstra
maintainerPATCH 3/6 sched, wait: Change the wait_exclusive control flow
In reply to Peter ZijlstraPurely a preparatory patch; it changes the control flow to match what will soon be generated by generic code so that that patch can be a unity transform.
--- include/linux/wait.h | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
include/linux/wait.h +5 -4
@@ -430,23 +430,24 @@ do { \#define __wait_event_interruptible_exclusive (wq, condition, ret) \ do { \ + __label__ __out; \ DEFINE_WAIT(__wait); \ \ for (;;) { \ prepare_to_wait_exclusive(&wq, &__wait, \ TASK_INTERRUPTIBLE); \ - if (condition) { \ - finish_wait(&wq, &__wait); \ + if (condition) \ break; \ - } \ if (signal_pending(current)) { \ ret = -ERESTARTSYS; \ abort_exclusive_wait(&wq, &__wait, \ TASK_INTERRUPTIBLE, NULL); \ - break; \ + goto __out; \ } \ schedule(); \ } \ + finish_wait(&wq, &__wait); \ +__out: ; \ } while (0)#define wait_event_interruptible_exclusive (wq, condition) \
Peter Zijlstra
maintainerPATCH 4/6 sched, wait: Collapse __wait_event macros
In reply to Peter ZijlstraThere's far too much duplication in the __wait_event macros; fix this.
With the previous patches changing the various __wait_event*() implementations to be more uniform; we can now collapse the lot without also changing generated code.
--- include/linux/tty.h | 21 +---- include/linux/wait.h | 192 +++++++++++++++------------------------------------ 2 files changed, 63 insertions(+), 150 deletions(-)
include/linux/tty.h +4 -17
@@ -679,23 +679,10 @@ static inline void tty_wait_until_sent_f })#define __wait_event_interruptible_tty (tty, wq, condition, ret) \ -do { \ - DEFINE_WAIT(__wait); \ - \ - for (;;) { \ - prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE); \ - if (condition) \ - break; \ - if (signal_pending(current)) { \ - ret = -ERESTARTSYS; \ - break; \ - } \ - tty_unlock(tty); \ - schedule(); \ - tty_lock(tty); \ - } \ - finish_wait(&wq, &__wait); \ -} while (0) + ___wait_event(wq, condition, TASK_INTERRUPTIBLE, 0, ret, \ + tty_unlock(tty); \ + schedule(); \ + tty_lock(tty))#ifdef CONFIG_PROC_FS extern void proc_tty_register_driver (struct tty_driver *);
include/linux/wait.h +59 -133
256 lines not on this page · raw message
Peter Zijlstra
maintainerPATCH 5/6 sched, wait: Also use ___wait_event() for __wait_event_hrtimeout
In reply to Peter ZijlstraThe actual core of __wait_event_hrtimeout() is identical to what ___wait_event() generates.
--- include/linux/wait.h | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
include/linux/wait.h +2 -13
@@ -338,7 +338,6 @@ do { \#define __wait_event_hrtimeout (wq, condition, timeout, state) \ ({ \ int __ret = 0; \ - DEFINE_WAIT(__wait); \ struct hrtimer_sleeper __t; \ \ hrtimer_init_on_stack(&__t.timer, CLOCK_MONOTONIC, \
@@ -349,25 +348,15 @@ do { \ current->timer_slack_ns, \ HRTIMER_MODE_REL); \ \ -for (;;) { \ -prepare_to_wait (&wq, &__wait, state); \ -if (condition) \ -break ; \ -if (state ==TASK_INTERRUPTIBLE && \ -signal_pending (current)) { \ - __ret = -ERESTARTSYS ; \ -break ; \ - } \ +___wait_event (wq, condition, state,0 , __ret, \ if (!__t.task) { \ __ret = -ETIME ; \break ; \ } \ -schedule (); \ - } \ + schedule()); \ \hrtimer_cancel (&__t.timer); \destroy_hrtimer_on_stack (&__t.timer); \ -finish_wait (&wq, &__wait); \ __ret; \ })
Peter Zijlstra
maintainerPATCH 6/6 sched, wait: Make the __wait_event*() interface more friendly
In reply to Peter ZijlstraChange all __wait_event*() implementations to match the corresponding wait_event*() signature for convenience.
In particular this does away with the weird 'ret' logic. Since there are __wait_event*() users this requires we update them too.
--- arch/mips/kernel/rtlx.c | 19 +++--- include/linux/tty.h | 10 +-- include/linux/wait.h | 113 +++++++++++++++++++--------------------- net/irda/af_irda.c | 5 - net/netfilter/ipvs/ip_vs_sync.c | 7 -- 5 files changed, 73 insertions(+), 81 deletions(-)
arch/mips/kernel/rtlx.c +9 -10
@@ -172,8 +172,9 @@ int rtlx_open(int index, int can_sleep)if (rtlx ==NULL ) {if ( (p =vpe_get_shared (tclimit)) ==NULL ) {if (can_sleep) { -__wait_event_interruptible (channel_wqs[index].lx_queue, - (p =vpe_get_shared (tclimit)), ret); + ret =__wait_event_interruptible ( + channel_wqs[index].lx_queue, + (p =vpe_get_shared (tclimit)));if (ret) goto out_fail; }else {
@@ -263,11 +264,10 @@ unsigned int rtlx_read_poll(int index, i/* data available to read? */ if (chan->lx_read == chan->lx_write) {if (can_sleep) { -int ret =0 ; - -__wait_event_interruptible (channel_wqs[index].lx_queue, +int ret =__wait_event_interruptible ( + channel_wqs[index].lx_queue, (chan->lx_read != chan->lx_write) || - sp_stopping, ret); + sp_stopping);if (ret)return ret;
@@ -440,14 +440,13 @@ static ssize_t file_write(struct file *f/* any space left... */ if (!rtlx_write_poll (minor)) { -int ret =0 ; +int ret;if (file->f_flags &O_NONBLOCK )return -EAGAIN ; -__wait_event_interruptible (channel_wqs[minor].rt_queue, -rtlx_write_poll (minor), - ret); + ret =__wait_event_interruptible (channel_wqs[minor].rt_queue, +rtlx_write_poll (minor));if (ret)return ret; }
include/linux/tty.h +5 -5
@@ -672,14 +672,14 @@ static inline void tty_wait_until_sent_f#define wait_event_interruptible_tty (tty, wq, condition) \ ({ \ int __ret = 0; \ - if (!(condition)) { \ - __wait_event_interruptible_tty(tty, wq, condition, __ret); \ - } \ + if (!(condition)) \ + __ret = __wait_event_interruptible_tty(tty, wq, \ + condition); \ __ret; \ }) -#define __wait_event_interruptible_tty (tty, wq, condition, ret) \ - ___wait_event(wq, condition, TASK_INTERRUPTIBLE, 0, ret, \ +#define __wait_event_interruptible_tty (tty, wq, condition) \ + ___wait_event(wq, condition, TASK_INTERRUPTIBLE, 0, 0, \ tty_unlock(tty); \ schedule(); \ tty_lock(tty))
include/linux/wait.h +55 -58
239 lines not on this page · raw message
net/irda/af_irda.c +2 -3
@@ -2563,9 +2563,8 @@ static int irda_getsockopt(struct socket jiffies + msecs_to_jiffies(val));/* Wait for IR-LMP to call us back */ -__wait_event_interruptible (self->query_wait, - (self->cachedaddr !=0 || self->errno == -ETIME ), - err); + err =__wait_event_interruptible (self->query_wait, + (self->cachedaddr !=0 || self->errno == -ETIME ));/* If watchdog is still activated, kill it! */ del_timer (&(self->watchdog));
net/netfilter/ipvs/ip_vs_sync.c +2 -5
@@ -1637,12 +1637,9 @@ static int sync_thread_master(void *datacontinue ; }while (ip_vs_send_sync_msg (tinfo->sock, sb->mesg) <0 ) { -int ret =0 ; - -__wait_event_interruptible (*sk_sleep (sk), +int ret =__wait_event_interruptible (*sk_sleep (sk),sock_writeable (sk) || -kthread_should_stop (), - ret); +kthread_should_stop ());if (unlikely (kthread_should_stop ())) goto done; }
Re: [PATCH 6/6] sched, wait: Make the __wait_event*() interface more friendly
Quoted 6 lines · Peter Zijlstra
Change all __wait_event*() implementations to match the corresponding wait_event*() signature for convenience.
In particular this does away with the weird 'ret' logic. Since there are __wait_event*() users this requires we update them too.
Quoted 8 lines
Signed-off-by Peter Zijlstra <[email protected]>--- arch/mips/kernel/rtlx.c | 19 +++--- include/linux/tty.h | 10 +-- include/linux/wait.h | 113 +++++++++++++++++++--------------------- net/irda/af_irda.c | 5 - net/netfilter/ipvs/ip_vs_sync.c | 7 -- 5 files changed, 73 insertions(+), 81 deletions(-)
Since now everyone seems to agree about this series, and since this particular patch changes generated code it would be really nice to split it into about ten per interface patches:
- __wait_event_interruptible()
- __wait_event_interruptible_timeout()
- __wait_event_interruptible_exclusive()
- __wait_event_interruptible_lock_irq()
- __wait_event_interruptible_tty()
- __wait_event()
- __wait_event_timeout()
- __wait_event_killable()
- __wait_event_interruptible_lock_irq_timeout()
The changes seem to be mostly isolated, they are even in separate patch hunks most of the time making a splitup relatively easy - with a handful of semantic interdependencies.
There's very little downside to doing it the split-up way, and there's a lot of bisection and bug review upside, should any of these changes prove problematic...
Thanks,
Ingo
Quoted 2 lines · Peter Zijlstra
Compile and boot tested on
x86_64.
Btw, I assume the odd binary size reduction is gone now, and code generation is generally identical?
Linus
Peter Zijlstra
maintainerQuoted 6 lines · Linus Torvalds
Quoted 2 lines · Peter Zijlstra
Compile and boot tested on
x86_64.Btw, I assume the odd binary size reduction is gone now, and code generation is generally identical?
Over patch 4; yes. Patches 1-3 generate different kernels, esp patch 2 has a large drop in size, 5 too is invariant. Patch 6 however increases code size again, but then it actually changes the generated code so that's somewhat expected.
But with patches 1-3 its all clear what changes and why; and having the big scary conversion patch 4 invariant is good.
Quoted 2 lines · Peter Zijlstra
Compile and boot tested on
x86_64.
Looks fine, but I'll try to actually read this series tomorrow. Not that I think this needs my review ;)
But at first glance, I think that something like below makes sense as 7/6.
Once again, of course I do not blame this series, but wait_event_timeout(wq, true, 0) still returns 0.
Oleg.
include/linux/wait.h +12 -7
@@ -184,9 +184,14 @@ wait_queue_head_t *bit_waitqueue(void *,bool __cond = (condition); \if (__cond && !__ret) \ __ret =1 ; \ - __cond || !__ret; \ + __cond; \ }) +#define ___wait_schedule_timeout (tout) \ + if (!tout) \ + break; \ + tout = schedule_timeout(tout) +#define ___wait_signal_pending (state) \ ((state == TASK_INTERRUPTIBLE && signal_pending(current)) || \ (state == TASK_KILLABLE && fatal_signal_pending(current)))
@@ -248,7 +253,7 @@ do { \#define __wait_event_timeout (wq, condition, timeout) \ ___wait_event(wq, ___wait_cond_timeout(condition), \ TASK_UNINTERRUPTIBLE, 0, timeout, \ - __ret = schedule_timeout(__ret)) + ___wait_schedule_timeout(__ret))/** * wait_event_timeout - sleep until a condition gets true or a timeout elapses
@@ -270,7 +275,7 @@ do { \#define wait_event_timeout (wq, condition, timeout) \ ({ \ long __ret = timeout; \ - if (!(condition)) \ + if (!___wait_cond_timeout(condition)) \ __ret = __wait_event_timeout(wq, condition, timeout); \ __ret; \ })
@@ -305,7 +310,7 @@ do { \#define __wait_event_interruptible_timeout (wq, condition, timeout) \ ___wait_event(wq, ___wait_cond_timeout(condition), \ TASK_INTERRUPTIBLE, 0, timeout, \ - __ret = schedule_timeout(__ret)) + ___wait_schedule_timeout(__ret))/** * wait_event_interruptible_timeout - sleep until a condition gets true or a timeout elapses
@@ -328,7 +333,7 @@ do { \#define wait_event_interruptible_timeout (wq, condition, timeout) \ ({ \ long __ret = timeout; \ - if (!(condition)) \ + if (!___wait_cond_timeout(condition)) \ __ret = __wait_event_interruptible_timeout(wq, \ condition, timeout); \ __ret; \
@@ -739,7 +744,7 @@ do { \___wait_event (wq,___wait_cond_timeout (condition), \TASK_INTERRUPTIBLE ,0 , ret, \ spin_unlock_irq(&lock); \ - __ret =schedule_timeout (__ret); \ +___wait_schedule_timeout (__ret); \ spin_lock_irq(&lock));/**
@@ -770,7 +775,7 @@ do { \ timeout) \ ({ \ long __ret = timeout; \ - if (!(condition)) \ + if (!___wait_cond_timeout(condition)) \ __ret = __wait_event_interruptible_lock_irq_timeout( \ wq, condition, lock, timeout); \ __ret; \
Quoted 3 lines · Oleg Nesterov
But at first glance, I think that something like below makes sense as 7/6.
Well, _something like_ perhaps,
include/linux/wait.h do { \ · line 747
Quoted 5 lines
include/linux/wait.h +1
-1___wait_event (wq,___wait_cond_timeout (condition), \TASK_INTERRUPTIBLE ,0 , ret, \ spin_unlock_irq(&lock); \ - __ret =schedule_timeout (__ret); \ +___wait_schedule_timeout (__ret); \
this is obviously wrong ;)
Oleg.
Peter Zijlstra
maintainerQuoted 2 lines · Oleg Nesterov
Once again, of course I do not blame this series, but
wait_event_timeout(wq, true, 0)still returns 0.
So we have:
#define ___wait_cond_timeout(condition) \
({ \
bool __cond = (condition); \
if (__cond && !__ret) \
__ret = 1; \
__cond || !__ret; \
})
#define ___wait_event(wq, condition, state, exclusive, ret, cmd) \
({ \
__label__ __out; \
DEFINE_WAIT(__wait); \
long __ret = ret; \
\
for (;;) { \
if (exclusive) \
prepare_to_wait_exclusive(&wq, &__wait, state); \
else \
prepare_to_wait(&wq, &__wait, state); \
\
if (condition) \
break; \
\
if (___wait_signal_pending(state)) { \
__ret = -ERESTARTSYS; \
if (exclusive) { \
abort_exclusive_wait(&wq, &__wait, \
state, NULL); \
goto __out; \
} \
break; \
} \
\
cmd; \
} \
finish_wait(&wq, &__wait); \__out: __ret; \
})
So wait_event_timeout(wq, true, 0) turns into:
({
DEFINE_WAIT(__wait);
long __ret = 0;
for (;;) {
prepare_to_wait(&wq, &__wait, TASK_UNINTERRUPTIBLE);
if (({
bool __cond = (true);
if (__cond && !__ret)
__ret = 1;
__cond || !__ret;
}))
break;
schedule();
}
finish_wait(&wq, &__wait);
__ret;
})Which; afaict, returns 1, not 0.
Review
Quoted 12 lines
(quoted diff) +6
-1+#define ___wait_schedule_timeout(tout) \ + if (!tout) \ + break; \ + tout = schedule_timeout(tout) + ___wait_event(wq, ___wait_cond_timeout(condition), \ TASK_INTERRUPTIBLE, 0, ret, \ spin_unlock_irq(&lock); \ - __ret = schedule_timeout(__ret); \ + ___wait_schedule_timeout(__ret); \ spin_lock_irq(&lock));
You can't do that; you'll break/return without the lock held.
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines · Oleg Nesterov
Once again, of course I do not blame this series, but
wait_event_timeout(wq, true, 0)still returns 0.So we have:
[...snip...]
So
wait_event_timeout(wq, true, 0)turns into:
Not really, because of fast-path check,
#define wait_event_timeout(wq, condition, timeout) \
({ \
long __ret = timeout; \
if (!(condition)) \
__ret = __wait_event_timeout(wq, condition, timeout); \
__ret; \
})we do not even call __wait_event_timeout() if "condition" is already true at the start.
Quoted 8 lines
include/linux/wait.h do { \ · line 748
Quoted 6 lines
include/linux/wait.h +1
-1___wait_event (wq,___wait_cond_timeout (condition), \TASK_INTERRUPTIBLE ,0 , ret, \ spin_unlock_irq(&lock); \ - __ret =schedule_timeout (__ret); \ +___wait_schedule_timeout (__ret); \ spin_lock_irq(&lock));You can't do that; you'll break/return without the lock held.
Yes, see another email, already noticed this.
And let me repeat just in case, I think this series is fine, just wait.h needs another minor/orthogonal fix.
Oleg.
Quoted 23 lines · Oleg Nesterov
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines · Oleg Nesterov
Once again, of course I do not blame this series, but
wait_event_timeout(wq, true, 0)still returns 0.So we have:
[...snip...]
So
wait_event_timeout(wq, true, 0)turns into:Not really, because of fast-path check,
#define wait_event_timeout(wq, condition, timeout) \ ({ \ long __ret = timeout; \ if (!(condition)) \ __ret = __wait_event_timeout(wq, condition, timeout); \ __ret; \ })
Quoted 2 lines
we do not even call
__wait_event_timeout()if "condition" is already true at the start.
But somehow I didn't realize that ___wait_cond_timeout() can be used as is, so the simple patch below should work?
Oleg.
include/linux/wait.h +3 -3
@@ -270,7 +270,7 @@ do { \#define wait_event_timeout (wq, condition, timeout) \ ({ \ long __ret = timeout; \ - if (!(condition)) \ + if (!___wait_cond_timeout(condition)) \ __ret = __wait_event_timeout(wq, condition, timeout); \ __ret; \ })
@@ -328,7 +328,7 @@ do { \#define wait_event_interruptible_timeout (wq, condition, timeout) \ ({ \ long __ret = timeout; \ - if (!(condition)) \ + if (!___wait_cond_timeout(condition)) \ __ret = __wait_event_interruptible_timeout(wq, \ condition, timeout); \ __ret; \
@@ -770,7 +770,7 @@ do { \ timeout) \ ({ \ long __ret = timeout; \ - if (!(condition)) \ + if (!___wait_cond_timeout(condition)) \ __ret = __wait_event_interruptible_lock_irq_timeout( \ wq, condition, lock, timeout); \ __ret; \
Peter Zijlstra
maintainerQuoted 2 lines · Oleg Nesterov
But somehow I didn't realize that
___wait_cond_timeout()can be used as is, so the simple patch below should work?
Yeah, should work.. But how often do people use timeout=0? Should we really care about that case to the effect of adding more code.
Quoted 6 lines · Peter Zijlstra
Quoted 2 lines · Oleg Nesterov
But somehow I didn't realize that
___wait_cond_timeout()can be used as is, so the simple patch below should work?Yeah, should work.. But how often do people use timeout=0?
I do not know. Perhaps never.
Quoted 2 lines
Should we
really care about that case to the effect of adding more code.
Again, I do not really know. But imo it would be better to fix this anyway, even if the problem is really minor. If nothing else, wait_event_timeout() and __wait_event_timeout() should have the same semantics.
And suppose that we ha a helper(timeout) which calls wait_event_timeout(), and checks the non-trivial condition inside. Now suppose that someone does
timeout = DEFAULT_TIMEOUT; if (option_nonblock) timeout = 0; ok = helper(timeout);
So do you think we should ignore this or I should send 7/6 with the changelog ?
(In fact I am going to send another patch on top of this series later.
At least, try to send for discussion because I know you dislike the idea to move the signal-pending checks out of line).
Oleg.
RFC 1/1 introduce prepare_to_wait_event()
In reply to Peter ZijlstraI didn't test is yet, and I did write the changelog.
But let me show the patch for review right now, before we all forget the details... On top of Peter's "wait: Collapse __wait_event macros -v4".
This patch moves the signal-pending checks and part of DEFINE_WAIT's code into the new helper: prepare_to_wait_event().
Yes, sure, prepare_to_wait_event() becomes a little bit slower than prepare_to_wait/prepare_to_wait_exclusive. But this is the slow path anyway, we are likely going to sleep. IMO, it is better to shrink .text, and on my build the difference is
- 5124686 2955056 10117120 18196862 115a97e vmlinux + 5123212 2955088 10117120 18195420 115a3dc vmlinux
The code with the patch is
#define ___wait_is_interruptible(state) \
(!__builtin_constant_p(state) || \
state == TASK_INTERRUPTIBLE || state == TASK_KILLABLE) \
#define ___wait_event(wq, condition, state, exclusive, ret, cmd) \
({ \
__label__ __out; \
wait_queue_t __wait; \
long __ret = ret; \
\
INIT_LIST_HEAD(&__wait.task_list); \
if (exclusive) \
__wait.flags = WQ_FLAG_EXCLUSIVE; \
else \
__wait.flags = 0; \
\
for (;;) { \
long intr = prepare_to_wait_event(&wq, &__wait, state); \
\
if (condition) \
break; \
\
if (___wait_is_interruptible(state) && intr) { \
__ret = intr; \
if (exclusive) { \
abort_exclusive_wait(&wq, &__wait, \
state, NULL); \
goto __out; \
} \
break; \
} \
\
cmd; \
} \
finish_wait(&wq, &__wait); \
__out: __ret; \
})Compiler should optimize out "long intr" if !interruptible/killable.
What do you think?
Oleg.
---
include/linux/wait.h +14 -10
@@ -187,27 +187,30 @@ wait_queue_head_t *bit_waitqueue(void *, int); __cond || !__ret; \ }) -#define ___wait_signal_pending (state) \ - ((state == TASK_INTERRUPTIBLE && signal_pending(current)) || \ - (state == TASK_KILLABLE && fatal_signal_pending(current))) +#define ___wait_is_interruptible (state) \ + (!__builtin_constant_p(state) || \ + state == TASK_INTERRUPTIBLE || state == TASK_KILLABLE) \#define ___wait_event (wq, condition, state, exclusive, ret, cmd) \ ({ \ __label__ __out; \ - DEFINE_WAIT(__wait); \ + wait_queue_t __wait; \ long __ret = ret; \ \ + INIT_LIST_HEAD(&__wait.task_list); \ + if (exclusive) \ + __wait.flags = WQ_FLAG_EXCLUSIVE; \ + else \ + __wait.flags = 0; \ + \ for (;;) { \ - if (exclusive) \ - prepare_to_wait_exclusive(&wq, &__wait, state); \ - else \ - prepare_to_wait(&wq, &__wait, state); \ + long intr = prepare_to_wait_event(&wq, &__wait, state); \ \ if (condition) \ break; \ \ - if (___wait_signal_pending(state)) { \ - __ret = -ERESTARTSYS; \ + if (___wait_is_interruptible(state) && intr) { \ + __ret = intr; \ if (exclusive) { \ abort_exclusive_wait(&wq, &__wait, \ state, NULL); \
@@ -794,6 +797,7 @@ extern long interruptible_sleep_on_timeout(wait_queue_head_t *q,*/ void prepare_to_wait (wait_queue_head_t *q,wait_queue_t *wait,int state);void prepare_to_wait_exclusive (wait_queue_head_t *q,wait_queue_t *wait,int state); +long prepare_to_wait_event (wait_queue_head_t *q,wait_queue_t *wait,int state);void finish_wait (wait_queue_head_t *q,wait_queue_t *wait);void abort_exclusive_wait (wait_queue_head_t *q,wait_queue_t *wait,unsigned int mode,void *key);
kernel/wait.c +24 -0
@@ -92,6 +92,30 @@ prepare_to_wait_exclusive(wait_queue_head_t *q, wait_queue_t *wait, int state) }EXPORT_SYMBOL (prepare_to_wait_exclusive); +long prepare_to_wait_event (wait_queue_head_t *q,wait_queue_t *wait,int state) +{ +unsigned long flags; + +if (signal_pending_state (state, current)) +return -ERESTARTSYS ; + + wait->private = current; + wait->func = autoremove_wake_function; + +spin_lock_irqsave (&q->lock, flags); +if (list_empty (&wait->task_list)) { +if (wait->flags &WQ_FLAG_EXCLUSIVE ) +__add_wait_queue_tail (q, wait); +else +__add_wait_queue (q, wait); + } +set_current_state (state); +spin_unlock_irqrestore (&q->lock, flags); + +return 0 ; +} +EXPORT_SYMBOL (prepare_to_wait_event); +/** * finish_wait - clean up after waiting in a queue * @q: waitqueue waited on
Peter Zijlstra
maintainerRe: [RFC] introduce prepare_to_wait_event()
Quoted 13 lines · Oleg Nesterov
This patch moves the signal-pending checks and part of DEFINE_WAIT's code into the new helper:
prepare_to_wait_event().Yes, sure,
prepare_to_wait_event()becomes a little bit slower than prepare_to_wait/prepare_to_wait_exclusive. But this is the slow path anyway, we are likely going to sleep. IMO, it is better to shrink .text, and on my build the difference is- 5124686 2955056 10117120 18196862 115a97e vmlinux + 5123212 2955088 10117120 18195420 115a3dc vmlinuxThe code with the patch is
Quoted 15 lines
#define ___wait_is_interruptible(state) \
(!__builtin_constant_p(state) || \
state == TASK_INTERRUPTIBLE || state == TASK_KILLABLE) \
#define ___wait_event(wq, condition, state, exclusive, ret, cmd) \
({ \
__label__ __out; \
wait_queue_t __wait; \
long __ret = ret; \
\
INIT_LIST_HEAD(&__wait.task_list); \
if (exclusive) \
__wait.flags = WQ_FLAG_EXCLUSIVE; \
else \
__wait.flags = 0; \__wait.flags = exclusive * WQ_FLAG_EXCLUSIVE;
or is that too obscure? ;-)
Quoted 3 lines
\
for (;;) { \
long intr = prepare_to_wait_event(&wq, &__wait, state); \int __intr = ...;
The interruptible bit doesn't actually need long; and local variables have __ prefixes in this context.
Quoted 20 lines
\
if (condition) \
break; \
\
if (___wait_is_interruptible(state) && intr) { \
__ret = intr; \
if (exclusive) { \
abort_exclusive_wait(&wq, &__wait, \
state, NULL); \
goto __out; \
} \
break; \
} \
\
cmd; \
} \
finish_wait(&wq, &__wait); \
__out: __ret; \
})Quoted 1 line
Compiler should optimize out "long intr" if !interruptible/killable.
Yeah, and I think even the if (0 && __intr) would suffice for the unused check; otherwise we'd have to adorn the thing with __maybe_unused.
Quoted 1 line
What do you think?
That would actually work I think.. the ___wait_is_interruptible() nicely does away with the unused code; the only slightly more expensive thing would be the prepare_to_wait_event() thing.
And if that really turns out to be a problem we could even re-use ___wait_is_interruptible() to call prepare_to_wait() instead.
Re: [RFC] introduce prepare_to_wait_event()
Quoted 10 lines · Peter Zijlstra
Quoted 4 lines · Oleg Nesterov
if (exclusive) \ __wait.flags = WQ_FLAG_EXCLUSIVE; \ else \ __wait.flags = 0; \__wait.flags = exclusive * WQ_FLAG_EXCLUSIVE;or is that too obscure? ;-)
I do not mind ;) The generated code should be the same.
Quoted 6 lines
Quoted 2 lines
for (;;) { \ long intr = prepare_to_wait_event(&wq, &__wait, state); \int __intr = ...;The interruptible bit doesn't actually need long;
Yes, it can be even "bool", but see below.
Quoted 2 lines
and local variables
have __ prefixes in this context.
Yes, yes, will fix.
Quoted 5 lines
Quoted 5 lines
if (condition) \ break; \ \ if (___wait_is_interruptible(state) && intr) { \ __ret = intr; \
Since typeof(__ret) == typeof(intr) gcc can (likely) simply do "mov r1, r2", so "long intr" make the code better.
I am not saying that "int intr" can make it worse, but to me "long" looks better in this context. But I wouldn't mind to change this.
Quoted 4 lines
Quoted 1 line
Compiler should optimize out "long intr" if !interruptible/killable.
Yeah, and I think even the if (0 &&
__intr) would suffice for the unused check; otherwise we'd have to adorn the thing with__maybe_unused.
Hmm yes, I didn't see any warning during the compilation, but perhaps __maybe_unused is needed, thanks.
Quoted 8 lines
Quoted 1 line
What do you think?
That would actually work I think.. the
___wait_is_interruptible()nicely does away with the unused code; the only slightly more expensive thing would be theprepare_to_wait_event()thing.And if that really turns out to be a problem we could even re-use
___wait_is_interruptible()to callprepare_to_wait()instead.
OK, thanks.
So I'll wait until your series is applied the resend it officially.
Oleg.
Peter Zijlstra
maintainerRe: [RFC] introduce prepare_to_wait_event()
Quoted 5 lines · Oleg Nesterov
Quoted 2 lines
Quoted 2 lines
if (___wait_is_interruptible(state) && intr) { \ __ret = intr; \Since
typeof(__ret)== typeof(intr) gcc can (likely) simply do "mov r1, r2", so "long intr" make the code better.
ok, fair enough; keep it long.