Skip to contentSkip to conversation

sched, wait: Collapse __wait_event macros -v4

PATCH In review Peter Zijlstra 4 participants 23 messages Sep 30, 2013 – Oct 1, 2013

Subsystems (6)
Networking MIPS netfilter Scheduler +2 more subsystems
lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

cover sched, wait: Collapse __wait_event macros -v4

While poking at the cpu hotplug implementation I had a desire to use wait_event() with schedule_preempt_disabled() and found the rediculous amount of duplication in linux/wait.h.

These patches attempt to cure some of this while also making it 'trivial' to generate new variants.

Compile and boot tested on x86_64.

Changes since -v3:
- added LKML to the CC (D'0h!!)
- introduced ___wait_cond_timeout() to avoid double evalutation of 'condition'

Changes since -v2:
- removed ___wait_cmd_lock_irq(), ___wait_cmd_lock_irq_timo()
- renamed ___wait_cmd_test_timeout() to ___wait_test_timeout()
- renamed ___wait_cmd_schedule_timeout() to ___wait_sched_timeout()
- fixed schedule_timeout() invocations in patch 6; don't restart a schedule_timeout() at the total timeout; rather continue where we left off for a total fixed timeout.
- use 'long' return type for ___wait_event() so that timeout values for schedule_timeout() fit.
- fix wait_event_interruptible_lock_irq_timeout() and use long return type.

---
 arch/mips/kernel/rtlx.c         |   19 +-
 include/linux/tty.h             |   28 +---
 include/linux/wait.h            |  268 +++++++++++++---------------------------
 net/irda/af_irda.c              |    5 
 net/netfilter/ipvs/ip_vs_sync.c |    7 -
 5 files changed, 108 insertions(+), 219 deletions(-)
author maintainer
#1 lore raw

PATCH 1/6 sched, wait: Make the signal_pending() checks consistent

There's two patterns to check signals in the __wait_event*() macros:

  if (!signal_pending(current)) {
	schedule();
	continue;

}
ret = -ERESTARTSYS;
break;

And the more natural:

  if (signal_pending(current)) {
	ret = -ERESTARTSYS;
	break;

}
schedule();

Change them all into the latter form.

---
 include/linux/tty.h  |   13 ++++++-------
 include/linux/wait.h |   35 ++++++++++++++++-------------------
 2 files changed, 22 insertions(+), 26 deletions(-)
include/linux/tty.h +6 -7
@@ -686,14 +686,13 @@ do {									\
 		prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE);	\
 		if (condition)						\
 			break;						\
-		if (!signal_pending(current)) {				\
-			tty_unlock(tty);					\
-			schedule();					\
-			tty_lock(tty);					\
-			continue;					\
+		if (signal_pending(current)) {				\
+			ret = -ERESTARTSYS;				\
+			break;						\
 		}							\
-		ret = -ERESTARTSYS;					\
-		break;							\
+		tty_unlock(tty);					\
+		schedule();						\
+		tty_lock(tty);						\
 	}								\
 	finish_wait(&wq, &__wait);					\
 } while (0)
include/linux/wait.h +16 -19
@@ -261,12 +261,11 @@ do {									\
 		prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE);	\
 		if (condition)						\
 			break;						\
-		if (!signal_pending(current)) {				\
-			schedule();					\
-			continue;					\
+		if (signal_pending(current)) {				\
+			ret = -ERESTARTSYS;				\
+			break;						\
 		}							\
-		ret = -ERESTARTSYS;					\
-		break;							\
+		schedule();						\
 	}								\
 	finish_wait(&wq, &__wait);					\
 } while (0)
@@ -302,14 +301,13 @@ do {									\
 		prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE);	\
 		if (condition)						\
 			break;						\
-		if (!signal_pending(current)) {				\
-			ret = schedule_timeout(ret);			\
-			if (!ret)					\
-				break;					\
-			continue;					\
+		if (signal_pending(current)) {				\
+			ret = -ERESTARTSYS;				\
+			break;						\
 		}							\
-		ret = -ERESTARTSYS;					\
-		break;							\
+		ret = schedule_timeout(ret);				\
+		if (!ret)						\
+			break;						\
 	}								\
 	if (!ret && (condition))					\
 		ret = 1;						\
@@ -439,14 +437,13 @@ do {									\
 			finish_wait(&wq, &__wait);			\
 			break;						\
 		}							\
-		if (!signal_pending(current)) {				\
-			schedule();					\
-			continue;					\
-		}							\
-		ret = -ERESTARTSYS;					\
-		abort_exclusive_wait(&wq, &__wait, 			\
+		if (signal_pending(current)) {				\
+			ret = -ERESTARTSYS;				\
+			abort_exclusive_wait(&wq, &__wait, 		\
 				TASK_INTERRUPTIBLE, NULL);		\
-		break;							\
+			break;						\
+		}							\
+		schedule();						\
 	}								\
 } while (0)
 
author maintainer
#2 lore raw

PATCH 2/6 sched, wait: Change timeout logic

In reply to Peter Zijlstra

Commit 4c663cf ("wait: fix false timeouts when using wait_event_timeout()") introduced an additional condition check after a timeout but there's a few issues;

- it forgot one site
- it put the check after the main loop; not at the actual timeout check.

Cure both; by wrapping the condition (as suggested by Oleg), this avoids double evaluation of 'condition' which could be quite big.

---
 include/linux/wait.h |   24 +++++++++++-------------
 1 file changed, 11 insertions(+), 13 deletions(-)
include/linux/wait.h +11 -13
@@ -179,6 +179,14 @@ wait_queue_head_t *bit_waitqueue(void *,
 #define wake_up_interruptible_sync_poll(x, m)				\
 	__wake_up_sync_key((x), TASK_INTERRUPTIBLE, 1, (void *) (m))
 
+#define ___wait_cond_timeout(condition, ret)				\
+({									\
+ 	bool __cond = (condition);					\
+ 	if (__cond && !ret)						\
+ 		ret = 1;						\
+ 	__cond || !ret;							\
+})
+
 #define __wait_event(wq, condition) 					\
 do {									\
 	DEFINE_WAIT(__wait);						\
@@ -217,14 +225,10 @@ do {									\
 									\
 	for (;;) {							\
 		prepare_to_wait(&wq, &__wait, TASK_UNINTERRUPTIBLE);	\
-		if (condition)						\
+		if (___wait_cond_timeout(condition, ret))		\
 			break;						\
 		ret = schedule_timeout(ret);				\
-		if (!ret)						\
-			break;						\
 	}								\
-	if (!ret && (condition))					\
-		ret = 1;						\
 	finish_wait(&wq, &__wait);					\
 } while (0)
 
@@ -299,18 +303,14 @@ do {									\
 									\
 	for (;;) {							\
 		prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE);	\
-		if (condition)						\
+		if (___wait_cond_timeout(condition, ret))		\
 			break;						\
 		if (signal_pending(current)) {				\
 			ret = -ERESTARTSYS;				\
 			break;						\
 		}							\
 		ret = schedule_timeout(ret);				\
-		if (!ret)						\
-			break;						\
 	}								\
-	if (!ret && (condition))					\
-		ret = 1;						\
 	finish_wait(&wq, &__wait);					\
 } while (0)
 
@@ -815,7 +815,7 @@ do {									\
 									\
 	for (;;) {							\
 		prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE);	\
-		if (condition)						\
+		if (___wait_cond_timeout(condition, ret))		\
 			break;						\
 		if (signal_pending(current)) {				\
 			ret = -ERESTARTSYS;				\
@@ -824,8 +824,6 @@ do {									\
 		spin_unlock_irq(&lock);					\
 		ret = schedule_timeout(ret);				\
 		spin_lock_irq(&lock);					\
-		if (!ret)						\
-			break;						\
 	}								\
 	finish_wait(&wq, &__wait);					\
 } while (0)
#3 lore raw

Re: [PATCH 2/6] sched, wait: Change timeout logic

Quoted 2 lines · Peter Zijlstra

Commit 4c663cf ("wait: fix false timeouts when using wait_event_timeout()")

Btw, unrelated to the patch itselt: you seem to use the old broken 7-character short SHA1 format.

Yes, 7 hex characters is generally still unique. But no, it won't stay so. The birthday paradox means that you start to get collisions more quickly than you'd think, and the original 7-character hex string default was done early in git development, when the BK repository was considered "big", and that one was approaching having 16-bit commit numbers.

Oh, how naive. In the eight+ years we've used git, we've flown past that old 16-bit limit in commit numbers (we're now around 400k commits), and we have over 3M objects. So we already have lots of collisions in 7 characters. Now, git is smart enough that when it picks a short form of the SHA1, it will never pick a collision, but since the tree keeps growing, they can happen later.

As a result, I currently strongly suggest kernel-developers do

    git config --global core.abbrev 12

to set the default abbreviated SHA1 count to 12 characters, not the old default of 7. That pushes out the collision worry to the point where we're talking "later generations" rather than "forseeable future".

Linus

author maintainer
#4 lore raw

Re: [PATCH 2/6] sched, wait: Change timeout logic

Quoted 4 lines · Linus Torvalds

As a result, I currently strongly suggest kernel-developers do

    git config --global core.abbrev 12

Done!

author maintainer
#5 lore raw

PATCH 3/6 sched, wait: Change the wait_exclusive control flow

In reply to Peter Zijlstra

Purely a preparatory patch; it changes the control flow to match what will soon be generated by generic code so that that patch can be a unity transform.

---
 include/linux/wait.h |    9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
include/linux/wait.h +5 -4
@@ -430,23 +430,24 @@ do {									\
 
 #define __wait_event_interruptible_exclusive(wq, condition, ret)	\
 do {									\
+	__label__ __out;						\
 	DEFINE_WAIT(__wait);						\
 									\
 	for (;;) {							\
 		prepare_to_wait_exclusive(&wq, &__wait,			\
 					TASK_INTERRUPTIBLE);		\
-		if (condition) {					\
-			finish_wait(&wq, &__wait);			\
+		if (condition)						\
 			break;						\
-		}							\
 		if (signal_pending(current)) {				\
 			ret = -ERESTARTSYS;				\
 			abort_exclusive_wait(&wq, &__wait, 		\
 				TASK_INTERRUPTIBLE, NULL);		\
-			break;						\
+			goto __out;					\
 		}							\
 		schedule();						\
 	}								\
+	finish_wait(&wq, &__wait);					\
+__out:	;								\
 } while (0)
 
 #define wait_event_interruptible_exclusive(wq, condition)		\
author maintainer
#6 lore raw

PATCH 4/6 sched, wait: Collapse __wait_event macros

In reply to Peter Zijlstra

There's far too much duplication in the __wait_event macros; fix this.

With the previous patches changing the various __wait_event*() implementations to be more uniform; we can now collapse the lot without also changing generated code.

---
 include/linux/tty.h  |   21 +----
 include/linux/wait.h |  192 +++++++++++++++------------------------------------
 2 files changed, 63 insertions(+), 150 deletions(-)
include/linux/tty.h +4 -17
@@ -679,23 +679,10 @@ static inline void tty_wait_until_sent_f
 })
 
 #define __wait_event_interruptible_tty(tty, wq, condition, ret)		\
-do {									\
-	DEFINE_WAIT(__wait);						\
-									\
-	for (;;) {							\
-		prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE);	\
-		if (condition)						\
-			break;						\
-		if (signal_pending(current)) {				\
-			ret = -ERESTARTSYS;				\
-			break;						\
-		}							\
-		tty_unlock(tty);					\
-		schedule();						\
-		tty_lock(tty);						\
-	}								\
-	finish_wait(&wq, &__wait);					\
-} while (0)
+	___wait_event(wq, condition, TASK_INTERRUPTIBLE, 0, ret,	\
+			tty_unlock(tty);				\
+			schedule();					\
+			tty_lock(tty))
 
 #ifdef CONFIG_PROC_FS
 extern void proc_tty_register_driver(struct tty_driver *);
include/linux/wait.h +59 -133

256 lines not on this page · raw message

author maintainer
#7 lore raw

PATCH 5/6 sched, wait: Also use ___wait_event() for __wait_event_hrtimeout

In reply to Peter Zijlstra

The actual core of __wait_event_hrtimeout() is identical to what ___wait_event() generates.

---
 include/linux/wait.h |   15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)
include/linux/wait.h +2 -13
@@ -338,7 +338,6 @@ do {									\
 #define __wait_event_hrtimeout(wq, condition, timeout, state)		\
 ({									\
 	int __ret = 0;							\
-	DEFINE_WAIT(__wait);						\
 	struct hrtimer_sleeper __t;					\
 									\
 	hrtimer_init_on_stack(&__t.timer, CLOCK_MONOTONIC,		\
@@ -349,25 +348,15 @@ do {									\
 				       current->timer_slack_ns,		\
 				       HRTIMER_MODE_REL);		\
 									\
-	for (;;) {							\
-		prepare_to_wait(&wq, &__wait, state);			\
-		if (condition)						\
-			break;						\
-		if (state == TASK_INTERRUPTIBLE &&			\
-		    signal_pending(current)) {				\
-			__ret = -ERESTARTSYS;				\
-			break;						\
-		}							\
+	___wait_event(wq, condition, state, 0, __ret,			\
 		if (!__t.task) {					\
 			__ret = -ETIME;					\
 			break;						\
 		}							\
-		schedule();						\
-	}								\
+		schedule());						\
 									\
 	hrtimer_cancel(&__t.timer);					\
 	destroy_hrtimer_on_stack(&__t.timer);				\
-	finish_wait(&wq, &__wait);					\
 	__ret;								\
 })
 
author maintainer
#8 lore raw

PATCH 6/6 sched, wait: Make the __wait_event*() interface more friendly

In reply to Peter Zijlstra

Change all __wait_event*() implementations to match the corresponding wait_event*() signature for convenience.

In particular this does away with the weird 'ret' logic. Since there are __wait_event*() users this requires we update them too.

---
 arch/mips/kernel/rtlx.c         |   19 +++---
 include/linux/tty.h             |   10 +--
 include/linux/wait.h            |  113 +++++++++++++++++++---------------------
 net/irda/af_irda.c              |    5 -
 net/netfilter/ipvs/ip_vs_sync.c |    7 --
 5 files changed, 73 insertions(+), 81 deletions(-)
arch/mips/kernel/rtlx.c +9 -10
@@ -172,8 +172,9 @@ int rtlx_open(int index, int can_sleep)
 	if (rtlx == NULL) {
 		if( (p = vpe_get_shared(tclimit)) == NULL) {
 		    if (can_sleep) {
-			__wait_event_interruptible(channel_wqs[index].lx_queue,
-				(p = vpe_get_shared(tclimit)), ret);
+			ret = __wait_event_interruptible(
+					channel_wqs[index].lx_queue,
+					(p = vpe_get_shared(tclimit)));
 			if (ret)
 				goto out_fail;
 		    } else {
@@ -263,11 +264,10 @@ unsigned int rtlx_read_poll(int index, i
 	/* data available to read? */
 	if (chan->lx_read == chan->lx_write) {
 		if (can_sleep) {
-			int ret = 0;
-
-			__wait_event_interruptible(channel_wqs[index].lx_queue,
+			int ret = __wait_event_interruptible(
+				channel_wqs[index].lx_queue,
 				(chan->lx_read != chan->lx_write) ||
-				sp_stopping, ret);
+				sp_stopping);
 			if (ret)
 				return ret;
 
@@ -440,14 +440,13 @@ static ssize_t file_write(struct file *f
 
 	/* any space left... */
 	if (!rtlx_write_poll(minor)) {
-		int ret = 0;
+		int ret;
 
 		if (file->f_flags & O_NONBLOCK)
 			return -EAGAIN;
 
-		__wait_event_interruptible(channel_wqs[minor].rt_queue,
-					   rtlx_write_poll(minor),
-					   ret);
+		ret = __wait_event_interruptible(channel_wqs[minor].rt_queue,
+					   rtlx_write_poll(minor));
 		if (ret)
 			return ret;
 	}
include/linux/tty.h +5 -5
@@ -672,14 +672,14 @@ static inline void tty_wait_until_sent_f
 #define wait_event_interruptible_tty(tty, wq, condition)		\
 ({									\
 	int __ret = 0;							\
-	if (!(condition)) {						\
-		__wait_event_interruptible_tty(tty, wq, condition, __ret);	\
-	}								\
+	if (!(condition))						\
+		__ret = __wait_event_interruptible_tty(tty, wq,		\
+						       condition);	\
 	__ret;								\
 })
 
-#define __wait_event_interruptible_tty(tty, wq, condition, ret)		\
-	___wait_event(wq, condition, TASK_INTERRUPTIBLE, 0, ret,	\
+#define __wait_event_interruptible_tty(tty, wq, condition)		\
+	___wait_event(wq, condition, TASK_INTERRUPTIBLE, 0, 0,		\
 			tty_unlock(tty);				\
 			schedule();					\
 			tty_lock(tty))
include/linux/wait.h +55 -58

239 lines not on this page · raw message

net/irda/af_irda.c +2 -3
@@ -2563,9 +2563,8 @@ static int irda_getsockopt(struct socket
 				  jiffies + msecs_to_jiffies(val));
 
 			/* Wait for IR-LMP to call us back */
-			__wait_event_interruptible(self->query_wait,
-			      (self->cachedaddr != 0 || self->errno == -ETIME),
-						   err);
+			err = __wait_event_interruptible(self->query_wait,
+			      (self->cachedaddr != 0 || self->errno == -ETIME));
 
 			/* If watchdog is still activated, kill it! */
 			del_timer(&(self->watchdog));
net/netfilter/ipvs/ip_vs_sync.c +2 -5
@@ -1637,12 +1637,9 @@ static int sync_thread_master(void *data
 			continue;
 		}
 		while (ip_vs_send_sync_msg(tinfo->sock, sb->mesg) < 0) {
-			int ret = 0;
-
-			__wait_event_interruptible(*sk_sleep(sk),
+			int ret = __wait_event_interruptible(*sk_sleep(sk),
 						   sock_writeable(sk) ||
-						   kthread_should_stop(),
-						   ret);
+						   kthread_should_stop());
 			if (unlikely(kthread_should_stop()))
 				goto done;
 		}
#9 lore raw

Re: [PATCH 6/6] sched, wait: Make the __wait_event*() interface more friendly

Quoted 6 lines · Peter Zijlstra

Change all __wait_event*() implementations to match the corresponding wait_event*() signature for convenience.

In particular this does away with the weird 'ret' logic. Since there are __wait_event*() users this requires we update them too.

Quoted 8 lines
---
 arch/mips/kernel/rtlx.c         |   19 +++---
 include/linux/tty.h             |   10 +--
 include/linux/wait.h            |  113 +++++++++++++++++++---------------------
 net/irda/af_irda.c              |    5 -
 net/netfilter/ipvs/ip_vs_sync.c |    7 --
 5 files changed, 73 insertions(+), 81 deletions(-)

Since now everyone seems to agree about this series, and since this particular patch changes generated code it would be really nice to split it into about ten per interface patches:

- __wait_event_interruptible()
- __wait_event_interruptible_timeout()
- __wait_event_interruptible_exclusive()
- __wait_event_interruptible_lock_irq()
- __wait_event_interruptible_tty()
- __wait_event()
- __wait_event_timeout()
- __wait_event_killable()
- __wait_event_interruptible_lock_irq_timeout()

The changes seem to be mostly isolated, they are even in separate patch hunks most of the time making a splitup relatively easy - with a handful of semantic interdependencies.

There's very little downside to doing it the split-up way, and there's a lot of bisection and bug review upside, should any of these changes prove problematic...

Thanks,

Ingo

#10 lore raw
In reply to Peter Zijlstra
Quoted 2 lines · Peter Zijlstra

Compile and boot tested on x86_64.

Btw, I assume the odd binary size reduction is gone now, and code generation is generally identical?

Linus

author maintainer
#11 lore raw
Quoted 6 lines · Linus Torvalds
Quoted 2 lines · Peter Zijlstra

Compile and boot tested on x86_64.

Btw, I assume the odd binary size reduction is gone now, and code generation is generally identical?

Over patch 4; yes. Patches 1-3 generate different kernels, esp patch 2 has a large drop in size, 5 too is invariant. Patch 6 however increases code size again, but then it actually changes the generated code so that's somewhat expected.

But with patches 1-3 its all clear what changes and why; and having the big scary conversion patch 4 invariant is good.

#12 lore raw
In reply to Peter Zijlstra
Quoted 2 lines · Peter Zijlstra

Compile and boot tested on x86_64.

Looks fine, but I'll try to actually read this series tomorrow. Not that I think this needs my review ;)

But at first glance, I think that something like below makes sense as 7/6.

Once again, of course I do not blame this series, but wait_event_timeout(wq, true, 0) still returns 0.

Oleg.

include/linux/wait.h +12 -7
@@ -184,9 +184,14 @@ wait_queue_head_t *bit_waitqueue(void *,
  	bool __cond = (condition);					\
  	if (__cond && !__ret)						\
  		__ret = 1;						\
- 	__cond || !__ret;						\
+ 	__cond;								\
 })
 
+#define ___wait_schedule_timeout(tout)					\
+	if (!tout)							\
+		break;							\
+	tout = schedule_timeout(tout)
+
 #define ___wait_signal_pending(state)					\
 	((state == TASK_INTERRUPTIBLE && signal_pending(current)) ||	\
 	 (state == TASK_KILLABLE && fatal_signal_pending(current)))
@@ -248,7 +253,7 @@ do {									\
 #define __wait_event_timeout(wq, condition, timeout)			\
 	___wait_event(wq, ___wait_cond_timeout(condition),		\
 		      TASK_UNINTERRUPTIBLE, 0, timeout,			\
-		      __ret = schedule_timeout(__ret))
+		      ___wait_schedule_timeout(__ret))
 
 /**
  * wait_event_timeout - sleep until a condition gets true or a timeout elapses
@@ -270,7 +275,7 @@ do {									\
 #define wait_event_timeout(wq, condition, timeout)			\
 ({									\
 	long __ret = timeout;						\
-	if (!(condition)) 						\
+	if (!___wait_cond_timeout(condition)) 				\
 		__ret = __wait_event_timeout(wq, condition, timeout);	\
 	__ret;								\
 })
@@ -305,7 +310,7 @@ do {									\
 #define __wait_event_interruptible_timeout(wq, condition, timeout)	\
 	___wait_event(wq, ___wait_cond_timeout(condition),		\
 		      TASK_INTERRUPTIBLE, 0, timeout,			\
-		      __ret = schedule_timeout(__ret))
+		      ___wait_schedule_timeout(__ret))
 
 /**
  * wait_event_interruptible_timeout - sleep until a condition gets true or a timeout elapses
@@ -328,7 +333,7 @@ do {									\
 #define wait_event_interruptible_timeout(wq, condition, timeout)	\
 ({									\
 	long __ret = timeout;						\
-	if (!(condition))						\
+	if (!___wait_cond_timeout(condition))				\
 		__ret = __wait_event_interruptible_timeout(wq, 		\
 						condition, timeout);	\
 	__ret;								\
@@ -739,7 +744,7 @@ do {									\
 	___wait_event(wq, ___wait_cond_timeout(condition),		\
 		      TASK_INTERRUPTIBLE, 0, ret,	      		\
 		      spin_unlock_irq(&lock);				\
-		      __ret = schedule_timeout(__ret);			\
+		      ___wait_schedule_timeout(__ret);			\
 		      spin_lock_irq(&lock));
 
 /**
@@ -770,7 +775,7 @@ do {									\
 						  timeout)		\
 ({									\
 	long __ret = timeout;						\
-	if (!(condition))						\
+	if (!___wait_cond_timeout(condition))				\
 		__ret = __wait_event_interruptible_lock_irq_timeout(	\
 					wq, condition, lock, timeout);	\
 	__ret;								\
#13 lore raw
Quoted 3 lines · Oleg Nesterov

But at first glance, I think that something like below makes sense as 7/6.

Well, _something like_ perhaps,

include/linux/wait.h do { \ · line 747

Quoted 5 lines
include/linux/wait.h +1 -1
 	___wait_event(wq, ___wait_cond_timeout(condition),		\
 		      TASK_INTERRUPTIBLE, 0, ret,	      		\
 		      spin_unlock_irq(&lock);				\
-		      __ret = schedule_timeout(__ret);			\
+		      ___wait_schedule_timeout(__ret);			\

this is obviously wrong ;)

Oleg.

author maintainer
#14 lore raw
In reply to Oleg Nesterov
Quoted 2 lines · Oleg Nesterov

Once again, of course I do not blame this series, but wait_event_timeout(wq, true, 0) still returns 0.

So we have:

#define ___wait_cond_timeout(condition) \
({ \
bool __cond = (condition); \
if (__cond && !__ret) \
__ret = 1; \
__cond || !__ret; \
})

#define ___wait_event(wq, condition, state, exclusive, ret, cmd) \

({									\
	__label__ __out;						\
	DEFINE_WAIT(__wait);						\
	long __ret = ret;						\
									\
	for (;;) {							\
		if (exclusive)						\
			prepare_to_wait_exclusive(&wq, &__wait, state); \
		else							\
			prepare_to_wait(&wq, &__wait, state);		\
									\
		if (condition)						\
			break;						\
									\
		if (___wait_signal_pending(state)) {			\
			__ret = -ERESTARTSYS;				\
			if (exclusive) {				\
				abort_exclusive_wait(&wq, &__wait, 	\
						     state, NULL); 	\
				goto __out;				\
			}						\
			break;						\
		}							\
									\
		cmd;							\
	}								\
	finish_wait(&wq, &__wait);					\

__out: __ret; \
})

So wait_event_timeout(wq, true, 0) turns into:

({
	DEFINE_WAIT(__wait);
	long __ret = 0;

	for (;;) {
		prepare_to_wait(&wq, &__wait, TASK_UNINTERRUPTIBLE);

		if (({
			bool __cond = (true);
			if (__cond && !__ret)
				__ret = 1;
			__cond || !__ret;
		     }))
			break;

		schedule();
	}
	finish_wait(&wq, &__wait);
	__ret;
})

Which; afaict, returns 1, not 0.

Review

Quoted 12 lines
(quoted diff) +6 -1
+#define ___wait_schedule_timeout(tout)					\
+	if (!tout)							\
+		break;							\
+	tout = schedule_timeout(tout)
+
 
 	___wait_event(wq, ___wait_cond_timeout(condition),		\
 		      TASK_INTERRUPTIBLE, 0, ret,	      		\
 		      spin_unlock_irq(&lock);				\
-		      __ret = schedule_timeout(__ret);			\
+		      ___wait_schedule_timeout(__ret);			\
 		      spin_lock_irq(&lock));

You can't do that; you'll break/return without the lock held.

#15 lore raw
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines · Oleg Nesterov

Once again, of course I do not blame this series, but wait_event_timeout(wq, true, 0) still returns 0.

So we have:

[...snip...]

So wait_event_timeout(wq, true, 0) turns into:

Not really, because of fast-path check,

	#define wait_event_timeout(wq, condition, timeout)			\
	({									\
		long __ret = timeout;						\
		if (!(condition)) 						\
			__ret = __wait_event_timeout(wq, condition, timeout);	\
		__ret;								\
	})

we do not even call __wait_event_timeout() if "condition" is already true at the start.

Quoted 8 lines

include/linux/wait.h do { \ · line 748

Quoted 6 lines
include/linux/wait.h +1 -1
 	___wait_event(wq, ___wait_cond_timeout(condition),		\
 		      TASK_INTERRUPTIBLE, 0, ret,	      		\
 		      spin_unlock_irq(&lock);				\
-		      __ret = schedule_timeout(__ret);			\
+		      ___wait_schedule_timeout(__ret);			\
 		      spin_lock_irq(&lock));

You can't do that; you'll break/return without the lock held.

Yes, see another email, already noticed this.

And let me repeat just in case, I think this series is fine, just wait.h needs another minor/orthogonal fix.

Oleg.

#16 lore raw
Quoted 23 lines · Oleg Nesterov
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines · Oleg Nesterov

Once again, of course I do not blame this series, but wait_event_timeout(wq, true, 0) still returns 0.

So we have:

[...snip...]

So wait_event_timeout(wq, true, 0) turns into:

Not really, because of fast-path check,

	#define wait_event_timeout(wq, condition, timeout)			\
	({									\
		long __ret = timeout;						\
		if (!(condition)) 						\
			__ret = __wait_event_timeout(wq, condition, timeout);	\
		__ret;								\
	})
Quoted 2 lines

we do not even call __wait_event_timeout() if "condition" is already true at the start.

But somehow I didn't realize that ___wait_cond_timeout() can be used as is, so the simple patch below should work?

Oleg.

include/linux/wait.h +3 -3
@@ -270,7 +270,7 @@ do {									\
 #define wait_event_timeout(wq, condition, timeout)			\
 ({									\
 	long __ret = timeout;						\
-	if (!(condition)) 						\
+	if (!___wait_cond_timeout(condition)) 				\
 		__ret = __wait_event_timeout(wq, condition, timeout);	\
 	__ret;								\
 })
@@ -328,7 +328,7 @@ do {									\
 #define wait_event_interruptible_timeout(wq, condition, timeout)	\
 ({									\
 	long __ret = timeout;						\
-	if (!(condition))						\
+	if (!___wait_cond_timeout(condition))				\
 		__ret = __wait_event_interruptible_timeout(wq, 		\
 						condition, timeout);	\
 	__ret;								\
@@ -770,7 +770,7 @@ do {									\
 						  timeout)		\
 ({									\
 	long __ret = timeout;						\
-	if (!(condition))						\
+	if (!___wait_cond_timeout(condition))				\
 		__ret = __wait_event_interruptible_lock_irq_timeout(	\
 					wq, condition, lock, timeout);	\
 	__ret;								\
author maintainer
#17 lore raw
Quoted 2 lines · Oleg Nesterov

But somehow I didn't realize that ___wait_cond_timeout() can be used as is, so the simple patch below should work?

Yeah, should work.. But how often do people use timeout=0? Should we really care about that case to the effect of adding more code.

#18 lore raw
Quoted 6 lines · Peter Zijlstra
Quoted 2 lines · Oleg Nesterov

But somehow I didn't realize that ___wait_cond_timeout() can be used as is, so the simple patch below should work?

Yeah, should work.. But how often do people use timeout=0?

I do not know. Perhaps never.

Quoted 2 lines

Should we
really care about that case to the effect of adding more code.

Again, I do not really know. But imo it would be better to fix this anyway, even if the problem is really minor. If nothing else, wait_event_timeout() and __wait_event_timeout() should have the same semantics.

And suppose that we ha a helper(timeout) which calls wait_event_timeout(), and checks the non-trivial condition inside. Now suppose that someone does

	timeout = DEFAULT_TIMEOUT;

	if (option_nonblock)
		timeout = 0;

	ok = helper(timeout);

So do you think we should ignore this or I should send 7/6 with the changelog ?

(In fact I am going to send another patch on top of this series later.
At least, try to send for discussion because I know you dislike the idea to move the signal-pending checks out of line).

Oleg.

#19 lore raw

RFC 1/1 introduce prepare_to_wait_event()

In reply to Peter Zijlstra

I didn't test is yet, and I did write the changelog.

But let me show the patch for review right now, before we all forget the details... On top of Peter's "wait: Collapse __wait_event macros -v4".

This patch moves the signal-pending checks and part of DEFINE_WAIT's code into the new helper: prepare_to_wait_event().

Yes, sure, prepare_to_wait_event() becomes a little bit slower than prepare_to_wait/prepare_to_wait_exclusive. But this is the slow path anyway, we are likely going to sleep. IMO, it is better to shrink .text, and on my build the difference is

	- 5124686 2955056 10117120        18196862        115a97e vmlinux
	+ 5123212 2955088 10117120        18195420        115a3dc vmlinux

The code with the patch is

	#define ___wait_is_interruptible(state)					\
		(!__builtin_constant_p(state) ||				\
			state == TASK_INTERRUPTIBLE || state == TASK_KILLABLE)	\

	#define ___wait_event(wq, condition, state, exclusive, ret, cmd)	\
	({									\
		__label__ __out;						\
		wait_queue_t __wait;						\
		long __ret = ret;						\
										\
		INIT_LIST_HEAD(&__wait.task_list);				\
		if (exclusive)							\
			__wait.flags = WQ_FLAG_EXCLUSIVE;			\
		else								\
			__wait.flags = 0;					\
										\
		for (;;) {							\
			long intr = prepare_to_wait_event(&wq, &__wait, state);	\
										\
			if (condition)						\
				break;						\
										\
			if (___wait_is_interruptible(state) && intr) {		\
				__ret = intr;					\
				if (exclusive) {				\
					abort_exclusive_wait(&wq, &__wait, 	\
							     state, NULL); 	\
					goto __out;				\
				}						\
				break;						\
			}							\
										\
			cmd;							\
		}								\
		finish_wait(&wq, &__wait);					\
	__out:	__ret;								\
	})

Compiler should optimize out "long intr" if !interruptible/killable.

What do you think?

Oleg.
---

include/linux/wait.h +14 -10
index bd4bd7b..aa758d3 100644
@@ -187,27 +187,30 @@ wait_queue_head_t *bit_waitqueue(void *, int);
  	__cond || !__ret;						\
 })
 
-#define ___wait_signal_pending(state)					\
-	((state == TASK_INTERRUPTIBLE && signal_pending(current)) ||	\
-	 (state == TASK_KILLABLE && fatal_signal_pending(current)))
+#define ___wait_is_interruptible(state)					\
+	(!__builtin_constant_p(state) ||				\
+		state == TASK_INTERRUPTIBLE || state == TASK_KILLABLE)	\
 
 #define ___wait_event(wq, condition, state, exclusive, ret, cmd)	\
 ({									\
 	__label__ __out;						\
-	DEFINE_WAIT(__wait);						\
+	wait_queue_t __wait;						\
 	long __ret = ret;						\
 									\
+	INIT_LIST_HEAD(&__wait.task_list);				\
+	if (exclusive)							\
+		__wait.flags = WQ_FLAG_EXCLUSIVE;			\
+	else								\
+		__wait.flags = 0;					\
+									\
 	for (;;) {							\
-		if (exclusive)						\
-			prepare_to_wait_exclusive(&wq, &__wait, state); \
-		else							\
-			prepare_to_wait(&wq, &__wait, state);		\
+		long intr = prepare_to_wait_event(&wq, &__wait, state);	\
 									\
 		if (condition)						\
 			break;						\
 									\
-		if (___wait_signal_pending(state)) {			\
-			__ret = -ERESTARTSYS;				\
+		if (___wait_is_interruptible(state) && intr) {		\
+			__ret = intr;					\
 			if (exclusive) {				\
 				abort_exclusive_wait(&wq, &__wait, 	\
 						     state, NULL); 	\
@@ -794,6 +797,7 @@ extern long interruptible_sleep_on_timeout(wait_queue_head_t *q,
  */
 void prepare_to_wait(wait_queue_head_t *q, wait_queue_t *wait, int state);
 void prepare_to_wait_exclusive(wait_queue_head_t *q, wait_queue_t *wait, int state);
+long prepare_to_wait_event(wait_queue_head_t *q, wait_queue_t *wait, int state);
 void finish_wait(wait_queue_head_t *q, wait_queue_t *wait);
 void abort_exclusive_wait(wait_queue_head_t *q, wait_queue_t *wait,
 			unsigned int mode, void *key);
kernel/wait.c +24 -0
index d550920..de21c63 100644
@@ -92,6 +92,30 @@ prepare_to_wait_exclusive(wait_queue_head_t *q, wait_queue_t *wait, int state)
 }
 EXPORT_SYMBOL(prepare_to_wait_exclusive);
 
+long prepare_to_wait_event(wait_queue_head_t *q, wait_queue_t *wait, int state)
+{
+	unsigned long flags;
+
+	if (signal_pending_state(state, current))
+		return -ERESTARTSYS;
+
+	wait->private = current;
+	wait->func = autoremove_wake_function;
+
+	spin_lock_irqsave(&q->lock, flags);
+	if (list_empty(&wait->task_list)) {
+		if (wait->flags & WQ_FLAG_EXCLUSIVE)
+			__add_wait_queue_tail(q, wait);
+		else
+			__add_wait_queue(q, wait);
+	}
+	set_current_state(state);
+	spin_unlock_irqrestore(&q->lock, flags);
+
+	return 0;
+}
+EXPORT_SYMBOL(prepare_to_wait_event);
+
 /**
  * finish_wait - clean up after waiting in a queue
  * @q: waitqueue waited on
author maintainer
#20 lore raw

Re: [RFC] introduce prepare_to_wait_event()

Quoted 13 lines · Oleg Nesterov

This patch moves the signal-pending checks and part of DEFINE_WAIT's code into the new helper: prepare_to_wait_event().

Yes, sure, prepare_to_wait_event() becomes a little bit slower than prepare_to_wait/prepare_to_wait_exclusive. But this is the slow path anyway, we are likely going to sleep. IMO, it is better to shrink .text, and on my build the difference is

	- 5124686 2955056 10117120        18196862        115a97e vmlinux
	+ 5123212 2955088 10117120        18195420        115a3dc vmlinux

The code with the patch is

Quoted 15 lines
	#define ___wait_is_interruptible(state)					\
		(!__builtin_constant_p(state) ||				\
			state == TASK_INTERRUPTIBLE || state == TASK_KILLABLE)	\

	#define ___wait_event(wq, condition, state, exclusive, ret, cmd)	\
	({									\
		__label__ __out;						\
		wait_queue_t __wait;						\
		long __ret = ret;						\
										\
		INIT_LIST_HEAD(&__wait.task_list);				\
		if (exclusive)							\
			__wait.flags = WQ_FLAG_EXCLUSIVE;			\
		else								\
			__wait.flags = 0;					\
		__wait.flags = exclusive * WQ_FLAG_EXCLUSIVE;

or is that too obscure? ;-)

Quoted 3 lines
										\
		for (;;) {							\
			long intr = prepare_to_wait_event(&wq, &__wait, state);	\
			int __intr = ...;

The interruptible bit doesn't actually need long; and local variables have __ prefixes in this context.

Quoted 20 lines
										\
			if (condition)						\
				break;						\
										\
			if (___wait_is_interruptible(state) && intr) {		\
				__ret = intr;					\
				if (exclusive) {				\
					abort_exclusive_wait(&wq, &__wait, 	\
							     state, NULL); 	\
					goto __out;				\
				}						\
				break;						\
			}							\
										\
			cmd;							\
		}								\
		finish_wait(&wq, &__wait);					\
	__out:	__ret;								\
	})
Quoted 1 line

Compiler should optimize out "long intr" if !interruptible/killable.

Yeah, and I think even the if (0 && __intr) would suffice for the unused check; otherwise we'd have to adorn the thing with __maybe_unused.

Quoted 1 line

What do you think?

That would actually work I think.. the ___wait_is_interruptible() nicely does away with the unused code; the only slightly more expensive thing would be the prepare_to_wait_event() thing.

And if that really turns out to be a problem we could even re-use ___wait_is_interruptible() to call prepare_to_wait() instead.

#21 lore raw

Re: [RFC] introduce prepare_to_wait_event()

Quoted 10 lines · Peter Zijlstra
Quoted 4 lines · Oleg Nesterov
		if (exclusive)							\
			__wait.flags = WQ_FLAG_EXCLUSIVE;			\
		else								\
			__wait.flags = 0;					\
		__wait.flags = exclusive * WQ_FLAG_EXCLUSIVE;

or is that too obscure? ;-)

I do not mind ;) The generated code should be the same.

Quoted 6 lines
Quoted 2 lines
		for (;;) {							\
			long intr = prepare_to_wait_event(&wq, &__wait, state);	\
			int __intr = ...;

The interruptible bit doesn't actually need long;

Yes, it can be even "bool", but see below.

Quoted 2 lines

and local variables
have __ prefixes in this context.

Yes, yes, will fix.

Quoted 5 lines
Quoted 5 lines
			if (condition)						\
				break;						\
										\
			if (___wait_is_interruptible(state) && intr) {		\
				__ret = intr;					\

Since typeof(__ret) == typeof(intr) gcc can (likely) simply do "mov r1, r2", so "long intr" make the code better.

I am not saying that "int intr" can make it worse, but to me "long" looks better in this context. But I wouldn't mind to change this.

Quoted 4 lines
Quoted 1 line

Compiler should optimize out "long intr" if !interruptible/killable.

Yeah, and I think even the if (0 && __intr) would suffice for the unused check; otherwise we'd have to adorn the thing with __maybe_unused.

Hmm yes, I didn't see any warning during the compilation, but perhaps __maybe_unused is needed, thanks.

Quoted 8 lines
Quoted 1 line

What do you think?

That would actually work I think.. the ___wait_is_interruptible() nicely does away with the unused code; the only slightly more expensive thing would be the prepare_to_wait_event() thing.

And if that really turns out to be a problem we could even re-use ___wait_is_interruptible() to call prepare_to_wait() instead.

OK, thanks.

So I'll wait until your series is applied the resend it officially.

Oleg.

author maintainer
#22 lore raw

Re: [RFC] introduce prepare_to_wait_event()

Quoted 5 lines · Oleg Nesterov
Quoted 2 lines
Quoted 2 lines
			if (___wait_is_interruptible(state) && intr) {		\
				__ret = intr;					\

Since typeof(__ret) == typeof(intr) gcc can (likely) simply do "mov r1, r2", so "long intr" make the code better.

ok, fair enough; keep it long.