hrtimer: make epoll_wait() use the hrtimer range feature
PATCH 1/1 hrtimer: make epoll_wait() use the hrtimer range feature
This make epoll use hrtimers for the timeout value which prevents epoll_wait() from timing out up to a millisecond early.
This mirrors the behavior of select() and poll().
--- fs/eventpoll.c | 35 +++++++++++++++++++---------------- fs/select.c | 2 +- include/linux/poll.h | 2 ++ 3 files changed, 22 insertions(+), 17 deletions(-)
fs/eventpoll.c +19 -16
@@ -77,9 +77,6 @@/* Maximum number of nesting allowed inside epoll sets */ #define EP_MAX_NESTS 4 -/* Maximum msec timeout value storeable in a long int */ -#define EP_MAX_MSTIMEO min(1000ULL * MAX_SCHEDULE_TIMEOUT / HZ, (LONG_MAX - 999ULL) / HZ) -#define EP_MAX_EVENTS (INT_MAX / sizeof(struct epoll_event))#define EP_UNACTIVE_PTR ((void *) -1L)
@@ -1116,18 +1113,22 @@ static int ep_send_events(struct eventpoll *ep,static int ep_poll (struct eventpoll *ep,struct epoll_event __user *events,int maxevents,long timeout) { -int res, eavail; +int res, eavail, timed_out =0 ;unsigned long flags; -long jtimeout; +long slack;wait_queue_t wait; - -/* -* Calculate the timeout by checking for the "infinite" value (-1) -* and the overflow condition. The passed timeout is in milliseconds, -* that why (t * HZ) / 1000. -*/ - jtimeout = (timeout <0 || timeout >=EP_MAX_MSTIMEO ) ? -MAX_SCHEDULE_TIMEOUT : (timeout *HZ +999 ) /1000 ; +struct timespec end_time; +ktime_t expires, *to =NULL ; + +if (timeout >0 ) { +ktime_get_ts (&end_time); +timespec_add_ns (&end_time, (u64 )timeout *NSEC_PER_MSEC ); + slack =estimate_accuracy (&end_time); + to = &expires; + *to =timespec_to_ktime (end_time); + }else if (timeout ==0 ) { + timed_out =1 ; + } retry:spin_lock_irqsave (&ep->lock, flags);
@@ -1149,7 +1150,7 @@ retry:* to TASK_INTERRUPTIBLE before doing the checks. */ set_current_state (TASK_INTERRUPTIBLE ); -if (!list_empty (&ep->rdllist) || !jtimeout) +if (!list_empty (&ep->rdllist) || timed_out)break ;if (signal_pending (current)) { res = -EINTR ;
@@ -1157,7 +1158,9 @@ retry: }spin_unlock_irqrestore (&ep->lock, flags); - jtimeout =schedule_timeout (jtimeout); +if (!schedule_hrtimeout_range (to, slack,HRTIMER_MODE_ABS )) + timed_out =1 ; +spin_lock_irqsave (&ep->lock, flags); }__remove_wait_queue (&ep->wq, &wait);
@@ -1175,7 +1178,7 @@ retry:* more luck. */ if (!res && eavail && - !(res =ep_send_events (ep, events, maxevents)) && jtimeout) + !(res =ep_send_events (ep, events, maxevents)) && !timed_out) goto retry;return res;
fs/select.c +1 -1
@@ -67,7 +67,7 @@ static long __estimate_accuracy(struct timespec *tv)return slack; } -static long estimate_accuracy (struct timespec *tv) +long estimate_accuracy (struct timespec *tv) {unsigned long ret;struct timespec now;
include/linux/poll.h +2 -0
@@ -73,6 +73,8 @@ extern void poll_initwait(struct poll_wqueues *pwq);extern void poll_freewait (struct poll_wqueues *pwq);extern int poll_schedule_timeout (struct poll_wqueues *pwq,int state,ktime_t *expires,unsigned long slack); +extern long estimate_accuracy (struct timespec *tv); +static inline int poll_schedule (struct poll_wqueues *pwq,int state) {
Signature · 1 line
1.7.2.1
I'd say this is an epoll patch, not an hrtimer patch. So I renamed it to "epoll: make epoll_wait() use the hrtimer range feature".
Davide looks after epoll, so let's cc him.
fs/eventpoll.c
Quoted 9 lines · On Sun, 8 Aug 2010 17:45:32 -0500 Shawn Bohrer <[email protected]> wrote:
This make epoll use hrtimers for the timeout value which prevents
epoll_wait()from timing out up to a millisecond early.This mirrors the behavior of
select()andpoll()....
fs/eventpoll.c +0
-0
Davide stuff ;)
fs/select.c static long __estimate_accuracy(struct timespec *tv) · line 70
Quoted 8 lines
fs/select.c +1
-1@@ -67,7 +67,7 @@ static long __estimate_accuracy(struct timespec *tv)return slack; } -static long estimate_accuracy (struct timespec *tv) +long estimate_accuracy (struct timespec *tv)
"estimate_accuracy" is a rotten name for a global symbol. I queued a preparatory patch which renames this to "select_estimate_accuracy".
Quoted 4 lines · Shawn Bohrer
This make epoll use hrtimers for the timeout value which prevents
epoll_wait()from timing out up to a millisecond early.This mirrors the behavior of
select()andpoll().
I saw this now, since I got notifications from Andrew's patch machine. Is this really needed?
Is that really worth the extra code in the fast path? Aren't we breaking existing behavior?
I have some doubts.
- Davide
Quoted 9 lines · Davide Libenzi
Quoted 4 lines · Shawn Bohrer
This make epoll use hrtimers for the timeout value which prevents
epoll_wait()from timing out up to a millisecond early.This mirrors the behavior of
select()andpoll().I saw this now, since I got notifications from Andrew's patch machine. Is this really needed?
Yes, as it is the last user space interface which is jiffies based.
Quoted 1 line
Is that really worth the extra code in the fast path?
It's not that much overhead and we had no complaints when we converted everything else
Quoted 1 line
Aren't we breaking existing behavior?
We had no problems when we moved all the other interfaces over
Quoted 1 line
I have some doubts.
/me not - though I need to look at the patch itself
Thanks,
tglx
Quoted 6 lines · Thomas Gleixner
Quoted 5 lines · Davide Libenzi
Quoted 3 lines · Shawn Bohrer
This make epoll use hrtimers for the timeout value which prevents
epoll_wait()from timing out up to a millisecond early.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
This mirrors the behavior of
select()andpoll().I saw this now, since I got notifications from Andrew's patch machine. Is this really needed?
Yes, as it is the last user space interface which is jiffies based.
What? You killed jiffies?! :)
Alright then, the patch itself, besides the different bahavior on timeout greater/equal to EP_MAX_MSTIMEO (which is not documented, hence does not exist), looks OK to me.
- Davide
Quoted 43 lines · Shawn Bohrer
fs/eventpoll.c +14
-10@@ -1116,18 +1113,22 @@ static int ep_send_events(struct eventpoll *ep, static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, int maxevents, long timeout) { - int res, eavail; + int res, eavail, timed_out = 0; unsigned long flags; - long jtimeout; + long slack; wait_queue_t wait; - - /* - * Calculate the timeout by checking for the "infinite" value (-1) - * and the overflow condition. The passed timeout is in milliseconds, - * that why (t * HZ) / 1000. - */ - jtimeout = (timeout < 0 || timeout >= EP_MAX_MSTIMEO) ? - MAX_SCHEDULE_TIMEOUT : (timeout * HZ + 999) / 1000; + struct timespec end_time; + ktime_t expires, *to = NULL; + + if (timeout > 0) { + ktime_get_ts(&end_time); + timespec_add_ns(&end_time, (u64)timeout * NSEC_PER_MSEC); + slack = estimate_accuracy(&end_time); + to = &expires; + *to = timespec_to_ktime(end_time); + } else if (timeout == 0) { + timed_out = 1; + }retry:
spin_lock_irqsave(&ep->lock, flags);fs/eventpoll.c +1
-1@@ -1149,7 +1150,7 @@ retry:* to TASK_INTERRUPTIBLE before doing the checks. */ set_current_state(TASK_INTERRUPTIBLE); - if (!list_empty(&ep->rdllist) || !jtimeout) + if (!list_empty(&ep->rdllist) || timed_out) break; if (signal_pending(current)) { res = -EINTR;
fs/eventpoll.c retry: · line 1166
Quoted 10 lines
fs/eventpoll.c +3
-1@@ -1157,7 +1158,9 @@ retry: } spin_unlock_irqrestore(&ep->lock, flags); - jtimeout = schedule_timeout(jtimeout); + if (!schedule_hrtimeout_range(to, slack, HRTIMER_MODE_ABS)) + timed_out = 1; + spin_lock_irqsave(&ep->lock, flags); } __remove_wait_queue(&ep->wq, &wait);
this code introduces a warning:fs/eventpoll.c: In function ‘ep_poll’:fs/eventpoll.c:1119: warning: ‘slack’ may be used uninitialized in this function
looks to me like you arent properly handling negative timeouts. certainly epoll_wait() passes the timeout value straight from userspace to ep_poll() without any error checking, so if userspace passes a negative timeout value, it looks like "slack" will be used uninitialized.
-mike
Quoted 59 lines · Mike Frysinger
Quoted 43 lines · Shawn Bohrer
fs/eventpoll.c +14
-10@@ -1116,18 +1113,22 @@ static int ep_send_events(struct eventpoll *ep, static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, int maxevents, long timeout) { - int res, eavail; + int res, eavail, timed_out = 0; unsigned long flags; - long jtimeout; + long slack; wait_queue_t wait; - - /* - * Calculate the timeout by checking for the "infinite" value (-1) - * and the overflow condition. The passed timeout is in milliseconds, - * that why (t * HZ) / 1000. - */ - jtimeout = (timeout < 0 || timeout >= EP_MAX_MSTIMEO) ? - MAX_SCHEDULE_TIMEOUT : (timeout * HZ + 999) / 1000; + struct timespec end_time; + ktime_t expires, *to = NULL; + + if (timeout > 0) { + ktime_get_ts(&end_time); + timespec_add_ns(&end_time, (u64)timeout * NSEC_PER_MSEC); + slack = estimate_accuracy(&end_time); + to = &expires; + *to = timespec_to_ktime(end_time); + } else if (timeout == 0) { + timed_out = 1; + }retry:
spin_lock_irqsave(&ep->lock, flags);fs/eventpoll.c +1
-1@@ -1149,7 +1150,7 @@ retry:* to TASK_INTERRUPTIBLE before doing the checks. */ set_current_state(TASK_INTERRUPTIBLE); - if (!list_empty(&ep->rdllist) || !jtimeout) + if (!list_empty(&ep->rdllist) || timed_out) break; if (signal_pending(current)) { res = -EINTR;fs/eventpoll.c retry: · line 1166
Quoted 10 lines
fs/eventpoll.c +3
-1@@ -1157,7 +1158,9 @@ retry: } spin_unlock_irqrestore(&ep->lock, flags); - jtimeout = schedule_timeout(jtimeout); + if (!schedule_hrtimeout_range(to, slack, HRTIMER_MODE_ABS)) + timed_out = 1; + spin_lock_irqsave(&ep->lock, flags); } __remove_wait_queue(&ep->wq, &wait);this code introduces a warning:
fs/eventpoll.c: In function ‘ep_poll’:fs/eventpoll.c:1119: warning: ‘slack’ may be used uninitialized in this function
Quoted 5 lines
looks to me like you arent properly handling negative timeouts. certainly
epoll_wait()passes the timeout value straight from userspace toep_poll()without any error checking, so if userspace passes a negative timeout value, it looks like "slack" will be used uninitialized.
If a negative timeout is passed in then 'to' remains NULL. When 'to is NULL schedule_hrtimeout_range() has an infinite timeout and the 'slack' parameter is never used. So technically everything should be fine here.
Of course it would be safest and best to simply initialize slack to 0. I can send a patch this evening with the fix.
--
Shawn
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 66 lines · Shawn Bohrer
Quoted 59 lines · Mike Frysinger
Quoted 43 lines · Shawn Bohrer
fs/eventpoll.c +14
-10@@ -1116,18 +1113,22 @@ static int ep_send_events(struct eventpoll *ep, static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, int maxevents, long timeout) { - int res, eavail; + int res, eavail, timed_out = 0; unsigned long flags; - long jtimeout; + long slack; wait_queue_t wait; - - /* - * Calculate the timeout by checking for the "infinite" value (-1) - * and the overflow condition. The passed timeout is in milliseconds, - * that why (t * HZ) / 1000. - */ - jtimeout = (timeout < 0 || timeout >= EP_MAX_MSTIMEO) ? - MAX_SCHEDULE_TIMEOUT : (timeout * HZ + 999) / 1000; + struct timespec end_time; + ktime_t expires, *to = NULL; + + if (timeout > 0) { + ktime_get_ts(&end_time); + timespec_add_ns(&end_time, (u64)timeout * NSEC_PER_MSEC); + slack = estimate_accuracy(&end_time); + to = &expires; + *to = timespec_to_ktime(end_time); + } else if (timeout == 0) { + timed_out = 1; + }retry:
spin_lock_irqsave(&ep->lock, flags);fs/eventpoll.c +1
-1@@ -1149,7 +1150,7 @@ retry:* to TASK_INTERRUPTIBLE before doing the checks. */ set_current_state(TASK_INTERRUPTIBLE); - if (!list_empty(&ep->rdllist) || !jtimeout) + if (!list_empty(&ep->rdllist) || timed_out) break; if (signal_pending(current)) { res = -EINTR;fs/eventpoll.c retry: · line 1166
Quoted 10 lines
fs/eventpoll.c +3
-1@@ -1157,7 +1158,9 @@ retry: } spin_unlock_irqrestore(&ep->lock, flags); - jtimeout = schedule_timeout(jtimeout); + if (!schedule_hrtimeout_range(to, slack, HRTIMER_MODE_ABS)) + timed_out = 1; + spin_lock_irqsave(&ep->lock, flags); } __remove_wait_queue(&ep->wq, &wait);this code introduces a warning:
fs/eventpoll.c: In function ‘ep_poll’:fs/eventpoll.c:1119: warning: ‘slack’ may be used uninitialized in this functionQuoted 5 lines
looks to me like you arent properly handling negative timeouts. certainly
epoll_wait()passes the timeout value straight from userspace toep_poll()without any error checking, so if userspace passes a negative timeout value, it looks like "slack" will be used uninitialized.
Quoted 4 lines
If a negative timeout is passed in then 'to' remains NULL. When 'to is NULL
schedule_hrtimeout_range()has an infinite timeout and the 'slack' parameter is never used. So technically everything should be fine here.
ok, but that depends on an external function never changing behavior and makes changing the API pretty hard since all callers must be closely analyzed
Quoted 2 lines
Of course it would be safest and best to simply initialize slack to 0. I can send a patch this evening with the fix.
thanks !
-mike
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/1 epoll: initialize slack for negative timeout values
When a negative timeout value is passed to epoll the 'slack' variable is currently uninitialized:
fs/eventpoll.c: In function ‘ep_poll’:fs/eventpoll.c:1119: warning: ‘slack’ may be used uninitialized in this function
In this case a NULL pointer is passed to schedule_hrtimeout_range() specifying an infinite timeout. The current implementation of schedule_hrtimeout_range() does not use slack in this case, but we should still initialize slack to 0 in case future implementations use it.
--- fs/eventpoll.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
fs/eventpoll.c +1 -1
@@ -1116,7 +1116,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, {int res, eavail, timed_out =0 ;unsigned long flags; -long slack; +long slack =0 ;wait_queue_t wait;struct timespec end_time;ktime_t expires, *to =NULL ;
Signature · 1 line
1.7.3.2
Re: [PATCH] epoll: initialize slack for negative timeout values
Quoted 10 lines · Shawn Bohrer
When a negative timeout value is passed to epoll the 'slack' variable is currently uninitialized:
fs/eventpoll.c: In function ‘ep_poll’:fs/eventpoll.c:1119: warning: ‘slack’ may be used uninitialized in this functionIn this case a NULL pointer is passed to
schedule_hrtimeout_range()specifying an infinite timeout. The current implementation ofschedule_hrtimeout_range()does not use slack in this case, but we should still initialize slack to 0 in case future implementations use it.
Thanks.
Quoted 1 line
Signed-off-by Shawn Bohrer <[email protected]>
Quoted 9 lines
--- fs/eventpoll.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)fs/eventpoll.c +0
-0
fs/eventpoll.c static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, · line 1122
Quoted 8 lines
fs/eventpoll.c +1
-1@@ -1116,7 +1116,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, {int res, eavail, timed_out =0 ;unsigned long flags; -long slack; +long slack =0 ;wait_queue_t wait;struct timespec end_time;ktime_t expires, *to =NULL ;
- Davide