udp ping pong with various process bindings (and correct cpu mappings)
Subsystems (4)
Here are the results of a 40 byte udpping (http://gentwo.org/ll) run on kernel from 2.6.22 to 2.6.30-rc3 on a Dell 1950 dual quad core 3.3Ghz. One system fixed 2.6.22 kernel version on the other are varied.
Nice graph at http://gentwo.org/results/udpping-results.pdf
Summary:
- Loss of ~1.5usec on fastest path (same cpu) since 2.6.22
- Different cpu same core looses 2-3 usecs vs. same cpu
- Different cpu different core looses ~ 8 usecs vs same cpu
- Maximum is usual if threads are on different sockets but sometimes the same socket different core is worse (2.6.26/2.6.27).
- Up to 9 usecs variance in a basic network operation just because of process placement.
Same CPU
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 83.03 82.9 82.89 82.92 82.94
2.6.23 83.35 82.81 82.83 82.86 82.96
2.6.24 82.66 82.56 82.64 82.73 82.65
2.6.25 84.28 84.29 84.37 84.3 84.31
2.6.26 84.72 84.38 84.41 84.68 84.55
2.6.27 84.56 84.44 84.41 84.58 84.5
2.6.28 84.7 84.43 84.47 84.48 84.52
2.6.29 84.91 84.67 84.69 84.75 84.76
2.6.30-rc2 84.94 84.72 84.69 84.93 84.82
2.6.30-rc3 84.88 84.7 84.73 84.89 84.8
Same core, different processor (l2 is shared)
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 84.6 84.71 84.52 84.53 84.59
2.6.23 84.59 84.5 84.33 84.34 84.44
2.6.24 84.28 84.3 84.38 84.28 84.31
2.6.25 86.12 85.8 86.2 86.04 86.04
2.6.26 86.61 86.46 86.49 86.7 86.57
2.6.27 87 87.01 87 86.95 86.99
2.6.28 86.53 86.44 86.26 86.24 86.37
2.6.29 85.88 85.94 86.1 85.69 85.9
2.6.30-rc2 86.03 85.93 85.99 86.06 86
2.6.30-rc3 85.73 85.88 85.67 85.94 85.81
Same Socket, different core (l2 not shared)
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 90.08 89.72 90 89.9 89.93
2.6.23 89.72 90.1 89.99 89.86 89.92
2.6.24 89.18 89.28 89.25 89.22 89.23
2.6.25 90.83 90.78 90.87 90.61 90.77
2.6.26 90.51 91.25 91.8 91.69 91.31
2.6.27 91.98 91.93 91.97 91.91 91.95
2.6.28 91.72 91.7 91.84 91.75 91.75
2.6.29 89.85 89.85 90.14 89.9 89.94
2.6.30-rc2 90.78 90.8 90.87 90.73 90.8
2.6.30-rc3 90.84 90.94 91.05 90.84 90.92
Different Socket
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 91.64 91.65 91.61 91.68 91.645
2.6.23 91.9 91.84 91.92 91.83 91.873
2.6.24 91.33 91.24 91.42 91.38 91.343
2.6.25 92.39 92.04 92.3 92.23 92.240
2.6.26 90.64 90.57 90.6 90.08 90.473
2.6.27 91.14 91.26 90.9 91.09 91.098
2.6.28 92.3 91.92 92.3 92.23 92.188
2.6.29 90.57 89.83 89.9 90.41 90.178
2.6.30-rc2 90.59 90.97 90.27 91.69 90.880
2.6.30-rc3 92.08 91.32 91.21 92.06 91.668
Quoted 54 lines · Christoph Lameter
Here are the results of a 40 byte udpping (http://gentwo.org/ll) run on kernel from 2.6.22 to 2.6.30-rc3 on a Dell 1950 dual quad core 3.3Ghz. One system fixed 2.6.22 kernel version on the other are varied.
Nice graph at http://gentwo.org/results/udpping-results.pdf
Summary:
- Loss of ~1.5usec on fastest path (same cpu) since 2.6.22
- Different cpu same core looses 2-3 usecs vs. same cpu
- Different cpu different core looses ~ 8 usecs vs same cpu
- Maximum is usual if threads are on different sockets but sometimes the same socket different core is worse (2.6.26/2.6.27).
- Up to 9 usecs variance in a basic network operation just because of process placement.Same CPU
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 83.03 82.9 82.89 82.92 82.94
2.6.23 83.35 82.81 82.83 82.86 82.96
2.6.24 82.66 82.56 82.64 82.73 82.65
2.6.25 84.28 84.29 84.37 84.3 84.31
2.6.26 84.72 84.38 84.41 84.68 84.55
2.6.27 84.56 84.44 84.41 84.58 84.5
2.6.28 84.7 84.43 84.47 84.48 84.52
2.6.29 84.91 84.67 84.69 84.75 84.76
2.6.30-rc2 84.94 84.72 84.69 84.93 84.82
2.6.30-rc3 84.88 84.7 84.73 84.89 84.8Same core, different processor (l2 is shared)
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 84.6 84.71 84.52 84.53 84.59
2.6.23 84.59 84.5 84.33 84.34 84.44
2.6.24 84.28 84.3 84.38 84.28 84.31
2.6.25 86.12 85.8 86.2 86.04 86.04
2.6.26 86.61 86.46 86.49 86.7 86.57
2.6.27 87 87.01 87 86.95 86.99
2.6.28 86.53 86.44 86.26 86.24 86.37
2.6.29 85.88 85.94 86.1 85.69 85.9
2.6.30-rc2 86.03 85.93 85.99 86.06 86
2.6.30-rc3 85.73 85.88 85.67 85.94 85.81Same Socket, different core (l2 not shared)
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 90.08 89.72 90 89.9 89.93
2.6.23 89.72 90.1 89.99 89.86 89.92
2.6.24 89.18 89.28 89.25 89.22 89.23
2.6.25 90.83 90.78 90.87 90.61 90.77
2.6.26 90.51 91.25 91.8 91.69 91.31
2.6.27 91.98 91.93 91.97 91.91 91.95
2.6.28 91.72 91.7 91.84 91.75 91.75
2.6.29 89.85 89.85 90.14 89.9 89.94
2.6.30-rc2 90.78 90.8 90.87 90.73 90.8
2.6.30-rc3 90.84 90.94 91.05 90.84 90.92
Quoted 14 lines
Different Socket
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 91.64 91.65 91.61 91.68 91.645
2.6.23 91.9 91.84 91.92 91.83 91.873
2.6.24 91.33 91.24 91.42 91.38 91.343
2.6.25 92.39 92.04 92.3 92.23 92.240
2.6.26 90.64 90.57 90.6 90.08 90.473
2.6.27 91.14 91.26 90.9 91.09 91.098
2.6.28 92.3 91.92 92.3 92.23 92.188
2.6.29 90.57 89.83 89.9 90.41 90.178
2.6.30-rc2 90.59 90.97 90.27 91.69 90.880
2.6.30-rc3 92.08 91.32 91.21 92.06 91.668
Thanks Christoph for doing this
I believe we can restore pre 2.6.25 performance level with litle changes.
[Problem is that on 2.6.25, UDP mem accounting forced us to add a callback to sock_def_write_space() at skb TX completion time. This function then wake up all thread(s) blocked in revfrom() syscall. Once awaken, thread(s) block again because no frame was received]
Davide Libenzi added a 'key' opaque argument to wakeups so that eventpoll can avoid unnecessary wakeups. This infrastructure could be used on other paths. (Most important being this one : receivers, because writers are rarely blocked because of sndbuffer filled)
commit 37e5540b3c9d838eb20f2ca8ea2eb8072271e403
Author: Davide Libenzi <[email protected]> Date: Tue Mar 31 15:24:21 2009 -0700
epoll keyed wakeups: make sockets use keyed wakeups
Add support for event-aware wakeups to the sockets code. Events are delivered to the wakeup target, so that epoll can avoid spurious wakeups for non-interesting events.
commit : 2dfa4eeab0fc7e8633974f2770945311b31eedf6
epoll keyed wakeups: teach epoll about hints coming with the wakeup key
Use the events hint now sent by some devices, to avoid unnecessary wakeups for events that are of no interest for the caller. This code handles both devices that are sending keyed events, and the ones that are not (and event the ones that sometimes send events, and sometimes don't).
We can add support for these key on regular socket code, so that a process waiting on receive wont be scheduled because a TX completion occured.
Standard way is using autoremove_wake_function() :
int autoremove_wake_function(wait_queue_t *wait, unsigned mode, int sync, void *key)
{
int ret = default_wake_function(wait, mode, sync, key);
if (ret)
list_del_init(&wait->task_list);
return ret;
}/* this function ignores "key" argument */
int default_wake_function(wait_queue_t *curr, unsigned mode, int sync,
void *key)
{
return try_to_wake_up(curr->private, mode, sync);
}While new 'keyed' events can do better :
static int ep_poll_callback(wait_queue_t *wait, unsigned mode, int sync, void *key)
{
int pwake = 0;
unsigned long flags;
struct epitem *epi = ep_item_from_wait(wait);
struct eventpoll *ep = epi->ep;
spin_lock_irqsave(&ep->lock, flags);...
/*
* Check the events coming with the callback. At this stage, not
* every device reports the events in the "key" parameter of the
* callback. We need to be able to handle both cases here, hence the
* test for "key" != NULL before the event match test. */
if (key && !((unsigned long) key & epi->event.events))
goto out_unlock;
}
I'll try to cook a patch in following days, unless someone beats me :)
Thanks
PATCH 1/1 net: Avoid extra wakeups of threads blocked in wait_for_packet()
In reply to Christoph LameterIn 2.6.25 we added UDP mem accounting.
This unfortunatly added a penalty when a frame is transmitted, since we have at TX completion time to call sock_wfree() to perform necessary memory accounting. This calls sock_def_write_space() and utimately scheduler if any thread is waiting on the socket.
Thread(s) waiting for an incoming frame was scheduled, then had to sleep again as event was meaningless.
(All threads waiting on a socket are using same sk_sleep anchor)
This adds lot of extra wakeups and increases latencies, as noted by Christoph Lameter, and slows down softirq handler.
Reference : http://marc.info/?l=linux-netdev&m=124060437012283&w=2
Fortunatly, Davide Libenzi recently added concept of keyed wakeups into kernel, and particularly for sockets (see commit 37e5540b3c9d838eb20f2ca8ea2eb8072271e403
epoll keyed wakeups: make sockets use keyed wakeups)
Davide goal was to optimize epoll, but this new wakeup infrastructure can help non epoll users as well, if they care to setup an appropriate handler.
This patch introduces new DEFINE_WAIT_FUNC() helper and uses it in wait_for_packet(), so that only relevant event can wakeup a thread blocked in this function.
Trace of function calls from bnx2 TX completion bnx2_poll_work() is :__kfree_skb()
skb_release_head_state()
sock_wfree()
sock_def_write_space()
__wake_up_sync_key()
__wake_up_common()
receiver_wake_function() : Stops here since thread is waiting for an INPUT--- include/linux/wait.h | 6 ++++-- net/core/datagram.c | 14 +++++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-)
include/linux/wait.h +4 -2
@@ -440,13 +440,15 @@ void abort_exclusive_wait(wait_queue_head_t *q, wait_queue_t *wait,int autoremove_wake_function (wait_queue_t *wait,unsigned mode,int sync,void *key);int wake_bit_function (wait_queue_t *wait,unsigned mode,int sync,void *key); -#define DEFINE_WAIT (name) \ +#define DEFINE_WAIT_FUNC (name, function) \ wait_queue_t name = { \ .private = current, \ - .func = autoremove_wake_function, \ + .func = function, \ .task_list = LIST_HEAD_INIT((name).task_list), \ } +#define DEFINE_WAIT (name) DEFINE_WAIT_FUNC(name, autoremove_wake_function) +#define DEFINE_WAIT_BIT (name, word, bit) \ struct wait_bit_queue name = { \ .key = __WAIT_BIT_KEY_INITIALIZER(word, bit), \
net/core/datagram.c +13 -1
@@ -64,13 +64,25 @@ static inline int connection_based(struct sock *sk)return sk->sk_type ==SOCK_SEQPACKET || sk->sk_type ==SOCK_STREAM ; } +static int receiver_wake_function (wait_queue_t *wait,unsigned mode,int sync, +void *key) +{ +unsigned long bits = (unsigned long )key; + +/* +* Avoid a wakeup if event not interesting for us +*/ +if (bits && !(bits & (POLLIN |POLLERR ))) +return 0 ; +return autoremove_wake_function (wait, mode, sync, key); +}/* * Wait for a packet.. */ static int wait_for_packet (struct sock *sk,int *err,long *timeo_p) {int error; -DEFINE_WAIT (wait); +DEFINE_WAIT_FUNC (wait, receiver_wake_function);prepare_to_wait_exclusive (sk->sk_sleep, &wait,TASK_INTERRUPTIBLE );
David Miller
maintainerRe: [PATCH] net: Avoid extra wakeups of threads blocked in wait_for_packet()
From: Eric Dumazet <[email protected]>
Date: Sat, 25 Apr 2009 17:47:23 +0200
Quoted 1 line
(All threads waiting on a socket are using same
sk_sleepanchor)
Great stuff Eric.
We've discussed splitting the wait queue up before, but shorter-term your idea is pretty cool too :-)
PATCH 1/1 poll: Avoid extra wakeups
Quoted 5 lines · David Miller
Great stuff Eric.
We've discussed splitting the wait queue up before, but shorter-term your idea is pretty cool too :-)
Well, I only got this idea because Davide did its previous work, he is the one who did the hard stuff :)
About poll()/select() improvements, I believe following patch should be fine too.
Note some lines on this patch are longer than 80 columns, I am aware of this but could not find an elegant/efficient way to avoid this.
Thank you
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call sock_wfree() which in turn schedules thread.
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
--- fs/select.c | 28 +++++++++++++++++++++++++--- include/linux/poll.h | 3 +++ 2 files changed, 28 insertions(+), 3 deletions(-)
fs/select.c +25 -3
@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);
@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)
@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);
@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll) +if (f_op && f_op->poll) { +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } mask = (*f_op->poll)(file, retval ?NULL : wait); + }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit;
@@ -685,8 +704,11 @@ static inline unsigned int do_pollfd(struct pollfd *pollfd, poll_table *pwait) mask =POLLNVAL ;if (file !=NULL ) { mask =DEFAULT_POLLMASK ; -if (file->f_op && file->f_op->poll) +if (file->f_op && file->f_op->poll) { +if (pwait) + pwait->key = pollfd->events |POLLERR |POLLHUP ; mask = file->f_op->poll (file, pwait); + }/* Mask out unneeded events. */ mask &= pollfd->events |POLLERR |POLLHUP ;fput_light (file, fput_needed);
include/linux/poll.h +3 -0
@@ -32,6 +32,7 @@ typedef void (*poll_queue_proc)(struct file *, wait_queue_head_t *, struct poll_typedef struct poll_table_struct {poll_queue_proc qproc; +unsigned long key; }poll_table ;static inline void poll_wait (struct file * filp,wait_queue_head_t * wait_address,poll_table *p)
@@ -43,10 +44,12 @@ static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addresstatic inline void init_poll_funcptr (poll_table *pt,poll_queue_proc qproc) { pt->qproc = qproc; + pt->key = ~0UL ;/* all events enabled */ }struct poll_table_entry {struct file *filp; +unsigned long key;wait_queue_t wait;wait_queue_head_t *wait_address; };
Re: [PATCH] poll: Avoid extra wakeups
Eric Dumazet wrote, On 04/26/2009 12:46 PM:
...
Quoted 1 line
[PATCH] poll: Avoid extra wakeups in select/pol
...
Quoted 5 lines
Signed-off-by Eric Dumazet <[email protected]>--- fs/select.c | 28 +++++++++++++++++++++++++--- include/linux/poll.h | 3 +++ 2 files changed, 28 insertions(+), 3 deletions(-)
Eric, I wonder why you've forgotten about linux-kernel@ folks...
Jarek P.
Re: [PATCH] poll: Avoid extra wakeups
Quoted 7 lines · Jarek Poplawski
Eric Dumazet wrote, On 04/26/2009 12:46 PM:
...
Quoted 1 line
[PATCH] poll: Avoid extra wakeups in select/pol
...
Quoted 9 lines
Quoted 5 lines
Signed-off-by Eric Dumazet <[email protected]>--- fs/select.c | 28 +++++++++++++++++++++++++--- include/linux/poll.h | 3 +++ 2 files changed, 28 insertions(+), 3 deletions(-)Eric, I wonder why you've forgotten about linux-kernel@ folks...
Ah yes, I forgot, I only did a 'reply all' on David's mail. I'll resubmit it anyway, since it was only a followup.
Re: [PATCH] poll: Avoid extra wakeups
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Date: Sun, 26 Apr 2009 12:46:39 +0200
Quoted 1 line
[PATCH] poll: Avoid extra wakeups in select/poll
Looks great to me:
But this has to go through something other than the networking tree, of course :-)
Re: [PATCH] poll: Avoid extra wakeups
Quoted 13 lines · David Miller
From: Eric Dumazet <[email protected]>
Date: Sun, 26 Apr 2009 12:46:39 +0200Quoted 1 line
[PATCH] poll: Avoid extra wakeups in select/poll
Looks great to me:
Acked-by David S. Miller <[email protected]>But this has to go through something other than the networking tree, of course :-)
Sure, I'll do that promptly :)
Thanks
Re: [PATCH] poll: Avoid extra wakeups
In reply to Eric DumazetQuoted 4 lines · Eric Dumazet
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
I wonder if the key could be used for more state. For example if you two processes are in recvmsg() on a socket and there's only a single packet incoming we only need to wake up the first waiter. Could that be done with keys too?
Quoted 4 lines
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
I'm late, but: very cool patch too.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Re: [PATCH] poll: Avoid extra wakeups
Quoted 9 lines · Andi Kleen
Quoted 3 lines · Eric Dumazet
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
I wonder if the key could be used for more state. For example if you two processes are in
recvmsg()on a socket and there's only a single packet incoming we only need to wake up the first waiter. Could that be done with keys too?
I am not sure its possible. I'll take a look.
Quoted 9 lines
Quoted 5 lines
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
I'm late, but: very cool patch too.
Acked-by Andi Kleen <[email protected]>
Thanks, I am going to send it again on lkml this time :)
PATCH 1/1 poll: Avoid extra wakeups in select/poll
In reply to Andi Kleen[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call sock_wfree() which in turn schedules thread.
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
--- fs/select.c | 28 +++++++++++++++++++++++++--- include/linux/poll.h | 3 +++ 2 files changed, 28 insertions(+), 3 deletions(-)
fs/select.c +25 -3
@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);
@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)
@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);
@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll) +if (f_op && f_op->poll) { +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } mask = (*f_op->poll)(file, retval ?NULL : wait); + }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit;
@@ -685,8 +704,11 @@ static inline unsigned int do_pollfd(struct pollfd *pollfd, poll_table *pwait) mask =POLLNVAL ;if (file !=NULL ) { mask =DEFAULT_POLLMASK ; -if (file->f_op && file->f_op->poll) +if (file->f_op && file->f_op->poll) { +if (pwait) + pwait->key = pollfd->events |POLLERR |POLLHUP ; mask = file->f_op->poll (file, pwait); + }/* Mask out unneeded events. */ mask &= pollfd->events |POLLERR |POLLHUP ;fput_light (file, fput_needed);
include/linux/poll.h +3 -0
@@ -32,6 +32,7 @@ typedef void (*poll_queue_proc)(struct file *, wait_queue_head_t *, struct poll_typedef struct poll_table_struct {poll_queue_proc qproc; +unsigned long key; }poll_table ;static inline void poll_wait (struct file * filp,wait_queue_head_t * wait_address,poll_table *p)
@@ -43,10 +44,12 @@ static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addresstatic inline void init_poll_funcptr (poll_table *pt,poll_queue_proc qproc) { pt->qproc = qproc; + pt->key = ~0UL ;/* all events enabled */ }struct poll_table_entry {struct file *filp; +unsigned long key;wait_queue_t wait;wait_queue_head_t *wait_address; };
Re: [PATCH] poll: Avoid extra wakeups in select/poll
For the udpping test load these patches have barely any effect:
git2p1 is the first version of the patch
git2p2 is the second version (this one)
Same CPU
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 83.03 82.9 82.89 82.92 82.935
2.6.23 83.35 82.81 82.83 82.86 82.9625
2.6.24 82.66 82.56 82.64 82.73 82.6475
2.6.25 84.28 84.29 84.37 84.3 84.31
2.6.26 84.72 84.38 84.41 84.68 84.5475
2.6.27 84.56 84.44 84.41 84.58 84.4975
2.6.28 84.7 84.43 84.47 84.48 84.52
2.6.29 84.91 84.67 84.69 84.75 84.755
2.6.30-rc2 84.94 84.72 84.69 84.93 84.82
2.6.30-rc3 84.88 84.7 84.73 84.89 84.8
2.6.30-rc3-git2p1 84.89 84.77 84.79 84.85 84.825
2.6.30-rc3-git2p2 84.91 84.79 84.78 84.8 84.82
Same Core
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 84.6 84.71 84.52 84.53 84.59
2.6.23 84.59 84.5 84.33 84.34 84.44
2.6.24 84.28 84.3 84.38 84.28 84.31
2.6.25 86.12 85.8 86.2 86.04 86.04
2.6.26 86.61 86.46 86.49 86.7 86.565
2.6.27 87 87.01 87 86.95 86.99
2.6.28 86.53 86.44 86.26 86.24 86.3675
2.6.29 85.88 85.94 86.1 85.69 85.9025
2.6.30-rc2 86.03 85.93 85.99 86.06 86.0025
2.6.30-rc3 85.73 85.88 85.67 85.94 85.805
2.6.30-rc3-git2p1 86.11 85.8 86.03 85.92 85.965
2.6.30-rc3-git2p2 86.04 85.96 85.89 86.04 85.9825
Same Socket
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 90.08 89.72 90 89.9 89.925
2.6.23 89.72 90.1 89.99 89.86 89.9175
2.6.24 89.18 89.28 89.25 89.22 89.2325
2.6.25 90.83 90.78 90.87 90.61 90.7725
2.6.26 90.51 91.25 91.8 91.69 91.3125
2.6.27 91.98 91.93 91.97 91.91 91.9475
2.6.28 91.72 91.7 91.84 91.75 91.7525
2.6.29 89.85 89.85 90.14 89.9 89.935
2.6.30-rc2 90.78 90.8 90.87 90.73 90.795
2.6.30-rc3 90.84 90.94 91.05 90.84 90.9175
2.6.30-rc3-git2p1 90.87 90.95 90.86 90.92 90.9
2.6.30-rc3-git2p2 91.09 91.01 90.97 91.06 91.0325
Different Socket
Kernel Test 1 Test 2 Test 3 Test 4 Average
2.6.22 91.64 91.65 91.61 91.68 91.645
2.6.23 91.9 91.84 91.92 91.83 91.873
2.6.24 91.33 91.24 91.42 91.38 91.343
2.6.25 92.39 92.04 92.3 92.23 92.240
2.6.26 90.64 90.57 90.6 90.08 90.473
2.6.27 91.14 91.26 90.9 91.09 91.098
2.6.28 92.3 91.92 92.3 92.23 92.188
2.6.29 90.57 89.83 89.9 90.41 90.178
2.6.30-rc2 90.59 90.97 90.27 91.69 90.880
2.6.30-rc3 92.08 91.32 91.21 92.06 91.668
2.6.30-rc3-git2p1 91.46 91.38 91.92 91.03 91.448
2.6.30-rc3-git2p2 91.39 90.47 90.03 90.62 90.628
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 4 lines · Christoph Lameter
For the udpping test load these patches have barely any effect:
git2p1 is the first version of the patch
git2p2 is the second version (this one)
But... udpping does *not* use poll() nor select(), unless I am mistaken ?
If you really want to test this patch with udpping, you might add a poll() call before recvfrom() :
while(1) {
+ struct pollfd pfd = { .fd = sock, .events = POLLIN};
+ poll(pfd, 1, -1);
nbytes = recvfrom(sock, msg, min(inblocksize, sizeof(msg)),
0, &inad, &inadlen);
if (nbytes < 0) {
perror("recvfrom");
break;
}
if (sendto(sock, msg, nbytes, 0, &inad, inadlen) < 0) {
perror("sendto");
break;
}
}Part about recvfrom() wakeup avoidance is in David net-2.6 tree, and saves 2 us on udpping here.
Is it what you call git2p1 ?
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 3 lines · Eric Dumazet
Part about
recvfrom()wakeup avoidance is in David net-2.6 tree, and saves 2 us on udpping here.Is it what you call git2p1 ?
No that is just the prior version of the poll/select improvements.
Which patch are you referring to?
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 11 lines · Christoph Lameter
Quoted 3 lines · Eric Dumazet
Part about
recvfrom()wakeup avoidance is in David net-2.6 tree, and saves 2 us on udpping here.Is it what you call git2p1 ?
No that is just the prior version of the poll/select improvements.
Which patch are you referring to?
The one that did improved your udpping 'bench' :)
[PATCH] net: Avoid extra wakeups of threads blocked in wait_for_packet()
In 2.6.25 we added UDP mem accounting.
This unfortunatly added a penalty when a frame is transmitted, since we have at TX completion time to call sock_wfree() to perform necessary memory accounting. This calls sock_def_write_space() and utimately scheduler if any thread is waiting on the socket.
Thread(s) waiting for an incoming frame was scheduled, then had to sleep again as event was meaningless.
(All threads waiting on a socket are using same sk_sleep anchor)
This adds lot of extra wakeups and increases latencies, as noted by Christoph Lameter, and slows down softirq handler.
Reference : http://marc.info/?l=linux-netdev&m=124060437012283&w=2
Fortunatly, Davide Libenzi recently added concept of keyed wakeups into kernel, and particularly for sockets (see commit 37e5540b3c9d838eb20f2ca8ea2eb8072271e403
epoll keyed wakeups: make sockets use keyed wakeups)
Davide goal was to optimize epoll, but this new wakeup infrastructure can help non epoll users as well, if they care to setup an appropriate handler.
This patch introduces new DEFINE_WAIT_FUNC() helper and uses it in wait_for_packet(), so that only relevant event can wakeup a thread blocked in this function.
Trace of function calls from bnx2 TX completion bnx2_poll_work() is :__kfree_skb()
skb_release_head_state()
sock_wfree()
sock_def_write_space()
__wake_up_sync_key()
__wake_up_common()
receiver_wake_function() : Stops here since thread is waiting for an INPUT--- include/linux/wait.h | 6 ++++-- net/core/datagram.c | 14 +++++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-)
include/linux/wait.h +4 -2
@@ -440,13 +440,15 @@ void abort_exclusive_wait(wait_queue_head_t *q, wait_queue_t *wait,int autoremove_wake_function (wait_queue_t *wait,unsigned mode,int sync,void *key);int wake_bit_function (wait_queue_t *wait,unsigned mode,int sync,void *key); -#define DEFINE_WAIT (name) \ +#define DEFINE_WAIT_FUNC (name, function) \ wait_queue_t name = { \ .private = current, \ - .func = autoremove_wake_function, \ + .func = function, \ .task_list = LIST_HEAD_INIT((name).task_list), \ } +#define DEFINE_WAIT (name) DEFINE_WAIT_FUNC(name, autoremove_wake_function) +#define DEFINE_WAIT_BIT (name, word, bit) \ struct wait_bit_queue name = { \ .key = __WAIT_BIT_KEY_INITIALIZER(word, bit), \
net/core/datagram.c +13 -1
@@ -64,13 +64,25 @@ static inline int connection_based(struct sock *sk)return sk->sk_type ==SOCK_SEQPACKET || sk->sk_type ==SOCK_STREAM ; } +static int receiver_wake_function (wait_queue_t *wait,unsigned mode,int sync, +void *key) +{ +unsigned long bits = (unsigned long )key; + +/* +* Avoid a wakeup if event not interesting for us +*/ +if (bits && !(bits & (POLLIN |POLLERR ))) +return 0 ; +return autoremove_wake_function (wait, mode, sync, key); +}/* * Wait for a packet.. */ static int wait_for_packet (struct sock *sk,int *err,long *timeo_p) {int error; -DEFINE_WAIT (wait); +DEFINE_WAIT_FUNC (wait, receiver_wake_function);prepare_to_wait_exclusive (sk->sk_sleep, &wait,TASK_INTERRUPTIBLE );
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 3 lines · Eric Dumazet
The one that did improved your udpping 'bench' :)
Well yes that is git2p1. The measurements that we took showed not much of an effect as you see.
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 9 lines · Christoph Lameter
Quoted 3 lines · Eric Dumazet
The one that did improved your udpping 'bench' :)
Well yes that is git2p1. The measurements that we took showed not much of an effect as you see.
It depends of coalescing parameters of NIC.
BNX2 interrupts first handle TX completions, then RX events. So If by the
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 6 lines · Eric Dumazet
Quoted 5 lines · Christoph Lameter
Quoted 3 lines · Eric Dumazet
The one that did improved your udpping 'bench' :)
Quoted 8 lines
Quoted 3 lines
Well yes that is git2p1. The measurements that we took showed not much of an effect as you see.
It depends of coalescing parameters of NIC.
BNX2 interrupts first handle TX completions, then RX events. So If by the
Sorry, hit wrong key.
So if by t
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 3 lines · Eric Dumazet
Sorry, hit wrong key.
So if by t
You lost 2 more characters..... Keyboard wonky?
Re: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Eric DumazetQuoted 2 lines · Eric Dumazet
BNX2 interrupts first handle TX completions, then RX events. So If by the
Guess there is more to come?
Re: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Eric DumazetQuoted 6 lines · Eric Dumazet
Quoted 5 lines · Christoph Lameter
Quoted 3 lines · Eric Dumazet
The one that did improved your udpping 'bench' :)
Quoted 9 lines
Quoted 3 lines
Well yes that is git2p1. The measurements that we took showed not much of an effect as you see.
It depends of coalescing parameters of NIC.
BNX2 interrupts first handle TX completions, then RX events. So If by the
Sorry for the previous message...
If by the time interrupt comes to the host, TX was handled right before RX event, the extra wakeup is not a problem, because incoming frame will be delivered into socket queue right before awaken thread tries to pull it.
On real workloads (many incoming/outgoing frames), then avoiding extra wakeups is a win, regardless of coalescing parameters and cpu affinities...
On uddpping, I had prior to the patch about 49000 wakeups per second, and after patch about 26000 wakeups per second (matches number of incoming udp messages per second)
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 3 lines · Eric Dumazet
On uddpping, I had prior to the patch about 49000 wakeups per second, and after patch about 26000 wakeups per second (matches number of incoming udp messages per second)
very nice. It might not show up as a real performance difference if the CPUs are not fully saturated during the test - but it could show up as a decrease in CPU utilization.
Also, if you run the test via 'perf stat -a ./test.sh' you should see a reduction in instructions executed:
aldebaran:~/linux/linux> perf stat -a sleep 1
Performance counter stats for 'sleep':
16128.045994 task clock ticks (msecs)
12876 context switches (events)
219 CPU migrations (events)
186144 pagefaults (events)
20911802763 CPU cycles (events)
19309416815 instructions (events)
199608554 cache references (events)
19990754 cache misses (events)Wall-clock time elapsed: 1008.882282 msecs
With -a it's measured system-wide, from start of test to end of test
- the results will be a lot more stable (and relevant) statistically than wall-clock time or CPU usage measurements. (both of which are rather imprecise in general)
Ingo
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 26 lines · Ingo Molnar
Quoted 3 lines · Eric Dumazet
On uddpping, I had prior to the patch about 49000 wakeups per second, and after patch about 26000 wakeups per second (matches number of incoming udp messages per second)
very nice. It might not show up as a real performance difference if the CPUs are not fully saturated during the test - but it could show up as a decrease in CPU utilization.
Also, if you run the test via 'perf stat -a ./test.sh' you should see a reduction in instructions executed:
aldebaran:~/linux/linux> perf stat -a sleep 1
Performance counter stats for 'sleep':
16128.045994 task clock ticks (msecs) 12876 context switches (events) 219 CPU migrations (events) 186144 pagefaults (events) 20911802763 CPU cycles (events) 19309416815 instructions (events) 199608554 cache references (events) 19990754 cache misses (events)
Quoted 6 lines
Wall-clock time elapsed: 1008.882282 msecs
With -a it's measured system-wide, from start of test to end of test
- the results will be a lot more stable (and relevant) statistically than wall-clock time or CPU usage measurements. (both of which are rather imprecise in general)
I tried this perf stuff and got strange results on a cpu burning bench, saturating my 8 cpus with a "while (1) ;" loop
# perf stat -a sleep 10
Performance counter stats for 'sleep':
80334.709038 task clock ticks (msecs)
80638 context switches (events)
4 CPU migrations (events)
468 pagefaults (events)
160694681969 CPU cycles (events)
160127154810 instructions (events)
686393 cache references (events)
230117 cache misses (events)Wall-clock time elapsed: 10041.531644 msecs
So its about 16069468196 cycles per second for 8 cpus Divide by 8 to get 2008683524 cycles per second per cpu,
which is not 3000000000 (E5450 @ 3.00GHz)
It seems strange a "jmp myself" uses one unhalted cycle per instruction and 0.5 halted cycle ...
Also, after using "perf stat", tbench results are 1778 MB/S instead of 2610 MB/s. Even if no perf stat running.
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 51 lines · Eric Dumazet
Quoted 26 lines · Ingo Molnar
Quoted 3 lines · Eric Dumazet
On uddpping, I had prior to the patch about 49000 wakeups per second, and after patch about 26000 wakeups per second (matches number of incoming udp messages per second)
very nice. It might not show up as a real performance difference if the CPUs are not fully saturated during the test - but it could show up as a decrease in CPU utilization.
Also, if you run the test via 'perf stat -a ./test.sh' you should see a reduction in instructions executed:
aldebaran:~/linux/linux> perf stat -a sleep 1
Performance counter stats for 'sleep':
16128.045994 task clock ticks (msecs) 12876 context switches (events) 219 CPU migrations (events) 186144 pagefaults (events) 20911802763 CPU cycles (events) 19309416815 instructions (events) 199608554 cache references (events) 19990754 cache misses (events)Quoted 6 lines
Wall-clock time elapsed: 1008.882282 msecs
With -a it's measured system-wide, from start of test to end of test
- the results will be a lot more stable (and relevant) statistically than wall-clock time or CPU usage measurements. (both of which are rather imprecise in general)I tried this perf stuff and got strange results on a cpu burning bench, saturating my 8 cpus with a "while (1) ;" loop
# perf stat -a sleep 10Performance counter stats for 'sleep':
80334.709038 task clock ticks (msecs) 80638 context switches (events) 4 CPU migrations (events) 468 pagefaults (events) 160694681969 CPU cycles (events) 160127154810 instructions (events) 686393 cache references (events) 230117 cache misses (events)
Quoted 5 lines
Wall-clock time elapsed: 10041.531644 msecs
So its about 16069468196 cycles per second for 8 cpus Divide by 8 to get 2008683524 cycles per second per cpu,
which is not 3000000000 (E5450 @ 3.00GHz)
What does "perf stat -l -a sleep 10" show? I suspect your counters are scaled by about 67%, due to counter over-commit. -l will show the scaling factor (and will scale up the results).
If so then i think this behavior is confusing, and i'll make -l default-enabled. (in fact i just committed this change to latest -tip and pushed it out)
To get only instructions and cycles, do:
perf stat -e instructions -e cycles
Quoted 5 lines
It seems strange a "jmp myself" uses one unhalted cycle per instruction and 0.5 halted cycle ...
Also, after using "perf stat", tbench results are 1778 MB/S instead of 2610 MB/s. Even if no perf stat running.
Hm, that would be a bug. Could you send the dmesg output of:
echo p > /proc/sysrq-trigger
echo p > /proc/sysrq-trigger
with counters running it will show something like:
[ 868.105712] SysRq : Show Regs [ 868.106544] [ 868.106544] CPU#1: ctrl: ffffffffffffffff [ 868.106544] CPU#1: status: 0000000000000000 [ 868.106544] CPU#1: overflow: 0000000000000000 [ 868.106544] CPU#1: fixed: 0000000000000000 [ 868.106544] CPU#1: used: 0000000000000000 [ 868.106544] CPU#1: gen-PMC0 ctrl: 00000000001300c0 [ 868.106544] CPU#1: gen-PMC0 count: 000000ffee889194 [ 868.106544] CPU#1: gen-PMC0 left: 0000000011e1791a [ 868.106544] CPU#1: gen-PMC1 ctrl: 000000000013003c [ 868.106544] CPU#1: gen-PMC1 count: 000000ffd2542438 [ 868.106544] CPU#1: gen-PMC1 left: 000000002dd17a8e
the counts should stay put (i.e. all counters should be disabled). If they move around - despite there being no 'perf stat -a' session running, that would be a bug.
Also, the overhead might be profile-able, via:
perf record -m 1024 sleep 10
(this records the profile into output.perf.)
followed by:
./perf-report | tail -20
to display a histogram, with kernel-space and user-space symbols mixed into a single profile.
(Pick up latest -tip to get perf-report built by default.)
Ingo
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 53 lines · Ingo Molnar
Quoted 51 lines · Eric Dumazet
Quoted 25 lines · Ingo Molnar
Quoted 3 lines · Eric Dumazet
On uddpping, I had prior to the patch about 49000 wakeups per second, and after patch about 26000 wakeups per second (matches number of incoming udp messages per second)
very nice. It might not show up as a real performance difference if the CPUs are not fully saturated during the test - but it could show up as a decrease in CPU utilization.
Also, if you run the test via 'perf stat -a ./test.sh' you should see a reduction in instructions executed:
aldebaran:~/linux/linux> perf stat -a sleep 1
Performance counter stats for 'sleep':
16128.045994 task clock ticks (msecs) 12876 context switches (events) 219 CPU migrations (events) 186144 pagefaults (events) 20911802763 CPU cycles (events) 19309416815 instructions (events) 199608554 cache references (events) 19990754 cache misses (events)Quoted 6 lines
Wall-clock time elapsed: 1008.882282 msecs
With -a it's measured system-wide, from start of test to end of test
- the results will be a lot more stable (and relevant) statistically than wall-clock time or CPU usage measurements. (both of which are rather imprecise in general)I tried this perf stuff and got strange results on a cpu burning bench, saturating my 8 cpus with a "while (1) ;" loop
# perf stat -a sleep 10Performance counter stats for 'sleep':
80334.709038 task clock ticks (msecs) 80638 context switches (events) 4 CPU migrations (events) 468 pagefaults (events) 160694681969 CPU cycles (events) 160127154810 instructions (events) 686393 cache references (events) 230117 cache misses (events)Wall-clock time elapsed: 10041.531644 msecs
Quoted 7 lines
Quoted 3 lines
So its about 16069468196 cycles per second for 8 cpus Divide by 8 to get 2008683524 cycles per second per cpu,
which is not 3000000000 (E5450 @ 3.00GHz)What does "perf stat -l -a sleep 10" show? I suspect your counters are scaled by about 67%, due to counter over-commit. -l will show the scaling factor (and will scale up the results).
Only difference I see with '-l' is cache misses not counted.
(tbench 8 running, so not one instruction per cycle)
# perf stat -l -a sleep 10
Performance counter stats for 'sleep':
80007.128844 task clock ticks (msecs)
6754642 context switches (events)
2 CPU migrations (events)
474 pagefaults (events)
160925719143 CPU cycles (events)
108482003620 instructions (events)
7584035056 cache references (events)
<not counted> cache missesWall-clock time elapsed: 10000.595448 msecs
# perf stat -a sleep 10
Performance counter stats for 'sleep':
80702.908287 task clock ticks (msecs)
6792588 context switches (events)
24 CPU migrations (events)
4867 pagefaults (events)
161957342744 CPU cycles (events)
109147553984 instructions (events)
7633190481 cache references (events)
22996234 cache misses (events)Wall-clock time elapsed: 10087.502391 msecs
Quoted 9 lines
If so then i think this behavior is confusing, and i'll make -l default-enabled. (in fact i just committed this change to latest -tip and pushed it out)
To get only instructions and cycles, do:
perf stat -e instructions -e cycles
# perf stat -e instructions -e cycles -a sleep 10
Performance counter stats for 'sleep':
109469842392 instructions (events) 162012922122 CPU cycles (events)
Wall-clock time elapsed: 10124.943544 msecs
I am wondering if cpus are not running at 2 GHz ;)
Quoted 13 lines
Quoted 5 lines
It seems strange a "jmp myself" uses one unhalted cycle per instruction and 0.5 halted cycle ...
Also, after using "perf stat", tbench results are 1778 MB/S instead of 2610 MB/s. Even if no perf stat running.
Hm, that would be a bug. Could you send the dmesg output of:
echo p >
/proc/sysrq-trigger
echo p >/proc/sysrq-triggerwith counters running it will show something like:
Quoted 13 lines
[ 868.105712] SysRq : Show Regs [ 868.106544] [ 868.106544] CPU#1: ctrl: ffffffffffffffff [ 868.106544] CPU#1: status: 0000000000000000 [ 868.106544] CPU#1: overflow: 0000000000000000 [ 868.106544] CPU#1: fixed: 0000000000000000 [ 868.106544] CPU#1: used: 0000000000000000 [ 868.106544] CPU#1: gen-PMC0 ctrl: 00000000001300c0 [ 868.106544] CPU#1: gen-PMC0 count: 000000ffee889194 [ 868.106544] CPU#1: gen-PMC0 left: 0000000011e1791a [ 868.106544] CPU#1: gen-PMC1 ctrl: 000000000013003c [ 868.106544] CPU#1: gen-PMC1 count: 000000ffd2542438 [ 868.106544] CPU#1: gen-PMC1 left: 000000002dd17a8e
They stay fix (but only CPU#0 is displayed)
Is perf able to display per cpu counters, and not aggregated values ?
[ 7894.426787] CPU#0: ctrl: ffffffffffffffff [ 7894.426788] CPU#0: status: 0000000000000000 [ 7894.426790] CPU#0: overflow: 0000000000000000 [ 7894.426792] CPU#0: fixed: 0000000000000000 [ 7894.426793] CPU#0: used: 0000000000000000 [ 7894.426796] CPU#0: gen-PMC0 ctrl: 0000000000134f2e [ 7894.426798] CPU#0: gen-PMC0 count: 000000ffb91e31e1 [ 7894.426799] CPU#0: gen-PMC0 left: 000000007fffffff [ 7894.426802] CPU#0: gen-PMC1 ctrl: 000000000013412e [ 7894.426804] CPU#0: gen-PMC1 count: 000000ff80312b23 [ 7894.426805] CPU#0: gen-PMC1 left: 000000007fffffff [ 7894.426807] CPU#0: fixed-PMC0 count: 000000ffacf54a68 [ 7894.426809] CPU#0: fixed-PMC1 count: 000000ffb71cfe02 [ 7894.426811] CPU#0: fixed-PMC2 count: 0000000000000000 [ 7905.522262] SysRq : Show Regs [ 7905.522277] [ 7905.522279] CPU#0: ctrl: ffffffffffffffff [ 7905.522281] CPU#0: status: 0000000000000000 [ 7905.522283] CPU#0: overflow: 0000000000000000 [ 7905.522284] CPU#0: fixed: 0000000000000000 [ 7905.522286] CPU#0: used: 0000000000000000 [ 7905.522288] CPU#0: gen-PMC0 ctrl: 0000000000134f2e [ 7905.522290] CPU#0: gen-PMC0 count: 000000ffb91e31e1 [ 7905.522292] CPU#0: gen-PMC0 left: 000000007fffffff [ 7905.522294] CPU#0: gen-PMC1 ctrl: 000000000013412e [ 7905.522296] CPU#0: gen-PMC1 count: 000000ff80312b23 [ 7905.522298] CPU#0: gen-PMC1 left: 000000007fffffff [ 7905.522299] CPU#0: fixed-PMC0 count: 000000ffacf54a68 [ 7905.522301] CPU#0: fixed-PMC1 count: 000000ffb71cfe02 [ 7905.522303] CPU#0: fixed-PMC2 count: 0000000000000000
Quoted 4 lines
the counts should stay put (i.e. all counters should be disabled). If they move around - despite there being no 'perf stat -a' session running, that would be a bug.
I rebooted my machine then got good results
# perf stat -e instructions -e cycles -a sleep 10
Performance counter stats for 'sleep':
240021659058 instructions (events) 240997984836 CPU cycles (events) << OK >>
Wall-clock time elapsed: 10041.499326 msecs
But if I use plain "perf stat -a sleep 10"
it seems I get wrong values again (16 G cycles/sec) for all next perf sessions
# perf stat -a sleep 10
Performance counter stats for 'sleep':
80332.718661 task clock ticks (msecs)
80602 context switches (events)
4 CPU migrations (events)
473 pagefaults (events)
160665397757 CPU cycles (events) << bad >>
160079277974 instructions (events)
793665 cache references (events)
226829 cache misses (events)Wall-clock time elapsed: 10041.302969 msecs
# perf stat -e cycles -a sleep 10
Performance counter stats for 'sleep':
160665176575 CPU cycles (events) << bad >>
Wall-clock time elapsed: 10041.491503 msecs
Quoted 7 lines
Also, the overhead might be profile-able, via:
perf record -m 1024 sleep 10
(this records the profile into output.perf.)
Quoted 8 lines
followed by:
./perf-report | tail -20to display a histogram, with kernel-space and user-space symbols mixed into a single profile.
(Pick up latest -tip to get perf-report built by default.)
Thanks, this is what I use currently
[BUG] perf_counter: change cpu frequencies
Quoted 3 lines · Eric Dumazet
But if I use plain "perf stat -a sleep 10"
it seems I get wrong values again (16 G cycles/sec) for all next perf sessions
Well, I confirm all my cpus switched from 3GHz to 2GHz, after
"perf stat -a sleep 10"
(but "perf stat -e instructions -e cycles -a sleep 10" doesnt trigger this problem)
Nothing logged, and /proc/cpuinfo stills reports 3 GHz frequencies
# cat unit.c
main() {
int i;
for (i = 0 ; i < 10000000; i++)
getppid();
}
# time ./unitreal 0m0.818s user 0m0.289s sys 0m0.529s # perf stat -a sleep 10 2>/dev/null # time ./unit
real 0m1.122s user 0m0.482s sys 0m0.640s
# tail -n 27 /proc/cpuinfo processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5450 @ 3.00GHz stepping : 6 cpu MHz : 3000.102 cache size : 6144 KB physical id : 1 siblings : 1 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority bogomips : 6000.01 clflush size : 64 power management:
# grep CPU_FREQ .config # CONFIG_CPU_FREQ is not set
perf_counter seems promising, but still... needs some bug hunting :)
Thank you
Re: [BUG] perf_counter: change cpu frequencies
Quoted 60 lines · Eric Dumazet
Quoted 3 lines · Eric Dumazet
But if I use plain "perf stat -a sleep 10"
it seems I get wrong values again (16 G cycles/sec) for all next perf sessionsWell, I confirm all my cpus switched from 3GHz to 2GHz, after
"perf stat -a sleep 10"
(but "perf stat -e instructions -e cycles -a sleep 10" doesnt trigger this problem)
Nothing logged, and
/proc/cpuinfostills reports 3 GHz frequencies# cat unit.c main() { int i; for (i = 0 ; i < 10000000; i++) getppid(); } # time ./unitreal 0m0.818s user 0m0.289s sys 0m0.529s # perf stat -a sleep 10 2>/dev/null # time ./unitreal 0m1.122s user 0m0.482s sys 0m0.640s# tail -n 27 /proc/cpuinfo processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5450 @ 3.00GHz stepping : 6 cpu MHz : 3000.102 cache size : 6144 KB physical id : 1 siblings : 1 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority bogomips : 6000.01 clflush size : 64 power management:
Quoted 6 lines
# grep CPU_FREQ .config # CONFIG_CPU_FREQ is not set
perf_counterseems promising, but still... needs some bug hunting :)
Update :
Mike Galbraith suggested me to try various things, and finally, I discovered this frequency change was probably a BIOS problem on my HP BL460c G1
System Options -> Power regulator for Proliant
[*] HP Dynamic Power Savings Mode
[ ] HP Static Low Power Mode
[ ] HP Static High Performance Mode
[ ] OS Control Mode
I switched it to 'OS Control Mode'
Then acpi-cpufreq could load, and no more frequencies changes on a "perf -a sleep 10" session, using or not cpufreq.
(Supported cpufreq speeds on these cpus : 1999 & 2999 MHz)
So it was a BIOS issue
# perf stat -a sleep 10
Performance counter stats for 'sleep':
80005.418223 task clock ticks (msecs)
80266 context switches (events)
3 CPU migrations (events)
486 pagefaults (events)
240013851624 CPU cycles (events) << good >>
239076501419 instructions (events)
679464 cache references (events)
<not counted> cache missesWall-clock time elapsed: 10000.468808 msecs
Thank you
Re: [BUG] perf_counter: change cpu frequencies
Quoted 81 lines · Eric Dumazet
Quoted 60 lines · Eric Dumazet
Quoted 3 lines · Eric Dumazet
But if I use plain "perf stat -a sleep 10"
it seems I get wrong values again (16 G cycles/sec) for all next perf sessionsWell, I confirm all my cpus switched from 3GHz to 2GHz, after
"perf stat -a sleep 10"
(but "perf stat -e instructions -e cycles -a sleep 10" doesnt trigger this problem)
Nothing logged, and
/proc/cpuinfostills reports 3 GHz frequencies# cat unit.c main() { int i; for (i = 0 ; i < 10000000; i++) getppid(); } # time ./unitreal 0m0.818s user 0m0.289s sys 0m0.529s # perf stat -a sleep 10 2>/dev/null # time ./unitreal 0m1.122s user 0m0.482s sys 0m0.640s# tail -n 27 /proc/cpuinfo processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5450 @ 3.00GHz stepping : 6 cpu MHz : 3000.102 cache size : 6144 KB physical id : 1 siblings : 1 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority bogomips : 6000.01 clflush size : 64 power management:Quoted 6 lines
# grep CPU_FREQ .config # CONFIG_CPU_FREQ is not set
perf_counterseems promising, but still... needs some bug hunting :)Update :
Mike Galbraith suggested me to try various things, and finally, I discovered this frequency change was probably a BIOS problem on my HP BL460c G1
System Options -> Power regulator for Proliant
[*] HP Dynamic Power Savings Mode
[ ] HP Static Low Power Mode
[ ] HP Static High Performance Mode
[ ] OS Control Mode
Quoted 7 lines
I switched it to 'OS Control Mode'
Then acpi-cpufreq could load, and no more frequencies changes on a "perf -a sleep 10" session, using or not cpufreq.
(Supported cpufreq speeds on these cpus : 1999 & 2999 MHz)So it was a BIOS issue
ah! That makes quite a bit of sense. The BIOS interfering with an OS feature ... Was that the default setting in the BIOS?
Quoted 13 lines
# perf stat -a sleep 10Performance counter stats for 'sleep':
80005.418223 task clock ticks (msecs) 80266 context switches (events) 3 CPU migrations (events) 486 pagefaults (events) 240013851624 CPU cycles (events) << good >> 239076501419 instructions (events) 679464 cache references (events) <not counted> cache misses
Quoted 1 line
Wall-clock time elapsed: 10000.468808 msecs
That looks perfect now.
It would also be really nice to have a sysrq-p dump of your PMU state before you've done any profiling. Is there any trace of the BIOS meddling with them, that we could detect (and warn about) during bootup?
Thanks,
Ingo
Re: [BUG] perf_counter: change cpu frequencies
Quoted 83 lines · Ingo Molnar
Quoted 81 lines · Eric Dumazet
Quoted 59 lines · Eric Dumazet
Quoted 3 lines · Eric Dumazet
But if I use plain "perf stat -a sleep 10"
it seems I get wrong values again (16 G cycles/sec) for all next perf sessionsWell, I confirm all my cpus switched from 3GHz to 2GHz, after
"perf stat -a sleep 10"
(but "perf stat -e instructions -e cycles -a sleep 10" doesnt trigger this problem)
Nothing logged, and
/proc/cpuinfostills reports 3 GHz frequencies# cat unit.c main() { int i; for (i = 0 ; i < 10000000; i++) getppid(); } # time ./unitreal 0m0.818s user 0m0.289s sys 0m0.529s # perf stat -a sleep 10 2>/dev/null # time ./unitreal 0m1.122s user 0m0.482s sys 0m0.640s# tail -n 27 /proc/cpuinfo processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5450 @ 3.00GHz stepping : 6 cpu MHz : 3000.102 cache size : 6144 KB physical id : 1 siblings : 1 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority bogomips : 6000.01 clflush size : 64 power management:Quoted 6 lines
# grep CPU_FREQ .config # CONFIG_CPU_FREQ is not set
perf_counterseems promising, but still... needs some bug hunting :)Update :
Mike Galbraith suggested me to try various things, and finally, I discovered this frequency change was probably a BIOS problem on my HP BL460c G1
System Options -> Power regulator for Proliant
[*] HP Dynamic Power Savings Mode
[ ] HP Static Low Power Mode
[ ] HP Static High Performance Mode
[ ] OS Control ModeI switched it to 'OS Control Mode'
Quoted 8 lines
Quoted 5 lines
Then acpi-cpufreq could load, and no more frequencies changes on a "perf -a sleep 10" session, using or not cpufreq.
(Supported cpufreq speeds on these cpus : 1999 & 2999 MHz)So it was a BIOS issue
ah! That makes quite a bit of sense. The BIOS interfering with an OS feature ... Was that the default setting in the BIOS?
This was default setting in BIOS, yes.
Quoted 14 lines
Quoted 14 lines
# perf stat -a sleep 10Performance counter stats for 'sleep':
80005.418223 task clock ticks (msecs) 80266 context switches (events) 3 CPU migrations (events) 486 pagefaults (events) 240013851624 CPU cycles (events) << good >> 239076501419 instructions (events) 679464 cache references (events) <not counted> cache misses
Quoted 8 lines
Quoted 1 line
Wall-clock time elapsed: 10000.468808 msecs
That looks perfect now.
It would also be really nice to have a sysrq-p dump of your PMU state before you've done any profiling. Is there any trace of the BIOS meddling with them, that we could detect (and warn about) during bootup?
Difference is that on BIOS set to 'OS Control Mode' I see one more entry in ACPI list :
[ 0.000000] ACPI: SSDT cfe5b000 004C9 (v01 HP SSDTP 00000001 INTL 20030228)
...
And these 8 additional lines after (one per cpu)
[ 0.706697] ACPI: SSDT cfe5c000 002DA (v01 HP SSDT0 00000001 INTL 20030228) [ 0.707250] ACPI: SSDT cfe5c300 002DA (v01 HP SSDT1 00000001 INTL 20030228) [ 0.707768] ACPI: SSDT cfe5c600 002DA (v01 HP SSDT2 00000001 INTL 20030228) [ 0.708376] ACPI: SSDT cfe5c900 002DF (v01 HP SSDT3 00000001 INTL 20030228) [ 0.708964] ACPI: SSDT cfe5cc00 002DA (v01 HP SSDT4 00000001 INTL 20030228) [ 0.709567] ACPI: SSDT cfe5cf00 002DA (v01 HP SSDT5 00000001 INTL 20030228) [ 0.710122] ACPI: SSDT cfe5d200 002DA (v01 HP SSDT6 00000001 INTL 20030228) [ 0.710713] ACPI: SSDT cfe5d500 002DA (v01 HP SSDT7 00000001 INTL 20030228)
Also, if this option is set to default (HP Dynamic Power Savings Mode) I get :
# modprobe acpi-cpufreq FATAL: Error inserting acpi_cpufreq (/lib/modules/2.6.30-rc4-tip-01560-gdd5fa92/kernel/arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.ko): No such device
but no kernel message logged.
Might be possible to add some kind of warning yes, I can test a patch if you want.
Re: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Eric DumazetQuoted 194 lines · Eric Dumazet
Quoted 53 lines · Ingo Molnar
Quoted 51 lines · Eric Dumazet
Quoted 25 lines · Ingo Molnar
Quoted 3 lines · Eric Dumazet
On uddpping, I had prior to the patch about 49000 wakeups per second, and after patch about 26000 wakeups per second (matches number of incoming udp messages per second)
very nice. It might not show up as a real performance difference if the CPUs are not fully saturated during the test - but it could show up as a decrease in CPU utilization.
Also, if you run the test via 'perf stat -a ./test.sh' you should see a reduction in instructions executed:
aldebaran:~/linux/linux> perf stat -a sleep 1
Performance counter stats for 'sleep':
16128.045994 task clock ticks (msecs) 12876 context switches (events) 219 CPU migrations (events) 186144 pagefaults (events) 20911802763 CPU cycles (events) 19309416815 instructions (events) 199608554 cache references (events) 19990754 cache misses (events)Quoted 6 lines
Wall-clock time elapsed: 1008.882282 msecs
With -a it's measured system-wide, from start of test to end of test
- the results will be a lot more stable (and relevant) statistically than wall-clock time or CPU usage measurements. (both of which are rather imprecise in general)I tried this perf stuff and got strange results on a cpu burning bench, saturating my 8 cpus with a "while (1) ;" loop
# perf stat -a sleep 10Performance counter stats for 'sleep':
80334.709038 task clock ticks (msecs) 80638 context switches (events) 4 CPU migrations (events) 468 pagefaults (events) 160694681969 CPU cycles (events) 160127154810 instructions (events) 686393 cache references (events) 230117 cache misses (events)Wall-clock time elapsed: 10041.531644 msecs
Quoted 7 lines
Quoted 3 lines
So its about 16069468196 cycles per second for 8 cpus Divide by 8 to get 2008683524 cycles per second per cpu,
which is not 3000000000 (E5450 @ 3.00GHz)What does "perf stat -l -a sleep 10" show? I suspect your counters are scaled by about 67%, due to counter over-commit. -l will show the scaling factor (and will scale up the results).
Only difference I see with '-l' is cache misses not counted.
(tbench 8 running, so not one instruction per cycle)
# perf stat -l -a sleep 10Performance counter stats for 'sleep':
80007.128844 task clock ticks (msecs) 6754642 context switches (events) 2 CPU migrations (events) 474 pagefaults (events) 160925719143 CPU cycles (events) 108482003620 instructions (events) 7584035056 cache references (events) <not counted> cache missesWall-clock time elapsed: 10000.595448 msecs
# perf stat -a sleep 10Performance counter stats for 'sleep':
80702.908287 task clock ticks (msecs) 6792588 context switches (events) 24 CPU migrations (events) 4867 pagefaults (events) 161957342744 CPU cycles (events) 109147553984 instructions (events) 7633190481 cache references (events) 22996234 cache misses (events)Wall-clock time elapsed: 10087.502391 msecs
Quoted 9 lines
If so then i think this behavior is confusing, and i'll make -l default-enabled. (in fact i just committed this change to latest -tip and pushed it out)
To get only instructions and cycles, do:
perf stat -e instructions -e cycles
# perf stat -e instructions -e cycles -a sleep 10Performance counter stats for 'sleep':
109469842392 instructions (events) 162012922122 CPU cycles (events)Wall-clock time elapsed: 10124.943544 msecs
I am wondering if cpus are not running at 2 GHz ;)
Quoted 13 lines
Quoted 5 lines
It seems strange a "jmp myself" uses one unhalted cycle per instruction and 0.5 halted cycle ...
Also, after using "perf stat", tbench results are 1778 MB/S instead of 2610 MB/s. Even if no perf stat running.
Hm, that would be a bug. Could you send the dmesg output of:
echo p >
/proc/sysrq-trigger
echo p >/proc/sysrq-triggerwith counters running it will show something like:
Quoted 13 lines
[ 868.105712] SysRq : Show Regs [ 868.106544] [ 868.106544] CPU#1: ctrl: ffffffffffffffff [ 868.106544] CPU#1: status: 0000000000000000 [ 868.106544] CPU#1: overflow: 0000000000000000 [ 868.106544] CPU#1: fixed: 0000000000000000 [ 868.106544] CPU#1: used: 0000000000000000 [ 868.106544] CPU#1: gen-PMC0 ctrl: 00000000001300c0 [ 868.106544] CPU#1: gen-PMC0 count: 000000ffee889194 [ 868.106544] CPU#1: gen-PMC0 left: 0000000011e1791a [ 868.106544] CPU#1: gen-PMC1 ctrl: 000000000013003c [ 868.106544] CPU#1: gen-PMC1 count: 000000ffd2542438 [ 868.106544] CPU#1: gen-PMC1 left: 000000002dd17a8eThey stay fix (but only CPU#0 is displayed)
Is perf able to display per cpu counters, and not aggregated values ?
[ 7894.426787] CPU#0: ctrl: ffffffffffffffff [ 7894.426788] CPU#0: status: 0000000000000000 [ 7894.426790] CPU#0: overflow: 0000000000000000 [ 7894.426792] CPU#0: fixed: 0000000000000000 [ 7894.426793] CPU#0: used: 0000000000000000 [ 7894.426796] CPU#0: gen-PMC0 ctrl: 0000000000134f2e [ 7894.426798] CPU#0: gen-PMC0 count: 000000ffb91e31e1 [ 7894.426799] CPU#0: gen-PMC0 left: 000000007fffffff [ 7894.426802] CPU#0: gen-PMC1 ctrl: 000000000013412e [ 7894.426804] CPU#0: gen-PMC1 count: 000000ff80312b23 [ 7894.426805] CPU#0: gen-PMC1 left: 000000007fffffff [ 7894.426807] CPU#0: fixed-PMC0 count: 000000ffacf54a68 [ 7894.426809] CPU#0: fixed-PMC1 count: 000000ffb71cfe02 [ 7894.426811] CPU#0: fixed-PMC2 count: 0000000000000000 [ 7905.522262] SysRq : Show Regs [ 7905.522277] [ 7905.522279] CPU#0: ctrl: ffffffffffffffff [ 7905.522281] CPU#0: status: 0000000000000000 [ 7905.522283] CPU#0: overflow: 0000000000000000 [ 7905.522284] CPU#0: fixed: 0000000000000000 [ 7905.522286] CPU#0: used: 0000000000000000 [ 7905.522288] CPU#0: gen-PMC0 ctrl: 0000000000134f2e [ 7905.522290] CPU#0: gen-PMC0 count: 000000ffb91e31e1 [ 7905.522292] CPU#0: gen-PMC0 left: 000000007fffffff [ 7905.522294] CPU#0: gen-PMC1 ctrl: 000000000013412e [ 7905.522296] CPU#0: gen-PMC1 count: 000000ff80312b23 [ 7905.522298] CPU#0: gen-PMC1 left: 000000007fffffff [ 7905.522299] CPU#0: fixed-PMC0 count: 000000ffacf54a68 [ 7905.522301] CPU#0: fixed-PMC1 count: 000000ffb71cfe02 [ 7905.522303] CPU#0: fixed-PMC2 count: 0000000000000000Quoted 4 lines
the counts should stay put (i.e. all counters should be disabled). If they move around - despite there being no 'perf stat -a' session running, that would be a bug.
I rebooted my machine then got good results
# perf stat -e instructions -e cycles -a sleep 10Performance counter stats for 'sleep':
Quoted 7 lines
240021659058 instructions (events) 240997984836 CPU cycles (events) << OK >>Wall-clock time elapsed: 10041.499326 msecs
But if I use plain "perf stat -a sleep 10"
it seems I get wrong values again (16 G cycles/sec) for all next perf sessions
I have to ask...
Is it possible that the machine runs at 3GHz initially, but slows down to 2GHz for cooling reasons? One thing to try would be to run powertop, which displays the frequencies. I get the following if mostly idle:
PowerTOP version 1.8 (C) 2007 Intel Corporation Cn Avg residency P-states (frequencies) C0 (cpu running) (14.1%) 2.17 Ghz 4.3% C1 0.0ms ( 0.0%) 1.67 Ghz 0.0% C2 0.5ms (16.2%) 1333 Mhz 0.0% C3 0.5ms (69.8%) 1000 Mhz 95.7%
And the following with an infinite loop running:
PowerTOP version 1.8 (C) 2007 Intel Corporation Cn Avg residency P-states (frequencies) C0 (cpu running) (54.3%) 2.17 Ghz 100.0% C1 0.0ms ( 0.0%) 1.67 Ghz 0.0% C2 1.2ms ( 1.7%) 1333 Mhz 0.0% C3 1.3ms (44.0%) 1000 Mhz 0.0%
But I am probably missing the point here...
Thanx, Paul
Quoted 48 lines
# perf stat -a sleep 10Performance counter stats for 'sleep':
80332.718661 task clock ticks (msecs) 80602 context switches (events) 4 CPU migrations (events) 473 pagefaults (events) 160665397757 CPU cycles (events) << bad >> 160079277974 instructions (events) 793665 cache references (events) 226829 cache misses (events)Wall-clock time elapsed: 10041.302969 msecs
# perf stat -e cycles -a sleep 10Performance counter stats for 'sleep':
160665176575 CPU cycles (events) << bad >>Wall-clock time elapsed: 10041.491503 msecs
Quoted 7 lines
Also, the overhead might be profile-able, via:
perf record -m 1024 sleep 10
(this records the profile into output.perf.)
Quoted 8 lines
followed by:
./perf-report | tail -20to display a histogram, with kernel-space and user-space symbols mixed into a single profile.
(Pick up latest -tip to get perf-report built by default.)
Thanks, this is what I use currently
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Eric DumazetQuoted 9 lines · On Tue, 28 Apr 2009 17:06:11 +0200 Eric Dumazet <[email protected]> wrote:
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call
sock_wfree()which in turn schedules thread.
Quoted 9 lines
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
Seems that this is a virtuous patch even though Christoph is struggling a bit to test it?
Quoted 47 lines
fs/select.c | 28 +++++++++++++++++++++++++--- include/linux/poll.h | 3 +++ 2 files changed, 28 insertions(+), 3 deletions(-)fs/select.c +12
-2@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll)
fs/select.c int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 441
Quoted 10 lines
fs/select.c +9
-0+if (f_op && f_op->poll) { +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } mask = (*f_op->poll)(file, retval ?NULL : wait); + }
<resizes xterm rather a lot>
Can we (and should we) avoid all that manipulation of wait->key if `retval' is zero?
Quoted 11 lines
include/linux/poll.h +1
-0@@ -32,6 +32,7 @@ typedef void (*poll_queue_proc)(struct file *, wait_queue_head_t *, struct poll_typedef struct poll_table_struct {poll_queue_proc qproc; +unsigned long key; }poll_table ;static inline void poll_wait (struct file * filp,wait_queue_head_t * wait_address,poll_table *p)
include/linux/poll.h static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addres · line 47
Quoted 4 lines
include/linux/poll.h +1
-0@@ -43,10 +44,12 @@ static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addresstatic inline void init_poll_funcptr (poll_table *pt,poll_queue_proc qproc) { pt->qproc = qproc; + pt->key = ~0UL ;/* all events enabled */
I kind of prefer to use plain old -1 for the all-ones pattern. Because it always just works, and doesn't send the reviewer off to check if the type was really u64 or something.
It's a bit ugly though.
Quoted 8 lines
include/linux/poll.h +1
-0}struct poll_table_entry {struct file *filp; +unsigned long key;wait_queue_t wait;wait_queue_head_t *wait_address; };
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 2 lines
Seems that this is a virtuous patch even though Christoph is struggling a bit to test it?
The main drawback is that the select/poll data structures will get larger. That could cause regression in theory. But I suspect the win in some situations is still worth it. Of course it would be nice if it handled more situations (like multiple reader etc.)
-Andi
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 8 lines · Andi Kleen
Quoted 2 lines
Seems that this is a virtuous patch even though Christoph is struggling a bit to test it?
The main drawback is that the select/poll data structures will get larger. That could cause regression in theory. But I suspect the win in some situations is still worth it. Of course it would be nice if it handled more situations (like multiple reader etc.)
On poll()/select() interface, we must wakeup every pollers, because we dont know if they really will consume the event
thread 1:poll();
<insert an exit() or something bad here>read();
thread 2:poll(); /* no return because event was 'granted' to thread 1 */ read();
We could try to optimize read()/recvfrom() because we can know if event is consumed, as its a whole syscall.
Ingo Molnar
Acked-byRe: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Andi KleenQuoted 5 lines · Andi Kleen
Quoted 2 lines
Seems that this is a virtuous patch even though Christoph is struggling a bit to test it?
The main drawback is that the select/poll data structures will get larger. That could cause regression in theory. [...]
Current size of struct poll_table_entry is 0x38 on 64-bit kernels. Adding the key will make it 0x40 - which is not only a power of two but also matches cache line size on most modern CPUs.
So the size of this structure is ideal now and arithmetics on the poll table have become simpler as well.
So the patch has my ack:
Ingo
Re: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Andrew MortonQuoted 84 lines · Andrew Morton
Quoted 9 lines · On Tue, 28 Apr 2009 17:06:11 +0200 Eric Dumazet <[email protected]> wrote:
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call
sock_wfree()which in turn schedules thread.Quoted 9 lines
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
Seems that this is a virtuous patch even though Christoph is struggling a bit to test it?
Quoted 47 lines
fs/select.c | 28 +++++++++++++++++++++++++--- include/linux/poll.h | 3 +++ 2 files changed, 28 insertions(+), 3 deletions(-)fs/select.c +12
-2@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll)fs/select.c int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 441
Quoted 10 lines
fs/select.c +9
-0+if (f_op && f_op->poll) { +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } mask = (*f_op->poll)(file, retval ?NULL : wait); + }<resizes xterm rather a lot>
Quoted 2 lines
Can we (and should we) avoid all that manipulation of
wait->keyif `retval' is zero?
yes, we could set wait to NULL as soon as retval is incremented. and also do :
mask = (*f_op->poll)(file, wait);
Quoted 39 lines
Quoted 12 lines
include/linux/poll.h +1
-0@@ -32,6 +32,7 @@ typedef void (*poll_queue_proc)(struct file *, wait_queue_head_t *, struct poll_typedef struct poll_table_struct {poll_queue_proc qproc; +unsigned long key; }poll_table ;static inline void poll_wait (struct file * filp,wait_queue_head_t * wait_address,poll_table *p)include/linux/poll.h static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addres · line 47
Quoted 4 lines
include/linux/poll.h +1
-0@@ -43,10 +44,12 @@ static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addresstatic inline void init_poll_funcptr (poll_table *pt,poll_queue_proc qproc) { pt->qproc = qproc; + pt->key = ~0UL ;/* all events enabled */ I kind of prefer to use plain old -1 for the all-ones pattern. Because it always just works, and doesn't send the reviewer off to check if the type was really u64 or something.
It's a bit ugly though.
include/linux/poll.h static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addres · line 55
Quoted 8 lines
include/linux/poll.h +1
-0}struct poll_table_entry {struct file *filp; +unsigned long key;wait_queue_t wait;wait_queue_head_t *wait_address; };--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 83 lines · Eric Dumazet
Quoted 82 lines · Andrew Morton
Quoted 9 lines · On Tue, 28 Apr 2009 17:06:11 +0200 Eric Dumazet <[email protected]> wrote:
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call
sock_wfree()which in turn schedules thread.Quoted 9 lines
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
Seems that this is a virtuous patch even though Christoph is struggling a bit to test it?
Quoted 47 lines
fs/select.c | 28 +++++++++++++++++++++++++--- include/linux/poll.h | 3 +++ 2 files changed, 28 insertions(+), 3 deletions(-)fs/select.c +12
-2@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll)fs/select.c int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 441
Quoted 10 lines
fs/select.c +9
-0+if (f_op && f_op->poll) { +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } mask = (*f_op->poll)(file, retval ?NULL : wait); + }<resizes xterm rather a lot>
Quoted 8 lines
Quoted 2 lines
Can we (and should we) avoid all that manipulation of
wait->keyif `retval' is zero?yes, we could set wait to NULL as soon as retval is incremented. and also do :
mask = (*f_op->poll)(file, wait);
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call sock_wfree() which in turn schedules thread.
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
--- fs/select.c | 33 +++++++++++++++++++++++++++++---- include/linux/poll.h | 3 +++ 2 files changed, 32 insertions(+), 4 deletions(-)
fs/select.c +29 -4
@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);
@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)
@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);
@@ -418,20 +429,31 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll) - mask = (*f_op->poll)(file, retval ?NULL : wait); +if (f_op && f_op->poll) { +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } + mask = (*f_op->poll)(file, wait); + }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit; retval++; + wait =NULL ; }if ((mask &POLLOUT_SET ) && (out & bit)) { res_out |= bit; retval++; + wait =NULL ; }if ((mask &POLLEX_SET ) && (ex & bit)) { res_ex |= bit; retval++; + wait =NULL ; } } }
@@ -685,8 +707,11 @@ static inline unsigned int do_pollfd(struct pollfd *pollfd, poll_table *pwait) mask =POLLNVAL ;if (file !=NULL ) { mask =DEFAULT_POLLMASK ; -if (file->f_op && file->f_op->poll) +if (file->f_op && file->f_op->poll) { +if (pwait) + pwait->key = pollfd->events |POLLERR |POLLHUP ; mask = file->f_op->poll (file, pwait); + }/* Mask out unneeded events. */ mask &= pollfd->events |POLLERR |POLLHUP ;fput_light (file, fput_needed);
include/linux/poll.h +3 -0
@@ -32,6 +32,7 @@ typedef void (*poll_queue_proc)(struct file *, wait_queue_head_t *, struct poll_typedef struct poll_table_struct {poll_queue_proc qproc; +unsigned long key; }poll_table ;static inline void poll_wait (struct file * filp,wait_queue_head_t * wait_address,poll_table *p)
@@ -43,10 +44,12 @@ static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addresstatic inline void init_poll_funcptr (poll_table *pt,poll_queue_proc qproc) { pt->qproc = qproc; + pt->key = ~0UL ;/* all events enabled */ }struct poll_table_entry {struct file *filp; +unsigned long key;wait_queue_t wait;wait_queue_head_t *wait_address; };
Re: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Eric DumazetQuoted 8 lines · Eric Dumazet
fs/select.c +3
-1@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll) +if (f_op && f_op->poll) { +if (wait) { + wait->key =POLLEX_SET ;
fs/select.c int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 444
Quoted 10 lines
fs/select.c +6
-0+if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } mask = (*f_op->poll)(file, retval ?NULL : wait); + }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit;
Please factor this whole 'if (file)' branch out into a helper. Typical indentation levels go from 1 to 3 tabs - 4 should be avoided if possible and 5 is pretty excessive already. This goes to eight.
Ingo
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 16 lines · Ingo Molnar
Quoted 14 lines · Eric Dumazet
(quoted diff) +8
-1@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time) if (file) { f_op = file->f_op; mask = DEFAULT_POLLMASK; - if (f_op && f_op->poll) + if (f_op && f_op->poll) { + if (wait) { + wait->key = POLLEX_SET; + if (in & bit) + wait->key |= POLLIN_SET; + if (out & bit) + wait->key |= POLLOUT_SET; + } mask = (*f_op->poll)(file, retval ? NULL : wait);
Quoted 9 lines
Review int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 444
Quoted 4 lines
(quoted diff) +1
-0+ } fput_light(file, fput_needed); if ((mask & POLLIN_SET) && (in & bit)) { res_in |= bit;Please factor this whole 'if (file)' branch out into a helper. Typical indentation levels go from 1 to 3 tabs - 4 should be avoided if possible and 5 is pretty excessive already. This goes to eight.
Thanks Ingo,
Here is v3 of patch, with your Acked-by included :)
This is IMHO clearer since helper immediatly follows POLLIN_SET / POLLOUT_SET / POLLEX_SET defines.
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call sock_wfree() which in turn schedules thread.
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
--- fs/select.c | 40 ++++++++++++++++++++++++++++++++++++---- include/linux/poll.h | 3 +++ 2 files changed, 39 insertions(+), 4 deletions(-)
fs/select.c +36 -4
@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);
@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)
@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);
@@ -362,6 +373,18 @@ get_max:#define POLLOUT_SET (POLLWRBAND | POLLWRNORM | POLLOUT | POLLERR)#define POLLEX_SET (POLLPRI) +static void wait_key_set (poll_table *wait,unsigned long in, +unsigned long out,unsigned long bit) +{ +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } +} +int do_select (int n,fd_set_bits *fds,struct timespec *end_time) {ktime_t expire, *to =NULL ;
@@ -418,20 +441,25 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll) - mask = (*f_op->poll)(file, retval ?NULL : wait); +if (f_op && f_op->poll) { +wait_key_set (wait, in, out, bit); + mask = (*f_op->poll)(file, wait); + }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit; retval++; + wait =NULL ; }if ((mask &POLLOUT_SET ) && (out & bit)) { res_out |= bit; retval++; + wait =NULL ; }if ((mask &POLLEX_SET ) && (ex & bit)) { res_ex |= bit; retval++; + wait =NULL ; } } }
@@ -685,8 +713,12 @@ static inline unsigned int do_pollfd(struct pollfd *pollfd, poll_table *pwait) mask =POLLNVAL ;if (file !=NULL ) { mask =DEFAULT_POLLMASK ; -if (file->f_op && file->f_op->poll) +if (file->f_op && file->f_op->poll) { +if (pwait) + pwait->key = pollfd->events | +POLLERR |POLLHUP ; mask = file->f_op->poll (file, pwait); + }/* Mask out unneeded events. */ mask &= pollfd->events |POLLERR |POLLHUP ;fput_light (file, fput_needed);
include/linux/poll.h +3 -0
@@ -32,6 +32,7 @@ typedef void (*poll_queue_proc)(struct file *, wait_queue_head_t *, struct poll_typedef struct poll_table_struct {poll_queue_proc qproc; +unsigned long key; }poll_table ;static inline void poll_wait (struct file * filp,wait_queue_head_t * wait_address,poll_table *p)
@@ -43,10 +44,12 @@ static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addresstatic inline void init_poll_funcptr (poll_table *pt,poll_queue_proc qproc) { pt->qproc = qproc; + pt->key = ~0UL ;/* all events enabled */ }struct poll_table_entry {struct file *filp; +unsigned long key;wait_queue_t wait;wait_queue_head_t *wait_address; };
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 105 lines · Eric Dumazet
Quoted 16 lines · Ingo Molnar
Quoted 14 lines · Eric Dumazet
(quoted diff) +8
-1@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time) if (file) { f_op = file->f_op; mask = DEFAULT_POLLMASK; - if (f_op && f_op->poll) + if (f_op && f_op->poll) { + if (wait) { + wait->key = POLLEX_SET; + if (in & bit) + wait->key |= POLLIN_SET; + if (out & bit) + wait->key |= POLLOUT_SET; + } mask = (*f_op->poll)(file, retval ? NULL : wait);Quoted 9 lines
fs/select.c int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 450
Quoted 4 lines
fs/select.c +1
-0+ }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit;Please factor this whole 'if (file)' branch out into a helper. Typical indentation levels go from 1 to 3 tabs - 4 should be avoided if possible and 5 is pretty excessive already. This goes to eight.
Thanks Ingo,
Here is v3 of patch, with your Acked-by included :)
This is IMHO clearer since helper immediatly follows
POLLIN_SET/POLLOUT_SET/POLLEX_SETdefines.[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call
sock_wfree()which in turn schedules thread.When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
Signed-off-by Eric Dumazet <[email protected]>Acked-by David S. Miller <[email protected]>Acked-by Andi Kleen <[email protected]>Acked-by Ingo Molnar <[email protected]>--- fs/select.c | 40 ++++++++++++++++++++++++++++++++++++---- include/linux/poll.h | 3 +++ 2 files changed, 39 insertions(+), 4 deletions(-)fs/select.c +13
-1@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);@@ -362,6 +373,18 @@ get_max:#define POLLOUT_SET (POLLWRBAND | POLLWRNORM | POLLOUT | POLLERR)#define POLLEX_SET (POLLPRI) +static void wait_key_set(poll_table *wait, unsigned long in,
fs/select.c get_max: · line 386
Quoted 10 lines
fs/select.c +10
-0+ unsigned long out, unsigned long bit) +{ +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } +}
should be inline perhaps?
Quoted 22 lines
fs/select.c +6
-2+int do_select (int n,fd_set_bits *fds,struct timespec *end_time) {ktime_t expire, *to =NULL ;@@ -418,20 +441,25 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll) - mask = (*f_op->poll)(file, retval ?NULL : wait); +if (f_op && f_op->poll) { +wait_key_set (wait, in, out, bit); + mask = (*f_op->poll)(file, wait); + }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit; retval++; + wait =NULL ; }if ((mask &POLLOUT_SET ) && (out & bit)) { res_out |= bit;
fs/select.c int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 465
Quoted 10 lines
fs/select.c +2
-0retval++; + wait =NULL ; }if ((mask &POLLEX_SET ) && (ex & bit)) { res_ex |= bit; retval++; + wait =NULL ; } } }
Looks much nicer now! [ I'd still suggest to factor out the guts of do_select() as its nesting is excessive that hurts its reviewability quite a bit - but now your patch does not make the situation any worse. ]
Ingo
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 104 lines · Ingo Molnar
Quoted 102 lines · Eric Dumazet
Quoted 15 lines · Ingo Molnar
Quoted 13 lines · Eric Dumazet
(quoted diff) +8
-1@@ -418,8 +429,16 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time) if (file) { f_op = file->f_op; mask = DEFAULT_POLLMASK; - if (f_op && f_op->poll) + if (f_op && f_op->poll) { + if (wait) { + wait->key = POLLEX_SET; + if (in & bit) + wait->key |= POLLIN_SET; + if (out & bit) + wait->key |= POLLOUT_SET; + }Quoted 9 lines
Review int do_select(int n, fd_set_bits *fds, struct timespec *end_time) · line 444
Quoted 5 lines
(quoted diff) +1
-0mask = (*f_op->poll)(file, retval ? NULL : wait); + } fput_light(file, fput_needed); if ((mask & POLLIN_SET) && (in & bit)) { res_in |= bit;Please factor this whole 'if (file)' branch out into a helper. Typical indentation levels go from 1 to 3 tabs - 4 should be avoided if possible and 5 is pretty excessive already. This goes to eight.
Thanks Ingo,
Here is v3 of patch, with your Acked-by included :)
This is IMHO clearer since helper immediatly follows
POLLIN_SET/POLLOUT_SET/POLLEX_SETdefines.[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call
sock_wfree()which in turn schedules thread.When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
Signed-off-by Eric Dumazet <[email protected]>Acked-by David S. Miller <[email protected]>Acked-by Andi Kleen <[email protected]>Acked-by Ingo Molnar <[email protected]>--- fs/select.c | 40 ++++++++++++++++++++++++++++++++++++---- include/linux/poll.h | 3 +++ 2 files changed, 39 insertions(+), 4 deletions(-)fs/select.c +12
-1@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);@@ -362,6 +373,18 @@ get_max:#define POLLOUT_SET (POLLWRBAND | POLLWRNORM | POLLOUT | POLLERR)#define POLLEX_SET (POLLPRI)
Quoted 13 lines
fs/select.c get_max: · line 386
Quoted 11 lines
fs/select.c +11
-0+static void wait_key_set (poll_table *wait,unsigned long in, +unsigned long out,unsigned long bit) +{ +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } +}should be inline perhaps?
Well, I thought current practice was not using inline for such trivial functions, as gcc already inlines them anyway.
Quoting Documentation/CodingStyle :
Often people argue that adding inline to functions that are static and used only once is always a win since there is no space tradeoff. While this is technically correct, gcc is capable of inlining these automatically without help, and the maintenance issue of removing the inline when a second user appears outweighs the potential value of the hint that tells gcc to do something it would have done anyway.
Anyway :
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call sock_wfree() which in turn schedules thread.
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
--- fs/select.c | 40 ++++++++++++++++++++++++++++++++++++---- include/linux/poll.h | 3 +++ 2 files changed, 39 insertions(+), 4 deletions(-)
fs/select.c +36 -4
@@ -168,7 +168,7 @@ static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)return table->entry++; } -static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +static int __pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) {struct poll_wqueues *pwq = wait->private;DECLARE_WAITQUEUE (dummy_wait, pwq->polling_task);
@@ -194,6 +194,16 @@ static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)return default_wake_function (&dummy_wait, mode, sync, key); } +static int pollwake (wait_queue_t *wait,unsigned mode,int sync,void *key) +{ +struct poll_table_entry *entry; + + entry = container_of(wait, struct poll_table_entry, wait); +if (key && !((unsigned long )key & entry->key)) +return 0 ; +return __pollwake (wait, mode, sync, key); +} +/* Add a new entry */ static void __pollwait (struct file *filp,wait_queue_head_t *wait_address,poll_table *p)
@@ -205,6 +215,7 @@ static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,get_file (filp); entry->filp = filp; entry->wait_address = wait_address; + entry->key = p->key;init_waitqueue_func_entry (&entry->wait, pollwake); entry->wait.private = pwq;add_wait_queue (wait_address, &entry->wait);
@@ -362,6 +373,18 @@ get_max:#define POLLOUT_SET (POLLWRBAND | POLLWRNORM | POLLOUT | POLLERR)#define POLLEX_SET (POLLPRI) +static inline void wait_key_set (poll_table *wait,unsigned long in, +unsigned long out,unsigned long bit) +{ +if (wait) { + wait->key =POLLEX_SET ; +if (in & bit) + wait->key |=POLLIN_SET ; +if (out & bit) + wait->key |=POLLOUT_SET ; + } +} +int do_select (int n,fd_set_bits *fds,struct timespec *end_time) {ktime_t expire, *to =NULL ;
@@ -418,20 +441,25 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time)if (file) { f_op = file->f_op; mask =DEFAULT_POLLMASK ; -if (f_op && f_op->poll) - mask = (*f_op->poll)(file, retval ?NULL : wait); +if (f_op && f_op->poll) { +wait_key_set (wait, in, out, bit); + mask = (*f_op->poll)(file, wait); + }fput_light (file, fput_needed);if ((mask &POLLIN_SET ) && (in & bit)) { res_in |= bit; retval++; + wait =NULL ; }if ((mask &POLLOUT_SET ) && (out & bit)) { res_out |= bit; retval++; + wait =NULL ; }if ((mask &POLLEX_SET ) && (ex & bit)) { res_ex |= bit; retval++; + wait =NULL ; } } }
@@ -685,8 +713,12 @@ static inline unsigned int do_pollfd(struct pollfd *pollfd, poll_table *pwait) mask =POLLNVAL ;if (file !=NULL ) { mask =DEFAULT_POLLMASK ; -if (file->f_op && file->f_op->poll) +if (file->f_op && file->f_op->poll) { +if (pwait) + pwait->key = pollfd->events | +POLLERR |POLLHUP ; mask = file->f_op->poll (file, pwait); + }/* Mask out unneeded events. */ mask &= pollfd->events |POLLERR |POLLHUP ;fput_light (file, fput_needed);
include/linux/poll.h +3 -0
@@ -32,6 +32,7 @@ typedef void (*poll_queue_proc)(struct file *, wait_queue_head_t *, struct poll_typedef struct poll_table_struct {poll_queue_proc qproc; +unsigned long key; }poll_table ;static inline void poll_wait (struct file * filp,wait_queue_head_t * wait_address,poll_table *p)
@@ -43,10 +44,12 @@ static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_addresstatic inline void init_poll_funcptr (poll_table *pt,poll_queue_proc qproc) { pt->qproc = qproc; + pt->key = ~0UL ;/* all events enabled */ }struct poll_table_entry {struct file *filp; +unsigned long key;wait_queue_t wait;wait_queue_head_t *wait_address; };
Re: [PATCH] poll: Avoid extra wakeups in select/poll
Quoted 4 lines · Eric Dumazet
Quoted 1 line
should be inline perhaps?
Well, I thought current practice was not using inline for such trivial functions, as gcc already inlines them anyway.
ok.
how about:
Quoted 4 lines
Quoted 4 lines
[ I'd still suggest to factor out the guts of
do_select()as
its nesting is excessive that hurts its reviewability quite a bit - but now your patch does not make the situation any worse. ]
We tend to shop for drive-by cleanups in visibly ugly code whenever someone wants to touch that code. Could go into a separate patch.
Ingo
Davide Libenzi
Acked-byRe: [PATCH] poll: Avoid extra wakeups in select/poll
In reply to Eric DumazetQuoted 9 lines · Eric Dumazet
[PATCH] poll: Avoid extra wakeups in select/poll
After introduction of keyed wakeups Davide Libenzi did on epoll, we are able to avoid spurious wakeups in poll()/select() code too.
For example, typical use of poll()/select() is to wait for incoming network frames on many sockets. But TX completion for UDP/TCP frames call
sock_wfree()which in turn schedules thread.
Quoted 8 lines
When scheduled, thread does a full scan of all polled fds and can sleep again, because nothing is really available. If number of fds is large, this cause significant load.
This patch makes select()/poll() aware of keyed wakeups and useless wakeups are avoided. This reduces number of context switches by about 50% on some setups, and work performed by sofirq handlers.
Looks fine to me Eric ...
- Davide
David Miller
maintainerRe: [PATCH] net: Avoid extra wakeups of threads blocked in wait_for_packet()
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Date: Sat, 25 Apr 2009 17:47:23 +0200
Quoted 8 lines
In 2.6.25 we added UDP mem accounting.
This unfortunatly added a penalty when a frame is transmitted, since we have at TX completion time to call
sock_wfree()to perform necessary memory accounting. This callssock_def_write_space()and utimately scheduler if any thread is waiting on the socket.
Thread(s) waiting for an incoming frame was scheduled, then had to sleep again as event was meaningless.
...
Quoted 3 lines
This patch introduces new
DEFINE_WAIT_FUNC()helper and uses it inwait_for_packet(), so that only relevant event can wakeup a thread blocked in this function.
Ok, I was going to give some time towards considering the alternative implementation of using 2 wait queues and what it would look like.
It didn't take long for me to figure out that this is so much simpler that it's not even worth trying the dual wait queue approach.
So I've applied this to net-2.6, thanks!
Now, if we want to fix this up in -stable we'll need to scratch our heads if we can't get the keyed wakeup patch in too. :-/