So, Poll is not scalable... what to do?
If poll is not scalable, which method should I use when writing multithreaded socket server?
What is the most efficient model to use?
Is there a "standard" model to use when writing a scalable multithreaded socket serve such as "io completion ports" on windows?
Quoted 1 line
From the "Microbenchmark comparing poll, kqueue, and
/dev/poll", kqueue is
the way to go. Am I correct?
What is the best solution to use on Linux?
Also, why is it that poll doesn not return with "close signal" when a thread-1 calls poll and thread-2 calls close on a sockfd1? It seems that poll only handles close signal when a client disconnects from the server. I've seen this mentioned here before, has it been fixed?
Thank you~~~
_________________________________________________________________
Quoted 1 line
From Beethoven to the Rolling Stones, your favorite music is always playing
on MSN Radio Plus. No ads, no talk. Trial month FREE! http://join.msn.com/?page=offers/premiumradio
Quoted 3 lines · kirk bae
If poll is not scalable, which method should I use when writing multithreaded socket server?
What is the most efficient model to use?
epoll, thread pools, AIO...
Quoted 2 lines · kirk bae
If poll is not scalable, which method should I use when writing multithreaded socket server?
Honnestly, if you're using threads (I mean lots of threads, such as one per connection), I don't think that poll performance will be your worst ennemy. The first thing to do is to handle the task switching yourself either with a publicly available coroutine library or with one of your own.
Take a look here for more a comparison of several available methods :
http://www.kegel.com/c10k.html
epoll is compared to other methods with numbers here :
http://www.xmailserver.org/linux-patches/nio-improve.html
Cheers,
Willy
In article <[email protected]>, Willy Tarreau <[email protected]> wrote:
| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote: | > If poll is not scalable, which method should I use when writing | > multithreaded socket server? | | Honnestly, if you're using threads (I mean lots of threads, such as one | per connection), I don't think that poll performance will be your worst | ennemy. The first thing to do is to handle the task switching yourself | either with a publicly available coroutine library or with one of your own.
It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.
| | Take a look here for more a comparison of several available methods : | | http://www.kegel.com/c10k.html | | epoll is compared to other methods with numbers here : | | http://www.xmailserver.org/linux-patches/nio-improve.html | | Cheers, | Willy | | - | To unsubscribe from this list: send the line "unsubscribe linux-kernel" in | the body of a message to [email protected] | More majordomo info at http://vger.kernel.org/majordomo-info.html | Please read the FAQ at http://www.tux.org/lkml/ |
Signature · 3 lines
bill davidsen <[email protected]> CTO, TMR Associates, Inc Doing interesting things with little computers since 1979.
Quoted 11 lines · bill davidsen
In article <[email protected]>, Willy Tarreau <[email protected]> wrote:
| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote: | > If poll is not scalable, which method should I use when writing | > multithreaded socket server? | | Honnestly, if you're using threads (I mean lots of threads, such as one | per connection), I don't think that poll performance will be your worst | ennemy. The first thing to do is to handle the task switching yourself | either with a publicly available coroutine library or with one of your own.
Quoted 4 lines
It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.
Pros & Cons:
*) Coroutines cost is basically its stack (8-16Kb). Threads there's a little bit more under the hood
*) No locks at all with coroutines
*) Coroutine context switch time was about 20 times faster last time I tried. I used this:
http://www.xmailserver.org/libpcl.html
against O(1)
*) Coroutines require a more careful coding then threads, expecially stackwise
*) Achieving SMP scalability/balancing with coroutines is not trivial while with threads it comes along (well, almost)
Coroutines are not the only alternative though. I/O driven state machines can make you save some stack space at expenses of less trivial coding.
- Davide
Quoted 18 lines · Davide Libenzi
Quoted 9 lines · bill davidsen
In article <[email protected]>, Willy Tarreau <[email protected]> wrote:
| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote: | > If poll is not scalable, which method should I use when writing | > multithreaded socket server? | | Honnestly, if you're using threads (I mean lots of threads, such as one | per connection), I don't think that poll performance will be your worst | ennemy. The first thing to do is to handle the task switching yourself | either with a publicly available coroutine library or with one of your own.Quoted 5 lines
It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.
Quoted 8 lines
Pros & Cons:
*) Coroutines cost is basically its stack (8-16Kb). Threads there's a little bit more under the hood
*) No locks at all with coroutines
*) Coroutine context switch time was about 20 times faster last time I tried. I used this:
According to lmbench, 2.4 does a context switch in 0.67us (on one type of uniprocessor cpu - the machines at osdl). 2.6.0-test9 manages 1.17us (174% longer). I have some patches that brings this to 0.80us (119%). This is with 2 active tasks, so O(1) doesn't get to show its advantage.
If you remove the realtime priority array (disable RT scheduling support) you could probably drop this figure below 2.4's. I should get some numbers
Perhaps if there is demand I could make RT scheduling a config option in my patchset. I think Andrea does something fancy like that. I'll have to take a look at his code.
That said, I'd be inclined to think an application that it context switch bound is broken by design. Although maybe there are some special cases.
2.4: http://khack.osdl.org/stp/282982/results/summary_report
2.6: http://khack.osdl.org/stp/282354/results/summary_report
np: http://khack.osdl.org/stp/283054/results/summary_report
Quoted 31 lines · Nick Piggin
Quoted 27 lines · Davide Libenzi
Quoted 11 lines · bill davidsen
In article <[email protected]>, Willy Tarreau <[email protected]> wrote:
| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote: | > If poll is not scalable, which method should I use when writing | > multithreaded socket server? | | Honnestly, if you're using threads (I mean lots of threads, such as one | per connection), I don't think that poll performance will be your worst | ennemy. The first thing to do is to handle the task switching yourself | either with a publicly available coroutine library or with one of your own.Quoted 5 lines
It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.
Pros & Cons:
*) Coroutines cost is basically its stack (8-16Kb). Threads there's a little bit more under the hood
*) No locks at all with coroutines
Quoted 8 lines
Quoted 3 lines
*) Coroutine context switch time was about 20 times faster last time I tried. I used this:
According to lmbench, 2.4 does a context switch in 0.67us (on one type of uniprocessor cpu - the machines at osdl). 2.6.0-test9 manages 1.17us (174% longer). I have some patches that brings this to 0.80us (119%). This is with 2 active tasks, so O(1) doesn't get to show its advantage.
It would seem that in the case of many sockets that there is still a need to determine when to run a coroutine. Using pthreads and NPTL lets the kernel do that by unblocking the thread, handles SMP cases, etc. And is source code portable.
On the other hand, I'm sure that with enough effort some solution-specific code could do better, assuming that there's nothing else in the machine. And depending on the base hardware and the computation involved, there could be a big win with SMP of the HT flavor.
I think any problem like this is an "it depends" case, but the assumption that polling doesn't scale is a good starting point.
Quoted 18 lines
If you remove the realtime priority array (disable RT scheduling support) you could probably drop this figure below 2.4's. I should get some numbers
Perhaps if there is demand I could make RT scheduling a config option in my patchset. I think Andrea does something fancy like that. I'll have to take a look at his code.
That said, I'd be inclined to think an application that it context switch bound is broken by design. Although maybe there are some special cases.
2.4: http://khack.osdl.org/stp/282982/results/summary_report
2.6: http://khack.osdl.org/stp/282354/results/summary_report
np: http://khack.osdl.org/stp/283054/results/summary_report
Signature · 3 lines
bill davidsen <[email protected]> CTO, TMR Associates, Inc Doing interesting things with little computers since 1979.
In article <[email protected]>, kirk bae <[email protected]> wrote:
| If poll is not scalable, which method should I use when writing | multithreaded socket server? | | What is the most efficient model to use? | | Is there a "standard" model to use when writing a scalable multithreaded | socket serve such as "io completion ports" on windows? |
In many cases people just run a thread per socket and use blocking i/o. Then the thread either does the work required or make an entry on a "work to do" queue.
You've had other suggestions, this is just for completeness.
Signature · 3 lines
bill davidsen <[email protected]> CTO, TMR Associates, Inc Doing interesting things with little computers since 1979.
Quoted 5 lines · In article <[email protected]> you wrote:
In many cases people just run a thread per socket and use blocking i/o. Then the thread either does the work required or make an entry on a "work to do" queue.
You've had other suggestions, this is just for completeness.
This is true for Windows and Java, but how many popular Linux Applications do it that way? There are much more non-blocking (erlang, boa) or process based (apache, ...)
Greetings
Bernd
Signature · 2 lines
eckes privat - http://www.eckes.org/ Project Freefire - http://www.freefire.org/
Quoted 3 lines
If poll is not scalable, which method should I use when writing multithreaded socket server?
'poll' is quite scalable. It's only not scalable if you make stupid
assumptions about how people use it.
Quoted 4 lines
What is the most efficient model to use?
Is there a "standard" model to use when writing a scalable multithreaded socket serve such as "io completion ports" on windows?
If all your sockets are equally active, having a bunch of threads each
'poll'ing 1,024 descriptors is perfectly scalable. 64,000 is not a problem, and that's about the kernel limit if memory serves me.
If you have active and inactive sockets, sort them. Have some threads that
'poll' on large numbers of inactive sockets and some that 'poll' on small numbers of active sockets. This will scale quite well as the expensive 'poll' calls are infrequent compared to the cheap ones.
Quoted 3 lines
From the "Microbenchmark comparing poll, kqueue, and
/dev/poll", kqueue is
the way to go. Am I correct?
Yes, if you're using a kernel that supports it. It will pretty much always
be more efficient than 'poll'.
Quoted 4 lines
Also, why is it that poll doesn not return with "close signal" when a thread-1 calls poll and thread-2 calls close on a sockfd1? It seems that poll only handles close signal when a client disconnects from the server. I've seen this mentioned here before, has it been fixed?
You can never ever make this work. There will always be a race condition no
matter what you do. (How can you ever be sure thread-2 doesn't all 'close' jsut before thread-1 calls 'poll'?) So do not ever do this. Do not ever release a shared resource while a thread is or might be using it because it might also be about to use it!
DS