Skip to contentSkip to conversation

So, Poll is not scalable... what to do?

DISCUSSION kirk bae 8 participants 10 messages Nov 11, 2003 – Nov 13, 2003

lore
Share & tools
lore

kirk bae

author
#0 lore raw

If poll is not scalable, which method should I use when writing multithreaded socket server?

What is the most efficient model to use?

Is there a "standard" model to use when writing a scalable multithreaded socket serve such as "io completion ports" on windows?

Quoted 1 line

From the "Microbenchmark comparing poll, kqueue, and /dev/poll", kqueue is

the way to go. Am I correct?

What is the best solution to use on Linux?

Also, why is it that poll doesn not return with "close signal" when a thread-1 calls poll and thread-2 calls close on a sockfd1? It seems that poll only handles close signal when a client disconnects from the server. I've seen this mentioned here before, has it been fixed?

Thank you~~~

_________________________________________________________________

Quoted 1 line

From Beethoven to the Rolling Stones, your favorite music is always playing

on MSN Radio Plus. No ads, no talk. Trial month FREE! http://join.msn.com/?page=offers/premiumradio

#1 lore raw
Quoted 3 lines · kirk bae

If poll is not scalable, which method should I use when writing multithreaded socket server?

What is the most efficient model to use?

epoll, thread pools, AIO...

#2 lore raw
In reply to kirk bae
Quoted 2 lines · kirk bae

If poll is not scalable, which method should I use when writing multithreaded socket server?

Honnestly, if you're using threads (I mean lots of threads, such as one per connection), I don't think that poll performance will be your worst ennemy. The first thing to do is to handle the task switching yourself either with a publicly available coroutine library or with one of your own.

Take a look here for more a comparison of several available methods :

    http://www.kegel.com/c10k.html

epoll is compared to other methods with numbers here :

http://www.xmailserver.org/linux-patches/nio-improve.html

Cheers,
Willy

#3 lore raw

In article <[email protected]>, Willy Tarreau <[email protected]> wrote:

| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote:
| > If poll is not scalable, which method should I use when writing 
| > multithreaded socket server?
| 
| Honnestly, if you're using threads (I mean lots of threads, such as one
| per connection), I don't think that poll performance will be your worst
| ennemy. The first thing to do is to handle the task switching yourself
| either with a publicly available coroutine library or with one of your own.

It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.

| 
| Take a look here for more a comparison of several available methods :
| 
|     http://www.kegel.com/c10k.html
| 
| epoll is compared to other methods with numbers here :
| 
|     http://www.xmailserver.org/linux-patches/nio-improve.html
| 
| Cheers,
| Willy
| 
| -
| To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
| the body of a message to [email protected]
| More majordomo info at  http://vger.kernel.org/majordomo-info.html
| Please read the FAQ at  http://www.tux.org/lkml/
| 
Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#4 lore raw
Quoted 11 lines · bill davidsen

In article <[email protected]>, Willy Tarreau <[email protected]> wrote:

| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote:
| > If poll is not scalable, which method should I use when writing 
| > multithreaded socket server?
| 
| Honnestly, if you're using threads (I mean lots of threads, such as one
| per connection), I don't think that poll performance will be your worst
| ennemy. The first thing to do is to handle the task switching yourself
| either with a publicly available coroutine library or with one of your own.
Quoted 4 lines

It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.

Pros & Cons:

*) Coroutines cost is basically its stack (8-16Kb). Threads there's a little bit more under the hood

*) No locks at all with coroutines

*) Coroutine context switch time was about 20 times faster last time I tried. I used this:

http://www.xmailserver.org/libpcl.html

against O(1)

*) Coroutines require a more careful coding then threads, expecially stackwise

*) Achieving SMP scalability/balancing with coroutines is not trivial while with threads it comes along (well, almost)

Coroutines are not the only alternative though. I/O driven state machines can make you save some stack space at expenses of less trivial coding.

- Davide

#5 lore raw
Quoted 18 lines · Davide Libenzi
Quoted 9 lines · bill davidsen

In article <[email protected]>, Willy Tarreau <[email protected]> wrote:

| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote:
| > If poll is not scalable, which method should I use when writing | > multithreaded socket server?
| | Honnestly, if you're using threads (I mean lots of threads, such as one
| per connection), I don't think that poll performance will be your worst
| ennemy. The first thing to do is to handle the task switching yourself
| either with a publicly available coroutine library or with one of your own.
Quoted 5 lines

It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.

Quoted 8 lines

Pros & Cons:

*) Coroutines cost is basically its stack (8-16Kb). Threads there's a little bit more under the hood

*) No locks at all with coroutines

*) Coroutine context switch time was about 20 times faster last time I tried. I used this:

According to lmbench, 2.4 does a context switch in 0.67us (on one type of uniprocessor cpu - the machines at osdl). 2.6.0-test9 manages 1.17us (174% longer). I have some patches that brings this to 0.80us (119%). This is with 2 active tasks, so O(1) doesn't get to show its advantage.

If you remove the realtime priority array (disable RT scheduling support) you could probably drop this figure below 2.4's. I should get some numbers

Perhaps if there is demand I could make RT scheduling a config option in my patchset. I think Andrea does something fancy like that. I'll have to take a look at his code.

That said, I'd be inclined to think an application that it context switch bound is broken by design. Although maybe there are some special cases.

2.4: http://khack.osdl.org/stp/282982/results/summary_report
2.6: http://khack.osdl.org/stp/282354/results/summary_report
np: http://khack.osdl.org/stp/283054/results/summary_report

#6 lore raw
Quoted 31 lines · Nick Piggin
Quoted 27 lines · Davide Libenzi
Quoted 11 lines · bill davidsen

In article <[email protected]>, Willy Tarreau <[email protected]> wrote:

| On Tue, Nov 11, 2003 at 05:52:42PM -0600, kirk bae wrote:
| > If poll is not scalable, which method should I use when writing 
| > multithreaded socket server?
| 
| Honnestly, if you're using threads (I mean lots of threads, such as one
| per connection), I don't think that poll performance will be your worst
| ennemy. The first thing to do is to handle the task switching yourself
| either with a publicly available coroutine library or with one of your own.
Quoted 5 lines

It's not clear that with 2.6 this is necessary or desirable. I'll let someone who worked on the new thread and/or futex development say more if they will, but I'm reasonable convinced that in most cases the kernel will do it better.

Pros & Cons:

*) Coroutines cost is basically its stack (8-16Kb). Threads there's a little bit more under the hood

*) No locks at all with coroutines

Quoted 8 lines
Quoted 3 lines

*) Coroutine context switch time was about 20 times faster last time I tried. I used this:

According to lmbench, 2.4 does a context switch in 0.67us (on one type of uniprocessor cpu - the machines at osdl). 2.6.0-test9 manages 1.17us (174% longer). I have some patches that brings this to 0.80us (119%). This is with 2 active tasks, so O(1) doesn't get to show its advantage.

It would seem that in the case of many sockets that there is still a need to determine when to run a coroutine. Using pthreads and NPTL lets the kernel do that by unblocking the thread, handles SMP cases, etc. And is source code portable.

On the other hand, I'm sure that with enough effort some solution-specific code could do better, assuming that there's nothing else in the machine. And depending on the base hardware and the computation involved, there could be a big win with SMP of the HT flavor.

I think any problem like this is an "it depends" case, but the assumption that polling doesn't scale is a good starting point.

Quoted 18 lines

If you remove the realtime priority array (disable RT scheduling support) you could probably drop this figure below 2.4's. I should get some numbers

Perhaps if there is demand I could make RT scheduling a config option in my patchset. I think Andrea does something fancy like that. I'll have to take a look at his code.

That said, I'd be inclined to think an application that it context switch bound is broken by design. Although maybe there are some special cases.

2.4: http://khack.osdl.org/stp/282982/results/summary_report
2.6: http://khack.osdl.org/stp/282354/results/summary_report
np: http://khack.osdl.org/stp/283054/results/summary_report

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#7 lore raw
In reply to kirk bae

In article <[email protected]>, kirk bae <[email protected]> wrote:

| If poll is not scalable, which method should I use when writing 
| multithreaded socket server?
| 
| What is the most efficient model to use?
| 
| Is there a "standard" model to use when writing a scalable multithreaded 
| socket serve such as "io completion ports" on windows?
| 

In many cases people just run a thread per socket and use blocking i/o. Then the thread either does the work required or make an entry on a "work to do" queue.

You've had other suggestions, this is just for completeness.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#8 lore raw
Quoted 5 lines · In article <[email protected]> you wrote:

In many cases people just run a thread per socket and use blocking i/o. Then the thread either does the work required or make an entry on a "work to do" queue.

You've had other suggestions, this is just for completeness.

This is true for Windows and Java, but how many popular Linux Applications do it that way? There are much more non-blocking (erlang, boa) or process based (apache, ...)

Greetings
Bernd

Signature · 2 lines
eckes privat - http://www.eckes.org/
Project Freefire - http://www.freefire.org/
#9 lore raw
In reply to kirk bae
Quoted 3 lines

If poll is not scalable, which method should I use when writing multithreaded socket server?

'poll' is quite scalable. It's only not scalable if you make stupid
assumptions about how people use it.

Quoted 4 lines

What is the most efficient model to use?

Is there a "standard" model to use when writing a scalable multithreaded socket serve such as "io completion ports" on windows?

If all your sockets are equally active, having a bunch of threads each
'poll'ing 1,024 descriptors is perfectly scalable. 64,000 is not a problem, and that's about the kernel limit if memory serves me.

If you have active and inactive sockets, sort them. Have some threads that
'poll' on large numbers of inactive sockets and some that 'poll' on small numbers of active sockets. This will scale quite well as the expensive 'poll' calls are infrequent compared to the cheap ones.

Quoted 3 lines

From the "Microbenchmark comparing poll, kqueue, and /dev/poll", kqueue is
the way to go. Am I correct?

Yes, if you're using a kernel that supports it. It will pretty much always
be more efficient than 'poll'.

Quoted 4 lines

Also, why is it that poll doesn not return with "close signal" when a thread-1 calls poll and thread-2 calls close on a sockfd1? It seems that poll only handles close signal when a client disconnects from the server. I've seen this mentioned here before, has it been fixed?

You can never ever make this work. There will always be a race condition no
matter what you do. (How can you ever be sure thread-2 doesn't all 'close' jsut before thread-1 calls 'poll'?) So do not ever do this. Do not ever release a shared resource while a thread is or might be using it because it might also be about to use it!

DS