Poll and Select not scaling
Quoted 7 lines · Micah
I have been trying to increase the scalabilty of an email server that has been ported to Linux. It was originally written for Netware, and there we are able to provide over 30,000 connections at any given time. On Linux however select stops working after the first 1024 connections. I have changed
include/linux/fs.hand updatedNR_FILEto be 81920.
On modern kernels, you no longer need to change NR_FILE; all you need to do is increase ulimit -n and a few proc entries; see
http://www.kegel.com/c10k.html#limits.filehandles
Quoted 3 lines
In test applications I have been able to create well over 30,000 connections but I am unable to do either a select or a poll on them.
select() is usually limited to 1024 file descriptors, so poll() is a slightly better choice. However, although it can handle 30000 file descriptors, the performance sucks; see http://www.kegel.com/dkftpbench/Poller_bench.html#results
You may wish to use an alternative to poll().
See http://www.kegel.com/c10k.html#strategies for a list of possibilities.
The only one that performs well on Linux at 30000 fds would be the real time signal stuff. That's a little tricky to use. Or you could recode your app to use one thread or process per 1000 fd's.
Solaris and FreeBSD both offer friendlier poll alternatives that perform quite well. You might consider using one of those operating systems if all else fails.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 1 line · Dan Kegel
select()is usually limited to 1024 file descriptors
oh hey, this limit is only a libc limit these days. you can do this:
#define MY_FD_SETSIZE (16384)
typedef struct {
__fd_mask __fds_bits[MY_FD_SETSIZE / __NFDBITS];
} my_fd_set;#define MY_FD_ZERO(_f) (memset((_f), 0, sizeof(my_fd_set)))
and do select()s of 16384 descriptors.
Quoted 3 lines
poll()is a slightly better choice. However, although it can handle 30000 file descriptors, the performance sucks; see http://www.kegel.com/dkftpbench/Poller_bench.html#results
poll() stops working at 16384 file descriptors (as of 2.2.14-foo original redhat 6.2 kernel). at least that's where i think it is, maybe it's 32768. it's limited by the maximum kmalloc() size of 128k. this is somewhat unfortunate. even though we know it'll stop performing well.
-dean
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/