Results
Hi all,
yesterday I played a bit with the chatroom benchmark.
I don't want to discuss about number (you are the right people to do that, not me).
--- What is the chat bench ---
Usage: chat_c ip_addr [num_rooms] [num_messages] [server_port]
ip_addr: server ip address that the client connects to.
[num_rooms]: the number of chat rooms.
default is 10.
[num_messages]: the number of messages sent by each chat room member.
default is 100.
[port]: server port number the client connects to.
default is 9999.
This benchmark includes both a client and server. The benchmark is modeled after a chat room. This benchmark will create a lot of threads, tcp connections, and send and receive a lot of messages. The client side of the benchmark will report the number of messages sent per second.
The number of chat rooms and messages will control the workload. The default number of chat rooms is 10.
The default number of messages is 100. The size of each message is 100 bytes. Both of these parameters are specified on the client side.
(1) Each chat room has 20 users.
(2) 10 chat rooms represents 20*10 or 200 users.
(3) For each user in the chat room, the client will make a connection to the server.
(4) 10 chat rooms will create 10*20 or 200 connections between the client and server.
(5) For each user (or connection) in the chat room, 1 'send' thread is created and 1 'receive' thread is created.
(6) 10 chat rooms will create 10*20*2 or 400 client threads and 400 server threads for a total of 800 threads.
(7) Each client 'send' thread will send the specified number of messages to the server.
(8) For 10 chat rooms and 100 messages, the client will send 10*20*100 or 20,000 messages. The server 'receive' thread will receive the corresponding number of messages.
(9) The chat room server will echo each of the messages back to the other users in the chat room.
(10) For 10 chat rooms and 100 messages, the server 'send' thread will send 10*20*100*19 or 380,000 messages. The client 'receive' thread will receive the corresponding number of messages.
(11) The client will report the message throughput at the end of the test.
(12) The server loops and accepts another start message from the client.
What I did:
Boot in single mode with apm off
./chat_s 127.0.0.1 &
./chat_c 127.0.0.1 30 1000 9999
And now the results:
2.4.19-ck7.results:Average throughput : 55928 messages per second 2.4.19.results:Average throughput : 44851 messages per second 2.5.33.results:Average throughput : 59522 messages per second 2.5.34.results:Average throughput : 62941 messages per second 2.5.36.results:Average throughput : 60858 messages per second
2.4.19-ck7 is preemption ON
2.5.33 and 2.5.34 are preemption ON
2.5.36 is preemption OFF (Robert, 2.5.36 with preemption ON oops at boot)
Ciao,
Paolo
Signature · 4 lines
Get your free email from www.linuxmail.org Powered by Outblaze
Just thought I'd bounce this one over to Andrew as a warm-n-fuzzy. There's also some kind of hint that preemption improves throughput here.
Quoted 15 lines · Paolo Ciarrocchi
What I did:
Boot in single mode with apm off
./chat_s 127.0.0.1 &
./chat_c 127.0.0.1 30 1000 9999And now the results:
2.4.19-ck7.results:Average throughput : 55928 messages per second 2.4.19.results:Average throughput : 44851 messages per second 2.5.33.results:Average throughput : 59522 messages per second 2.5.34.results:Average throughput : 62941 messages per second 2.5.36.results:Average throughput : 60858 messages per second2.4.19-ck7 is preemption ON
2.5.33 and 2.5.34 are preemption ON
2.5.36 is preemption OFF (Robert, 2.5.36 with preemption ON oops at boot)
Signature · 1 line
Daniel
Quoted 2 lines · Daniel Phillips
Just thought I'd bounce this one over to Andrew as a warm-n-fuzzy.
Is a scheduler test, isn't it?
Quoted 2 lines
There's also some kind of hint that preemption improves throughput here.
One needs to treat any test which involves networking to localhost with caution. They tend to show large (+/- 10% or more) swings in throughput from one run to the next. Some sort of cache associativity thing; not sure.
Running the test between separate machines is much, much more repeatable.
We'll get the cpu-local hot pages list code going soon; that may provide some benefit to this sort of thing. Even on uniprocessor.
Quoted 5 lines · Andrew Morton
Quoted 2 lines · Daniel Phillips
Just thought I'd bounce this one over to Andrew as a warm-n-fuzzy.
Is a scheduler test, isn't it?
Looks like it's inspired by volano, and yes, mainly scheduling, but I suppose the messages put a little load on the VM as well.
Signature · 1 line
Daniel
Quoted 7 lines
Quoted 7 lines
And now the results:
2.4.19-ck7.results:Average throughput : 55928 messages per second 2.4.19.results:Average throughput : 44851 messages per second 2.5.33.results:Average throughput : 59522 messages per second 2.5.34.results:Average throughput : 62941 messages per second 2.5.36.results:Average throughput : 60858 messages per second
Quoted 6 lines
Quoted 3 lines
2.4.19-ck7 is preemption ON
2.5.33 and 2.5.34 are preemption ON
2.5.36 is preemption OFF (Robert, 2.5.36 with preemption ON oops at boot)There's also some kind of hint that preemption improves throughput here.
Any chance of doing two runs on exactly the same kernel, one with preempt on, and the other with preempt off? That's a much nicer hint ;-)
M.
Quoted 5 lines · On Thu, 19 Sep 2002 01:29:30 -0700 Andrew Morton <[email protected]> wrote:
Quoted 2 lines · Daniel Phillips
Just thought I'd bounce this one over to Andrew as a warm-n-fuzzy.
Is a scheduler test, isn't it?
Mixed. Before the O(1) scheduler, I distilled this test into "hackbench" (which I now call schedbench). Do well on this, and you'll do well on the lots-of-threads-wakingup case: [BTW, 2.5 rocks at this 8)]. Ideally "hackbench n" should scale with n:
/* Test groups of 20 processes spraying to 20 receivers */ #include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <sys/time.h>
#include <sys/select.h>
#define DATASIZE 100
static unsigned int loops = 100;
static int use_pipes = 0;
static void barf(const char *msg)
{
fprintf(stderr, "%s (error: %s)\n", msg, strerror(errno));
exit(1);
}static void fdpair(int fds[2])
{
if (use_pipes) {
if (pipe(fds) == 0)
return;
} else {
if (socketpair(AF_UNIX, SOCK_STREAM, 0, fds) == 0)
return;
}
barf("Creating fdpair");
}/* Block until we're ready to go */
static void ready(int ready_out, int wakefd)
{
char dummy;
fd_set fds;
FD_ZERO(&fds);
FD_SET(wakefd, &fds);
/* Tell them we're ready. */
if (write(ready_out, &dummy, 1) != 1)
barf("CLIENT: ready write");
/* Wait for "GO" signal */
if (select(wakefd+1, &fds, NULL, NULL, NULL) != 1)
barf("select");
}/* Sender sprays loops messages down each file descriptor */ static void sender(unsigned int num_fds,
int out_fd[num_fds],
int ready_out,
int wakefd)
{
char data[DATASIZE];
unsigned int i, j;
ready(ready_out, wakefd);
/* Now pump to every receiver. */
for (i = 0; i < loops; i++) {
for (j = 0; j < num_fds; j++) {
int ret, done = 0;
again:
ret = write(out_fd[j], data + done, sizeof(data)-done);
if (ret < 0)
barf("SENDER: write");
done += ret;
if (done < sizeof(data))
goto again;
}
}
}/* One receiver per fd */
static void receiver(unsigned int num_packets,
int in_fd,
int ready_out,
int wakefd)
{
unsigned int i;
/* Wait for start... */
ready(ready_out, wakefd);
/* Receive them all */
for (i = 0; i < num_packets; i++) {
char data[DATASIZE];
int ret, done = 0;
again:
ret = read(in_fd, data + done, DATASIZE - done);
if (ret < 0)
barf("SERVER: read");
done += ret;
if (done < DATASIZE)
goto again;
}
}/* One group of senders and receivers */
static unsigned int group(unsigned int num_fds,
int ready_out,
int wakefd)
{
unsigned int i;
unsigned int out_fds[num_fds];
for (i = 0; i < num_fds; i++) {
int fds[2];
/* Create the pipe between client and server */
fdpair(fds);
/* Fork the receiver. */
switch (fork()) {
case -1: barf("fork()");
case 0:
close(fds[1]);
receiver(num_fds*loops, fds[0], ready_out, wakefd);
exit(0);
}
out_fds[i] = fds[1];
close(fds[0]);
}
/* Now we have all the fds, fork the senders */
for (i = 0; i < num_fds; i++) {
switch (fork()) {
case -1: barf("fork()");
case 0:
sender(num_fds, out_fds, ready_out, wakefd);
exit(0);
}
}
/* Close the fds we have left */
for (i = 0; i < num_fds; i++)
close(out_fds[i]);
/* Return number of children to reap */
return num_fds * 2;
}int main(int argc, char *argv[])
{
unsigned int i, num_groups, total_children;
struct timeval start, stop, diff;
unsigned int num_fds = 20;
int readyfds[2], wakefds[2];
char dummy;
if (argv[1] && strcmp(argv[1], "-pipe") == 0) {
use_pipes = 1;
argc--;
argv++;
}
if (argc != 2 || (num_groups = atoi(argv[1])) == 0)
barf("Usage: hackbench [-pipe] <num groups>\n");
fdpair(readyfds);
fdpair(wakefds);
total_children = 0;
for (i = 0; i < num_groups; i++)
total_children += group(num_fds, readyfds[1], wakefds[0]);
/* Wait for everyone to be ready */
for (i = 0; i < total_children; i++)
if (read(readyfds[0], &dummy, 1) != 1)
barf("Reading for readyfds");
gettimeofday(&start, NULL);
/* Kick them off */
if (write(wakefds[1], &dummy, 1) != 1)
barf("Writing to start them");
/* Reap them all */
for (i = 0; i < total_children; i++) {
int status;
wait(&status);
if (!WIFEXITED(status))
exit(1);
}
gettimeofday(&stop, NULL);
/* Print time... */
timersub(&stop, &start, &diff);
printf("Time: %lu.%03lu\n", diff.tv_sec, diff.tv_usec/1000);
exit(0);
}Signature · 2 lines
there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy