2.5.50-BK + 24 CPUs
Quoted 10 lines · Anton
schedules:
56283 total
41984pipe_wait
9746do_work
1949do_exit
1834sys_wait4ie during the compile we scheduled 56283 times, and 41984 of them were caused by pipes.
The linux pipe implementation has only a page sized buffer - with 4 kB pages, transfering 1 MB through a pipe means at 512 context switches.
--
Manfred
Quoted 2 lines · Anton
Quoted 2 lines
ie during the compile we scheduled 56283 times, and 41984 of them were caused by pipes.
Quoted 2 lines · Manfred Spraul
The linux pipe implementation has only a page sized buffer - with 4 kB pages, transfering 1 MB through a pipe means at 512 context switches.
Hmm. What happened to that pipe buffer size increase patch? That sounds like it might help here, but only if those things are trying to shove more than 4KB through the pipe at a time.
Bill
Quoted 3 lines · William Lee Irwin III
Hmm. What happened to that pipe buffer size increase patch? That sounds like it might help here, but only if those things are trying to shove more than 4KB through the pipe at a time.
You probably mean the zero-copy pipe patches, which I think really should go in. The most recent version of the diffs I saw didn't use the zero copy bits unless the trasnfers were quite large so it should be ok and not pessimize small transfers.
That patch has been gathering cobwebs for more than a year now when I first did it, let's push this in already :-)
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines
Hmm. What happened to that pipe buffer size increase patch? That sounds like it might help here, but only if those things are trying to shove more than 4KB through the pipe at a time.
Quoted 6 lines · David S. Miller
You probably mean the zero-copy pipe patches, which I think really should go in. The most recent version of the diffs I saw didn't use the zero copy bits unless the trasnfers were quite large so it should be ok and not pessimize small transfers.
That patch has been gathering cobwebs for more than a year now when I first did it, let's push this in already :-)
I was actually referring to one that explicitly used larger pipe buffers, but this sounds useful too.
Bill
Quoted 9 lines · David S. Miller
Quoted 5 lines · William Lee Irwin III
Hmm. What happened to that pipe buffer size increase patch? That sounds like it might help here, but only if those things are trying to shove more than 4KB through the pipe at a time.
Quoted 9 lines
You probably mean the zero-copy pipe patches, which I think really should go in. The most recent version of the diffs I saw didn't use the zero copy bits unless the trasnfers were quite large so it should be ok and not pessimize small transfers.
That patch has been gathering cobwebs for more than a year now when I first did it, let's push this in already :-)
Unfortunately zero-copy doesn't help to avoid the schedules:
Zero copy just avoid the copy to kernel - you still need one schedule for each page to be transfered.
writer calls
for(;;){
prepare_data(buf);
write(fd,buf,PAGE_SIZE);
}
reader calls
for(;;) {
read(fd,buf,PAGE_SIZE);
use_data(buf);}
What's needed is a large kernel buffer - I've seen buffers between 64 and 256 kB in other unices. zero copy only helps lmbench and other apps where the whole working set fits into the cpu cache.
The difference between
main-mem->cache;cache->main_mem [non-zerocopy]
and
main-mem->main-mem [zerocopy, the copy to kernel is skipped]
is small.
--
Manfred
From: Manfred Spraul <[email protected]>
Date: Mon, 09 Dec 2002 18:03:10 +0100
Unfortunately zero-copy doesn't help to avoid the schedules:
Zero copy just avoid the copy to kernel - you still need one schedule for each page to be transfered.
The zerocopy patches copied up to 64k (or rather, 16 pages, something like that) at once, that's going to lead to 16 times less schedules.
The 64k number was decided arbitrarily (it's what freebsd's pipe code uses) and it can be experimented with.
Quoted 7 lines · David S. Miller
From: Manfred Spraul <[email protected]>
Date: Mon, 09 Dec 2002 18:03:10 +0100Unfortunately zero-copy doesn't help to avoid the schedules:
Zero copy just avoid the copy to kernel - you still need one schedule for each page to be transfered.
Quoted 7 lines
The zerocopy patches copied up to 64k (or rather, 16 pages, something like that) at once, that's going to lead to 16 times less schedules.
The 64k number was decided arbitrarily (it's what freebsd's pipe code uses) and it can be experimented with.
Only if user space writes in 64 kB chunks - if user space writes 4 kB chunks, then zerocopy doesn't help much against schedule [depending on the implementation, it halves the number of schedules]. And page table tricks (COW) tricks are not acceptable.
--
Manfred