tcp bandwidth on athlon
Quoted 7 lines
I ran oprofile with
bw_tcpand retired instructions on athlon showed:samples %-age symbol name
903640 75.4825csum_partial_copy_generic
Quoted 7 lines
In Carl Staelin and Larry McVoy's 98 Usenix paper they wrote:
It is interesting to compare pipes with TCP because the TCP benchmark is identical to the pipe benchmark except for the transport mechanism. Ideally, the TCP bandwidth would be as good as the pipe bandwidth. It is not widely known that the
Well, TCP will have a little more overhead than a pipe - the network stack having to take care of a few more things.
Quoted 7 lines
majority of the TCP cost is in the bcopy, the checksum, and the network interface driver. The checksum and the driver may be safely eliminated in the loopback case and if the costs have been eliminated, then TCP should be just as fast as pipes. From the pipe and TCP results [...]
it is easy to see that Solaris and HP-UX have done this optimization.
Much has happened since 1998: hw checksum offload, sendfile, amongst others. Note that this checksum is performed while copying the data from/to user space (though not that often in rx code path). However, while we dont look at the checksum on the rx side for TCP, since the loopback driver will have set ip_summed to CHECKSUM_UNNECESSARY, on the send side we dont bother, because we have to do the copy in any case. Marginal difference, although perhaps worth doing. I had a patch for this which showed no difference in a VolanoMark (yes, I know), benchmark.
Quoted 4 lines
Processor Pipe TCP
Athlon/1330 840.66 73.75 (or 150 MB/sec - see below)
k6-2/475 65.15 52.45
PIII * 1/700 Xeon 539.73 446.16
Hmm, so if K6 and Xeon can scrounge up 80% of pipe performance, why is the Athlon an order of magnitude off at 8%? How did your Athlon perform in other tests relative to these other procs?
Quoted 7 lines
I tried compiling the athlon kernel without
X86_USE_PPRO_CHECKSUMbut that didn't really change tcp bandwidth.kernel Pipe TCP
2.4.19rc2aa1 860.97 74.27 2.4.19rc2aa1-nocsum 853.18 74.16
Well, that would simply change how the checksum is calculated, and in this case, I believe the substantial latency is from the copy.
Review
Quoted 6 lines
There was a change in
bw_tcp.cthat has a 2x impact on the computed bandwidth. I have two versions:ls -gl
LM*/src/bw_tcp.c(quoted diff) +0
-2-r--r--r-- 1 rwhron 3553 Jul 23 2001 LMbench.old/src/bw_tcp.c -r--r--r-- 1 rwhron 3799 Sep 27 2001 LMbench2/src/bw_tcp.c
I only see the bitkeeper version thats almost a year old, online, where is the later version from?
Quoted 1 line
Both LMbench trees have the same version:
...
Quoted 2 lines
ident doesn't specify a version in
tcp_bw.c, but diff shows a difference.
A change in your test causes a 2x difference in performance, and you dont give us the diff? :) :)
Quoted 1 line
Socket bandwidth using localhost: 75.85 MB/sec
...
Quoted 1 line
Socket bandwidth using localhost: 150.21 MB/sec
Where are the complete profiles from these runs? Also, any chance you have network stats before/after? I looked on your site but couldnt find the tcp_bw runs..
thanks,
Nivedita