Performance udp vs. tcp
Subsystems (2)
Hi,
I just ran some bonnie++ tests to measure the nfs performance of udp vs. tcp, expecting tcp to be better (after reading this list and some newsgroups). Amazingly, it turned out that udp with rsize/wsize 16k was the best, taking the average over all tests. tcp was a little better in some of the large-file tests, but especially with creating/stating/deleting 16 thousands of small file, sequentially and randomly, udp was better than tcp in every test (between 5-15 % faster). I rann all the tests also with differenct tcp windows sizes, and a larger window size speeded up udp and tcp more or less in the same way.
I was very amazed by this, but I guess this could be caused by our network, which is a fully switched "100mbit for client/1gbit for servers" network, so we have almost no packet loss.
If I got the discussions on the list and in the newsgroups right, on a lossy network udp would slow down due to the packet loss and resends.
So, can my result be correct for a really fast and stable network? Or is it some "bonnie++ likes udp" effect? :-)
cu,
Frank
Signature · 15 lines
Dipl.-Inform. Frank Steiner Web: http://www.bio.ifi.lmu.de/~steiner/ Lehrstuhl f. Bioinformatik Mail: http://www.bio.ifi.lmu.de/~steiner/m/ LMU, Amalienstr. 17 Phone: +49 89 2180-4049 80333 Muenchen, Germany Fax: +49 89 2180-99-4049 ------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
I forgot to say: This is kernel 2.6.8.1 on server and client.
Signature · 15 lines
Dipl.-Inform. Frank Steiner Web: http://www.bio.ifi.lmu.de/~steiner/ Lehrstuhl f. Bioinformatik Mail: http://www.bio.ifi.lmu.de/~steiner/m/ LMU, Amalienstr. 17 Phone: +49 89 2180-4049 80333 Muenchen, Germany Fax: +49 89 2180-99-4049 * Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. * ------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 14 lines · Frank Steiner
Hi,
I just ran some bonnie++ tests to measure the nfs performance of udp vs. tcp, expecting tcp to be better (after reading this list and some newsgroups).
Amazingly, it turned out that udp with rsize/wsize 16k was the best, taking the average over all tests. tcp was a little better in some of the large-file
tests, but especially with creating/stating/deleting 16 thousands of small file, sequentially and randomly, udp was better than tcp in every test (between 5-15 % faster). I rann all the tests also with differenct tcp windows
sizes, and a larger window size speeded up udp and tcp more or less in the same way.
With a metadata-heavy workload like this the [rw]size makes little difference.
Quoted 9 lines
I was very amazed by this, but I guess this could be caused by our network, which is a fully switched "100mbit for client/1gbit for servers" network, so we have almost no packet loss.
If I got the discussions on the list and in the newsgroups right, on a lossy network udp would slow down due to the packet loss and resends.
So, can my result be correct for a really fast and stable network? Or is it some "bonnie++ likes udp" effect? :-)
There's nothing unusual here. On a low-latency uncongested unlossy network, it's not surprising that UDP is faster. As soon as any of those factors goes away, TCP starts winning.
Greg.
Signature · 12 lines
Greg Banks, R&D Software Engineer, SGI Australian Software Group. I don't speak for SGI. ------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 3 lines · Greg Banks
There's nothing unusual here. On a low-latency uncongested unlossy network, it's not surprising that UDP is faster. As soon as any of those factors goes away, TCP starts winning.
Ok, thanks! I just wasn't sure about that.
Signature · 15 lines
Dipl.-Inform. Frank Steiner Web: http://www.bio.ifi.lmu.de/~steiner/ Lehrstuhl f. Bioinformatik Mail: http://www.bio.ifi.lmu.de/~steiner/m/ LMU, Amalienstr. 17 Phone: +49 89 2180-4049 80333 Muenchen, Germany Fax: +49 89 2180-99-4049 ------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 3 lines · Frank Steiner
I was very amazed by this, but I guess this could be caused by our network, which is a fully switched "100mbit for client/1gbit for servers" network, so we have almost no packet loss.
I don't know why you are so suprised. TCP has somewhat more protocol overhead than UDP. So on a solid network it should be faster.
You can use larger packets with TCP than you can with UDP which can offset the overhead some.
Ian
-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 3 lines · [email protected] wrote:
On Mon, 23 Aug 2004, Frank Steiner wrote:
I don't know why you are so suprised. TCP has somewhat more protocol overhead than UDP. So on a solid network it should be faster.
I guess that was just because I was blended by the "tcp should be faster" quotes, where I didn't read carefully enough to get the full story... My fault :-(
Signature · 15 lines
Dipl.-Inform. Frank Steiner Web: http://www.bio.ifi.lmu.de/~steiner/ Lehrstuhl f. Bioinformatik Mail: http://www.bio.ifi.lmu.de/~steiner/m/ LMU, Amalienstr. 17 Phone: +49 89 2180-4049 80333 Muenchen, Germany Fax: +49 89 2180-99-4049 * Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. * ------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs