From: Andy Furniss Date: Tue, 03 Apr 2007 19:27:38 +0000 Subject: Re: [LARTC] Xen and tc problems Message-Id: <4612AAAA.4070608@andyfurniss.entadsl.com> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: lartc@vger.kernel.org Padala, Pradeep wrote: > Hi, > > I am trying to shape traffic to two VMs hosted in Xen. There seems to be > very little information regarding this. I found this web page > http://www.ioncannon.net/system-administration/57/limiting-bandwidth-usa > ge-on-xen-linux-setup/ and followed the instructions. But, the real > bandwidth experienced from clients always seems to exceed the set rate. > Part of the problem may be because of the way Xen bridging is setup. > There are many interfaces that the packets go through. So, I switched to > the Xen routed networking, in which dom0 simply sees two virtual > interfaces for the VM, which are kind of PPP connections to the eth0 > interfaces in VM. > > eth0 +---- vif1.0 -- eth0 in VM1 > | > | > +---- vif2.0 -- eth1 in VM2 > > Say, I want to limit the bandwidth to VM1 to 100mbit and VM2 to 500mbit > (eth0 is a 1gbit interface), I used to following commands. > > iptables -t mangle -F POSTROUTING > tc qdisc add dev eth0 root handle 1: htb r2q 1000 > iptables -t mangle -A POSTROUTING -s $vm1_ip -j CLASSIFY --set-class 1:1 > iptables -t mangle -A POSTROUTING -d $vm1_ip -j CLASSIFY --set-class 1:1 > tc class add dev eth0 parent 1: classid 1:1 htb rate 512mbit > iptables -t mangle -A POSTROUTING -s $vm2_ip -j CLASSIFY --set-class 1:2 > iptables -t mangle -A POSTROUTING -d $vm2_ip -j CLASSIFY --set-class 1:2 > tc class add dev eth0 parent 1: classid 1:2 htb rate 512mbit > > I setup a web server in VM1 and download a 1GB file from another machine > that is on the same network (actually on the same enclosure). I always > see wire speeds on the client side. I have tried many configurations > including adding a sfq, pfifo, tbf class under the leaf classes, but > either the rate becomes too low (because packets are dropped at the > leaves) or too high. > > Part of the problem lies in the fact the vif1.0 has already received the > traffice, so it has to be overlimited at eth0, instead of dropping. So, > I tried a simple tbf within the VM. That doesn't work either with very > low speeds. Xen VMs don't have very precise clocks, so that might be one > reason why the reliable tbf is also not performing well. > > I also set the burst sizes manually and the speed again becomes > exceptionally low. > > Please let me know if you have any ideas on why this is happening. I can > paste the stats as well, if required. I've never used zen, but IIRC it uses GSO Generic Segmentation offload - like some gig nics. For nics you need to turn it off with ethtool -k, so you could try and see if you can do the same for virtual nics. Andy. _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc