egress bandwidth not limited / limited extremely
I'm just-so-fresh to the list, so hello everyone.
I'm having a realy hard time with setting up very simple bandwidth management. What I am trying to do is setup a 10Mbit interface to send at only 1920kbit to most of the network (to make sure it hardly ever tops 2mbit). I did succeed with the ingress traffic... With egress however, I get very odd results at different speeds.
With HTB:
If i try to throttle at 0.5mbit for example, a windows box on 1mbit ADSL line is capable of downloading at ~64k from the box i'm limiting, while a bsd box on a 4mbit DSL line happily downloads at ~200k. Limiting to 2mbit looks like there are no results at all
With TBF:
The dowloads from the limited box are always ~5k, at least to what I've observed with different configurations (even with limits at 2mbit).
I'm doing all of that directly on the Internet with public IPs.
I think I've read all the TLDP and LARTC to no results, so would greatly appreciate any help with strictly cutting the bandwidth ath ~1920kbit. What am I missing?
The scrips I've tried (or tried their variants):
#tc qdisc add dev eth0 root handle 1: htb default 20 #tc class add dev eth0 parent 1: classid 1:1 htb rate 10mbit #tc class add dev eth0 parent 1:1 classid 1:10 htb rate 9.5mbit #tc class add dev eth0 parent 1:1 classid 1:20 htb rate 0.5mbit #tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 #tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 #tc filter add dev eth0 parent 1: protocol ip u32 match ip dst a.b.c.d/24 flowid 1:10
#tc qdisc add dev eth0 root tbf rate 0.5mbit burst 5kb latency 70ms peakrate 1mbit minburst 1540
(yes that does 0.5mbit limiting, but it would be a start...)
#tc qdisc add dev eth0 root handle 1: htb default 20
#tc class add dev eth0 parent 1: classid 1:1 htb rate 1920kbit #tc class add dev eth0 parent 1:1 classid 1:10 htb rate 128kbit prio 1 #tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1792kbit prio 2 #tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 #tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 #tc filter add dev eth0 parent 1:0 protocol ip prio 10 u32 match ip tos 0x10 0xff flowid 1:10
#tc filter add dev eth0 parent 1:0 protocol ip prio 10 u32 match ip protocol 1 0xff flowid 1:10
I use the latest iproute2 from the gentoo portage, marked as stable on amd64:
sys-apps/iproute2-2.6.16.20060323/
I manually setup to NIC at server boot, with:
#!/bin/sh
mii-tool -F 10baseT-FD eth0
(but I had the same problems before I did that when the box was in a private LAN)
The server details are:
(serenity:~) # uname -a89zx
Linux serenity 2.6.18-gentoo-r4 #1 SMP Mon Jan 15 12:33:56 CET 2007 x86_64 Intel(R) Pentium(R) D CPU 3.00GHz GenuineIntel GNU/Linux (this is genkernel generated)
(serenity:~) # grep -i config_hz /etc/kernels/kernel-config-x86_64-2.6.18-gentoo-r4 # CONFIG_HZ_100 is not setCONFIG_HZ_250=y
# CONFIG_HZ_1000 is not set
CONFIG_HZ%0
(serenity:~) # lspci | grep -i ether
05:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5714 Gigabit Ethernet (rev a3)
05:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5714 Gigabit Ethernet (rev a3)
(serenity:~) # lsmod
Module Size Used by sch_sfq 7424 0 sch_htb 18944 1 act_police 8480 1 cls_u32 9480 2 sch_ingress 5248 1 sch_tbf 8320 0 ipt_LOG 8192 1 xt_limit 4352 1 xt_state 3840 1 ip_conntrack 53092 1 xt_state xt_tcpudp 4864 5 iptable_filter 4736 1 iptable_mangle 4480 0 ip_tables 22312 2 iptable_filter,iptable_mangle x_tables 18824 5
ipt_LOG,xt_limit,xt_state,xt_tcpudp,ip_tables
(serenity:~) # iptables -nL
Chain INPUT (policy DROP)
target prot opt source destination ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:20
flags:0x17/0x02
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:21
flags:0x17/0x02
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
flags:0x17/0x02
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
flags:0x17/0x02
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp
dpts:20000:20100 flags:0x17/0x02
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state
RELATED,ESTABLISHED
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 8 LOG all -- 0.0.0.0/0 0.0.0.0/0 limit: avg
3/hour burst 5 LOG flags 0 level 4 prefix `iptables-IN-policy: '
Chain FORWARD (policy DROP)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
(the mangle tables are empty)
Regards,
Signature · 6 lines
Roman Sku³a (http://roman.skula.com) _______________________________________________ LARTC mailing list [email protected] http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] egress bandwidth not limited / limited
Quoted 2 lines · Roman Skula
05:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5714 Gigabit Ethernet (rev a3)
You may need to turn off segmentation offload with ethtool -k
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Andy Furniss napisa³(a):
Quoted 4 lines
Quoted 2 lines · Roman Skula
05:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5714 Gigabit Ethernet (rev a3)
You may need to turn off segmentation offload with ethtool -k
A huge, wet kiss for you, this turned my sky blue again. :)
Many thanks!
Best of wishes,
Signature · 7 lines
Roman Sku³a (http://roman.skula.com) _______________________________________________ LARTC mailing list [email protected] http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Hi there,
Quoted 4 lines
Quoted 2 lines
05:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5714 Gigabit Ethernet (rev a3)
You may need to turn off segmentation offload with ethtool -k
That's very interesting. Do you have any more information on this subject? Does tso in general distrupt accuracy of tc ops or is it the case only with Broadcom nics? Any links would be very much appreciated.
Thanks for response in advance!
cheers,
Marek Kierdelewicz
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 7 lines · Marek Kierdelewicz
Hi there,
Quoted 2 lines
Quoted 2 lines
05:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5714 Gigabit Ethernet (rev a3)
Quoted 6 lines
Quoted 1 line
You may need to turn off segmentation offload with ethtool -k
That's very interesting. Do you have any more information on this subject? Does tso in general distrupt accuracy of tc ops or is it the case only with Broadcom nics? Any links would be very much appreciated.
I don't have any gig eth, but it's not just Broadcom.
There have been other posts on here where people have needed to increase
htb's mtu parameter to get htb to handle it - despite the interface
being 1500. For shaping traffic for a slow link that is not going to be nice - you could end up dropping multiple tcp segments at once, and it will hurt jitter. I don't imagine it's right to do it for shaping at gig speed either, maybe the default long queue on eth saves dropping.
I suppose those using gig eth and shaping routed traffic at 1500 MTU will not notice unless they try and shape locally generated aswell.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] egress bandwidth not limited / limited
In reply to Roman SkulaQuoted 10 lines · Roman Skula
Andy Furniss napisa³(a):
Quoted 6 lines
Quoted 2 lines · Roman Skula
05:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5714 Gigabit Ethernet (rev a3)
You may need to turn off segmentation offload with ethtool -k
A huge, wet kiss for you, this turned my sky blue again. :)
Lol - I forgot to mention: be carefull about using htb default, your arp/other link layer end up there. If you don't specify a default htb will let them through unshaped, you can make a filter to catch the unclassified IP traffic and send it to the class you want.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc