Skip to contentSkip to conversation

htb not hard limiting correctly

LARTC Abraham van der Merwe 2 participants 3 messages Sep 21, 2003 – Sep 22, 2003

lore
Share & tools
lore
#0 lore raw

Hi!

This morning one of our major links died and suddenly I was streaming 1mbit of traffic through 256kbit (one of our backup links), but it just doesn't work. The client is still doing in the region of 350kbit/s (the link capacity is 512kbit). Here is my config:

------------< snip <------< snip <------< snip <------------
tc qdisc add dev eth1 root handle 1: htb default 4 r2q 5
tc class add dev eth1 parent 1: classid 1:1 htb rate 512kbit
tc class add dev eth1 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth1 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth1 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth1 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth1 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth1 handle 4: parent 1:4 sfq perturb 10 limit 84 tc qdisc add dev eth0 root handle 1: htb default 4 r2q 5
tc class add dev eth0 parent 1: classid 1:1 htb rate 512kbit
tc class add dev eth0 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth0 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth0 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth0 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth0 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth0 handle 4: parent 1:4 sfq perturb 10 limit 84 ------------< snip <------< snip <------< snip <------------

I'm tagging packets with the netfilter CLASSIFY patch. I've got a user-defined chain in the mangle table and I redirect all traffic from the POSTROUTING chain into that and add rules like this:

iptables -t mangle -N qos
iptables -t mangle -A POSTROUTING -j qos

and then these rules for the ftc rules above:

------------< snip <------< snip <------< snip <------------ iptables -t mangle -A qos -o eth1 -d 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth1 -d 66.8.28.32/27 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.32/27 -j CLASSIFY --set-class 1:3 ------------< snip <------< snip <------< snip <------------

The class which is not shaping correctly is 1:3 - it has a rate of 64kbit and a ceil of 256kbit so it should be limited to 256kbit even if there is 512kbit of traffic available.

Any ideas what I did wrong or is this a bug?

Signature · 16 lines
Regards
 Abraham

Bernard Shaw is an excellent man; he has not an enemy in the world, and
none of his friends like him either.
		-- Oscar Wilde

___________________________________________________
 Abraham vd Merwe - Frogfoot Networks CC
 9 Kinnaird Court, 33 Main Street, Newlands, 7700
 Phone: +27 21 686 1665 Cell: +27 82 565 4451
 Http: http://www.frogfoot.net/ Email: [email protected]

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#1 lore raw

On Sun, 21 Sep 2003, Abraham van der Merwe wrote:
I am not sure, but I think if you classify your packet in PREROUTING chain it will works fine

Quoted 50 lines

Hi!

This morning one of our major links died and suddenly I was streaming 1mbit of traffic through 256kbit (one of our backup links), but it just doesn't work. The client is still doing in the region of 350kbit/s (the link capacity is 512kbit). Here is my config:

------------< snip <------< snip <------< snip <------------
tc qdisc add dev eth1 root handle 1: htb default 4 r2q 5
tc class add dev eth1 parent 1: classid 1:1 htb rate 512kbit
tc class add dev eth1 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth1 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth1 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth1 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth1 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth1 handle 4: parent 1:4 sfq perturb 10 limit 84 tc qdisc add dev eth0 root handle 1: htb default 4 r2q 5
tc class add dev eth0 parent 1: classid 1:1 htb rate 512kbit
tc class add dev eth0 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth0 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth0 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth0 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth0 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth0 handle 4: parent 1:4 sfq perturb 10 limit 84 ------------< snip <------< snip <------< snip <------------

I'm tagging packets with the netfilter CLASSIFY patch. I've got a user-defined chain in the mangle table and I redirect all traffic from the POSTROUTING chain into that and add rules like this:

iptables -t mangle -N qos
iptables -t mangle -A POSTROUTING -j qos

and then these rules for the ftc rules above:

------------< snip <------< snip <------< snip <------------ iptables -t mangle -A qos -o eth1 -d 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth1 -d 66.8.28.32/27 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.32/27 -j CLASSIFY --set-class 1:3 ------------< snip <------< snip <------< snip <------------

Quoted 7 lines

The class which is not shaping correctly is 1:3 - it has a rate of 64kbit and a ceil of 256kbit so it should be limited to 256kbit even if there is 512kbit of traffic available.

Any ideas what I did wrong or is this a bug?

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#2 lore raw
In reply to Abraham van der Merwe

Hi Mahmoud >@2003.09.22_11:44:01_+0200

(a) You can't put it in PREROUTING - the CLASSIFY patch don't allow that:

from net/ipv4/netfilter/ipt_CLASSIFY.c:

------------< snip <------< snip <------< snip <------------

    if (hook_mask & ~(1 << NF_IP_POST_ROUTING)) {
        printk(KERN_ERR "CLASSIFY: only valid in POST_ROUTING.\n");
        return 0;
    }

------------< snip <------< snip <------< snip <------------

Would you mind telling me why you suggested PREROUTING?

(b)

I discovered some serious problems in my ruleset below which I fixed in the meantime (Notice all the qdiscs have the same major id - they're supposed to be unique for each qdisc, i.e. the sfq qdisc's and the two htb qdisc's should have different major numbers)

(c)

One thing I discovered as well is that if all traffic goes through one of our backup links, the line is so congested that I still get around double the traffic incoming on the qos box itself. If I check on a box behind it you can see that that class is shaped correctly, the requests are just flooding in at such a pace that it still eats up twice the bandwidth!

Quoted 65 lines

I am not sure, but I think if you classify your packet in PREROUTING chain it will works fine

Quoted 50 lines

Hi!

This morning one of our major links died and suddenly I was streaming 1mbit of traffic through 256kbit (one of our backup links), but it just doesn't work. The client is still doing in the region of 350kbit/s (the link capacity is 512kbit). Here is my config:

------------< snip <------< snip <------< snip <------------
tc qdisc add dev eth1 root handle 1: htb default 4 r2q 5
tc class add dev eth1 parent 1: classid 1:1 htb rate 512kbit
tc class add dev eth1 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth1 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth1 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth1 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth1 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth1 handle 4: parent 1:4 sfq perturb 10 limit 84 tc qdisc add dev eth0 root handle 1: htb default 4 r2q 5
tc class add dev eth0 parent 1: classid 1:1 htb rate 512kbit
tc class add dev eth0 parent 1:1 classid 1:2 htb rate 320kbit ceil 512kbit prio 1 tc qdisc add dev eth0 handle 2: parent 1:2 sfq perturb 10 limit 84 tc class add dev eth0 parent 1:1 classid 1:3 htb rate 64kbit ceil 256kbit prio 3 tc qdisc add dev eth0 handle 3: parent 1:3 sfq perturb 10 limit 42 tc class add dev eth0 parent 1:1 classid 1:4 htb rate 128kbit ceil 512kbit prio 2 tc qdisc add dev eth0 handle 4: parent 1:4 sfq perturb 10 limit 84 ------------< snip <------< snip <------< snip <------------

I'm tagging packets with the netfilter CLASSIFY patch. I've got a user-defined chain in the mangle table and I redirect all traffic from the POSTROUTING chain into that and add rules like this:

iptables -t mangle -N qos
iptables -t mangle -A POSTROUTING -j qos

and then these rules for the ftc rules above:

------------< snip <------< snip <------< snip <------------ iptables -t mangle -A qos -o eth1 -d 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth1 -d 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth1 -d 66.8.28.32/27 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.92/30 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 66.8.28.160/27 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.55.0/24 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.54.80/29 -j CLASSIFY --set-class 1:2 iptables -t mangle -A qos -o eth0 -s 168.210.56.0/24 -j CLASSIFY --set-class 1:3 iptables -t mangle -A qos -o eth0 -s 66.8.28.32/27 -j CLASSIFY --set-class 1:3 ------------< snip <------< snip <------< snip <------------

Quoted 7 lines

The class which is not shaping correctly is 1:3 - it has a rate of 64kbit and a ceil of 256kbit so it should be limited to 256kbit even if there is 512kbit of traffic available.

Any ideas what I did wrong or is this a bug?

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Signature · 14 lines
Regards
 Abraham

But Officer, I stopped for the last one, and it was green!

___________________________________________________
 Abraham vd Merwe - Frogfoot Networks CC
 9 Kinnaird Court, 33 Main Street, Newlands, 7700
 Phone: +27 21 686 1665 Cell: +27 82 565 4451
 Http: http://www.frogfoot.net/ Email: [email protected]

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/