Skip to contentSkip to conversation

HTB Weird Shaping Question(Bug?). Please Help!

LARTC Leo 2 participants 2 messages Apr 26, 2005

lore
Share & tools
lore

Leo

author
#0 lore raw

Hi all,

I have a script that allocates an ADSL(1500K/256K) bandwidth to three users.

My idea is to allow each user having a guarentee rate, while each one is allowed to burst to the max rate while no one is using the bandwidth.

I use imq0 for uplink (for some mobility reason) and imq1 for downlink. Everything works smoothly except for VoIP traffic.

There are three VoIP phones attached to a computer. They are set up for different providers. It is possible that all of them are engaged at the same time. And it does happen that all of them are using G711u codec, which consuming around 110kbit each for each direction (i.e. the total bandwidth consumed is larger than the physical link rate 256K). In this situation, it slowed down the whole network affecting all classes, which should not be happening. I have done a bit of testing, and isolated the problem to the uplink congestion.

Please refer to the script I attached blow. I classify all the udp traffic(generated by the VoIP applications) to 1:110. I continously generate ping traffic on the computer and the router to a remote computer in the ISP network. The pings from the computer are classified as 1:105, while the pings from the router are classified as 1:800. The max uplink speed is 220Kbit determined by observation.

In the first test, I limit the SUBCLASS_OUTRATE to 200Kbit. Both pings are around 20ms before I start the VoIP services. However, once I start the services, the pings jump up to 1800ms.

In the second test, I limit the SUBCLASS_OUTRATE to 180Kbit. The pings jump up to 80ms, which is perfectly acceptable.

After a few tests, I noticed that 180Kbit is a magic number, anything exceed that will generate 1800ms pings, and below it is 80ms.

In my senario, the weird point is that the determining factor is the ceiling, but not the rate. That's the "rate" for other class doesn't seem to give bandwidth to packets in the corresponding class unless the ceil for the 1:110 is low enough!

I attached my script and "tc -s class show" below. I truncated part of the script and the results to make it short.

Please shine me a light!

Thanks heaps,
Leo

P.S. My router is Linksys WRT54G. I've tried top, it doesn't LOOK LIKE a CPU load problem.

Script:
####################################
#!/bin/sh

#ADDRs
ROUTER2.168.1.1
LEO_LAP_WL2.168.1.31
LEO_LAP2.168.1.32
LEO_DES2.168.1.33
PONY_DES2.168.1.34
ERIC_DES2.168.1.35
ERIC_LAP2.168.1.36

# Interfaces
OUTQ=imq0
INQ=imq1

# Speed Rates
ROOT_OUTRATE 0kbit
ROOT_INRATE10kbit

CLASS_OUTRATE 0kbit
CLASS_INRATE10kbit

SUBCLASS_OUTRATE0kbit
SUBCLASS_INRATE10kbit

# Init Interfaces
ip link set ${INQ} up
ip link set ${OUTQ} up
iptables -t mangle -F
iptables -t mangle -A POSTROUTING -o ppp0 -j IMQ --todev 0 iptables -t mangle -A PREROUTING -i ppp0 -j IMQ --todev 1

#Outbound Traffic
tc qdisc del dev ${OUTQ} root

tc qdisc add dev ${OUTQ} root handle 1: htb default 800
tc class add dev ${OUTQ} parent 1: classid 1:1 htb rate ${ROOT_OUTRATE}

#Leo
tc class add dev ${OUTQ} parent 1:1 classid 1:100 htb rate 17kbps ceil ${CLASS_OUTRATE} prio 0
tc class add dev ${OUTQ} parent 1:100 classid 1:105 htb rate 5kbps ceil ${SUBCLASS_OUTRATE} prio 3 # Other
tc class add dev ${OUTQ} parent 1:100 classid 1:110 htb rate 4kbps ceil ${SUBCLASS_OUTRATE} prio 0 # VoIP
tc class add dev ${OUTQ} parent 1:100 classid 1:120 htb rate 1kbps ceil ${SUBCLASS_OUTRATE} prio 0 # SSH
tc class add dev ${OUTQ} parent 1:100 classid 1:130 htb rate 1kbps ceil ${SUBCLASS_OUTRATE} prio 1 # Telnet
tc class add dev ${OUTQ} parent 1:100 classid 1:140 htb rate 2kbps ceil ${SUBCLASS_OUTRATE} prio 1 # TermSrv
tc class add dev ${OUTQ} parent 1:100 classid 1:150 htb rate 4kbps ceil ${SUBCLASS_OUTRATE} prio 2 # Mail

#Pony
tc class add dev ${OUTQ} parent 1:1 classid 1:200 htb rate 5kbps ceil ${CLASS_OUTRATE} prio 4

#Eric
tc class add dev ${OUTQ} parent 1:1 classid 1:300 htb rate 5kbps ceil ${CLASS_OUTRATE} prio 4

#Other
tc class add dev ${OUTQ} parent 1:1 classid 1:800 htb rate 0.5kbps ceil ${CLASS_OUTRATE} prio 5

tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 1 fw classid 1:105 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 2 fw classid 1:200 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 3 fw classid 1:300 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 4 fw classid 1:800 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 5 fw classid 1:110 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 6 fw classid 1:120 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 7 fw classid 1:130 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 8 fw classid 1:140 tc filter add dev ${OUTQ} parent 1:0 protocol ip handle 9 fw classid 1:150

tc qdisc add dev ${OUTQ} parent 1:105 handle 105: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:200 handle 200: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:300 handle 300: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:800 handle 800: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:110 handle 110: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:120 handle 120: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:130 handle 130: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:140 handle 140: pfifo limit 5 tc qdisc add dev ${OUTQ} parent 1:150 handle 150: pfifo limit 5

iptables -t mangle -A POSTROUTING --src ${LEO_DES} -p udp -j MARK --set-mark 5
iptables -t mangle -A POSTROUTING --src ${LEO_DES} -p udp -j RETURN iptables -t mangle -A POSTROUTING --src ${LEO_DES} -p gre -j MARK --set-mark 5
iptables -t mangle -A POSTROUTING --src ${LEO_DES} -p gre -j RETURN
...
...
######################################################################

######## Long delay 1800ms #######

class htb 1:110 parent 1:100 leaf 110: prio 0 rate 32Kbit ceil 200Kbit burst 1639b cburst 1856b
Sent 3108142 bytes 15541 pkts (dropped 2725, overlimits 0) rate 25459bps 127pps backlog 3p
lended: 2522 borrowed: 13016 giants: 0
tokens: -1918 ctokens: -2823

class htb 1:1 root rate 220Kbit ceil 220Kbit burst 1880b cburst 1880b
Sent 3137858 bytes 15954 pkts (dropped 0, overlimits 0) rate 25493bps 129pps
lended: 4990 borrowed: 0 giants: 0
tokens: 45460 ctokens: 45460

class htb 1:100 parent 1:1 rate 136Kbit ceil 220Kbit burst 1773b cburst 1880b
Sent 3120904 bytes 15754 pkts (dropped 0, overlimits 0) rate 25397bps 128pps
lended: 8026 borrowed: 4990 giants: 0
tokens: -6802 ctokens: 45460

##############################

######### Short Delay 80ms #############

class htb 1:110 parent 1:100 leaf 110: prio 0 rate 32Kbit ceil 180Kbit burst 1639b cburst 1829b
Sent 2675988 bytes 13461 pkts (dropped 4043, overlimits 0) rate 22888bps 114pps backlog 5p
lended: 2542 borrowed: 10914 giants: 0
tokens: -4451 ctokens: -5694

class htb 1:1 root rate 220Kbit ceil 220Kbit burst 1880b cburst 1880b
Sent 2927810 bytes 15992 pkts (dropped 0, overlimits 0) rate 22935bps 116pps
lended: 3295 borrowed: 0 giants: 0
tokens: 45460 ctokens: 45460

class htb 1:100 parent 1:1 rate 136Kbit ceil 220Kbit burst 1773b cburst 1880b
Sent 2801155 bytes 14888 pkts (dropped 0, overlimits 0) rate 22834bps 115pps
lended: 7650 borrowed: 3275 giants: 0
tokens: -8277 ctokens: 45460

##################################
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

#1 lore raw
Quoted 15 lines · Leo

In the first test, I limit the SUBCLASS_OUTRATE to 200Kbit. Both pings are around 20ms before I start the VoIP services. However, once I start the services, the pings jump up to 1800ms.

In the second test, I limit the SUBCLASS_OUTRATE to 180Kbit. The pings jump up to 80ms, which is perfectly acceptable.

After a few tests, I noticed that 180Kbit is a magic number, anything exceed that will generate 1800ms pings, and below it is 80ms.

In my senario, the weird point is that the determining factor is the ceiling, but not the rate. That's the "rate" for other class doesn't seem to give bandwidth to packets in the corresponding class unless the ceil for the 1:110 is low enough!

Quoted 4 lines

I attached my script and "tc -s class show" below. I truncated part of the script and the results to make it short.

Please shine me a light!

It's because the link is dsl and there are lots of overheads on each packet (and they vary with packet size). HTB rates are based on ip packet length and with lots of small packets like voip the difference can be alot.

The 1800ms latency is not caused by a queue within htb it's in your modem/router because it can't send >180kbit ip level for voip.

You can patch HTB and TC to make things perfect - you could set a ceil very close to your sync rate then. You need to know exactly what type of dsl you are on to find your overhead though. If your modem/router gives ATM cell counts you can deduce it from those.

There is a very good thesis and patch info here -

http://www.adsl-optimizer.dk/

Andy.

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc