Skip to contentSkip to conversation

$100 USD to the first person that can provide the rules/scripts that will solve the QOS late

LARTC Greg Freeman 3 participants 13 messages Dec 3, 2003 – Dec 5, 2003

lore
Share & tools
lore
#0 lore raw

To stress the urgency and importance of my questions, I am willing to pay $100 to the first person that can provide me with the scripts/ rules that will work in my SnapGear firewalls that will solve the problems I am having.

Please see the following post:

Linux QOS and prioritization of real-time data (RTP/VoIP)

Thank you!

#1 lore raw

Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

Quoted 4 lines · Greg Freeman

To stress the urgency and importance of my questions, I am willing to pay $100 to the first person that can provide me with the scripts/ rules that will work in my SnapGear firewalls that will solve the problems I am having.

Maybe this can help :

- use the htb qdisc
- for your RTP/VoIP class, use a short prio qdisc, not a sfq qdisc
- preserve a minimum bandwidth for RTP/Voip
example : limit all non RTP/Voip traffic to 600kbps so RTP/VoIP has always
40kbps immediatly available. This can be done with the ceil parameter if you use htb. If you don't do this and RTP/VoIP needs bandwidth, the other traffic has to throttle down and that can take some time.
- if you use htb, give the RTP/VoIP class a lower prio BUT !!!! make sure that that class NEVER sends more data then the rate you give to that class.

Good luck.

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.openprojects.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#2 lore raw

RE: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman

I tried the following rules and had the same bad latency results for the corp site to remote site. Below are the rules I tried which gave the bad (avg. 350ms) latency:

----corp rules----

tc qdisc add dev ipsec0 handle 1: root cbq bandwidth 600kbit avpkt 1000 tc class add dev ipsec0 parent 1: classid 1:1 cbq bandwidth 240kbit avpkt 500 prio 1 rate 240kbit bounded isolated
tc filter add dev ipsec0 parent 1:0 protocol ip prio 100 route tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.1.20 flowid 1:1
tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.1.21 flowid 1:1
tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.1.22 flowid 1:1

----remote rules------

tc qdisc add dev ipsec0 handle 1: root cbq bandwidth 580kbit avpkt 1000 tc class add dev ipsec0 parent 1: classid 1:1 cbq bandwidth 240kbit avpkt 500 prio 1 rate 240kbit bounded isolated
tc filter add dev ipsec0 parent 1:0 protocol ip prio 100 route tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.0.7 flowid 1:1

Any help would be greatly appreciated.

Please see the following post:

Linux QOS and prioritization of real-time data (RTP/VoIP)

________________________________

From: Greg Freeman
Sent: Wednesday, December 03, 2003 9:04 AM
To: [email protected]
Subject: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS latency & bandwidth allocation issue !!!!
Importance: High

To stress the urgency and importance of my questions, I am willing to pay $100 to the first person that can provide me with the scripts/ rules that will work in my SnapGear firewalls that will solve the problems I am having.

Please see the following post:

Linux QOS and prioritization of real-time data (RTP/VoIP)

Thank you!
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#3 lore raw

Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman
Quoted 32 lines · Greg Freeman

I tried the following rules and had the same bad latency results for the corp site to remote site. Below are the rules I tried which gave the bad (avg. 350ms) latency:

----corp rules----

tc qdisc add dev ipsec0 handle 1: root cbq bandwidth 600kbit avpkt 1000 tc class add dev ipsec0 parent 1: classid 1:1 cbq bandwidth 240kbit avpkt 500 prio 1 rate 240kbit bounded isolated
tc filter add dev ipsec0 parent 1:0 protocol ip prio 100 route tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.1.20 flowid 1:1
tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.1.21 flowid 1:1
tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.1.22 flowid 1:1

----remote rules------

tc qdisc add dev ipsec0 handle 1: root cbq bandwidth 580kbit avpkt 1000 tc class add dev ipsec0 parent 1: classid 1:1 cbq bandwidth 240kbit avpkt 500 prio 1 rate 240kbit bounded isolated
tc filter add dev ipsec0 parent 1:0 protocol ip prio 100 route tc filter add dev ipsec0 parent 1:0 protocol ip prio 1 u32 match ip dst 10.0.0.7 flowid 1:1

Quoted 1 line

Any help would be greatly appreciated.

The bandwidth must be your NIC bandwidth. So 10mbit or 100mbit. And remove the isolated parameter.

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.openprojects.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#4 lore raw

Re: [LARTC] $100 USD to the first person that can provide the rules/scripts

In reply to Greg Freeman

Greg,

You do not say in your original article how your 640kbit/640kbit link is attached, eth0, eth1 or pppX? If the interface is dedicated only for WAN usage, or if the WAN be via a gateway? If your VPN endpoint hosts are on the same or a different box to that with the 640kbit link.

You did not say if pinging between the two tunnel endpoint addresses (i.e. their non-10.x.x.x addresses) also had any latency reduction during the times of increased latency through the tunnel. I'm trying to establish if the 640kbit link is the point where the queuing occurs, or if its never saturated and the problem with your TC rule set.

Since I expect the 640kbit link is your bottle neck, and that there is a quantity of non-VPN traffic also passing through this point (web browsing, email, internet downloading), rules MUST also be applied here.
It is this point causing your latency issue, the device interface queue
is whats holding all the packets and thus adding propagation delay.
While the traffic going into the IPSEC it being controlled, there is
nothing I can see in your configuration which arbitrates bandwidth usage between webbrowsing and VPN data over the 640kbit link.

I am not the most knowledgable person about VPN and TC operation, but I would look for a solution using ipchains to MARK the DS/TOS field of the outside IP header of VPN packets only for VoIP/HiPriority packets, this has to be done on the box which is a VPN endpoint at the point each packet enters the tunnel. Maybe the inside TOS field is already copied into the outside TOS field during encapsulation (this is what I believe IPIP and GRE encaps would do). Use tcpdump on both the inside and outside interfaces, if possible to see. I would expect the UDP packets to already be marked with a "low latency" TOS (due to the nature of the applications), if this is the case then you have enough information to distinguish your important traffic from the rest over the 640kbit link.
Other than this suggestion I'm not sure how you can mark the outside IP
packet header based on the inside IP, UDP/TCP content during encapsulation (I'd like to know myself). I do not think any TC rules need to be applied to the ipsec0 interface, I believe you only need to ensure the packets have been marked at this point.

The next thing to do is, at the box with the 640kbit link, use your TC rule set to provide bandwidth reservation based on the protocol byte (VPN traffic, ESP/AH, etc..) and the IP DS/TOS header marking to select them to be placed in a higher priority queue.

Don't forget to account with the packet length increase due to the extra layer of headers of VPN traffic, both in overall bandwidth calculation and average packet length.

If your 640kbit link were say DSL and your DSL providers equipment was between the two 640kbit link interfaces, then there are also other issues with managing the inbound packet queue length at your ISP. Check out the "WonderShaper" (on google) for information on the problem and how the TC rules in it help the situation, but basically your Internet downloads need to be controlled to prevent queueing by your ISP when it sends inbound data to you.

Hope I read your original article correctly and am I'm pitching at the real problem,

Darryl

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#5 lore raw

RE: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman

Stef,
The current set of rules I have in it are as follows:

tc qdisc add dev ipsec0 root handle 1: htb default 20
tc class add dev ipsec0 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k
tc class add dev ipsec0 parent 1:1 classid 1:10 htb rate 240kbit tc class add dev ipsec0 parent 1:2 classid 1:20 htb rate 400kbit tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10
tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20
iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.20 -j MARK --set-mark 10 iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.21 -j MARK --set-mark 10 iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.22 -j MARK --set-mark 10

This still does not solve the latency (or even seem to affect it, but perhaps this is a better route in trying to solve this issue?) Please let me know you thoughts,

Thanks

Greg

-----Original Message-----
From: Stef Coene [mailto:[email protected]]
Sent: Thursday, December 04, 2003 10:42 AM
To: Greg Freeman
Subject: Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS latency & bandwidth allocation issue !!!!

Quoted 10 lines · Greg Freeman

Thanks Stef,

If I knew how and what (to specifically state with the htb qdisc scripts I would try.

Can I just substitute the word prio for sfq?

If you could type the rules I should use it would help greatly, and if

it works I would be happy to pay you.

Before I help you with the htb rules, do you have a kernel and tc binary with htb support?

Stef

--
[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.openprojects.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#6 lore raw

Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman
Quoted 22 lines · Greg Freeman

Stef,
The current set of rules I have in it are as follows:

tc qdisc add dev ipsec0 root handle 1: htb default 20
tc class add dev ipsec0 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k
tc class add dev ipsec0 parent 1:1 classid 1:10 htb rate 240kbit tc class add dev ipsec0 parent 1:2 classid 1:20 htb rate 400kbit tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10
tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20
iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.20 -j MARK --set-mark 10 iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.21 -j MARK --set-mark 10 iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.22 -j MARK --set-mark 10

Quoted 3 lines

This still does not solve the latency (or even seem to affect it, but perhaps this is a better route in trying to solve this issue?) Please let me know you thoughts,

Try this :

tc qdisc del dev ipsec0 root
tc qdisc add dev ipsec0 root handle 1: htb default 20
tc class add dev ipsec0 parent 1: classid 1:1 htb rate 600kbit ceil 600kbit tc class add dev ipsec0 parent 1:1 classid 1:10 htb rate 240kbit ceil 600kbit tc class add dev ipsec0 parent 1:2 classid 1:20 htb rate 400kbit ceil 500kbit tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 0x10 fw flowid 1:10
tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 0x20 fw flowid 1:20
iptables -F -t mangle
iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.20 -j MARK --set-mark 0x10 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.21 -j MARK --set-mark 0x10 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.22 -j MARK --set-mark 0x10

10.1.1.20/21/22 are the VoIP systems ?

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.openprojects.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#7 lore raw

RE: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman

Good guess, 10.0.0.7 is actually the VoIP phone system, which can handle multiple calls. 10.0.1.20-22 are actually IP phones

I just pasted your other changes and was about to apply them. I will try it first, then the below and let you know the results.

-----Original Message-----
From: Stef Coene [mailto:[email protected]]
Sent: Thursday, December 04, 2003 11:37 AM
To: Greg Freeman
Cc: [email protected]
Subject: Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS latency & bandwidth allocation issue !!!!

Quoted 24 lines · Greg Freeman

Stef,
The current set of rules I have in it are as follows:

tc qdisc add dev ipsec0 root handle 1: htb default 20 tc class add dev

ipsec0 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k tc

class add dev ipsec0 parent 1:1 classid 1:10 htb rate 240kbit tc class

add dev ipsec0 parent 1:2 classid 1:20 htb rate 400kbit tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10 tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20 iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark

20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.20 -j MARK --set-mark 10 iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.21 -j MARK --set-mark 10 iptables -A PREROUTING -t mangle -p ALL -j MARK --set-mark 20 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.22 -j MARK --set-mark 10

Quoted 4 lines

This still does not solve the latency (or even seem to affect it, but perhaps this is a better route in trying to solve this issue?) Please

let me know you thoughts,

Try this :

tc qdisc del dev ipsec0 root
tc qdisc add dev ipsec0 root handle 1: htb default 20 tc class add dev ipsec0 parent 1: classid 1:1 htb rate 600kbit ceil 600kbit tc class add dev ipsec0 parent 1:1 classid 1:10 htb rate 240kbit ceil 600kbit tc class add dev ipsec0 parent 1:2 classid 1:20 htb rate 400kbit ceil 500kbit tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 0x10 fw flowid 1:10 tc filter add dev ipsec0 parent 1:0 prio 0 protocol ip handle 0x20 fw flowid 1:20 iptables -F -t mangle iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.20 -j MARK --set-mark 0x10 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.21 -j MARK --set-mark 0x10 iptables -A PREROUTING -t mangle -p ALL -d 10.0.1.22 -j MARK --set-mark 0x10

10.1.1.20/21/22 are the VoIP systems ?

Stef

--
[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.openprojects.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#8 lore raw

RE: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman

Removed all the isolated, latency result was the same. Tried the new rule(s) on the site1 and again had a latency of 318ms. However, dropped packets increased to the highest I have seen it... 17%. The min latency was 30ms, the max was 881ms

-----Original Message-----
From: Stef Coene [mailto:[email protected]]
Sent: Thursday, December 04, 2003 11:41 AM
To: Greg Freeman
Subject: Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS latency & bandwidth allocation issue !!!!

Quoted 2 lines · Greg Freeman

Thanks! Should I remove it from any/all sections that mentions isolated then?

Yes.

Stef

--
[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.openprojects.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#9 lore raw

Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman
Quoted 4 lines · Greg Freeman

Removed all the isolated, latency result was the same. Tried the new rule(s) on the site1 and again had a latency of 318ms. However, dropped packets increased to the highest I have seen it... 17%. The min latency was 30ms, the max was 881ms

Removing the isolated parameter will not fix anything. But i did some tests and if I specified the isolated parameter, the cbq setup was broken.

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.openprojects.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#10 lore raw

Re: [LARTC] $100 USD to the first person that can provide the rules/scripts

In reply to Greg Freeman
Quoted 33 lines · Greg Freeman

Thank you Darryl,

Hopefully the below will help clarify.

(below was run on the site1 firewall) Does the RX line look bad to you for Eth0?

ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:D0:CF:01:A6:52

         inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:20061386 errors:206047 dropped:5857547
overruns:138637 frame:0
         TX packets:8390751 errors:0 dropped:0 overruns:1 carrier:0
         collisions:0 txqueuelen:100 
         Interrupt:5 

eth1 Link encap:Ethernet HWaddr 00:D0:CF:01:A6:53

         inet addr:24.239.167.x  Bcast:24.237.169.255
Mask:255.255.254.0
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:4246757 errors:0 dropped:0 overruns:0 frame:0
         TX packets:7470351 errors:0 dropped:0 overruns:6 carrier:0
         collisions:48974 txqueuelen:100 
         Interrupt:8 

ipsec0 Link encap:Ethernet HWaddr 00:D0:CF:01:A6:53

         inet addr:24.239.167.x  Mask:255.255.254.0
         UP RUNNING NOARP  MTU:16260  Metric:1
         RX packets:625899 errors:0 dropped:1 overruns:0 frame:0
         TX packets:658838 errors:373 dropped:1 overruns:0 carrier:373
         collisions:0 txqueuelen:10 
Quoted 5 lines

(external IP has been modified)

"640kbps" is the quoted sDSL speed from the ISP, actually see about 580kbps. The DSL connections are attached to the etho on each firewall,

One thing I didn't ask before and had presumed from the configuration, does your VoIP data get sent over the VPN ? I had presumed yes.

I would guess from your configuration of site1 the Internet is presented to you by ethernet on eth1 and that its the only thing on that interface ?

Your output of eth0 maybe because you are running a mismatching ifconfig tools version to the kernel version, check the response from 'cat /proc/net/dev' manually.

Quoted 6 lines

IPSEC tunnel configured as a tunnel to the "remote network" and not as a gateway (this is done on both sides -site1 & site2). Site1 VPN endpoint is a SnapGear SME550, site 2 is a SnapGear Pro. Both firewall have a 640kbps SDSL connection.

I'm not familar with SnapGear, I would like to know what VoIP handsets you are running ?

I'm not sure what you mean by "remote network" as opposed to "gateway".
Since the definition of a "gateway" means to provide access to a
"remote network".

Quoted 2 lines

If you have any ideas on what rules I should try please let me know!

I don't know if this will even run, what I'm trying to say would look something like this:

#######################################################################
#
# Remote site end (with 3 phones)
#
# Mark VoIP protocol for special treatment
iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.1.20 --set-tos 0x10 iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.1.21 --set-tos 0x10 iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.1.22 --set-tos 0x10

# eth1, which is your Internet/DSL interface
# Setup the basic reservation of 240kbit
tc qdisc add dev eth1 root handle 1: htb default 20
tc class add dev eth1 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k tc class add dev eth1 parent 1:1 classid 1:10 htb rate 240kbit
tc class add dev eth1 parent 1:2 classid 1:20 htb rate 400kbit

# Match IP Protocol type 47 (Check your VPN technology uses this protocol ID # number in the IP header)
# Match Low Through put TOS (Check the inner IP-TOS value is copied into the # outer IP-TOS value by your protocol stack, use
# "tcpdump -i eth1 -n -s 1500 -v" and capture some VPN data with UDP inside) # Match the IP address of the VPN tunnel endpoint, however if you are on DSL

#  with dynamic IP then you might as well scrap this value, otherwise change
#  "0.0.0.0/0" to just "10.0.0.something/32" or remove
#  "match ip src 0.0.0.0/0" completely.
#  This would enforce the IP of the tunnel endpoint gets special treatment so
#  that no one else on the lan (like peoples workstations) couldn't start
#  using their own VPN with TOS=0x10 and take this bandwidth.

tc filter add dev eth1 parent 1: prio 1 protocol ip u32 match ip protocol 47 0xff match ip tos 0x10 0xff match ip src 0.0.0.0/0 flowid 1:10

# Everything else gets flowid 1:20
tc filter add dev eth1 parent 1: prio 2 protocol ip prio 21 u32 match ip dst 0.0.0.0/0 flowid 1:20

#####################################################################
#
# Corp site end
#
# Mark VoIP protocol from gatekeeper/switch for special treatment iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.0.7 --set-tos 0x10

# eth1, which is your Internet/DSL interface
# Setup the basic reservation of 240kbit
tc qdisc add dev eth1 root handle 1: htb default 20
tc class add dev eth1 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k tc class add dev eth1 parent 1:1 classid 1:10 htb rate 240kbit
tc class add dev eth1 parent 1:2 classid 1:20 htb rate 400kbit

# Match IP Protocol type 47 (Check your VPN technology uses this protocol ID # number in the IP header)
# Match Low Through put TOS (Check the inner IP-TOS value is copied into the # outer IP-TOS value by your protocol stack, use
# "tcpdump -i eth1 -n -s 1500 -v" and capture some VPN data with UDP inside) # Match the IP address of the VPN tunnel endpoint, however if you are on DSL

#  with dynamic IP then you might as well scrap this value, otherwise change
#  "0.0.0.0/0" to just "10.0.0.something/32" or remove
#  "match ip src 0.0.0.0/0" completely.
#  This would enforce the IP of the tunnel endpoint gets special treatment so
#  that no one else on the lan (like peoples workstations) couldn't start
#  using their own VPN with TOS=0x10 and take this bandwidth.

tc filter add dev eth1 parent 1: prio 1 protocol ip u32 match ip protocol 47 0xff match ip tos 0x10 0xff match ip src 0.0.0.0/0 flowid 1:10

# Everything else gets flowid 1:20
tc filter add dev eth1 parent 1: prio 2 protocol ip prio 21 u32 match ip dst 0.0.0.0/0 flowid 1:20

Quoted 6 lines

PS. I am new to posting, and would like to update the thread, but I don't know how to a reply to the thread, I seem to only make a new post.

You have to subscribe and reply from the address you have subscribed as.

List-Subscribe: <http://mailman.ds9a.nl/mailman/listinfo/lartc>,
<mailto:[email protected]?subject=subscribe>

Quoted 5 lines

Thanks for your suggestions, I will take a look at the Wondershaper, I saw some articles mention it but I didn't know if I could use it on the firewalls.

Good use on low bandwidth links.

Quoted 73 lines

-----Original Message-----
From: Darryl Miles [mailto:[email protected]]
Sent: Thursday, December 04, 2003 10:39 AM
To: Greg Freeman
Cc: [email protected]
Subject: Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS latency & bandwidth allocation issue !!!!

Greg,

You do not say in your original article how your 640kbit/640kbit link is attached, eth0, eth1 or pppX? If the interface is dedicated only for WAN usage, or if the WAN be via a gateway? If your VPN endpoint hosts are on the same or a different box to that with the 640kbit link.

You did not say if pinging between the two tunnel endpoint addresses (i.e. their non-10.x.x.x addresses) also had any latency reduction during the times of increased latency through the tunnel. I'm trying to establish if the 640kbit link is the point where the queuing occurs, or if its never saturated and the problem with your TC rule set.

Since I expect the 640kbit link is your bottle neck, and that there is a quantity of non-VPN traffic also passing through this point (web browsing, email, internet downloading), rules MUST also be applied here.

It is this point causing your latency issue, the device interface queue is whats holding all the packets and thus adding propagation delay. While the traffic going into the IPSEC it being controlled, there is nothing I can see in your configuration which arbitrates bandwidth usage between webbrowsing and VPN data over the 640kbit link.

I am not the most knowledgable person about VPN and TC operation, but I would look for a solution using ipchains to MARK the DS/TOS field of the outside IP header of VPN packets only for VoIP/HiPriority packets, this has to be done on the box which is a VPN endpoint at the point each packet enters the tunnel. Maybe the inside TOS field is already copied into the outside TOS field during encapsulation (this is what I believe IPIP and GRE encaps would do). Use tcpdump on both the inside and outside interfaces, if possible to see. I would expect the UDP packets to already be marked with a "low latency" TOS (due to the nature of the applications), if this is the case then you have enough information to distinguish your important traffic from the rest over the 640kbit link. Other than this suggestion I'm not sure how you can mark the outside IP packet header based on the inside IP, UDP/TCP content during encapsulation (I'd like to know myself). I do not think any TC rules need to be applied to the ipsec0 interface, I believe you only need to ensure the packets have been marked at this point.

The next thing to do is, at the box with the 640kbit link, use your TC rule set to provide bandwidth reservation based on the protocol byte (VPN traffic, ESP/AH, etc..) and the IP DS/TOS header marking to select them to be placed in a higher priority queue.

Don't forget to account with the packet length increase due to the extra layer of headers of VPN traffic, both in overall bandwidth calculation and average packet length.

If your 640kbit link were say DSL and your DSL providers equipment was between the two 640kbit link interfaces, then there are also other issues with managing the inbound packet queue length at your ISP. Check out the "WonderShaper" (on google) for information on the problem and how the TC rules in it help the situation, but basically your Internet downloads need to be controlled to prevent queueing by your ISP when it sends inbound data to you.

Quoted 11 lines

Hope I read your original article correctly and am I'm pitching at the real problem,

Darryl

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#11 lore raw

Re: [LARTC] $100 USD to the first person that can provide the rules/scripts

In reply to Greg Freeman
Quoted 5 lines · Darryl Miles

# Match the IP address of the VPN tunnel endpoint, however if you are on DSL
# with dynamic IP then you might as well scrap this value, otherwise change
# "0.0.0.0/0" to just "10.0.0.something/32" or remove

Opps...

to just "24.239.167.x/32", the point is if you are on dynamic IP its probably more trouble than its worth to also check this, it can be omited ("match ip src 0.0.0.0/0") and the config will still work.

This is the same as Steff's config (just 4 lines mangled by cut'n'paste):

Quoted 14 lines

tc qdisc add dev eth1 root handle 1: htb default 20 tc class add dev eth1 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k tc class add dev eth1 parent 1:1 classid 1:10 htb rate 240kbit tc class add dev eth1 parent 1:2 classid 1:20 htb rate 400kbit

tc qdisc add dev eth1 root handle 1: htb default 20

tc class add dev eth1 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k

tc class add dev eth1 parent 1:1 classid 1:10 htb rate 240kbit

tc class add dev eth1 parent 1:2 classid 1:20 htb rate 400kbit

Signature · 8 lines
Darryl L. Miles
M: 07968 320 114



_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#12 lore raw

RE: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS

In reply to Greg Freeman

Thanks Patrick,
But I am not sure if it is possible to apply it to the embedded Linux .

-----Original Message-----
From: Patrick McHardy [mailto:[email protected]]
Sent: Thursday, December 04, 2003 5:42 PM
To: Greg Freeman
Subject: Re: [LARTC] $100 USD to the first person that can provide the rules/scripts that will solve the QOS latency & bandwidth allocation issue !!!!

See trash.net/~kabet/hfsc for a packet scheduler which allows delay and bandwidth decoupling.
Maybe that helps.

Regards,
Patrick

Quoted 6 lines · Greg Freeman

To stress the urgency and importance of my questions, I am willing to

pay $100 to the first person that can provide me with the scripts/ rules that will work in my SnapGear firewalls that will solve the problems I am having.

Quoted 6 lines

Please see the following post:

Linux QOS and prioritization of real-time data (RTP/VoIP)

Thank you!

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/