Qos with 2 internet connections problems
Hi all, ive got 2 internet connections set up via the nano howto (which are working great) and we are running NAT.
Was looking in to qos mainly to stop large http downloads/ftp downloads from hogging the line so that browsing for other users doesnt slow to a crawl, but if the line is free and no one is doing anything then for it to use the available bandwith. The wondershaper sounded exactly what i wanted, i obviously had to customize this slightly to get my other connection to be shaped as well.
my connections are 2 adsl connections running at 512kbit:256kbit, i compiled the kernel with all the tos + iptables stuff (eventually) so that the script ran without any errors, once it had i did a download on each pipe and then tried to ping jolt.co.uk and google.co.uk, jolt.co.uk normally is about 15ms, so i was hoping that it would maybe be 200ms with my new shaping, but it was coming at about 900ms+ google was the same. Browsing other website was also to a crawl (what i was trying to avoid) but i did notice that when traffic came from our mail server through my gateway it was only receiving at about 25k/sec instead of 2mb/sec etc as its on a 100mbit switch. So something was obviously shaping, so i suppose i will now need to look in to specifying some of the hosts which are local to me but are on the outside interfaces and for them not to be shaped.. But still no better off :(
Ive been reading various howtos like the adsl bandwith management howto, but these havnt been updated in about 3 years and they mention various techniques which were being tried back then to help with these kind of situations, so what i really want is a howto which shows all these new features? or is this adsl bandwith management still current ? :)
my network diagram
Lan machines -> Linux Router -> Alcatel Router -> ADSL Lan Machine -> linux router -> alcatel router2 -> ADSL
Alcatel router -> Linux router
Alcatel router -> mail server etc
my modified wondershaper script is below: if anyone could point out some errors and or maybe point me to somewhere where i could learn how to do what i want better please let me know.
#!/bin/bash -x
# Wonder Shaper
# please read the README before filling out these values # # Set the following values to somewhat less than your actual download # and uplink speed. In kilobits. Also set the device that is to be shaped.
DOWNLINKQ2
UPLINK%6
DEV=eth1
DEV2=eth2
DOWNLINKlanQ2
UPLINKlan%6
DEVlan=eth1
# low priority OUTGOING traffic - you can leave this blank if you want # low priority source netmasks NOPRIOHOSTSRC # low priority destination netmasks NOPRIOHOSTDST # low priority source ports NOPRIOPORTSRC # low priority destination ports NOPRIOPORTDST
# Now remove the following two lines :-)
#echo Please read the documentation in 'README' first #exit
#pipe1
if [ "$1" = "status" ]
then
tc -s qdisc ls dev $DEV
tc -s class ls dev $DEV
exit
fi
# clean existing down- and uplink qdiscs, hide errors tc qdisc del dev $DEV root 2> /dev/null > /dev/null tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null
###### uplink
# install root HTB, point default traffic to 1:20:
tc qdisc add dev $DEV root handle 1: htb default 20
# shape everything at $UPLINK speed - this prevents huge queues in your # DSL modem which destroy latency:
tc class add dev $DEV parent 1: classid 1:1 htb rate ${UPLINK}kbit burst 6k
# high prio class 1:10:
tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${UPLINK}kbit \
burst 6k prio 1
# bulk & default class 1:20 - gets slightly less traffic, # and a lower priority:
tc class add dev $DEV parent 1:1 classid 1:20 htb rate $[9*$UPLINK/10]kbit \
burst 6k prio 2
tc class add dev $DEV parent 1:1 classid 1:30 htb rate $[8*$UPLINK/10]kbit \
burst 6k prio 2
# all get Stochastic Fairness: tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10
# TOS Minimum Delay (ssh, NOT scp) in 1:10:
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
match ip tos 0x10 0xff flowid 1:10
# ICMP (ip protocol 1) in the interactive class 1:10 so we
# can do measurements & impress our friends:
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
match ip protocol 1 0xff flowid 1:10# To speed up downloads while an upload is going on, put ACK packets in
# the interactive class:
tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u16 0x0000 0xffc0 at 2 \
match u8 0x10 0xff at 33 \
flowid 1:10
# rest is 'non-interactive' ie 'bulk' and ends up in 1:20
# some traffic however suffers a worse fate
for a in $NOPRIOPORTDST
do
tc filter add dev $DEV parent 1: protocol ip prio 14 u32 \
match ip dport $a 0xffff flowid 1:30
donefor a in $NOPRIOPORTSRC
do
tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \
match ip sport $a 0xffff flowid 1:30done
for a in $NOPRIOHOSTSRC
do
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 \
match ip src $a flowid 1:30
done
for a in $NOPRIOHOSTDST
do
tc filter add dev $DEV parent 1: protocol ip prio 17 u32 \
match ip dst $a flowid 1:30
done
# rest is 'non-interactive' ie 'bulk' and ends up in 1:20
tc filter add dev $DEV parent 1: protocol ip prio 18 u32 \
match ip dst 0.0.0.0/0 flowid 1:20
########## downlink #############
# slow downloads down to somewhat less than the real speed to prevent # queuing at our ISP. Tune to see how high you can set it. # ISPs tend to have *huge* queues to make sure big downloads are fast # # attach ingress policer:
tc qdisc add dev $DEV handle ffff: ingress
# filter *everything* to it (0.0.0.0/0), drop everything that's # coming in too fast:
tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src \
0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1
###########################################################################
#pipe2
#########################################################################
if [ "$1" = "status" ]
then
tc -s qdisc ls dev $DEV2
tc -s class ls dev $DEV2
exit
fi
# clean existing down- and uplink qdiscs, hide errors tc qdisc del dev $DEV2 root 2> /dev/null > /dev/null tc qdisc del dev $DEV2 ingress 2> /dev/null > /dev/null
if [ "$1" = "stop" ]
then
exit
fi
###### uplink
# install root HTB, point default traffic to 1:20:
tc qdisc add dev $DEV2 root handle 2: htb default 20
# shape everything at $UPLINK speed - this prevents huge queues in your # DSL modem which destroy latency:
tc class add dev $DEV2 parent 2: classid 2:1 htb rate ${UPLINK}kbit burst 6k
# high prio class 1:10:
tc class add dev $DEV2 parent 2:1 classid 2:10 htb rate ${UPLINK}kbit \
burst 6k prio 1
# bulk & default class 1:20 - gets slightly less traffic, # and a lower priority:
tc class add dev $DEV2 parent 2:1 classid 2:20 htb rate $[9*$UPLINK/10]kbit \
burst 6k prio 2
tc class add dev $DEV2 parent 2:1 classid 2:30 htb rate $[8*$UPLINK/10]kbit \
burst 6k prio 2
# all get Stochastic Fairness: tc qdisc add dev $DEV2 parent 2:10 handle 10: sfq perturb 10 tc qdisc add dev $DEV2 parent 2:20 handle 20: sfq perturb 10 tc qdisc add dev $DEV2 parent 2:30 handle 30: sfq perturb 10
# TOS Minimum Delay (ssh, NOT scp) in 1:10:
tc filter add dev $DEV2 parent 2:0 protocol ip prio 10 u32 \
match ip tos 0x10 0xff flowid 2:10
# ICMP (ip protocol 1) in the interactive class 2:10 so we
# can do measurements & impress our friends:
tc filter add dev $DEV2 parent 2:0 protocol ip prio 10 u32 \
match ip protocol 1 0xff flowid 2:10# To speed up downloads while an upload is going on, put ACK packets in
# the interactive class:
tc filter add dev $DEV2 parent 2: protocol ip prio 10 u32 \
match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u16 0x0000 0xffc0 at 2 \
match u8 0x10 0xff at 33 \
flowid 2:10
# rest is 'non-interactive' ie 'bulk' and ends up in 1:20
# some traffic however suffers a worse fate
for a in $NOPRIOPORTDST
do
tc filter add dev $DEV2 parent 2: protocol ip prio 14 u32 \
match ip dport $a 0xffff flowid 2:30
donefor a in $NOPRIOPORTSRC
do
tc filter add dev $DEV2 parent 2: protocol ip prio 15 u32 \
match ip sport $a 0xffff flowid 2:30done
for a in $NOPRIOHOSTSRC
do
tc filter add dev $DEV2 parent 2: protocol ip prio 16 u32 \
match ip src $a flowid 2:30
done
for a in $NOPRIOHOSTDST
do
tc filter add dev $DEV2 parent 2: protocol ip prio 17 u32 \
match ip dst $a flowid 2:30
done
# rest is 'non-interactive' ie 'bulk' and ends up in 2:20
tc filter add dev $DEV2 parent 2: protocol ip prio 18 u32 \
match ip dst 0.0.0.0/0 flowid 2:20
########## downlink #############
# slow downloads down to somewhat less than the real speed to prevent # queuing at our ISP. Tune to see how high you can set it. # ISPs tend to have *huge* queues to make sure big downloads are fast # # attach ingress policer:
tc qdisc add dev $DEV2 handle ffff: ingress
# filter *everything* to it (0.0.0.0/0), drop everything that's # coming in too fast:
tc filter add dev $DEV2 parent ffff: protocol ip prio 50 u32 match ip src \
0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1
cheers
anthony
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 22 lines · Anthony Letchet
Hi all, ive got 2 internet connections set up via the nano howto (which are working great) and we are running NAT.
Was looking in to qos mainly to stop large http downloads/ftp downloads from hogging the line so that browsing for other users doesnt slow to a crawl, but if the line is free and no one is doing anything then for it to use the available bandwith. The wondershaper sounded exactly what i wanted, i obviously had to customize this slightly to get my other connection to be shaped as well.
my connections are 2 adsl connections running at 512kbit:256kbit, i compiled the kernel with all the tos + iptables stuff (eventually) so that the script ran without any errors, once it had i did a download on each pipe and then tried to ping jolt.co.uk and google.co.uk, jolt.co.uk normally is about 15ms, so i was hoping that it would maybe be 200ms with my new shaping, but it was coming at about 900ms+ google was the same. Browsing other website was also to a crawl (what i was trying to avoid) but i did notice that when traffic came from our mail server through my gateway it was only receiving at about 25k/sec instead of 2mb/sec etc as its on a 100mbit switch. So something was obviously shaping, so i suppose i will now need to look in to specifying some of the hosts which are local to me but are on the outside interfaces and for them not to be shaped.. But still no better off :(
Quoted 6 lines
Ive been reading various howtos like the adsl bandwith management howto, but these havnt been updated in about 3 years and they mention various techniques which were being tried back then to help with these kind of situations, so what i really want is a howto which shows all these new features? or is this adsl bandwith management still current ? :)
I would recommend reading the masters thesis of Jesper Dangaard Brouer at http://www.adsl-optimizer.dk/thesis/
Altough he didnt release software yet (there is some code & patches in the pdf file) which could lead to the best solution, you could estimate the adsl overhead with real knowledge, and not like it says in some scripts (put x kbit less than the link bandwith, or 5% less,...)
Quoted 19 lines
my network diagram
Lan machines -> Linux Router -> Alcatel Router -> ADSL Lan Machine -> linux router -> alcatel router2 -> ADSL
Alcatel router -> Linux router
Alcatel router -> mail server etcmy modified wondershaper script is below: if anyone could point out some errors and or maybe point me to somewhere where i could learn how to do what i want better please let me know.
#!/bin/bash -x
# Wonder Shaper
Quoted 7 lines
# please read the README before filling out these values # # Set the following values to somewhat less than your actual download # and uplink speed. In kilobits. Also set the device that is to be shaped.DOWNLINKQ2
UPLINK%6
Too much. Because ATM overhead (5/53), 463/231 is the actual maximum bandwith.
And you have yet to consider the per packet overhead. Read the thesis I mentioned.
Quoted 7 lines
DEV=eth1
DEV2=eth2DOWNLINKlanQ2
UPLINKlan%6
DEVlan=eth1
Did you use the DEVlan variable?
eth1 is LAN and external at the same time?
Have you looked at http://routeskeeper.sourceforge.net/
or http://selab.edu.ms/twiki/bin/view/Networking/RoutesKeeperProject
Regards,
Francisco.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
On Tuesday 05 April 2005 18:36, Francisco Pereira wrote:
<snip>
Quoted 6 lines
I would recommend reading the masters thesis of Jesper Dangaard Brouer at http://www.adsl-optimizer.dk/thesis/
Altough he didnt release software yet (there is some code & patches in the pdf file) which could lead to the best solution, you could estimate the adsl overhead with real knowledge, and not like it says in some scripts (put x kbit less than the link bandwith, or 5% less,...)
I have been eagerly awaiting the release of that software myself.
Signature · 9 lines
Jason Boxman Perl Programmer / *NIX Systems Administrator Shimberg Center for Affordable Housing | University of Florida http://edseek.com/ - Linux and FOSS stuff _______________________________________________ LARTC mailing list [email protected] http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 11 lines · Jason Boxman
On Tuesday 05 April 2005 18:36, Francisco Pereira wrote:
<snip>Quoted 6 lines
I would recommend reading the masters thesis of Jesper Dangaard Brouer at http://www.adsl-optimizer.dk/thesis/
Altough he didnt release software yet (there is some code & patches in the pdf file) which could lead to the best solution, you could estimate the adsl overhead with real knowledge, and not like it says in some scripts (put x kbit less than the link bandwith, or 5% less,...)
Quoted 2 lines
I have been eagerly awaiting the release of that software myself.
I am testing a version at the moment based on Ed Wildgoose's - if that didn't work for you though, I can't see why this one should.
Mine is for UK pppoatm but if you know your overheads it's easy to modify - Do you? there are tables in the doc linked to.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 6 lines · Anthony Letchet
Lan machines -> Linux Router -> Alcatel Router -> ADSL Lan Machine -> linux router -> alcatel router2 -> ADSL
Alcatel router -> Linux router
Alcatel router -> mail server etc
So not all traffic goes through Linux router?
As Francisco says you need to back off from link rates - even if you do perfect calculation you can only max egress - you won't build up a queue if you are upto the limit for ingress.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 10 lines · Jason Boxman
On Tuesday 05 April 2005 18:36, Francisco Pereira wrote:
<snip>Quoted 6 lines
I would recommend reading the masters thesis of Jesper Dangaard Brouer at http://www.adsl-optimizer.dk/thesis/
Altough he didnt release software yet (there is some code & patches in the pdf file) which could lead to the best solution, you could estimate the adsl overhead with real knowledge, and not like it says in some scripts (put x kbit less than the link bandwith, or 5% less,...)I have been eagerly awaiting the release of that software myself.
Sorry, I have not released the software yet... I have been delayed by the birth of my little new daughter... have not gotten much sleep lately.
(http://www.trykdenaf.dk/gallery/silke_fodsel)
I can release the patches and a "beta" version of the scripts, if people will give me some feedback on the tar.gz distribution file and can live with too much debug information/output from the graph-module.
One of my friends are trying out the tar.gz distribution file today. He will hopefully give me some positive feedback tomorrow, wether he succesfully can follow the install instruction and have a functional system.
Hilsen
Jesper Brouer
--
-------------------------------------------------------------------
Research Assistant and Network Administrator Dept. of Computer Science, University of Copenhagen
E-mail: [email protected], Direct Tel.: 353 21438
-------------------------------------------------------------------
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
At Wed, 6 Apr 2005 17:56:13 +0200 (CEST),
hawk wrote:
Hey
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
I would recommend reading the masters thesis of Jesper Dangaard Brouer at http://www.adsl-optimizer.dk/thesis/
I have been eagerly awaiting the release of that software myself.
Sorry, I have not released the software yet... I have been delayed by the birth of my little new daughter... have not gotten much sleep lately.
(http://www.trykdenaf.dk/gallery/silke_fodsel)
Jesper have now released a version of his software.
http://www.adsl-optimizer.dk/ADSL-optimizer/
http://www.adsl-optimizer.dk/ADSL-optimizer/download/ADSL-optimizer-0.03.tar.gz
Per Marker Mortensen <[email protected]>
research assistant - distlab.dk
office N220 -- dept. of Computer Science, University of Copenhagen direct +45 35321438 -- mobile +45 20413070 -- home +45 32592041
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc