Packet loss with htb+sfq+l7filter
Hi all!
I'm trying to shape traffic in a dorm's network (4 mbit symmetrical internet link, about 200 computers, heavy p2p usage). The router is a p4xeon running linux 2.6.9 with the qnet patches
(http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get
Quoted 1 line
20% packet loss and a lot of duplicates. Any ideas? I attach my shaping
script.
Thank you very much in advance,
Eduardo
router:~# cat htb.new #!/bin/sh IFOUT=eth0 IFIN=eth2
# cleaning tc qdisc del dev $IFOUT root &>/dev/null tc qdisc del dev $IFOUT ingress &>/dev/null
# link's capacity
CEILP0
# 1:10 interactive traffic with the highest prio (dns, ssh...) # 1:20 interactive traffic with lower prio (radios, vcn, x11...) # 1:30 bulk (http, ftp, cvs...)
# 1:40 the rest (p2p mostly)
tc qdisc add dev $IFOUT root handle 1: htb default 40
tc class add dev $IFOUT parent 1: classid 1:1 htb rate ${CEIL}kbps ceil ${CEIL}kbps
tc class add dev $IFOUT parent 1:1 classid 1:10 htb rate 80kbps ceil 80kbps prio 0
tc class add dev $IFOUT parent 1:1 classid 1:20 htb rate 10kbps ceil 100kbps prio 1
tc class add dev $IFOUT parent 1:1 classid 1:30 htb rate 400kbps ceil ${CEIL}kbps prio 2
tc class add dev $IFOUT parent 1:1 classid 1:40 htb rate 10kbps ceil ${CEIL}kbps prio 3
tc qdisc add dev $IFOUT parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:40 handle 40: sfq perturb 10
tc filter add dev $IFOUT parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev $IFOUT parent 1:0 protocol ip prio 2 handle 2 fw classid 1:20
tc filter add dev $IFOUT parent 1:0 protocol ip prio 3 handle 3 fw classid 1:30
tc filter add dev $IFOUT parent 1:0 protocol ip prio 4 handle 4 fw classid 1:40
NF="/usr/local/sbin/iptables -t mangle -A PREROUTING " NFl7="/usr/local/sbin/iptables -t mangle -A POSTROUTING -m layer7 "
/usr/local/sbin/iptables -F -t mangle
# by tos $NF -p icmp -j MARK --set-mark 0x1 $NF -p icmp -j RETURN $NF -m tos --tos 0x8 -j MARK --set-mark 0x3 #scp $NF -m tos --tos 0x8 -j RETURN $NF -m tos --tos Minimize-Cost -j MARK --set-mark 0x1 $NF -m tos --tos Minimize-Cost -j RETURN $NF -m tos --tos Maximize-Throughput -j MARK --set-mark 0x4 $NF -m tos --tos Maximize-Throughput -j RETURN $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1 $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN
# by layer 7 $NFl7 --l7proto http -j MARK --set-mark 0x3 $NFl7 --l7proto http -j RETURN $NFl7 --l7proto ftp -j MARK --set-mark 0x3 $NFl7 --l7proto ftp -j RETURN $NFl7 --l7proto skype -j MARK --set-mark 0x2 $NFl7 --l7proto skype -j RETURN $NFl7 --l7proto msnmessenger -j MARK --set-mark 0x1 $NFl7 --l7proto msnmessenger -j RETURN $NFl7 --l7proto msn-filetransfer -j MARK --set-mark 0x3 $NFl7 --l7proto msn-filetransfer -j RETURN $NFl7 --l7proto jabber -j MARK --set-mark 0x1 $NFl7 --l7proto jabber -j RETURN $NFl7 --l7proto smtp -j MARK --set-mark 0x3 $NFl7 --l7proto smtp -j RETURN $NFl7 --l7proto pop3 -j MARK --set-mark 0x3 $NFl7 --l7proto pop3 -j RETURN $NFl7 --l7proto ssh -j MARK --set-mark 0x2 $NFl7 --l7proto ssh -j RETURN $NFl7 --l7proto dns -j MARK --set-mark 0x1 $NFl7 --l7proto dns -j RETURN $NFl7 --l7proto telnet -j MARK --set-mark 0x2 $NFl7 --l7proto telnet -j RETURN $NFl7 --l7proto cvs -j MARK --set-mark 0x3 $NFl7 --l7proto cvs -j RETURN $NFl7 --l7proto irc -j MARK --set-mark 0x1 $NFl7 --l7proto irc -j RETURN $NFl7 --l7proto yahoo -j MARK --set-mark 0x1 $NFl7 --l7proto yahoo -j RETURN $NFl7 --l7proto counterstrike -j MARK --set-mark 0x1 $NFl7 --l7proto counterstrike -j RETURN $NFl7 --l7proto rstp -j MARK --set-mark 0x2 $NFl7 --l7proto rstp -j RETURN $NFl7 --l7proto live365 -j MARK --set-mark 0x2 $NFl7 --l7proto live365 -j RETURN $NFl7 --l7proto h323 -j MARK --set-mark 0x2 $NFl7 --l7proto h323 -j RETURN $NFl7 --l7proto quake-halflife -j MARK --set-mark 0x1 $NFl7 --l7proto quake-halflife -j RETURN $NFl7 --l7proto rdp -j MARK --set-mark 0x2 $NFl7 --l7proto rdp -j RETURN $NFl7 --l7proto rlogin -j MARK --set-mark 0x2 $NFl7 --l7proto rlogin -j RETURN $NFl7 --l7proto shoutcast -j MARK --set-mark 0x2 $NFl7 --l7proto shoutcast -j RETURN $NFl7 --l7proto validcertssl -j MARK --set-mark 0x3 $NFl7 --l7proto validcertssl -j RETURN $NFl7 --l7proto vnc -j MARK --set-mark 0x2 $NFl7 --l7proto vnc -j RETURN $NFl7 --l7proto x11 -j MARK --set-mark 0x2 $NFl7 --l7proto x11 -j RETURN # end
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Ypu said ypu've got 4Mbit. As far as I'm concerned 4Mbit= 1024*4@96 So link capacity should be 4000kbit (rate and ceil of base class) Am I wrong?
----- Original Message -----
From: "Eduardo Fernández" <[email protected]>
To: <[email protected]>
Sent: Monday, November 15, 2004 3:52 PM
Subject: [LARTC] Packet loss with htb+sfq+l7filter
Quoted 125 lines
Hi all!
I'm trying to shape traffic in a dorm's network (4 mbit symmetrical internet link, about 200 computers, heavy p2p usage). The router is a p4xeon running linux 2.6.9 with the qnet patches
(http://kem.p.lodz.pl/~peter/qnet/). When I activateip_forwardI getQuoted 1 line
20% packet loss and a lot of duplicates. Any ideas? I attach my shaping
script.
Thank you very much in advance,
Eduardo
router:~# cat htb.new #!/bin/sh IFOUT=eth0 IFIN=eth2# cleaning tc qdisc del dev $IFOUT root &>/dev/null tc qdisc del dev $IFOUT ingress &>/dev/null# link's capacity
CEILP0# 1:10 interactive traffic with the highest prio (dns, ssh...) # 1:20 interactive traffic with lower prio (radios, vcn, x11...) # 1:30 bulk (http, ftp, cvs...)
# 1:40 the rest (p2p mostly)
tc qdisc add dev $IFOUT root handle 1: htb default 40
tc class add dev $IFOUT parent 1: classid 1:1 htb rate ${CEIL}kbps ceil ${CEIL}kbps
tc class add dev $IFOUT parent 1:1 classid 1:10 htb rate 80kbps ceil 80kbps prio 0
tc class add dev $IFOUT parent 1:1 classid 1:20 htb rate 10kbps ceil 100kbps prio 1
tc class add dev $IFOUT parent 1:1 classid 1:30 htb rate 400kbps ceil ${CEIL}kbps prio 2
tc class add dev $IFOUT parent 1:1 classid 1:40 htb rate 10kbps ceil ${CEIL}kbps prio 3tc qdisc add dev $IFOUT parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:40 handle 40: sfq perturb 10
tc filter add dev $IFOUT parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev $IFOUT parent 1:0 protocol ip prio 2 handle 2 fw classid 1:20
tc filter add dev $IFOUT parent 1:0 protocol ip prio 3 handle 3 fw classid 1:30
tc filter add dev $IFOUT parent 1:0 protocol ip prio 4 handle 4 fw classid 1:40NF="/usr/local/sbin/iptables -t mangle -A PREROUTING " NFl7="/usr/local/sbin/iptables -t mangle -A POSTROUTING -m layer7 "
/usr/local/sbin/iptables-F -t mangle# by tos $NF -p icmp -j MARK --set-mark 0x1 $NF -p icmp -j RETURN $NF -m tos --tos 0x8 -j MARK --set-mark 0x3 #scp $NF -m tos --tos 0x8 -j RETURN $NF -m tos --tos Minimize-Cost -j MARK --set-mark 0x1 $NF -m tos --tos Minimize-Cost -j RETURN $NF -m tos --tos Maximize-Throughput -j MARK --set-mark 0x4 $NF -m tos --tos Maximize-Throughput -j RETURN $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1 $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN# by layer 7 $NFl7 --l7proto http -j MARK --set-mark 0x3 $NFl7 --l7proto http -j RETURN $NFl7 --l7proto ftp -j MARK --set-mark 0x3 $NFl7 --l7proto ftp -j RETURN $NFl7 --l7proto skype -j MARK --set-mark 0x2 $NFl7 --l7proto skype -j RETURN $NFl7 --l7proto msnmessenger -j MARK --set-mark 0x1 $NFl7 --l7proto msnmessenger -j RETURN $NFl7 --l7proto msn-filetransfer -j MARK --set-mark 0x3 $NFl7 --l7proto msn-filetransfer -j RETURN $NFl7 --l7proto jabber -j MARK --set-mark 0x1 $NFl7 --l7proto jabber -j RETURN $NFl7 --l7proto smtp -j MARK --set-mark 0x3 $NFl7 --l7proto smtp -j RETURN $NFl7 --l7proto pop3 -j MARK --set-mark 0x3 $NFl7 --l7proto pop3 -j RETURN $NFl7 --l7proto ssh -j MARK --set-mark 0x2 $NFl7 --l7proto ssh -j RETURN $NFl7 --l7proto dns -j MARK --set-mark 0x1 $NFl7 --l7proto dns -j RETURN $NFl7 --l7proto telnet -j MARK --set-mark 0x2 $NFl7 --l7proto telnet -j RETURN $NFl7 --l7proto cvs -j MARK --set-mark 0x3 $NFl7 --l7proto cvs -j RETURN $NFl7 --l7proto irc -j MARK --set-mark 0x1 $NFl7 --l7proto irc -j RETURN $NFl7 --l7proto yahoo -j MARK --set-mark 0x1 $NFl7 --l7proto yahoo -j RETURN $NFl7 --l7proto counterstrike -j MARK --set-mark 0x1 $NFl7 --l7proto counterstrike -j RETURN $NFl7 --l7proto rstp -j MARK --set-mark 0x2 $NFl7 --l7proto rstp -j RETURN $NFl7 --l7proto live365 -j MARK --set-mark 0x2 $NFl7 --l7proto live365 -j RETURN $NFl7 --l7proto h323 -j MARK --set-mark 0x2 $NFl7 --l7proto h323 -j RETURN $NFl7 --l7proto quake-halflife -j MARK --set-mark 0x1 $NFl7 --l7proto quake-halflife -j RETURN $NFl7 --l7proto rdp -j MARK --set-mark 0x2 $NFl7 --l7proto rdp -j RETURN $NFl7 --l7proto rlogin -j MARK --set-mark 0x2 $NFl7 --l7proto rlogin -j RETURN $NFl7 --l7proto shoutcast -j MARK --set-mark 0x2 $NFl7 --l7proto shoutcast -j RETURN $NFl7 --l7proto validcertssl -j MARK --set-mark 0x3 $NFl7 --l7proto validcertssl -j RETURN $NFl7 --l7proto vnc -j MARK --set-mark 0x2 $NFl7 --l7proto vnc -j RETURN $NFl7 --l7proto x11 -j MARK --set-mark 0x2 $NFl7 --l7proto x11 -j RETURN # end
Quoted 3 lines
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Yeah, 4 mbit, that is, 512 kbps. Notice the command line:
CEILP0
(...) rate ${CEIL}kbps ceil ${CEIL}kbps
^^^^ ^^^^
Quoted 3 lines · mjoachimiak
Ypu said ypu've got 4Mbit. As far as I'm concerned 4Mbit= 1024*4@96 So link capacity should be 4000kbit (rate and ceil of base class) Am I wrong?
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi,
Quoted 2 lines · Eduardo Fernández
(http://kem.p.lodz.pl/~peter/qnet/). When I activate
ip_forwardI getQuoted 1 line
20% packet loss and a lot of duplicates. Any ideas? I attach my shaping
well we've done something similar but instead of using a route we used a bridge (acting as traffic shaper and firewall). So you don't need ip_forward maybe this is worth a try in your setup? In general a box like your should easily handle much more traffic that 4+4MBit/s.
Regards,
Eicke
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi Eicke,
But setting up a bridge there'll be another machine routing, so I'll need ip_forward either way, and I won't avoid the problem if it's really ip_forward, I will only move it to another machine.
Regards,
Eduardo
Quoted 8 lines · Eicke Friedrich
Hi,
Quoted 2 lines · Eduardo Fernández
(http://kem.p.lodz.pl/~peter/qnet/). When I activate
ip_forwardI getQuoted 1 line
20% packet loss and a lot of duplicates. Any ideas? I attach my shaping
well we've done something similar but instead of using a route we used a bridge (acting as traffic shaper and firewall). So you don't need
ip_forwardmaybe this is worth a try in your setup? In general a box like your should easily handle much more traffic that 4+4MBit/s.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 5 lines · Eduardo Fernández
Yeah, 4 mbit, that is, 512 kbps. Notice the command line:
CEILP0
(...) rate ${CEIL}kbps ceil ${CEIL}kbps^^^^ ^^^^
It's still a bit too close to link speed - uplink allow for overheads - downlink you need to be below link speed to have a queue - its fill rate depending on the % of link used.
How are you measuring packet loss?
Andy.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/