Skip to contentSkip to conversation

Packet loss with htb+sfq+l7filter

LARTC Eduardo Fernández 4 participants 6 messages Nov 15, 2004 – Nov 16, 2004

lore
Share & tools
lore
#0 lore raw

Hi all!

I'm trying to shape traffic in a dorm's network (4 mbit symmetrical internet link, about 200 computers, heavy p2p usage). The router is a p4xeon running linux 2.6.9 with the qnet patches
(http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get

Quoted 1 line

20% packet loss and a lot of duplicates. Any ideas? I attach my shaping

script.

Thank you very much in advance,

Eduardo

router:~# cat htb.new
#!/bin/sh
IFOUT=eth0
IFIN=eth2
# cleaning
tc qdisc del dev $IFOUT root   &>/dev/null
tc qdisc del dev $IFOUT ingress &>/dev/null

# link's capacity
CEILP0

# 1:10 interactive traffic with the highest prio (dns, ssh...) # 1:20 interactive traffic with lower prio (radios, vcn, x11...) # 1:30 bulk (http, ftp, cvs...)
# 1:40 the rest (p2p mostly)
tc qdisc add dev $IFOUT root handle 1: htb default 40
tc class add dev $IFOUT parent 1: classid 1:1 htb rate ${CEIL}kbps ceil ${CEIL}kbps
tc class add dev $IFOUT parent 1:1 classid 1:10 htb rate 80kbps ceil 80kbps prio 0
tc class add dev $IFOUT parent 1:1 classid 1:20 htb rate 10kbps ceil 100kbps prio 1
tc class add dev $IFOUT parent 1:1 classid 1:30 htb rate 400kbps ceil ${CEIL}kbps prio 2
tc class add dev $IFOUT parent 1:1 classid 1:40 htb rate 10kbps ceil ${CEIL}kbps prio 3

tc qdisc add dev $IFOUT parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:40 handle 40: sfq perturb 10

tc filter add dev $IFOUT parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev $IFOUT parent 1:0 protocol ip prio 2 handle 2 fw classid 1:20
tc filter add dev $IFOUT parent 1:0 protocol ip prio 3 handle 3 fw classid 1:30
tc filter add dev $IFOUT parent 1:0 protocol ip prio 4 handle 4 fw classid 1:40

NF="/usr/local/sbin/iptables -t mangle -A PREROUTING " NFl7="/usr/local/sbin/iptables -t mangle -A POSTROUTING -m layer7 "

/usr/local/sbin/iptables -F -t mangle

# by tos
$NF -p icmp -j MARK --set-mark 0x1
$NF -p icmp -j RETURN
$NF -m tos --tos 0x8 -j MARK --set-mark 0x3 #scp
$NF -m tos --tos 0x8 -j RETURN
$NF -m tos --tos Minimize-Cost -j MARK --set-mark 0x1
$NF -m tos --tos Minimize-Cost -j RETURN
$NF -m tos --tos Maximize-Throughput -j MARK --set-mark 0x4
$NF -m tos --tos Maximize-Throughput -j RETURN
$NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1
$NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN
# by layer 7
$NFl7 --l7proto http -j MARK --set-mark 0x3
$NFl7 --l7proto http -j RETURN
$NFl7 --l7proto ftp -j MARK --set-mark 0x3
$NFl7 --l7proto ftp -j RETURN
$NFl7 --l7proto skype -j MARK --set-mark 0x2
$NFl7 --l7proto skype -j RETURN
$NFl7 --l7proto msnmessenger -j MARK --set-mark 0x1
$NFl7 --l7proto msnmessenger -j RETURN
$NFl7 --l7proto msn-filetransfer -j MARK --set-mark 0x3
$NFl7 --l7proto msn-filetransfer -j RETURN
$NFl7 --l7proto jabber -j MARK --set-mark 0x1
$NFl7 --l7proto jabber -j RETURN
$NFl7 --l7proto smtp -j MARK --set-mark 0x3
$NFl7 --l7proto smtp -j RETURN
$NFl7 --l7proto pop3 -j MARK --set-mark 0x3
$NFl7 --l7proto pop3 -j RETURN
$NFl7 --l7proto ssh -j MARK --set-mark 0x2
$NFl7 --l7proto ssh -j RETURN
$NFl7 --l7proto dns -j MARK --set-mark 0x1
$NFl7 --l7proto dns -j RETURN
$NFl7 --l7proto telnet -j MARK --set-mark 0x2
$NFl7 --l7proto telnet -j RETURN
$NFl7 --l7proto cvs -j MARK --set-mark 0x3
$NFl7 --l7proto cvs -j RETURN
$NFl7 --l7proto irc -j MARK --set-mark 0x1
$NFl7 --l7proto irc -j RETURN
$NFl7 --l7proto yahoo -j MARK --set-mark 0x1
$NFl7 --l7proto yahoo -j RETURN
$NFl7 --l7proto counterstrike -j MARK --set-mark 0x1
$NFl7 --l7proto counterstrike -j RETURN
$NFl7 --l7proto rstp -j MARK --set-mark 0x2
$NFl7 --l7proto rstp -j RETURN
$NFl7 --l7proto live365 -j MARK --set-mark 0x2
$NFl7 --l7proto live365 -j RETURN
$NFl7 --l7proto h323 -j MARK --set-mark 0x2
$NFl7 --l7proto h323 -j RETURN
$NFl7 --l7proto quake-halflife -j MARK --set-mark 0x1
$NFl7 --l7proto quake-halflife -j RETURN
$NFl7 --l7proto rdp -j MARK --set-mark 0x2
$NFl7 --l7proto rdp -j RETURN
$NFl7 --l7proto rlogin -j MARK --set-mark 0x2
$NFl7 --l7proto rlogin -j RETURN
$NFl7 --l7proto shoutcast -j MARK --set-mark 0x2
$NFl7 --l7proto shoutcast -j RETURN
$NFl7 --l7proto validcertssl -j MARK --set-mark 0x3
$NFl7 --l7proto validcertssl -j RETURN
$NFl7 --l7proto vnc -j MARK --set-mark 0x2
$NFl7 --l7proto vnc -j RETURN
$NFl7 --l7proto x11 -j MARK --set-mark 0x2
$NFl7 --l7proto x11 -j RETURN
# end

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw

Ypu said ypu've got 4Mbit. As far as I'm concerned 4Mbit= 1024*4@96 So link capacity should be 4000kbit (rate and ceil of base class) Am I wrong?
----- Original Message -----
From: "Eduardo Fernández" <[email protected]>
To: <[email protected]>
Sent: Monday, November 15, 2004 3:52 PM
Subject: [LARTC] Packet loss with htb+sfq+l7filter

Quoted 125 lines

Hi all!

I'm trying to shape traffic in a dorm's network (4 mbit symmetrical internet link, about 200 computers, heavy p2p usage). The router is a p4xeon running linux 2.6.9 with the qnet patches
(http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get

Quoted 1 line

20% packet loss and a lot of duplicates. Any ideas? I attach my shaping

script.

Thank you very much in advance,

Eduardo

router:~# cat htb.new
#!/bin/sh
IFOUT=eth0
IFIN=eth2
# cleaning
tc qdisc del dev $IFOUT root   &>/dev/null
tc qdisc del dev $IFOUT ingress &>/dev/null

# link's capacity
CEILP0

# 1:10 interactive traffic with the highest prio (dns, ssh...) # 1:20 interactive traffic with lower prio (radios, vcn, x11...) # 1:30 bulk (http, ftp, cvs...)
# 1:40 the rest (p2p mostly)
tc qdisc add dev $IFOUT root handle 1: htb default 40
tc class add dev $IFOUT parent 1: classid 1:1 htb rate ${CEIL}kbps ceil ${CEIL}kbps
tc class add dev $IFOUT parent 1:1 classid 1:10 htb rate 80kbps ceil 80kbps prio 0
tc class add dev $IFOUT parent 1:1 classid 1:20 htb rate 10kbps ceil 100kbps prio 1
tc class add dev $IFOUT parent 1:1 classid 1:30 htb rate 400kbps ceil ${CEIL}kbps prio 2
tc class add dev $IFOUT parent 1:1 classid 1:40 htb rate 10kbps ceil ${CEIL}kbps prio 3

tc qdisc add dev $IFOUT parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:40 handle 40: sfq perturb 10

tc filter add dev $IFOUT parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev $IFOUT parent 1:0 protocol ip prio 2 handle 2 fw classid 1:20
tc filter add dev $IFOUT parent 1:0 protocol ip prio 3 handle 3 fw classid 1:30
tc filter add dev $IFOUT parent 1:0 protocol ip prio 4 handle 4 fw classid 1:40

NF="/usr/local/sbin/iptables -t mangle -A PREROUTING " NFl7="/usr/local/sbin/iptables -t mangle -A POSTROUTING -m layer7 "

/usr/local/sbin/iptables -F -t mangle

# by tos
$NF -p icmp -j MARK --set-mark 0x1
$NF -p icmp -j RETURN
$NF -m tos --tos 0x8 -j MARK --set-mark 0x3 #scp
$NF -m tos --tos 0x8 -j RETURN
$NF -m tos --tos Minimize-Cost -j MARK --set-mark 0x1
$NF -m tos --tos Minimize-Cost -j RETURN
$NF -m tos --tos Maximize-Throughput -j MARK --set-mark 0x4
$NF -m tos --tos Maximize-Throughput -j RETURN
$NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1
$NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN
# by layer 7
$NFl7 --l7proto http -j MARK --set-mark 0x3
$NFl7 --l7proto http -j RETURN
$NFl7 --l7proto ftp -j MARK --set-mark 0x3
$NFl7 --l7proto ftp -j RETURN
$NFl7 --l7proto skype -j MARK --set-mark 0x2
$NFl7 --l7proto skype -j RETURN
$NFl7 --l7proto msnmessenger -j MARK --set-mark 0x1
$NFl7 --l7proto msnmessenger -j RETURN
$NFl7 --l7proto msn-filetransfer -j MARK --set-mark 0x3
$NFl7 --l7proto msn-filetransfer -j RETURN
$NFl7 --l7proto jabber -j MARK --set-mark 0x1
$NFl7 --l7proto jabber -j RETURN
$NFl7 --l7proto smtp -j MARK --set-mark 0x3
$NFl7 --l7proto smtp -j RETURN
$NFl7 --l7proto pop3 -j MARK --set-mark 0x3
$NFl7 --l7proto pop3 -j RETURN
$NFl7 --l7proto ssh -j MARK --set-mark 0x2
$NFl7 --l7proto ssh -j RETURN
$NFl7 --l7proto dns -j MARK --set-mark 0x1
$NFl7 --l7proto dns -j RETURN
$NFl7 --l7proto telnet -j MARK --set-mark 0x2
$NFl7 --l7proto telnet -j RETURN
$NFl7 --l7proto cvs -j MARK --set-mark 0x3
$NFl7 --l7proto cvs -j RETURN
$NFl7 --l7proto irc -j MARK --set-mark 0x1
$NFl7 --l7proto irc -j RETURN
$NFl7 --l7proto yahoo -j MARK --set-mark 0x1
$NFl7 --l7proto yahoo -j RETURN
$NFl7 --l7proto counterstrike -j MARK --set-mark 0x1
$NFl7 --l7proto counterstrike -j RETURN
$NFl7 --l7proto rstp -j MARK --set-mark 0x2
$NFl7 --l7proto rstp -j RETURN
$NFl7 --l7proto live365 -j MARK --set-mark 0x2
$NFl7 --l7proto live365 -j RETURN
$NFl7 --l7proto h323 -j MARK --set-mark 0x2
$NFl7 --l7proto h323 -j RETURN
$NFl7 --l7proto quake-halflife -j MARK --set-mark 0x1
$NFl7 --l7proto quake-halflife -j RETURN
$NFl7 --l7proto rdp -j MARK --set-mark 0x2
$NFl7 --l7proto rdp -j RETURN
$NFl7 --l7proto rlogin -j MARK --set-mark 0x2
$NFl7 --l7proto rlogin -j RETURN
$NFl7 --l7proto shoutcast -j MARK --set-mark 0x2
$NFl7 --l7proto shoutcast -j RETURN
$NFl7 --l7proto validcertssl -j MARK --set-mark 0x3
$NFl7 --l7proto validcertssl -j RETURN
$NFl7 --l7proto vnc -j MARK --set-mark 0x2
$NFl7 --l7proto vnc -j RETURN
$NFl7 --l7proto x11 -j MARK --set-mark 0x2
$NFl7 --l7proto x11 -j RETURN
# end
Quoted 3 lines

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#2 lore raw
In reply to Eduardo Fernández

Yeah, 4 mbit, that is, 512 kbps. Notice the command line:

CEILP0
(...) rate ${CEIL}kbps ceil ${CEIL}kbps

                  ^^^^             ^^^^
Quoted 3 lines · mjoachimiak

Ypu said ypu've got 4Mbit. As far as I'm concerned 4Mbit= 1024*4@96 So link capacity should be 4000kbit (rate and ceil of base class) Am I wrong?

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#3 lore raw
In reply to Eduardo Fernández

Hi,

Quoted 2 lines · Eduardo Fernández

(http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get

Quoted 1 line

20% packet loss and a lot of duplicates. Any ideas? I attach my shaping

well we've done something similar but instead of using a route we used a bridge (acting as traffic shaper and firewall). So you don't need ip_forward maybe this is worth a try in your setup? In general a box like your should easily handle much more traffic that 4+4MBit/s.

Regards,
Eicke

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#4 lore raw
In reply to Eduardo Fernández

Hi Eicke,

But setting up a bridge there'll be another machine routing, so I'll need ip_forward either way, and I won't avoid the problem if it's really ip_forward, I will only move it to another machine.

Regards,

Eduardo

Quoted 8 lines · Eicke Friedrich

Hi,

Quoted 2 lines · Eduardo Fernández

(http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get

Quoted 1 line

20% packet loss and a lot of duplicates. Any ideas? I attach my shaping

well we've done something similar but instead of using a route we used a bridge (acting as traffic shaper and firewall). So you don't need ip_forward maybe this is worth a try in your setup? In general a box like your should easily handle much more traffic that 4+4MBit/s.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#5 lore raw
In reply to Eduardo Fernández
Quoted 5 lines · Eduardo Fernández

Yeah, 4 mbit, that is, 512 kbps. Notice the command line:

CEILP0
(...) rate ${CEIL}kbps ceil ${CEIL}kbps

                  ^^^^             ^^^^

It's still a bit too close to link speed - uplink allow for overheads - downlink you need to be below link speed to have a queue - its fill rate depending on the % of link used.

How are you measuring packet loss?

Andy.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/