Shaping traffic over a linux bridge
Greetings,
I am using tc/cbq to do some traffic shaping over a linux bridge. My system is running the 2.4.20 kernel with the latest bridge-nf and ebtables patches. I am also running ebtables 2.0.3 and iptables 1.2.8.
Last week, since I had yet to discover ebtables/bridging I was testing the shaping with a basic NAT setup using totally seperate networks on each ethernet interface.
I now have a fully functional bridge. I was amazed to see how easy it is! I love it!
But now I am trying to add tc back into the mix. I have looked at the simple and real life examples, and a typical mark line would be something like
ebtables -A FORWARD -p ipv4 -i eth0 -j mark --set-mark 2 --mark-target CONTINUE
So now I have two questions.
How many bits are available for marking? On docum.org, I saw examples for mark values up to '5', which leads me to assume that there are at least 3 bits, meaning I can mark up to '7' (or is it '8'?). What is the numeric range here?
And what I am wanting to do is possible, right? I haven't found any real life examples for traffic shaping over a bridge yet (links appreciated). I have found this though:
"When you create a bridge with the bridge-utils, you get a new device :
br0. You can shape traffic on this device, but you can not use iptables to mark packets and the fw filter to use that mark. But you can use the u32 filter."
Does this mean I need to use some other process besides normal mangling?
What it comes down to are two issues (that are almost the same). I don't know how to mark packets coming in from the LAN, and I don't know which interface to bind tc to.
Here is a diagram of my setup:
64.119.201.0/24 ---- eth1 [bridge, br0] eth0 ---- 64.119.201.1 (router)
And here is the script I am playing with. I have tried binding tc and ebtables and iptables to all three interfaces (eth0,eth1,br0) and it doesn't seem to change anything.. but then again I don't know if my packets are even being marked. Should I be using a 2.5.x kernel instead of the patched 2.4.20? I need this for production use.. stability is important.
#!/bin/sh
OUTSIDE="eth0"
INSIDE="eth1"
BRIDGE="br0"
LAN="64.119.201.0/24"
OPTION="allot 1514 maxburst 20 avpkt 1000 prio 3"
ebtables -P INPUT ACCEPT
ebtables -P OUTPUT ACCEPT
ebtables -P FORWARD ACCEPT
ebtables -F
ebtables -t nat -F
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -F
iptables -t nat -F
echo "Loading ebtables rules"
ebtables -A FORWARD -p ipv4 -i eth1 --ip-source 64.119.201.114 -j mark --set-mark 2 --mark-target CONTINUE
ebtables -A FORWARD -p ipv4 -j mark --set-mark 1 --mark-target CONTINUE
echo "Loading iptables rules"
iptables -A PREROUTING -i eth1 -s $LAN -t mangle -j MARK --set-mark 1 iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2
###############################################################################
# i have tried $INSIDE, $OUTSIDE, and $BRIDGE here DEV="dev $INSIDE" RATE_TOTkbit SERVERSP0kbit SERVERS_WEIGHTPkbit GENPkbit GEN_WEIGHT=5kbit
echo "Clearing qdiscs"
tc qdisc del dev br0 root
tc qdisc del dev eth0 root
tc qdisc del dev eth1 root
echo "Inserting qdiscs"
tc qdisc add $DEV root handle 10: cbq bandwidth $RATE_TOT avpkt 1000 tc class add $DEV parent 10:0 classid 10:2 cbq bandwidth $RATE_TOT rate $RATE_TOT $OPTION bounded
tc qdisc add $DEV parent 10:2 handle 20: cbq bandwidth $RATE_TOT allot 1514 avpkt 1000
tc class add $DEV parent 20: classid 20:2 cbq bandwidth $RATE_TOT rate $RATE_TOT $OPTION prio 3
tc class add $DEV parent 20:2 classid 20:10 cbq bandwidth $RATE_TOT rate $SERVERS $OPTION weight $SERVERS_WEIGHT bounded
tc class add $DEV parent 20:2 classid 20:20 cbq bandwidth $RATE_TOT rate
$GEN $OPTION weight $GEN_WEIGHT bounded
echo "Adding tc filters"
tc filter add $DEV parent 10: protocol ip prio 3 handle 1 fw classid 10:2 tc filter add $DEV parent 10: protocol ip prio 3 handle 2 fw classid 10:2
tc filter add $DEV parent 20: protocol ip prio 3 handle 1 fw classid 20:20 tc filter add $DEV parent 20: protocol ip prio 3 handle 2 fw classid 20:10
echo "eth0:"
tc -s qdisc ls dev eth0
echo "eth1:"
tc -s qdisc ls dev eth1
echo "br0:"
tc -s qdisc ls dev br0
Hugh Buchanan
Userfriendly.com
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
I should add some additional comments.
I have gone through most of the LARTC archives dealing with tc.. it seems a lot of people have attempted this, but no one ever posts solutions to these things.
There are a bunch of archive posts I found somewhat helpful.
http://lists.netfilter.org/pipermail/netfilter/2002-May/034041.html
http://mailman.ds9a.nl/pipermail/lartc/2003q1/007571.html
Using those two posts, I decided to give u32 filtering a try to avoid the mangling confusion I have.. and it doesn't seem to change much.
Shaping works perfectly using NAT/MASQUERADE and iptables mangling. When I throw bridging into the mix it stops working.
I have yet to try a 2.5.x kernel.. I would prefer not to, but I suppose since I have no ideas right now on how to proceed, I better try 2.5.x.
If anyone figures this out, send me your address.. i'll send you a box of cookies or something.
Hugh
Quoted 123 lines · Hugh Buchanan
Greetings,
I am using tc/cbq to do some traffic shaping over a linux bridge. My system is running the 2.4.20 kernel with the latest bridge-nf and ebtables patches. I am also running ebtables 2.0.3 and iptables 1.2.8.
Last week, since I had yet to discover ebtables/bridging I was testing the shaping with a basic NAT setup using totally seperate networks on each ethernet interface.
I now have a fully functional bridge. I was amazed to see how easy it is! I love it!
But now I am trying to add tc back into the mix. I have looked at the simple and real life examples, and a typical mark line would be something like
ebtables -A FORWARD -p ipv4 -i eth0 -j mark --set-mark 2 --mark-target CONTINUESo now I have two questions.
How many bits are available for marking? On docum.org, I saw examples for mark values up to '5', which leads me to assume that there are at least 3 bits, meaning I can mark up to '7' (or is it '8'?). What is the numeric range here?
And what I am wanting to do is possible, right? I haven't found any real life examples for traffic shaping over a bridge yet (links appreciated). I have found this though:
"When you create a bridge with the bridge-utils, you get a new device :
br0. You can shape traffic on this device, but you can not use iptables to mark packets and the fw filter to use that mark. But you can use the u32 filter."Does this mean I need to use some other process besides normal mangling?
What it comes down to are two issues (that are almost the same). I don't know how to mark packets coming in from the LAN, and I don't know which interface to bind tc to.
Here is a diagram of my setup:
64.119.201.0/24 ---- eth1 [bridge, br0] eth0 ---- 64.119.201.1 (router)
And here is the script I am playing with. I have tried binding tc and ebtables and iptables to all three interfaces (eth0,eth1,br0) and it doesn't seem to change anything.. but then again I don't know if my packets are even being marked. Should I be using a 2.5.x kernel instead of the patched 2.4.20? I need this for production use.. stability is important.
#!/bin/sh
OUTSIDE="eth0"
INSIDE="eth1"
BRIDGE="br0"
LAN="64.119.201.0/24"
OPTION="allot 1514 maxburst 20 avpkt 1000 prio 3"ebtables -P INPUT ACCEPT
ebtables -P OUTPUT ACCEPT
ebtables -P FORWARD ACCEPT
ebtables -F
ebtables -t nat -Fiptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -F
iptables -t nat -Fecho "Loading ebtables rules"
ebtables -A FORWARD -p ipv4 -i eth1 --ip-source 64.119.201.114 -j mark --set-mark 2 --mark-target CONTINUE
ebtables -A FORWARD -p ipv4 -j mark --set-mark 1 --mark-target CONTINUEecho "Loading iptables rules"
iptables -A PREROUTING -i eth1 -s $LAN -t mangle -j MARK --set-mark 1 iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2###############################################################################
# i have tried $INSIDE, $OUTSIDE, and $BRIDGE here DEV="dev $INSIDE" RATE_TOTkbit SERVERSP0kbit SERVERS_WEIGHTPkbit GENPkbit GEN_WEIGHT=5kbitecho "Clearing qdiscs"
tc qdisc del dev br0 root
tc qdisc del dev eth0 root
tc qdisc del dev eth1 rootecho "Inserting qdiscs"
tc qdisc add $DEV root handle 10: cbq bandwidth $RATE_TOT avpkt 1000 tc class add $DEV parent 10:0 classid 10:2 cbq bandwidth $RATE_TOT rate $RATE_TOT $OPTION boundedtc qdisc add $DEV parent 10:2 handle 20: cbq bandwidth $RATE_TOT allot 1514 avpkt 1000
tc class add $DEV parent 20: classid 20:2 cbq bandwidth $RATE_TOT rate $RATE_TOT $OPTION prio 3
tc class add $DEV parent 20:2 classid 20:10 cbq bandwidth $RATE_TOT rate $SERVERS $OPTION weight $SERVERS_WEIGHT bounded
tc class add $DEV parent 20:2 classid 20:20 cbq bandwidth $RATE_TOT rate
$GEN $OPTION weight $GEN_WEIGHT boundedecho "Adding tc filters"
tc filter add $DEV parent 10: protocol ip prio 3 handle 1 fw classid 10:2 tc filter add $DEV parent 10: protocol ip prio 3 handle 2 fw classid 10:2tc filter add $DEV parent 20: protocol ip prio 3 handle 1 fw classid 20:20
tc filter add $DEV parent 20: protocol ip prio 3 handle 2 fw classid 20:10echo "eth0:"
tc -s qdisc ls dev eth0
echo "eth1:"
tc -s qdisc ls dev eth1
echo "br0:"
tc -s qdisc ls dev br0
Quoted 8 lines
Hugh Buchanan
Userfriendly.com_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
I have since fixed my problem. I am not sure if it's useful to anyone, but I'll briefly describe what I did to get it working.
In the end, I am not using the 'iptables' program at all. I am using ebtables to mangle the packets on eth1 as they come in. What I was doing before with ebtables was mostly right.. I have pretty simple ebtable lines that mark the packets. Packet marks can be 32 bits.
And my shaping is very different now too.. I shape on the outside (eth0) interface.. I can shape incoming traffic going over the eth1 interface as well, although I don't need it.
I also find that cbq doesn't work too happily.. HTB works like a charm, and really is more useful for my application. I ended up needing to use the 'tc' binary supplied by the HTB people.
I am not completely done yet.. but at the very least it works.. it is now just a matter of setting up all of the classes/rules for the 200+ MAC addresses I'll be shaping.
Hugh
Quoted 20 lines · Hugh Buchanan
I should add some additional comments.
I have gone through most of the LARTC archives dealing with tc.. it seems a lot of people have attempted this, but no one ever posts solutions to these things.
There are a bunch of archive posts I found somewhat helpful.
http://lists.netfilter.org/pipermail/netfilter/2002-May/034041.html
http://mailman.ds9a.nl/pipermail/lartc/2003q1/007571.htmlUsing those two posts, I decided to give u32 filtering a try to avoid the mangling confusion I have.. and it doesn't seem to change much.
Shaping works perfectly using NAT/MASQUERADE and iptables mangling. When I throw bridging into the mix it stops working.
I have yet to try a 2.5.x kernel.. I would prefer not to, but I suppose since I have no ideas right now on how to proceed, I better try 2.5.x.
Quoted 135 lines
If anyone figures this out, send me your address.. i'll send you a box of cookies or something.
Hugh
Quoted 127 lines · Hugh Buchanan
Greetings,
I am using tc/cbq to do some traffic shaping over a linux bridge. My system is running the 2.4.20 kernel with the latest bridge-nf and ebtables patches. I am also running ebtables 2.0.3 and iptables 1.2.8.
Last week, since I had yet to discover ebtables/bridging I was testing the shaping with a basic NAT setup using totally seperate networks on each ethernet interface.
I now have a fully functional bridge. I was amazed to see how easy it is! I love it!
But now I am trying to add tc back into the mix. I have looked at the simple and real life examples, and a typical mark line would be something like
ebtables -A FORWARD -p ipv4 -i eth0 -j mark --set-mark 2 --mark-target CONTINUESo now I have two questions.
How many bits are available for marking? On docum.org, I saw examples for mark values up to '5', which leads me to assume that there are at least 3 bits, meaning I can mark up to '7' (or is it '8'?). What is the numeric range here?
And what I am wanting to do is possible, right? I haven't found any real life examples for traffic shaping over a bridge yet (links appreciated). I have found this though:
"When you create a bridge with the bridge-utils, you get a new device :
br0. You can shape traffic on this device, but you can not use iptables to mark packets and the fw filter to use that mark. But you can use the u32 filter."Does this mean I need to use some other process besides normal mangling?
What it comes down to are two issues (that are almost the same). I don't know how to mark packets coming in from the LAN, and I don't know which interface to bind tc to.
Here is a diagram of my setup:
64.119.201.0/24 ---- eth1 [bridge, br0] eth0 ---- 64.119.201.1 (router)
And here is the script I am playing with. I have tried binding tc and ebtables and iptables to all three interfaces (eth0,eth1,br0) and it doesn't seem to change anything.. but then again I don't know if my packets are even being marked. Should I be using a 2.5.x kernel instead of the patched 2.4.20? I need this for production use.. stability is important.
#!/bin/sh
OUTSIDE="eth0"
INSIDE="eth1"
BRIDGE="br0"
LAN="64.119.201.0/24"
OPTION="allot 1514 maxburst 20 avpkt 1000 prio 3"ebtables -P INPUT ACCEPT
ebtables -P OUTPUT ACCEPT
ebtables -P FORWARD ACCEPT
ebtables -F
ebtables -t nat -Fiptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -F
iptables -t nat -Fecho "Loading ebtables rules"
ebtables -A FORWARD -p ipv4 -i eth1 --ip-source 64.119.201.114 -j mark --set-mark 2 --mark-target CONTINUE
ebtables -A FORWARD -p ipv4 -j mark --set-mark 1 --mark-target CONTINUEecho "Loading iptables rules"
iptables -A PREROUTING -i eth1 -s $LAN -t mangle -j MARK --set-mark 1 iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2###############################################################################
# i have tried $INSIDE, $OUTSIDE, and $BRIDGE here DEV="dev $INSIDE" RATE_TOTkbit SERVERSP0kbit SERVERS_WEIGHTPkbit GENPkbit GEN_WEIGHT=5kbitecho "Clearing qdiscs"
tc qdisc del dev br0 root
tc qdisc del dev eth0 root
tc qdisc del dev eth1 rootecho "Inserting qdiscs"
tc qdisc add $DEV root handle 10: cbq bandwidth $RATE_TOT avpkt 1000 tc class add $DEV parent 10:0 classid 10:2 cbq bandwidth $RATE_TOT rate $RATE_TOT $OPTION boundedtc qdisc add $DEV parent 10:2 handle 20: cbq bandwidth $RATE_TOT allot 1514 avpkt 1000
tc class add $DEV parent 20: classid 20:2 cbq bandwidth $RATE_TOT rate $RATE_TOT $OPTION prio 3
tc class add $DEV parent 20:2 classid 20:10 cbq bandwidth $RATE_TOT rate $SERVERS $OPTION weight $SERVERS_WEIGHT bounded
tc class add $DEV parent 20:2 classid 20:20 cbq bandwidth $RATE_TOT rate
$GEN $OPTION weight $GEN_WEIGHT boundedecho "Adding tc filters"
tc filter add $DEV parent 10: protocol ip prio 3 handle 1 fw classid 10:2
tc filter add $DEV parent 10: protocol ip prio 3 handle 2 fw classid 10:2tc filter add $DEV parent 20: protocol ip prio 3 handle 1 fw classid 20:20
tc filter add $DEV parent 20: protocol ip prio 3 handle 2 fw classid 20:10echo "eth0:"
tc -s qdisc ls dev eth0
echo "eth1:"
tc -s qdisc ls dev eth1
echo "br0:"
tc -s qdisc ls dev br0Hugh Buchanan
Userfriendly.com
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/