tc and CONNMARK
Hi!
I'm building a firewall/traffic shaper using tc and connmark but I note a strange behaviour.
Configuration:
eth0 - lan interface - rate 600kbit/s connection with mark X (download) eth1 - wan interface - rate 600kbit/s connection with mark X (upload)
Starting a single connection (upload or download) traffic is limited correctly but if I start two connection (one in upload and the other in download) I note that the one in upload runs correctly (how in the previous test) but the one in download stalls and then send some packets and then stalls alternatively.
I don't understand why!
always (and only) the download connection has this behaviour.
two connection have the same mark (in netfilter table), is it a problem for the shaper?
also I think about ack packets but they have a very short size (66 bytes) so I believe they are not the cause for this problem.
thanks in advance,
Fabio Marcone
A message that is not in the archive (sent off-list or not yet received): [email protected]
Thanks for your reply,
Quoted 3 lines
A first idea, without saw your rules, it's that you limit the wrong side of communication, however no one can help you without understand better your environment.
it's not possible because if I test only download flow or upload flow, limit rate is ok.
Quoted 1 line
Post your tc/iptables rules and commands.
I'm developing a firewall/router with some services and configuration it's not so easy to understand.
Quoted 2 lines
P.s. Use two different mark for different update/download traffic
I'm using connmark so both flow of connection are marked with same mark (CONNMARK) but shaper rule are set on different interfaces:
mark 10 on eth0 -> classid 1:6
mark 10 on eth1 -> classid 2:6
where classes are build with these commands:
tc class add dev eth0 parent 1:1 classid 1:6 htb rate 600kbit ceil 600kbit burst 6k cburst 64k quantum 1600 tc class add dev eth0 parent 2:1 classid 2:6 htb rate 600kbit ceil 600kbit burst 6k cburst 64k quantum 1600
and filters are:
tc filter add dev eth0 parent 1: protocol ip handle 10 fw flowid 1:6 tc filter add dev eth1 parent 2: protocol ip handle 10 fw flowid 2:6
do you think is a connmark problem?
Fabio Marcone ha scritto:
Quoted 5 lines
Thanks for your reply,
Quoted 3 lines
A first idea, without saw your rules, it's that you limit the wrong
side of communication, however no one can help you without
understand better your environment.
Quoted 7 lines
it's not possible because if I test only download flow or upload flow, limit rate is ok.
Quoted 1 line
Post your tc/iptables rules and commands.
I'm developing a firewall/router with some services and configuration
it's not so easy to understand.
Ok, but at least an environment where we can test or understand it's need!
Quoted 4 lines
Quoted 2 lines
P.s. Use two different mark for different update/download traffic
I'm using connmark so both flow of connection are marked with same mark (CONNMARK) but shaper rule are set on different interfaces: mark
10 on eth0 -> classid 1:6 mark 10 on eth1 -> classid 2:6
For me it's better if you use different marks.
After, where do you mark the packets (in iptables)? iptables/kernel version?
Quoted 6 lines
where classes are build with these commands: tc class add dev eth0 parent 1:1 classid 1:6 htb rate 600kbit ceil 600kbit burst 6k cburst 64k quantum 1600 tc class add dev eth0 parent 2:1 classid 2:6 htb rate 600kbit ceil 600kbit burst 6k cburst 64k quantum 1600
and filters are: tc filter add dev eth0 parent 1: protocol ip handle 10 fw flowid 1:6 tc filter add dev eth1 parent 2: protocol ip handle 10 fw flowid 2:6
do you think is a connmark problem?
Normally I don't use connmark because when I try some time ago to use it, I found some "not marked" problems, so I switch to classid. Better and cleaner for me.
Michele
hello,
Quoted 1 line
For me it's better if you use different marks.
I note that packets are marked correctly but in the wan interface tc filter send them on the wrong class. is the same problem you had it?
Quoted 3 lines
After, where do you mark the packets (in iptables)? iptables/kernel version?
I mark packets in iptables (mangle table PREROUTING chain):
iptables -t mangle -A PREROUTING -p TCP -m mac --mac-source xx:xx:xx:xx:xx:xx--dport 443 -j MARK --set-mark 8
iptables version: 1.4.3.2
kernel version: 2.6.29.3
both patched to use IMQ devices.
Quoted 4 lines
Normally I don't use connmark because when I try some time ago to use it, I found some "not marked" problems, so I switch to classid. Better and cleaner for me.
what kind of problems? What do you mean with "I switch to classid" ?
thanks in advance,
Fabio
A message that is not in the archive (sent off-list or not yet received): [email protected]
Just realised I'd sent this direct not via list... must remember to check address lines before sending in future.
2009/10/6 Richard Horton <[email protected]>:
Quoted 54 lines
2009/10/6 Fabio Marcone <[email protected]>:
Quoted 2 lines
what kind of problems? What do you mean with "I switch to classid" ?
Classid in this context I would assume to mean using the diff serv code point field (was the ToS field) in the IP Header and making queuing decisions based on the diffserv value rather than using marking in the packet structure.
If you use Classid there are two routes you can go do;
Route 1 uses 'class selectors' and provides 8 levels of discrimination. Route 2 goes down the full diffserv path and provides 3 levels of traffic - Expedited forwards, Assured Forwarding and 'Best' Effort (which is anything but best).
The EF class is designed for real-time jitter (variation of delay) sensitive traffic, AF is designed for loss-intolerant traffic and provides 4 levels of forwarding each with 3 levels of drop precedence and BE is well... whatever is left over.
Using the AFxy (x class, y drop precedence) you can define some complex schemes with each class having a guaranteed bandwidth with packets being dropped on the basis of their drop precedence...
For instance to mark rtp and give it decent bandwidth/priority and sip slightly lower...
(Using port 20000/20001 for rtp, 5060 for sip)
iptables -A INPUT -t mangle -p udp -dport 5060 -j DSCP --set-dscp-class AF41 iptables -A INPUT -t mangle -p udp -dport 20000 -j DSCP --set-dscp-class EF iptables -A INPUT -t mangle -p udp -dport 20001 -j DSCP --set-dscp class EFtc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 classid 1:10 parent 1: htb rate 10mbps ceil 10mbps prio 10 tc class add dev eth0 classid 1:20 parent 1: htb rate 30mbps ceil 40mbps prio 5 tc class add dev eth0 classid 1:30 parent 1: htb rate 60mbps ceil 100mbps prio 0tc filter add dev eth0 parent 1:0 protocol ip prio 10 u32 \
match ip tos0x880xffflowid 1:20
tc filter add dev eth0 parent 1:0 protocol ip prio 10 u32 \
match ip tos0xB80xffflowid 1:30--
Richard Horton
Users are like a virus: Each causing a thousand tiny crises until the host finally dies.
http://www.solstans.co.uk - Solstans Japanese Bobtails and Norwegian Forest Cats
http://www.pbase.com/arimus - My online photogallery
Signature · 5 lines
Richard Horton Users are like a virus: Each causing a thousand tiny crises until the host finally dies. http://www.solstans.co.uk - Solstans Japanese Bobtails and Norwegian Forest Cats http://www.pbase.com/arimus - My online photogallery
Fabio Marcone ha scritto:
Quoted 6 lines
hello,
Quoted 1 line
For me it's better if you use different marks.
I note that packets are marked correctly but in the wan interface tc filter send them on the wrong class. is the same problem you had it?
If I remember correctly, yes.
Quoted 2 lines
iptables -t mangle -A PREROUTING -p TCP -m mac --mac-source xx:xx:xx:xx:xx:xx--dport 443 -j MARK --set-mark 8
Why not forward or postrouting? Do you choose prerouting for use the mac addrs?
Quoted 3 lines
iptables version: 1.4.3.2 kernel version: 2.6.29.3
both patched to use IMQ devices.
Haven't you said this!
In _all_ my installations, I always skip to use imq. I don't know why, but it's simple to use that are already included inside the vanilla kernel
Quoted 4 lines
Quoted 3 lines
Normally I don't use connmark because when I try some time ago to use it, I found some "not marked" problems, so I switch to classid.
Better and cleaner for me.what kind of problems?
ip filter don't match my data. But it's true that I didn't lost a lot of time following that solution because... I find CLASSIFY
Quoted 2 lines
What do you mean with "I switch to classid" ?
It's a wrong definition, sorry. The right one it CLASSIFY!
Simple example, assuming that you have 192.168.1.0/24, need to limit at 5mb the ip .100 on both sides on flow and all the others goes at 1mbit all together. eth0 lan, eth1 are wan.
tc qdisc add dev eth0 root handle 1: htb default 1000
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit tc class add dev eth0 parent 1:1 classid 1:100 htb rate 1mbit ceil 5mbit tc class add dev eth0 parent 1:1 classid 1:1000 htb rate 1mbit ceil 1mbit
tc qdisc add dev eth1 root handle 2: htb default 1000
tc class add dev eth1 parent 2: classid 2:1 htb rate 100mbit ceil 100mbit tc class add dev eth1 parent 2:1 classid 2:100 htb rate 1mbit ceil 5mbit tc class add dev eth1 parent 2:1 classid 2:1000 htb rate 1mbit ceil 1mbit
iptables -t mangle -F FORWARD
iptables -t mangle -A FORWARD -o eth0 -d 192.168.1.100 -j CLASSIFY --set-class 1:100
iptables -t mangle -A FORWARD -o eth1 -s 192.168.1.100 -j CLASSIFY --set-class 5:100
#not need since class [12]:1000 are already a fetch-all for the unclassified iptables -t mangle -A OUTPUT -o eth0 -j CLASSIFY --set-class 1:1000 iptables -t mangle -A OUTPUT -o eth1 -j CLASSIFY --set-class 5:1000
That's all.
Of course, a rule reorganization it's a must.
Michele
Hello,
Quoted 5 lines
Quoted 3 lines
iptables -t mangle -A PREROUTING -p TCP -m mac --mac-source xx:xx:xx:xx:xx:xx--dport 443 -j MARK --set-mark 8
Why not forward or postrouting? Do you choose prerouting for use the mac addrs?
yes, I recognize workstation by macaddress and workstations are divided into groups. Different groups can use different rates related at the same IP/protocol/port. So, to limit traffic in upload I need to know if a packet is related to a connection or an another one.
Quoted 8 lines
Quoted 4 lines
iptables version: 1.4.3.2 kernel version: 2.6.29.3
both patched to use IMQ devices.
Haven't you said this!
In _all_ my installations, I always skip to use imq. I don't know why, but it's simple to use that are already included inside the vanilla kernel
I use IMQ for services exported by router (like http, smtp, ecc ecc) so I can limit upload and download traffic of a particular service available on a particular interface.
Quoted 28 lines
Quoted 5 lines
Quoted 4 lines
Normally I don't use connmark because when I try some time ago to use it, I found some "not marked" problems, so I switch to classid.
Better and cleaner for me.what kind of problems?
ip filter don't match my data. But it's true that I didn't lost a lot of time following that solution because... I find CLASSIFY
Quoted 2 lines
What do you mean with "I switch to classid" ?
It's a wrong definition, sorry. The right one it CLASSIFY!
Simple example, assuming that you have 192.168.1.0/24, need to limit at 5mb the ip .100 on both sides on flow and all the others goes at 1mbit all together. eth0 lan, eth1 are wan.
tc qdisc add dev eth0 root handle 1: htb default 1000
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit tc class add dev eth0 parent 1:1 classid 1:100 htb rate 1mbit ceil 5mbit tc class add dev eth0 parent 1:1 classid 1:1000 htb rate 1mbit ceil 1mbittc qdisc add dev eth1 root handle 2: htb default 1000
tc class add dev eth1 parent 2: classid 2:1 htb rate 100mbit ceil 100mbit tc class add dev eth1 parent 2:1 classid 2:100 htb rate 1mbit ceil 5mbit tc class add dev eth1 parent 2:1 classid 2:1000 htb rate 1mbit ceil 1mbit
Quoted 8 lines
iptables -t mangle -F FORWARD
iptables -t mangle -A FORWARD -o eth0 -d 192.168.1.100 -j CLASSIFY --set-class 1:100
iptables -t mangle -A FORWARD -o eth1 -s 192.168.1.100 -j CLASSIFY --set-class 5:100
#not need since class [12]:1000 are already a fetch-all for the unclassified iptables -t mangle -A OUTPUT -o eth0 -j CLASSIFY --set-class 1:1000 iptables -t mangle -A OUTPUT -o eth1 -j CLASSIFY --set-class 5:1000
Thanks a lot I didn't known CLASSIFY.
Fabio
Re: tc and CONNMARK [SOLVED]
In reply to Fabio MarconeThanks to all for your help.
Quoted 2 lines
Starting a single connection (upload or download) traffic is limited correctly but if I start two connection (one in upload and the other in download) I note that the one in upload runs correctly (how in the previous test) but the one in download stalls and then send some packets and then stalls alternatively.
I solve using sfq to htb leaves, in this way this two flows share the bandwidth correctly.
Fabio