Questions about IMQ
from http://luxik.cdi.cz/~patrick/imq/index.html
By default you have two imq devices (imq0 and imq1).
That means if you don't specify numdevs below the default is 2, right?
from http://luxik.cdi.cz/~patrick/imq/index.html
modprobe imq numdevs=1
tc qdisc add dev imq0 handle 1: root htb default 1 tc class add dev imq0 parent 1: classid 1:1 htb rate 1mbit tc qdisc add dev imq0 parent 1:1 handle 10: htb default 5
We're adding an htb as the qdisc for a child class of htb ? Why? Isn't that just wasting time? Can't all 10: stuff be done with 1:
instead?
So above the total rate for all imq traffic is limited to 1Mbit. As far as I see, it all goes out 1:1, and below we shape that.
tc class add dev imq0 parent 10: handle 10:1 htb rate 256kbit burst 30k prio 1 tc class add dev imq0 parent 10: handle 10:2 htb rate 256kbit burst 30k prio 2 tc class add dev imq0 parent 10: handle 10:5 htb rate 1mbit prio 3
tc qdisc add dev imq0 parent 10:1 handle 21:0 pfifo tc qdisc add dev imq0 parent 10:2 handle 22:0 sfq tc qdisc add dev imq0 parent 10:3 handle 23:0 sfq
Should this 10:3 be 10:5 ?
tc filter add dev imq0 protocol ip pref 1 parent 10: handle 1 fw classid 10:1 tc filter add dev imq0 protocol ip pref 2 parent 10: handle 2 fw classid 10:2
iptables -t mangle -A PREROUTING -i ppp0 -j IMQ
This is a little confusing. I gather -j IMQ means to mark the packet so that AFTER the mangle table is done, the IMQ device will steal the packet. Is that right? If there were more than one imq device, how would we know which one gets the packet?
iptables -t mangle -A PREROUTING -i ppp0 -p tcp -m tos --tos minimize-delay -m state --state ESTABLISHED -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i ppp0 -p tcp --sport 80 --dport 1024: -m state --state ESTABLISHED -j MARK --set-mark 2
ip link set imq0 upSo here's what I imagine happens. Please confirm or correct. The packet goes through mangle table and maybe gets marked for later classification and maybe gets marked for imq. Then the imq hook steals away those that were marked for imq. It enqueues them and dequeues them according to its classes and the marks. At this point the skb dev is imq0 ? Or still ppp0 ?
When the packet is eventually dequeued (if not dropped) then it goes where? I'm hoping it goes to the beginning of pre-routing so we can apply conntrack/nat/mangle rules to it with -i imq0.
I suspect this is not the case, since I see in the patch code
nf_reinject(skb, info, NF_ACCEPT)
I'm not even sure netfilter supports what I want.
I see in
http://netfilter.samba.org/documentation/HOWTO//netfilter-hacking-HOWTO-3.html
5.NF_REPEAT: call this hook again.
but what's "this hook" ? Is it the imq hook or pre_routing ?
My goal here is to protect conntrack from attack by rate limiting the packets not from known connections. To do that I need to send them to imq before conntrack sees them. Unfortunately, conntrack does all the work that I want to avoid for those packets in prerouting, and conntrack sees them before mangle. But maybe I could restrict all operations that involve conntrack to packets with dev imq (or the imq mark), which I hope would result in conntrack NOT seeing the packets with other devices. I could instead send all those packets from other devices (or without the mark) to imq, where I would look them up in the conntrack table (but not add them!) to see whether they belong in the rate limited class or not. Then when (if) they're released they should go through conntrack, nat, mangle, etc.
Perhaps even better than changing the dev to imq0 would be a way for netfilter rules to match on the imq mark. Then I wouldn't have to worry about whether rp_filter would still work.
from http://luxik.cdi.cz/~patrick/imq/faq.html
4. When do packets reach the device (qdisc) ?
The imq device registers NF_IP_PRE_ROUTING (for ingress) and NF_IP_POST_ROUTING (egress) netfilter hooks. These hooks are also registered by iptables. Hooks can be registered with different priorities which determine the order in which the registered functions will be called. Packet delivery to the imq device in NF_IP_PRE_ROUTING happens directly after the mangle table has been passed (not in the table itself!). In NF_IP_POST_ROUTING packets reach the device after ALL tables have been passed. This means you will be able to use netfilter marks for classifying incoming and outgoing packets. Packets seen in NF_IP_PRE_ROUTING include the ones that will be dropped by packet filtering later (since they already occupied bandwidth), in NF_IP_POST_ROUTING only packets which already passed packet filtering are seen.
from include/linux/netfilter_ipv4.h
enum nf_ip_hook_priorities {
NF_IP_PRI_FIRST = INT_MIN,
NF_IP_PRI_CONNTRACK = -200,
NF_IP_PRI_MANGLE = -150,
NF_IP_PRI_NAT_DST = -100,
NF_IP_PRI_FILTER = 0,
NF_IP_PRI_NAT_SRC = 100,
NF_IP_PRI_LAST = INT_MAX,};
So after mangle means first conntrack, then mangle, then IMQ, then ...
It might be worth mentioning this somewhere in doc.
One other thing I worry about.net/ipv4/netfilter/ip_queue.c contains:
* Packets arrive here from netfilter for queuing to userspace.
* All of them must be fed back via nf_reinject() or Alexey will kill Rusty. */
static int netfilter_receive(struct sk_buff *skb,
I notice the patch returning NF_QUEUE.
Will Rusty survive if IMQ ends up dropping packets ?
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi.
Alot question, i hope i can answer them to your satisfaction.
Quoted 6 lines · Don Cohen
from http://luxik.cdi.cz/~patrick/imq/index.html
By default you have two imq devices (imq0 and imq1)That means if you don't specify numdevs below the default is 2, right?
right.
Quoted 11 lines
from http://luxik.cdi.cz/~patrick/imq/index.html
modprobe imq numdevs=1
tc qdisc add dev imq0 handle 1: root htb default 1 tc class add dev imq0 parent 1: classid 1:1 htb rate 1mbit tc qdisc add dev imq0 parent 1:1 handle 10: htb default 5We're adding an htb as the qdisc for a child class of htb ? Why? Isn't that just wasting time? Can't all 10: stuff be done with 1:
instead?
The root qdisc is used for delay simulation, 10:0 is the "real" qdisc ( http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm#prio )
Quoted 7 lines
So above the total rate for all imq traffic is limited to 1Mbit. As far as I see, it all goes out 1:1, and below we shape that.
tc class add dev imq0 parent 10: handle 10:1 htb rate 256kbit burst 30k prio 1 tc class add dev imq0 parent 10: handle 10:2 htb rate 256kbit burst 30k prio 2 tc class add dev imq0 parent 10: handle 10:5 htb rate 1mbit prio 3
Quoted 6 lines
tc qdisc add dev imq0 parent 10:1 handle 21:0 pfifo tc qdisc add dev imq0 parent 10:2 handle 22:0 sfq tc qdisc add dev imq0 parent 10:3 handle 23:0 sfq
Should this 10:3 be 10:5 ?
Yes you're right. Someone else already told me but i forgot to correct it.
Quoted 5 lines
tc filter add dev imq0 protocol ip pref 1 parent 10: handle 1 fw classid 10:1 tc filter add dev imq0 protocol ip pref 2 parent 10: handle 2 fw classid 10:2
Quoted 7 lines
iptables -t mangle -A PREROUTING -i ppp0 -j IMQ
This is a little confusing. I gather -j IMQ means to mark the packet so that AFTER the mangle table is done, the IMQ device will steal the packet. Is that right? If there were more than one imq device, how would we know which one gets the packet?
I put it in this order on purpose because it thought it would show people the imq device does not get to see the packet
during mangle table traversal but afterwards. It probably IS confusing so i'm going to change it the next days.
If more than one imq device is used you specify the one which should get the packet with --todev argument to IMQ target.
Quoted 7 lines
iptables -t mangle -A PREROUTING -i ppp0 -p tcp -m tos --tos minimize-delay -m state --state ESTABLISHED -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i ppp0 -p tcp --sport 80 --dport 1024: -m state --state ESTABLISHED -j MARK --set-mark 2
ip link set imq0 upQuoted 7 lines
So here's what I imagine happens. Please confirm or correct. The packet goes through mangle table and maybe gets marked for later classification and maybe gets marked for imq. Then the imq hook steals away those that were marked for imq. It enqueues them and dequeues them according to its classes and the marks. At this point the skb dev is imq0 ? Or still ppp0 ?
skb->dev doesn't get changed if thats what you mean ..
Quoted 5 lines
When the packet is eventually dequeued (if not dropped) then it goes where? I'm hoping it goes to the beginning of pre-routing so we can apply conntrack/nat/mangle rules to it with -i imq0.
No it doesn't. I think it doesn't make any sense to use any kind of iptables rules on packets
passing imq because all of them come from/go to real devices which you can use in your
rules.
Quoted 9 lines
I suspect this is not the case, since I see in the patch code
nf_reinject(skb, info, NF_ACCEPT)
I'm not even sure netfilter supports what I want.
I see in
http://netfilter.samba.org/documentation/HOWTO//netfilter-hacking-HOWTO-3.html
5.NF_REPEAT: call this hook again.
but what's "this hook" ? Is it the imq hook orpre_routing?
it's imq hook. from net/core/netfilter.c:nf_reinject(...)
{
...
if (verdict = NF_REPEAT) {
elem = elem->prev;
verdict = NF_ACCEPT;
}...
}
Quoted 8 lines
My goal here is to protect conntrack from attack by rate limiting the packets not from known connections. To do that I need to send them to imq before conntrack sees them. Unfortunately, conntrack does all the work that I want to avoid for those packets in prerouting, and conntrack sees them before mangle. But maybe I could restrict all
you can easily change this order. i guess you already noticed if you looked at the imq source.
but are you sure this is necessary ? i guess your connection must be extremly fast if someone
wants to dos you through a connection tracking table fillup attack ...
Quoted 10 lines
operations that involve conntrack to packets with dev imq (or the imq mark), which I hope would result in conntrack NOT seeing the packets with other devices. I could instead send all those packets from other devices (or without the mark) to imq, where I would look them up in the conntrack table (but not add them!) to see whether they belong in the rate limited class or not. Then when (if) they're released they should go through conntrack, nat, mangle, etc.
Quoted 4 lines
Perhaps even better than changing the dev to imq0 would be a way for netfilter rules to match on the imq mark. Then I wouldn't have to worry about whether
rp_filterwould still work.
Changing skb->dev to imq0 would result in something like this:
... -> NF_HOOK(..) -> imq -> qdisc -> reinject -> continue NF_HOOK -> ... -> dev_queue_xmit -> qdisc -> imq -> reinject (CRASH!)
Quoted 31 lines
from http://luxik.cdi.cz/~patrick/imq/faq.html
4. When do packets reach the device (qdisc) ?
The imq device registers
NF_IP_PRE_ROUTING(for ingress) andNF_IP_POST_ROUTING(egress) netfilter hooks. These hooks are also registered by iptables. Hooks can be registered with different priorities which determine the order in which the registered functions will be called. Packet delivery to the imq device inNF_IP_PRE_ROUTINGhappens directly after the mangle table has been passed (not in the table itself!). InNF_IP_POST_ROUTINGpackets reach the device after ALL tables have been passed. This means you will be able to use netfilter marks for classifying incoming and outgoing packets. Packets seen inNF_IP_PRE_ROUTINGinclude the ones that will be dropped by packet filtering later (since they already occupied bandwidth), inNF_IP_POST_ROUTINGonly packets which already passed packet filtering are seen.from
include/linux/netfilter_ipv4.henum nf_ip_hook_priorities { NF_IP_PRI_FIRST = INT_MIN, NF_IP_PRI_CONNTRACK = -200, NF_IP_PRI_MANGLE = -150, NF_IP_PRI_NAT_DST = -100, NF_IP_PRI_FILTER = 0, NF_IP_PRI_NAT_SRC = 100, NF_IP_PRI_LAST = INT_MAX, };
Quoted 4 lines
So after mangle means first conntrack, then mangle, then IMQ, then ...
It might be worth mentioning this somewhere in doc.
hmm i guess i may explanation is the worst way to describe this simple fact :)
(its supposed to mean the same thing)
Quoted 9 lines
One other thing I worry about.
net/ipv4/netfilter/ip_queue.ccontains:
* Packets arrive here from netfilter for queuing to userspace.
* All of them must be fed back vianf_reinject()or Alexey will kill Rusty. */
static int netfilter_receive(structsk_buff*skb,
Quoted 3 lines
I notice the patch returning
NF_QUEUE.
Will Rusty survive if IMQ ends up dropping packets ?
If you look at the imq source you find a imq_skb_destructor, i though about adding a comment that
it's meant to save rusty's life. if skb's are freed inside qdiscs kfree_skb will call the destructor which
will do necessary things to protect rusty :)
Bye,
Patrick
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 6 lines · Patrick McHardy
Quoted 4 lines
We're adding an htb as the qdisc for a child class of htb ? Why? Isn't that just wasting time? Can't all 10: stuff be done with 1:
instead?The root qdisc is used for delay simulation, 10:0 is the "real" qdisc ( http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm#prio )
So I was right, just to waste time. That was not part of the spec, as I recall. So I suggest it be removed from the example.
Quoted 2 lines
If more than one imq device is used you specify the one which should get the packet with --todev argument to IMQ target.
Be sure to put that in the doc. I didn't see it there. I suppose the default is imq0 ?
Quoted 1 line
skb->devdoesn't get changed if thats what you mean ..
Ok, important to know that. I gather there is currently no way to read the imq mark from netfilter.
Quoted 8 lines
Quoted 4 lines
When the packet is eventually dequeued (if not dropped) then it goes where? I'm hoping it goes to the beginning of pre-routing so we can apply conntrack/nat/mangle rules to it with -i imq0.
No it doesn't. I think it doesn't make any sense to use any kind of iptables rules on packets
passing imq because all of them come from/go to real devices which you can use in your rules.
But if you could read the imq mark then it would make a lot of sense. These two things in combination would allow me to do what I want without changing the code. As it is, it looks like I need a local variant of IMQ that runs before conntrack. (On the other hand, this is probably the more efficient solution anyhow.)
Quoted 10 lines
Quoted 8 lines
I suspect this is not the case, since I see in the patch code
nf_reinject(skb, info, NF_ACCEPT)
I'm not even sure netfilter supports what I want.
I see in
http://netfilter.samba.org/documentation/HOWTO//netfilter-hacking-HOWTO-3.html
5.NF_REPEAT: call this hook again.
but what's "this hook" ? Is it the imq hook orpre_routing?it's imq hook. from
net/core/netfilter.c:nf_reinject(...)
As I thought, there's no convenient way for you to do what I want.
Quoted 2 lines
you can easily change this order. i guess you already noticed if you looked at the imq source.
Right. But this is not a change that everyone would want.
Quoted 3 lines
but are you sure this is necessary ? i guess your connection must be extremly fast if someone
wants to dos you through a connection tracking table fillup attack ...
My idea of extremely fast has changed recently. Maybe it's a bit ahead of yours. First, I'm interested in protecting against attacks from inside the firewall, and these are typically connected at 100Mbit. Is that fast enough? Next I've been playing with gigabit cards. Finally I visited sprint a few weeks ago and they're not interested in anything as slow as one gigabit. Although, for a firewall, I admit that seems fast enough for the time being.
Quoted 3 lines
Changing
skb->devto imq0 would result in something like this:
... ->NF_HOOK(..)-> imq -> qdisc -> reinject -> continueNF_HOOK-> ... ->dev_queue_xmit-> qdisc -> imq -> reinject (CRASH!)
If you mean it could result in infinite loops, yes, but this is not the first invention of infinite loops. If your rules do the right things then the loops can also be avoided. Besides, that requires my other request, that the reinject go back to the beginning of the prerouting hook. Without that it was completely plausible that the skb dev could have been changed. But I'm not complaining. I just wanted to know.
Quoted 5 lines
If you look at the imq source you find a
imq_skb_destructor, i though about adding a comment that
it's meant to save rusty's life. if skb's are freed inside qdiscskfree_skbwill call the destructor which
will do necessary things to protect rusty :)
Ok, I wouldn't want to contribute to his early demise. This tends to confirm my first guess, which was that the important thing here was to free skbs when they are no longer in use. I guess user mode can't free them, but perhaps the better solution would have been to free them before a copy is sent to user space and then recreating them if the copy ever came back. But I digress...
Thanks for all the answers.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 11 lines · Don Cohen
Quoted 6 lines · Patrick McHardy
Quoted 4 lines
We're adding an htb as the qdisc for a child class of htb ? Why? Isn't that just wasting time? Can't all 10: stuff be done with 1:
instead?The root qdisc is used for delay simulation, 10:0 is the "real" qdisc ( http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm#prio )
So I was right, just to waste time. That was not part of the spec, as I recall. So I suggest it be removed from the example.
I don't know if its really necessary, but as imq is a software device it probably is .. maybe devik
can answer this ...
Quoted 8 lines
Quoted 4 lines
If more than one imq device is used you specify the one which should get the packet with --todev argument to IMQ target.
Be sure to put that in the doc. I didn't see it there. I suppose the default is imq0 ?
yes ..
Quoted 7 lines
Quoted 3 lines
skb->devdoesn't get changed if thats what you mean ..Ok, important to know that. I gather there is currently no way to read the imq mark from netfilter.
currently not, but if you need something like this just change the mark match to match skb->imq_flags
instead of skb->nfmark ... you should then look at include/linux/imq.h to see the meaning of the different bits.
Quoted 11 lines
Quoted 10 lines
Quoted 6 lines
When the packet is eventually dequeued (if not dropped) then it goes where? I'm hoping it goes to the beginning of pre-routing so we can apply conntrack/nat/mangle rules to it with -i imq0.
No it doesn't. I think it doesn't make any sense to use any kind of iptables rules on packets
passing imq because all of them come from/go to real devices which you can use in your rules.
Quoted 6 lines
But if you could read the imq mark then it would make a lot of sense. These two things in combination would allow me to do what I want without changing the code. As it is, it looks like I need a local variant of IMQ that runs before conntrack. (On the other hand, this is probably the more efficient solution anyhow.)
more efficient maybe, but you will loose the possibility to only donate bandwidth to established and assured
connections by using the state match ..
Quoted 49 lines
Quoted 10 lines
Quoted 10 lines
I suspect this is not the case, since I see in the patch code
nf_reinject(skb, info, NF_ACCEPT)
I'm not even sure netfilter supports what I want.
I see in
http://netfilter.samba.org/documentation/HOWTO//netfilter-hacking-HOWTO-3.html
5.NF_REPEAT: call this hook again.
but what's "this hook" ? Is it the imq hook orpre_routing?Quoted 2 lines
it's imq hook. from
net/core/netfilter.c:nf_reinject(...)As I thought, there's no convenient way for you to do what I want.
Quoted 2 lines
you can easily change this order. i guess you already noticed if you looked at the imq source.
Right. But this is not a change that everyone would want.
Quoted 3 lines
but are you sure this is necessary ? i guess your connection must be extremly fast if someone
wants to dos you through a connection tracking table fillup attack ...My idea of extremely fast has changed recently. Maybe it's a bit ahead of yours. First, I'm interested in protecting against attacks from inside the firewall, and these are typically connected at 100Mbit. Is that fast enough? Next I've been playing with gigabit cards. Finally I visited sprint a few weeks ago and they're not interested in anything as slow as one gigabit. Although, for a firewall, I admit that seems fast enough for the time being.
Quoted 3 lines
Changing
skb->devto imq0 would result in something like this:
... ->NF_HOOK(..)-> imq -> qdisc -> reinject -> continueNF_HOOK-> ... ->dev_queue_xmit-> qdisc -> imq -> reinject (CRASH!)If you mean it could result in infinite loops, yes, but this is not the first invention of infinite loops. If your rules do the right things then the loops can also be avoided. Besides, that requires my other request, that the reinject go back to the beginning of the prerouting hook. Without that it was completely plausible that the skb dev could have been changed. But I'm not complaining. I just wanted to know.
Quoted 5 lines
If you look at the imq source you find a
imq_skb_destructor, i though about adding a comment that
it's meant to save rusty's life. if skb's are freed inside qdiscskfree_skbwill call the destructor which
will do necessary things to protect rusty :)
Quoted 7 lines
Ok, I wouldn't want to contribute to his early demise. This tends to confirm my first guess, which was that the important thing here was to free skbs when they are no longer in use. I guess user mode can't free them, but perhaps the better solution would have been to free them before a copy is sent to user space and then recreating them if the copy ever came back. But I digress...
userspace ?? imq never sends anything to userspace, but if it really would then you're right, userspace can't free skbs.
also the destructor doesn't free them but it releases references hold on the real devices which are taken before by netfilter
so the real device doesn't vanish while the packet is out of netfilters control ..
Quoted 4 lines
Thanks for all the answers.
Your welcome,
Patrick
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/