nwfilter: probe for inverted ctdir
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
For reference of people new to this thread, here is the start of the thread:
https://www.redhat.com/archives/libvir-list/2013-March/msg01403.html
This concerns changes to libvirt to cope with the newly discovered (by us :-) difference in interpretation of ctdir by different versions of netfilter.
Quoted 37 lines · Stefan Berger
Quoted 19 lines · Stefan Berger
Quoted 9 lines · Eric Blake
Quoted 8 lines · Laine Stump
My opinion is that the patch we should apply should be a simple patch that just removes use of --ctdir. According to the netfilter developer who responded to the thread on libvirt-users, it doesn't add any extra security not already provided by conntrack:
https://www.redhat.com/archives/libvirt-users/2013-March/msg00121.html
https://www.redhat.com/archives/libvirt-users/2013-March/msg00128.htmlQuoted 6 lines
Quoted 3 lines
Not being an expert on netfilter internals, I can't dispute his claim.
Does anyone else have an opinion?
What filters specifically caused the use of --ctdir, and are they broken
if we omit the use of --ctdir?It depends on how you write the filters that the --ctdir is being used.
Quoted 10 lines
iirc: The effect of the --ctdir usage is that if one has an incoming rule and and outgoing rule with the same IP address on the 'other' side the check for an ESTABLISHED state is not enough to ACCEPT the traffic, if one was to remove one of the rules while communication in both directions was occurring and an immediate cut of the traffic in one way was expected. The effect so far was that if the rule for the incoming rule was removed it would cut the incoming traffic immediately while the traffic in outgoing direction was uninterrupted. I think that if we remove this now the traffic in both directions will continue. I will verify tomorrow.
Verified. I have a ping running from the VM to destination 'A' and from 'A' to the VM. The --ctdir enforces the direction of the traffic and if one of the following rules is removed, the ping is immediately cut.
<rule action='accept' direction='out' priority='500'>
Quoted 8 lines
<icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/></rule>
The ping is not cut anymore upon removal of one of the above rules if --ctdir was to be removed entirely.
Okay, as I understand from your description, the difference is that when a ping in one direction is already in action, and you remove the rule allowing that ping, that existing ping "session" will continue to be allowed *if* there is still a rule allowing pings in the other direction. Is that correct? I'm guessing that *new* attempts to ping in that direction will no longer be allowed though, is that also correct?
For the benefit of Pablo and the other netfilter developers, can you paste the iptables commands that are generated for the two rules above? Possibly they can suggest alternative rules that have the desired effect.
If they have no alternatives, then I do now agree that we shouldn't just scrap --ctdir. Then we have two choices:
1) take your patch, which hopefully will successfully guess the polarity of --ctdir correctly in all cases.
2) switch unconditionally to the new "correct" polarity of --ctdir, release-note the heck out of it, and require that any distro with a kernel old enough to have the old style --ctdir backport at least the one patch to netfilter to change that.
My comments about (1): while I'll again say that the patch truly is poetic in its ability to overcome obstacles, it's really a workaround for a bug, and as time goes on will become less and less relevant (and more and more difficult to explain/rationalize). I really think I would prefer to be broken on very old distros rather than have that patch in the tree (although others may have a different opinion, and I would gracefully withdraw my objection in that case).
About (2): Fedora at least as far back as F16 has a new enough kernel/iptables (iptables-1.4.12) that it uses the new polarity for --ctdir, and no older Fedora is supported. RHEL6 and CentOS6 still have iptables-1.4.7, so they have the old polarity. I'd be willing to bet that RHEL6 would take the netfilter patch to change --ctdir; of course that would leave us with temporary brokenness for anybody who upgraded their kernel without upgrading libvirt, or vice-versa.
(That temporary breakage has me a bit concerned, actually, and may mandate a patch like yours, at least applied downstream-only for RHEL/CentOS builds of libvirt.)
Can maintainers for other distros comment on the version of iptables their distros have in all releases still being supported? It would be good to know just who would be broken by changing this (and who would be fixed, of course)
Quoted 59 lines · Laine Stump
For reference of people new to this thread, here is the start of the thread:
https://www.redhat.com/archives/libvir-list/2013-March/msg01403.htmlThis concerns changes to libvirt to cope with the newly discovered (by us :-) difference in interpretation of ctdir by different versions of netfilter.
Quoted 41 lines · Stefan Berger
Quoted 18 lines · Stefan Berger
Quoted 9 lines · Eric Blake
Quoted 8 lines · Laine Stump
My opinion is that the patch we should apply should be a simple patch that just removes use of --ctdir. According to the netfilter developer who responded to the thread on libvirt-users, it doesn't add any extra security not already provided by conntrack:
https://www.redhat.com/archives/libvirt-users/2013-March/msg00121.html
https://www.redhat.com/archives/libvirt-users/2013-March/msg00128.htmlQuoted 6 lines
Quoted 3 lines
Not being an expert on netfilter internals, I can't dispute his claim.
Does anyone else have an opinion?
What filters specifically caused the use of --ctdir, and are they broken
if we omit the use of --ctdir?It depends on how you write the filters that the --ctdir is being used.
Quoted 10 lines
iirc: The effect of the --ctdir usage is that if one has an incoming rule and and outgoing rule with the same IP address on the 'other' side the check for an ESTABLISHED state is not enough to ACCEPT the traffic, if one was to remove one of the rules while communication in both directions was occurring and an immediate cut of the traffic in one way was expected. The effect so far was that if the rule for the incoming rule was removed it would cut the incoming traffic immediately while the traffic in outgoing direction was uninterrupted. I think that if we remove this now the traffic in both directions will continue. I will verify tomorrow.
Verified. I have a ping running from the VM to destination 'A' and from 'A' to the VM. The --ctdir enforces the direction of the traffic and if one of the following rules is removed, the ping is immediately cut.
<rule action='accept' direction='out' priority='500'> <icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule>Quoted 2 lines
The ping is not cut anymore upon removal of one of the above rules if --ctdir was to be removed entirely.
Okay, as I understand from your description, the difference is that when a ping in one direction is already in action, and you remove the rule allowing that ping, that existing ping "session" will continue to be allowed *if* there is still a rule allowing pings in the other direction. Is that correct? I'm guessing that *new* attempts to ping in that direction will no longer be allowed though, is that also correct?
Quoted 3 lines
For the benefit of Pablo and the other netfilter developers, can you paste the iptables commands that are generated for the two rules above? Possibly they can suggest alternative rules that have the desired effect.
First off, there are multiple ways one can write the filtering rules in nwfilter, either stateless or stateful:
http://libvirt.org/formatnwfilter.html#nwfwrite
Thus the filter here is only one example how one can write a stateful filter for traffic from/to a VM:
<filter name='ctdirtest' chain='ipv4' priority='-700'> <uuid>582c2fe6-569a-f366-58fb-f995f1a559ce</uuid>
<rule action='accept' direction='out' priority='500'>
<icmp/>
</rule>
<rule action='accept' direction='in' priority='500'>
<icmp/>
</rule>
<rule action='drop' direction='inout' priority='500'>
<all/> </rule>
</filter>
The filter above creates the following types of rules -- some rules are omitted that goto into these user-defined rules.
Chain FI-vnet0 (1 references)
pkts bytes target prot opt in out source
destination
6 504 RETURN icmp -- * * 0.0.0.0/0
0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL
0 0 RETURN icmp -- * * 0.0.0.0/0
0.0.0.0/0 state ESTABLISHED ctdir REPLY
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0Chain FO-vnet0 (1 references)
pkts bytes target prot opt in out source
destination
0 0 ACCEPT icmp -- * * 0.0.0.0/0
0.0.0.0/0 state ESTABLISHED ctdir REPLY
0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL
53 3235 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
Chain HI-vnet0 (1 references)
pkts bytes target prot opt in out source
destination
0 0 RETURN icmp -- * * 0.0.0.0/0
0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL
0 0 RETURN icmp -- * * 0.0.0.0/0
0.0.0.0/0 state ESTABLISHED ctdir REPLY
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0Those may not be the most efficient rules, yet they provide the possibility to remove a rule from the XML and cut the connection immediately due to the ctdir usage.
Quoted 6 lines
If they have no alternatives, then I do now agree that we shouldn't just scrap --ctdir. Then we have two choices:
1) take your patch, which hopefully will successfully guess the polarity of --ctdir correctly in all cases.
Why 'guess' -- it 'probes' for it and does actual testing.
Quoted 5 lines
2) switch unconditionally to the new "correct" polarity of --ctdir, release-note the heck out of it, and require that any distro with a kernel old enough to have the old style --ctdir backport at least the one patch to netfilter to change that.
Not a good idea IMHO because it would break on older kernels.
Quoted 9 lines
My comments about (1): while I'll again say that the patch truly is poetic in its ability to overcome obstacles, it's really a workaround for a bug, and as time goes on will become less and less relevant (and more and more difficult to explain/rationalize). I really think I would prefer to be broken on very old distros rather than have that patch in the tree (although others may have a different opinion, and I would gracefully withdraw my objection in that case).
Quoted 7 lines
About (2): Fedora at least as far back as F16 has a new enough kernel/iptables (iptables-1.4.12) that it uses the new polarity for --ctdir, and no older Fedora is supported. RHEL6 and CentOS6 still have iptables-1.4.7, so they have the old polarity. I'd be willing to bet that RHEL6 would take the netfilter patch to change --ctdir; of course that would leave us with temporary brokenness for anybody who upgraded their kernel without upgrading libvirt, or vice-versa.
You not only need to convert the filtering rules (and only do it once) but also re-program the users minds that may be used to that odd mixup of directions.
Quoted 90 lines · Stefan Berger
Quoted 59 lines · Laine Stump
For reference of people new to this thread, here is the start of the thread:
https://www.redhat.com/archives/libvir-list/2013-March/msg01403.html
This concerns changes to libvirt to cope with the newly discovered (by us :-) difference in interpretation of ctdir by different versions of netfilter.
Quoted 35 lines · Stefan Berger
Quoted 18 lines · Stefan Berger
Quoted 9 lines · Eric Blake
Quoted 8 lines · Laine Stump
My opinion is that the patch we should apply should be a simple patch that just removes use of --ctdir. According to the netfilter developer who responded to the thread on libvirt-users, it doesn't add any extra security not already provided by conntrack:
https://www.redhat.com/archives/libvirt-users/2013-March/msg00121.html
https://www.redhat.com/archives/libvirt-users/2013-March/msg00128.htmlQuoted 6 lines
Quoted 3 lines
Not being an expert on netfilter internals, I can't dispute his claim.
Does anyone else have an opinion?
What filters specifically caused the use of --ctdir, and are they broken
if we omit the use of --ctdir?It depends on how you write the filters that the --ctdir is being used.
Quoted 10 lines
iirc: The effect of the --ctdir usage is that if one has an incoming rule and and outgoing rule with the same IP address on the 'other' side the check for an ESTABLISHED state is not enough to ACCEPT the traffic, if one was to remove one of the rules while communication in both directions was occurring and an immediate cut of the traffic in one way was expected. The effect so far was that if the rule for the incoming rule was removed it would cut the incoming traffic immediately while the traffic in outgoing direction was uninterrupted. I think that if we remove this now the traffic in both directions will continue. I will verify tomorrow.
Verified. I have a ping running from the VM to destination 'A' and from 'A' to the VM. The --ctdir enforces the direction of the traffic and if one of the following rules is removed, the ping is immediately cut.
<rule action='accept' direction='out' priority='500'>
Quoted 8 lines
<icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/></rule>
The ping is not cut anymore upon removal of one of the above rules if --ctdir was to be removed entirely.
Okay, as I understand from your description, the difference is that when a ping in one direction is already in action, and you remove the rule allowing that ping, that existing ping "session" will continue to be allowed *if* there is still a rule allowing pings in the other direction. Is that correct? I'm guessing that *new* attempts to ping in that direction will no longer be allowed though, is that also correct?
Quoted 3 lines
For the benefit of Pablo and the other netfilter developers, can you paste the iptables commands that are generated for the two rules above? Possibly they can suggest alternative rules that have the desired effect.
First off, there are multiple ways one can write the filtering rules in nwfilter, either stateless or stateful:
http://libvirt.org/formatnwfilter.html#nwfwrite
Thus the filter here is only one example how one can write a stateful filter for traffic from/to a VM:
<filter name='ctdirtest' chain='ipv4' priority='-700'> <uuid>582c2fe6-569a-f366-58fb-f995f1a559ce</uuid>
<rule action='accept' direction='out' priority='500'><icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule> <rule action='drop' direction='inout' priority='500'> <all/></rule>
</filter>The filter above creates the following types of rules -- some rules are omitted that goto into these user-defined rules.
Quoted 8 lines
Chain FI-vnet0 (1 references)
pkts bytes target prot opt in out source
destination6 504 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL 0 0 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state ESTABLISHED ctdir REPLY 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
Conntrack is already internally validating that directions are correct for you, so no need for those --ctdir. Let me explain why:
If conntrack gets an ICMP echo reply entering through the NEW state, it will consider it invalid since it is not coming as reply to an ICMP echo request.
[ In case you want to have a look at the internal implementation.
See linux/net/ipv4/netfilter/nf_conntrack_proto_icmp.c, function icmp_new(), the valid ICMP message that can enter through the NEW state are:
static const u_int8_t valid_new[] = {
[ICMP_ECHO] = 1,
[ICMP_TIMESTAMP] = 1,
[ICMP_INFO_REQUEST] = 1,
[ICMP_ADDRESS] = 1
};]
So using:
0.0.0.0/0 state NEW,ESTABLISHED
is just fine if you drop INVALID traffic in your rule-set:
iptables -A F0-vnet0 -m state INVALID -j LOG --log-prefix "invalid: " iptables -A F0-vnet0 -m state INVALID -j DROP
Which are the common rules that you add in a sane stateful rule-set (the log line could be removed if you want to discard traffic silently).
Same thing for TCP traffic. The connection tracking table already validates in its internal state machine that the correct state transitions are actually happening. That validation includes that packets are coming in the good direction.
In sum: The --ctdir is not providing more security. We did not have it originally in the `state' match, it was a late extension to the conntrack match.
My advice here: Just rely on conntrack states and drop invalid traffic, it will do the direction validation that you're trying to achieve with that rule-set.
Hope that it helps.
Regards.
Quoted 104 lines · Pablo Neira Ayuso
Quoted 89 lines · Stefan Berger
Quoted 59 lines · Laine Stump
For reference of people new to this thread, here is the start of the thread:
https://www.redhat.com/archives/libvir-list/2013-March/msg01403.htmlThis concerns changes to libvirt to cope with the newly discovered (by us :-) difference in interpretation of ctdir by different versions of netfilter.
Quoted 41 lines · Stefan Berger
Quoted 18 lines · Stefan Berger
Quoted 9 lines · Eric Blake
Quoted 8 lines · Laine Stump
My opinion is that the patch we should apply should be a simple patch that just removes use of --ctdir. According to the netfilter developer who responded to the thread on libvirt-users, it doesn't add any extra security not already provided by conntrack:
https://www.redhat.com/archives/libvirt-users/2013-March/msg00121.html
https://www.redhat.com/archives/libvirt-users/2013-March/msg00128.htmlQuoted 6 lines
Quoted 3 lines
Not being an expert on netfilter internals, I can't dispute his claim.
Does anyone else have an opinion?
What filters specifically caused the use of --ctdir, and are they broken
if we omit the use of --ctdir?It depends on how you write the filters that the --ctdir is being used.
Quoted 10 lines
iirc: The effect of the --ctdir usage is that if one has an incoming rule and and outgoing rule with the same IP address on the 'other' side the check for an ESTABLISHED state is not enough to ACCEPT the traffic, if one was to remove one of the rules while communication in both directions was occurring and an immediate cut of the traffic in one way was expected. The effect so far was that if the rule for the incoming rule was removed it would cut the incoming traffic immediately while the traffic in outgoing direction was uninterrupted. I think that if we remove this now the traffic in both directions will continue. I will verify tomorrow.
Verified. I have a ping running from the VM to destination 'A' and from 'A' to the VM. The --ctdir enforces the direction of the traffic and if one of the following rules is removed, the ping is immediately cut.
<rule action='accept' direction='out' priority='500'> <icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule>Quoted 2 lines
The ping is not cut anymore upon removal of one of the above rules if --ctdir was to be removed entirely.
Okay, as I understand from your description, the difference is that when a ping in one direction is already in action, and you remove the rule allowing that ping, that existing ping "session" will continue to be allowed *if* there is still a rule allowing pings in the other direction. Is that correct? I'm guessing that *new* attempts to ping in that direction will no longer be allowed though, is that also correct?
Quoted 3 lines
For the benefit of Pablo and the other netfilter developers, can you paste the iptables commands that are generated for the two rules above? Possibly they can suggest alternative rules that have the desired effect.
First off, there are multiple ways one can write the filtering rules in nwfilter, either stateless or stateful:
http://libvirt.org/formatnwfilter.html#nwfwrite
Thus the filter here is only one example how one can write a stateful filter for traffic from/to a VM:
<filter name='ctdirtest' chain='ipv4' priority='-700'> <uuid>582c2fe6-569a-f366-58fb-f995f1a559ce</uuid>
<rule action='accept' direction='out' priority='500'><icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule> <rule action='drop' direction='inout' priority='500'> <all/></rule>
</filter>The filter above creates the following types of rules -- some rules are omitted that goto into these user-defined rules.
Quoted 8 lines
Chain FI-vnet0 (1 references)
pkts bytes target prot opt in out source
destination6 504 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL 0 0 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state ESTABLISHED ctdir REPLY 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0Conntrack is already internally validating that directions are correct for you, so no need for those --ctdir. Let me explain why:
If conntrack gets an ICMP echo reply entering through the NEW state, it will consider it invalid since it is not coming as reply to an ICMP echo request.
[...]
Quoted 8 lines
In sum: The --ctdir is not providing more security. We did not have it originally in the `state' match, it was a late extension to the conntrack match.
My advice here: Just rely on conntrack states and drop invalid traffic, it will do the direction validation that you're trying to achieve with that rule-set.
I don't see that removing a filtering rule, as can be done by an nwfilter user, invalidates the connection tracking state so that a rule dropping upon INVALID state would then kick in. IMO the connection is still in ESTABLISHED state and thus will act on a rule checking on ESTABLISHED state. A simple test here:
iptables -I INPUT 1 -m state --state INVALID -j DROP iptables -I INPUT 2 -p icmp -m state --state ESTABLISHED -j ACCEPT iptables -I INPUT 3 -p icmp -j ACCEPT
Now ping that machine. Pings should work now
Following what you said
iptables -D INPUT -p icmp -j ACCEPT
should now cause the first rule to kick in for that ICMP stream now that the rule is gone. This is not the case with my machine and the ping simply continues -- in this case I have used a RHEL 6 installation with 2.6.32 kernel.
IMO we still need --ctdir.
Stefan
Quoted 127 lines · Stefan Berger
Quoted 104 lines · Pablo Neira Ayuso
Quoted 89 lines · Stefan Berger
Quoted 59 lines · Laine Stump
For reference of people new to this thread, here is the start of the thread:
https://www.redhat.com/archives/libvir-list/2013-March/msg01403.html
This concerns changes to libvirt to cope with the newly discovered (by us :-) difference in interpretation of ctdir by different versions of netfilter.
Quoted 35 lines · Stefan Berger
Quoted 18 lines · Stefan Berger
Quoted 9 lines · Eric Blake
Quoted 8 lines · Laine Stump
My opinion is that the patch we should apply should be a simple patch that just removes use of --ctdir. According to the netfilter developer who responded to the thread on libvirt-users, it doesn't add any extra security not already provided by conntrack:
https://www.redhat.com/archives/libvirt-users/2013-March/msg00121.html
https://www.redhat.com/archives/libvirt-users/2013-March/msg00128.htmlQuoted 6 lines
Quoted 3 lines
Not being an expert on netfilter internals, I can't dispute his claim.
Does anyone else have an opinion?
What filters specifically caused the use of --ctdir, and are they broken
if we omit the use of --ctdir?It depends on how you write the filters that the --ctdir is being used.
Quoted 10 lines
iirc: The effect of the --ctdir usage is that if one has an incoming rule and and outgoing rule with the same IP address on the 'other' side the check for an ESTABLISHED state is not enough to ACCEPT the traffic, if one was to remove one of the rules while communication in both directions was occurring and an immediate cut of the traffic in one way was expected. The effect so far was that if the rule for the incoming rule was removed it would cut the incoming traffic immediately while the traffic in outgoing direction was uninterrupted. I think that if we remove this now the traffic in both directions will continue. I will verify tomorrow.
Verified. I have a ping running from the VM to destination 'A' and from 'A' to the VM. The --ctdir enforces the direction of the traffic and if one of the following rules is removed, the ping is immediately cut.
<rule action='accept' direction='out' priority='500'>
Quoted 8 lines
<icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/></rule>
The ping is not cut anymore upon removal of one of the above rules if --ctdir was to be removed entirely.
Okay, as I understand from your description, the difference is that when a ping in one direction is already in action, and you remove the rule allowing that ping, that existing ping "session" will continue to be allowed *if* there is still a rule allowing pings in the other direction. Is that correct? I'm guessing that *new* attempts to ping in that direction will no longer be allowed though, is that also correct?
Quoted 3 lines
For the benefit of Pablo and the other netfilter developers, can you paste the iptables commands that are generated for the two rules above? Possibly they can suggest alternative rules that have the desired effect.
First off, there are multiple ways one can write the filtering rules in nwfilter, either stateless or stateful:
http://libvirt.org/formatnwfilter.html#nwfwrite
Thus the filter here is only one example how one can write a stateful filter for traffic from/to a VM:
<filter name='ctdirtest' chain='ipv4' priority='-700'> <uuid>582c2fe6-569a-f366-58fb-f995f1a559ce</uuid>
<rule action='accept' direction='out' priority='500'><icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule> <rule action='drop' direction='inout' priority='500'> <all/></rule>
</filter>The filter above creates the following types of rules -- some rules are omitted that goto into these user-defined rules.
Quoted 8 lines
Chain FI-vnet0 (1 references)
pkts bytes target prot opt in out source
destination6 504 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL 0 0 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state ESTABLISHED ctdir REPLY 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0Conntrack is already internally validating that directions are correct for you, so no need for those --ctdir. Let me explain why:
If conntrack gets an ICMP echo reply entering through the NEW state, it will consider it invalid since it is not coming as reply to an ICMP echo request.
[...]
Quoted 8 lines
In sum: The --ctdir is not providing more security. We did not have it originally in the `state' match, it was a late extension to the conntrack match.
My advice here: Just rely on conntrack states and drop invalid traffic, it will do the direction validation that you're trying to achieve with that rule-set.
I don't see that removing a filtering rule, as can be done by an nwfilter user, invalidates the connection tracking state so that a rule dropping upon INVALID state would then kick in. IMO the connection is still in ESTABLISHED state and thus will act on a rule checking on ESTABLISHED state. A simple test here:
iptables -I INPUT 1 -m state --state INVALID -j DROP iptables -I INPUT 2 -p icmp -m state --state ESTABLISHED -j ACCEPT iptables -I INPUT 3 -p icmp -j ACCEPT
Now ping that machine. Pings should work now
Quoted 8 lines
Following what you said
iptables -D INPUT -p icmp -j ACCEPT
should now cause the first rule to kick in for that ICMP stream now that the rule is gone. This is not the case with my machine and the ping simply continues -- in this case I have used a RHEL 6 installation with 2.6.32 kernel.
If default policy is DROP, then no rules will match, so the ping will be dropped.
The rule with the INVALID state only matches if, for example, conntrack sees an ICMP echo reply without having seen an echo request before.
Quoted 127 lines · Pablo Neira Ayuso
Quoted 126 lines · Stefan Berger
Quoted 103 lines · Pablo Neira Ayuso
Quoted 88 lines · Stefan Berger
Quoted 59 lines · Laine Stump
For reference of people new to this thread, here is the start of the thread:
https://www.redhat.com/archives/libvir-list/2013-March/msg01403.htmlThis concerns changes to libvirt to cope with the newly discovered (by us :-) difference in interpretation of ctdir by different versions of netfilter.
Quoted 41 lines · Stefan Berger
Quoted 18 lines · Stefan Berger
Quoted 9 lines · Eric Blake
Quoted 8 lines · Laine Stump
My opinion is that the patch we should apply should be a simple patch that just removes use of --ctdir. According to the netfilter developer who responded to the thread on libvirt-users, it doesn't add any extra security not already provided by conntrack:
https://www.redhat.com/archives/libvirt-users/2013-March/msg00121.html
https://www.redhat.com/archives/libvirt-users/2013-March/msg00128.htmlQuoted 6 lines
Quoted 3 lines
Not being an expert on netfilter internals, I can't dispute his claim.
Does anyone else have an opinion?
What filters specifically caused the use of --ctdir, and are they broken
if we omit the use of --ctdir?It depends on how you write the filters that the --ctdir is being used.
Quoted 10 lines
iirc: The effect of the --ctdir usage is that if one has an incoming rule and and outgoing rule with the same IP address on the 'other' side the check for an ESTABLISHED state is not enough to ACCEPT the traffic, if one was to remove one of the rules while communication in both directions was occurring and an immediate cut of the traffic in one way was expected. The effect so far was that if the rule for the incoming rule was removed it would cut the incoming traffic immediately while the traffic in outgoing direction was uninterrupted. I think that if we remove this now the traffic in both directions will continue. I will verify tomorrow.
Verified. I have a ping running from the VM to destination 'A' and from 'A' to the VM. The --ctdir enforces the direction of the traffic and if one of the following rules is removed, the ping is immediately cut.
<rule action='accept' direction='out' priority='500'> <icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule>Quoted 2 lines
The ping is not cut anymore upon removal of one of the above rules if --ctdir was to be removed entirely.
Okay, as I understand from your description, the difference is that when a ping in one direction is already in action, and you remove the rule allowing that ping, that existing ping "session" will continue to be allowed *if* there is still a rule allowing pings in the other direction. Is that correct? I'm guessing that *new* attempts to ping in that direction will no longer be allowed though, is that also correct?
Quoted 3 lines
For the benefit of Pablo and the other netfilter developers, can you paste the iptables commands that are generated for the two rules above? Possibly they can suggest alternative rules that have the desired effect.
First off, there are multiple ways one can write the filtering rules in nwfilter, either stateless or stateful:
http://libvirt.org/formatnwfilter.html#nwfwrite
Thus the filter here is only one example how one can write a stateful filter for traffic from/to a VM:
<filter name='ctdirtest' chain='ipv4' priority='-700'> <uuid>582c2fe6-569a-f366-58fb-f995f1a559ce</uuid>
<rule action='accept' direction='out' priority='500'><icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule> <rule action='drop' direction='inout' priority='500'> <all/></rule>
</filter>The filter above creates the following types of rules -- some rules are omitted that goto into these user-defined rules.
Quoted 8 lines
Chain FI-vnet0 (1 references)
pkts bytes target prot opt in out source
destination6 504 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL 0 0 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state ESTABLISHED ctdir REPLY 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0Conntrack is already internally validating that directions are correct for you, so no need for those --ctdir. Let me explain why:
If conntrack gets an ICMP echo reply entering through the NEW state, it will consider it invalid since it is not coming as reply to an ICMP echo request.
[...]
Quoted 7 lines
In sum: The --ctdir is not providing more security. We did not have it originally in the `state' match, it was a late extension to the conntrack match.
My advice here: Just rely on conntrack states and drop invalid traffic, it will do the direction validation that you're trying to achieve with that rule-set.
I don't see that removing a filtering rule, as can be done by an nwfilter user, invalidates the connection tracking state so that a rule dropping upon INVALID state would then kick in. IMO the connection is still in ESTABLISHED state and thus will act on a rule checking on ESTABLISHED state. A simple test here:
iptables -I INPUT 1 -m state --state INVALID -j DROP iptables -I INPUT 2 -p icmp -m state --state ESTABLISHED -j ACCEPT iptables -I INPUT 3 -p icmp -j ACCEPT
Now ping that machine. Pings should work now
Following what you said
Quoted 8 lines
Quoted 6 lines
iptables -D INPUT -p icmp -j ACCEPT
should now cause the first rule to kick in for that ICMP stream now that the rule is gone. This is not the case with my machine and the ping simply continues -- in this case I have used a RHEL 6 installation with 2.6.32 kernel.
If default policy is DROP, then no rules will match, so the ping will be dropped.
Unless it runs into a rule '-m state --state ESTABLISHED -j ACCEPT', which I would say is typical for stateful filtering.
The point is '--ctdir' helped us before to cut off traffic and we still need it.
Quoted 5 lines
The rule with the INVALID state only matches if, for example, conntrack sees an ICMP echo reply without having seen an echo request before.
Well, yeah, but this is not the case we're after.
Stefan
Quoted 134 lines · Stefan Berger
Quoted 133 lines · Pablo Neira Ayuso
Quoted 132 lines · Stefan Berger
Quoted 103 lines · Pablo Neira Ayuso
Quoted 88 lines · Stefan Berger
Quoted 59 lines · Laine Stump
For reference of people new to this thread, here is the start of the thread:
https://www.redhat.com/archives/libvir-list/2013-March/msg01403.html
This concerns changes to libvirt to cope with the newly discovered (by us :-) difference in interpretation of ctdir by different versions of netfilter.
Quoted 35 lines · Stefan Berger
Quoted 18 lines · Stefan Berger
Quoted 9 lines · Eric Blake
Quoted 8 lines · Laine Stump
My opinion is that the patch we should apply should be a simple patch that just removes use of --ctdir. According to the netfilter developer who responded to the thread on libvirt-users, it doesn't add any extra security not already provided by conntrack:
https://www.redhat.com/archives/libvirt-users/2013-March/msg00121.html
https://www.redhat.com/archives/libvirt-users/2013-March/msg00128.htmlQuoted 6 lines
Quoted 3 lines
Not being an expert on netfilter internals, I can't dispute his claim.
Does anyone else have an opinion?
What filters specifically caused the use of --ctdir, and are they broken
if we omit the use of --ctdir?It depends on how you write the filters that the --ctdir is being used.
Quoted 10 lines
iirc: The effect of the --ctdir usage is that if one has an incoming rule and and outgoing rule with the same IP address on the 'other' side the check for an ESTABLISHED state is not enough to ACCEPT the traffic, if one was to remove one of the rules while communication in both directions was occurring and an immediate cut of the traffic in one way was expected. The effect so far was that if the rule for the incoming rule was removed it would cut the incoming traffic immediately while the traffic in outgoing direction was uninterrupted. I think that if we remove this now the traffic in both directions will continue. I will verify tomorrow.
Verified. I have a ping running from the VM to destination 'A' and from 'A' to the VM. The --ctdir enforces the direction of the traffic and if one of the following rules is removed, the ping is immediately cut.
<rule action='accept' direction='out' priority='500'>
Quoted 8 lines
<icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/></rule>
The ping is not cut anymore upon removal of one of the above rules if --ctdir was to be removed entirely.
Okay, as I understand from your description, the difference is that when a ping in one direction is already in action, and you remove the rule allowing that ping, that existing ping "session" will continue to be allowed *if* there is still a rule allowing pings in the other direction. Is that correct? I'm guessing that *new* attempts to ping in that direction will no longer be allowed though, is that also correct?
Quoted 3 lines
For the benefit of Pablo and the other netfilter developers, can you paste the iptables commands that are generated for the two rules above? Possibly they can suggest alternative rules that have the desired effect.
First off, there are multiple ways one can write the filtering rules in nwfilter, either stateless or stateful:
http://libvirt.org/formatnwfilter.html#nwfwrite
Thus the filter here is only one example how one can write a stateful filter for traffic from/to a VM:
<filter name='ctdirtest' chain='ipv4' priority='-700'> <uuid>582c2fe6-569a-f366-58fb-f995f1a559ce</uuid>
<rule action='accept' direction='out' priority='500'><icmp/> </rule> <rule action='accept' direction='in' priority='500'> <icmp/> </rule> <rule action='drop' direction='inout' priority='500'> <all/></rule>
</filter>The filter above creates the following types of rules -- some rules are omitted that goto into these user-defined rules.
Quoted 8 lines
Chain FI-vnet0 (1 references)
pkts bytes target prot opt in out source
destination6 504 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED ctdir ORIGINAL 0 0 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0 state ESTABLISHED ctdir REPLY 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0Conntrack is already internally validating that directions are correct for you, so no need for those --ctdir. Let me explain why:
If conntrack gets an ICMP echo reply entering through the NEW state, it will consider it invalid since it is not coming as reply to an ICMP echo request.
[...]
Quoted 7 lines
In sum: The --ctdir is not providing more security. We did not have it originally in the `state' match, it was a late extension to the conntrack match.
My advice here: Just rely on conntrack states and drop invalid traffic, it will do the direction validation that you're trying to achieve with that rule-set.
I don't see that removing a filtering rule, as can be done by an nwfilter user, invalidates the connection tracking state so that a rule dropping upon INVALID state would then kick in. IMO the connection is still in ESTABLISHED state and thus will act on a rule checking on ESTABLISHED state. A simple test here:
iptables -I INPUT 1 -m state --state INVALID -j DROP iptables -I INPUT 2 -p icmp -m state --state ESTABLISHED -j ACCEPT iptables -I INPUT 3 -p icmp -j ACCEPT
Now ping that machine. Pings should work now
Following what you said
iptables -D INPUT -p icmp -j ACCEPT
should now cause the first rule to kick in for that ICMP stream now that the rule is gone. This is not the case with my machine and the ping simply continues -- in this case I have used a RHEL 6 installation with 2.6.32 kernel.
Quoted 5 lines
Quoted 2 lines
If default policy is DROP, then no rules will match, so the ping will be dropped.
Unless it runs into a rule '-m state --state ESTABLISHED -j ACCEPT', which I would say is typical for stateful filtering.
Not sure what you mean. The first packet of an ICMP echo request will not ever match -m state --state ESTABLISHED.
Quoted 9 lines
The point is '--ctdir' helped us before to cut off traffic and we still need it.
Quoted 5 lines
The rule with the INVALID state only matches if, for example, conntrack sees an ICMP echo reply without having seen an echo request before.
Well, yeah, but this is not the case we're after.