Creating rules without the /sbin/iptables command?
Hi,
I'm writing a tool for creating a rulesset. It's meant for users who do understand the way a firewall works, but don't have any netfilter-specific knowledge. It can output a bash-script or create the rules itself.
My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something? I'm going to release the tool under the GPL soon, so that shouldn't be a problem.
Regards,
Victor
Quoted 4 lines · Victor Julien
My program (written in c) creates rules by opening a pipe to
/sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?
You could use iptables libs that stand in /usr/lib/iptables, just like iptables does.
You should however ask netfilter developpers mailing list.
Signature · 2 lines
http://www.netexit.com/~sid/ PGP KeyID: 157E98EE FingerPrint: FA62226DA9E72FA8AECAA240008B480E157E98EE
Quoted 2 lines
Quoted 2 lines
Hi! I'm your friendly neighbourhood signature virus. Copy me to your signature file and help me spread!
Hi Cedric,
Okay, thanx i will do that. I thought that netfilter-devel was only for development of netfilter itself. My bad ;)
By the way, are the iptables libs in /usr/lib/iptables meant for use by third party tools, or is this considered a dirty hack?
Regards,
Victor
Quoted 12 lines · Cedric Blancher
Quoted 1 line · Victor Julien
My program (written in c) creates rules by opening a pipe to
/sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?You could use iptables libs that stand in
/usr/lib/iptables, just like iptables does.You should however ask netfilter developpers mailing list.
Quoted 8 lines · Cedric Blancher
Quoted 4 lines · Victor Julien
My program (written in c) creates rules by opening a pipe to
/sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?You could use iptables libs that stand in
/usr/lib/iptables, just like iptables does.
Or actually the preferred interface for this type of operations is to use iptables-restore, the batch version of iptables. The speed difference from using iptables-restore and direct calls is pretty minimal by any means.
libiptc is an internal interface of the iptables source tree and is subject to change at any time. This should not be used directly unless you have very good reasons.
Regards
Henrik
Hi Hendrik,
This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?
Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?
Regards,
Victor
Quoted 19 lines · Henrik Nordstrom
Quoted 6 lines · Cedric Blancher
Quoted 1 line · Victor Julien
My program (written in c) creates rules by opening a pipe to
/sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?You could use iptables libs that stand in
/usr/lib/iptables, just like iptables does.Or actually the preferred interface for this type of operations is to use iptables-restore, the batch version of iptables. The speed difference from using iptables-restore and direct calls is pretty minimal by any means.
libiptc is an internal interface of the iptables source tree and is
subject to change at any time. This should not be used directly unless you have very good reasons.Regards
Henrik
Quoted 6 lines · Cedric Blancher
Quoted 4 lines · Victor Julien
My program (written in c) creates rules by opening a pipe to
/sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?
Quoted 6 lines
You could use iptables libs that stand in
/usr/lib/iptables, just like iptables does.You should however ask netfilter developpers mailing list.
The above solution is probably better than mine but I write out my new rules to a file in iptables-save format and then use iptables-restore -n <filename>. This is dramatically faster than calling iptables - John
Signature · 9 lines
John A. Sullivan III Chief Technology Officer Nexus Management +1 207-985-7880 [email protected] --- If you are interested in helping to develop a GPL enterprise class VPN/Firewall/Security device management console, please visit http://iscs.sourceforge.net
I use it all the time in the ISCS project -- just remember to use the -n argument. It is much, much faster. However, you need to supply the input file in the proper and not well documented format - John
Quoted 40 lines · Victor Julien
Hi Hendrik,
This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?
Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?Regards,
VictorQuoted 25 lines · Henrik Nordstrom
Quoted 9 lines · Cedric Blancher
Quoted 4 lines · Victor Julien
My program (written in c) creates rules by opening a pipe to
/sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?You could use iptables libs that stand in
/usr/lib/iptables, just like iptables does.Or actually the preferred interface for this type of operations is to use iptables-restore, the batch version of iptables. The speed difference from using iptables-restore and direct calls is pretty minimal by any means.
libiptc is an internal interface of the iptables source tree and is subject to change at any time. This should not be used directly unless you have very good reasons.
Regards
Henrik
Signature · 5 lines
John A. Sullivan III Chief Technology Officer Nexus Management +1 207-985-7880 [email protected]
Quoted 3 lines · Victor Julien
This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?
You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.
Quoted 4 lines
Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?
Such C function would only be marginally faster than having a pipe to iptables-restore.
iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.
iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences
Syntax for iptables-restore input is
*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.
:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.
COMMIT
Uploads the current table (specified by *tablename) to the kernel.
#....
Input lines starting with # are assumed to be comments and are ignored
Any other lines are assumed to be iptables commands per the iptables syntax specification.
iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.
Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.
Regards
Henrik
Quoted 52 lines · Henrik Nordstrom
Quoted 3 lines · Victor Julien
This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?
You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.
Quoted 4 lines
Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?Such C function would only be marginally faster than having a pipe to iptables-restore.
iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.
iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences
Syntax for iptables-restore input is
*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.COMMIT
Uploads the current table (specified by *tablename) to the kernel.#....
Input lines starting with # are assumed to be comments and are ignoredAny other lines are assumed to be iptables commands per the iptables syntax specification.
iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.
Quoted 7 lines
Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.
Regards
Henrik
I found that there were a few other differences. As far as I can tell, you can only have one table section per file, in other words if I have a file such as
*nat
some rules
COMMIT
*filter
some more rules
COMMIT
*nat
some more rules
COMMIT
only the first set of NAT rules commits. I also found that I needed to use an explicit declaration of implicit matches, e.g., *filter
-A FORWARD -p 6 --dport 80 -j ACCEPT - does not work
-A FORWARD -p 6 -m tcp --dport 80 -j ACCEPT - does work COMMIT
I did find that the performance difference for even small rule sets was noticeable and, for large rule sets, it was absolutely stunning - John
Signature · 5 lines
John A. Sullivan III Chief Technology Officer Nexus Management +1 207-985-7880 [email protected]
Quoted 14 lines · John A. Sullivan III
I found that there were a few other differences. As far as I can tell, you can only have one table section per file, in other words if I have a file such as
*nat
some rules
COMMIT
*filter
some more rules
COMMIT
*nat
some more rules
COMMIT
Quoted 1 line
only the first set of NAT rules commits.
If this is the case then it is a bug somewhere, most likely in libiptc..
seems to work fine here however.. tested with
*filter
-A INPUT -p tcp --dport 45 -j ACCEPT
COMMIT
*nat
-A PREROUTING -p tcp --dport 45 -j ACCEPT
COMMIT
*filter
-A OUTPUT -j ACCEPT
COMMIT
and all three rules was added just fine.
also I do not see any reason why it should not work. The libiptc handle is completely closed on COMMIT so there should not be any memory of you already have used this table when the new *tablename is reached.
Note: when doing this you MUST use the --noflush (-n) option. If not the second reference to the table will clear the rules added by the first..
Quoted 7 lines
I also found that I needed to use an explicit declaration of implicit matches, e.g.,
*filter
(quoted diff) +0
-2-A FORWARD -p 6 --dport 80 -j ACCEPT - does not work -A FORWARD -p 6 -m tcp --dport 80 -j ACCEPT - does workCOMMIT
Works fine here without.. and considering that the exact same iptables.c is used in both commands for parsing the commands I don't see why it should not work in any version of iptables-restore..
The fact that iptables-save prints the protocol match is only due to the fact that this is actually what the rules looks like if you are protocol match options.. there is some heavy duty "magics" in iptables command parsing if a protocol is specified but no protocol match. There is an interesting comment in iptables.c which summarises this magic protocol match function quite well ;-)
/* If you listen carefully, you can
actually hear this code suck. */
Regards
Henrik
Quoted 60 lines · Henrik Nordstrom
Quoted 14 lines · John A. Sullivan III
I found that there were a few other differences. As far as I can tell, you can only have one table section per file, in other words if I have a file such as
*nat
some rules
COMMIT
*filter
some more rules
COMMIT
*nat
some more rules
COMMITQuoted 1 line
only the first set of NAT rules commits.
If this is the case then it is a bug somewhere, most likely in libiptc..
seems to work fine here however.. tested with
*filter
-A INPUT -p tcp --dport 45 -j ACCEPT
COMMIT
*nat
-A PREROUTING -p tcp --dport 45 -j ACCEPT
COMMIT
*filter
-A OUTPUT -j ACCEPT
COMMITand all three rules was added just fine.
also I do not see any reason why it should not work. The libiptc handle is completely closed on COMMIT so there should not be any memory of you already have used this table when the new *tablename is reached.
Note: when doing this you MUST use the --noflush (-n) option. If not the second reference to the table will clear the rules added by the first..
Quoted 7 lines
I also found that I needed to use an explicit declaration of implicit matches, e.g.,
*filter
(quoted diff) +0
-2-A FORWARD -p 6 --dport 80 -j ACCEPT - does not work -A FORWARD -p 6 -m tcp --dport 80 -j ACCEPT - does workCOMMIT
Works fine here without.. and considering that the exact same
iptables.cis used in both commands for parsing the commands I don't see why it should not work in any version of iptables-restore..The fact that iptables-save prints the protocol match is only due to the fact that this is actually what the rules looks like if you are protocol match options.. there is some heavy duty "magics" in iptables command parsing if a protocol is specified but no protocol match. There is an interesting comment in
iptables.cwhich summarises this magic protocol match function quite well ;-)
Quoted 6 lines
/* If you listen carefully, you can
actually hear this code suck. */Regards
Henrik
Interesting. Thanks for the update. I know the last time I tested was with a pretty old iptables - I don't know the version number as I have long since upgraded. Perhaps I was braincramping but I'm almost certain it choked if I didn't use the explicit match. Thanks again - John
Signature · 5 lines
John A. Sullivan III Chief Technology Officer Nexus Management +1 207-985-7880 [email protected]
Just curious, having made my own decision to not use the kinda-interfaces and just call the provided iptables executables; has anyone run valgrind
(http://valgrind.kde.org/) on iptables? I did a simple one like:
valgrind --leak-check=yes iptables -A PREROUTING -t mangle -s 192.168.1.0/24 -j DROP
and got a few...I would be cautious about using these routines in a long-running app unless they were looked at more.
__________________________________
Do you Yahoo!?
Yahoo! Mail - More reliable, more storage, less spam
http://mail.yahoo.com
Quoted 44 lines · Henrik Nordstrom
Quoted 1 line · Victor Julien
This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?
You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.
Quoted 3 lines
Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?Such C function would only be marginally faster than having a pipe to iptables-restore.
iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.
iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences
Syntax for iptables-restore input is
*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.COMMIT
Uploads the current table (specified by *tablename) to the kernel.#....
Input lines starting with # are assumed to be comments and are ignoredAny other lines are assumed to be iptables commands per the iptables syntax specification.
iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.
Quoted 9 lines
Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.
Regards
Henrik
Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':
*filter
:FORWARD DROP
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT COMMIT
and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):
*filter
:FORWARD DROP
-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT COMMIT
and then 'iptables-restore -n', right?
But the easiest way is to recreate the initial ruleset with the updated rules would be:
*filter
:FORWARD DROP
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT COMMIT
and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?
The last method should still be way faster than my current method, i guess. Is this right?
Regards,
Victor
Quoted 17 lines · Victor Julien
Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):
*filter
:FORWARD DROP(quoted diff) +0
-2-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
Quoted 1 line
COMMIT
Or you could
*filer
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT COMMIT
Quoted 1 line
and then 'iptables-restore -n', right?
Right. Or you could even have iptables-restore -n running on a pipe already.. (but beware of memoryleaks in libiptc)
Quoted 9 lines
But the easiest way is to recreate the initial ruleset with the updated rules would be:
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
Depends on your application, but yes I find this easier most times.
Quoted 2 lines
The last method should still be way faster than my current method, i guess. Is this right?
iptables-restore is quicker than iptables as soon as you have more than one modification to the table. On single-rule operations it is identical to iptables.
if you use a pipe to a already running iptables-restore then it isfaster than iptables even on single rule operations.
In addition iptables-restore allows you to atomically replace the complete ruleset without race windows. This can not be done with iptables and with iptables packets will be forwarded while the new ruleset is only partiailly installed...
Regards
Henrik
Quoted 6 lines · Scott MacKay
valgrind --leak-check=yes iptables -A PREROUTING -t mangle -s 192.168.1.0/24 -j DROP
and got a few...I would be cautious about using these routines in a long-running app unless they were looked at more.
libiptc is not really intended for long-running applications. The only official use of this is iptables and iptables-restore which usually does not run for a long time, so there has not been very much attention to memory leaks etc.. but bugfixes are usually accepted when found.
Note that the design of libiptc will indicate some static memory leaks due to modules being loaded etc unless you compile it as a single binary. These leaks are not interesting to have fixed. The type of leaks interesting to have fixed is the leaks which grows over time as more and more operations have been executed.
Regards
Henrik
Quoted 2 lines · Victor Julien
Okay, thanx i will do that. I thought that netfilter-devel was only for development of netfilter itself. My bad ;)
No, you can discuss third party tools development around iptables stuff also.
Quoted 2 lines
By the way, are the iptables libs in
/usr/lib/iptablesmeant for use by third party tools, or is this considered a dirty hack?
They are meant to be used by iptables itself. But you can use them with your own soft. That's no dirty hack.
Signature · 2 lines
http://www.netexit.com/~sid/ PGP KeyID: 157E98EE FingerPrint: FA62226DA9E72FA8AECAA240008B480E157E98EE
Quoted 2 lines
Quoted 2 lines
Hi! I'm your friendly neighbourhood signature virus. Copy me to your signature file and help me spread!
Quoted 86 lines · Victor Julien
Quoted 56 lines · Henrik Nordstrom
Quoted 3 lines · Victor Julien
This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?
You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.
Quoted 4 lines
Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?Such C function would only be marginally faster than having a pipe to iptables-restore.
iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.
iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences
Syntax for iptables-restore input is
*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.COMMIT
Uploads the current table (specified by *tablename) to the kernel.#....
Input lines starting with # are assumed to be comments and are ignoredAny other lines are assumed to be iptables commands per the iptables syntax specification.
iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.
Quoted 9 lines
Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.
Regards
HenrikOkay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):
*filter
:FORWARD DROP(quoted diff) +0
-2-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPTCOMMIT
Quoted 1 line
and then 'iptables-restore -n', right?
I believe that is correct although I also believe you can dispense with the :FORWARD DROP since you already had it in the first rule set and could also use
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport -j ACCEPT
to just replace the rule although I confess to never having tried that in an iptables-restore file
Quoted 11 lines
But the easiest way is to recreate the initial ruleset with the updated rules would be:
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
Quoted 3 lines
and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?
Yes, although when ISCS is released (http://iscs.sourceforge.net), it will provide an alternative to having to track rule order (massive oversimplification here but it is not the topic at hand).
Quoted 3 lines
The last method should still be way faster than my current method, i guess. Is this right?
I have found it to be dramatically faster - John
Quoted 3 lines
Regards,
Victor
Signature · 3 lines
Open Source Development Corporation Financially Sustainable open source development http://www.opensourcedevelopmentcorp.com
Quoted 23 lines · Henrik Nordstrom
Quoted 15 lines · Victor Julien
Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):
*filter
:FORWARD DROP(quoted diff) +0
-2-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPTQuoted 1 line
COMMIT
Or you could
Quoted 9 lines
*filer
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT COMMITQuoted 1 line
and then 'iptables-restore -n', right?
Right. Or you could even have iptables-restore -n running on a pipe already.. (but beware of memoryleaks in libiptc)
Okay, i think i'll rather open the pipe on demand.
Quoted 11 lines
Quoted 8 lines
But the easiest way is to recreate the initial ruleset with the updated rules would be:
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
Depends on your application, but yes I find this easier most times.
Good! But if i have a rule for accounting, how does it work then? If you consider this (useless) example:
*filter
:FORWARD DROP
# account the traffic for 192.168.0.1 -A FORWARD -p tcp -s 192.168.0.1 -A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT COMMIT
if i now want to reload the rule set to
*filter
:FORWARD DROP
# account the traffic for 192.168.0.1 -A FORWARD -p tcp -s 192.168.0.1 -A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.12 --dport 80 -j ACCEPT COMMIT
using 'iptables-restore' (without noflush) is there a way to preserve the accounting data from the first rule? Is this what the -c option of iptables-restore is for?
Quoted 6 lines
Quoted 1 line
The last method should still be way faster than my current method, i guess. Is this right?
iptables-restore is quicker than iptables as soon as you have more than one modification to the table. On single-rule operations it is identical to iptables.
Quoted 7 lines
if you use a pipe to a already running iptables-restore then it isfaster than iptables even on single rule operations.
In addition iptables-restore allows you to atomically replace the complete ruleset without race windows. This can not be done with iptables and with iptables packets will be forwarded while the new ruleset is only partiailly installed...
Sounds like what i'm looking for...
Regards,
Victor
John A. Sullivan III wrote:
<snip>
Quoted 36 lines
Quoted 18 lines
Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):
*filter
:FORWARD DROP(quoted diff) +0
-2-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPTCOMMIT
Quoted 1 line
and then 'iptables-restore -n', right?
I believe that is correct although I also believe you can dispense with the :FORWARD DROP since you already had it in the first rule set and could also use
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport -j ACCEPT
to just replace the rule although I confess to never having tried that in an iptables-restore fileQuoted 9 lines
But the easiest way is to recreate the initial ruleset with the updated rules would be:
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
Quoted 5 lines
Quoted 1 line
and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?
Yes, although when ISCS is released (http://iscs.sourceforge.net), it will provide an alternative to having to track rule order (massive oversimplification here but it is not the topic at hand).
But how will ISCS handle this problem? Will it also create input for iptables-restore? Or do you have some other method?
Quoted 4 lines
Quoted 2 lines
The last method should still be way faster than my current method, i guess. Is this right?
I have found it to be dramatically faster - John
Good!
Regards,
Victor
Quoted 3 lines
Quoted 3 lines
Regards,
Victor
Quoted 5 lines · Victor Julien
John A. Sullivan III wrote:
<snip>
Quoted 1 line
Quoted 1 line
<snip>
Quoted 18 lines
Quoted 10 lines
Quoted 10 lines
But the easiest way is to recreate the initial ruleset with the updated rules would be:
*filter
:FORWARD DROP(quoted diff) +0
-3-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPTCOMMIT
Quoted 7 lines
Quoted 3 lines
and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?
Yes, although when ISCS is released (http://iscs.sourceforge.net), it will provide an alternative to having to track rule order (massive oversimplification here but it is not the topic at hand).
Quoted 2 lines
But how will ISCS handle this problem? Will it also create input for iptables-restore? Or do you have some other method?
Yes, it creates input files for iptables restore and is also able to create configuration files for other firewalls (as well as the VPN configuration, the router configuration, the NAT configuration and any local DHCP configuration and, hopefully soon, all the layer2 configuration for the individual gateways). However, rather than relying exclusively upon rule order, it uses an approach we call "Best Match". This allows us to automate the creation of all rules from a high level description of the environment. The idea is to eliminate the need for an administrator to create rules. Instead, we interpret the environment, e.g., "give these three teams access to the new joint development project data", and create consistent rules for access control, user authentication, encryption, data authentication, routing, NAT, etc, to produce that environment.
Quoted 15 lines
Quoted 6 lines
Quoted 4 lines
The last method should still be way faster than my current method, i guess. Is this right?
I have found it to be dramatically faster - John
Good!
Regards,
VictorQuoted 3 lines
Quoted 3 lines
Regards,
Victor
Signature · 5 lines
John A. Sullivan III Chief Technology Officer Nexus Management +1 207-985-7880 [email protected]
Quoted 2 lines · Victor Julien
using 'iptables-restore' (without noflush) is there a way to preserve the accounting data from the first rule?
Then you must use incremental commands, keepint the accounting rule untouched.
Quoted 1 line
Is this what the -c option of iptables-restore is for?
No. This instructs iptables-restore to restore any packet/byte counters found in the input data. Without this option any such counters is ignored and the installed rules have zeroed counters.
Regards
Henrik
Quoted 6 lines · Henrik Nordstrom
Quoted 1 line · Victor Julien
using 'iptables-restore' (without noflush) is there a way to preserve the accounting data from the first rule?
Then you must use incremental commands, keepint the accounting rule untouched.
Hmmm, that i don't want.
Quoted 4 lines
Quoted 2 lines
Is this what the -c option of iptables-restore is for?
No. This instructs iptables-restore to restore any packet/byte counters found in the input data. Without this option any such counters is ignored and the installed rules have zeroed counters.
So i have to read the current counters first by parsing 'iptables-save'-output. And i can then recreate the rules and set the counters to their old values?
Using 'iptables-restore' can i flush one chain, but leave another untouched? If so, i can create an accounting chain which will not be flushed...
Regards,
Victor
Quoted 4 lines
Regards
Henrik
Quoted 4 lines · Victor Julien
Quoted 2 lines
Then you must use incremental commands, keepint the accounting rule untouched.
Hmmm, that i don't want.
Then your other option is to dump the existing rule, parse the counter and include this in the input.
Quoted 3 lines
So i have to read the current counters first by parsing 'iptables-save'-output. And i can then recreate the rules and set the counters to their old values?
Yes.
Quoted 2 lines
Using 'iptables-restore' can i flush one chain, but leave another untouched?
Yes, by using disabling the implicit flush (--noflush option) and then use the flush command explicitly.
Quoted 2 lines
If so, i can create an accounting chain which will not be flushed...
True. Probably much better approach than what is discussed above.
Regards
Regards
Henrik
Quoted 21 lines · Henrik Nordstrom
Quoted 3 lines · Victor Julien
Quoted 1 line
Then you must use incremental commands, keepint the accounting rule untouched.
Hmmm, that i don't want.
Then your other option is to dump the existing rule, parse the counter and include this in the input.
Quoted 1 line
So i have to read the current counters first by parsing 'iptables-save'-output. And i can then recreate the rules and set the counters to their old values?
Yes.
Quoted 1 line
Using 'iptables-restore' can i flush one chain, but leave another untouched?
Quoted 7 lines
Yes, by using disabling the implicit flush (--noflush option) and then use the flush command explicitly.
Quoted 1 line
If so, i can create an accounting chain which will not be flushed...
True. Probably much better approach than what is discussed above.
So the best option would be to create an accounting chain (e.g ACCOUNT). If i, for example, want to recreate the rules in the INPUT chain it should work like this:
Create a file for iptables-restore input:
*filter
:INPUT DROP
# flush this chain -F INPUT # now recreate the rules -A INPUT -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT ...more rules... COMMIT
Now i would have to call 'iptables-restore --noflush'. This will clear the entire INPUT chain, but will leave all other (including my ACCOUNT chain) chains untouched.
Right?
Regards,
Victor
Quoted 8 lines · Victor Julien
So the best option would be to create an accounting chain (e.g ACCOUNT). If i, for example, want to recreate the rules in the INPUT chain it should work like this:
Create a file for iptables-restore input:
*filter
:INPUT DROP
This line is redundant. You only need policy lines if you want to change the policy.
Quoted 10 lines
# flush this chain -F INPUT # now recreate the rules -A INPUT -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT ...more rules... COMMITNow i would have to call 'iptables-restore --noflush'. This will clear the entire INPUT chain, but will leave all other (including my ACCOUNT chain) chains untouched.
Correct.
But dont forget to insert a jump to the ACCOUNT chain where needed..
*filter
-F INPUT
-A INPUT -j ACCOUNT
-A INTPUT -p tcp .... -j ACCEPT
....
COMMIT
Regards
Henrik
Quoted 8 lines · Henrik Nordstrom
Quoted 6 lines
So the best option would be to create an accounting chain (e.g ACCOUNT). If i, for example, want to recreate the rules in the INPUT chain it should work like this:
Create a file for iptables-restore input:
*filter
:INPUT DROPThis line is redundant. You only need policy lines if you want to change the policy.
But i do need it if i want to (re)set chain counters, right?
Like this:
:INPUT DROP [123:45678]
Quoted 19 lines
Quoted 9 lines
# flush this chain -F INPUT # now recreate the rules -A INPUT -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT ...more rules... COMMITNow i would have to call 'iptables-restore --noflush'. This will clear the entire INPUT chain, but will leave all other (including my ACCOUNT chain) chains untouched.
Correct.
But dont forget to insert a jump to the ACCOUNT chain where needed..
*filter
(quoted diff) +0
-3-F INPUT -A INPUT -j ACCOUNT -A INTPUT -p tcp .... -j ACCEPT
Quoted 2 lines
COMMIT
Well, i think i know all i need now, thanks a lot for your help! Now i stand for the daunting task of implmenting this in my current codebase ;-)
Well i guess that will have to wait a bit, my current version is quite stable (for me a least) and i plan to release it soon...
Regards,
Victor
Quoted 7 lines · Victor Julien
Quoted 2 lines
This line is redundant. You only need policy lines if you want to change the policy.
But i do need it if i want to (re)set chain counters, right?
Like this:
:INPUT DROP [123:45678]
Yes.
Or if you don't the old counters should be kept.
Regards
Henrik
Adding packet metadata
In reply to Henrik NordstromHello,
I wanted to mark up packets with information as they pass thru the system. Basically, I want to associate some data with a packet during PREROUTING/raw (to read info prior to conntrack) and then get that information back during PREROUTING/mangle. Is there a module already written which could perform this? It is not something I would need to have accessible to the iptables program, though I will probably need to adjust some code to make it accessible to my ip_queue userspace application...
-Scott
__________________________________
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html
Re: Adding packet metadata
Quoted 5 lines · Scott MacKay
I wanted to mark up packets with information as they pass thru the system. Basically, I want to associate some data with a packet during PREROUTING/raw (to read info prior to conntrack) and then get that information back during PREROUTING/mangle. Is there a module already written which could perform this?
MARK should be able to do this.
Quoted 3 lines
It is not something I would need to have accessible to the iptables program, though I will probably need to adjust some code to make it accessible to my
ip_queueuserspace application...
The MARK value is available (read-only) via ip_queue.
Regards
Henrik
Re: Adding packet metadata
Quoted 10 lines
Quoted 6 lines
I wanted to mark up packets with information as they pass thru the system. Basically, I want to associate some data with a packet during PREROUTING/raw (to read info prior to conntrack) and then get that information back during PREROUTING/mangle. Is there a module already written which could perform this?
MARK should be able to do this.
An interesting question about MARK field in SKB structure is: Can I use all 32-bit of "nfmark" for adding metadata? Iptables front-end (from command line) seems to take mark value as a 8-bit number.
Mike-Ro-Chanel
Re: Adding packet metadata
Quoted 2 lines · Mike-Ro-Chanel
An interesting question about MARK field in SKB structure is: Can I use all 32-bit of "nfmark" for adding metadata?
Yes.
Quoted 2 lines
Iptables front-end (from command line) seems to take mark value as a 8-bit number.
What makes you think so?
There is a slight signed bug in the string_to_number function which prevents the use of the 2^31 bit, easily fixed by changing it to use unsigned.
Regards
Henrik
Re: Adding packet metadata
Quoted 6 lines · Henrik Nordstrom
Quoted 1 line · Mike-Ro-Chanel
An interesting question about MARK field in SKB structure is: Can I use all 32-bit of "nfmark" for adding metadata?
Quoted 8 lines
Yes.
Quoted 2 lines
Iptables front-end (from command line) seems to take mark value as a 8-bit number.
What makes you think so?
Now I understand, it is my error on iptables usage.
Quoted 2 lines
There is a slight signed bug in the
string_to_numberfunction which prevents the use of the 2^31 bit, easily fixed by changing it to use unsigned.
Ok, thanks for the tip!
Mike-Ro-Chanel
Re: Adding packet metadata
In reply to Henrik NordstromQuoted 5 lines · --- Henrik Nordstrom
Quoted 1 line · Scott MacKay
I wanted to mark up packets with information as
they pass thru the
Quoted 1 line
system. Basically, I want to associate some data
Quoted 8 lines
with a packet during
Quoted 1 line
PREROUTING/raw (to read info prior to conntrack)
and then get that
Quoted 1 line
information back during PREROUTING/mangle. Is
there a module already
Quoted 1 line
written which could perform this?
MARK should be able to do this.
I actually was thinking of using the MARK for a different purpose unfortunantly. Since it is a nice 32bit word, I was thinking of even using only part of it, but I need it for setting up some match settings for actual rules.
Quoted 8 lines
Quoted 1 line
It is not something I would need to have
accessible to the iptables
Quoted 1 line
program, though I will probably need to adjust
some code to make it
Quoted 1 line
accessible to my
ip_queueuserspace application...The MARK value is available (read-only) via
ip_queue.
That would be my general problem. The only bit that seems transmitted back is the payload for evaluation and update. Even if I wanted to use MARK, it would not allow me to alter the value.
__________________________________
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html
Re: Adding packet metadata
Quoted 5 lines · Scott MacKay
I actually was thinking of using the MARK for a different purpose unfortunantly. Since it is a nice 32bit word, I was thinking of even using only part of it, but I need it for setting up some match settings for actual rules.
Then you only need a new MARK target as the existing one does not support masked operations (only the mark match does..).
Quoted 6 lines
Quoted 2 lines
The MARK value is available (read-only) via
ip_queue.That would be my general problem. The only bit that seems transmitted back is the payload for evaluation and update. Even if I wanted to use MARK, it would not allow me to alter the value.
There is a patch somewhere to extend ip_queue to be able to return the mark.. and no matter what approach you take something similar is needed if you from ip_queue want to return other information than the verdict and optional payload substitution.
Regards
Henrik