Skip to contentSkip to conversation

Creating rules without the /sbin/iptables command?

DISCUSSION Victor Julien 7 participants 34 messages Mar 17, 2004 – Mar 24, 2004 netfilter

lore
Share & tools
lore
#0 lore raw

Hi,

I'm writing a tool for creating a rulesset. It's meant for users who do understand the way a firewall works, but don't have any netfilter-specific knowledge. It can output a bash-script or create the rules itself.

My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something? I'm going to release the tool under the GPL soon, so that shouldn't be a problem.

Regards,
Victor

#1 lore raw
Quoted 4 lines · Victor Julien

My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?

You could use iptables libs that stand in /usr/lib/iptables, just like iptables does.

You should however ask netfilter developpers mailing list.

Signature · 2 lines
http://www.netexit.com/~sid/
PGP KeyID: 157E98EE FingerPrint: FA62226DA9E72FA8AECAA240008B480E157E98EE
Quoted 2 lines
Quoted 2 lines

Hi! I'm your friendly neighbourhood signature virus. Copy me to your signature file and help me spread!

#2 lore raw

Hi Cedric,

Okay, thanx i will do that. I thought that netfilter-devel was only for development of netfilter itself. My bad ;)

By the way, are the iptables libs in /usr/lib/iptables meant for use by third party tools, or is this considered a dirty hack?

Regards,
Victor

Quoted 12 lines · Cedric Blancher
Quoted 1 line · Victor Julien

My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?

You could use iptables libs that stand in /usr/lib/iptables, just like iptables does.

You should however ask netfilter developpers mailing list.

#3 lore raw
Quoted 2 lines · Victor Julien

Okay, thanx i will do that. I thought that netfilter-devel was only for development of netfilter itself. My bad ;)

No, you can discuss third party tools development around iptables stuff also.

Quoted 2 lines

By the way, are the iptables libs in /usr/lib/iptables meant for use by third party tools, or is this considered a dirty hack?

They are meant to be used by iptables itself. But you can use them with your own soft. That's no dirty hack.

Signature · 2 lines
http://www.netexit.com/~sid/
PGP KeyID: 157E98EE FingerPrint: FA62226DA9E72FA8AECAA240008B480E157E98EE
Quoted 2 lines
Quoted 2 lines

Hi! I'm your friendly neighbourhood signature virus. Copy me to your signature file and help me spread!

#4 lore raw
In reply to Cedric Blancher
Quoted 8 lines · Cedric Blancher
Quoted 4 lines · Victor Julien

My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?

You could use iptables libs that stand in /usr/lib/iptables, just like iptables does.

Or actually the preferred interface for this type of operations is to use iptables-restore, the batch version of iptables. The speed difference from using iptables-restore and direct calls is pretty minimal by any means.

libiptc is an internal interface of the iptables source tree and is subject to change at any time. This should not be used directly unless you have very good reasons.

Regards
Henrik

#5 lore raw

Hi Hendrik,

This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?

Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?

Regards,
Victor

Quoted 19 lines · Henrik Nordstrom
Quoted 6 lines · Cedric Blancher
Quoted 1 line · Victor Julien

My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?

You could use iptables libs that stand in /usr/lib/iptables, just like iptables does.

Or actually the preferred interface for this type of operations is to use iptables-restore, the batch version of iptables. The speed difference from using iptables-restore and direct calls is pretty minimal by any means.

libiptc is an internal interface of the iptables source tree and is
subject to change at any time. This should not be used directly unless you have very good reasons.

Regards
Henrik

#6 lore raw

I use it all the time in the ISCS project -- just remember to use the -n argument. It is much, much faster. However, you need to supply the input file in the proper and not well documented format - John

Quoted 40 lines · Victor Julien

Hi Hendrik,

This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?

Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?

Regards,
Victor

Quoted 25 lines · Henrik Nordstrom
Quoted 9 lines · Cedric Blancher
Quoted 4 lines · Victor Julien

My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?

You could use iptables libs that stand in /usr/lib/iptables, just like iptables does.

Or actually the preferred interface for this type of operations is to use iptables-restore, the batch version of iptables. The speed difference from using iptables-restore and direct calls is pretty minimal by any means.

libiptc is an internal interface of the iptables source tree and is subject to change at any time. This should not be used directly unless you have very good reasons.

Regards
Henrik

Signature · 5 lines
John A. Sullivan III
Chief Technology Officer
Nexus Management
+1 207-985-7880
[email protected]
#7 lore raw
In reply to Victor Julien
Quoted 3 lines · Victor Julien

This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?

You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.

Quoted 4 lines

Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?

Such C function would only be marginally faster than having a pipe to iptables-restore.

iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.

iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences

Syntax for iptables-restore input is

*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.

:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.

COMMIT
Uploads the current table (specified by *tablename) to the kernel.

#....
Input lines starting with # are assumed to be comments and are ignored

Any other lines are assumed to be iptables commands per the iptables syntax specification.

iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.

Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.

Regards
Henrik

#8 lore raw
Quoted 52 lines · Henrik Nordstrom
Quoted 3 lines · Victor Julien

This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?

You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.

Quoted 4 lines

Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?

Such C function would only be marginally faster than having a pipe to iptables-restore.

iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.

iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences

Syntax for iptables-restore input is

*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.

:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.

COMMIT
Uploads the current table (specified by *tablename) to the kernel.

#....
Input lines starting with # are assumed to be comments and are ignored

Any other lines are assumed to be iptables commands per the iptables syntax specification.

iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.

Quoted 7 lines

Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.

Regards
Henrik

I found that there were a few other differences. As far as I can tell, you can only have one table section per file, in other words if I have a file such as
*nat
some rules
COMMIT
*filter
some more rules
COMMIT
*nat
some more rules
COMMIT

only the first set of NAT rules commits. I also found that I needed to use an explicit declaration of implicit matches, e.g., *filter
-A FORWARD -p 6 --dport 80 -j ACCEPT - does not work
-A FORWARD -p 6 -m tcp --dport 80 -j ACCEPT - does work COMMIT

I did find that the performance difference for even small rule sets was noticeable and, for large rule sets, it was absolutely stunning - John

Signature · 5 lines
John A. Sullivan III
Chief Technology Officer
Nexus Management
+1 207-985-7880
[email protected]
#9 lore raw
Quoted 14 lines · John A. Sullivan III

I found that there were a few other differences. As far as I can tell, you can only have one table section per file, in other words if I have a file such as

*nat
some rules
COMMIT
*filter
some more rules
COMMIT
*nat
some more rules
COMMIT

Quoted 1 line

only the first set of NAT rules commits.

If this is the case then it is a bug somewhere, most likely in libiptc..

seems to work fine here however.. tested with

*filter
-A INPUT -p tcp --dport 45 -j ACCEPT
COMMIT
*nat
-A PREROUTING -p tcp --dport 45 -j ACCEPT
COMMIT
*filter
-A OUTPUT -j ACCEPT
COMMIT

and all three rules was added just fine.

also I do not see any reason why it should not work. The libiptc handle is completely closed on COMMIT so there should not be any memory of you already have used this table when the new *tablename is reached.

Note: when doing this you MUST use the --noflush (-n) option. If not the second reference to the table will clear the rules added by the first..

Quoted 7 lines

I also found that I needed to use an explicit declaration of implicit matches, e.g.,

*filter

(quoted diff) +0 -2
-A FORWARD -p 6 --dport 80 -j ACCEPT - does not work
-A FORWARD -p 6 -m tcp --dport 80 -j ACCEPT - does work

COMMIT

Works fine here without.. and considering that the exact same iptables.c is used in both commands for parsing the commands I don't see why it should not work in any version of iptables-restore..

The fact that iptables-save prints the protocol match is only due to the fact that this is actually what the rules looks like if you are protocol match options.. there is some heavy duty "magics" in iptables command parsing if a protocol is specified but no protocol match. There is an interesting comment in iptables.c which summarises this magic protocol match function quite well ;-)

/* If you listen carefully, you can
actually hear this code suck. */

Regards
Henrik

#10 lore raw
Quoted 60 lines · Henrik Nordstrom
Quoted 14 lines · John A. Sullivan III

I found that there were a few other differences. As far as I can tell, you can only have one table section per file, in other words if I have a file such as

*nat
some rules
COMMIT
*filter
some more rules
COMMIT
*nat
some more rules
COMMIT

Quoted 1 line

only the first set of NAT rules commits.

If this is the case then it is a bug somewhere, most likely in libiptc..

seems to work fine here however.. tested with

*filter
-A INPUT -p tcp --dport 45 -j ACCEPT
COMMIT
*nat
-A PREROUTING -p tcp --dport 45 -j ACCEPT
COMMIT
*filter
-A OUTPUT -j ACCEPT
COMMIT

and all three rules was added just fine.

also I do not see any reason why it should not work. The libiptc handle is completely closed on COMMIT so there should not be any memory of you already have used this table when the new *tablename is reached.

Note: when doing this you MUST use the --noflush (-n) option. If not the second reference to the table will clear the rules added by the first..

Quoted 7 lines

I also found that I needed to use an explicit declaration of implicit matches, e.g.,

*filter

(quoted diff) +0 -2
-A FORWARD -p 6 --dport 80 -j ACCEPT - does not work
-A FORWARD -p 6 -m tcp --dport 80 -j ACCEPT - does work

COMMIT

Works fine here without.. and considering that the exact same iptables.c is used in both commands for parsing the commands I don't see why it should not work in any version of iptables-restore..

The fact that iptables-save prints the protocol match is only due to the fact that this is actually what the rules looks like if you are protocol match options.. there is some heavy duty "magics" in iptables command parsing if a protocol is specified but no protocol match. There is an interesting comment in iptables.c which summarises this magic protocol match function quite well ;-)

Quoted 6 lines

/* If you listen carefully, you can
actually hear this code suck. */

Regards
Henrik

Interesting. Thanks for the update. I know the last time I tested was with a pretty old iptables - I don't know the version number as I have long since upgraded. Perhaps I was braincramping but I'm almost certain it choked if I didn't use the explicit match. Thanks again - John

Signature · 5 lines
John A. Sullivan III
Chief Technology Officer
Nexus Management
+1 207-985-7880
[email protected]
#11 lore raw
In reply to Henrik Nordstrom

Just curious, having made my own decision to not use the kinda-interfaces and just call the provided iptables executables; has anyone run valgrind
(http://valgrind.kde.org/) on iptables? I did a simple one like:

valgrind --leak-check=yes iptables -A PREROUTING -t mangle -s 192.168.1.0/24 -j DROP

and got a few...I would be cautious about using these routines in a long-running app unless they were looked at more.

__________________________________
Do you Yahoo!?
Yahoo! Mail - More reliable, more storage, less spam
http://mail.yahoo.com

#12 lore raw
Quoted 6 lines · Scott MacKay

valgrind --leak-check=yes iptables -A PREROUTING -t mangle -s 192.168.1.0/24 -j DROP

and got a few...I would be cautious about using these routines in a long-running app unless they were looked at more.

libiptc is not really intended for long-running applications. The only official use of this is iptables and iptables-restore which usually does not run for a long time, so there has not been very much attention to memory leaks etc.. but bugfixes are usually accepted when found.

Note that the design of libiptc will indicate some static memory leaks due to modules being loaded etc unless you compile it as a single binary. These leaks are not interesting to have fixed. The type of leaks interesting to have fixed is the leaks which grows over time as more and more operations have been executed.

Regards
Henrik

#13 lore raw
In reply to Henrik Nordstrom
Quoted 44 lines · Henrik Nordstrom
Quoted 1 line · Victor Julien

This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?

You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.

Quoted 3 lines

Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?

Such C function would only be marginally faster than having a pipe to iptables-restore.

iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.

iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences

Syntax for iptables-restore input is

*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.

:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.

COMMIT
Uploads the current table (specified by *tablename) to the kernel.

#....
Input lines starting with # are assumed to be comments and are ignored

Any other lines are assumed to be iptables commands per the iptables syntax specification.

iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.

Quoted 9 lines

Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.

Regards
Henrik

Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':

*filter
:FORWARD DROP
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT COMMIT

and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):

*filter
:FORWARD DROP
-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT -I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT COMMIT

and then 'iptables-restore -n', right?

But the easiest way is to recreate the initial ruleset with the updated rules would be:

*filter
:FORWARD DROP
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT -A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT COMMIT

and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?

The last method should still be way faster than my current method, i guess. Is this right?

Regards,
Victor

#14 lore raw
Quoted 17 lines · Victor Julien

Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):

*filter
:FORWARD DROP

(quoted diff) +0 -2
-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
Quoted 1 line

COMMIT

Or you could

*filer
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT COMMIT

Quoted 1 line

and then 'iptables-restore -n', right?

Right. Or you could even have iptables-restore -n running on a pipe already.. (but beware of memoryleaks in libiptc)

Quoted 9 lines

But the easiest way is to recreate the initial ruleset with the updated rules would be:

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

Depends on your application, but yes I find this easier most times.

Quoted 2 lines

The last method should still be way faster than my current method, i guess. Is this right?

iptables-restore is quicker than iptables as soon as you have more than one modification to the table. On single-rule operations it is identical to iptables.

if you use a pipe to a already running iptables-restore then it isfaster than iptables even on single rule operations.

In addition iptables-restore allows you to atomically replace the complete ruleset without race windows. This can not be done with iptables and with iptables packets will be forwarded while the new ruleset is only partiailly installed...

Regards
Henrik

#15 lore raw
Quoted 23 lines · Henrik Nordstrom
Quoted 15 lines · Victor Julien

Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):

*filter
:FORWARD DROP

(quoted diff) +0 -2
-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
Quoted 1 line

COMMIT

Or you could

Quoted 9 lines

*filer
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT COMMIT

Quoted 1 line

and then 'iptables-restore -n', right?

Right. Or you could even have iptables-restore -n running on a pipe already.. (but beware of memoryleaks in libiptc)

Okay, i think i'll rather open the pipe on demand.

Quoted 11 lines
Quoted 8 lines

But the easiest way is to recreate the initial ruleset with the updated rules would be:

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

Depends on your application, but yes I find this easier most times.

Good! But if i have a rule for accounting, how does it work then? If you consider this (useless) example:

*filter
:FORWARD DROP

# account the traffic for 192.168.0.1
-A FORWARD -p tcp -s 192.168.0.1
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
COMMIT

if i now want to reload the rule set to

*filter
:FORWARD DROP

# account the traffic for 192.168.0.1
-A FORWARD -p tcp -s 192.168.0.1
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.12 --dport 80 -j ACCEPT
COMMIT

using 'iptables-restore' (without noflush) is there a way to preserve the accounting data from the first rule? Is this what the -c option of iptables-restore is for?

Quoted 6 lines
Quoted 1 line

The last method should still be way faster than my current method, i guess. Is this right?

iptables-restore is quicker than iptables as soon as you have more than one modification to the table. On single-rule operations it is identical to iptables.

Quoted 7 lines

if you use a pipe to a already running iptables-restore then it isfaster than iptables even on single rule operations.

In addition iptables-restore allows you to atomically replace the complete ruleset without race windows. This can not be done with iptables and with iptables packets will be forwarded while the new ruleset is only partiailly installed...

Sounds like what i'm looking for...

Regards,
Victor

#16 lore raw
Quoted 2 lines · Victor Julien

using 'iptables-restore' (without noflush) is there a way to preserve the accounting data from the first rule?

Then you must use incremental commands, keepint the accounting rule untouched.

Quoted 1 line

Is this what the -c option of iptables-restore is for?

No. This instructs iptables-restore to restore any packet/byte counters found in the input data. Without this option any such counters is ignored and the installed rules have zeroed counters.

Regards
Henrik

#17 lore raw
Quoted 6 lines · Henrik Nordstrom
Quoted 1 line · Victor Julien

using 'iptables-restore' (without noflush) is there a way to preserve the accounting data from the first rule?

Then you must use incremental commands, keepint the accounting rule untouched.

Hmmm, that i don't want.

Quoted 4 lines
Quoted 2 lines

Is this what the -c option of iptables-restore is for?

No. This instructs iptables-restore to restore any packet/byte counters found in the input data. Without this option any such counters is ignored and the installed rules have zeroed counters.

So i have to read the current counters first by parsing 'iptables-save'-output. And i can then recreate the rules and set the counters to their old values?

Using 'iptables-restore' can i flush one chain, but leave another untouched? If so, i can create an accounting chain which will not be flushed...

Regards,
Victor

Quoted 4 lines

Regards
Henrik

#18 lore raw
Quoted 4 lines · Victor Julien
Quoted 2 lines

Then you must use incremental commands, keepint the accounting rule untouched.

Hmmm, that i don't want.

Then your other option is to dump the existing rule, parse the counter and include this in the input.

Quoted 3 lines

So i have to read the current counters first by parsing 'iptables-save'-output. And i can then recreate the rules and set the counters to their old values?

Yes.

Quoted 2 lines

Using 'iptables-restore' can i flush one chain, but leave another untouched?

Yes, by using disabling the implicit flush (--noflush option) and then use the flush command explicitly.

Quoted 2 lines

If so, i can create an accounting chain which will not be flushed...

True. Probably much better approach than what is discussed above.

Regards
Regards
Henrik

#19 lore raw
Quoted 21 lines · Henrik Nordstrom
Quoted 3 lines · Victor Julien
Quoted 1 line

Then you must use incremental commands, keepint the accounting rule untouched.

Hmmm, that i don't want.

Then your other option is to dump the existing rule, parse the counter and include this in the input.

Quoted 1 line

So i have to read the current counters first by parsing 'iptables-save'-output. And i can then recreate the rules and set the counters to their old values?

Yes.

Quoted 1 line

Using 'iptables-restore' can i flush one chain, but leave another untouched?

Quoted 7 lines

Yes, by using disabling the implicit flush (--noflush option) and then use the flush command explicitly.

Quoted 1 line

If so, i can create an accounting chain which will not be flushed...

True. Probably much better approach than what is discussed above.

So the best option would be to create an accounting chain (e.g ACCOUNT). If i, for example, want to recreate the rules in the INPUT chain it should work like this:

Create a file for iptables-restore input:

*filter
:INPUT DROP

# flush this chain
-F INPUT
# now recreate the rules
-A INPUT -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
...more rules...
COMMIT

Now i would have to call 'iptables-restore --noflush'. This will clear the entire INPUT chain, but will leave all other (including my ACCOUNT chain) chains untouched.

Right?

Regards,
Victor

#20 lore raw
Quoted 8 lines · Victor Julien

So the best option would be to create an accounting chain (e.g ACCOUNT). If i, for example, want to recreate the rules in the INPUT chain it should work like this:

Create a file for iptables-restore input:

*filter
:INPUT DROP

This line is redundant. You only need policy lines if you want to change the policy.

Quoted 10 lines
# flush this chain
-F INPUT
# now recreate the rules
-A INPUT -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
...more rules...
COMMIT

Now i would have to call 'iptables-restore --noflush'. This will clear the entire INPUT chain, but will leave all other (including my ACCOUNT chain) chains untouched.

Correct.

But dont forget to insert a jump to the ACCOUNT chain where needed..

*filter
-F INPUT
-A INPUT -j ACCOUNT
-A INTPUT -p tcp .... -j ACCEPT
....
COMMIT

Regards
Henrik

#21 lore raw
Quoted 8 lines · Henrik Nordstrom
Quoted 6 lines

So the best option would be to create an accounting chain (e.g ACCOUNT). If i, for example, want to recreate the rules in the INPUT chain it should work like this:

Create a file for iptables-restore input:

*filter
:INPUT DROP

This line is redundant. You only need policy lines if you want to change the policy.

But i do need it if i want to (re)set chain counters, right?

Like this:
:INPUT DROP [123:45678]

Quoted 19 lines
Quoted 9 lines
# flush this chain
-F INPUT
# now recreate the rules
-A INPUT -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
...more rules...
COMMIT

Now i would have to call 'iptables-restore --noflush'. This will clear the entire INPUT chain, but will leave all other (including my ACCOUNT chain) chains untouched.

Correct.

But dont forget to insert a jump to the ACCOUNT chain where needed..

*filter

(quoted diff) +0 -3
-F INPUT
-A INPUT -j ACCOUNT
-A INTPUT -p tcp .... -j ACCEPT
....
Quoted 2 lines

COMMIT

Well, i think i know all i need now, thanks a lot for your help! Now i stand for the daunting task of implmenting this in my current codebase ;-)

Well i guess that will have to wait a bit, my current version is quite stable (for me a least) and i plan to release it soon...

Regards,
Victor

#22 lore raw
Quoted 7 lines · Victor Julien
Quoted 2 lines

This line is redundant. You only need policy lines if you want to change the policy.

But i do need it if i want to (re)set chain counters, right?

Like this:
:INPUT DROP [123:45678]

Yes.

Or if you don't the old counters should be kept.

Regards
Henrik

#23 lore raw
In reply to Victor Julien
Quoted 86 lines · Victor Julien
Quoted 56 lines · Henrik Nordstrom
Quoted 3 lines · Victor Julien

This might be a big improvement, but it leaves me with one possible problem. When adding and removing rules on-the-fly i can't use this method, right?

You can. There is a --noflush option to iptables-restore to not flush the existing rules, allowing you to use iptables-restore to add/modify/delete existing rules without having to specify the whole ruleset.

Quoted 4 lines

Wouldn't it be nice if there was an c function which i could call, which would do all the checking and other stuff the commandline iptables does,
but, because its a c-function, way faster? Would it be easy (or even
possible) to implement such a function?

Such C function would only be marginally faster than having a pipe to iptables-restore.

iptables-restore has the exact same capabilities as iptables, only difference is that iptables-restore is batch oriented compiling all your changes and then uploading them into the kernel in one single call while iptables makes one modification to the table per call.

iptables-restore accepts the exact same set of commands as iptables with only some minor syntactical sugar differences

Syntax for iptables-restore input is

*tablename
start working on rules in table "tablename". Equivalent to the -t option to iptables.

:chainname policy
specifies the default policy for chain "chainname". Equivalent to the iptables -P command. Probably also works just fine to use the -P command but I have not tried.

COMMIT
Uploads the current table (specified by *tablename) to the kernel.

#....
Input lines starting with # are assumed to be comments and are ignored

Any other lines are assumed to be iptables commands per the iptables syntax specification.

iptables commands may be prefixed by [packetcnt:bytecnt] which gets automatically translated into --set-counters packetcnt bytecnt before the command is executed within iptables-restore. The two syntaxes are equivalent.

Quoted 9 lines

Don't be fooled by the fact that iptables-save only uses the -A command or by the name iptables-restore. What iptables-restore is is a batch version of iptables with all the capabilities of iptables, not just a tool to restore complete tables saved by iptables-save.

Regards
Henrik

Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):

*filter
:FORWARD DROP

(quoted diff) +0 -2
-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT

COMMIT

Quoted 1 line

and then 'iptables-restore -n', right?

I believe that is correct although I also believe you can dispense with the :FORWARD DROP since you already had it in the first rule set and could also use
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport -j ACCEPT
to just replace the rule although I confess to never having tried that in an iptables-restore file

Quoted 11 lines

But the easiest way is to recreate the initial ruleset with the updated rules would be:

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

Quoted 3 lines

and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?

Yes, although when ISCS is released (http://iscs.sourceforge.net), it will provide an alternative to having to track rule order (massive oversimplification here but it is not the topic at hand).

Quoted 3 lines

The last method should still be way faster than my current method, i guess. Is this right?

I have found it to be dramatically faster - John

Quoted 3 lines

Regards,
Victor

Signature · 3 lines
Open Source Development Corporation
Financially Sustainable open source development
http://www.opensourcedevelopmentcorp.com
#24 lore raw

John A. Sullivan III wrote:

<snip>

Quoted 36 lines
Quoted 18 lines

Okay, lets see if I understand what you mean. Say i have an initial ruleset which looks like this, loaded with 'iptables-restore':

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

and sometime later i want to replace 192.168.0.2 by 192.168.0.4 (in exacty the same place):

*filter
:FORWARD DROP

(quoted diff) +0 -2
-D FORWARD -p tcp -s 192.168.0.2 --dport 80 -j ACCEPT
-I FORWARD 2 -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT

COMMIT

Quoted 1 line

and then 'iptables-restore -n', right?

I believe that is correct although I also believe you can dispense with the :FORWARD DROP since you already had it in the first rule set and could also use
-R FORWARD 2 -p tcp -s 192.168.0.4 --dport -j ACCEPT
to just replace the rule although I confess to never having tried that in an iptables-restore file

Quoted 9 lines

But the easiest way is to recreate the initial ruleset with the updated rules would be:

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

Quoted 5 lines
Quoted 1 line

and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?

Yes, although when ISCS is released (http://iscs.sourceforge.net), it will provide an alternative to having to track rule order (massive oversimplification here but it is not the topic at hand).

But how will ISCS handle this problem? Will it also create input for iptables-restore? Or do you have some other method?

Quoted 4 lines
Quoted 2 lines

The last method should still be way faster than my current method, i guess. Is this right?

I have found it to be dramatically faster - John

Good!

Regards,
Victor

Quoted 3 lines
Quoted 3 lines

Regards,
Victor

#25 lore raw
Quoted 5 lines · Victor Julien

John A. Sullivan III wrote:

<snip>

Quoted 1 line
Quoted 1 line

<snip>

Quoted 18 lines
Quoted 10 lines
Quoted 10 lines

But the easiest way is to recreate the initial ruleset with the updated rules would be:

*filter
:FORWARD DROP

(quoted diff) +0 -3
-A FORWARD -p tcp -s 192.168.0.1 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.4 --dport 80 -j ACCEPT
-A FORWARD -p tcp -s 192.168.0.3 --dport 80 -j ACCEPT

COMMIT

Quoted 7 lines
Quoted 3 lines

and then just call iptables-restore. This way i wont have to calculate where i want to insert the rules, this can be quite complex on many changes in large rulessets. Is this correct?

Yes, although when ISCS is released (http://iscs.sourceforge.net), it will provide an alternative to having to track rule order (massive oversimplification here but it is not the topic at hand).

Quoted 2 lines

But how will ISCS handle this problem? Will it also create input for iptables-restore? Or do you have some other method?

Yes, it creates input files for iptables restore and is also able to create configuration files for other firewalls (as well as the VPN configuration, the router configuration, the NAT configuration and any local DHCP configuration and, hopefully soon, all the layer2 configuration for the individual gateways). However, rather than relying exclusively upon rule order, it uses an approach we call "Best Match". This allows us to automate the creation of all rules from a high level description of the environment. The idea is to eliminate the need for an administrator to create rules. Instead, we interpret the environment, e.g., "give these three teams access to the new joint development project data", and create consistent rules for access control, user authentication, encryption, data authentication, routing, NAT, etc, to produce that environment.

Quoted 15 lines
Quoted 6 lines
Quoted 4 lines

The last method should still be way faster than my current method, i guess. Is this right?

I have found it to be dramatically faster - John

Good!

Regards,
Victor

Quoted 3 lines
Quoted 3 lines

Regards,
Victor

Signature · 5 lines
John A. Sullivan III
Chief Technology Officer
Nexus Management
+1 207-985-7880
[email protected]
#26 lore raw

Adding packet metadata

In reply to Henrik Nordstrom

Hello,
I wanted to mark up packets with information as they pass thru the system. Basically, I want to associate some data with a packet during PREROUTING/raw (to read info prior to conntrack) and then get that information back during PREROUTING/mangle. Is there a module already written which could perform this? It is not something I would need to have accessible to the iptables program, though I will probably need to adjust some code to make it accessible to my ip_queue userspace application...

-Scott

__________________________________
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html

#27 lore raw

Re: Adding packet metadata

Quoted 5 lines · Scott MacKay

I wanted to mark up packets with information as they pass thru the system. Basically, I want to associate some data with a packet during PREROUTING/raw (to read info prior to conntrack) and then get that information back during PREROUTING/mangle. Is there a module already written which could perform this?

MARK should be able to do this.

Quoted 3 lines

It is not something I would need to have accessible to the iptables program, though I will probably need to adjust some code to make it accessible to my ip_queue userspace application...

The MARK value is available (read-only) via ip_queue.

Regards
Henrik

#28 lore raw

Re: Adding packet metadata

Quoted 10 lines
Quoted 6 lines

I wanted to mark up packets with information as they pass thru the system. Basically, I want to associate some data with a packet during PREROUTING/raw (to read info prior to conntrack) and then get that information back during PREROUTING/mangle. Is there a module already written which could perform this?

MARK should be able to do this.

An interesting question about MARK field in SKB structure is: Can I use all 32-bit of "nfmark" for adding metadata? Iptables front-end (from command line) seems to take mark value as a 8-bit number.

Mike-Ro-Chanel

#29 lore raw

Re: Adding packet metadata

Quoted 2 lines · Mike-Ro-Chanel

An interesting question about MARK field in SKB structure is: Can I use all 32-bit of "nfmark" for adding metadata?

Yes.

Quoted 2 lines

Iptables front-end (from command line) seems to take mark value as a 8-bit number.

What makes you think so?

There is a slight signed bug in the string_to_number function which prevents the use of the 2^31 bit, easily fixed by changing it to use unsigned.

Regards
Henrik

#30 lore raw

Re: Adding packet metadata

Quoted 6 lines · Henrik Nordstrom
Quoted 1 line · Mike-Ro-Chanel

An interesting question about MARK field in SKB structure is: Can I use all 32-bit of "nfmark" for adding metadata?

Quoted 8 lines

Yes.

Quoted 2 lines

Iptables front-end (from command line) seems to take mark value as a 8-bit number.

What makes you think so?

Now I understand, it is my error on iptables usage.

Quoted 2 lines

There is a slight signed bug in the string_to_number function which prevents the use of the 2^31 bit, easily fixed by changing it to use unsigned.

Ok, thanks for the tip!

Mike-Ro-Chanel

#31 lore raw

Re: Adding packet metadata

In reply to Henrik Nordstrom
Quoted 5 lines · --- Henrik Nordstrom
Quoted 1 line · Scott MacKay

I wanted to mark up packets with information as

they pass thru the

Quoted 1 line

system. Basically, I want to associate some data

Quoted 8 lines

with a packet during

Quoted 1 line

PREROUTING/raw (to read info prior to conntrack)

and then get that

Quoted 1 line

information back during PREROUTING/mangle. Is

there a module already

Quoted 1 line

written which could perform this?

MARK should be able to do this.

I actually was thinking of using the MARK for a different purpose unfortunantly. Since it is a nice 32bit word, I was thinking of even using only part of it, but I need it for setting up some match settings for actual rules.

Quoted 8 lines
Quoted 1 line

It is not something I would need to have

accessible to the iptables

Quoted 1 line

program, though I will probably need to adjust

some code to make it

Quoted 1 line

accessible to my ip_queue userspace application...

The MARK value is available (read-only) via
ip_queue.

That would be my general problem. The only bit that seems transmitted back is the payload for evaluation and update. Even if I wanted to use MARK, it would not allow me to alter the value.

__________________________________
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html

#32 lore raw

Re: Adding packet metadata

Quoted 5 lines · Scott MacKay

I actually was thinking of using the MARK for a different purpose unfortunantly. Since it is a nice 32bit word, I was thinking of even using only part of it, but I need it for setting up some match settings for actual rules.

Then you only need a new MARK target as the existing one does not support masked operations (only the mark match does..).

Quoted 6 lines
Quoted 2 lines

The MARK value is available (read-only) via
ip_queue.

That would be my general problem. The only bit that seems transmitted back is the payload for evaluation and update. Even if I wanted to use MARK, it would not allow me to alter the value.

There is a patch somewhere to extend ip_queue to be able to return the mark.. and no matter what approach you take something similar is needed if you from ip_queue want to return other information than the verdict and optional payload substitution.

Regards
Henrik

#33 lore raw
In reply to Cedric Blancher
Quoted 6 lines · Cedric Blancher
Quoted 4 lines · Victor Julien

My program (written in c) creates rules by opening a pipe to /sbin/iptables. However this is quite slow with large rulessets and on slow hardware. Is there another way, like an iptables librarycall or something?

Quoted 6 lines

You could use iptables libs that stand in /usr/lib/iptables, just like iptables does.

You should however ask netfilter developpers mailing list.

The above solution is probably better than mine but I write out my new rules to a file in iptables-save format and then use iptables-restore -n <filename>. This is dramatically faster than calling iptables - John

Signature · 9 lines
John A. Sullivan III
Chief Technology Officer
Nexus Management
+1 207-985-7880
[email protected]
---
If you are interested in helping to develop a GPL enterprise class
VPN/Firewall/Security device management console, please visit
http://iscs.sourceforge.net