Skip to contentSkip to conversation

pgsql-ulogd2

DISCUSSION Mr Dash Four 4 participants 18 messages Jul 13, 2012 – Jul 17, 2012 netfilter

lore
Share & tools
lore
#0 lore raw

I just came across the pgsql script for the ulogd2 daemon supplied with the latest sources and since I intend to deploy it (upgrading my system from syslog-ng) I thought to ask about a couple of ideas I have.

As I see it, the script does not have any security/permission policies created or implemented. Is such feature planned?

If not, I think I have enough PostgreSQL experience and could alter that script to include such implementation, though I might need help with the NFLOG/ULOGD2 part as I am fairly new to this.

The idea I have is that the ulogd2 daemon should only be allowed INSERT permissions (nothing else) to the log tables, so that even if someone is able to hijack the ulogd2 connection to PostgreSQL somehow, they won't be able to see what has been logged, let alone alter it or delete it.

For certain views, I am sure there is a need for SELECT permission and for others there would even be a need for USAGE or REFERENCES privileges.

I tried to email the author of that script (Pierre - [email protected]), but my emails are not getting through for some reason. Thanks!

#1 lore raw

Hello,

Quoted 3 lines · Mr Dash Four

I just came across the pgsql script for the ulogd2 daemon supplied with the latest sources and since I intend to deploy it (upgrading my system from syslog-ng) I thought to ask about a couple of ideas I have.

As I see it, the script does not have any security/permission policies created or implemented. Is such feature planned?

No work is planned on that but it is a good idea.

Quoted 2 lines

If not, I think I have enough PostgreSQL experience and could alter that script to include such implementation, though I might need help with the NFLOG/ULOGD2 part as I am fairly new to this.

I'm here to help ;)

Quoted 1 line

The idea I have is that the ulogd2 daemon should only be allowed INSERT permissions (nothing else) to the log tables, so that even if someone is able to hijack the ulogd2 connection to PostgreSQL somehow, they won't be able to see what has been logged, let alone alter it or delete it.

That's a sane setup.

Quoted 4 lines

For certain views, I am sure there is a need for SELECT permission and for others there would even be a need for USAGE or REFERENCES privileges.

I tried to email the author of that script (Pierre - [email protected]), but my emails are not getting through for some reason. Thanks!

INL company is dead but I'm sure that Pierre is still reading this ML from another mail ;)

BR,

Signature · 2 lines
Eric Leblond 
Blog: http://home.regit.org/ - Portfolio: http://regit.500px.com/
#2 lore raw
Quoted 3 lines

No work is planned on that but it is a good idea.

Yeah, I thought as much.

Quoted 5 lines
Quoted 2 lines

If not, I think I have enough PostgreSQL experience and could alter that script to include such implementation, though I might need help with the NFLOG/ULOGD2 part as I am fairly new to this.

I'm here to help ;)

OK, for now I have some general queries with regards to ulogd2. When I gain more knowledge (and experience) my queries will become more specific. For now though I have to sort out the following:

In the manual there is a very good example on how to use NFLOG (as a iptables target), but I have no idea how to apply and select NFACCT and NFCT. How do I use this?

OK, I defined 2 stacks which use both NFCT and NFACCT, but how do I tell iptables how to use them? I presume this is in a similar way to NFLOG (with a target), but in that manual there are no examples (or even hints) given on how to use that.

Also, there are quite a few plugins which I have just a vague idea what they do (they are not documented in the manual - I presume because they were recently introduced) - GPRINT springs to mind, NACCT is the other. Is there a description on the purpose of these and their configuration parameters, if any?

Probably my last 2 queries for now: when I define a stack, I use a series of plugins to define a chain - from the source of where the packet originates, to a cascade of "filters" and then the output plugin.

Does the order of "filter" plugins matters (in other words, are they traversed from left-to-right in the order they are specified in the "stack" statement in ulogd.conf) or is this order irrelevant?

And the second query: usually every plugin, when defined in a stack statement has its own section when the various plugin parameters are defined. Say, "emu1" has its own "[emu1]" section in ulogd.conf and there I could define various parameters related to the LOGEMU plug in.

There are some plugins though, which do not have such section, which follows me to conclude that either defining that section is not mandatory (in which case I presume the plugin "assumes" default values) or, the plugin in question does not have any values to be specified, so no such section is needed. Am I correct in this assumption?

Quoted 5 lines
Quoted 2 lines

The idea I have is that the ulogd2 daemon should only be allowed INSERT permissions (nothing else) to the log tables, so that even if someone is able to hijack the ulogd2 connection to PostgreSQL somehow, they won't be able to see what has been logged, let alone alter it or delete it.

That's a sane setup.

I will dig much more deeper into this once I know how ulogd works as my knowledge of ulogd2 is not that great (yet!). My initial instinct is that I may have to alter this script slightly, because from what I gather ulogd2 is "sensing" the structure of the logging table by reading the "description" of it in PostgreSQL. That particular operation usually requires higher-than-needed privileges and in order to avoid that I might need to find other ways to impose the "insert-only" restriction, which, lets be clear, is what ulogd needs. Again, I might be wrong with this, but that is what my initial observations with the script and ulogd suggest.

Quoted 3 lines

INL company is dead but I'm sure that Pierre is still reading this ML from another mail ;)

Ah, that explains it! Well, if he reads this thread, I would love to get some feedback when I am confident enough to start digging into that script. ;-)

#3 lore raw

Hello,

Quoted 9 lines · Mr Dash Four
Quoted 2 lines

No work is planned on that but it is a good idea.

Yeah, I thought as much.

Quoted 5 lines
Quoted 2 lines

If not, I think I have enough PostgreSQL experience and could alter that script to include such implementation, though I might need help with the NFLOG/ULOGD2 part as I am fairly new to this.

I'm here to help ;)

Quoted 7 lines

OK, for now I have some general queries with regards to ulogd2. When I gain more knowledge (and experience) my queries will become more specific. For now though I have to sort out the following:

In the manual there is a very good example on how to use NFLOG (as a iptables target), but I have no idea how to apply and select NFACCT and NFCT. How do I use this?

For NFCT, you simply need to have nfnetlink_conntrack loaded.

Things are more complicated for NFACCT and I've just wrote a page dedicated to this point:
https://home.regit.org/2012/07/flow-accounting-with-netfilter-and-ulogd2/

Quoted 9 lines

OK, I defined 2 stacks which use both NFCT and NFACCT, but how do I tell iptables how to use them? I presume this is in a similar way to NFLOG (with a target), but in that manual there are no examples (or even hints) given on how to use that.

Also, there are quite a few plugins which I have just a vague idea what they do (they are not documented in the manual - I presume because they were recently introduced) - GPRINT springs to mind, NACCT is the other.

GPRINT is a one line per item textual log. NACCT is an tab separated output format dedicated to NFACCT data and that can be used to import data in other tools (Astaro does so).

Quoted 2 lines

Is there a description on the purpose of these and their configuration parameters, if any?

To now which keys are used and whch config var are available, you can use the -i option of ulogd:

ulogd -i libexec/ulogd/ulogd_filter_MARK.so
Name: MARK
Config options:
Var: mark (Integer, Default: 0)
Var: mask (Integer, Default: -1)
Input keys:
Key: ct.mark (unsigned int 32, optional)
Key: oob.mark (unsigned int 32, optional)
Output keys:
No statically defined keys

Quoted 7 lines

Probably my last 2 queries for now: when I define a stack, I use a series of plugins to define a chain - from the source of where the packet originates, to a cascade of "filters" and then the output plugin.

Does the order of "filter" plugins matters (in other words, are they traversed from left-to-right in the order they are specified in the "stack" statement in ulogd.conf) or is this order irrelevant?

This is read from end to start. All that matter is that for a plugin the needed keys are exported by another plugin on the left.

Quoted 10 lines

And the second query: usually every plugin, when defined in a stack statement has its own section when the various plugin parameters are defined. Say, "emu1" has its own "[emu1]" section in ulogd.conf and there I could define various parameters related to the LOGEMU plug in.

There are some plugins though, which do not have such section, which follows me to conclude that either defining that section is not mandatory (in which case I presume the plugin "assumes" default values) or, the plugin in question does not have any values to be specified, so no such section is needed. Am I correct in this assumption?

Yes, there is default value. But for at least INPUT plugins, you need to specify a section. I really don't remember why but this is the case.

Quoted 6 lines
Quoted 6 lines
Quoted 3 lines

The idea I have is that the ulogd2 daemon should only be allowed INSERT permissions (nothing else) to the log tables, so that even if someone is able to hijack the ulogd2 connection to PostgreSQL somehow, they won't be able to see what has been logged, let alone alter it or delete it.

That's a sane setup.

Quoted 10 lines

I will dig much more deeper into this once I know how ulogd works as my knowledge of ulogd2 is not that great (yet!). My initial instinct is that I may have to alter this script slightly, because from what I gather ulogd2 is "sensing" the structure of the logging table by reading the "description" of it in PostgreSQL. That particular operation usually requires higher-than-needed privileges and in order to avoid that I might need to find other ways to impose the "insert-only" restriction, which, lets be clear, is what ulogd needs. Again, I might be wrong with this, but that is what my initial observations with the script and ulogd suggest.

Yes, there's a discovery system that may require some tuning. But it can be made on a separate table use for the only purpose of describing the variables.

Quoted 7 lines
Quoted 3 lines

INL company is dead but I'm sure that Pierre is still reading this ML from another mail ;)

Ah, that explains it! Well, if he reads this thread, I would love to get some feedback when I am confident enough to start digging into that script. ;-)

BR,

Signature · 2 lines
Eric Leblond 
Blog: http://home.regit.org/ - Portfolio: http://regit.500px.com/
#4 lore raw
Quoted 3 lines

For NFCT, you simply need to have nfnetlink_conntrack loaded.

I did, but I also made the mistake of including a few filters in that stack, which were incompatible and that was the reason I did not get any NFCT logs. Once that was corrected I started seeing connection tracking logged.

I have another question with regards to this: Is it possible to limit (by a separate filter or otherwise) the reporting and restrict it, to say, a specific set of interfaces or specific source/destination IP addresses/subnets?

Currently, NFCT reports absolutely everything, which is not what I really want as I have to sift through thousands of logs, not to mention that by reporting everything the system load is much higher.

So, is there a way for me to do that, somehow?

Quoted 4 lines

Things are more complicated for NFACCT and I've just wrote a page dedicated to this point:
https://home.regit.org/2012/07/flow-accounting-with-netfilter-and-ulogd2/

Very helpful, thanks! I don't see it as complicated at all, certainly no more complicated than the existing accounting techniques.

Quoted 6 lines
Quoted 2 lines

Is there a description on the purpose of these and their configuration parameters, if any?

To now which keys are used and whch config var are available, you can use the -i option of ulogd:

Quoted 10 lines

ulogd -i libexec/ulogd/ulogd_filter_MARK.so Name: MARK Config options:
Var: mark (Integer, Default: 0)
Var: mask (Integer, Default: -1)
Input keys:
Key: ct.mark (unsigned int 32, optional)
Key: oob.mark (unsigned int 32, optional)
Output keys:
No statically defined keys

Or look at the source code, which is what I did at the end.

Quoted 7 lines
Quoted 2 lines

I will dig much more deeper into this once I know how ulogd works as my knowledge of ulogd2 is not that great (yet!). My initial instinct is that I may have to alter this script slightly, because from what I gather ulogd2 is "sensing" the structure of the logging table by reading the "description" of it in PostgreSQL. That particular operation usually requires higher-than-needed privileges and in order to avoid that I might need to find other ways to impose the "insert-only" restriction, which, lets be clear, is what ulogd needs. Again, I might be wrong with this, but that is what my initial observations with the script and ulogd suggest.

Yes, there's a discovery system that may require some tuning. But it can be made on a separate table use for the only purpose of describing the variables.

Yep, the pgsql plugin makes extensive use of pg_namespace, pg_class and pg_attribute which are system tables. These contain definitions of every single object registered on that database server and is a major security risk (as I pointed out, if that ulogd connection to the database server is hijacked, then the attacker could find out what is on that database without any problems, which is not good).

I had in mind exactly what you've suggested above - use a separate, manually-registered table containing the table columns and their mapping to ulogd2 parameters - much less risk and everything is configurable, though the downside is that the two tables need to be synchronised if the structure of the main ulogd table changes (columns renamed or added).

#5 lore raw
Quoted 4 lines

Yep, the pgsql plugin makes extensive use of pg_namespace, pg_class and pg_attribute which are system tables. These contain definitions of every single object registered on that database server and is a major security risk (as I pointed out, if that ulogd connection to the database server is hijacked, then the attacker could find out what is on that database without any problems, which is not good).

I had in mind exactly what you've suggested above - use a separate, manually-registered table containing the table columns and their mapping to ulogd2 parameters - much less risk and everything is configurable, though the downside is that the two tables need to be synchronised if the structure of the main ulogd table changes (columns renamed or added).

One other thing which I forgot to ask: currently, the pgsql plugin uses an unencrypted connection to the database server. I haven't studied the underlying source code which handles pgsql-specific functions, but in principle it is possible to use encrypted (SSL) connections to the server by using client/server certificates. Has this been attempted and considered to be included as an option to that particular plugin?

If not, would there be any objections if such feature is implemented (obviously, there will be a need to add additional parameters to that plugin to configure the type of connection, client/server/ca certificates etc)?

#6 lore raw
In reply to Mr Dash Four

Hello,

Quoted 12 lines · Mr Dash Four
Quoted 2 lines

For NFCT, you simply need to have nfnetlink_conntrack loaded.

I did, but I also made the mistake of including a few filters in that stack, which were incompatible and that was the reason I did not get any NFCT logs. Once that was corrected I started seeing connection tracking logged.

I have another question with regards to this: Is it possible to limit (by a separate filter or otherwise) the reporting and restrict it, to say, a specific set of interfaces or specific source/destination IP addresses/subnets?

Quoted 5 lines

Currently, NFCT reports absolutely everything, which is not what I really want as I have to sift through thousands of logs, not to mention that by reporting everything the system load is much higher.

So, is there a way for me to do that, somehow?

Not now but I'm working on it: Pablo has made a filter system in libnetfilter_conntrack. I will used it to filter.

Quoted 7 lines
Quoted 5 lines

Things are more complicated for NFACCT and I've just wrote a page dedicated to this point:
https://home.regit.org/2012/07/flow-accounting-with-netfilter-and-ulogd2/

Very helpful, thanks! I don't see it as complicated at all, certainly no more complicated than the existing accounting techniques.

Yes, not that complicated, documentation was just lacking.

Quoted 19 lines
Quoted 8 lines
Quoted 4 lines

Is there a description on the purpose of these and their configuration parameters, if any?

To now which keys are used and whch config var are available, you can use the -i option of ulogd:

Quoted 11 lines

ulogd -i libexec/ulogd/ulogd_filter_MARK.so
Name: MARK
Config options:
Var: mark (Integer, Default: 0)
Var: mask (Integer, Default: -1)
Input keys:
Key: ct.mark (unsigned int 32, optional)
Key: oob.mark (unsigned int 32, optional)
Output keys:
No statically defined keys

Quoted 1 line

Or look at the source code, which is what I did at the end.

:)

Quoted 17 lines
Quoted 13 lines
Quoted 12 lines

I will dig much more deeper into this once I know how ulogd works as my knowledge of ulogd2 is not that great (yet!). My initial instinct is that I may have to alter this script slightly, because from what I gather ulogd2 is "sensing" the structure of the logging table by reading the "description" of it in PostgreSQL. That particular operation usually requires higher-than-needed privileges and in order to avoid that I might need to find other ways to impose the "insert-only" restriction, which, lets be clear, is what ulogd needs. Again, I might be wrong with this, but that is what my initial observations with the script and ulogd suggest.

Quoted 4 lines

Yes, there's a discovery system that may require some tuning. But it can be made on a separate table use for the only purpose of describing the variables.

Quoted 6 lines

Yep, the pgsql plugin makes extensive use of pg_namespace, pg_class and pg_attribute which are system tables. These contain definitions of every single object registered on that database server and is a major security risk (as I pointed out, if that ulogd connection to the database server is hijacked, then the attacker could find out what is on that database without any problems, which is not good).

You're right.

Quoted 5 lines

I had in mind exactly what you've suggested above - use a separate, manually-registered table containing the table columns and their mapping to ulogd2 parameters - much less risk and everything is configurable, though the downside is that the two tables need to be synchronised if the structure of the main ulogd table changes (columns renamed or added).

It seems the safest way.

BR,

Signature · 2 lines
Eric Leblond 
Blog: http://home.regit.org/ - Portfolio: http://regit.500px.com/
#7 lore raw
Quoted 9 lines
Quoted 5 lines

Currently, NFCT reports absolutely everything, which is not what I really want as I have to sift through thousands of logs, not to mention that by reporting everything the system load is much higher.

So, is there a way for me to do that, somehow?

Not now but I'm working on it: Pablo has made a filter system in libnetfilter_conntrack. I will used it to filter.

This would be some awesome feature. I think this new filter which implements "custom" restrictions should not be for a particular input filter, but rather be universal. In other words, to be able to customise, say, certain IP addresses/subnets, certain ethernet interfaces etc, and then used anywhere in stack statements - NFCT, NFLOG and so on.

If this is implemented, it will certainly make ulogd2 very powerful and flexible at the same time - a bit like what syslog-ng is to the old syslog ;-)

The specific reason I raised this issue is because on the main firewall we have here, if I deploy ulogd2 and use NFCT at its present form, I will get the logs from all 7 interfaces, and it would make it an absolutely huge task to sift through all these logs and "match" the various entries (OK, doing it through the database will help up a bit, but not a lot).

If I am able to place a "custom" filter with different "filter" values in each separate stack, redirecting input to different places, then I would be able to track down what I want quite easily.

Quoted 5 lines
Quoted 2 lines

I had in mind exactly what you've suggested above - use a separate, manually-registered table containing the table columns and their mapping to ulogd2 parameters - much less risk and everything is configurable, though the downside is that the two tables need to be synchronised if the structure of the main ulogd table changes (columns renamed or added).

It seems the safest way.

It looks that way, doesn't it? In the coming days I'll look at the PGSQL implementation code to see whether SSL connection to the database server is a possibility with this plug in - it will be another good security feature if that is possible to be implemented.

#8 lore raw

Hello,

Quoted 7 lines · Mr Dash Four
Quoted 7 lines
Quoted 6 lines

Currently, NFCT reports absolutely everything, which is not what I really want as I have to sift through thousands of logs, not to mention that by reporting everything the system load is much higher.

So, is there a way for me to do that, somehow?

Quoted 6 lines
Quoted 3 lines

Not now but I'm working on it: Pablo has made a filter system in libnetfilter_conntrack. I will used it to filter.

This would be some awesome feature. I think this new filter which implements "custom" restrictions should not be for a particular input filter, but rather be universal.

Yes and no ;) (maybe maybe)

Quoted 4 lines

In other words, to be able to
customise, say, certain IP addresses/subnets, certain ethernet interfaces etc, and then used anywhere in stack statements - NFCT, NFLOG and so on.

If this universal filter is an excellent idea, it will suffer from some performance issue. The NFCT filter I'm currently implementing does the filtering inside the kernel which is really efficient. For NFLOG, kernel filtering can be done via iptables.
Implementing a userpsace filter is indeed a good idea but I don't think it is possible to do a kernel filtering ulogd module. The ulogd architecture will hardly deal with that because you will not be able to access to the constructor (in the input plugin) from the filter module.

Quoted 4 lines

If this is implemented, it will certainly make ulogd2 very powerful and flexible at the same time - a bit like what syslog-ng is to the old syslog ;-)

lol, the stack system is already a really big improvment.

Quoted 11 lines

The specific reason I raised this issue is because on the main firewall we have here, if I deploy ulogd2 and use NFCT at its present form, I will get the logs from all 7 interfaces, and it would make it an absolutely huge task to sift through all these logs and "match" the various entries (OK, doing it through the database will help up a bit, but not a lot).

If I am able to place a "custom" filter with different "filter" values in each separate stack, redirecting input to different places, then I would be able to track down what I want quite easily.

You should be able to do a per-network filtering with my current work. I should have a patch ready today.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

I had in mind exactly what you've suggested above - use a separate, manually-registered table containing the table columns and their mapping to ulogd2 parameters - much less risk and everything is configurable, though the downside is that the two tables need to be synchronised if the structure of the main ulogd table changes (columns renamed or added).

Quoted 6 lines
Quoted 2 lines

It seems the safest way.

It looks that way, doesn't it? In the coming days I'll look at the PGSQL implementation code to see whether SSL connection to the database server is a possibility with this plug in - it will be another good security feature if that is possible to be implemented.

Fine!

BR,

Signature · 2 lines
Eric Leblond 
Blog: http://home.regit.org/ - Portfolio: http://regit.500px.com/
#9 lore raw
Quoted 10 lines
Quoted 4 lines

In other words, to be able to
customise, say, certain IP addresses/subnets, certain ethernet interfaces etc, and then used anywhere in stack statements - NFCT, NFLOG and so on.

If this universal filter is an excellent idea, it will suffer from some performance issue. The NFCT filter I'm currently implementing does the filtering inside the kernel which is really efficient. For NFLOG, kernel filtering can be done via iptables.

Yeah, sorry, I've got carried away a bit there. :-)

Of course, with NFLOG things are much more easier.

Quoted 6 lines
Quoted 2 lines

If I am able to place a "custom" filter with different "filter" values in each separate stack, redirecting input to different places, then I would be able to track down what I want quite easily.

You should be able to do a per-network filtering with my current work. I should have a patch ready today.

Looking forward to it.

Quoted 5 lines
Quoted 2 lines

In the coming days I'll look at the PGSQL implementation code to see whether SSL connection to the database server is a possibility with this plug in - it will be another good security feature if that is possible to be implemented.

Fine!

I managed to get some preliminary code working yesterday and later today when I get home I'll get the chance to test it with the real plug in to see whether SSL works. If it does, I'll propose some changes to include SSL capability to the PGSQL plugin.

#10 lore raw
Quoted 2 lines

I managed to get some preliminary code working yesterday and later today when I get home I'll get the chance to test it with the real plug in to see whether SSL works. If it does, I'll propose some changes to include SSL capability to the PGSQL plugin.

OK, I've just done some testing using the PGSQL plugin with hard-coded values to create a secure (SSL) connection and verify how that plugin works. In short - it is absolutely flawless!

I am able to establish 4 types of SSL connections between ulogd and my PostgreSQL database server and conducted tests on various (known) attacks to see how it behaves - it passes with flying colours!

In the mean time, I found a bug and a "maybe" bug:

1. pgsql-ulogd2 script:

CREATE TABLE ulog2 (
[...]
timestamp timestamp NOT NULL default 'now'

The above will insert a string, equivalent to the current timestamp when the table was created as a default value (which PostgreSQL will automatically convert to timestamp) and *not* when a record is inserted into the table. The correct syntax is:

"timestamp" timestamp NOT NULL default now()

2. When I start using NFCT and it does the logging to the database, I am getting endless sequences of the following statement in the main ulogd.log file:

<5> ulogd_inpflow_NFCT.c:697 unsupported message type

I don't know whether there is something I am doing wrong or whether there is a message which ulogd cannot interpret. Maybe a bug, maybe not, I don't know.

So, for SSL implementation in the PGSQL plugin, this is how I see it: PostgreSQL (the server) allows for 6 different types of SSL connections to be established between it and a client [1]:

Option Description
~~~~~~ ~~~~~~~~~~~
disable only try a non-SSL connection
allow first try a non-SSL connection; if that fails, try an SSL connection
prefer (default) first try an SSL connection; if that fails, try a non-SSL connection
require only try an SSL connection
verify-ca only try an SSL connection, and verify that the server certificate is issued by a trusted CA
verify-full only try an SSL connection, verify that the server certificate is issued by a trusted CA and that the server host name matches that in the certificate

The above option is configured with the "sslmode" connection option ("sslmode=verfify-full" for example). In earlier PostgreSQL versions (below 8, I think), there was only one possible option and that was "requiressl" ("requiressl=1" is equivalent to "sslmode=require" above, while "requiressl=0" is the equivalent to "sslmode=prefer").

As far as verification goes, there is one additional parameter which could also be deployed on the PostgreSQL server to check request and check the client certificate as well. So verification and checking could be a dual process on both the client and the server.

In addition to the above parameter, 4 other connection options can be specified:

sslcert - This parameter specifies the file name of the client SSL certificate, replacing the default ‘~/.postgresql/postgresql.crt’. This parameter is ignored if an SSL connection is not made.

sslkey - This parameter specifies the location for the secret key used for the client certificate. It can either specify a file name that will be used instead of the default ‘~/.postgresql/postgresql.key’, or it can specify a key obtained from an external “engine” (engines are OpenSSL loadable modules). An external engine specification should consist of a colon-separated engine name and an engine-specific key identifier. This parameter is ignored if an SSL connection is not made. If this key is protected with a password, this will be asked when the connection is made. On the server side, it is stored internally and asked once when the server starts up. on the client side it is asked every time an attempt for a connection is made.

sslrootcert - This parameter specifies the name of a file containing SSL certificate authority (CA) certificate(s). If the file exists, the server's certificate will be verified to be signed by one of these authorities. The default is ‘~/.postgresql/root.crt’.

sslcrl - This parameter specifies the file name of the SSL certificate revocation list (CRL). Certificates listed in this file, if it exists, will be rejected while attempting to authenticate the server's certificate. The default is ‘~/.postgresql/root.crl’.

So, the way I see it, we can mirror all of the above 5 options, including their default values quite easily and let PGconn do the rest (please note that the PG .so object library takes care of opening the certificate files as well as checking and validating the various certificates contained within - this is done transparently to the client - PGSQL plugin in this case). The modifications to the PGSQL plugin would be minimal and will consists of little more than adding the above parameters and then passing their values as connection options to PGconn.

As far as hardening the access to the system tables goes, the way I see it there are 2 possible options:

1. Change the meaning of the "table" plugin parameter to point to a user-defined table, which contains a single row where all column names of the main ulogd table are stored. This option *will* brake backward compatibility!

2. Keep the existing "table" option and add a separate parameter (probably called "columns" or "table_columns" or something like that), which will contain a single row with the description of all usable columns in the "table" table where the main ulogd logging will take place.

I think option 2 is better, though at the expense of adding an extra configuration parameter, in addition to the 5 new ones for SSL support I listed above.

A few other observations and a query: Currently, there isn't a "user-friendly" NCFT view implemented in the PGSQL plugin, similar to the one designed for NFLOG (the one implemented for NFLOG is called ulog, I think) where the various numeric and raw values are "deciphered" for the end user and presented in a nice format to be understood more easily.

All NFCT logs are currently dumped, quite literally, in the ulogd2_ct table. Such a view would be very easy to create - I could do it and submit it together with the above changes if there is interest.

One last question - is there a way where I could get *all* possible values which a given filter plugin interprets, like source/destination IP address/subnet, interface names etc?

The reason I am asking this is because if I could use or implement a command line parameter (similar to "ulogd -i"), which asks every filter plugin to list the names of the parameters it processes from the input plugin, then I could build a reliable set of parameters which are then used to build a complete database script to be used in the creating the database schema which is then used by ulogd2.

Currently, I am only guessing what each plugin processes - it is like a black box, which isn't good-enough for me.

[1] - PostgreSQL v9.0 Programmer's Manual - http://www.network-theory.co.uk/docs/postgresql9/vol2/DatabaseConnectionControlFunctions.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

maintainer
#11 lore raw
In reply to Mr Dash Four
Quoted 6 lines · Mr Dash Four

The specific reason I raised this issue is because on the main firewall we have here, if I deploy ulogd2 and use NFCT at its present form, I will get the logs from all 7 interfaces, and it would make it an absolutely huge task to sift through all these logs and "match" the various entries (OK, doing it through the database will help up a bit, but not a lot).

Try something like
iptables -t raw -A PREROUTING (thingsyoudontwant) -j CT --ctevents related

(i don't think its possible to disable event delivery completely
at the moment without disabling conntrack ).

#12 lore raw
Quoted 8 lines · Florian Westphal
Quoted 6 lines · Mr Dash Four

The specific reason I raised this issue is because on the main firewall we have here, if I deploy ulogd2 and use NFCT at its present form, I will get the logs from all 7 interfaces, and it would make it an absolutely huge task to sift through all these logs and "match" the various entries (OK, doing it through the database will help up a bit, but not a lot).

Quoted 5 lines

Try something like
iptables -t raw -A PREROUTING (thingsyoudontwant) -j CT --ctevents related

(i don't think its possible to disable event delivery completely
at the moment without disabling conntrack ).

You can do it via:

/proc/sys/net/netfilter/nf_conntrack_events

#13 lore raw
In reply to Florian Westphal
Quoted 9 lines
Quoted 8 lines

The specific reason I raised this issue is because on the main firewall we have here, if I deploy ulogd2 and use NFCT at its present form, I will get the logs from all 7 interfaces, and it would make it an absolutely huge task to sift through all these logs and "match" the various entries (OK, doing it through the database will help up a bit, but not a lot).

Quoted 3 lines

Try something like
iptables -t raw -A PREROUTING (thingsyoudontwant) -j CT --ctevents related

What do you mean by "(thingsyoudontwant)"? How would that affect tracking (sorry, I probably need to brush-up on the CT a bit)?

maintainer
#14 lore raw
Quoted 8 lines · Mr Dash Four
Quoted 8 lines
Quoted 7 lines

The specific reason I raised this issue is because on the main firewall we have here, if I deploy ulogd2 and use NFCT at its present form, I will get the logs from all 7 interfaces, and it would make it an absolutely huge task to sift through all these logs and "match" the various entries (OK, doing it through the database will help up a bit, but not a lot).

Quoted 4 lines
Quoted 2 lines

Try something like
iptables -t raw -A PREROUTING (thingsyoudontwant) -j CT --ctevents related

What do you mean by "(thingsyoudontwant)"? How would that affect tracking (sorry, I probably need to brush-up on the CT a bit)?

It specifies what events to generate; Events that aren't generated can't be seen by NFCT. Tracking itself is not influenced.

As Pablo pointed out, you could also try disabling event reporting completely via sysctl and only enable events for those networks/services you're interested in.

So e.g. if you are only interested in logging the start and end of connections coming from 192.168.1/24 on eth0 and going out via eth7 you could try

echo 0 > /proc/sys/net/netfilter/nf_conntrack_events

iptables -t raw -A PREROUTING -i eth0 -o eth7 \
	 -s 192.168.1.0/24 -j CT --ctevents new,related,destroy
#15 lore raw
Quoted 21 lines · Florian Westphal
Quoted 8 lines · Mr Dash Four
Quoted 8 lines
Quoted 7 lines

The specific reason I raised this issue is because on the main firewall we have here, if I deploy ulogd2 and use NFCT at its present form, I will get the logs from all 7 interfaces, and it would make it an absolutely huge task to sift through all these logs and "match" the various entries (OK, doing it through the database will help up a bit, but not a lot).

Quoted 4 lines
Quoted 2 lines

Try something like
iptables -t raw -A PREROUTING (thingsyoudontwant) -j CT --ctevents related

What do you mean by "(thingsyoudontwant)"? How would that affect tracking (sorry, I probably need to brush-up on the CT a bit)?

It specifies what events to generate; Events that aren't generated can't be seen by NFCT. Tracking itself is not influenced.

As Pablo pointed out, you could also try disabling event reporting completely via sysctl and only enable events for those networks/services you're interested in.

Quoted 5 lines

So e.g. if you are only interested in logging the start and end of connections coming from 192.168.1/24 on eth0 and going out via eth7 you could try

echo 0 > /proc/sys/net/netfilter/nf_conntrack_events

oops, sorry, this is not what I meant to say. This line above *completely* disables events. Thus, the line below does not have effect anymore.

Quoted 2 lines
iptables -t raw -A PREROUTING -i eth0 -o eth7 \
	 -s 192.168.1.0/24 -j CT --ctevents new,related,destroy

It should be hard to add some "none" for --ctevents so you can add to the following rules below:

iptables -t raw -A PREROUTING -i eth0 -o eth7 \
	 -s 192.168.1.0/24 -j CT --ctevents new,related,destroy
iptables -t raw -A PREROUTING -i eth0 -o eth7 \
	 ! -s 192.168.1.0/24 -j CT --ctevents none

Not to report events for others. Note that the "none" is missing now in iptables.

Thus, we will only get events coming from 192.168.1.0/24.

Still, for better per-process (not global) filtering some ulogd2 filter that uses the libnetfilter_conntrack filtering based on Berkeley Socket Filters can help.

I remember that I posted some patches long time ago to add unicast-based ctevent. These days that could be easier by reusing the user-space cthelper infrastructure that went into the kernel for 3.7.

#16 lore raw
Quoted 13 lines
Quoted 4 lines
iptables -t raw -A PREROUTING -i eth0 -o eth7 \
	 -s 192.168.1.0/24 -j CT --ctevents new,related,destroy

It should be hard to add some "none" for --ctevents so you can add to the following rules below:

iptables -t raw -A PREROUTING -i eth0 -o eth7 \
	 -s 192.168.1.0/24 -j CT --ctevents new,related,destroy
iptables -t raw -A PREROUTING -i eth0 -o eth7 \
	 ! -s 192.168.1.0/24 -j CT --ctevents none
Quoted 5 lines

Not to report events for others. Note that the "none" is missing now in iptables.

Thus, we will only get events coming from 192.168.1.0/24.

I am not entirely certain whether I could do more damage to my setup if I do the above. Let me explain:

On all interfaces, with the exception of one, I have a limited set of rules, which are traversed only when a new connection is established (i.e. the state is NEW). In all other cases, I have -j ACCEPT when the state is RELATED or ESTABLISHED. This was done primarily to lower the load on the firewall machine and also to save me from writing iptables rules for both sides of a connection.

I am not sure if I apply the above, how would that affect the setup I just described and I am not sure whether it would make things worse. Again, my aim is to be able to configure full logging of particular type of connections via NFCT (from their inception to their closure), depending on the interface and source/destiantion IP address/subnet, possibly without affecting the above setup too much.

I know how to do that with packets via NFLOG - that is easy enough for me, but I am struggling with NFCT, unfortunately.

#17 lore raw
In reply to Eric Leblond
Quoted 15 lines · Eric Leblond

Hello,

Quoted 12 lines · Mr Dash Four
Quoted 2 lines

For NFCT, you simply need to have nfnetlink_conntrack loaded.

I did, but I also made the mistake of including a few filters in that stack, which were incompatible and that was the reason I did not get any NFCT logs. Once that was corrected I started seeing connection tracking logged.

I have another question with regards to this: Is it possible to limit (by a separate filter or otherwise) the reporting and restrict it, to say, a specific set of interfaces or specific source/destination IP addresses/subnets?

Quoted 8 lines
Quoted 5 lines

Currently, NFCT reports absolutely everything, which is not what I really want as I have to sift through thousands of logs, not to mention that by reporting everything the system load is much higher.

So, is there a way for me to do that, somehow?

Not now but I'm working on it: Pablo has made a filter system in libnetfilter_conntrack. I will used it to filter.

You can also use the CT target to filter conntrack events. It's a global configurable parameter though, but it's easy.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html