Skip to contentSkip to conversation

Misc improvements and bugfixes

PATCH Applied Eric Leblond 2 participants 49 messages Mar 23, 2008 – Apr 29, 2008 netfilter

lore
Share & tools
lore
#0 lore raw

cover Misc improvements and bugfixes

Hello,

This patchset contains some code cleaning and two structural changes.

The first four patches are code cleaning:
* Fix SQL reconnection algorithm.
* Suppress ip_as_string configuration variable.
* Fix type of raw.mac_len key in NFLOG.
* Fix type of raw.mac_len key in ULOG.

The next set is more interesting as it introduces handling of MAC address via a dedicated module named MAC2STR. Source hardware address is now stored in a dedicated table of the MySQL and pgSQL databases.

Patches which implement this modification are:
* New MAC2STR plugin for hwmac address conversion.
* MAC address handling in MySQL output plugin.
* MAC address handling in PgSQL output plugin.

The next set of patches try to solve a classical netfilter logging problem. As logging is not a verdict, it is not possible to know in userspace if a packet has been accepted or dropped. A frequent workaround is to use the prefix of the ULOG or NFLOG target to indicate the decision taken on the packet.

The idea behind this patchset is to dedicate a group (and a plugin instance in ulogd2) to accepted packets and an other instance to dropped packets. To differentiate the instances a new option, state, has been added to NFLOG and ULOG plugins. Default value of the option is 0 which is NF_DROP value.

The value of the option is sent to the output module as "raw.state" key. This value is stored in a dedicated table of the PgSQL and MySQL output modules.

This modification is implemented in the following patches:
* Add state option to NFLOG input plugin.
* Add state support to MySQL schema.
* Add state option to ULOG input plugin.
* Add state support to PGSQL output module.

The remaining patches are :
* Add hook output to ULOG input module.
* Update default configuration to fit last changes.
* Fix computation of length of mac address.

Patchset statistics :

 doc/mysql-ulogd2.sql              |   78 +++++++++++++++-----------
 doc/pgsql-ulogd2.sql              |   59 +++++++++++++++++---
 filter/Makefile.am                |    6 ++-
 filter/ulogd_filter_MAC2STR.c     |  111 +++++++++++++++++++++++++++++++++++++
 include/ulogd/db.h                |   13 ++---
 input/packet/ulogd_inppkt_NFLOG.c |   26 +++++++-
 input/packet/ulogd_inppkt_ULOG.c  |   33 ++++++++++-
 ulogd.conf.in                     |   10 +++-
 util/db.c                         |   20 ++----
 util/printpkt.c                   |    2 +-
 10 files changed, 286 insertions(+), 72 deletions(-)

BR,
--
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/

#1 lore raw

PATCH 1/14 Fix SQL reconnection algorithm.

This patch fixes a problem in SQL reconnection algorithm which is managed in the db.c file for PgSQL and MySQL. In case of problem during request execution a new connection to the database was immediatly started without closing the previous one. The consequence was to block the database by having too much simultaneous open connections.

This patch fixes the problem by disconnectinng from the database after a request failure and trying to reconnect after a delay which is by default of 2 secondes. This delay can be customized via the reconnect configuration variable in the database configuration section.

---
 include/ulogd/db.h |    2 ++
 util/db.c          |    9 +++++++--
 2 files changed, 9 insertions(+), 2 deletions(-)
include/ulogd/db.h +2 -0
index 94cdbcb..62819f2 100644
@@ -23,6 +23,7 @@ struct db_instance {
 	struct db_driver *driver;
 };
 #define TIME_ERR		((time_t)-1)	/* Be paranoid */
+#define RECONNECT_DEFAULT	2
 
 #define DB_CES							\
 		{						\
@@ -33,6 +34,7 @@ struct db_instance {
 		{						\
 			.key = "reconnect",			\
 			.type = CONFIG_TYPE_INT,		\
+			.u.value = RECONNECT_DEFAULT,		\
 		},						\
 		{						\
 			.key = "ip_as_string",			\
util/db.c +7 -2
index 49f6c29..b9ca8db 100644
@@ -176,6 +176,8 @@ static int _init_reconnect(struct ulogd_pluginstance *upi)
 	struct db_instance *di = (struct db_instance *) upi->private;
 
 	if (reconnect_ce(upi->config_kset).u.value) {
+		if (time(NULL) < di->reconnect)
+			return -1;
 		di->reconnect = time(NULL);
 		if (di->reconnect != TIME_ERR) {
 			ulogd_log(ULOGD_ERROR, "no connection to database, "
@@ -312,8 +314,11 @@ static int __interp_db(struct ulogd_pluginstance *upi)
 
 	/* now we have created our statement, insert it */
 
-	if (di->driver->execute(upi, di->stmt, strlen(di->stmt)) < 0)
-		return _init_db(upi);
+	if (di->driver->execute(upi, di->stmt, strlen(di->stmt)) < 0) {
+		/* error occur, database connexion need to be closed */
+		di->driver->close_db(upi);
+		return _init_reconnect(upi);
+	}
 
 	return 0;
 }
Signature · 1 line
1.5.2.5
maintainer Applied
#2 lore raw

Re: [ULOGD PATCH 01/14] Fix SQL reconnection algorithm.

Quoted 10 lines · Eric Leblond

This patch fixes a problem in SQL reconnection algorithm which is managed in the db.c file for PgSQL and MySQL. In case of problem during request execution a new connection to the database was immediatly started without closing the previous one. The consequence was to block the database by having too much simultaneous open connections.

This patch fixes the problem by disconnectinng from the database after a request failure and trying to reconnect after a delay which is by default of 2 secondes. This delay can be customized via the reconnect configuration variable in the database configuration section.

Applied. Thanks Eric. BTW, an early impression, would it be possible to handle db reconnections using the timer infrastructure?

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#3 lore raw

PATCH 2/14 Suppress ip_as_string configuration variable.

In reply to Eric Leblond

This patch suppress a now unused option. Each database module have now to be used with a defined IP storage type.

---
 include/ulogd/db.h |   11 +++--------
 util/db.c          |   11 -----------
 2 files changed, 3 insertions(+), 19 deletions(-)
include/ulogd/db.h +3 -8
index 62819f2..b0b7295 100644
@@ -37,10 +37,6 @@ struct db_instance {
 			.u.value = RECONNECT_DEFAULT,		\
 		},						\
 		{						\
-			.key = "ip_as_string",			\
-			.type = CONFIG_TYPE_INT,		\
-		},						\
-		{						\
 			.key = "connect_timeout",		\
 			.type = CONFIG_TYPE_INT,		\
 		},						\
@@ -50,12 +46,11 @@ struct db_instance {
 			.options = CONFIG_OPT_MANDATORY,	\
 		}
 
-#define DB_CE_NUM	5
+#define DB_CE_NUM	4
 #define table_ce(x)	(x->ces[0])
 #define reconnect_ce(x)	(x->ces[1])
-#define asstring_ce(x)	(x->ces[2])
-#define timeout_ce(x)	(x->ces[3])
-#define procedure_ce(x)	(x->ces[4])
+#define timeout_ce(x)	(x->ces[2])
+#define procedure_ce(x)	(x->ces[3])
 
 void ulogd_db_signal(struct ulogd_pluginstance *upi, int signal);
 int ulogd_db_start(struct ulogd_pluginstance *upi);
util/db.c +0 -11
index b9ca8db..3cfe057 100644
@@ -265,17 +265,6 @@ static int __interp_db(struct ulogd_pluginstance *upi)
 			sprintf(di->stmt_ins, "%u,", res->u.value.ui16);
 			break;
 		case ULOGD_RET_IPADDR:
-			if (asstring_ce(upi->config_kset).u.value) {
-				memset(&addr, 0, sizeof(addr));
-				addr.s_addr = ntohl(res->u.value.ui32);
-				*(di->stmt_ins++) = '\'';
-				tmpstr = inet_ntoa(addr);
-				di->driver->escape_string(upi, di->stmt_ins,
-							  tmpstr, strlen(tmpstr));
-				di->stmt_ins = di->stmt + strlen(di->stmt);
-				sprintf(di->stmt_ins, "',");
-				break;
-			}
 			/* fallthrough when logging IP as u_int32_t */
 		case ULOGD_RET_UINT32:
 			sprintf(di->stmt_ins, "%u,", res->u.value.ui32);
Signature · 1 line
1.5.2.5
#4 lore raw

Re: [ULOGD PATCH 02/14] Suppress ip_as_string configuration variable.

Quoted 2 lines · Eric Leblond

This patch suppress a now unused option. Each database module have now to be used with a defined IP storage type.

As far as I can remember, this parameter was introduced because some sysadmins queried the sql database directly to look for certain IPs. I know that this isn't good for performance but it has some clients in ulogd 1.x.

Does mysql have something like the PgSQL's inet type? Looking at the database definitions, we are storing the IPs in binary format in mysql and in inet type in PgSQL. Does this mean that there's no replacement for the ip_as_string in mysql?

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#5 lore raw

Re: [ULOGD PATCH 02/14] Suppress ip_as_string configuration variable.

Hello,

Quoted 10 lines · Pablo Neira Ayuso
Quoted 2 lines · Eric Leblond

This patch suppress a now unused option. Each database module have now to be used with a defined IP storage type.

As far as I can remember, this parameter was introduced because some sysadmins queried the sql database directly to look for certain IPs. I know that this isn't good for performance but it has some clients in ulogd 1.x.

Does mysql have something like the PgSQL's inet type?

No, there is no advanced types in MySQL.

Quoted 4 lines

Looking at the
database definitions, we are storing the IPs in binary format in mysql and in inet type in PgSQL. Does this mean that there's no replacement for the ip_as_string in mysql?

I think it could be achieved by working on a VIEW which could display the binary field in IP string format. An other solution could be to provide two database schemas (as it was before).

In any cases, it is not related to ulogd C code: the list of keys (with their associated type) is given by the content of a table. Thus, there is no need for the option ip_as_string.

BR,

Signature · 3 lines
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/
#6 lore raw

Re: [ULOGD PATCH 02/14] Suppress ip_as_string configuration variable.

Quoted 15 lines · Eric Leblond

Hello,

Quoted 9 lines · Pablo Neira Ayuso
Quoted 2 lines · Eric Leblond

This patch suppress a now unused option. Each database module have now to be used with a defined IP storage type.

As far as I can remember, this parameter was introduced because some sysadmins queried the sql database directly to look for certain IPs. I know that this isn't good for performance but it has some clients in ulogd 1.x.

Does mysql have something like the PgSQL's inet type?

No, there is no advanced types in MySQL.

Quoted 8 lines
Quoted 4 lines

Looking at the
database definitions, we are storing the IPs in binary format in mysql and in inet type in PgSQL. Does this mean that there's no replacement for the ip_as_string in mysql?

I think it could be achieved by working on a VIEW which could display the binary field in IP string format. An other solution could be to provide two database schemas (as it was before).

I prefer the view option.

Quoted 3 lines

In any cases, it is not related to ulogd C code: the list of keys (with their associated type) is given by the content of a table. Thus, there is no need for the option ip_as_string.

OK. Applied, thanks.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#7 lore raw

PATCH 3/14 Fix type of raw.mac_len key in NFLOG.

In reply to Eric Leblond

Type of the raw.mac_len key was set to string but this is an unsigned interger. This patch fix this in the NFLOG module.

---
 input/packet/ulogd_inppkt_NFLOG.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
input/packet/ulogd_inppkt_NFLOG.c +1 -1
index 166e048..467d42f 100644
@@ -209,7 +209,7 @@ static struct ulogd_key output_keys[] = {
 		},
 	},
 	{ 
-		.type = ULOGD_RET_STRING, 
+		.type = ULOGD_RET_UINT16, 
 		.flags = ULOGD_RETF_NONE, 
 		.name = "raw.mac_len", 
 	},
Signature · 1 line
1.5.2.5
maintainer Applied
#8 lore raw

Re: [ULOGD PATCH 03/14] Fix type of raw.mac_len key in NFLOG.

Quoted 2 lines · Eric Leblond

Type of the raw.mac_len key was set to string but this is an unsigned interger. This patch fix this in the NFLOG module.

Applied. Thanks Eric.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#9 lore raw

PATCH 4/14 Fix type of raw.mac_len key in ULOG.

In reply to Eric Leblond

Type of the raw.mac_len key was set to string but this is an unsigned interger. This patch fix this in the ULOG module.

---
 input/packet/ulogd_inppkt_ULOG.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
input/packet/ulogd_inppkt_ULOG.c +1 -1
index c49345d..a1d52f2 100644
@@ -143,7 +143,7 @@ static struct ulogd_key output_keys[] = {
 		.name = "oob.out", 
 	},
 	{ 
-		.type = ULOGD_RET_STRING, 
+		.type = ULOGD_RET_UINT16, 
 		.flags = ULOGD_RETF_NONE, 
 		.name = "raw.mac_len", 
 	},
Signature · 1 line
1.5.2.5
#10 lore raw

Re: [ULOGD PATCH 04/14] Fix type of raw.mac_len key in ULOG.

Quoted 2 lines · Eric Leblond

Type of the raw.mac_len key was set to string but this is an unsigned interger. This patch fix this in the ULOG module.

Also applied. Thanks. Please, next time put together patches that fix the same problem (3/14 and 4/14).

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#11 lore raw

PATCH 5/14 New MAC2STR plugin for hwmac address conversion.

In reply to Eric Leblond

This patch introduces a new plugin MAC2STR which is in charge of conversion to string of MAC address. It is used by database output plugin to store MAC related information.

---
 filter/Makefile.am            |    6 ++-
 filter/ulogd_filter_MAC2STR.c |  111 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 116 insertions(+), 1 deletions(-)
 create mode 100644 filter/ulogd_filter_MAC2STR.c
filter/Makefile.am +5 -1
index 52b639c..958a5de 100644
@@ -4,7 +4,8 @@ INCLUDES = $(all_includes) -I$(top_srcdir)/include
 
 pkglib_LTLIBRARIES = ulogd_filter_IFINDEX.la ulogd_filter_PWSNIFF.la \
 		     ulogd_filter_PRINTPKT.la ulogd_filter_PRINTFLOW.la \
-		     ulogd_filter_IP2STR.la ulogd_filter_IP2BIN.la
+		     ulogd_filter_IP2STR.la ulogd_filter_IP2BIN.la \
+		     ulogd_filter_MAC2STR.la
 
 ulogd_filter_IFINDEX_la_SOURCES = ulogd_filter_IFINDEX.c
 ulogd_filter_IFINDEX_la_LDFLAGS = -module -lnfnetlink
@@ -18,6 +19,9 @@ ulogd_filter_IP2STR_la_LDFLAGS = -module
 ulogd_filter_IP2BIN_la_SOURCES = ulogd_filter_IP2BIN.c
 ulogd_filter_IP2BIN_la_LDFLAGS = -module
 
+ulogd_filter_MAC2STR_la_SOURCES = ulogd_filter_MAC2STR.c
+ulogd_filter_MAC2STR_la_LDFLAGS = -module
+
 ulogd_filter_PRINTPKT_la_SOURCES = ulogd_filter_PRINTPKT.c ../util/printpkt.c
 ulogd_filter_PRINTPKT_la_LDFLAGS = -module
 
filter/ulogd_filter_MAC2STR.c new file +111 -0

111 lines not on this page · raw message

Signature · 1 line
1.5.2.5
#12 lore raw

Re: [ULOGD PATCH 05/14] New MAC2STR plugin for hwmac address conversion.

Quoted 3 lines · Eric Leblond

This patch introduces a new plugin MAC2STR which is in charge of conversion to string of MAC address. It is used by database output plugin to store MAC related information.

Also applied. Thanks.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#13 lore raw

PATCH 6/14 MAC address handling in MySQL output plugin.

In reply to Eric Leblond

This patch adds MAC address handling to the MySQL output plugin.

---
 doc/mysql-ulogd2.sql |   31 +++++++++++++------------------
 1 files changed, 13 insertions(+), 18 deletions(-)
doc/mysql-ulogd2.sql +13 -18
index 1265eba..bb31339 100644
@@ -72,14 +72,12 @@ ALTER TABLE ulog2 ADD KEY `timestamp` (`timestamp`);
 
 CREATE TABLE `mac` (
   `_mac_id` bigint unsigned NOT NULL,
-  `mac_saddr` binary(12) default NULL,
-  `mac_daddr` binary(12) default NULL,
+  `mac_saddr` varchar(32) default NULL,
   `mac_protocol` smallint(5) default NULL
 ) ENGINE=INNODB;
 
 ALTER TABLE mac ADD UNIQUE KEY `_mac_id` (`_mac_id`);
 ALTER TABLE mac ADD KEY `mac_saddr` (`mac_saddr`);
-ALTER TABLE mac ADD KEY `mac_daddr` (`mac_daddr`);
 ALTER TABLE mac ADD KEY `index_mac_id` (`_mac_id`);
 
 CREATE TABLE `tcp` (
@@ -213,10 +211,9 @@ CREATE SQL SECURITY INVOKER VIEW `ulog` AS
 	icmpv6_code,
 	icmpv6_echoid,
 	icmpv6_echoseq,
-	icmpv6_csum
---	mac_saddr,
---	mac_daddr,
---	mac_protocol,
+	icmpv6_csum,
+	mac_saddr as mac_saddr_str,
+	mac_protocol as oob_protocol
         FROM ulog2 LEFT JOIN tcp ON ulog2._id = tcp._tcp_id LEFT JOIN udp ON ulog2._id = udp._udp_id
                 LEFT JOIN icmp ON ulog2._id = icmp._icmp_id LEFT JOIN mac ON ulog2._id = mac._mac_id
                 LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id;
@@ -571,13 +568,12 @@ delimiter $$
 DROP PROCEDURE IF EXISTS PACKET_ADD_MAC;
 CREATE PROCEDURE PACKET_ADD_MAC(
 		IN `id` int(10) unsigned,
-		IN `_saddr` binary(12),
-		IN `_daddr` binary(12),
+		IN `_saddr` varchar(32),
 		IN `_protocol` smallint(5)
 		)
 BEGIN
-	INSERT INTO mac (_mac_id, mac_saddr, mac_daddr, mac_protocol) VALUES
-	(id, _saddr, _daddr, _protocol);
+	INSERT INTO mac (_mac_id, mac_saddr, mac_protocol) VALUES
+	(id, _saddr, _protocol);
 END
 $$
 
@@ -627,10 +623,9 @@ CREATE FUNCTION INSERT_PACKET_FULL(
 		icmpv6_code tinyint(3) unsigned,
 		icmpv6_echoid smallint(5) unsigned,
 		icmpv6_echoseq smallint(5) unsigned,
-		icmpv6_csum int(10) unsigned
---		mac_saddr binary(12),
---		mac_daddr binary(12),
---		mac_protocol smallint(5)
+		icmpv6_csum int(10) unsigned,
+		mac_saddr varchar(32),
+		mac_protocol smallint(5)
 		) RETURNS bigint unsigned
 READS SQL DATA
 BEGIN
@@ -652,9 +647,9 @@ BEGIN
 		CALL PACKET_ADD_ICMPV6(@lastid, icmpv6_type, icmpv6_code, icmpv6_echoid,
 				       icmpv6_echoseq, icmpv6_csum);
 	END IF;
---	IF mac_protocol IS NOT NULL THEN
---		CALL PACKET_ADD_MAC(@lastid, mac_saddr, mac_daddr, mac_protocol);
---	END IF;
+	IF mac_protocol IS NOT NULL THEN
+		CALL PACKET_ADD_MAC(@lastid, mac_saddr, mac_protocol);
+	END IF;
 	RETURN @lastid;
 END
 $$
Signature · 1 line
1.5.2.5
maintainer Applied
#14 lore raw

Re: [ULOGD PATCH 06/14] MAC address handling in MySQL output plugin.

Quoted 1 line · Eric Leblond

This patch adds MAC address handling to the MySQL output plugin.

Applied. Thanks. Please, next time also include in the description that you have deleted the mac_saddr field (I assume that it makes sense since it does not provide any useful information for the logging purposes)

BTW, we have to freeze changes in the databases before the final release, otherwise we'll have to include upgrade scripts to convert the database from version X to Y as it happens to other apps (like bugzilla). I know that this would happen sooner or later if we have to add features but still introducing such kind of scripts would be really a pain. I don't think that we should go that way.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#15 lore raw

Re: [ULOGD PATCH 06/14] MAC address handling in MySQL output plugin.

Hello,

Quoted 7 lines · Pablo Neira Ayuso
Quoted 1 line · Eric Leblond

This patch adds MAC address handling to the MySQL output plugin.

Applied. Thanks. Please, next time also include in the description that you have deleted the mac_saddr field (I assume that it makes sense since it does not provide any useful information for the logging purposes)

Quoted 6 lines

BTW, we have to freeze changes in the databases before the final release, otherwise we'll have to include upgrade scripts to convert the database from version X to Y as it happens to other apps (like bugzilla). I know that this would happen sooner or later if we have to add features but still introducing such kind of scripts would be really a pain. I don't think that we should go that way.

Pierre Chifflier and I will soon start working on improving the SQL schemas. Some things remind to be done on them before freezing.

BR,

Signature · 3 lines
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/
#16 lore raw

Re: [ULOGD PATCH 06/14] MAC address handling in MySQL output plugin.

Quoted 16 lines · Eric Leblond

Hello,

Quoted 6 lines · Pablo Neira Ayuso
Quoted 1 line · Eric Leblond

This patch adds MAC address handling to the MySQL output plugin.

Applied. Thanks. Please, next time also include in the description that you have deleted the mac_saddr field (I assume that it makes sense since it does not provide any useful information for the logging purposes)

Quoted 6 lines

BTW, we have to freeze changes in the databases before the final release, otherwise we'll have to include upgrade scripts to convert the database from version X to Y as it happens to other apps (like bugzilla). I know that this would happen sooner or later if we have to add features but still introducing such kind of scripts would be really a pain. I don't think that we should go that way.

Quoted 2 lines

Pierre Chifflier and I will soon start working on improving the SQL schemas. Some things remind to be done on them before freezing.

Great. Wait for your news. I'll also have a look at it.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#17 lore raw

PATCH 7/14 MAC address handling in PgSQL output plugin.

In reply to Eric Leblond

This patch adds MAC address handling to the postgresql output plugin.

---
 doc/pgsql-ulogd2.sql |   23 +++++++++++++++++++----
 1 files changed, 19 insertions(+), 4 deletions(-)
doc/pgsql-ulogd2.sql +19 -4
index f4359ed..fc4aa1b 100644
@@ -68,12 +68,10 @@ CREATE INDEX ulog2_timestamp ON ulog2(timestamp);
 CREATE TABLE mac (
   _mac_id bigint PRIMARY KEY UNIQUE NOT NULL,
   mac_saddr macaddr default NULL,
-  mac_daddr macaddr default NULL,
   mac_protocol smallint default NULL
 ) WITH (OIDS=FALSE);
 
 CREATE INDEX mac_saddr ON mac(mac_saddr);
-CREATE INDEX mac_daddr ON mac(mac_daddr);
 
 CREATE TABLE tcp (
   _tcp_id bigint PRIMARY KEY UNIQUE NOT NULL,
@@ -191,7 +189,9 @@ CREATE OR REPLACE VIEW ulog AS
         icmpv6_code,
         icmpv6_echoid,
         icmpv6_echoseq,
-        icmpv6_csum
+        icmpv6_csum,
+        mac_saddr AS mac_saddr_str,
+        mac_protocol AS oob_protocol
         FROM ulog2 LEFT JOIN tcp ON ulog2._id = tcp._tcp_id LEFT JOIN udp ON ulog2._id = udp._udp_id
                 LEFT JOIN icmp ON ulog2._id = icmp._icmp_id LEFT JOIN mac ON ulog2._id = mac._mac_id
                 LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id;
@@ -433,6 +433,16 @@ RETURNS bigint AS $$
         SELECT currval('ulog2__id_seq');
 $$ LANGUAGE SQL SECURITY INVOKER;
 
+CREATE OR REPLACE FUNCTION INSERT_MAC(
+                IN mac_id bigint,
+                IN mac_saddr macaddr,
+                IN mac_protocol integer
+        )
+RETURNS bigint AS $$
+        INSERT INTO mac (_mac_id,mac_saddr,mac_protocol)
+                VALUES ($1,$2,$3);
+        SELECT currval('ulog2__id_seq');
+$$ LANGUAGE SQL SECURITY INVOKER;
 
 -- this function requires plpgsql
 -- su -c "createlang plpgsql ulog2" postgres
@@ -480,7 +490,9 @@ CREATE OR REPLACE FUNCTION INSERT_PACKET_FULL(
                 IN icmpv6_code integer,
                 IN icmpv6_echoid integer,
                 IN icmpv6_echoseq integer,
-                IN icmpv6_csum integer
+                IN icmpv6_csum integer,
+                IN mac_saddr varchar(32),
+                IN mac_protocol integer
         )
 RETURNS bigint AS $$
 DECLARE
@@ -496,6 +508,9 @@ BEGIN
         ELSIF (ip_protocol = 58) THEN
                 PERFORM INSERT_ICMPV6(_id,$40,$41,$42,$43,$44);
         END IF;
+        IF (mac_saddr IS NOT NULL) THEN
+                PERFORM INSERT_MAC(_id,$45::macaddr,$46);
+        END IF;
         RETURN _id;
 END
 $$ LANGUAGE plpgsql SECURITY INVOKER;
Signature · 1 line
1.5.2.5
maintainer Applied
#18 lore raw

Re: [ULOGD PATCH 07/14] MAC address handling in PgSQL output plugin.

Quoted 1 line · Eric Leblond

This patch adds MAC address handling to the postgresql output plugin.

Applied. Thanks Eric.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#19 lore raw

PATCH 8/14 Add state option to NFLOG input plugin.

In reply to Eric Leblond

This patch adds support for "state" option to the NFLOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

This patch also fixes a bug in definition of seq_global_ce macro.

---
 input/packet/ulogd_inppkt_NFLOG.c |   24 +++++++++++++++++++++---
 1 files changed, 21 insertions(+), 3 deletions(-)
input/packet/ulogd_inppkt_NFLOG.c +21 -3
index 467d42f..7562687 100644
@@ -34,7 +34,7 @@ struct nflog_input {
 /* configuration entries */
 
 static struct config_keyset libulog_kset = {
-	.num_ces = 7,
+	.num_ces = 8,
 	.ces = {
 		{
 			.key 	 = "bufsize",
@@ -78,6 +78,13 @@ static struct config_keyset libulog_kset = {
 			.options = CONFIG_OPT_NONE,
 			.u.value = 0,
 		},
+		{
+			.key	 = "state",
+			.type	 = CONFIG_TYPE_INT,
+			.options = CONFIG_OPT_NONE,
+			.u.value = 0,
+		},
+
 	}
 };
 
@@ -86,8 +93,9 @@ static struct config_keyset libulog_kset = {
 #define rmem_ce(x)	(x->ces[2])
 #define af_ce(x)	(x->ces[3])
 #define unbind_ce(x)	(x->ces[4])
-#define seq_ce(x)	(x->ces[4])
-#define seq_global_ce(x)	(x->ces[5])
+#define seq_ce(x)	(x->ces[5])
+#define seq_global_ce(x)	(x->ces[6])
+#define state_ce(x)	(x->ces[7])
 
 enum nflog_keys {
 	NFLOG_KEY_RAW_MAC = 0,
@@ -107,6 +115,7 @@ enum nflog_keys {
 	NFLOG_KEY_OOB_FAMILY,
 	NFLOG_KEY_OOB_PROTOCOL,
 	NFLOG_KEY_OOB_UID,
+	NFLOG_KEY_RAW_STATE,
 };
 
 static struct ulogd_key output_keys[] = {
@@ -246,6 +255,12 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.uid",
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "raw.state",
+	},
+
 
 };
 
@@ -270,6 +285,9 @@ interp_packet(struct ulogd_pluginstance *upi, struct nflog_data *ldata)
 	ret[NFLOG_KEY_OOB_FAMILY].u.value.ui8 = af_ce(upi->config_kset).u.value;
 	ret[NFLOG_KEY_OOB_FAMILY].flags |= ULOGD_RETF_VALID;
 
+	ret[NFLOG_KEY_RAW_STATE].u.value.ui8 = state_ce(upi->config_kset).u.value;
+	ret[NFLOG_KEY_RAW_STATE].flags |= ULOGD_RETF_VALID;
+
 	if (ph) {
 		/* FIXME */
 		ret[NFLOG_KEY_OOB_HOOK].u.value.ui8 = ph->hook;
Signature · 1 line
1.5.2.5
#20 lore raw

Re: [ULOGD PATCH 08/14] Add state option to NFLOG input plugin.

Quoted 3 lines · Eric Leblond

This patch adds support for "state" option to the NFLOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

What is the exact purpose of the "state" option? The use of the term "state" for this seems to me a bit confusing as users may think that it
is related with "stateful filtering". Please, develop the idea a bit more.

I have kept back patches 8/14 to 13/14 until we end discussing this.

Quoted 1 line

This patch also fixes a bug in definition of seq_global_ce macro.

Please, split this into two patches next time as they are not related.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#21 lore raw

Re: [ULOGD PATCH 08/14] Add state option to NFLOG input plugin.

Quoted 8 lines · Pablo Neira Ayuso
Quoted 3 lines · Eric Leblond

This patch adds support for "state" option to the NFLOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

What is the exact purpose of the "state" option? The use of the term "state" for this seems to me a bit confusing as users may think that it
is related with "stateful filtering". Please, develop the idea a bit more.

Hmm, you are quiet right, 'state' may be a bad choice. In fact, the idea is to be able to give a context relative to the logging. For example, I planned to use it with:
if state = 0 then packet has been dropped (typical -j NFLOG followed by -j DROP) if state = 1 then this is -j NFLOG followed by -j ACCEPT
With that usage choice, the word 'decision' would be better than 'state'.

I think other people may think to other usage. For example, it could be used to indicate the severity of the logged "attack". In this case, the 'decision' keyword is not really a good choice.

The only words that came to my mind and that would be better than 'state' are 'flag' or 'context' but you may have a better idea.

Quoted 5 lines

I have kept back patches 8/14 to 13/14 until we end discussing this.

Quoted 1 line

This patch also fixes a bug in definition of seq_global_ce macro.

Please, split this into two patches next time as they are not related.

Ok.

BR,

Signature · 3 lines
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/
#22 lore raw

Re: [ULOGD PATCH 08/14] Add state option to NFLOG input plugin.

Quoted 16 lines · Eric Leblond
Quoted 7 lines · Pablo Neira Ayuso
Quoted 3 lines · Eric Leblond

This patch adds support for "state" option to the NFLOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

What is the exact purpose of the "state" option? The use of the term "state" for this seems to me a bit confusing as users may think that it
is related with "stateful filtering". Please, develop the idea a bit more.

Hmm, you are quiet right, 'state' may be a bad choice. In fact, the idea is to be able to give a context relative to the logging. For example, I planned to use it with:
if state = 0 then packet has been dropped (typical -j NFLOG followed by -j DROP) if state = 1 then this is -j NFLOG followed by -j ACCEPT
With that usage choice, the word 'decision' would be better than 'state'.

Quoted 3 lines

I think other people may think to other usage. For example, it could be used to indicate the severity of the logged "attack". In this case, the 'decision' keyword is not really a good choice.

I see, however, why isn't --nflog-prefix enough to label the logs from iptables?

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#23 lore raw

Re: [ULOGD PATCH 08/14] Add state option to NFLOG input plugin.

Hello,

Quoted 17 lines · Pablo Neira Ayuso
Quoted 16 lines · Eric Leblond
Quoted 7 lines · Pablo Neira Ayuso
Quoted 3 lines · Eric Leblond

This patch adds support for "state" option to the NFLOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

What is the exact purpose of the "state" option? The use of the term "state" for this seems to me a bit confusing as users may think that it
is related with "stateful filtering". Please, develop the idea a bit more.

Hmm, you are quiet right, 'state' may be a bad choice. In fact, the idea is to be able to give a context relative to the logging. For example, I planned to use it with:
if state = 0 then packet has been dropped (typical -j NFLOG followed by -j DROP) if state = 1 then this is -j NFLOG followed by -j ACCEPT
With that usage choice, the word 'decision' would be better than 'state'.

Quoted 6 lines
Quoted 3 lines

I think other people may think to other usage. For example, it could be used to indicate the severity of the logged "attack". In this case, the 'decision' keyword is not really a good choice.

I see, however, why isn't --nflog-prefix enough to label the logs from iptables?

Yes, but it forces user to define a system to be able to know if decision is DROP or ACCEPT. For example, all prefix have to be labelled like [DA]:$MYSTRING. Well, it works but it overload prefix which is less human readable. Furthermore, it steals some bits in the prefix field which is in ULOG rather small.

BR,

Signature · 3 lines
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/
#24 lore raw

Re: [ULOGD PATCH 08/14] Add state option to NFLOG input plugin.

Hello,

Quoted 8 lines · Eric Leblond

Hello,

Quoted 5 lines · Pablo Neira Ayuso
Quoted 3 lines · Eric Leblond
Quoted 1 line · Pablo Neira Ayuso

Eric Leblond wrote:

'decision' keyword is not really a good choice.

Quoted 8 lines
Quoted 2 lines

I see, however, why isn't --nflog-prefix enough to label the logs from iptables?

Yes, but it forces user to define a system to be able to know if decision is DROP or ACCEPT. For example, all prefix have to be labelled like [DA]:$MYSTRING. Well, it works but it overload prefix which is less human readable. Furthermore, it steals some bits in the prefix field which is in ULOG rather small.

Not receiving a response to this mail make me thing about what I will need to do to have the same feature without the 'state' flag:

A standard logging packet logging table contains packet that have been dropped or accepted. One of the most relevant question for an administrator is: "Which packet have been dropped with this criteria ?". Thus, we need an easy and *fast* mean to differentiate dropped and aceepted packet. IMHO, a 'state' flag is one of the easiest and cleanest solution. The computation overload is really small compares to a parsing of the prefix. Furthermore, this approach is not disruptive with firewall rules generator who already have their log prefix.

BR,

Signature · 3 lines
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/
#25 lore raw

Re: [ULOGD PATCH 08/14] Add state option to NFLOG input plugin.

Quoted 21 lines · Eric Leblond

Hello,

Quoted 6 lines · Eric Leblond

Hello,

Quoted 3 lines · Pablo Neira Ayuso
Quoted 2 lines · Eric Leblond
Quoted 1 line · Pablo Neira Ayuso

Eric Leblond wrote:

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

'decision' keyword is not really a good choice.

I see, however, why isn't --nflog-prefix enough to label the logs from iptables?

Yes, but it forces user to define a system to be able to know if decision is DROP or ACCEPT. For example, all prefix have to be labelled like [DA]:$MYSTRING. Well, it works but it overload prefix which is less human readable. Furthermore, it steals some bits in the prefix field which is in ULOG rather small.

Not receiving a response to this mail make me thing about what I will need to do to have the same feature without the 'state' flag:

Quoted 8 lines

A standard logging packet logging table contains packet that have been dropped or accepted. One of the most relevant question for an administrator is: "Which packet have been dropped with this criteria ?". Thus, we need an easy and *fast* mean to differentiate dropped and aceepted packet. IMHO, a 'state' flag is one of the easiest and cleanest solution. The computation overload is really small compares to a parsing of the prefix. Furthermore, this approach is not disruptive with firewall rules generator who already have their log prefix.

I see. It makes sense. Please, could you resend this patch using 'label' instead of 'state' which seems more to me more appropriate for this feature?

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#26 lore raw

PATCH 1/7 Add label option to NFLOG input plugin.

This patch adds support for "label" option to the NFLOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

This patch also fixes a bug in definition of seq_global_ce macro.

---
:100644 100644 195f6cc... e67ae7a... M input/packet/ulogd_inppkt_NFLOG.c

 input/packet/ulogd_inppkt_NFLOG.c |   24 +++++++++++++++++++++---
 1 files changed, 21 insertions(+), 3 deletions(-)
input/packet/ulogd_inppkt_NFLOG.c +21 -3
index 195f6cc..e67ae7a 100644
@@ -34,7 +34,7 @@ struct nflog_input {
 /* configuration entries */
 
 static struct config_keyset libulog_kset = {
-	.num_ces = 7,
+	.num_ces = 8,
 	.ces = {
 		{
 			.key 	 = "bufsize",
@@ -78,6 +78,13 @@ static struct config_keyset libulog_kset = {
 			.options = CONFIG_OPT_NONE,
 			.u.value = 0,
 		},
+		{
+			.key	 = "label",
+			.type	 = CONFIG_TYPE_INT,
+			.options = CONFIG_OPT_NONE,
+			.u.value = 0,
+		},
+
 	}
 };
 
@@ -86,8 +93,9 @@ static struct config_keyset libulog_kset = {
 #define rmem_ce(x)	(x->ces[2])
 #define af_ce(x)	(x->ces[3])
 #define unbind_ce(x)	(x->ces[4])
-#define seq_ce(x)	(x->ces[4])
-#define seq_global_ce(x)	(x->ces[5])
+#define seq_ce(x)	(x->ces[5])
+#define seq_global_ce(x)	(x->ces[6])
+#define label_ce(x)	(x->ces[7])
 
 enum nflog_keys {
 	NFLOG_KEY_RAW_MAC = 0,
@@ -107,6 +115,7 @@ enum nflog_keys {
 	NFLOG_KEY_OOB_FAMILY,
 	NFLOG_KEY_OOB_PROTOCOL,
 	NFLOG_KEY_OOB_UID,
+	NFLOG_KEY_RAW_STATE,
 };
 
 static struct ulogd_key output_keys[] = {
@@ -246,6 +255,12 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.uid",
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "raw.label",
+	},
+
 
 };
 
@@ -270,6 +285,9 @@ interp_packet(struct ulogd_pluginstance *upi, struct nflog_data *ldata)
 	ret[NFLOG_KEY_OOB_FAMILY].u.value.ui8 = af_ce(upi->config_kset).u.value;
 	ret[NFLOG_KEY_OOB_FAMILY].flags |= ULOGD_RETF_VALID;
 
+	ret[NFLOG_KEY_RAW_STATE].u.value.ui8 = label_ce(upi->config_kset).u.value;
+	ret[NFLOG_KEY_RAW_STATE].flags |= ULOGD_RETF_VALID;
+
 	if (ph) {
 		/* FIXME */
 		ret[NFLOG_KEY_OOB_HOOK].u.value.ui8 = ph->hook;
Signature · 1 line
1.5.2.5
#27 lore raw

Re: [ULOGD PATCH 1/7] Add label option to NFLOG input plugin.

input/packet/ulogd_inppkt_NFLOG.c static struct config_keyset libulog_kset = { · line 89

Quoted 13 lines · Eric Leblond
input/packet/ulogd_inppkt_NFLOG.c +7 -0
@@ -78,6 +78,13 @@ static struct config_keyset libulog_kset = {
 			.options = CONFIG_OPT_NONE,
 			.u.value = 0,
 		},
+		{
+			.key	 = "label",
+			.type	 = CONFIG_TYPE_INT,
+			.options = CONFIG_OPT_NONE,
+			.u.value = 0,
+		},
+
 	}
 };

Wouldn't it better if we use a string instead of an integer? Thus, this will really work as labeler.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#28 lore raw

Re: [ULOGD PATCH 1/7] Add label option to NFLOG input plugin.

Hello,

Quoted 17 lines · Pablo Neira Ayuso

input/packet/ulogd_inppkt_NFLOG.c static struct config_keyset libulog_kset = { · line 89

Quoted 13 lines · Eric Leblond
input/packet/ulogd_inppkt_NFLOG.c +7 -0
@@ -78,6 +78,13 @@ static struct config_keyset libulog_kset = {
 			.options = CONFIG_OPT_NONE,
 			.u.value = 0,
 		},
+		{
+			.key	 = "label",
+			.type	 = CONFIG_TYPE_INT,
+			.options = CONFIG_OPT_NONE,
+			.u.value = 0,
+		},
+
 	}
 };

Wouldn't it better if we use a string instead of an integer? Thus, this will really work as labeler.

We already have the log prefix for a string label usage. I think it will brought confusion to have both a string log prefix and a string label. Furthermore, the goal of the label is to be able to classify packets and an integer will be faster (in the database), stricter (to avoid confusion) and smaller (to store) than a string.

BR,

Signature · 3 lines
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/
#29 lore raw

Re: [ULOGD PATCH 1/7] Add label option to NFLOG input plugin.

Quoted 20 lines · Eric Leblond

Hello,

Quoted 16 lines · Pablo Neira Ayuso

input/packet/ulogd_inppkt_NFLOG.c static struct config_keyset libulog_kset = { · line 89

Quoted 13 lines · Eric Leblond
input/packet/ulogd_inppkt_NFLOG.c +7 -0
@@ -78,6 +78,13 @@ static struct config_keyset libulog_kset = {
 			.options = CONFIG_OPT_NONE,
 			.u.value = 0,
 		},
+		{
+			.key	 = "label",
+			.type	 = CONFIG_TYPE_INT,
+			.options = CONFIG_OPT_NONE,
+			.u.value = 0,
+		},
+
 	}
 };

Wouldn't it better if we use a string instead of an integer? Thus, this will really work as labeler.

Quoted 5 lines

We already have the log prefix for a string label usage. I think it will brought confusion to have both a string log prefix and a string label. Furthermore, the goal of the label is to be able to classify packets and an integer will be faster (in the database), stricter (to avoid confusion) and smaller (to store) than a string.

OK, then I'd prefer if we change the name again to "numberic_label". Sorry for the annoyance.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#30 lore raw

cover Resend: rename label to numeric_label

Hi,

Here's the resend of my patchet with label being renamed as numeric_label. IMHO, we would use raw_label in the SQL schema and thus already sent patches should be applied.

BR,
--
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/

#31 lore raw

PATCH 1/3 Add numeric_label option to ULOG input plugin.

This patch adds support for "numeric_label" option to the ULOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

---
 input/packet/ulogd_inppkt_ULOG.c |   18 +++++++++++++++++-
 1 files changed, 17 insertions(+), 1 deletions(-)
input/packet/ulogd_inppkt_ULOG.c +17 -1
index 5112281..253bbe0 100644
@@ -34,7 +34,7 @@ struct ulog_input {
 /* configuration entries */
 
 static struct config_keyset libulog_kset = {
-	.num_ces = 3,
+	.num_ces = 4,
 	.ces = {
 	{
 		.key 	 = "bufsize",
@@ -54,6 +54,13 @@ static struct config_keyset libulog_kset = {
 		.options = CONFIG_OPT_NONE,
 		.u.value = ULOGD_RMEM_DEFAULT,
 	},
+	{
+		.key	 = "numeric_label",
+		.type	 = CONFIG_TYPE_INT,
+		.options = CONFIG_OPT_NONE,
+		.u.value = 0,
+	},
+
 	}
 };
 enum ulog_keys {
@@ -71,6 +78,7 @@ enum ulog_keys {
 	ULOG_KEY_RAW_MAC_LEN,
 	ULOG_KEY_OOB_FAMILY,
 	ULOG_KEY_OOB_PROTOCOL,
+	ULOG_KEY_RAW_LABEL,
 };
 
 static struct ulogd_key output_keys[] = {
@@ -167,6 +175,11 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.protocol",
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "raw.label",
+	},
 
 };
 
@@ -181,6 +194,9 @@ static int interp_packet(struct ulogd_pluginstance *ip, ulog_packet_msg_t *pkt)
 		ret[ULOG_KEY_RAW_MAC_LEN].flags |= ULOGD_RETF_VALID;
 	}
 
+	ret[ULOG_KEY_RAW_LABEL].u.value.ui8 = ip->config_kset->ces[3].u.value;
+	ret[ULOG_KEY_RAW_LABEL].flags |= ULOGD_RETF_VALID;
+
 	/* include pointer to raw ipv4 packet */
 	ret[ULOG_KEY_RAW_PCKT].u.value.ptr = pkt->payload;
 	ret[ULOG_KEY_RAW_PCKT].flags |= ULOGD_RETF_VALID;
Signature · 1 line
1.5.4.3
#32 lore raw

PATCH 2/3 Add numeric_label option to NFLOG input plugin.

This patch adds support for "numeric_label" option to the NFLOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

---
 input/packet/ulogd_inppkt_NFLOG.c |   20 +++++++++++++++++++-
 1 files changed, 19 insertions(+), 1 deletions(-)
input/packet/ulogd_inppkt_NFLOG.c +19 -1
index 9887e86..bb69bb6 100644
@@ -34,7 +34,7 @@ struct nflog_input {
 /* configuration entries */
 
 static struct config_keyset libulog_kset = {
-	.num_ces = 7,
+	.num_ces = 8,
 	.ces = {
 		{
 			.key 	 = "bufsize",
@@ -78,6 +78,13 @@ static struct config_keyset libulog_kset = {
 			.options = CONFIG_OPT_NONE,
 			.u.value = 0,
 		},
+		{
+			.key	 = "numeric_label",
+			.type	 = CONFIG_TYPE_INT,
+			.options = CONFIG_OPT_NONE,
+			.u.value = 0,
+		},
+
 	}
 };
 
@@ -88,6 +95,7 @@ static struct config_keyset libulog_kset = {
 #define unbind_ce(x)	(x->ces[4])
 #define seq_ce(x)	(x->ces[5])
 #define seq_global_ce(x)	(x->ces[6])
+#define label_ce(x)	(x->ces[7])
 
 enum nflog_keys {
 	NFLOG_KEY_RAW_MAC = 0,
@@ -108,6 +116,7 @@ enum nflog_keys {
 	NFLOG_KEY_OOB_PROTOCOL,
 	NFLOG_KEY_OOB_UID,
 	NFLOG_KEY_OOB_GID,
+	NFLOG_KEY_RAW_LABEL,
 };
 
 static struct ulogd_key output_keys[] = {
@@ -252,6 +261,12 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.gid",
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "raw.label",
+	},
+
 };
 
 static inline int 
@@ -275,6 +290,9 @@ interp_packet(struct ulogd_pluginstance *upi, struct nflog_data *ldata)
 	ret[NFLOG_KEY_OOB_FAMILY].u.value.ui8 = af_ce(upi->config_kset).u.value;
 	ret[NFLOG_KEY_OOB_FAMILY].flags |= ULOGD_RETF_VALID;
 
+	ret[NFLOG_KEY_RAW_LABEL].u.value.ui8 = label_ce(upi->config_kset).u.value;
+	ret[NFLOG_KEY_RAW_LABEL].flags |= ULOGD_RETF_VALID;
+
 	if (ph) {
 		/* FIXME */
 		ret[NFLOG_KEY_OOB_HOOK].u.value.ui8 = ph->hook;
Signature · 1 line
1.5.4.3
#33 lore raw

PATCH 3/3 Update default configuration to fit last changes

Default stacks needed to be modified to be in sync MAC2STR plugin which is used by database output modules. The new label configuration variable has been added to some ULOG and NFLOG instances to show that it exists.

---
 ulogd.conf.in |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
ulogd.conf.in +2 -0
index 3def8d1..7f78242 100644
@@ -97,6 +97,7 @@ group=0
 [log2]
 group=1 # Group has to be different from the one use in log1
 addressfamily=10 # 10 is value of AF_INET6
+numeric_label=1 # label can be used to define the decision applied to packet
 
 # ebtables logging through NFLOG
 [log3]
@@ -106,6 +107,7 @@ addressfamily=7 # 7 is value of AF_BRIDGE
 [ulog1]
 # netlink multicast group (the same as the iptables --ulog-nlgroup param)
 nlgroup=1
+#numeric_label=0 # optionnal argument
 
 [emu1]
 file="/var/log/ulogd_syslogemu.log"
Signature · 1 line
1.5.4.3
#34 lore raw

Re: [ULOGD PATCH 0/3] Resend: rename label to numeric_label

In reply to Eric Leblond
Quoted 5 lines · Eric Leblond

Hi,

Here's the resend of my patchet with label being renamed as numeric_label. IMHO, we would use raw_label in the SQL schema and thus already sent patches should be applied.

I have applied your patches and lastest Pierre's 2/4.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
#35 lore raw

PATCH 2/7 Add label support to MySQL schema.

In reply to Pablo Neira Ayuso

This patch adds support for label_t table in the MySQL schema.

---
:100644 100644 ed8d982... 72575b3... M doc/mysql-ulogd2.sql

 doc/mysql-ulogd2.sql |   37 ++++++++++++++++++++++++++++++++++---
 1 files changed, 34 insertions(+), 3 deletions(-)
doc/mysql-ulogd2.sql +34 -3
index ed8d982..72575b3 100644
@@ -139,6 +139,19 @@ CREATE TABLE `icmpv6` (
 ALTER TABLE icmpv6 ADD UNIQUE KEY `key_icmpv6_id` (`_icmpv6_id`);
 ALTER TABLE icmpv6 ADD KEY `index_icmpv6_id` (`_icmpv6_id`);
 
+-- State
+CREATE TABLE `label_t` (
+  `_label_id` bigint unsigned NOT NULL,
+  label tinyint(3) unsigned
+) ENGINE=INNODB;
+
+ALTER TABLE label_t ADD UNIQUE KEY `_label_id` (`_label_id`);
+ALTER TABLE label_t ADD KEY `index_label_id` (`_label_id`);
+ALTER TABLE label_t ADD KEY `label` (`label`);
+ALTER TABLE label_t ADD FOREIGN KEY (_label_id) REFERENCES ulog2 (_id);
+
+INSERT INTO _extensions (ext_name,table_name,join_name) VALUES
+        ('label','label_t','_label_id');
 
 -- views
 
@@ -213,10 +226,11 @@ CREATE SQL SECURITY INVOKER VIEW `ulog` AS
 	icmpv6_echoseq,
 	icmpv6_csum,
 	mac_saddr as mac_saddr_str,
-	mac_protocol as oob_protocol
+	mac_protocol as oob_protocol,
+	label as raw_label
         FROM ulog2 LEFT JOIN tcp ON ulog2._id = tcp._tcp_id LEFT JOIN udp ON ulog2._id = udp._udp_id
                 LEFT JOIN icmp ON ulog2._id = icmp._icmp_id LEFT JOIN mac ON ulog2._id = mac._mac_id
-                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id;
+                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id LEFT JOIN label_t ON ulog2._id = label_t._label_id;
 
 
 -- shortcuts
@@ -578,6 +592,18 @@ END
 $$
 
 delimiter $$
+DROP PROCEDURE IF EXISTS PACKET_ADD_LABEL;
+CREATE PROCEDURE PACKET_ADD_LABEL(
+		IN `id` int(10) unsigned,
+		IN `_label` tinyint(4)
+		)
+BEGIN
+	INSERT INTO label_t (_label_id, label) VALUES (id, _label);
+END
+$$
+
+
+delimiter $$
 DROP FUNCTION IF EXISTS INSERT_PACKET_FULL;
 CREATE FUNCTION INSERT_PACKET_FULL(
 		_oob_time_sec int(10) unsigned,
@@ -625,7 +651,8 @@ CREATE FUNCTION INSERT_PACKET_FULL(
 		icmpv6_echoseq smallint(5) unsigned,
 		icmpv6_csum int(10) unsigned,
 		mac_saddr varchar(32),
-		mac_protocol smallint(5)
+		mac_protocol smallint(5),
+		label tinyint(4)
 		) RETURNS bigint unsigned
 READS SQL DATA
 BEGIN
@@ -650,6 +677,10 @@ BEGIN
 	IF mac_protocol IS NOT NULL THEN
 		CALL PACKET_ADD_MAC(@lastid, mac_saddr, mac_protocol);
 	END IF;
+	IF label IS NOT NULL THEN
+		CALL PACKET_ADD_LABEL(@lastid, label);
+	END IF;
+
 	RETURN @lastid;
 END
 $$
Signature · 1 line
1.5.2.5
#36 lore raw

PATCH 3/7 Add label option to ULOG input plugin.

In reply to Pablo Neira Ayuso

This patch adds support for "label" option to the ULOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

---
:100644 100644 d30c07e... c3a44da... M input/packet/ulogd_inppkt_ULOG.c

 input/packet/ulogd_inppkt_ULOG.c |   18 +++++++++++++++++-
 1 files changed, 17 insertions(+), 1 deletions(-)
input/packet/ulogd_inppkt_ULOG.c +17 -1
index d30c07e..c3a44da 100644
@@ -34,7 +34,7 @@ struct ulog_input {
 /* configuration entries */
 
 static struct config_keyset libulog_kset = {
-	.num_ces = 3,
+	.num_ces = 4,
 	.ces = {
 	{
 		.key 	 = "bufsize",
@@ -54,6 +54,13 @@ static struct config_keyset libulog_kset = {
 		.options = CONFIG_OPT_NONE,
 		.u.value = ULOGD_RMEM_DEFAULT,
 	},
+	{
+		.key	 = "label",
+		.type	 = CONFIG_TYPE_INT,
+		.options = CONFIG_OPT_NONE,
+		.u.value = 0,
+	},
+
 	}
 };
 enum ulog_keys {
@@ -70,6 +77,7 @@ enum ulog_keys {
 	ULOG_KEY_RAW_MAC_LEN,
 	ULOG_KEY_OOB_FAMILY,
 	ULOG_KEY_OOB_PROTOCOL,
+	ULOG_KEY_RAW_STATE,
 };
 
 static struct ulogd_key output_keys[] = {
@@ -157,6 +165,11 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.protocol",
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "raw.label",
+	},
 
 };
 
@@ -171,6 +184,9 @@ static int interp_packet(struct ulogd_pluginstance *ip, ulog_packet_msg_t *pkt)
 		ret[ULOG_KEY_RAW_MAC_LEN].flags |= ULOGD_RETF_VALID;
 	}
 
+	ret[ULOG_KEY_RAW_STATE].u.value.ui8 = ip->config_kset->ces[3].u.value;
+	ret[ULOG_KEY_RAW_STATE].flags |= ULOGD_RETF_VALID;
+
 	/* include pointer to raw ipv4 packet */
 	ret[ULOG_KEY_RAW_PCKT].u.value.ptr = pkt->payload;
 	ret[ULOG_KEY_RAW_PCKT].flags |= ULOGD_RETF_VALID;
Signature · 1 line
1.5.2.5
#37 lore raw

PATCH 4/7 Add hook output to ULOG input module.

In reply to Pablo Neira Ayuso

This patch adds oob.hook to the list of output key sof ULOG input plugin.

---
:100644 100644 c3a44da... ad88f28... M input/packet/ulogd_inppkt_ULOG.c

 input/packet/ulogd_inppkt_ULOG.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)
input/packet/ulogd_inppkt_ULOG.c +13 -0
index c3a44da..ad88f28 100644
@@ -74,6 +74,7 @@ enum ulog_keys {
 	ULOG_KEY_OOB_MARK,
 	ULOG_KEY_OOB_IN,
 	ULOG_KEY_OOB_OUT,
+	ULOG_KEY_OOB_HOOK,
 	ULOG_KEY_RAW_MAC_LEN,
 	ULOG_KEY_OOB_FAMILY,
 	ULOG_KEY_OOB_PROTOCOL,
@@ -150,6 +151,15 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.out", 
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "oob.hook",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_NETFILTER,
+			.field_id = IPFIX_NF_hook,
+		},
+	},
 	{ 
 		.type = ULOGD_RET_UINT16, 
 		.flags = ULOGD_RETF_NONE, 
@@ -217,6 +227,9 @@ static int interp_packet(struct ulogd_pluginstance *ip, ulog_packet_msg_t *pkt)
 	ret[ULOG_KEY_OOB_OUT].u.value.ptr = pkt->outdev_name;
 	ret[ULOG_KEY_OOB_OUT].flags |= ULOGD_RETF_VALID;
 
+	ret[ULOG_KEY_OOB_HOOK].u.value.ui8 = pkt->hook;
+	ret[ULOG_KEY_OOB_HOOK].flags |= ULOGD_RETF_VALID;
+
 	/* ULOG is IPv4 only */
 	ret[ULOG_KEY_OOB_FAMILY].u.value.ui8 = AF_INET;
 	ret[ULOG_KEY_OOB_FAMILY].flags |= ULOGD_RETF_VALID;
Signature · 1 line
1.5.2.5
#38 lore raw

PATCH 5/7 Add label support to PGSQL output module.

In reply to Pablo Neira Ayuso

This patch adds raw.label support to the PGSQL module. It creates a label_t table to store the label of the packets.

---
:100644 100644 cfef267... c974744... M doc/pgsql-ulogd2.sql

 doc/pgsql-ulogd2.sql |   40 ++++++++++++++++++++++++++++++++++------
 1 files changed, 34 insertions(+), 6 deletions(-)
doc/pgsql-ulogd2.sql +34 -6
index cfef267..c974744 100644
@@ -25,6 +25,7 @@ CREATE TABLE _extensions (
   join_name varchar(64) NOT NULL
 ) WITH (OIDS=FALSE);
 
+DROP TABLE IF EXISTS label_t CASCADE;
 DROP TABLE IF EXISTS mac CASCADE;
 DROP TABLE IF EXISTS tcp CASCADE;
 DROP TABLE IF EXISTS udp CASCADE;
@@ -65,6 +66,13 @@ CREATE INDEX ulog2_ip_saddr ON ulog2(ip_saddr_str);
 CREATE INDEX ulog2_ip_daddr ON ulog2(ip_daddr_str);
 CREATE INDEX ulog2_timestamp ON ulog2(timestamp);
 
+CREATE TABLE label_t (
+  _label_id bigint PRIMARY KEY UNIQUE NOT NULL,
+  label smallint default NULL
+) WITH (OIDS=FALSE);
+
+CREATE INDEX label ON label_t(label);
+
 CREATE TABLE mac (
   _mac_id bigint PRIMARY KEY UNIQUE NOT NULL,
   mac_saddr macaddr default NULL,
@@ -191,10 +199,12 @@ CREATE OR REPLACE VIEW ulog AS
         icmpv6_echoseq,
         icmpv6_csum,
         mac_saddr AS mac_saddr_str,
-        mac_protocol AS oob_protocol
+        mac_protocol AS oob_protocol,
+        label AS raw_label
         FROM ulog2 LEFT JOIN tcp ON ulog2._id = tcp._tcp_id LEFT JOIN udp ON ulog2._id = udp._udp_id
                 LEFT JOIN icmp ON ulog2._id = icmp._icmp_id LEFT JOIN mac ON ulog2._id = mac._mac_id
-                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id;
+                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id
+                LEFT JOIN label_t ON ulog2._id = label_t._label_id;
 
 -- shortcuts
 CREATE OR REPLACE VIEW view_tcp_quad AS
@@ -306,6 +316,8 @@ INSERT INTO _extensions (ext_name,table_name,join_name) VALUES
 
 CREATE OR REPLACE FUNCTION ULOG2_DROP_FOREIGN_KEYS()
 RETURNS void AS $$
+  ALTER TABLE label_t DROP CONSTRAINT label_id_fk;
+  ALTER TABLE mac DROP CONSTRAINT mac_id_fk;
   ALTER TABLE icmpv6 DROP CONSTRAINT icmpv6_id_fk;
   ALTER TABLE icmp DROP CONSTRAINT icmp_id_fk;
   ALTER TABLE udp  DROP CONSTRAINT udp_id_fk;
@@ -319,6 +331,8 @@ RETURNS void AS $$
   ALTER TABLE udp  ADD CONSTRAINT udp_id_fk  FOREIGN KEY (_udp_id)  REFERENCES ulog2(_id);
   ALTER TABLE icmp ADD CONSTRAINT icmp_id_fk FOREIGN KEY (_icmp_id) REFERENCES ulog2(_id);
   ALTER TABLE icmpv6 ADD CONSTRAINT icmpv6_id_fk FOREIGN KEY (_icmpv6_id) REFERENCES ulog2(_id);
+  ALTER TABLE mac ADD CONSTRAINT mac_id_fk FOREIGN KEY (_mac_id) REFERENCES ulog2(_id);
+  ALTER TABLE label_t ADD CONSTRAINT label_id_fk FOREIGN KEY (_label_id) REFERENCES ulog2(_id);
 $$ LANGUAGE SQL SECURITY INVOKER;
 
 
@@ -433,6 +447,16 @@ RETURNS bigint AS $$
         SELECT currval('ulog2__id_seq');
 $$ LANGUAGE SQL SECURITY INVOKER;
 
+CREATE OR REPLACE FUNCTION INSERT_STATE(
+                IN label_id bigint,
+                IN label integer
+        )
+RETURNS bigint AS $$
+        INSERT INTO label_t (_label_id,label)
+                VALUES ($1,$2);
+        SELECT currval('ulog2__id_seq');
+$$ LANGUAGE SQL SECURITY INVOKER;
+
 CREATE OR REPLACE FUNCTION INSERT_MAC(
                 IN mac_id bigint,
                 IN mac_saddr macaddr,
@@ -492,7 +516,8 @@ CREATE OR REPLACE FUNCTION INSERT_PACKET_FULL(
                 IN icmpv6_echoseq integer,
                 IN icmpv6_csum integer,
                 IN mac_saddr varchar(32),
-                IN mac_protocol integer
+                IN mac_protocol integer,
+                IN label integer
         )
 RETURNS bigint AS $$
 DECLARE
@@ -511,13 +536,13 @@ BEGIN
         IF (mac_saddr IS NOT NULL) THEN
                 PERFORM INSERT_MAC(_id,$45::macaddr,$46);
         END IF;
+        IF (label IS NOT NULL) THEN
+                PERFORM INSERT_STATE(_id,$47);
+        END IF;
         RETURN _id;
 END
 $$ LANGUAGE plpgsql SECURITY INVOKER;
 
-
-
-
 CREATE OR REPLACE FUNCTION DELETE_PACKET(
                 IN _packet_id bigint
         )
@@ -526,6 +551,8 @@ RETURNS void AS $$
   DELETE FROM icmp  WHERE icmp._icmp_id = $1;
   DELETE FROM tcp   WHERE tcp._tcp_id   = $1;
   DELETE FROM udp   WHERE udp._udp_id   = $1;
+  DELETE FROM mac   WHERE mac._mac_id   = $1;
+  DELETE FROM label_t   WHERE label_t._label_id   = $1;
   DELETE FROM ulog2 WHERE ulog2._id     = $1;
 $$ LANGUAGE SQL SECURITY INVOKER;
 
@@ -564,6 +591,7 @@ RETURNS void AS $$
   DELETE FROM tcp WHERE _tcp_id NOT IN (SELECT _id FROM ulog2);
   -- XXX note: could be rewritten (need to see what is more efficient) as:
   -- DELETE FROM tcp WHERE _tcp_id IN (SELECT tcp._tcp_id FROM tcp LEFT OUTER JOIN ulog2  ON (tcp._tcp_id = ulog2._id) WHERE ulog2._id IS NULL);
+  DELETE FROM label_t WHERE _label_id NOT IN (SELECT _id FROM ulog2);
   DELETE FROM mac WHERE _mac_id NOT IN (SELECT _id FROM ulog2);
   DELETE FROM udp WHERE _udp_id NOT IN (SELECT _id FROM ulog2);
   DELETE FROM icmp WHERE _icmp_id NOT IN (SELECT _id FROM ulog2);
Signature · 1 line
1.5.2.5
#39 lore raw

PATCH 6/7 Update default configuration to fit last changes

In reply to Pablo Neira Ayuso

Default stacks needed to be modified to be in sync MAC2STR plugin which is used by database output modules. The new label configuration variable has been added to some ULOG and NFLOG instances to show that it exists.

---
:100644 100644 3def8d1... 578d56b... M ulogd.conf.in

 ulogd.conf.in |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
ulogd.conf.in +2 -0
index 3def8d1..578d56b 100644
@@ -97,6 +97,7 @@ group=0
 [log2]
 group=1 # Group has to be different from the one use in log1
 addressfamily=10 # 10 is value of AF_INET6
+label=1 # State can be used to define the decision applied to packet
 
 # ebtables logging through NFLOG
 [log3]
@@ -106,6 +107,7 @@ addressfamily=7 # 7 is value of AF_BRIDGE
 [ulog1]
 # netlink multicast group (the same as the iptables --ulog-nlgroup param)
 nlgroup=1
+#label=0 # optionnal argument
 
 [emu1]
 file="/var/log/ulogd_syslogemu.log"
Signature · 1 line
1.5.2.5
#40 lore raw

PATCH 7/7 Convert SQL procedure to function in MySQL plugins.

In reply to Pablo Neira Ayuso

DB output plugins have been modified to use function instead of procedure. This patch convert the conntrack insertion procedure to function.

---
:100644 100644 72575b3... b40c961... M doc/mysql-ulogd2.sql

 doc/mysql-ulogd2.sql |   56 +++++++++++++++++++++++++------------------------
 1 files changed, 29 insertions(+), 27 deletions(-)
doc/mysql-ulogd2.sql +29 -27
index 72575b3..b40c961 100644
@@ -703,46 +703,48 @@ END
 $$
 
 delimiter $$
-DROP PROCEDURE IF EXISTS INSERT_CT;
-CREATE PROCEDURE INSERT_CT(
-		IN `_orig_ip_saddr` binary(16),
-		IN `_orig_ip_daddr` binary(16),
-		IN `_orig_ip_protocol` tinyint(3) unsigned,
-		IN `_orig_l4_sport` int(5),
-		IN `_orig_l4_dport` int(5),
-		IN `_orig_bytes` bigint,
-		IN `_orig_packets` bigint,
-		IN `_reply_ip_saddr` binary(16),
-		IN `_reply_ip_daddr` binary(16),
-		IN `_reply_ip_protocol` tinyint(3) unsigned,
-		IN `_reply_l4_sport` int(5),
-		IN `_reply_l4_dport` int(5),
-		IN `_reply_bytes` bigint,
-		IN `_reply_packets` bigint,
-		IN `_icmp_code` tinyint(3),
-		IN `_icmp_type` tinyint(3),
-		IN `_ct_mark` bigint,
-		IN `_flow_start_sec` int(10),
-		IN `_flow_start_usec` int(10),
-		IN `_flow_end_sec` int(10),
-		IN `_flow_end_usec` int(10)
-		)
+DROP FUNCTION IF EXISTS INSERT_CT;
+CREATE FUNCTION INSERT_CT(
+		`_oob_family` bigint,
+		`_orig_ip_saddr` binary(16),
+		`_orig_ip_daddr` binary(16),
+		`_orig_ip_protocol` tinyint(3) unsigned,
+		`_orig_l4_sport` int(5),
+		`_orig_l4_dport` int(5),
+		`_orig_bytes` bigint,
+		`_orig_packets` bigint,
+		`_reply_ip_saddr` binary(16),
+		`_reply_ip_daddr` binary(16),
+		`_reply_ip_protocol` tinyint(3) unsigned,
+		`_reply_l4_sport` int(5),
+		`_reply_l4_dport` int(5),
+		`_reply_bytes` bigint,
+		`_reply_packets` bigint,
+		`_icmp_code` tinyint(3),
+		`_icmp_type` tinyint(3),
+		`_ct_mark` bigint,
+		`_flow_start_sec` int(10),
+		`_flow_start_usec` int(10),
+		`_flow_end_sec` int(10),
+		`_flow_end_usec` int(10)
+		) RETURNS bigint unsigned
+READS SQL DATA
 BEGIN
-	INSERT INTO ulog2_ct (orig_ip_saddr, orig_ip_daddr, orig_ip_protocol,
+	INSERT INTO ulog2_ct (oob_family, orig_ip_saddr, orig_ip_daddr, orig_ip_protocol,
 		orig_l4_sport, orig_l4_dport, orig_bytes, orig_packets,
 		reply_ip_saddr, reply_ip_daddr, reply_ip_protocol,
 		reply_l4_sport, reply_l4_dport, reply_bytes, reply_packets,
 		icmp_code, icmp_type, ct_mark, 
 		flow_start_sec, flow_start_usec,
 		flow_end_sec, flow_end_usec)
- 	VALUES (_orig_ip_saddr, _orig_ip_daddr, _orig_ip_protocol,
+ 	VALUES (_oob_family, _orig_ip_saddr, _orig_ip_daddr, _orig_ip_protocol,
 		_orig_l4_sport, _orig_l4_dport, _orig_bytes, _orig_packets,
 		_reply_ip_saddr, _reply_ip_daddr, _reply_ip_protocol,
 		_reply_l4_sport, _reply_l4_dport, _reply_bytes, _reply_packets,
 		_icmp_code, _icmp_type, _ct_mark,
 		_flow_start_sec, _flow_start_usec,
 		_flow_end_sec, _flow_end_usec);
-
+	RETURN LAST_INSERT_ID();
 END
 $$
 
Signature · 1 line
1.5.2.5
#42 lore raw

PATCH 9/14 Add state support to MySQL schema.

In reply to Eric Leblond

This patch adds support for state_t table in the MySQL schema.

---
 doc/mysql-ulogd2.sql |   51 +++++++++++++++++++++++++++++++++----------------
 1 files changed, 34 insertions(+), 17 deletions(-)
doc/mysql-ulogd2.sql +34 -17
index bb31339..304173b 100644
@@ -139,6 +139,19 @@ CREATE TABLE `icmpv6` (
 ALTER TABLE icmpv6 ADD UNIQUE KEY `key_icmpv6_id` (`_icmpv6_id`);
 ALTER TABLE icmpv6 ADD KEY `index_icmpv6_id` (`_icmpv6_id`);
 
+-- State
+CREATE TABLE `state_t` (
+  `_state_id` bigint unsigned NOT NULL,
+  state tinyint(3) unsigned
+) ENGINE=INNODB;
+
+ALTER TABLE state_t ADD UNIQUE KEY `_state_id` (`_state_id`);
+ALTER TABLE state_t ADD KEY `index_state_id` (`_state_id`);
+ALTER TABLE state_t ADD KEY `state` (`state`);
+ALTER TABLE state_t ADD FOREIGN KEY (_state_id) REFERENCES ulog2 (_id);
+
+INSERT INTO _extensions (ext_name,table_name,join_name) VALUES
+        ('state','state_t','_state_id');
 
 -- views
 
@@ -213,10 +226,11 @@ CREATE SQL SECURITY INVOKER VIEW `ulog` AS
 	icmpv6_echoseq,
 	icmpv6_csum,
 	mac_saddr as mac_saddr_str,
-	mac_protocol as oob_protocol
+	mac_protocol as oob_protocol,
+	state as raw_state
         FROM ulog2 LEFT JOIN tcp ON ulog2._id = tcp._tcp_id LEFT JOIN udp ON ulog2._id = udp._udp_id
                 LEFT JOIN icmp ON ulog2._id = icmp._icmp_id LEFT JOIN mac ON ulog2._id = mac._mac_id
-                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id;
+                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id LEFT JOIN state_t ON ulog2._id = state_t._state_id;
 
 
 -- shortcuts
@@ -329,20 +343,6 @@ INSERT INTO ip_proto (_proto_id,proto_name,proto_desc) VALUES
         (41,'ipv6','Internet Protocol, version 6'),
         (58,'ipv6-icmp','ICMP for IPv6');
 
--- State
-CREATE TABLE `state_t` (
-  `_state_id` bigint unsigned NOT NULL,
-  state tinyint(3) unsigned
-) ENGINE=INNODB;
-
-ALTER TABLE state_t ADD UNIQUE KEY `_state_id` (`_state_id`);
-ALTER TABLE state_t ADD KEY `index_state_id` (`_state_id`);
-ALTER TABLE state_t ADD KEY `state` (`state`);
-ALTER TABLE state_t ADD FOREIGN KEY (_state_id) REFERENCES ulog2 (_id);
-
-INSERT INTO _extensions (ext_name,table_name,join_name) VALUES
-        ('state','state_t','_state_id');
-
 -- NuFW specific
 
 CREATE TABLE `nufw` (
@@ -578,6 +578,18 @@ END
 $$
 
 delimiter $$
+DROP PROCEDURE IF EXISTS PACKET_ADD_STATE;
+CREATE PROCEDURE PACKET_ADD_STATE(
+		IN `id` int(10) unsigned,
+		IN `_state` tinyint(4)
+		)
+BEGIN
+	INSERT INTO state_t (_state_id, state) VALUES (id, _state);
+END
+$$
+
+
+delimiter $$
 DROP FUNCTION IF EXISTS INSERT_PACKET_FULL;
 CREATE FUNCTION INSERT_PACKET_FULL(
 		_oob_time_sec int(10) unsigned,
@@ -625,7 +637,8 @@ CREATE FUNCTION INSERT_PACKET_FULL(
 		icmpv6_echoseq smallint(5) unsigned,
 		icmpv6_csum int(10) unsigned,
 		mac_saddr varchar(32),
-		mac_protocol smallint(5)
+		mac_protocol smallint(5),
+		state tinyint(4)
 		) RETURNS bigint unsigned
 READS SQL DATA
 BEGIN
@@ -650,6 +663,10 @@ BEGIN
 	IF mac_protocol IS NOT NULL THEN
 		CALL PACKET_ADD_MAC(@lastid, mac_saddr, mac_protocol);
 	END IF;
+	IF state IS NOT NULL THEN
+		CALL PACKET_ADD_STATE(@lastid, state);
+	END IF;
+
 	RETURN @lastid;
 END
 $$
Signature · 1 line
1.5.2.5
#43 lore raw

PATCH 10/14 Add state option to ULOG input plugin.

In reply to Eric Leblond

This patch adds support for "state" option to the ULOG plugin. For example, it can be used by another module to determine if the packet has been dropped, rejected or accepted.

---
 input/packet/ulogd_inppkt_ULOG.c |   18 +++++++++++++++++-
 1 files changed, 17 insertions(+), 1 deletions(-)
input/packet/ulogd_inppkt_ULOG.c +17 -1
index a1d52f2..9139e7d 100644
@@ -34,7 +34,7 @@ struct ulog_input {
 /* configuration entries */
 
 static struct config_keyset libulog_kset = {
-	.num_ces = 3,
+	.num_ces = 4,
 	.ces = {
 	{
 		.key 	 = "bufsize",
@@ -54,6 +54,13 @@ static struct config_keyset libulog_kset = {
 		.options = CONFIG_OPT_NONE,
 		.u.value = ULOGD_RMEM_DEFAULT,
 	},
+	{
+		.key	 = "state",
+		.type	 = CONFIG_TYPE_INT,
+		.options = CONFIG_OPT_NONE,
+		.u.value = 0,
+	},
+
 	}
 };
 enum ulog_keys {
@@ -70,6 +77,7 @@ enum ulog_keys {
 	ULOG_KEY_RAW_MAC_LEN,
 	ULOG_KEY_OOB_FAMILY,
 	ULOG_KEY_OOB_PROTOCOL,
+	ULOG_KEY_RAW_STATE,
 };
 
 static struct ulogd_key output_keys[] = {
@@ -157,6 +165,11 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.protocol",
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "raw.state",
+	},
 
 };
 
@@ -171,6 +184,9 @@ static int interp_packet(struct ulogd_pluginstance *ip, ulog_packet_msg_t *pkt)
 		ret[ULOG_KEY_RAW_MAC_LEN].flags |= ULOGD_RETF_VALID;
 	}
 
+	ret[ULOG_KEY_RAW_STATE].u.value.ui8 = ip->config_kset->ces[3].u.value;
+	ret[ULOG_KEY_RAW_STATE].flags |= ULOGD_RETF_VALID;
+
 	/* include pointer to raw ipv4 packet */
 	ret[ULOG_KEY_RAW_PCKT].u.value.ptr = pkt->payload;
 	ret[ULOG_KEY_RAW_PCKT].flags |= ULOGD_RETF_VALID;
Signature · 1 line
1.5.2.5
#44 lore raw

PATCH 11/14 Add hook output to ULOG input module.

In reply to Eric Leblond

This patch adds oob.hook to the list of output key sof ULOG input plugin.

---
 input/packet/ulogd_inppkt_ULOG.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)
input/packet/ulogd_inppkt_ULOG.c +13 -0
index 9139e7d..e2a32ba 100644
@@ -74,6 +74,7 @@ enum ulog_keys {
 	ULOG_KEY_OOB_MARK,
 	ULOG_KEY_OOB_IN,
 	ULOG_KEY_OOB_OUT,
+	ULOG_KEY_OOB_HOOK,
 	ULOG_KEY_RAW_MAC_LEN,
 	ULOG_KEY_OOB_FAMILY,
 	ULOG_KEY_OOB_PROTOCOL,
@@ -150,6 +151,15 @@ static struct ulogd_key output_keys[] = {
 		.flags = ULOGD_RETF_NONE,
 		.name = "oob.out", 
 	},
+	{
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "oob.hook",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_NETFILTER,
+			.field_id = IPFIX_NF_hook,
+		},
+	},
 	{ 
 		.type = ULOGD_RET_UINT16, 
 		.flags = ULOGD_RETF_NONE, 
@@ -217,6 +227,9 @@ static int interp_packet(struct ulogd_pluginstance *ip, ulog_packet_msg_t *pkt)
 	ret[ULOG_KEY_OOB_OUT].u.value.ptr = pkt->outdev_name;
 	ret[ULOG_KEY_OOB_OUT].flags |= ULOGD_RETF_VALID;
 
+	ret[ULOG_KEY_OOB_HOOK].u.value.ui8 = pkt->hook;
+	ret[ULOG_KEY_OOB_HOOK].flags |= ULOGD_RETF_VALID;
+
 	/* ULOG is IPv4 only */
 	ret[ULOG_KEY_OOB_FAMILY].u.value.ui8 = AF_INET;
 	ret[ULOG_KEY_OOB_FAMILY].flags |= ULOGD_RETF_VALID;
Signature · 1 line
1.5.2.5
#45 lore raw

PATCH 12/14 Add state support to PGSQL output module.

In reply to Eric Leblond

This patch adds raw.state support to the PGSQL module. It creates a state_t table to store the state of the packets.

---
 doc/pgsql-ulogd2.sql |   40 ++++++++++++++++++++++++++++++++++------
 1 files changed, 34 insertions(+), 6 deletions(-)
doc/pgsql-ulogd2.sql +34 -6
index fc4aa1b..b18f4a8 100644
@@ -25,6 +25,7 @@ CREATE TABLE _extensions (
   join_name varchar(64) NOT NULL
 ) WITH (OIDS=FALSE);
 
+DROP TABLE IF EXISTS state_t CASCADE;
 DROP TABLE IF EXISTS mac CASCADE;
 DROP TABLE IF EXISTS tcp CASCADE;
 DROP TABLE IF EXISTS udp CASCADE;
@@ -65,6 +66,13 @@ CREATE INDEX ulog2_ip_saddr ON ulog2(ip_saddr_str);
 CREATE INDEX ulog2_ip_daddr ON ulog2(ip_daddr_str);
 CREATE INDEX ulog2_timestamp ON ulog2(timestamp);
 
+CREATE TABLE state_t (
+  _state_id bigint PRIMARY KEY UNIQUE NOT NULL,
+  state smallint default NULL
+) WITH (OIDS=FALSE);
+
+CREATE INDEX state ON state_t(state);
+
 CREATE TABLE mac (
   _mac_id bigint PRIMARY KEY UNIQUE NOT NULL,
   mac_saddr macaddr default NULL,
@@ -191,10 +199,12 @@ CREATE OR REPLACE VIEW ulog AS
         icmpv6_echoseq,
         icmpv6_csum,
         mac_saddr AS mac_saddr_str,
-        mac_protocol AS oob_protocol
+        mac_protocol AS oob_protocol,
+        state AS raw_state
         FROM ulog2 LEFT JOIN tcp ON ulog2._id = tcp._tcp_id LEFT JOIN udp ON ulog2._id = udp._udp_id
                 LEFT JOIN icmp ON ulog2._id = icmp._icmp_id LEFT JOIN mac ON ulog2._id = mac._mac_id
-                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id;
+                LEFT JOIN icmpv6 ON ulog2._id = icmpv6._icmpv6_id
+                LEFT JOIN state_t ON ulog2._id = state_t._state_id;
 
 -- shortcuts
 CREATE OR REPLACE VIEW view_tcp_quad AS
@@ -306,6 +316,8 @@ INSERT INTO _extensions (ext_name,table_name,join_name) VALUES
 
 CREATE OR REPLACE FUNCTION ULOG2_DROP_FOREIGN_KEYS()
 RETURNS void AS $$
+  ALTER TABLE state_t DROP CONSTRAINT state_id_fk;
+  ALTER TABLE mac DROP CONSTRAINT mac_id_fk;
   ALTER TABLE icmpv6 DROP CONSTRAINT icmpv6_id_fk;
   ALTER TABLE icmp DROP CONSTRAINT icmp_id_fk;
   ALTER TABLE udp  DROP CONSTRAINT udp_id_fk;
@@ -319,6 +331,8 @@ RETURNS void AS $$
   ALTER TABLE udp  ADD CONSTRAINT udp_id_fk  FOREIGN KEY (_udp_id)  REFERENCES ulog2(_id);
   ALTER TABLE icmp ADD CONSTRAINT icmp_id_fk FOREIGN KEY (_icmp_id) REFERENCES ulog2(_id);
   ALTER TABLE icmpv6 ADD CONSTRAINT icmpv6_id_fk FOREIGN KEY (_icmpv6_id) REFERENCES ulog2(_id);
+  ALTER TABLE mac ADD CONSTRAINT mac_id_fk FOREIGN KEY (_mac_id) REFERENCES ulog2(_id);
+  ALTER TABLE state_t ADD CONSTRAINT state_id_fk FOREIGN KEY (_state_id) REFERENCES ulog2(_id);
 $$ LANGUAGE SQL SECURITY INVOKER;
 
 
@@ -433,6 +447,16 @@ RETURNS bigint AS $$
         SELECT currval('ulog2__id_seq');
 $$ LANGUAGE SQL SECURITY INVOKER;
 
+CREATE OR REPLACE FUNCTION INSERT_STATE(
+                IN state_id bigint,
+                IN state integer
+        )
+RETURNS bigint AS $$
+        INSERT INTO state_t (_state_id,state)
+                VALUES ($1,$2);
+        SELECT currval('ulog2__id_seq');
+$$ LANGUAGE SQL SECURITY INVOKER;
+
 CREATE OR REPLACE FUNCTION INSERT_MAC(
                 IN mac_id bigint,
                 IN mac_saddr macaddr,
@@ -492,7 +516,8 @@ CREATE OR REPLACE FUNCTION INSERT_PACKET_FULL(
                 IN icmpv6_echoseq integer,
                 IN icmpv6_csum integer,
                 IN mac_saddr varchar(32),
-                IN mac_protocol integer
+                IN mac_protocol integer,
+                IN state integer
         )
 RETURNS bigint AS $$
 DECLARE
@@ -511,13 +536,13 @@ BEGIN
         IF (mac_saddr IS NOT NULL) THEN
                 PERFORM INSERT_MAC(_id,$45::macaddr,$46);
         END IF;
+        IF (state IS NOT NULL) THEN
+                PERFORM INSERT_STATE(_id,$47);
+        END IF;
         RETURN _id;
 END
 $$ LANGUAGE plpgsql SECURITY INVOKER;
 
-
-
-
 CREATE OR REPLACE FUNCTION DELETE_PACKET(
                 IN _packet_id bigint
         )
@@ -526,6 +551,8 @@ RETURNS void AS $$
   DELETE FROM icmp  WHERE icmp._icmp_id = $1;
   DELETE FROM tcp   WHERE tcp._tcp_id   = $1;
   DELETE FROM udp   WHERE udp._udp_id   = $1;
+  DELETE FROM mac   WHERE mac._mac_id   = $1;
+  DELETE FROM state_t   WHERE state_t._state_id   = $1;
   DELETE FROM ulog2 WHERE ulog2._id     = $1;
 $$ LANGUAGE SQL SECURITY INVOKER;
 
@@ -564,6 +591,7 @@ RETURNS void AS $$
   DELETE FROM tcp WHERE _tcp_id NOT IN (SELECT _id FROM ulog2);
   -- XXX note: could be rewritten (need to see what is more efficient) as:
   -- DELETE FROM tcp WHERE _tcp_id IN (SELECT tcp._tcp_id FROM tcp LEFT OUTER JOIN ulog2  ON (tcp._tcp_id = ulog2._id) WHERE ulog2._id IS NULL);
+  DELETE FROM state_t WHERE _state_id NOT IN (SELECT _id FROM ulog2);
   DELETE FROM mac WHERE _mac_id NOT IN (SELECT _id FROM ulog2);
   DELETE FROM udp WHERE _udp_id NOT IN (SELECT _id FROM ulog2);
   DELETE FROM icmp WHERE _icmp_id NOT IN (SELECT _id FROM ulog2);
Signature · 1 line
1.5.2.5
#46 lore raw

PATCH 13/14 Update default configuration to fit last changes

In reply to Eric Leblond

Default stacks needed to be modified to be in sync MAC2STR plugin which is used by database output modules. The new state configuration variable has been added to some ULOG and NFLOG instances to show that it exists.

---
 ulogd.conf.in |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)
ulogd.conf.in +8 -2
index ff46146..d18ed5d 100644
@@ -38,6 +38,7 @@ plugin="@libdir@/ulogd/ulogd_inpflow_NFCT.so"
 plugin="@libdir@/ulogd/ulogd_filter_IFINDEX.so"
 plugin="@libdir@/ulogd/ulogd_filter_IP2STR.so"
 plugin="@libdir@/ulogd/ulogd_filter_IP2BIN.so"
+plugin="@libdir@/ulogd/ulogd_filter_MAC2STR.so"
 plugin="@libdir@/ulogd/ulogd_filter_PRINTPKT.so"
 plugin="@libdir@/ulogd/ulogd_filter_PRINTFLOW.so"
 plugin="@libdir@/ulogd/ulogd_output_LOGEMU.so"
@@ -70,10 +71,13 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so"
 #stack=log1:NFLOG,base1:BASE,pcap1:PCAP
 
 # this is a stack for logging packet to MySQL
-#stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2bin1:IP2BIN,mysql1:MYSQL
+#stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2bin1:IP2BIN,mac2str1:MAC2STR,mysql1:MYSQL
+
+# this is a stack for logging packet to PGsql via ULOG
+#stack=ulog1:ULOG,base1:BASE,ip2str1:IP2STR,mac2str1:MAC2STR,pgsql1:PGSQL
 
 # this is a stack for logging IPv6 packet to PGsql after a collect via NFLOG
-#stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,pgsql1:PGSQL
+#stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,mac2str1:MAC2STR,pgsql1:PGSQL
 
 # this is a stack for logging ebtables packets to syslog after a collect via NFLOG
 #stack=log3:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,sys1:SYSLOG
@@ -92,6 +96,7 @@ group=0
 [log2]
 group=1 # Group has to be different from the one use in log1
 addressfamily=10 # 10 is value of AF_INET6
+state=1 # State can be used to define the decision applied to packet
 
 # ebtables logging through NFLOG
 [log3]
@@ -101,6 +106,7 @@ addressfamily=7 # 7 is value of AF_BRIDGE
 [ulog1]
 # netlink multicast group (the same as the iptables --ulog-nlgroup param)
 nlgroup=1
+#state=0 # optionnal argument
 
 [emu1]
 file="/var/log/ulogd_syslogemu.log"
Signature · 1 line
1.5.2.5
#47 lore raw

PATCH 14/14 Fix computation of length of mac address.

In reply to Eric Leblond

Length of MAC address was set to big and thus display was wrong. This misbehaviour was also causing to read datas out of the correct range.

---
 util/printpkt.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
util/printpkt.c +1 -1
index 6636ac3..b62eed8 100644
@@ -428,7 +428,7 @@ int printpkt_print(struct ulogd_key *res, char *buf)
 	/* FIXME: configurable */
 	if (pp_is_valid(res, KEY_RAW_MAC)) {
 		unsigned char *mac = (unsigned char *) GET_VALUE(res, KEY_RAW_MAC).ptr;
-		int i, len = GET_VALUE(res, KEY_RAW_MACLEN).ui16 * 2;
+		int i, len = GET_VALUE(res, KEY_RAW_MACLEN).ui16;
 
 		buf_cur += sprintf(buf_cur, "MAC=");
 		for (i = 0; i < len; i++)
Signature · 1 line
1.5.2.5
maintainer Applied
#48 lore raw

Re: [ULOGD PATCH 14/14] Fix computation of length of mac address.

Quoted 2 lines · Eric Leblond

Length of MAC address was set to big and thus display was wrong. This misbehaviour was also causing to read datas out of the correct range.

Applied. Thanks.

Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers