Skip to content

RFC

Eric Leblond netfilter-devel 1 version Applied

Status from this message: Applied. Thanks. BTW, one question:

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest34 patches 46 repliesApplied

Patches in v1 Applied

#Subject and filesSizeRepliesApplied
1/34Introduce new SQL schema.doc/mysql-ulogd2.sql doc/pgsql-ulogd2.sql include/ulogd/db.h +2 more+1120 -2010 replies
2/34Do not propagate one conntrack event via 2 messagesinput/flow/ulogd_inpflow_NFCT.c+168 -686 replies
3/34Use an enum to clarify code.util/printflow.c+28 -151 reply
4/34Adapt printflow for one conntrack entry per line format.include/ulogd/printflow.h util/printflow.c+123 -401 reply
5/34Add --info option which displays information about plugin.src/ulogd.c+133 -81 reply
6/34New version of SQL schema.doc/mysql-ulogd2.sql doc/pgsql-ulogd2.sql+213 -871 reply
7/34Changed sighup_handler_print to fallback to continue using old descriptor on new file opening failure.output/ulogd_output_LOGEMU.c+4 -11 reply
8/34Added explicit null termination of the hostname bufferfilter/raw2packet/ulogd_raw2packet_LOCAL.c+1 -01 reply
9/34For OPRINT, changed sighup_handler_print to fallback to continue using old descriptor on new file opening failure.output/ulogd_output_OPRINT.c+4 -11 reply
10/34Add some missing line break.output/ulogd_output_OPRINT.c+2 -21 reply
11/34Put O at the real end of the string.filter/ulogd_filter_PWSNIFF.c+2 -21 reply
12/34Changed to show pcap file name when open failed.output/pcap/ulogd_output_PCAP.c+2 -11 reply
13/34Display filename in the other error case.output/pcap/ulogd_output_PCAP.c+2 -11 reply
14/34Request at least autoconf 2.50 (needed for large file support macro).configure.in+1 -01 reply
15/34MySQL client library does not reconnect automatically since 5.0.output/mysql/ulogd_output_MYSQL.c+14 -01 reply
16/34Introduce IP2STR module which convert IP to string.filter/Makefile.am filter/ulogd_filter_IP2STR.c+203 -11 reply
17/34Suppress key relative to IPv6 address.filter/raw2packet/ulogd_raw2packet_BASE.c include/ulogd/printpkt.h ulogd.conf.in +1 more+21 -451 reply
18/34Update schema for PostgreSQL.doc/pgsql-ulogd2.sql+23 -231 reply
19/34Fix options for pgsql moduleoutput/pgsql/ulogd_output_PGSQL.c+2 -21 reply
20/34Mark ID as inactive (sequence in pg schema)output/pgsql/ulogd_output_PGSQL.c+2 -11 reply
21/34Add IP2BIN module: convert IP address to binary string.filter/Makefile.am filter/ulogd_filter_IP2BIN.c+233 -11 reply
22/34Fix description and indenting.filter/ulogd_filter_IP2STR.c+2 -41 reply
23/34Print RAW as raw string.util/db.c+1 -31 reply
24/34Fix IPv4 output.filter/ulogd_filter_IP2BIN.c+2 -02 replies
25/34Set oob.family as VALID key.input/packet/ulogd_inppkt_NFLOG.c+1 -01 reply
26/34Modify IPv6 parser to fill oob_family.filter/raw2packet/ulogd_raw2packet_BASE.c+4 -01 reply
27/34Free insertion function result (mysql)output/mysql/ulogd_output_MYSQL.c+5 -01 reply
28/34Update SQL schemadoc/mysql-ulogd2.sql doc/pgsql-ulogd2.sql+170 -751 reply
29/34Fix some place were oob_family was used instead of _oob_family.doc/mysql-ulogd2.sql+2 -21 reply
30/34Convert SQL procedure to function call.util/db.c+1 -11 reply
31/34Switch from INNER JOIN to LEFT JOIN in ulog view.doc/mysql-ulogd2.sql+4 -41 reply
32/34Add state extension.doc/mysql-ulogd2.sql+28 -01 reply
33/34ULOGD_RET_IPADDR is for IPv4 or IPv6 address: fix description.src/ulogd.c+1 -41 reply
34/34Modify insert functions to accept standard integers to avoid casts.doc/pgsql-ulogd2.sql output/pgsql/ulogd_output_PGSQL.c+72 -841 reply

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [ULOGD RFC PATCH 0/34]

Hello,

This patchset contains patches for ulogd2 from Pierre Chifflier and I.

As discussed during Netfilter workshop, the goal of this patchset is to provide a new and modern SQL logging schema. Some colateral patchs are present in the patchset due to the state of Ulogd2. As stated by Holger, people using ulogd2 now are early adopters and we tried to improve usability of ulogd2. For example, we've added a --info switch to ulogd2 to be able to display option of a plugin.

But, the main work is on SQL logging. Ulogd 1.x schema was really bad. It lacks index and the way data are stored (one big line per entry full of NULL fields) is not efficient for databases.

Thus, we propose new schemas for MySQL and PGsql which use advanced database feature without complication on developper side. In fact, the SQL related C code did not change very much. The main change is the use of a call to a SQL function instead of using a SQL query. The advantage of doing this is to hide the complexity of the database to developpers and let people knowing databases work on their side without bothering us.

I will finished this mail by a description of the avantages of the new schema. It uses a set of small dedicated tables (a TCP tables for example). From an SQL point of view this is more efficient as we limit the number of NULL fields (storage of empty datas has a cost). The schema has some SQL views (virtual table) and some of them provides an near complete backward compatility with the existing one.

One other advantage of the new schema is that extension (like nufw one) can used without changing anything for non-aware system.

This patchset should not conflict with Holger patchset (if NFCT related work is omitted). I can do the merge work if some is needed, just let me know.

BR,
--
Eric Leblond <[email protected]>
INL: http://www.inl.fr/