Protocol auto detection, bugfixes and doc
cover Protocol auto detection, bugfixes and doc
Hi,
First patch is the most significative. It modify the behaviour of ulogd for packet based logging. Before the patch, it was necessary to define the type of IP protocol that will be used in an NFLOG input. A side effect was that it was impossible to log in a single netlink group for IPv4 and IPv6. This patch let the BASE filter do this job of finding the protocol.
This patchset also contains two small fixes and/or improvements:
* A fix of ulogd info option
* A review of default config file which has been modified to follow the latest modifications
PATCH 1/3 Do not set oob.family in config as it can be found in packet.
WHen using NFLOG or ULOG, obb.family (protocol IPv4 or IPv6) has to be setup manually in ulogd.conf configuration file. This was used to have the BASE filter parse accordingly the packet. This patch suppress oob.family as output keys of NFLOG and ULOG and let the BASE filter determine the family of the packet by itself (by parsing the raw header).
A good side effect is to be able to log in IPv6 and IPv4 in the same group. Before that, two loggers have to be setup separatly.
---
:100644 100644 48f2993... 62a9a87... M filter/raw2packet/ulogd_raw2packet_BASE.c
:100644 100644 be46fa2... a85ff44... M input/packet/ulogd_inppkt_NFLOG.c
:100644 100644 cf44474... 77087a4... M input/packet/ulogd_inppkt_ULOG.c
filter/raw2packet/ulogd_raw2packet_BASE.c | 32 ++++++++++++++++++++-------- input/packet/ulogd_inppkt_NFLOG.c | 15 ------------- input/packet/ulogd_inppkt_ULOG.c | 9 -------- 3 files changed, 23 insertions(+), 33 deletions(-)
filter/raw2packet/ulogd_raw2packet_BASE.c +23 -9
@@ -44,6 +44,7 @@#include <ulogd/ipfix_protocol.h> enum output_keys { +KEY_OOB_FAMILY ,KEY_IP_SADDR ,KEY_IP_DADDR ,KEY_IP_PROTOCOL ,
@@ -98,6 +99,11 @@ enum output_keys { };static struct ulogd_key iphdr_rets[] = { + [KEY_OOB_FAMILY ] = { + .type =ULOGD_RET_UINT8 , + .flags =ULOGD_RETF_NONE , + .name ="oob.family" , + }, [KEY_IP_SADDR ] = { .type =ULOGD_RET_IPADDR , .flags =ULOGD_RETF_NONE ,
@@ -819,15 +825,27 @@ out:static int _interp_pkt (struct ulogd_pluginstance *pi) { +struct ulogd_key *ret = pi->output.keys; +struct iphdr *iph = pi->input.keys[0 ].u.source->u.value.ptr;u_int32_t len = pi->input.keys[1 ].u.source->u.value.ui32;u_int8_t family = pi->input.keys[2 ].u.source->u.value.ui8; -switch (family) { -case AF_INET : -return _interp_iphdr (pi, len); -case AF_INET6 : -return _interp_ipv6hdr (pi, len); +switch (iph->version) { +case 4 : + ret[KEY_OOB_FAMILY ].u.value.ui8 =AF_INET ; + ret[KEY_OOB_FAMILY ].flags |=ULOGD_RETF_VALID ; + +return _interp_iphdr (pi, len); +case 6 : + ret[KEY_OOB_FAMILY ].u.value.ui8 =AF_INET6 ; + ret[KEY_OOB_FAMILY ].flags |=ULOGD_RETF_VALID ; + +return _interp_ipv6hdr (pi, len); +default : +/* unknown protocol */ +return 0 ; } +return 0 ; }
@@ -847,10 +865,6 @@ static struct ulogd_key base_inp[] = { .vendor = IPFIX_VENDOR_NETFILTER, .field_id = IPFIX_NF_rawpacket_length, }, - }, - { - .type = ULOGD_RET_UINT8, - .name = "oob.family", } };
input/packet/ulogd_inppkt_NFLOG.c +0 -15
@@ -55,12 +55,6 @@ static struct config_keyset libulog_kset = { .u.value = NFLOG_RMEM_DEFAULT, }, { - .key = "addressfamily", - .type = CONFIG_TYPE_INT, - .options = CONFIG_OPT_NONE, - .u.value = AF_INET, - }, - { .key = "unbind", .type = CONFIG_TYPE_INT, .options = CONFIG_OPT_NONE,
@@ -104,7 +98,6 @@ enum nflog_keys {NFLOG_KEY_RAW_MAC_LEN ,NFLOG_KEY_OOB_SEQ_LOCAL ,NFLOG_KEY_OOB_SEQ_GLOBAL , -NFLOG_KEY_OOB_FAMILY , NFLOG_KEY_OOB_PROTOCOL, };
@@ -231,11 +224,6 @@ static struct ulogd_key output_keys[] = { }, }, { - .type = ULOGD_RET_UINT8, - .flags = ULOGD_RETF_NONE, - .name = "oob.family", - }, - { .type = ULOGD_RET_UINT16, .flags = ULOGD_RETF_NONE, .name = "oob.protocol",
@@ -258,9 +246,6 @@ interp_packet(struct ulogd_pluginstance *upi, struct nflog_data *ldata)u_int32_t outdev =nflog_get_outdev (ldata);u_int32_t seq; - ret[NFLOG_KEY_OOB_FAMILY ].u.value.ui8 =af_ce (upi->config_kset).u.value; - ret[NFLOG_KEY_OOB_FAMILY ].flags |=ULOGD_RETF_VALID ; -if (ph) {/* FIXME */ ret[NFLOG_KEY_OOB_HOOK ].u.value.ui8 = ph->hook;
input/packet/ulogd_inppkt_ULOG.c +0 -9
@@ -68,7 +68,6 @@ enum ulog_keys {ULOG_KEY_OOB_IN ,ULOG_KEY_OOB_OUT ,ULOG_KEY_RAW_MAC_LEN , -ULOG_KEY_OOB_FAMILY , ULOG_KEY_OOB_PROTOCOL, };
@@ -148,11 +147,6 @@ static struct ulogd_key output_keys[] = { .name = "raw.mac_len", }, { - .type = ULOGD_RET_UINT8, - .flags = ULOGD_RETF_NONE, - .name = "oob.family", - }, - { .type = ULOGD_RET_UINT16, .flags = ULOGD_RETF_NONE, .name = "oob.protocol",
@@ -201,9 +195,6 @@ static int interp_packet(struct ulogd_pluginstance *ip, ulog_packet_msg_t *pkt) ret[ULOG_KEY_OOB_OUT ].u.value.ptr = pkt->outdev_name; ret[ULOG_KEY_OOB_OUT ].flags |=ULOGD_RETF_VALID ; -/* ULOG is IPv4 only */ - ret[ULOG_KEY_OOB_FAMILY ].u.value.ui8 =AF_INET ; - ret[ULOG_KEY_OOB_FAMILY ].flags |=ULOGD_RETF_VALID ;/* Undef in ULOG but necessary */ ret[ULOG_KEY_OOB_PROTOCOL ].u.value.ui16 =0 ; ret[ULOG_KEY_OOB_PROTOCOL ].flags |=ULOGD_RETF_VALID ;
Signature · 1 line
1.5.2.5
Re: [ULOGD PATCH 1/3] Do not set oob.family in config as it can be found in packet.
Hi Eric
Quoted 6 lines · Eric Leblond
WHen using NFLOG or ULOG, obb.family (protocol IPv4 or IPv6) has to be setup manually in ulogd.conf configuration file. This was used to have the BASE filter parse accordingly the packet. This patch suppress oob.family as output keys of NFLOG and ULOG and let the BASE filter determine the family of the packet by itself (by parsing the raw header).
That field is necessary if NFLOG logging should also work from within ebtables (I sent ebtables patches recently).
Sending from ebtables oob.family is AF_BRIDGE, since there could also come arp packets to be logged.
Configuring the addressfamily manually within the configuration file is how you then register the log handler for AF_BRIDGE. It is not always necessary to have that handler registered, so it's good practice to decide this by configuration file, i think.
What you say?
peter
Signature · 5 lines
:: e n d i a n :: open source - open minds :: peter warasin :: http://www.endian.com :: [email protected]
begin:vcard
fn:Peter Warasin
n:;Peter Warasin
org:Endian GmbH/Srl
adr:;;Pillhof 47;Frangart/Frangarto;BZ;I-39010;Italien/Italia email;internet:[email protected]
tel;work:+39 0471 631763
tel;fax:+39 0471 631764
x-mozilla-html:FALSE
url:http://www.endian.com
version:2.1
end:vcard
Re: [ULOGD PATCH 1/3] Do not set oob.family in config as it can be found in packet.
Hi,
Quoted 8 lines · Peter Warasin
Hi Eric
Eric Leblond wrote:
That field is necessary if NFLOG logging should also work from within ebtables (I sent ebtables patches recently).
Sending from ebtables oob.family isAF_BRIDGE, since there could also come arp packets to be logged.
Thanks for pointing this.
Quoted 6 lines
Configuring the addressfamily manually within the configuration file is how you then register the log handler for
AF_BRIDGE. It is not always necessary to have that handler registered, so it's good practice to decide this by configuration file, i think.What you say?
The main motivation of this patch was to get around the problem of being force to declare one loggers for IPv4 and one for IPv6. Adding AF_BRIDGE into account, make me ask to forget about this patch.
I will put focus on this when documentating the issue in the configuration file.
BR,
Signature · 3 lines
Eric Leblond INL: http://www.inl.fr/ NuFW: http://www.nufw.org/
Re: [ULOGD PATCH 1/3] Do not set oob.family in config as it can be found in packet.
In reply to Eric LeblondQuoted 9 lines · Eric Leblond
WHen using NFLOG or ULOG, obb.family (protocol IPv4 or IPv6) has to be setup manually in ulogd.conf configuration file. This was used to have the BASE filter parse accordingly the packet. This patch suppress oob.family as output keys of NFLOG and ULOG and let the BASE filter determine the family of the packet by itself (by parsing the raw header).
A good side effect is to be able to log in IPv6 and IPv4 in the same group. Before that, two loggers have to be setup separatly.
Applied. Thanks Eric.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
Re: [ULOGD PATCH 1/3] Do not set oob.family in config as it can be found in packet.
Hi Pablo
Quoted 3 lines · Pablo Neira Ayuso
Quoted 2 lines · Eric Leblond
WHen using NFLOG or ULOG, obb.family (protocol IPv4 or IPv6) has to be setup manually in ulogd.conf configuration file. This was
..
Quoted 1 line
Applied. Thanks Eric.
Attention, what's with AF_BRIDGE then?
It's not more possible to register a log handler for AF_BRIDGE with this patch applied.
Packets coming from ebtables will use that protocol.
peter
Signature · 5 lines
:: e n d i a n :: open source - open minds :: peter warasin :: http://www.endian.com :: [email protected]
begin:vcard
fn:Peter Warasin
n:;Peter Warasin
org:Endian GmbH/Srl
adr:;;Pillhof 47;Frangart/Frangarto;BZ;I-39010;Italien/Italia email;internet:[email protected]
tel;work:+39 0471 631763
tel;fax:+39 0471 631764
x-mozilla-html:FALSE
url:http://www.endian.com
version:2.1
end:vcard
PATCH 2/3 Fix ulogd --info when displaying some default value.
In reply to Eric LeblondTHis patch fixes a crash in ulogd2 when dealing with default value given as string.
---
:100644 100644 fe4106a... de2fd96... M src/ulogd.c
src/ulogd.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
src/ulogd.c +1 -1
@@ -290,7 +290,7 @@ void get_plugin_infos(struct ulogd_plugin *me)case CONFIG_TYPE_STRING :printf ("String" );printf (", Default: %s" , - me->config_kset->ces[i].u.value); + me->config_kset->ces[i].u.string);break ;case CONFIG_TYPE_INT :printf ("Integer" );
Signature · 1 line
1.5.2.5
Re: [ULOGD PATCH 2/3] Fix ulogd --info when displaying some default value.
Quoted 2 lines · Eric Leblond
THis patch fixes a crash in ulogd2 when dealing with default value given as string.
Applied. Thanks.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
PATCH 3/3 Add examples to config file.
In reply to Eric LeblondThis patch adds some examples of stack to the configuration file. It also fixes some comments to avoid confusion. IP2BIN has been added to the list of loaded modules.
---
:100644 100644 465b224... 3fc3a41... M ulogd.conf.in
ulogd.conf.in | 30 +++++++++++++++++++++--------- 1 files changed, 21 insertions(+), 9 deletions(-)
ulogd.conf.in +21 -9
@@ -36,6 +36,7 @@ plugin="@libdir@/ulogd/ulogd_inppkt_NFLOG.so" plugin="@libdir@/ulogd/ulogd_inpflow_NFCT.so" plugin="@libdir@/ulogd/ulogd_filter_IFINDEX.so" plugin="@libdir@/ulogd/ulogd_filter_IP2STR.so" +plugin="@libdir@/ulogd/ulogd_filter_IP2BIN.so" plugin="@libdir@/ulogd/ulogd_filter_PRINTPKT.so" plugin="@libdir@/ulogd/ulogd_filter_PRINTFLOW.so" plugin="@libdir@/ulogd/ulogd_output_LOGEMU.so"
@@ -45,11 +46,8 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" # this is a stack for packet-based logging via LOGEMU #stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU -# this is a stack for IPv6 packet-based logging via LOGEMU -#stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU - # this is a stack for ULOG packet-based logging via LOGEMU -#stack=ulog1:ULOG,base1:BASE,print1:PRINTPKT,emu1:LOGEMU +#stack=ulog1:ULOG,base1:BASE,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU # this is a stack for flow-based logging via LOGEMU #stack=ct1:NFCT,print1:PRINTFLOW,emu1:LOGEMU
@@ -57,17 +55,23 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" # this is a stack for flow-based logging via OPRINT #stack=ct1:NFCT,op1:OPRINT +# this is a stack for NFLOG packet logging to MySQL +#stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2bin1:IP2BIN,mysql1:MYSQL + +# this is a stack for NFLOG packet logging to PGsql +#stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,pgsql1:PGSQL + +# this is a stack for flow-based logging to MySQL +#stack=ct1:NFCT,ip2bin1:IP2BIN,mysql2:MYSQL + [ct1] [log1] -# netlink multicast group (the same as the iptables --ulog-nlgroup param) +# netlink multicast group (the same as the iptables --nflog-group param) group=0 -[log2] -group=1 -addressfamily=10 - [ulog1] +# netlink multicast group (the same as the iptables --ulog-nlgroup param) nlgroup=1 [emu1]
@@ -85,3 +89,11 @@ user="nupik" table="ulog" pass="changeme" procedure="INSERT_PACKET_FULL" + +[pgsql1] +db="nulog" +host="localhost" +user="nupik" +table="ulog" +pass="changeme" +procedure="INSERT_PACKET_FULL"
Signature · 1 line
1.5.2.5
PATCH RESEND 1/1 Improve ulogd.conf.in
This patch documents IPv4 and IPv6 usage by adding some examples of stacks. It also adds IP2BIN to the list of loaded plugins and fixes some comments.
---
:100644 100644 465b224... 6d76909... M ulogd.conf.in
ulogd.conf.in | 33 +++++++++++++++++++++++++++------ 1 files changed, 27 insertions(+), 6 deletions(-)
ulogd.conf.in +27 -6
@@ -36,20 +36,21 @@ plugin="@libdir@/ulogd/ulogd_inppkt_NFLOG.so" plugin="@libdir@/ulogd/ulogd_inpflow_NFCT.so" plugin="@libdir@/ulogd/ulogd_filter_IFINDEX.so" plugin="@libdir@/ulogd/ulogd_filter_IP2STR.so" +plugin="@libdir@/ulogd/ulogd_filter_IP2BIN.so" plugin="@libdir@/ulogd/ulogd_filter_PRINTPKT.so" plugin="@libdir@/ulogd/ulogd_filter_PRINTFLOW.so" plugin="@libdir@/ulogd/ulogd_output_LOGEMU.so" plugin="@libdir@/ulogd/ulogd_output_OPRINT.so" plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" -# this is a stack for packet-based logging via LOGEMU +# this is a stack for IPv4 packet-based logging via LOGEMU #stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU # this is a stack for IPv6 packet-based logging via LOGEMU #stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU # this is a stack for ULOG packet-based logging via LOGEMU -#stack=ulog1:ULOG,base1:BASE,print1:PRINTPKT,emu1:LOGEMU +#stack=ulog1:ULOG,base1:BASE,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU # this is a stack for flow-based logging via LOGEMU #stack=ct1:NFCT,print1:PRINTFLOW,emu1:LOGEMU
@@ -57,17 +58,29 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" # this is a stack for flow-based logging via OPRINT #stack=ct1:NFCT,op1:OPRINT +# this is a stack for logging packet to MySQL +#stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2bin1:IP2BIN,mysql1:MYSQL + +# this is a stack for logging IPv6 packet to PGsql after a collect via NFLOG +#stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,pgsql1:PGSQL + +# this is a stack for flow-based logging to MySQL +#stack=ct1:NFCT,ip2bin1:IP2BIN,mysql2:MYSQL + [ct1] +# IPv4 logging through NFLOG [log1] -# netlink multicast group (the same as the iptables --ulog-nlgroup param) +# netlink multicast group (the same as the iptables --nflog-group param) group=0 -[log2] -group=1 -addressfamily=10 +# IPv6 logging through NFLOG +[log1] +group=1 # Group has to be different from the one use in log1 +addressfamily=10 # 10 is value of AF_INET6 [ulog1] +# netlink multicast group (the same as the iptables --ulog-nlgroup param) nlgroup=1 [emu1]
@@ -85,3 +98,11 @@ user="nupik" table="ulog" pass="changeme" procedure="INSERT_PACKET_FULL" + +[pgsql1] +db="nulog" +host="localhost" +user="nupik" +table="ulog" +pass="changeme" +procedure="INSERT_PACKET_FULL"
Signature · 1 line
1.5.2.5
Pablo Neira Ayuso
maintainerRe: [RESEND PATCH] Improve ulogd.conf.in
Quoted 2 lines · Eric Leblond
This patch documents IPv4 and IPv6 usage by adding some examples of stacks. It also adds IP2BIN to the list of loaded plugins and fixes some comments.
This one does not apply correctly. Could you fix it a send it back to me, please? Thanks Eric.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
PATCH 1/1 Document IPv4 and IPv6 logging difference in NFLOG
Hi,
Here's an update of the patch. This patch is usefull if the modification in r7348 are reverted.
This patch documents difference between IPv4 and IPv6 logging.
--- ulogd.conf.in | 17 +++++++++++++---- 1 files changed, 13 insertions(+), 4 deletions(-)
ulogd.conf.in +13 -4
@@ -43,9 +43,12 @@ plugin="@libdir@/ulogd/ulogd_output_LOGEMU.so" plugin="@libdir@/ulogd/ulogd_output_OPRINT.so" plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" -# this is a stack for packet-based logging via LOGEMU +# this is a stack for IPv4 packet-based logging via LOGEMU #stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU +# this is a stack for IPv6 packet-based logging via LOGEMU +#stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU + # this is a stack for ULOG packet-based logging via LOGEMU #stack=ulog1:ULOG,base1:BASE,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU
@@ -55,21 +58,27 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" # this is a stack for flow-based logging via OPRINT #stack=ct1:NFCT,op1:OPRINT -# this is a stack for NFLOG packet logging to MySQL +# this is a stack for logging packet to MySQL #stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2bin1:IP2BIN,mysql1:MYSQL -# this is a stack for NFLOG packet logging to PGsql -#stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,pgsql1:PGSQL +# this is a stack for logging IPv6 packet to PGsql after a collect via NFLOG +#stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,pgsql1:PGSQL # this is a stack for flow-based logging to MySQL #stack=ct1:NFCT,ip2bin1:IP2BIN,mysql2:MYSQL [ct1] +# IPv4 logging through NFLOG [log1] # netlink multicast group (the same as the iptables --nflog-group param) group=0 +# IPv6 logging through NFLOG +[log1] +group=1 # Group has to be different from the one use in log1 +addressfamily=10 # 10 is value of AF_INET6 + [ulog1] # netlink multicast group (the same as the iptables --ulog-nlgroup param) nlgroup=1
Signature · 1 line
1.5.2.5
Re: [ULOGD PATCH] Document IPv4 and IPv6 logging difference in NFLOG
Quoted 2 lines · Eric Leblond
Here's an update of the patch. This patch is usefull if the modification in r7348 are reverted.
Reverted.
Quoted 1 line
This patch documents difference between IPv4 and IPv6 logging.
Applied. Thanks.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
Re: [ULOGD PATCH 3/3] Add examples to config file.
In reply to Eric LeblondQuoted 3 lines · Eric Leblond
This patch adds some examples of stack to the configuration file. It also fixes some comments to avoid confusion. IP2BIN has been added to the list of loaded modules.
Applied. Thanks.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers