UNIXSOCK plugin (v5)
Hi,
Here is an updated version of the UNIXSOCK plugin.
Changes for v5:
- align all structures on 64 bits instead of 32 This has been tested on a sparc64 (thanks to Jan)
- update pcap2ulog to handle different pcap formats
Changes for v4:
- use structures instead of reading integers directly
- all structures and fields are now aligned (4 bytes)
- update pcap2ulog script to use aligned data as well
Changes for v3:
- if the bufsize value (from conf) is 0, try to guess the buffer size using getsockopt(SOL_SOCKET, SO_RCVBUF ..)
- set default path to /var/run/ulogd/ulogd2.sock with mode 0600
- add new configuration options perms, owner and group to be able to set these parameters on the created socket
- change the input algorithm to make it more robust against junk data (which can happen if the client is desynchronized)
- fixed a few bugs
Description:
[1] add new input plugin UNISOCK, using a unix socket. This allows userspace applications to send packets to ulogd, for example to send packets offline. It uses a key-length-value protocol to handle optional fields or extensions.
[2] is a Perl script to load a PCAP file and send it to ulogd using the UNIXSOCK plugin.
Pierre
PATCH 1/2 Add new input plugin UNIXSOCK
This input plugins creates a unix socket which can be used to log packets. Scripts or applications can connect to the socket (only one client allowed per socket) and send data in a Key-Length-Value format (including the payload).
--- input/packet/Makefile.am | 5 +- input/packet/ulogd_inppkt_UNIXSOCK.c | 815 ++++++++++++++++++++++++++++++++++ ulogd.conf.in | 7 + 3 files changed, 826 insertions(+), 1 deletions(-) create mode 100644 input/packet/ulogd_inppkt_UNIXSOCK.c
input/packet/Makefile.am +4 -1
@@ -3,7 +3,7 @@ AM_CPPFLAGS = $(all_includes) -I$(top_srcdir)/include AM_CFLAGS=-fPIC -Wall LIBS= -pkglib_LTLIBRARIES = ulogd_inppkt_NFLOG.la ulogd_inppkt_ULOG.la +pkglib_LTLIBRARIES = ulogd_inppkt_NFLOG.la ulogd_inppkt_ULOG.la ulogd_inppkt_UNIXSOCK.la ulogd_inppkt_NFLOG_la_SOURCES = ulogd_inppkt_NFLOG.c ulogd_inppkt_NFLOG_la_LDFLAGS = -avoid-version -module $(LIBNETFILTER_LOG_LIBS)
@@ -12,3 +12,6 @@ ulogd_inppkt_NFLOG_la_CFLAGS = $(AM_CFLAGS) $(LIBNETFILTER_LOG_CFLAGS) ulogd_inppkt_ULOG_la_SOURCES = ulogd_inppkt_ULOG.c ulogd_inppkt_ULOG_la_LDFLAGS = -avoid-version -module ulogd_inppkt_ULOG_la_LIBADD = ../../libipulog/libipulog.la + +ulogd_inppkt_UNIXSOCK_la_SOURCES = ulogd_inppkt_UNIXSOCK.c +ulogd_inppkt_UNIXSOCK_la_LDFLAGS = -avoid-version -module
input/packet/ulogd_inppkt_UNIXSOCK.c new file +815 -0
815 lines not on this page · raw message
ulogd.conf.in +7 -0
@@ -27,6 +27,7 @@ loglevel=1 plugin="@libdir@/ulogd/ulogd_inppkt_NFLOG.so" #plugin="@libdir@/ulogd/ulogd_inppkt_ULOG.so" +#plugin="@libdir@/ulogd/ulogd_inppkt_UNIXSOCK.so" plugin="@libdir@/ulogd/ulogd_inpflow_NFCT.so" plugin="@libdir@/ulogd/ulogd_filter_IFINDEX.so" plugin="@libdir@/ulogd/ulogd_filter_IP2STR.so"
@@ -75,6 +76,9 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" # this is a stack for logging packets to syslog after a collect via NFLOG #stack=log3:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,sys1:SYSLOG +# this is a stack for logging packets to syslog after a collect via NuFW +#stack=nuauth1:UNIXSOCK,base1:BASE,ip2str1:IP2STR,print1:PRINTPKT,sys1:SYSLOG + # this is a stack for flow-based logging to MySQL #stack=ct1:NFCT,ip2bin1:IP2BIN,mysql2:MYSQL
@@ -137,6 +141,9 @@ numeric_label=1 # you can label the log info based on the packet verdict nlgroup=1 #numeric_label=0 # optional argument +[nuauth1] +socket_path="/tmp/nuauth_ulogd2.sock" + [emu1] file="/var/log/ulogd_syslogemu.log" sync=1
Signature · 1 line
1.7.0
Pablo Neira Ayuso
maintainerRe: [PATCH 1/2] Add new input plugin UNIXSOCK
Hi Pierre,
Quoted 4 lines · Pierre Chifflier
This input plugins creates a unix socket which can be used to log packets. Scripts or applications can connect to the socket (only one client allowed per socket) and send data in a Key-Length-Value format (including the payload).
I'm proposing some minor changes, they are mostly cleanups and comestic.
BTW, how does the "big picture" of this look like? I guess that your nufw daemon is listening to packets via libnetfilter_queue and then it sends unixsock messages that include the packet + extra authentication information to ulogd2 to log them, right?
Quoted 4 lines
Signed-off-by Pierre Chifflier <[email protected]>--- input/packet/Makefile.am | 5 +- input/packet/ulogd_inppkt_UNIXSOCK.c | 815 ++++++++++++++++++++++++++++++++++
could this be extended in the future to support flow-based information? I mean, does this really fit into the "packet" directory of ulogd2?
Quoted 30 lines
ulogd.conf.in | 7 + 3 files changed, 826 insertions(+), 1 deletions(-) create mode 100644 input/packet/ulogd_inppkt_UNIXSOCK.cinput/packet/Makefile.am +4
-1@@ -3,7 +3,7 @@ AM_CPPFLAGS = $(all_includes) -I$(top_srcdir)/include AM_CFLAGS=-fPIC -Wall LIBS= -pkglib_LTLIBRARIES = ulogd_inppkt_NFLOG.la ulogd_inppkt_ULOG.la +pkglib_LTLIBRARIES = ulogd_inppkt_NFLOG.la ulogd_inppkt_ULOG.la ulogd_inppkt_UNIXSOCK.la ulogd_inppkt_NFLOG_la_SOURCES = ulogd_inppkt_NFLOG.c ulogd_inppkt_NFLOG_la_LDFLAGS = -avoid-version -module $(LIBNETFILTER_LOG_LIBS)@@ -12,3 +12,6 @@ ulogd_inppkt_NFLOG_la_CFLAGS = $(AM_CFLAGS) $(LIBNETFILTER_LOG_CFLAGS) ulogd_inppkt_ULOG_la_SOURCES = ulogd_inppkt_ULOG.c ulogd_inppkt_ULOG_la_LDFLAGS = -avoid-version -module ulogd_inppkt_ULOG_la_LIBADD = ../../libipulog/libipulog.la + +ulogd_inppkt_UNIXSOCK_la_SOURCES = ulogd_inppkt_UNIXSOCK.c +ulogd_inppkt_UNIXSOCK_la_LDFLAGS = -avoid-version -moduleinput/packet/ulogd_inppkt_UNIXSOCK.c new file +0
-0
input/packet/ulogd_inppkt_UNIXSOCK.c line 3
Quoted 3 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +3
-0@@ -0,0 +1,815 @@ +/* +** Copyright(C) 2008-2010 INL +** Written by Pierre Chifflier <[email protected]>
Minor nitpick: please, add licensing terms here in a short header.
Quoted 18 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +18
-0+*/ + +#include <unistd.h> +#include <stdlib.h> +#include <netinet/ether.h> +#include <netinet/in.h> +#include <netinet/ip.h> +#include <netinet/ip6.h> +#include <sys/stat.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <pwd.h> +#include <grp.h> +#include <errno.h> + +#include <ulogd/ulogd.h> +
input/packet/ulogd_inppkt_UNIXSOCK.c line 31
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+/* Default size of the receive buffer for the unix socket +0 means that ulogd will use getsockopt(SO_RCVBUF) to determine it +at runtime */ +#define UNIXSOCK_BUFSIZE_DEFAULT 0 + +/* Default permissions of created socket (in octal mode) */ +#define UNIXSOCK_PERMS_DEFAULT 0600 + +/* Default unix socket path */ +#define UNIXSOCK_UNIXPATH_DEFAULT "/var/run/ulogd/ulogd2.sock"
These extra lines below are unnecessarly, I'd say. They don't help to make the code more readable.
input/packet/ulogd_inppkt_UNIXSOCK.c line 37
Quoted 6 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +6
-0+ + +/* Unique value used as a signature to ensure received data is really +* a packet +*/ +#define ULOGD_SOCKET_MARK 0x41c90fd4
Same here
Quoted 202 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +202
-0+ + +struct unixsock_input { +char *path; +char *unixsock_buf; +unsigned int unixsock_perms; +unsigned int unixsock_buf_avail; +unsigned int unixsock_buf_size; +struct ulogd_fd unixsock_server_fd; +struct ulogd_fd unixsock_instance_fd; +}; + +enum nflog_keys { +UNIXSOCK_KEY_RAW_MAC =0 , +UNIXSOCK_KEY_RAW_PCKT , +UNIXSOCK_KEY_RAW_PCKTLEN , +UNIXSOCK_KEY_RAW_PCKTCOUNT , +UNIXSOCK_KEY_OOB_PREFIX , +UNIXSOCK_KEY_OOB_TIME_SEC , +UNIXSOCK_KEY_OOB_TIME_USEC , +UNIXSOCK_KEY_OOB_MARK , +UNIXSOCK_KEY_OOB_IN , +UNIXSOCK_KEY_OOB_OUT , +UNIXSOCK_KEY_OOB_HOOK , +UNIXSOCK_KEY_RAW_MAC_LEN , +UNIXSOCK_KEY_OOB_SEQ_LOCAL , +UNIXSOCK_KEY_OOB_SEQ_GLOBAL , +UNIXSOCK_KEY_OOB_FAMILY , +UNIXSOCK_KEY_OOB_PROTOCOL , +UNIXSOCK_KEY_OOB_UID , +UNIXSOCK_KEY_OOB_GID , +UNIXSOCK_KEY_RAW_LABEL , +UNIXSOCK_KEY_RAW_TYPE , +UNIXSOCK_KEY_RAW_MAC_SADDR , +UNIXSOCK_KEY_RAW_MAC_ADDRLEN , +UNIXSOCK_KEY_NUFW_USER_NAME , +UNIXSOCK_KEY_NUFW_USER_ID , +UNIXSOCK_KEY_NUFW_OS_NAME , +UNIXSOCK_KEY_NUFW_OS_REL , +UNIXSOCK_KEY_NUFW_OS_VERS , +UNIXSOCK_KEY_NUFW_APP_NAME , +}; + +static struct ulogd_key output_keys[] = { + [UNIXSOCK_KEY_RAW_MAC ] = { + .type =ULOGD_RET_RAW , + .flags =ULOGD_RETF_NONE , + .name ="raw.mac" , + }, + [UNIXSOCK_KEY_RAW_MAC_SADDR ] = { + .type =ULOGD_RET_RAW , + .flags =ULOGD_RETF_NONE , + .name ="raw.mac.saddr" , + .ipfix = { + .vendor =IPFIX_VENDOR_IETF , + .field_id = IPFIX_sourceMacAddress, + }, + }, + [UNIXSOCK_KEY_RAW_PCKT ] = { + .type =ULOGD_RET_RAW , + .flags =ULOGD_RETF_NONE , + .name ="raw.pkt" , + .ipfix = { + .vendor =IPFIX_VENDOR_NETFILTER , + .field_id = IPFIX_NF_rawpacket, + }, + }, + [UNIXSOCK_KEY_RAW_PCKTLEN ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="raw.pktlen" , + .ipfix = { + .vendor =IPFIX_VENDOR_NETFILTER , + .field_id = IPFIX_NF_rawpacket_length, + }, + }, + [UNIXSOCK_KEY_RAW_PCKTCOUNT ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="raw.pktcount" , + .ipfix = { + .vendor =IPFIX_VENDOR_IETF , + .field_id = IPFIX_packetDeltaCount, + }, + }, + [UNIXSOCK_KEY_OOB_PREFIX ] = { + .type =ULOGD_RET_STRING , + .flags =ULOGD_RETF_NONE , + .name ="oob.prefix" , + .ipfix = { + .vendor =IPFIX_VENDOR_NETFILTER , + .field_id = IPFIX_NF_prefix, + }, + }, + [UNIXSOCK_KEY_OOB_TIME_SEC ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="oob.time.sec" , + .ipfix = { + .vendor =IPFIX_VENDOR_IETF , + .field_id = IPFIX_flowStartSeconds, + }, + }, + [UNIXSOCK_KEY_OOB_TIME_USEC ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="oob.time.usec" , + .ipfix = { + .vendor =IPFIX_VENDOR_IETF , + .field_id = IPFIX_flowStartMicroSeconds, + }, + }, + [UNIXSOCK_KEY_OOB_MARK ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="oob.mark" , + .ipfix = { + .vendor =IPFIX_VENDOR_NETFILTER , + .field_id = IPFIX_NF_mark, + }, + }, + [UNIXSOCK_KEY_OOB_IN ] = { + .type =ULOGD_RET_STRING , + .flags =ULOGD_RETF_NONE , + .name ="oob.in" , + .ipfix = { + .vendor =IPFIX_VENDOR_IETF , + .field_id = IPFIX_ingressInterface, + }, + }, + [UNIXSOCK_KEY_OOB_OUT ] = { + .type =ULOGD_RET_STRING , + .flags =ULOGD_RETF_NONE , + .name ="oob.out" , + .ipfix = { + .vendor =IPFIX_VENDOR_IETF , + .field_id = IPFIX_egressInterface, + }, + }, + [UNIXSOCK_KEY_OOB_HOOK ] = { + .type =ULOGD_RET_UINT8 , + .flags =ULOGD_RETF_NONE , + .name ="oob.hook" , + .ipfix = { + .vendor =IPFIX_VENDOR_NETFILTER , + .field_id = IPFIX_NF_hook, + }, + }, + [UNIXSOCK_KEY_RAW_MAC_LEN ] = { + .type =ULOGD_RET_UINT16 , + .flags =ULOGD_RETF_NONE , + .name ="raw.mac_len" , + }, + [UNIXSOCK_KEY_RAW_MAC_ADDRLEN ] = { + .type =ULOGD_RET_UINT16 , + .flags =ULOGD_RETF_NONE , + .name ="raw.mac.addrlen" , + }, + + [UNIXSOCK_KEY_OOB_SEQ_LOCAL ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="oob.seq.local" , + .ipfix = { + .vendor =IPFIX_VENDOR_NETFILTER , + .field_id = IPFIX_NF_seq_local, + }, + }, + [UNIXSOCK_KEY_OOB_SEQ_GLOBAL ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="oob.seq.global" , + .ipfix = { + .vendor =IPFIX_VENDOR_NETFILTER , + .field_id = IPFIX_NF_seq_global, + }, + }, + [UNIXSOCK_KEY_OOB_FAMILY ] = { + .type =ULOGD_RET_UINT8 , + .flags =ULOGD_RETF_NONE , + .name ="oob.family" , + }, + [UNIXSOCK_KEY_OOB_PROTOCOL ] = { + .type =ULOGD_RET_UINT16 , + .flags =ULOGD_RETF_NONE , + .name ="oob.protocol" , + }, + [UNIXSOCK_KEY_OOB_UID ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="oob.uid" , + }, + [UNIXSOCK_KEY_OOB_GID ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="oob.gid" , + }, + [UNIXSOCK_KEY_RAW_LABEL ] = { + .type =ULOGD_RET_UINT8 , + .flags =ULOGD_RETF_NONE , + .name ="raw.label" , + },
input/packet/ulogd_inppkt_UNIXSOCK.c line 249
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+ [UNIXSOCK_KEY_RAW_TYPE] = { + .type = ULOGD_RET_UINT16, + .flags = ULOGD_RETF_NONE, + .name = "raw.type", + }, + [UNIXSOCK_KEY_NUFW_USER_NAME] = { + .type = ULOGD_RET_STRING, + .flags = ULOGD_RETF_NONE, + .name = "nufw.user.name", + },
Could we define some more generic instead of NUFW? I know it's your thing but others may benefit from these fields I guess, right?
Well, this is not important anyway, I don't mind too much.
Quoted 56 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +56
-0+ [UNIXSOCK_KEY_NUFW_USER_ID ] = { + .type =ULOGD_RET_UINT32 , + .flags =ULOGD_RETF_NONE , + .name ="nufw.user.id" , + }, + [UNIXSOCK_KEY_NUFW_OS_NAME ] = { + .type =ULOGD_RET_STRING , + .flags =ULOGD_RETF_NONE , + .name ="nufw.os.name" , + }, + [UNIXSOCK_KEY_NUFW_OS_REL ] = { + .type =ULOGD_RET_STRING , + .flags =ULOGD_RETF_NONE , + .name ="nufw.os.rel" , + }, + [UNIXSOCK_KEY_NUFW_OS_VERS ] = { + .type =ULOGD_RET_STRING , + .flags =ULOGD_RETF_NONE , + .name ="nufw.os.vers" , + }, + [UNIXSOCK_KEY_NUFW_APP_NAME ] = { + .type =ULOGD_RET_STRING , + .flags =ULOGD_RETF_NONE , + .name ="nufw.app.name" , + }, +}; + +static struct config_keyset libunixsock_kset = { + .num_ces =5 , + .ces = { + { + .key ="socket_path" , + .type =CONFIG_TYPE_STRING , + .options =CONFIG_OPT_NONE , + .u.string =UNIXSOCK_UNIXPATH_DEFAULT , + }, + { + .key ="bufsize" , + .type =CONFIG_TYPE_INT , + .options =CONFIG_OPT_NONE , + .u.value =UNIXSOCK_BUFSIZE_DEFAULT , + }, + { + .key ="perms" , + .type =CONFIG_TYPE_INT , + .options =CONFIG_OPT_NONE , + .u.value =UNIXSOCK_PERMS_DEFAULT , + }, + { + .key ="owner" , + .type =CONFIG_TYPE_STRING , + .options =CONFIG_OPT_NONE , + }, + { + .key ="group" , + .type =CONFIG_TYPE_STRING ,
input/packet/ulogd_inppkt_UNIXSOCK.c line 315
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+ .options = CONFIG_OPT_NONE, + }, + }, +}; + +#define unixpath_ce (x) ((x)->ces[0]) +#define bufsize_ce (x) ((x)->ces[1]) +#define perms_ce (x) ((x)->ces[2]) +#define owner_ce (x) ((x)->ces[3]) +#define group_ce (x) ((x)->ces[4])
I know that this is xyz_ce() macros are the "standard way" in ulogd2 but it's ugly. I would do the following:
enum {
UNIXSOCK_OPT_UNIXPATH = 0,
UNIXSOCK_OPT_BUFSIZE,
UNIXSOCK_OPT_PERM,
UNIXSOCK_OPT_OWNER,
UNIXSOCK_OPT_GROUP,
};
Then, we can do the transition easily to several functions like:
ulogd_option_get_str(UNIXSOCK_OPT_UNIXPATH, upi->config_kset); ulogd_option_get_u32(UNIXSOCK_OPT_PERM, upi->config_kset);
and so on... this is something that I had in my head. I'm not asking you to do so but to tell you about my intentions. If we start defining the enums now, we can do the transition in the near future.
Quoted 15 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +15
-0+ + +enum ulogd2_option_type { +ULOGD2_OPT_UNUSED =0 , +ULOGD2_OPT_PREFIX ,/* log prefix (string) */ +ULOGD2_OPT_OOB_IN ,/* input device (string) */ +ULOGD2_OPT_OOB_OUT ,/* output device (string) */ +ULOGD2_OPT_OOB_TIME_SEC ,/* packet arrival time (u_int32_t) */ + +ULOGD2_OPT_USER =200 ,/* user name (string) */ +ULOGD2_OPT_USERID ,/* user id (u_int32_t) */ +ULOGD2_OPT_OSNAME ,/* OS name (string) */ +ULOGD2_OPT_OSREL ,/* OS release (string) */ +ULOGD2_OPT_OSVERS ,/* OS version (string) */ +ULOGD2_OPT_APPNAME ,/* application name (string) */
input/packet/ulogd_inppkt_UNIXSOCK.c line 340
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+ ULOGD2_OPT_STATE, /* connection state: 0 (drop), 1 (open), 2 (established), 3 (close), 4 (unknown) */ +}; + +struct ulogd_unixsock_packet_t { +uint32_t marker; +uint16_t total_size; +uint16_t payload_length; +uint32_t reserved; +struct iphdr payload; +} __attribute__((packed));
Probably you can add a version here (even if unused, I'd suggest some bits of the reserved field), so if you have a change the header layout you can support both versions without breaking backward compatibility.
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ +struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8
Minor question: why align this to 64 bits?
Same thing, I'd remove those extra lines here below.
Quoted 6 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +6
-0+ + +static int handle_packet (struct ulogd_pluginstance *upi,struct ulogd_unixsock_packet_t *pkt,u_int16_t total_len) +{ +char *data; +struct iphdr *ip;
input/packet/ulogd_inppkt_UNIXSOCK.c line 364
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+struct ulogd_key *ret = upi->output.keys; +u_int8_t oob_family; +u_int16_t payload_len; +u_int16_t option_number; +u_int16_t option_length; +char *buf; +struct ulogd_unixsock_option_t *option; +int new_offset; +char *options_start; +
unnecessary line break below?
input/packet/ulogd_inppkt_UNIXSOCK.c line 368
Quoted 4 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +4
-0+ulogd_log (ULOGD_DEBUG , +"ulogd2: handling packet\n" ); + + data =NULL ;
initialize data variable above to NULL. Remove extra line?
input/packet/ulogd_inppkt_UNIXSOCK.c line 372
Quoted 4 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +4
-0+ + payload_len =ntohs (pkt->payload_length); + + ip = &pkt->payload;
unnecessary extra lines.
input/packet/ulogd_inppkt_UNIXSOCK.c line 380
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ +if (ip->version ==4 ) + oob_family =AF_INET ; +else if (ip->version ==6 ) + oob_family =AF_INET6 ; +else oob_family =0 ; + +okey_set_u8 (&ret[UNIXSOCK_KEY_OOB_FAMILY ], oob_family);
remove extra line.
input/packet/ulogd_inppkt_UNIXSOCK.c line 388
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ +okey_set_ptr (&ret[UNIXSOCK_KEY_RAW_PCKT ], ip); +okey_set_u32 (&ret[UNIXSOCK_KEY_RAW_PCKTLEN ], payload_len); + +/* options */ +if (total_len > payload_len +sizeof (u_int16_t)) { +/* option starts at the next aligned address after the payload */ + new_offset =ALIGN_SIZE + ((payload_len -1 ) & ~(ALIGN_SIZE -1 ));
I'd define a macro like NLMSG_ALIGN(), those who are familiar with Netlink would easily notice what you're doing :-).
input/packet/ulogd_inppkt_UNIXSOCK.c line 401
Quoted 13 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +13
-0+ options_start = (char *)ip + new_offset; + data = options_start; +/* 8 here is offsetof(ulogd_unixsock_packet_t,payload) - offsetof(ulogd_unixsock_packet_t,total_size) */ + total_len -= (new_offset +8 ); + +while ( (data - options_start) < total_len) { + + option = (void *)data; + + option_number =ntohs (option->option_id); + option_length =ntohs (option->option_length); + data +=2 *sizeof (u_int16_t); + buf = data;
Check include/linux/netlink.h and include/net/netlink.h. I think that a couple of macros would be nicer for this parsing.
input/packet/ulogd_inppkt_UNIXSOCK.c line 412
Quoted 11 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +11
-0+ +/* next option is also aligned */ + new_offset =ALIGN_SIZE + ((option_length -1 ) & ~(ALIGN_SIZE -1 )); + + data += new_offset; + +ulogd_log (ULOGD_DEBUG , +"ulogd2: option %d (len %d) `%s'\n" , + option_number, option_length, buf); + +switch (option_number) {
If you put the case in the same line of the switch you save one indent and lines are less likely to wrap around after 80 chars per column :-)
Quoted 48 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +48
-0+case ULOGD2_OPT_PREFIX : +okey_set_ptr (&ret[UNIXSOCK_KEY_OOB_PREFIX ], buf); +break ; +case ULOGD2_OPT_OOB_IN : +okey_set_ptr (&ret[UNIXSOCK_KEY_OOB_IN ], buf); +break ; +case ULOGD2_OPT_OOB_OUT : +okey_set_ptr (&ret[UNIXSOCK_KEY_OOB_OUT ], buf); +break ; +case ULOGD2_OPT_OOB_TIME_SEC : +okey_set_u32 (&ret[UNIXSOCK_KEY_OOB_TIME_SEC ], *(u_int32_t *)buf); +break ; +case ULOGD2_OPT_USER : +okey_set_ptr (&ret[UNIXSOCK_KEY_NUFW_USER_NAME ], buf); +break ; +case ULOGD2_OPT_USERID : +okey_set_u32 (&ret[UNIXSOCK_KEY_NUFW_USER_ID ], *(u_int32_t *)buf); +break ; +case ULOGD2_OPT_OSNAME : +okey_set_ptr (&ret[UNIXSOCK_KEY_NUFW_OS_NAME ], buf); +break ; +case ULOGD2_OPT_OSREL : +okey_set_ptr (&ret[UNIXSOCK_KEY_NUFW_OS_REL ], buf); +break ; +case ULOGD2_OPT_OSVERS : +okey_set_ptr (&ret[UNIXSOCK_KEY_NUFW_OS_VERS ], buf); +break ; +case ULOGD2_OPT_APPNAME : +okey_set_ptr (&ret[UNIXSOCK_KEY_NUFW_APP_NAME ], buf); +break ; +case ULOGD2_OPT_STATE : +okey_set_u8 (&ret[UNIXSOCK_KEY_RAW_LABEL ], *(u_int8_t *)buf); +break ; +default : +ulogd_log (ULOGD_NOTICE , +"ulogd2: unknown option %d\n" , + option_number); +break ; + }; + } + } + +/* number of packets */ +okey_set_u32 (&ret[UNIXSOCK_KEY_RAW_PCKTCOUNT ],1 ); + +ulogd_propagate_results (upi); + +return 0 ;
input/packet/ulogd_inppkt_UNIXSOCK.c line 470
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+} + +static int _create_unix_socket (const char *unix_path) +{ +int ret =-1 ; +struct sockaddr_un server_sock; +int s; + + s =socket (AF_UNIX ,SOCK_STREAM ,0 ); +if (s <0 )
ulogd_log(ULOGD_ERROR, ...)
input/packet/ulogd_inppkt_UNIXSOCK.c line 478
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+return -1 ; + + server_sock.sun_family =AF_UNIX ; +strncpy (server_sock.sun_path, unix_path,sizeof (server_sock.sun_path)); + server_sock.sun_path[sizeof (server_sock.sun_path)-1 ] ='\0' ; + +/* remove existing socket, if any */ +unlink (unix_path);
Probably it's a good idea to check if there's another socket still open. You can avoid having two ulogd2 instances by mistake.
Quoted 100 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +100
-0+ + ret =bind (s, (struct sockaddr *)&server_sock,sizeof (server_sock)); +if (ret <0 ) { +ulogd_log (ULOGD_ERROR , +"ulogd2: could not bind to unix socket \'%s\'\n" , + server_sock.sun_path); +close (s); +return -1 ; + } + + ret =listen (s,10 ); +if (ret <0 ) { +ulogd_log (ULOGD_ERROR , +"ulogd2: could not bind to unix socket \'%s\'\n" , + server_sock.sun_path); +close (s); +return -1 ; + } + + +return s; +} + +static int _unix_socket_set_permissions (struct ulogd_pluginstance *upi) +{ +const char *socket_path; +const char *owner =owner_ce (upi->config_kset).u.string; +const char *group =group_ce (upi->config_kset).u.string; +uid_t uid = (uid_t)-1 ; +gid_t gid = (gid_t)-1 ; + + socket_path =unixpath_ce (upi->config_kset).u.string; + +if (chmod (socket_path,perms_ce (upi->config_kset).u.value) <0 ) { +ulogd_log (ULOGD_ERROR ,"Could not set permissions on unix socket\n" ); +return -1 ; + } + +if (owner &&strlen (owner)>0 ) { +struct passwd *p =getpwnam (owner); + +if (p ==NULL ) { +ulogd_log (ULOGD_ERROR ,"Invalid owner specified for unix socket (%s)\n" , owner); +return -1 ; + } + + uid = p->pw_uid; + } + +if (group &&strlen (group)>0 ) { +struct group *g =getgrnam (group); + +if (g ==NULL ) { +ulogd_log (ULOGD_ERROR ,"Invalid group specified for unix socket (%s)\n" , group); +return -1 ; + } + + gid = g->gr_gid; + } + +if (chown (socket_path, uid, gid) <0 ) { +ulogd_log (ULOGD_ERROR ,"Could not set owner/group of unix socket\n" ); +return -1 ; + } + +return 0 ; +} + +/* warning: this code is NOT reentrant ! */ +static void _timer_unregister_cb (struct ulogd_timer *a,void *param) +{ +struct unixsock_input *ui = param; + +if (ui->unixsock_instance_fd.fd >=0 ) { +ulogd_log (ULOGD_DEBUG ," removing client from list\n" ); +ulogd_unregister_fd (&ui->unixsock_instance_fd); +close (ui->unixsock_instance_fd.fd); + ui->unixsock_instance_fd.fd =-1 ; + ui->unixsock_buf_avail =0 ; + } +} + +static void _disconnect_client (struct unixsock_input *ui) +{ +struct ulogd_timer *t =malloc (sizeof (struct ulogd_timer )); + +/* we can't call ulogd_unregister_fd fd, it will segfault +* (unable to remove an entry while inside llist_for_each_entry) +* so we schedule removal for next loop +*/ +ulogd_init_timer (t, ui, _timer_unregister_cb); +ulogd_add_timer (t,0 ); +} + +/* callback called from ulogd core when fd is readable */ +static int unixsock_instance_read_cb (int fd,unsigned int what,void *param) +{ +struct ulogd_pluginstance *upi = param; +struct unixsock_input *ui = (struct unixsock_input *)upi->private; +int len;
input/packet/ulogd_inppkt_UNIXSOCK.c line 588
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+u_int16_t needed_len; +u_int32_t packet_sig; +struct ulogd_unixsock_packet_t *unixsock_packet; + +char buf[4096 ]; + +if (!(what &ULOGD_FD_READ )) +return 0 ; + + len =read (fd, buf,sizeof (buf));
Ah, looking at this, I think that it would be a good idea to ignore SIGPIPE to avoid crashing ulogd2 if the other peer has gone.
Quoted 4 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +4
-0+if (len <0 ) { +ulogd_log (ULOGD_NOTICE ," read returned %d, errno is %d (%s)\n" , + len, errno,strerror (errno)); +exit (-1 );
input/packet/ulogd_inppkt_UNIXSOCK.c line 602
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+return len; + } +if (len ==0 ) { +_disconnect_client (ui); +ulogd_log (ULOGD_DEBUG ," client disconnected\n" ); +return 0 ; + } + +//ulogd_log(ULOGD_DEBUG, " read %d bytes\n", len); +//ulogd_log(ULOGD_DEBUG, " buffer [%s]\n", buf);
Remove or leave the lines above, but those double slashs looks a bit ugly to me.
input/packet/ulogd_inppkt_UNIXSOCK.c line 613
Quoted 11 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +11
-0+ +if (ui->unixsock_buf_avail + len > ui->unixsock_buf_size) { +ulogd_log (ULOGD_NOTICE , +"We are losing events. Please consider using the clause " +"bufsize\n" ); +return -1 ; + } + +memcpy (ui->unixsock_buf + ui->unixsock_buf_avail, + buf, + len);
We need that line break above, really?
input/packet/ulogd_inppkt_UNIXSOCK.c line 621
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ ui->unixsock_buf_avail += len; + + do { + unixsock_packet = (void*)ui->unixsock_buf; + packet_sig = ntohl(unixsock_packet->marker); + if (packet_sig != ULOGD_SOCKET_MARK) { + ulogd_log(ULOGD_ERROR, + "ulogd2: invalid packet marked received (read %lx, expected %lx), closing socket.\n",
This line above goes over 80 chars per column?
Quoted 24 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +24
-0+ packet_sig, ULOGD_SOCKET_MARK); +_disconnect_client (ui); +return -1 ; + + } + + needed_len =ntohs (unixsock_packet->total_size); + +if (ui->unixsock_buf_avail >= needed_len +sizeof (u_int32_t)) { +ulogd_log (ULOGD_DEBUG ," We have enough data (%d bytes required), handling packet\n" , needed_len); + +if (handle_packet (upi, unixsock_packet, needed_len) !=0 ) { +return -1 ; + } +/* consume data */ + ui->unixsock_buf_avail -= (sizeof (u_int32_t) + needed_len); +if (ui->unixsock_buf_avail >0 ) { +/* we need to shift data .. */ +memmove (ui->unixsock_buf, + ui->unixsock_buf + (sizeof (u_int32_t) + needed_len) , + ui->unixsock_buf_avail); + }else { +/* input buffer is empty, do not loop */ +return 0 ;
input/packet/ulogd_inppkt_UNIXSOCK.c line 655
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+ } + + }else { +ulogd_log (ULOGD_DEBUG ," We have %d bytes, but need %d. Requesting more\n" , + ui->unixsock_buf_avail, needed_len +sizeof (u_int32_t)); +return 0 ; + } + +/* handle_packet has shifted data in buffer */ + }while (1 );
I'd put this while(1) above for readability so I don't need to go until the end of the code to notice that this is an endless loop ;-).
Quoted 35 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +35
-0+ +return 0 ; +} + +/* callback called from ulogd core when fd is readable */ +static int unixsock_server_read_cb (int fd,unsigned int what,void *param) +{ +struct ulogd_pluginstance *upi = param; +struct unixsock_input *ui = (struct unixsock_input *)upi->private; +socklen_t len; +int s; +struct sockaddr_storage saddr; + +if (!(what &ULOGD_FD_READ )) +return 0 ; + +ulogd_log (ULOGD_DEBUG ,"New server connected on unixsock socket\n" ); + + len =sizeof (saddr); + s =accept (fd, (struct sockaddr *)&saddr, &len); +if (s <0 ) { +ulogd_log (ULOGD_NOTICE , +" error while accepting new unixsock client, errno is %d (%s)\n" , + errno,strerror (errno)); +return len; + } + +if (ui->unixsock_instance_fd.fd >=0 ) { +ulogd_log (ULOGD_NOTICE ,"a client is already connecting, rejecting new connection" ); +close (s); +return 0 ; + } + + ui->unixsock_instance_fd.fd = s; + ui->unixsock_instance_fd.cb = &unixsock_instance_read_cb;
input/packet/ulogd_inppkt_UNIXSOCK.c line 700
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+ ui->unixsock_instance_fd.data = upi; + ui->unixsock_instance_fd.when =ULOGD_FD_READ ; + +if (ulogd_register_fd (&ui->unixsock_instance_fd) <0 ) { +ulogd_log (ULOGD_ERROR ,"unable to register client fd to ulogd\n" ); +return -1 ; + } + +return 0 ; +}
unnecessary extra lines below.
Quoted 138 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +115
-0+ + + + +static int configure (struct ulogd_pluginstance *upi, +struct ulogd_pluginstance_stack *stack) +{ +ulogd_log (ULOGD_DEBUG ,"parsing config file section `%s', " +"plugin `%s'\n" , upi->id, upi->plugin->name); + +config_parse_file (upi->id, upi->config_kset); +return 0 ; +} + +static int start (struct ulogd_pluginstance *upi) +{ +struct unixsock_input *ui = (struct unixsock_input *) upi->private; +int fd; + +ulogd_log (ULOGD_DEBUG ,"Starting plugin `%s'\n" , + upi->plugin->name); + + ui->path =unixpath_ce (upi->config_kset).u.string; + +ulogd_log (ULOGD_DEBUG ,"Creating Unix socket `%s'\n" , + ui->path); + fd =_create_unix_socket (ui->path); +if (fd <0 ) { +ulogd_log (ULOGD_ERROR ,"Unable to create unix socket on `%s'\n" , + ui->path); +return -1 ; + } + +if (_unix_socket_set_permissions (upi) <0 ) { +return -1 ; + } + + ui->unixsock_buf_avail =0 ; + ui->unixsock_buf_size =bufsize_ce (upi->config_kset).u.value; + +if (ui->unixsock_buf_size ==0 ) { +int fd_bufsize =0 ; +socklen_t optlen =sizeof (fd_bufsize); + +if (getsockopt (fd,SOL_SOCKET ,SO_RCVBUF , &fd_bufsize, &optlen) <0 ) { +ulogd_log (ULOGD_ERROR , +"Could not determine socket buffer size. You have to use the clause " +"bufsize\n" ); +return -1 ; + } +ulogd_log (ULOGD_DEBUG ,"bufsize is %d\n" , fd_bufsize); + + ui->unixsock_buf_size = fd_bufsize; + } + ui->unixsock_buf =malloc (ui->unixsock_buf_size); + + ui->unixsock_server_fd.fd = fd; + ui->unixsock_server_fd.cb = &unixsock_server_read_cb; + ui->unixsock_server_fd.data = upi; + ui->unixsock_server_fd.when =ULOGD_FD_READ ; + + ui->unixsock_instance_fd.fd =-1 ; + ui->unixsock_instance_fd.cb = &unixsock_instance_read_cb; + ui->unixsock_instance_fd.data = upi; + ui->unixsock_instance_fd.when =ULOGD_FD_READ ; + +if (ulogd_register_fd (&ui->unixsock_server_fd) <0 ) { +ulogd_log (ULOGD_ERROR ,"Unable to register fd to ulogd\n" ); +return -1 ; + } + +return 0 ; +} + +static int stop (struct ulogd_pluginstance *upi) +{ +struct unixsock_input *ui = (struct unixsock_input *) upi->private; +char *unix_path =unixpath_ce (upi->config_kset).u.string; + +ulogd_log (ULOGD_DEBUG ,"Stopping plugin `%s'\n" , + upi->plugin->name); + +if (unix_path) +unlink (unix_path); + +free (ui->unixsock_buf); + +return 0 ; +} + + + + +struct ulogd_plugin libunixsock_plugin = { + .name ="UNIXSOCK" , + .input = { + .type =ULOGD_DTYPE_SOURCE , + }, + .output = { + .type =ULOGD_DTYPE_RAW , + .keys = output_keys, + .num_keys =ARRAY_SIZE (output_keys), + }, + .priv_size =sizeof (struct unixsock_input ), + .configure = &configure, + .start = &start, + .stop = &stop, + .config_kset = &libunixsock_kset, + .version =ULOGD_VERSION , +}; + +static void __attribute__ ((constructor))init (void ) +{ +ulogd_register_plugin (&libunixsock_plugin); +}ulogd.conf.in +4
-0@@ -27,6 +27,7 @@ loglevel=1 plugin="@libdir@/ulogd/ulogd_inppkt_NFLOG.so" #plugin="@libdir@/ulogd/ulogd_inppkt_ULOG.so" +#plugin="@libdir@/ulogd/ulogd_inppkt_UNIXSOCK.so" plugin="@libdir@/ulogd/ulogd_inpflow_NFCT.so" plugin="@libdir@/ulogd/ulogd_filter_IFINDEX.so" plugin="@libdir@/ulogd/ulogd_filter_IP2STR.so"@@ -75,6 +76,9 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" # this is a stack for logging packets to syslog after a collect via NFLOG #stack=log3:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,sys1:SYSLOG +# this is a stack for logging packets to syslog after a collect via NuFW +#stack=nuauth1:UNIXSOCK,base1:BASE,ip2str1:IP2STR,print1:PRINTPKT,sys1:SYSLOG + # this is a stack for flow-based logging to MySQL #stack=ct1:NFCT,ip2bin1:IP2BIN,mysql2:MYSQL
ulogd.conf.in numeric_label=1 # you can label the log info based on the packet verdict · line 149
Quoted 9 lines
ulogd.conf.in +3
-0@@ -137,6 +141,9 @@ numeric_label=1 # you can label the log info based on the packet verdict nlgroup=1 #numeric_label=0 # optional argument +[nuauth1] +socket_path="/tmp/nuauth_ulogd2.sock" + [emu1] file="/var/log/ulogd_syslogemu.log" sync=1
Interesting :-). I'm sorry that I couldn't provide you feedback until now.
Re: [PATCH 1/2] Add new input plugin UNIXSOCK
Quoted 9 lines · Pablo Neira Ayuso
Hi Pierre,
Quoted 4 lines · Pierre Chifflier
This input plugins creates a unix socket which can be used to log packets. Scripts or applications can connect to the socket (only one client allowed per socket) and send data in a Key-Length-Value format (including the payload).
I'm proposing some minor changes, they are mostly cleanups and comestic.
Hi Pablo,
Thanks for the review. I have fixed almost every points you have mentioned, there are only a few points to discuss.
Quoted 5 lines
BTW, how does the "big picture" of this look like? I guess that your nufw daemon is listening to packets via
libnetfilter_queueand then it sends unixsock messages that include the packet + extra authentication information to ulogd2 to log them, right?
Yes. We use ulogd as a central point for logging information, since packets (and decision) can come either from iptables (for non-user rules) or from the nufw+nuauth chain (for user filtering rules). When logging from nufw, we use the unixsock message to include the original packet (so the standard code from ulogd2 will be used) and add our additional information (user, application, etc).
nflog + (drop|accept)
/ \
netfilter ulogd2
\ /
(nfq) nufw --> nuauthYet this code has nothing specific to nufw, it can be used by any application to inject data with extra information.
Quoted 2 lines
could this be extended in the future to support flow-based information? I mean, does this really fit into the "packet" directory of ulogd2?
Well, I don't have any fixed opinion on this. If you want I can move this plugin to the flow directory (in some way it is similar to IPFIX).
[...]
Quoted 7 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 3
Quoted 5 lines
+++
b/input/packet/ulogd_inppkt_UNIXSOCK.cinput/packet/ulogd_inppkt_UNIXSOCK.c new file +3
-0@@ -0,0 +1,815 @@ +/* +** Copyright(C) 2008-2010 INL +** Written by Pierre Chifflier <[email protected]> Minor nitpick: please, add licensing terms here in a short header.
Done
[...]
Quoted 5 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 31
Quoted 2 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +2
-0+/* Default unix socket path */ +#define UNIXSOCK_UNIXPATH_DEFAULT "/var/run/ulogd/ulogd2.sock"These extra lines below are unnecessarly, I'd say. They don't help to make the code more readable.
Fixed
Quoted 6 lines
Quoted 6 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +5
-0+ +/* Unique value used as a signature to ensure received data is really +* a packet +*/ +#define ULOGD_SOCKET_MARK 0x41c90fd4Same here
Fixed
[...]
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 249
Quoted 5 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +5
-0+ [UNIXSOCK_KEY_NUFW_USER_NAME] = { + .type = ULOGD_RET_STRING, + .flags = ULOGD_RETF_NONE, + .name = "nufw.user.name", + },Could we define some more generic instead of NUFW? I know it's your thing but others may benefit from these fields I guess, right?
Well, this is not important anyway, I don't mind too much.
Sure. Should I use something like 'EXTRA' instead of NUFW in the constants names ?
[...]
Quoted 18 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 315
Quoted 6 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +6
-0+ +#define unixpath_ce (x) ((x)->ces[0]) +#define bufsize_ce (x) ((x)->ces[1]) +#define perms_ce (x) ((x)->ces[2]) +#define owner_ce (x) ((x)->ces[3]) +#define group_ce (x) ((x)->ces[4])I know that this is
xyz_ce()macros are the "standard way" in ulogd2 but it's ugly. I would do the following:enum {
UNIXSOCK_OPT_UNIXPATH= 0,
UNIXSOCK_OPT_BUFSIZE,
UNIXSOCK_OPT_PERM,
UNIXSOCK_OPT_OWNER,
UNIXSOCK_OPT_GROUP,
};
Quoted 8 lines
Then, we can do the transition easily to several functions like:
ulogd_option_get_str(UNIXSOCK_OPT_UNIXPATH, upi->config_kset);ulogd_option_get_u32(UNIXSOCK_OPT_PERM, upi->config_kset);and so on... this is something that I had in my head. I'm not asking you to do so but to tell you about my intentions. If we start defining the enums now, we can do the transition in the near future.
I'd like that too ! I've defined the enum (and kept the xyz_ce macros for now). If you want, I can add these functions in another patch ?
Quoted 4 lines
Quoted 4 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +4
-0+ +struct ulogd_unixsock_packet_t { +uint32_t marker; +uint16_t total_size;
Quoted 9 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 340
Quoted 4 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +4
-0+uint16_t payload_length; +uint32_t reserved; +struct iphdr payload; +} __attribute__((packed));Probably you can add a version here (even if unused, I'd suggest some bits of the reserved field), so if you have a change the header layout you can support both versions without breaking backward compatibility.
Indeed, thanks for the suggestion.
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ +struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8Minor question: why align this to 64 bits?
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
Quoted 2 lines
Same thing, I'd remove those extra lines here below.
I've removed empty lines and extra line breaks.
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 388
Quoted 5 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +5
-0+ +/* options */ +if (total_len > payload_len +sizeof (u_int16_t)) { +/* option starts at the next aligned address after the payload */ + new_offset =ALIGN_SIZE + ((payload_len -1 ) & ~(ALIGN_SIZE -1 ));I'd define a macro like
NLMSG_ALIGN(), those who are familiar with Netlink would easily notice what you're doing :-).
Will do :)
[...]
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 478
Quoted 7 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +7
-0+ + server_sock.sun_family =AF_UNIX ; +strncpy (server_sock.sun_path, unix_path,sizeof (server_sock.sun_path)); + server_sock.sun_path[sizeof (server_sock.sun_path)-1 ] ='\0' ; + +/* remove existing socket, if any */ +unlink (unix_path);Probably it's a good idea to check if there's another socket still open. You can avoid having two ulogd2 instances by mistake.
Agreed, it's safer.
[...]
Quoted 11 lines
Quoted 11 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +11
-0+/* callback called from ulogd core when fd is readable */ +static int unixsock_instance_read_cb (int fd,unsigned int what,void *param) +{ +struct ulogd_pluginstance *upi = param; +struct unixsock_input *ui = (struct unixsock_input *)upi->private; +int len; +u_int16_t needed_len; +u_int32_t packet_sig; +struct ulogd_unixsock_packet_t *unixsock_packet; + +char buf[4096 ];
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 588
Quoted 5 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +5
-0+ +if (!(what &ULOGD_FD_READ )) +return 0 ; + + len =read (fd, buf,sizeof (buf));Ah, looking at this, I think that it would be a good idea to ignore SIGPIPE to avoid crashing ulogd2 if the other peer has gone.
Yes, I didn't want to do that in the plugin itself. Can I put that in src/ulogd.c ?
Quoted 10 lines
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +9
-0+if (len <0 ) { +ulogd_log (ULOGD_NOTICE ," read returned %d, errno is %d (%s)\n" , + len, errno,strerror (errno)); +exit (-1 ); +return len; + } +if (len ==0 ) { +_disconnect_client (ui); +ulogd_log (ULOGD_DEBUG ," client disconnected\n" );
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 602
Quoted 5 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +5
-0+return 0 ; + } + +//ulogd_log(ULOGD_DEBUG, " read %d bytes\n", len); +//ulogd_log(ULOGD_DEBUG, " buffer [%s]\n", buf); Remove or leave the lines above, but those double slashs looks a bit ugly to me.
Oops, debug code (removed).
Quoted 15 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 613
Quoted 12 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +11
-0+ +if (ui->unixsock_buf_avail + len > ui->unixsock_buf_size) { +ulogd_log (ULOGD_NOTICE , +"We are losing events. Please consider using the clause " +"bufsize\n" ); +return -1 ; + } + +memcpy (ui->unixsock_buf + ui->unixsock_buf_avail, + buf, + len);We need that line break above, really?
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 621
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ ui->unixsock_buf_avail += len; + + do { + unixsock_packet = (void*)ui->unixsock_buf; + packet_sig = ntohl(unixsock_packet->marker); + if (packet_sig != ULOGD_SOCKET_MARK) { + ulogd_log(ULOGD_ERROR, + "ulogd2: invalid packet marked received (read %lx, expected %lx), closing socket.\n",This line above goes over 80 chars per column?
I've changed some lines to be more 80-chars friendly.
Quoted 30 lines
Quoted 30 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +29
-0+ packet_sig, ULOGD_SOCKET_MARK); +_disconnect_client (ui); +return -1 ; + + } + + needed_len =ntohs (unixsock_packet->total_size); + +if (ui->unixsock_buf_avail >= needed_len +sizeof (u_int32_t)) { +ulogd_log (ULOGD_DEBUG ," We have enough data (%d bytes required), handling packet\n" , needed_len); + +if (handle_packet (upi, unixsock_packet, needed_len) !=0 ) { +return -1 ; + } +/* consume data */ + ui->unixsock_buf_avail -= (sizeof (u_int32_t) + needed_len); +if (ui->unixsock_buf_avail >0 ) { +/* we need to shift data .. */ +memmove (ui->unixsock_buf, + ui->unixsock_buf + (sizeof (u_int32_t) + needed_len) , + ui->unixsock_buf_avail); + }else { +/* input buffer is empty, do not loop */ +return 0 ; + } + + }else { +ulogd_log (ULOGD_DEBUG ," We have %d bytes, but need %d. Requesting more\n" , + ui->unixsock_buf_avail, needed_len +sizeof (u_int32_t));
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 655
Quoted 5 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +5
-0+return 0 ; + } + +/* handle_packet has shifted data in buffer */ + }while (1 );I'd put this while(1) above for readability so I don't need to go until the end of the code to notice that this is an endless loop ;-).
Heh :)
[...]
Thanks again,
Pierre
Pablo Neira Ayuso
maintainerRe: [PATCH 1/2] Add new input plugin UNIXSOCK
Hi Pierre,
Quoted 2 lines · Pierre Chifflier
Thanks for the review. I have fixed almost every points you have mentioned, there are only a few points to discuss.
Thanks.
Quoted 12 lines
Quoted 4 lines
BTW, how does the "big picture" of this look like? I guess that your nufw daemon is listening to packets via
libnetfilter_queueand then it sends unixsock messages that include the packet + extra authentication information to ulogd2 to log them, right?Yes. We use ulogd as a central point for logging information, since packets (and decision) can come either from iptables (for non-user rules) or from the nufw+nuauth chain (for user filtering rules). When logging from nufw, we use the unixsock message to include the original packet (so the standard code from ulogd2 will be used) and add our additional information (user, application, etc).
Quoted 5 lines
nflog + (drop|accept)
/ \
netfilter ulogd2\ / (nfq) nufw --> nuauth
Interesting, I have a feeling that it was similar to this ;-).
Quoted 2 lines
Yet this code has nothing specific to nufw, it can be used by any application to inject data with extra information.
I think so, but they would need to add new input keys to store that new extra information or use the _NUFW_ keys. We can add a comment after the _NUFW_ telling something that "add your new keys here".
Quoted 5 lines
Quoted 2 lines
could this be extended in the future to support flow-based information? I mean, does this really fit into the "packet" directory of ulogd2?
Well, I don't have any fixed opinion on this. If you want I can move this plugin to the flow directory (in some way it is similar to IPFIX).
My question is if this plugin is specific for packet-logging. If so, the packet/ directory is fine.
Quoted 8 lines
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 249
Quoted 5 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +5
-0+ [UNIXSOCK_KEY_NUFW_USER_NAME] = { + .type = ULOGD_RET_STRING, + .flags = ULOGD_RETF_NONE, + .name = "nufw.user.name", + },Could we define some more generic instead of NUFW? I know it's your thing but others may benefit from these fields I guess, right?
Quoted 4 lines
Quoted 1 line
Well, this is not important anyway, I don't mind too much.
Sure. Should I use something like 'EXTRA' instead of NUFW in the constants names ?
I don't mind too much, I leave it up to you.
Quoted 23 lines
[...]
Quoted 22 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 315
Quoted 6 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +6
-0+ +#define unixpath_ce (x) ((x)->ces[0]) +#define bufsize_ce (x) ((x)->ces[1]) +#define perms_ce (x) ((x)->ces[2]) +#define owner_ce (x) ((x)->ces[3]) +#define group_ce (x) ((x)->ces[4])I know that this is
xyz_ce()macros are the "standard way" in ulogd2 but it's ugly. I would do the following:enum {
UNIXSOCK_OPT_UNIXPATH= 0,
UNIXSOCK_OPT_BUFSIZE,
UNIXSOCK_OPT_PERM,
UNIXSOCK_OPT_OWNER,
UNIXSOCK_OPT_GROUP,
};Then, we can do the transition easily to several functions like:
ulogd_option_get_str(UNIXSOCK_OPT_UNIXPATH, upi->config_kset);ulogd_option_get_u32(UNIXSOCK_OPT_PERM, upi->config_kset);
Quoted 6 lines
Quoted 3 lines
and so on... this is something that I had in my head. I'm not asking you to do so but to tell you about my intentions. If we start defining the enums now, we can do the transition in the near future.
I'd like that too ! I've defined the enum (and kept the
xyz_cemacros for now). If you want, I can add these functions in another patch ?
Sure, go ahead if you want :-).
[...]
Quoted 10 lines
Quoted 9 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ +struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8Minor question: why align this to 64 bits?
Quoted 3 lines
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
I would need to look into this in more detail, not sure where the problem is. I think that you can use something like `struct nlattr' (see include/linux/netlink.h) and then nla_put() to add attributes in the TLV format (see lib/nlattr.c). Those are align-safe. I'm using something similar for conntrackd for the synchronization messages (src/build.c and src/parse.c).
Quoted 19 lines
[...]
Quoted 10 lines
Quoted 10 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +10
-0+/* callback called from ulogd core when fd is readable */ +static int unixsock_instance_read_cb (int fd,unsigned int what,void *param) +{ +struct ulogd_pluginstance *upi = param; +struct unixsock_input *ui = (struct unixsock_input *)upi->private; +int len; +u_int16_t needed_len; +u_int32_t packet_sig; +struct ulogd_unixsock_packet_t *unixsock_packet; +Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 588
Quoted 6 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +6
-0+char buf[4096 ]; + +if (!(what &ULOGD_FD_READ )) +return 0 ; + + len =read (fd, buf,sizeof (buf));Ah, looking at this, I think that it would be a good idea to ignore SIGPIPE to avoid crashing ulogd2 if the other peer has gone.
Quoted 2 lines
Yes, I didn't want to do that in the plugin itself. Can I put that in
src/ulogd.c?
Yes, you can add a comment in ulogd.c telling that at least UNIXSOCK needs this so we don't forget ;).
Re: [PATCH 1/2] Add new input plugin UNIXSOCK
Quoted 10 lines · Pablo Neira Ayuso
Quoted 10 lines
Quoted 9 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +8
-0+ +struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8Minor question: why align this to 64 bits?
Quoted 6 lines
Quoted 3 lines
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
I would need to look into this in more detail, not sure where the problem is.
The problem is that *(uint64_t *)x as used in unixsock.c would otherwise cause an unaligned access. Copying from an odd address is thus more costly.
Re: [PATCH 1/2] Add new input plugin UNIXSOCK
In reply to Pablo Neira AyusoQuoted 13 lines · Pablo Neira Ayuso
Quoted 9 lines
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 7 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +7
-0+struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8Minor question: why align this to 64 bits?
Quoted 3 lines
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
Quoted 7 lines
I would need to look into this in more detail, not sure where the problem is. I think that you can use something like `struct nlattr' (see
include/linux/netlink.h) and thennla_put()to add attributes in the TLV format (seelib/nlattr.c). Those are align-safe. I'm using something similar for conntrackd for the synchronization messages (src/build.candsrc/parse.c).
Yes, this is very similar though NLA_ALIGNTO is set to 4 which will cause problems with 64 bits integers.
The other way to solve this would be to read integers byte per byte, like in [1], but I found this not very elegant (and is likely to be slow compared to aligned access).
Or do you have any preferred solution ? Maybe using nlattr + one special function for dealing with 64 bits variables ?
Pierre
Pablo Neira Ayuso
maintainerRe: [PATCH 1/2] Add new input plugin UNIXSOCK
Quoted 19 lines · Pierre Chifflier
Quoted 11 lines · Pablo Neira Ayuso
Quoted 11 lines
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 7 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +7
-0+struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8Minor question: why align this to 64 bits?
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
Quoted 6 lines
I would need to look into this in more detail, not sure where the problem is. I think that you can use something like `struct nlattr' (see
include/linux/netlink.h) and thennla_put()to add attributes in the TLV format (seelib/nlattr.c). Those are align-safe. I'm using something similar for conntrackd for the synchronization messages (src/build.candsrc/parse.c).
Quoted 8 lines
Yes, this is very similar though
NLA_ALIGNTOis set to 4 which will cause problems with 64 bits integers.
The other way to solve this would be to read integers byte per byte, like in [1], but I found this not very elegant (and is likely to be slow compared to aligned access).Or do you have any preferred solution ? Maybe using nlattr + one special function for dealing with 64 bits variables ?
Aligning this to 8 bytes seems fine to me (messages would be bigger though). I think that you can use something like:
struct ulogd_unixsock_option_t {
uint32_t option_id;
uint32_t option_length;
char option_value[0];
} __attribute__((packed));So you can benefit from the extra bytes that would be added as padding.
[ULOGD2] UNIXSOCK plugin (v6)
Hi,
Here is the sixth version with (I think) every changed included.
* empty lines removed,
* enum added for future transition of xyz_ce macros,
* alignment changed and tested,
* socket no more automatically removed when starting,
* code more 80-chars compliant
BR,
Pierre
PATCH 1/2 Add new input plugin UNIXSOCK
In reply to Pablo Neira AyusoFrom: Pierre Chifflier <[email protected]>
This input plugins creates a unix socket which can be used to log packets. Scripts or applications can connect to the socket (only one client allowed per socket) and send data in a Key-Length-Value format (including the payload).
--- input/packet/Makefile.am | 5 +- input/packet/ulogd_inppkt_UNIXSOCK.c | 826 ++++++++++++++++++++++++++++++++++ ulogd.conf.in | 7 + 3 files changed, 837 insertions(+), 1 deletions(-) create mode 100644 input/packet/ulogd_inppkt_UNIXSOCK.c
input/packet/Makefile.am +4 -1
@@ -3,7 +3,7 @@ AM_CPPFLAGS = $(all_includes) -I$(top_srcdir)/include AM_CFLAGS=-fPIC -Wall LIBS= -pkglib_LTLIBRARIES = ulogd_inppkt_NFLOG.la ulogd_inppkt_ULOG.la +pkglib_LTLIBRARIES = ulogd_inppkt_NFLOG.la ulogd_inppkt_ULOG.la ulogd_inppkt_UNIXSOCK.la ulogd_inppkt_NFLOG_la_SOURCES = ulogd_inppkt_NFLOG.c ulogd_inppkt_NFLOG_la_LDFLAGS = -avoid-version -module $(LIBNETFILTER_LOG_LIBS)
@@ -12,3 +12,6 @@ ulogd_inppkt_NFLOG_la_CFLAGS = $(AM_CFLAGS) $(LIBNETFILTER_LOG_CFLAGS) ulogd_inppkt_ULOG_la_SOURCES = ulogd_inppkt_ULOG.c ulogd_inppkt_ULOG_la_LDFLAGS = -avoid-version -module ulogd_inppkt_ULOG_la_LIBADD = ../../libipulog/libipulog.la + +ulogd_inppkt_UNIXSOCK_la_SOURCES = ulogd_inppkt_UNIXSOCK.c +ulogd_inppkt_UNIXSOCK_la_LDFLAGS = -avoid-version -module
input/packet/ulogd_inppkt_UNIXSOCK.c new file +826 -0
826 lines not on this page · raw message
ulogd.conf.in +7 -0
@@ -27,6 +27,7 @@ loglevel=1 plugin="@libdir@/ulogd/ulogd_inppkt_NFLOG.so" #plugin="@libdir@/ulogd/ulogd_inppkt_ULOG.so" +#plugin="@libdir@/ulogd/ulogd_inppkt_UNIXSOCK.so" plugin="@libdir@/ulogd/ulogd_inpflow_NFCT.so" plugin="@libdir@/ulogd/ulogd_filter_IFINDEX.so" plugin="@libdir@/ulogd/ulogd_filter_IP2STR.so"
@@ -75,6 +76,9 @@ plugin="@libdir@/ulogd/ulogd_raw2packet_BASE.so" # this is a stack for logging packets to syslog after a collect via NFLOG #stack=log3:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,sys1:SYSLOG +# this is a stack for logging packets to syslog after a collect via NuFW +#stack=nuauth1:UNIXSOCK,base1:BASE,ip2str1:IP2STR,print1:PRINTPKT,sys1:SYSLOG + # this is a stack for flow-based logging to MySQL #stack=ct1:NFCT,ip2bin1:IP2BIN,mysql2:MYSQL
@@ -137,6 +141,9 @@ numeric_label=1 # you can label the log info based on the packet verdict nlgroup=1 #numeric_label=0 # optional argument +[nuauth1] +socket_path="/tmp/nuauth_ulogd2.sock" + [emu1] file="/var/log/ulogd_syslogemu.log" sync=1
Signature · 6 lines
1.7.0 -- To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/2 Add helper script pcap2ulog
In reply to Pablo Neira AyusoFrom: Pierre Chifflier <[email protected]>
This script uses the Net::Pcap Perl library to parse an pcap file and send packets to ulogd2 throught the UNIXSOCK input module.
--- contrib/pcap2ulog | 166 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 166 insertions(+), 0 deletions(-) create mode 100755 contrib/pcap2ulog
contrib/pcap2ulog new file +166 -0
@@ -0,0 +1,166 @@ +#!/usr/bin/perl -w + +# Copyright (C) 2009-2010 Pierre Chifflier <[email protected]> +# +# This program is free software; you can redistribute it and/or modify +# it under the same terms as Perl itself, either Perl version 5.8.4 or, +# at your option, any later version of Perl 5 you may have available. + +use strict; + +use IO::Socket; +use Net::Pcap; + +my $ULOGD_SOCKET_MARK = 0x41c90fd4; + +my $dumpfile = shift or die "Unable to open pcap file"; +my($pcap_t, $err); +my($ulogd_client); +my $socketfile = "/var/run/ulogd2.sock"; +my $data_buffer; +my $linktype; +my $proto_offset; + +my %linktype_to_offset = ( + Net::Pcap::DLT_LINUX_SLL => 14, + Net::Pcap::DLT_EN10MB => 12, +); + +sub connect_ulogd2 { + (-S $socketfile) or die "ulogd2 socket $socketfile does not exist - is ulogd running ?"; + + $ulogd_client = IO::Socket::UNIX->new(Peer => $socketfile, + Type => SOCK_STREAM ) or die $!; + $ulogd_client->autoflush(0); +} + +sub print_padding +{ + my ($offset) = @_; + my $padding; + my $align = 8; + my $data; + + $padding = ($align - ($offset % $align)) % $align; + #print "offset: $offset padding $padding\n"; + + $data = "\0" x $padding; + $data_buffer .= $data; +} + +sub process_pkt { + my($user, $hdr, $pkt) = @_; + + if (($user ne "xyz") or !defined($hdr) or !defined($pkt)) { + print("Bad args passed to callback\n"); + print("Bad user data\n"), if ($user ne "xyz"); + print("Bad pkthdr\n"), if (!defined($hdr)); + print("Bad pkt data\n"), if (!defined($pkt)); + print("not ok\n"); + exit; + } + + #print "Header: len $hdr->{len}\n"; + #my $len = length $pkt; + #print "Packet length: $len\n"; + + my $size = length($pkt) - ($proto_offset+2); + + #my $pcaphdr = unpack ("H*", substr ($pkt, 0, 16)); + #printf("pcap hdr: $pcaphdr\n"); + my $proto = unpack ("H*", substr ($pkt, $proto_offset, 2)); + #printf("proto: $proto\n"); + + if ($proto ne "0800") { + print "ignoring packet with proto $proto\n"; + return; + } + + #my $ip_firstbyte = unpack ("H*", substr ($pkt, $proto_offset+2, 2)); + #printf("ip_firstbyte: $ip_firstbyte\n"); + + # decode packet for a SLL: + # packet type (sent by us: 4) + # link layer address type: 1 + # link layer address length: 6 + # src dst + # protocol (IP, ARP, PPP, SNMP ...) + # data + my $srcmac = substr ($pkt, 6, 6); + + (my $hex_src = unpack("H*", $srcmac)) =~ s/(..)/$1:/g; + chop $hex_src; + #printf "source mac: $hex_src\n"; + + my $hex_dst = "\0"; + + # format data + my $data; + + $data_buffer = undef; + + # ulogd packet signature + $data = pack ('N', $ULOGD_SOCKET_MARK); + + $data_buffer .= $data; + + my $options_num=2; + my $options_len=length($hex_src) + length($hex_dst); + # total length (will be filled later) + my $total_size = 0; + $data = pack ('n', $total_size); + $data_buffer .= $data; + + # reserved + payload length + payload + $data = pack ('Nna*', 0, $size, substr($pkt,$proto_offset+2,$size)); + $data_buffer .= $data; + print_padding($size); + + # options + my $OOB_IN = 2; + $data = pack ('NNa*', $OOB_IN, length($hex_src), $hex_src); + $data_buffer .= $data; + print_padding(length($hex_src)); + my $OOB_OUT = 3; + $data = pack ('NNa*', $OOB_OUT, length($hex_dst), $hex_dst); + $data_buffer .= $data; + print_padding(length($hex_dst)); + + # replace total size in buffer + my $l = length($data_buffer) - 4; + substr($data_buffer, 4, 2) = pack('n', $l); + + #(my $hex = unpack("H*", $data_buffer)) =~ s/(..)/$1 /g; + #print "$l will be encoded as " . unpack("H*", pack('n', $l)) . "\n"; + #print $hex, "\n"; + + print $ulogd_client $data_buffer; + + $ulogd_client->flush; + + #exit; +} + + +connect_ulogd2 or die $!; + +$pcap_t = Net::Pcap::open_offline($dumpfile, \$err); +if (!defined($pcap_t)) { + print("Net::Pcap::dump_open failed: ", Net::Pcap::geterr($pcap_t), "\n"); + exit; +} + +$linktype = Net::Pcap::pcap_datalink($pcap_t); + +if (not exists $linktype_to_offset{$linktype}) { + print("Unsupported link type ", Net::Pcap::pcap_datalink_val_to_name($linktype), "\n"); + exit 1; +} + +$proto_offset = $linktype_to_offset{$linktype}; + +Net::Pcap::loop($pcap_t, -1, \&process_pkt, "xyz"); +Net::Pcap::close($pcap_t); + + +
Signature · 1 line
1.7.0
Re: [PATCH 1/2] Add new input plugin UNIXSOCK
In reply to Pablo Neira AyusoQuoted 11 lines · Pablo Neira Ayuso
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
+#define
ALIGN_SIZE8Minor question: why align this to 64 bits?
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
I would need to look into this in more detail, not sure where the problem is. I think that you can use something like `struct nlattr' (see
include/linux/netlink.h) and thennla_put()to add attributes in the TLV format (seelib/nlattr.c). Those are align-safe. I'm using something similar for conntrackd for the synchronization messages (src/build.candsrc/parse.c).
If they are align-safe, what's this? :-)
18:41 ares:/home/jengelh # conntrack -L
Bus error
18:41 ares:/home/jengelh # file which conntrack /usr/sbin/conntrack: ELF 32-bit MSB executable, SPARC32PLUS, V8+ Required, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.4, not stripped
Re: [PATCH 1/2] Add new input plugin UNIXSOCK
Quoted 9 lines · Jan Engelhardt
If they are align-safe, what's this? :-)
18:41 ares:/home/jengelh # conntrack -L
Bus error
18:41 ares:/home/jengelh # filewhich conntrack/usr/sbin/conntrack: ELF 32-bit MSB executable, SPARC32PLUS, V8+ Required, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.4, not stripped
(gdb) r -L
Starting program: /home/jengelh/code/conntrack-tools/src/conntrack -L
Program received signal SIGBUS, Bus error.0xf7fa0c24 in __parse_counters (attr=0xffff9a0c, ct=0x301c8, dir=0)
at parse.c:319
319 = __be64_to_cpu(*(u_int64_t *)
(gdb) l
314 = ntohl(*(u_int32_t *)
315 NFA_DATA(tb[CTA_COUNTERS32_PACKETS-1]));
316
317 if (tb[CTA_COUNTERS_PACKETS-1])
318 ct->counters[dir].packets
319 = __be64_to_cpu(*(u_int64_t *)
320 NFA_DATA(tb[CTA_COUNTERS_PACKETS-1]));
321
322 switch(dir) {
323 case __DIR_ORIG:
(gdb) p NFA_DATA(tb[CTA_COUNTERS_PACKETS-1])
$1 = (void *) 0xffff9a14
.......................^ There's the problem
(gdb) bt
#0 0xf7fa0c24 in __parse_counters (attr=0xffff9a0c, ct=0x301c8, dir=0)
at parse.c:319
#1 0xf7fa14dc in __parse_conntrack (nlh=0xffff9980, cda=0xffff9878,
ct=0x301c8) at parse.c:494
#2 0xf7f9ec7c in __callback (nlh=0xffff9980, nfa=0xffff9878, data=0x301b8)
at callback.c:30
#3 0xf7f7e4a8 in nfnl_step () from /usr/lib/libnfnetlink.so.0
#4 0xf7f7e6b4 in nfnl_process () from /usr/lib/libnfnetlink.so.0
#5 0xf7f7fc80 in nfnl_catch () from /usr/lib/libnfnetlink.so.0
#6 0xf7f9db04 in nfct_query (h=0x30150, qt=NFCT_Q_DUMP, data=0xffffd190)
at api.c:801#7 0x00015048 in main (argc=2, argv=0xffffd2d4) at conntrack.c:1275
Pablo Neira Ayuso
maintainerlibnetfilter_conntrack alignment issue [was Re: [PATCH 1/2] Add new input plugin UNIXSOCK]
In reply to Jan EngelhardtQuoted 7 lines · Jan Engelhardt
Quoted 5 lines · Pablo Neira Ayuso
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
+#define
ALIGN_SIZE8Minor question: why align this to 64 bits?
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
I would need to look into this in more detail, not sure where the problem is. I think that you can use something like `struct nlattr' (see
include/linux/netlink.h) and thennla_put()to add attributes in the TLV format (seelib/nlattr.c). Those are align-safe. I'm using something similar for conntrackd for the synchronization messages (src/build.candsrc/parse.c).If they are align-safe, what's this? :-)
Could you tell me where did I say that libnetfilter_conntrack is using those functions that I recommended?
Quoted 6 lines
18:41 ares:/home/jengelh # conntrack -L
Bus error
18:41 ares:/home/jengelh # filewhich conntrack/usr/sbin/conntrack: ELF 32-bit MSB executable, SPARC32PLUS, V8+ Required, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.4, not stripped
The following patch should fix the problem that you are reporting. Let me know if you have more issues.
parse: fix access to u64 attributes in netlink messages
From: Pablo Neira Ayuso <[email protected]>
This patch fixes parsing of 64 bits attributes (that are unaligned) in ctnetlink. It would be better to add nfnl_get_uX() functions similar to those in include/net/netlink.h to libnfnetlink to avoid this sort of errors.
--- src/conntrack/parse.c | 30 +++++++++++++++++++----------- 1 files changed, 19 insertions(+), 11 deletions(-)
src/conntrack/parse.c +19 -11
@@ -276,9 +276,11 @@ static void __parse_protoinfo_dccp(const struct nfattr *attr,set_bit (ATTR_DCCP_ROLE , ct->set); }if (tb[CTA_PROTOINFO_DCCP_SEQ -1 ]) { - ct->protoinfo.dccp.handshake_seq = -__be64_to_cpu (*(u_int64_t *) -NFA_DATA (tb[CTA_PROTOINFO_DCCP_SEQ -1 ])); +u_int64_t tmp; +memcpy (&tmp, +NFA_DATA (tb[CTA_PROTOINFO_DCCP_SEQ -1 ]), +sizeof (tmp)); + ct->protoinfo.dccp.handshake_seq =__be64_to_cpu (tmp);set_bit (ATTR_DCCP_HANDSHAKE_SEQ , ct->set); } }
@@ -314,10 +316,13 @@ static void __parse_counters(const struct nfattr *attr, = ntohl(*(u_int32_t *)NFA_DATA (tb[CTA_COUNTERS32_PACKETS -1 ])); -if (tb[CTA_COUNTERS_PACKETS -1 ]) - ct->counters[dir].packets - =__be64_to_cpu (*(u_int64_t *) -NFA_DATA (tb[CTA_COUNTERS_PACKETS -1 ])); +if (tb[CTA_COUNTERS_PACKETS -1 ]) { +u_int64_t tmp; +memcpy (&tmp, +NFA_DATA (tb[CTA_COUNTERS_PACKETS -1 ]), +sizeof (tmp)); + ct->counters[dir].packets =__be64_to_cpu (tmp); + }switch (dir) {case __DIR_ORIG:
@@ -335,10 +340,13 @@ static void __parse_counters(const struct nfattr *attr, = ntohl(*(u_int32_t *)NFA_DATA (tb[CTA_COUNTERS32_BYTES -1 ])); -if (tb[CTA_COUNTERS_BYTES -1 ]) - ct->counters[dir].bytes - =__be64_to_cpu (*(u_int64_t *) -NFA_DATA (tb[CTA_COUNTERS_BYTES -1 ])); +if (tb[CTA_COUNTERS_BYTES -1 ]) { +u_int64_t tmp; +memcpy (&tmp, +NFA_DATA (tb[CTA_COUNTERS_BYTES -1 ]), +sizeof (tmp)); + ct->counters[dir].bytes =__be64_to_cpu (tmp); + }switch (dir) {case __DIR_ORIG:
Re: libnetfilter_conntrack alignment issue [was Re: [PATCH 1/2] Add new input plugin UNIXSOCK]
Quoted 10 lines · Pablo Neira Ayuso
Quoted 7 lines
18:41 ares:/home/jengelh # conntrack -L
Bus error
18:41 ares:/home/jengelh # filewhich conntrack/usr/sbin/conntrack: ELF 32-bit MSB executable, SPARC32PLUS, V8+ Required, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.4, not strippedThe following patch should fix the problem that you are reporting. Let me know if you have more issues.
The patch works, thanks.
Re: [PATCH 1/2] Add new input plugin UNIXSOCK
In reply to Pierre ChifflierQuoted 20 lines · Pierre Chifflier
Quoted 11 lines · Pablo Neira Ayuso
Quoted 11 lines
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 7 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +7
-0+struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8Minor question: why align this to 64 bits?
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
Quoted 7 lines
I would need to look into this in more detail, not sure where the problem is. I think that you can use something like `struct nlattr' (see
include/linux/netlink.h) and thennla_put()to add attributes in the TLV format (seelib/nlattr.c). Those are align-safe. I'm using something similar for conntrackd for the synchronization messages (src/build.candsrc/parse.c).
Quoted 8 lines
Yes, this is very similar though
NLA_ALIGNTOis set to 4 which will cause problems with 64 bits integers.
The other way to solve this would be to read integers byte per byte, like in [1], but I found this not very elegant (and is likely to be slow compared to aligned access).Or do you have any preferred solution ? Maybe using nlattr + one special function for dealing with 64 bits variables ?
I'd suggest to add a function that reads the data in 4 byte quantities.
Pablo Neira Ayuso
maintainerRe: [PATCH 1/2] Add new input plugin UNIXSOCK
Quoted 26 lines · Patrick McHardy
Quoted 25 lines · Pierre Chifflier
Quoted 11 lines · Pablo Neira Ayuso
Quoted 11 lines
Quoted 8 lines
input/packet/ulogd_inppkt_UNIXSOCK.c line 348
Quoted 7 lines
input/packet/ulogd_inppkt_UNIXSOCK.c new file +7
-0+struct ulogd_unixsock_option_t { +uint16_t option_id; +uint16_t option_length; +char option_value[0 ]; +} __attribute__((packed)); + +#define ALIGN_SIZE 8Minor question: why align this to 64 bits?
I originally used an alignment to 32 bits, but Jan noticed it would break if using options/values on 64 bits (and a test confirmed that). I took 64 bits as the biggest allowed value for integers.
Quoted 7 lines
I would need to look into this in more detail, not sure where the problem is. I think that you can use something like `struct nlattr' (see
include/linux/netlink.h) and thennla_put()to add attributes in the TLV format (seelib/nlattr.c). Those are align-safe. I'm using something similar for conntrackd for the synchronization messages (src/build.candsrc/parse.c).Yes, this is very similar though
NLA_ALIGNTOis set to 4 which will cause problems with 64 bits integers.
The other way to solve this would be to read integers byte per byte, like in [1], but I found this not very elegant (and is likely to be slow compared to aligned access).
Quoted 5 lines
Quoted 2 lines
Or do you have any preferred solution ? Maybe using nlattr + one special function for dealing with 64 bits variables ?
I'd suggest to add a function that reads the data in 4 byte quantities.
Would memcpy() be enough? If so, we can go back to the 4 bytes alignment.
Re: [PATCH 1/2] Add new input plugin UNIXSOCK
Quoted 7 lines · Pablo Neira Ayuso
Quoted 4 lines · Patrick McHardy
Quoted 2 lines
Or do you have any preferred solution ? Maybe using nlattr + one special function for dealing with 64 bits variables ?
I'd suggest to add a function that reads the data in 4 byte quantities.
Would
memcpy()be enough? If so, we can go back to the 4 bytes alignment.
Both are fine.
PATCH 2/2 Add helper script pcap2ulog
In reply to Pierre ChifflierThis script uses the Net::Pcap Perl library to parse an pcap file and send packets to ulogd2 throught the UNIXSOCK input module.
--- contrib/pcap2ulog | 166 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 166 insertions(+), 0 deletions(-) create mode 100755 contrib/pcap2ulog
contrib/pcap2ulog new file +166 -0
@@ -0,0 +1,166 @@ +#!/usr/bin/perl -w + +# Copyright (C) 2009-2010 Pierre Chifflier <[email protected]> +# +# This program is free software; you can redistribute it and/or modify +# it under the same terms as Perl itself, either Perl version 5.8.4 or, +# at your option, any later version of Perl 5 you may have available. + +use strict; + +use IO::Socket; +use Net::Pcap; + +my $ULOGD_SOCKET_MARK = 0x41c90fd4; + +my $dumpfile = shift or die "Unable to open pcap file"; +my($pcap_t, $err); +my($ulogd_client); +my $socketfile = "/var/run/ulogd2.sock"; +my $data_buffer; +my $linktype; +my $proto_offset; + +my %linktype_to_offset = ( + Net::Pcap::DLT_LINUX_SLL => 14, + Net::Pcap::DLT_EN10MB => 12, +); + +sub connect_ulogd2 { + (-S $socketfile) or die "ulogd2 socket $socketfile does not exist - is ulogd running ?"; + + $ulogd_client = IO::Socket::UNIX->new(Peer => $socketfile, + Type => SOCK_STREAM ) or die $!; + $ulogd_client->autoflush(0); +} + +sub print_padding +{ + my ($offset) = @_; + my $padding; + my $align = 8; + my $data; + + $padding = ($align - ($offset % $align)) % $align; + #print "offset: $offset padding $padding\n"; + + $data = "\0" x $padding; + $data_buffer .= $data; +} + +sub process_pkt { + my($user, $hdr, $pkt) = @_; + + if (($user ne "xyz") or !defined($hdr) or !defined($pkt)) { + print("Bad args passed to callback\n"); + print("Bad user data\n"), if ($user ne "xyz"); + print("Bad pkthdr\n"), if (!defined($hdr)); + print("Bad pkt data\n"), if (!defined($pkt)); + print("not ok\n"); + exit; + } + + #print "Header: len $hdr->{len}\n"; + #my $len = length $pkt; + #print "Packet length: $len\n"; + + my $size = length($pkt) - ($proto_offset+2); + + #my $pcaphdr = unpack ("H*", substr ($pkt, 0, 16)); + #printf("pcap hdr: $pcaphdr\n"); + my $proto = unpack ("H*", substr ($pkt, $proto_offset, 2)); + #printf("proto: $proto\n"); + + if ($proto ne "0800") { + print "ignoring packet with proto $proto\n"; + return; + } + + #my $ip_firstbyte = unpack ("H*", substr ($pkt, $proto_offset+2, 2)); + #printf("ip_firstbyte: $ip_firstbyte\n"); + + # decode packet for a SLL: + # packet type (sent by us: 4) + # link layer address type: 1 + # link layer address length: 6 + # src dst + # protocol (IP, ARP, PPP, SNMP ...) + # data + my $srcmac = substr ($pkt, 6, 6); + + (my $hex_src = unpack("H*", $srcmac)) =~ s/(..)/$1:/g; + chop $hex_src; + #printf "source mac: $hex_src\n"; + + my $hex_dst = "\0"; + + # format data + my $data; + + $data_buffer = undef; + + # ulogd packet signature + $data = pack ('N', $ULOGD_SOCKET_MARK); + + $data_buffer .= $data; + + my $options_num=2; + my $options_len=length($hex_src) + length($hex_dst); + # total length (will be filled later) + my $total_size = 0; + $data = pack ('n', $total_size); + $data_buffer .= $data; + + # payload length + reserved + payload + $data = pack ('nNa*', $size, 0, substr($pkt,$proto_offset+2,$size)); + $data_buffer .= $data; + print_padding($size); + + # options + my $OOB_IN = 2; + $data = pack ('nna*', $OOB_IN, length($hex_src), $hex_src); + $data_buffer .= $data; + print_padding(length($hex_src)); + my $OOB_OUT = 3; + $data = pack ('nna*', $OOB_OUT, length($hex_dst), $hex_dst); + $data_buffer .= $data; + print_padding(length($hex_dst)); + + # replace total size in buffer + my $l = length($data_buffer) - 4; + substr($data_buffer, 4, 2) = pack('n', $l); + + #(my $hex = unpack("H*", $data_buffer)) =~ s/(..)/$1 /g; + #print "$l will be encoded as " . unpack("H*", pack('n', $l)) . "\n"; + #print $hex, "\n"; + + print $ulogd_client $data_buffer; + + $ulogd_client->flush; + + #exit; +} + + +connect_ulogd2 or die $!; + +$pcap_t = Net::Pcap::open_offline($dumpfile, \$err); +if (!defined($pcap_t)) { + print("Net::Pcap::dump_open failed: ", Net::Pcap::geterr($pcap_t), "\n"); + exit; +} + +$linktype = Net::Pcap::pcap_datalink($pcap_t); + +if (not exists $linktype_to_offset{$linktype}) { + print("Unsupported link type ", Net::Pcap::pcap_datalink_val_to_name($linktype), "\n"); + exit 1; +} + +$proto_offset = $linktype_to_offset{$linktype}; + +Net::Pcap::loop($pcap_t, -1, \&process_pkt, "xyz"); +Net::Pcap::close($pcap_t); + + +
Signature · 1 line
1.7.0
Hi,
Quoted 8 lines · Pierre Chifflier
Hi,
Here is an updated version of the UNIXSOCK plugin.
Changes for v5:
- align all structures on 64 bits instead of 32 This has been tested on a sparc64 (thanks to Jan)
- update pcap2ulog to handle different pcap formats
As change with previous version are quiet small, I don't think there will be a "long" discussion anymore.
I will give it some tests and push this patchset to the tree if I did not detect problem with it.
BR,
Quoted 27 lines
Changes for v4:
- use structures instead of reading integers directly
- all structures and fields are now aligned (4 bytes)
- update pcap2ulog script to use aligned data as wellChanges for v3:
- if the bufsize value (from conf) is 0, try to guess the buffer size usinggetsockopt(SOL_SOCKET, SO_RCVBUF ..)
- set default path to/var/run/ulogd/ulogd2.sockwith mode 0600
- add new configuration options perms, owner and group to be able to set these parameters on the created socket
- change the input algorithm to make it more robust against junk data (which can happen if the client is desynchronized)
- fixed a few bugsDescription:
[1] add new input plugin UNISOCK, using a unix socket. This allows userspace applications to send packets to ulogd, for example to send packets offline. It uses a key-length-value protocol to handle optional fields or extensions.
[2] is a Perl script to load a PCAP file and send it to ulogd using the UNIXSOCK plugin.
Pierre
Signature · 3 lines
Eric Leblond <[email protected]> INL: http://www.inl.fr/ NuFW: http://www.nufw.org/